2024-02-15T18:09:47.146 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-15T18:09:47.146 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_lflores@teuthology --name lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} --verbose --archive /home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561675 -- /home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561675/orig.config.yaml 2024-02-15T18:09:47.195 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-15T18:09:47.290 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561675 branch: wip-yuri10-testing-2024-02-08-0854-pacific description: rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} email: null first_in_suite: false job_id: '7561675' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/pacific v16.2.4 ' name: lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-yuri10-testing-2024-02-08-0854-pacific ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(PG_DEGRADED\) - Degraded data redundancy - \(PG_ - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-tracker-64343-pacific sha1: bedfc49c6a823fb0ff75911e476e1f3cb464ad3e owner: scheduled_lflores@teuthology priority: 25 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 964 sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-tracker-64343-pacific suite_path: /home/teuthworker/src/github.com_ljflores_ceph_bedfc49c6a823fb0ff75911e476e1f3cb464ad3e/qa suite_relpath: qa suite_repo: https://github.com/ljflores/ceph.git suite_sha1: bedfc49c6a823fb0ff75911e476e1f3cb464ad3e targets: smithi038.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEd41pZ143Adv9uvpx+XlukqK5F+agsvwBdxonJpz8AHilhRrPX5g0VWtWM0VufLhQp8BISm/7dTbXjZcJltzNo= smithi121.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOZ9r2D1mj/bG7GzVc470FwrHqZw9kuRYD6jtyL+QgMEKVZ53Sbt7UlfCnUk/hTnZLHHznVyp1T4GMNzJFZdAXk= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - pexec: clients: - sudo modprobe -r ceph - sudo modprobe ceph disable_send_metrics=on - install: exclude_packages: - ceph-volume tag: v16.2.4 - print: '**** done install task...' - cephadm: avoid_pacific_features: true cephadm_branch: v16.2.4 cephadm_git_url: https://github.com/ceph/ceph image: docker.io/ceph/ceph:v16.2.4 roleless: true - print: '**** done starting v16.2.4' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - kclient: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-15_17:32:20 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; sleep 30 ; done - ceph orch ps - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: lflores verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-02-15T18:09:47.290 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ljflores_ceph_bedfc49c6a823fb0ff75911e476e1f3cb464ad3e/qa; will attempt to use it 2024-02-15T18:09:47.291 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ljflores_ceph_bedfc49c6a823fb0ff75911e476e1f3cb464ad3e/qa/tasks 2024-02-15T18:09:47.291 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-15T18:09:47.293 INFO:teuthology.task.internal:Checking packages... 2024-02-15T18:09:47.317 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '0e714d9a4bd2a821113e6318adb87bd06cf81ec1' 2024-02-15T18:09:47.317 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-15T18:09:47.317 INFO:teuthology.packaging:ref: None 2024-02-15T18:09:47.317 INFO:teuthology.packaging:tag: None 2024-02-15T18:09:47.317 INFO:teuthology.packaging:branch: wip-yuri10-testing-2024-02-08-0854-pacific 2024-02-15T18:09:47.317 INFO:teuthology.packaging:sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:09:47.317 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri10-testing-2024-02-08-0854-pacific 2024-02-15T18:09:47.456 INFO:teuthology.task.internal:Found packages for ceph version 16.2.14-521.g0e714d9a 2024-02-15T18:09:47.457 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-15T18:09:47.464 INFO:teuthology.task.internal:no buildpackages task found 2024-02-15T18:09:47.464 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-15T18:09:47.483 INFO:teuthology.task.internal:Saving configuration 2024-02-15T18:09:47.502 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-15T18:09:47.509 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-15T18:09:47.535 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561675', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-15 18:04:35.389983', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEd41pZ143Adv9uvpx+XlukqK5F+agsvwBdxonJpz8AHilhRrPX5g0VWtWM0VufLhQp8BISm/7dTbXjZcJltzNo='} 2024-02-15T18:09:47.556 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561675', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-15 18:04:35.388771', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOZ9r2D1mj/bG7GzVc470FwrHqZw9kuRYD6jtyL+QgMEKVZ53Sbt7UlfCnUk/hTnZLHHznVyp1T4GMNzJFZdAXk='} 2024-02-15T18:09:47.556 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-15T18:09:47.562 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-02-15T18:09:47.562 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-02-15T18:09:47.562 INFO:teuthology.run_tasks:Running task console_log... 2024-02-15T18:09:47.639 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ffa5f316f70>, signals=[15]) 2024-02-15T18:09:47.640 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-15T18:09:47.647 INFO:teuthology.task.internal:Opening connections... 2024-02-15T18:09:47.647 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi038.front.sepia.ceph.com 2024-02-15T18:09:47.649 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T18:09:47.728 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi121.front.sepia.ceph.com 2024-02-15T18:09:47.729 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T18:09:47.800 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-15T18:09:47.807 DEBUG:teuthology.orchestra.run.smithi038:> uname -m 2024-02-15T18:09:47.830 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64 2024-02-15T18:09:47.831 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release 2024-02-15T18:09:47.893 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream" 2024-02-15T18:09:47.893 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="8" 2024-02-15T18:09:47.893 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos" 2024-02-15T18:09:47.893 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora" 2024-02-15T18:09:47.893 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="8" 2024-02-15T18:09:47.893 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el8" 2024-02-15T18:09:47.893 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 8" 2024-02-15T18:09:47.893 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31" 2024-02-15T18:09:47.893 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-02-15T18:09:47.893 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/" 2024-02-15T18:09:47.893 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-15T18:09:47.894 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-15T18:09:47.894 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-15T18:09:47.894 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2024-02-15T18:09:47.920 DEBUG:teuthology.orchestra.run.smithi121:> uname -m 2024-02-15T18:09:47.942 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2024-02-15T18:09:47.943 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2024-02-15T18:09:48.004 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Stream" 2024-02-15T18:09:48.004 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="8" 2024-02-15T18:09:48.004 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos" 2024-02-15T18:09:48.004 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora" 2024-02-15T18:09:48.005 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="8" 2024-02-15T18:09:48.005 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el8" 2024-02-15T18:09:48.005 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Stream 8" 2024-02-15T18:09:48.005 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2024-02-15T18:09:48.005 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-02-15T18:09:48.005 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://centos.org/" 2024-02-15T18:09:48.005 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-15T18:09:48.005 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-15T18:09:48.005 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-15T18:09:48.006 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2024-02-15T18:09:48.027 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-15T18:09:48.037 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-15T18:09:48.043 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-15T18:09:48.043 DEBUG:teuthology.orchestra.run.smithi038:> test '!' -e /home/ubuntu/cephtest 2024-02-15T18:09:48.045 DEBUG:teuthology.orchestra.run.smithi121:> test '!' -e /home/ubuntu/cephtest 2024-02-15T18:09:48.066 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-15T18:09:48.072 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-15T18:09:48.072 DEBUG:teuthology.orchestra.run.smithi038:> test -z $(ls -A /var/lib/ceph) 2024-02-15T18:09:48.106 DEBUG:teuthology.orchestra.run.smithi121:> test -z $(ls -A /var/lib/ceph) 2024-02-15T18:09:48.138 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-15T18:09:48.194 INFO:teuthology.run_tasks:Running task kernel... 2024-02-15T18:09:48.210 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-15T18:09:48.211 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-15T18:09:48.211 DEBUG:teuthology.orchestra.run.smithi038:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-15T18:09:48.211 DEBUG:teuthology.orchestra.run.smithi121:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-15T18:09:48.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-15T18:09:48.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-15T18:09:48.233 DEBUG:teuthology.orchestra.run.smithi121:> uname -r 2024-02-15T18:09:48.234 DEBUG:teuthology.orchestra.run.smithi038:> uname -r 2024-02-15T18:09:48.294 INFO:teuthology.orchestra.run.smithi121.stdout:4.18.0-489.el8.x86_64 2024-02-15T18:09:48.295 INFO:teuthology.task.kernel:Running kernel on smithi121: 4.18.0-489.el8.x86_64 2024-02-15T18:09:48.295 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum install -y kernel 2024-02-15T18:09:48.296 INFO:teuthology.orchestra.run.smithi038.stdout:4.18.0-489.el8.x86_64 2024-02-15T18:09:48.296 INFO:teuthology.task.kernel:Running kernel on smithi038: 4.18.0-489.el8.x86_64 2024-02-15T18:09:48.296 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum install -y kernel 2024-02-15T18:10:21.424 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 8 - AppStream 912 kB/s | 27 MB 00:30 2024-02-15T18:10:27.174 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 8 - BaseOS 18 MB/s | 10 MB 00:00 2024-02-15T18:10:29.299 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 8 - Extras 42 kB/s | 18 kB 00:00 2024-02-15T18:10:30.991 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 8 - Extras common packages 4.4 kB/s | 6.9 kB 00:01 2024-02-15T18:10:31.301 INFO:teuthology.orchestra.run.smithi038.stdout:Copr repo for python3-asyncssh owned by ceph 17 kB/s | 3.5 kB 00:00 2024-02-15T18:10:36.213 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 3.4 MB/s | 16 MB 00:04 2024-02-15T18:10:40.569 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 52 kB/s | 24 kB 00:00 2024-02-15T18:10:43.749 INFO:teuthology.orchestra.run.smithi038.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-02-15T18:10:43.749 INFO:teuthology.orchestra.run.smithi038.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-02-15T18:10:43.873 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:10:43.874 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:10:43.874 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-02-15T18:10:43.874 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:10:43.875 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-02-15T18:10:43.875 INFO:teuthology.orchestra.run.smithi038.stdout: kernel x86_64 4.18.0-539.el8 baseos 10 M 2024-02-15T18:10:43.875 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-02-15T18:10:43.875 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-core x86_64 4.18.0-539.el8 baseos 43 M 2024-02-15T18:10:43.875 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules x86_64 4.18.0-539.el8 baseos 36 M 2024-02-15T18:10:43.875 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:10:43.875 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-02-15T18:10:43.875 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:10:43.875 INFO:teuthology.orchestra.run.smithi038.stdout:Install 3 Packages 2024-02-15T18:10:43.875 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:10:43.875 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 90 M 2024-02-15T18:10:43.875 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 97 M 2024-02-15T18:10:43.875 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-02-15T18:10:44.328 INFO:teuthology.orchestra.run.smithi038.stdout:(1/3): kernel-4.18.0-539.el8.x86_64.rpm 28 MB/s | 10 MB 00:00 2024-02-15T18:10:44.962 INFO:teuthology.orchestra.run.smithi038.stdout:(2/3): kernel-modules-4.18.0-539.el8.x86_64.rpm 36 MB/s | 36 MB 00:01 2024-02-15T18:10:45.404 INFO:teuthology.orchestra.run.smithi038.stdout:(3/3): kernel-core-4.18.0-539.el8.x86_64.rpm 30 MB/s | 43 MB 00:01 2024-02-15T18:10:45.404 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:10:45.404 INFO:teuthology.orchestra.run.smithi038.stdout:Total 59 MB/s | 90 MB 00:01 2024-02-15T18:10:46.117 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-02-15T18:10:46.550 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-02-15T18:10:46.551 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-02-15T18:10:47.854 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-02-15T18:10:47.874 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-02-15T18:10:50.805 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-02-15T18:10:51.304 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-core-4.18.0-539.el8.x86_64 1/3 2024-02-15T18:10:55.050 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-core-4.18.0-539.el8.x86_64 1/3 2024-02-15T18:10:55.445 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-modules-4.18.0-539.el8.x86_64 2/3 2024-02-15T18:10:59.129 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-modules-4.18.0-539.el8.x86_64 2/3 2024-02-15T18:10:59.433 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-4.18.0-539.el8.x86_64 3/3 2024-02-15T18:11:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-core-4.18.0-539.el8.x86_64 3/3 2024-02-15T18:11:22.892 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-modules-4.18.0-539.el8.x86_64 3/3 2024-02-15T18:11:23.266 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-4.18.0-539.el8.x86_64 3/3 2024-02-15T18:11:23.267 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-4.18.0-539.el8.x86_64 1/3 2024-02-15T18:11:23.267 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-core-4.18.0-539.el8.x86_64 2/3 2024-02-15T18:11:23.891 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-modules-4.18.0-539.el8.x86_64 3/3 2024-02-15T18:11:23.891 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:11:23.891 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-02-15T18:11:23.891 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-4.18.0-539.el8.x86_64 kernel-core-4.18.0-539.el8.x86_64 2024-02-15T18:11:23.891 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules-4.18.0-539.el8.x86_64 2024-02-15T18:11:23.891 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:11:23.891 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:11:24.638 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q kernel | sort -rV | head -n 1 2024-02-15T18:11:24.916 INFO:teuthology.orchestra.run.smithi038.stdout:kernel-4.18.0-539.el8.x86_64 2024-02-15T18:11:24.916 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-539.el8.x86_64 2024-02-15T18:11:24.916 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-539.el8.x86_64 2024-02-15T18:11:24.916 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-15T18:11:24.916 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-15T18:11:24.916 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-02-15T18:11:24.917 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi038.front.sepia.ceph.com, path=None, version=distro) 2024-02-15T18:11:24.917 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q kernel | sort -rV | head -n 1 2024-02-15T18:11:25.137 INFO:teuthology.orchestra.run.smithi038.stdout:kernel-4.18.0-539.el8.x86_64 2024-02-15T18:11:25.137 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-539.el8.x86_64 2024-02-15T18:11:25.137 DEBUG:teuthology.orchestra.run.smithi038:> sudo rpm -qi grub2-tools 2024-02-15T18:11:25.172 INFO:teuthology.orchestra.run.smithi038.stdout:Name : grub2-tools 2024-02-15T18:11:25.172 INFO:teuthology.orchestra.run.smithi038.stdout:Epoch : 1 2024-02-15T18:11:25.172 INFO:teuthology.orchestra.run.smithi038.stdout:Version : 2.02 2024-02-15T18:11:25.172 INFO:teuthology.orchestra.run.smithi038.stdout:Release : 129.el8 2024-02-15T18:11:25.173 INFO:teuthology.orchestra.run.smithi038.stdout:Architecture: x86_64 2024-02-15T18:11:25.173 INFO:teuthology.orchestra.run.smithi038.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-02-15T18:11:25.173 INFO:teuthology.orchestra.run.smithi038.stdout:Group : System Environment/Base 2024-02-15T18:11:25.173 INFO:teuthology.orchestra.run.smithi038.stdout:Size : 9265586 2024-02-15T18:11:25.173 INFO:teuthology.orchestra.run.smithi038.stdout:License : GPLv3+ 2024-02-15T18:11:25.173 INFO:teuthology.orchestra.run.smithi038.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-02-15T18:11:25.173 INFO:teuthology.orchestra.run.smithi038.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-02-15T18:11:25.173 INFO:teuthology.orchestra.run.smithi038.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-02-15T18:11:25.173 INFO:teuthology.orchestra.run.smithi038.stdout:Build Host : kbuilder.bsys.centos.org 2024-02-15T18:11:25.173 INFO:teuthology.orchestra.run.smithi038.stdout:Relocations : (not relocatable) 2024-02-15T18:11:25.173 INFO:teuthology.orchestra.run.smithi038.stdout:Packager : CentOS BuildSystem 2024-02-15T18:11:25.173 INFO:teuthology.orchestra.run.smithi038.stdout:Vendor : CentOS 2024-02-15T18:11:25.173 INFO:teuthology.orchestra.run.smithi038.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-15T18:11:25.173 INFO:teuthology.orchestra.run.smithi038.stdout:Summary : Support tools for GRUB. 2024-02-15T18:11:25.174 INFO:teuthology.orchestra.run.smithi038.stdout:Description : 2024-02-15T18:11:25.174 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:11:25.174 INFO:teuthology.orchestra.run.smithi038.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-15T18:11:25.174 INFO:teuthology.orchestra.run.smithi038.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-15T18:11:25.174 INFO:teuthology.orchestra.run.smithi038.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-15T18:11:25.174 INFO:teuthology.orchestra.run.smithi038.stdout:hardware devices. 2024-02-15T18:11:25.174 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:11:25.174 INFO:teuthology.orchestra.run.smithi038.stdout:This subpackage provides tools for support of all platforms. 2024-02-15T18:11:25.175 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-15T18:11:25.175 INFO:teuthology.task.kernel:Updating grub on smithi038 to boot 4.18.0-539.el8.x86_64 2024-02-15T18:11:25.175 DEBUG:teuthology.orchestra.run.smithi038:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-15T18:11:26.297 INFO:teuthology.orchestra.run.smithi038.stderr:Generating grub configuration file ... 2024-02-15T18:11:28.849 INFO:teuthology.orchestra.run.smithi038.stderr:done 2024-02-15T18:11:28.851 DEBUG:teuthology.orchestra.run.smithi038:> mktemp 2024-02-15T18:11:28.905 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.HT7M6PdrDt 2024-02-15T18:11:28.906 DEBUG:teuthology.orchestra.run.smithi038:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HT7M6PdrDt 2024-02-15T18:11:28.938 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.HT7M6PdrDt 2024-02-15T18:11:29.082 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.HT7M6PdrDt is 6KB 2024-02-15T18:11:29.131 DEBUG:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.HT7M6PdrDt 2024-02-15T18:11:29.149 DEBUG:teuthology.orchestra.run.smithi038:> sudo /bin/ls /boot/loader/entries || true 2024-02-15T18:11:29.215 INFO:teuthology.orchestra.run.smithi038.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-02-15T18:11:29.215 INFO:teuthology.orchestra.run.smithi038.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-02-15T18:11:29.215 INFO:teuthology.orchestra.run.smithi038.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-02-15T18:11:29.215 INFO:teuthology.orchestra.run.smithi038.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-539.el8.x86_64.conf 2024-02-15T18:11:29.216 DEBUG:teuthology.orchestra.run.smithi038:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-539.el8.x86_64 2024-02-15T18:11:29.379 DEBUG:teuthology.orchestra.run.smithi038:> sudo shutdown -r now 2024-02-15T18:11:59.382 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-02-15T18:11:59.383 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-02-15T18:11:59.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T18:12:05.508 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 8 - AppStream 207 kB/s | 27 MB 02:14 2024-02-15T18:12:10.783 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 8 - BaseOS 24 MB/s | 10 MB 00:00 2024-02-15T18:12:12.692 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 8 - Extras 59 kB/s | 18 kB 00:00 2024-02-15T18:12:13.201 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 8 - Extras common packages 17 kB/s | 6.9 kB 00:00 2024-02-15T18:12:13.436 INFO:teuthology.orchestra.run.smithi121.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2024-02-15T18:12:15.809 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 7.1 MB/s | 16 MB 00:02 2024-02-15T18:12:17.856 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.38 2024-02-15T18:12:19.627 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 421 kB/s | 24 kB 00:00 2024-02-15T18:12:22.574 INFO:teuthology.orchestra.run.smithi121.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-02-15T18:12:22.574 INFO:teuthology.orchestra.run.smithi121.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-02-15T18:12:22.688 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:12:22.690 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:12:22.690 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2024-02-15T18:12:22.690 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:12:22.691 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-02-15T18:12:22.691 INFO:teuthology.orchestra.run.smithi121.stdout: kernel x86_64 4.18.0-539.el8 baseos 10 M 2024-02-15T18:12:22.691 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-02-15T18:12:22.691 INFO:teuthology.orchestra.run.smithi121.stdout: kernel-core x86_64 4.18.0-539.el8 baseos 43 M 2024-02-15T18:12:22.691 INFO:teuthology.orchestra.run.smithi121.stdout: kernel-modules x86_64 4.18.0-539.el8 baseos 36 M 2024-02-15T18:12:22.691 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:12:22.691 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-02-15T18:12:22.692 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:12:22.692 INFO:teuthology.orchestra.run.smithi121.stdout:Install 3 Packages 2024-02-15T18:12:22.692 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:12:22.692 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 90 M 2024-02-15T18:12:22.692 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 97 M 2024-02-15T18:12:22.692 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-02-15T18:12:23.095 INFO:teuthology.orchestra.run.smithi121.stdout:(1/3): kernel-4.18.0-539.el8.x86_64.rpm 28 MB/s | 10 MB 00:00 2024-02-15T18:12:23.754 INFO:teuthology.orchestra.run.smithi121.stdout:(2/3): kernel-modules-4.18.0-539.el8.x86_64.rpm 35 MB/s | 36 MB 00:01 2024-02-15T18:12:24.188 INFO:teuthology.orchestra.run.smithi121.stdout:(3/3): kernel-core-4.18.0-539.el8.x86_64.rpm 30 MB/s | 43 MB 00:01 2024-02-15T18:12:24.188 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:12:24.188 INFO:teuthology.orchestra.run.smithi121.stdout:Total 60 MB/s | 90 MB 00:01 2024-02-15T18:12:24.872 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-02-15T18:12:25.290 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-02-15T18:12:25.290 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-02-15T18:12:26.524 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-02-15T18:12:26.543 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-02-15T18:12:26.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-02-15T18:12:26.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T18:12:29.302 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-02-15T18:12:29.746 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : kernel-core-4.18.0-539.el8.x86_64 1/3 2024-02-15T18:12:33.331 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: kernel-core-4.18.0-539.el8.x86_64 1/3 2024-02-15T18:12:33.728 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : kernel-modules-4.18.0-539.el8.x86_64 2/3 2024-02-15T18:12:37.156 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: kernel-modules-4.18.0-539.el8.x86_64 2/3 2024-02-15T18:12:37.442 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : kernel-4.18.0-539.el8.x86_64 3/3 2024-02-15T18:12:59.255 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: kernel-core-4.18.0-539.el8.x86_64 3/3 2024-02-15T18:12:59.279 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: kernel-modules-4.18.0-539.el8.x86_64 3/3 2024-02-15T18:12:59.618 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: kernel-4.18.0-539.el8.x86_64 3/3 2024-02-15T18:12:59.619 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : kernel-4.18.0-539.el8.x86_64 1/3 2024-02-15T18:12:59.619 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : kernel-core-4.18.0-539.el8.x86_64 2/3 2024-02-15T18:13:00.123 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : kernel-modules-4.18.0-539.el8.x86_64 3/3 2024-02-15T18:13:00.123 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:13:00.123 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-02-15T18:13:00.123 INFO:teuthology.orchestra.run.smithi121.stdout: kernel-4.18.0-539.el8.x86_64 kernel-core-4.18.0-539.el8.x86_64 2024-02-15T18:13:00.123 INFO:teuthology.orchestra.run.smithi121.stdout: kernel-modules-4.18.0-539.el8.x86_64 2024-02-15T18:13:00.124 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:13:00.124 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:13:00.274 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q kernel | sort -rV | head -n 1 2024-02-15T18:13:00.529 INFO:teuthology.orchestra.run.smithi121.stdout:kernel-4.18.0-539.el8.x86_64 2024-02-15T18:13:00.530 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-539.el8.x86_64 2024-02-15T18:13:00.530 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-539.el8.x86_64 2024-02-15T18:13:00.530 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-15T18:13:00.530 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-15T18:13:00.530 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-02-15T18:13:00.530 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi121.front.sepia.ceph.com, path=None, version=distro) 2024-02-15T18:13:00.530 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q kernel | sort -rV | head -n 1 2024-02-15T18:13:00.741 INFO:teuthology.orchestra.run.smithi121.stdout:kernel-4.18.0-539.el8.x86_64 2024-02-15T18:13:00.742 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-539.el8.x86_64 2024-02-15T18:13:00.742 DEBUG:teuthology.orchestra.run.smithi121:> sudo rpm -qi grub2-tools 2024-02-15T18:13:00.775 INFO:teuthology.orchestra.run.smithi121.stdout:Name : grub2-tools 2024-02-15T18:13:00.775 INFO:teuthology.orchestra.run.smithi121.stdout:Epoch : 1 2024-02-15T18:13:00.775 INFO:teuthology.orchestra.run.smithi121.stdout:Version : 2.02 2024-02-15T18:13:00.775 INFO:teuthology.orchestra.run.smithi121.stdout:Release : 129.el8 2024-02-15T18:13:00.775 INFO:teuthology.orchestra.run.smithi121.stdout:Architecture: x86_64 2024-02-15T18:13:00.775 INFO:teuthology.orchestra.run.smithi121.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-02-15T18:13:00.775 INFO:teuthology.orchestra.run.smithi121.stdout:Group : System Environment/Base 2024-02-15T18:13:00.775 INFO:teuthology.orchestra.run.smithi121.stdout:Size : 9265586 2024-02-15T18:13:00.776 INFO:teuthology.orchestra.run.smithi121.stdout:License : GPLv3+ 2024-02-15T18:13:00.776 INFO:teuthology.orchestra.run.smithi121.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-02-15T18:13:00.776 INFO:teuthology.orchestra.run.smithi121.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-02-15T18:13:00.776 INFO:teuthology.orchestra.run.smithi121.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-02-15T18:13:00.776 INFO:teuthology.orchestra.run.smithi121.stdout:Build Host : kbuilder.bsys.centos.org 2024-02-15T18:13:00.776 INFO:teuthology.orchestra.run.smithi121.stdout:Relocations : (not relocatable) 2024-02-15T18:13:00.776 INFO:teuthology.orchestra.run.smithi121.stdout:Packager : CentOS BuildSystem 2024-02-15T18:13:00.776 INFO:teuthology.orchestra.run.smithi121.stdout:Vendor : CentOS 2024-02-15T18:13:00.776 INFO:teuthology.orchestra.run.smithi121.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-15T18:13:00.776 INFO:teuthology.orchestra.run.smithi121.stdout:Summary : Support tools for GRUB. 2024-02-15T18:13:00.776 INFO:teuthology.orchestra.run.smithi121.stdout:Description : 2024-02-15T18:13:00.776 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:13:00.776 INFO:teuthology.orchestra.run.smithi121.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-15T18:13:00.776 INFO:teuthology.orchestra.run.smithi121.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-15T18:13:00.777 INFO:teuthology.orchestra.run.smithi121.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-15T18:13:00.777 INFO:teuthology.orchestra.run.smithi121.stdout:hardware devices. 2024-02-15T18:13:00.777 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:13:00.777 INFO:teuthology.orchestra.run.smithi121.stdout:This subpackage provides tools for support of all platforms. 2024-02-15T18:13:00.778 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-15T18:13:00.778 INFO:teuthology.task.kernel:Updating grub on smithi121 to boot 4.18.0-539.el8.x86_64 2024-02-15T18:13:00.778 DEBUG:teuthology.orchestra.run.smithi121:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-15T18:13:02.533 INFO:teuthology.orchestra.run.smithi121.stderr:Generating grub configuration file ... 2024-02-15T18:13:05.005 INFO:teuthology.orchestra.run.smithi121.stderr:done 2024-02-15T18:13:05.007 DEBUG:teuthology.orchestra.run.smithi121:> mktemp 2024-02-15T18:13:05.036 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.CTfaJK9ziZ 2024-02-15T18:13:05.037 DEBUG:teuthology.orchestra.run.smithi121:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.CTfaJK9ziZ 2024-02-15T18:13:05.110 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0666 /tmp/tmp.CTfaJK9ziZ 2024-02-15T18:13:05.245 DEBUG:teuthology.orchestra.remote:smithi121:/tmp/tmp.CTfaJK9ziZ is 6KB 2024-02-15T18:13:05.293 DEBUG:teuthology.orchestra.run.smithi121:> rm -fr /tmp/tmp.CTfaJK9ziZ 2024-02-15T18:13:05.311 DEBUG:teuthology.orchestra.run.smithi121:> sudo /bin/ls /boot/loader/entries || true 2024-02-15T18:13:05.385 INFO:teuthology.orchestra.run.smithi121.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-02-15T18:13:05.385 INFO:teuthology.orchestra.run.smithi121.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-02-15T18:13:05.385 INFO:teuthology.orchestra.run.smithi121.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-02-15T18:13:05.385 INFO:teuthology.orchestra.run.smithi121.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-539.el8.x86_64.conf 2024-02-15T18:13:05.385 DEBUG:teuthology.orchestra.run.smithi121:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-539.el8.x86_64 2024-02-15T18:13:05.537 DEBUG:teuthology.orchestra.run.smithi121:> sudo shutdown -r now 2024-02-15T18:13:26.881 DEBUG:teuthology.orchestra.remote:timed out 2024-02-15T18:13:35.548 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-02-15T18:13:35.549 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-02-15T18:13:35.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T18:13:38.885 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-02-15T18:13:38.886 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T18:13:39.294 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-02-15T18:13:39.620 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-02-15T18:13:39.620 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "4.18.0-539.el8.x86_64"... 2024-02-15T18:13:39.620 DEBUG:teuthology.orchestra.run.smithi038:> uname -r 2024-02-15T18:13:39.640 INFO:teuthology.orchestra.run.smithi038.stdout:4.18.0-539.el8.x86_64 2024-02-15T18:13:39.641 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-539.el8.x86_64 vs 4.18.0-539.el8.x86_64 2024-02-15T18:13:39.641 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-15T18:13:39.641 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-15T18:13:40.641 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-02-15T18:13:40.642 DEBUG:teuthology.orchestra.run.smithi038:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-15T18:13:40.714 INFO:teuthology.orchestra.run.smithi038.stdout:ttyS1 2024-02-15T18:13:40.727 DEBUG:teuthology.parallel:result is None 2024-02-15T18:13:54.108 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.121 2024-02-15T18:14:03.109 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-02-15T18:14:03.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T18:14:06.176 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.121 2024-02-15T18:14:18.177 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-02-15T18:14:18.178 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T18:14:36.608 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.121 2024-02-15T18:14:51.610 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-02-15T18:14:51.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T18:14:52.854 DEBUG:teuthology.orchestra.run.smithi121:> true 2024-02-15T18:14:53.205 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-02-15T18:14:53.205 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "4.18.0-539.el8.x86_64"... 2024-02-15T18:14:53.205 DEBUG:teuthology.orchestra.run.smithi121:> uname -r 2024-02-15T18:14:53.263 INFO:teuthology.orchestra.run.smithi121.stdout:4.18.0-539.el8.x86_64 2024-02-15T18:14:53.263 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-539.el8.x86_64 vs 4.18.0-539.el8.x86_64 2024-02-15T18:14:53.263 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-15T18:14:53.264 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-15T18:14:54.264 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-02-15T18:14:54.265 DEBUG:teuthology.orchestra.run.smithi121:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-15T18:14:54.340 INFO:teuthology.orchestra.run.smithi121.stdout:ttyS1 2024-02-15T18:14:54.355 DEBUG:teuthology.parallel:result is None 2024-02-15T18:14:54.355 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-15T18:14:54.362 INFO:teuthology.task.internal:Creating test directory... 2024-02-15T18:14:54.362 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-15T18:14:54.365 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-15T18:14:54.386 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-15T18:14:54.392 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-15T18:14:54.402 INFO:teuthology.task.internal:Creating archive directory... 2024-02-15T18:14:54.402 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-15T18:14:54.428 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-15T18:14:54.464 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-15T18:14:54.471 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-15T18:14:54.471 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-15T18:14:54.502 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-15T18:14:54.537 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-15T18:14:54.547 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-15T18:14:54.554 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-15T18:14:54.568 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-15T18:14:54.568 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-15T18:14:54.575 INFO:teuthology.task.internal:Configuring sudo... 2024-02-15T18:14:54.575 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-15T18:14:54.577 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-15T18:14:54.616 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-15T18:14:54.625 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-15T18:14:54.625 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-15T18:14:54.649 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-15T18:14:54.674 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-15T18:14:54.755 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-15T18:14:54.803 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-15T18:14:54.889 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-15T18:14:54.935 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-02-15T18:14:54.935 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-15T18:14:55.004 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-15T18:14:55.052 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-15T18:14:55.104 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-15T18:14:55.190 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-15T18:14:55.232 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-02-15T18:14:55.232 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-15T18:14:55.298 DEBUG:teuthology.orchestra.run.smithi038:> sudo service rsyslog restart 2024-02-15T18:14:55.301 DEBUG:teuthology.orchestra.run.smithi121:> sudo service rsyslog restart 2024-02-15T18:14:55.367 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-15T18:14:55.388 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-15T18:14:55.796 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-15T18:14:55.805 INFO:teuthology.task.internal:Starting timer... 2024-02-15T18:14:55.805 INFO:teuthology.run_tasks:Running task pcp... 2024-02-15T18:14:55.863 INFO:teuthology.run_tasks:Running task selinux... 2024-02-15T18:14:55.871 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-15T18:14:55.871 DEBUG:teuthology.orchestra.run.smithi038:> sudo service auditd rotate 2024-02-15T18:14:55.969 INFO:teuthology.orchestra.run.smithi038.stdout:Rotating logs: 2024-02-15T18:14:55.971 DEBUG:teuthology.orchestra.run.smithi121:> sudo service auditd rotate 2024-02-15T18:14:56.035 INFO:teuthology.orchestra.run.smithi121.stdout:Rotating logs: 2024-02-15T18:14:56.037 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-15T18:14:56.037 DEBUG:teuthology.orchestra.run.smithi038:> /usr/sbin/getenforce 2024-02-15T18:14:56.093 INFO:teuthology.orchestra.run.smithi038.stdout:Permissive 2024-02-15T18:14:56.094 DEBUG:teuthology.orchestra.run.smithi121:> /usr/sbin/getenforce 2024-02-15T18:14:56.127 INFO:teuthology.orchestra.run.smithi121.stdout:Permissive 2024-02-15T18:14:56.127 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi038.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi121.front.sepia.ceph.com': 'permissive'} 2024-02-15T18:14:56.127 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-15T18:14:56.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-15T18:14:56.156 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-15T18:14:56.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-15T18:14:56.193 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-15T18:14:56.194 DEBUG:teuthology.orchestra.run.smithi038:> sudo /usr/sbin/setenforce permissive 2024-02-15T18:14:56.225 DEBUG:teuthology.orchestra.run.smithi121:> sudo /usr/sbin/setenforce permissive 2024-02-15T18:14:56.269 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-15T18:14:56.279 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-15T18:14:56.295 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-15T18:14:56.297 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi038.front.sepia.ceph.com,smithi121.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-15T18:20:57.080 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi038.front.sepia.ceph.com'), Remote(name='ubuntu@smithi121.front.sepia.ceph.com')] 2024-02-15T18:20:57.081 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-02-15T18:20:57.082 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T18:20:57.160 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-02-15T18:20:57.231 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-02-15T18:20:57.232 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-02-15T18:20:57.232 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T18:20:57.306 DEBUG:teuthology.orchestra.run.smithi121:> true 2024-02-15T18:20:57.379 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-02-15T18:20:57.380 INFO:teuthology.run_tasks:Running task clock... 2024-02-15T18:20:57.391 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-15T18:20:57.391 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-15T18:20:57.392 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-15T18:20:57.394 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-15T18:20:57.394 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-15T18:20:57.425 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-15T18:20:57.440 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-15T18:20:57.451 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-15T18:20:57.465 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-15T18:20:57.503 INFO:teuthology.orchestra.run.smithi038.stderr:sudo: ntpd: command not found 2024-02-15T18:20:57.515 INFO:teuthology.orchestra.run.smithi038.stdout:506 Cannot talk to daemon 2024-02-15T18:20:57.523 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: ntpd: command not found 2024-02-15T18:20:57.528 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-15T18:20:57.534 INFO:teuthology.orchestra.run.smithi121.stdout:506 Cannot talk to daemon 2024-02-15T18:20:57.542 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-15T18:20:57.548 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-15T18:20:57.561 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-15T18:20:57.614 INFO:teuthology.orchestra.run.smithi038.stderr:bash: ntpq: command not found 2024-02-15T18:20:57.617 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-15T18:20:57.617 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-02-15T18:20:57.617 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-15T18:20:57.617 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-15T18:20:57.617 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-15T18:20:57.618 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-15T18:20:57.639 INFO:teuthology.orchestra.run.smithi121.stderr:bash: ntpq: command not found 2024-02-15T18:20:57.644 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-15T18:20:57.644 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2024-02-15T18:20:57.644 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-15T18:20:57.644 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-15T18:20:57.644 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-15T18:20:57.644 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-15T18:20:57.644 INFO:teuthology.run_tasks:Running task pexec... 2024-02-15T18:20:57.654 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-15T18:20:57.655 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi038.front.sepia.ceph.com 2024-02-15T18:20:57.655 DEBUG:teuthology.orchestra.run.smithi038:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-15T18:20:57.655 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi121.front.sepia.ceph.com 2024-02-15T18:20:57.655 DEBUG:teuthology.orchestra.run.smithi121:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-15T18:20:58.462 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:35 ago on Thu 15 Feb 2024 06:19:23 PM UTC. 2024-02-15T18:20:58.478 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:28 ago on Thu 15 Feb 2024 06:19:30 PM UTC. 2024-02-15T18:21:00.494 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:21:00.495 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:21:00.496 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2024-02-15T18:21:00.496 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:21:00.496 INFO:teuthology.orchestra.run.smithi121.stdout:Resetting modules: 2024-02-15T18:21:00.496 INFO:teuthology.orchestra.run.smithi121.stdout: container-tools 2024-02-15T18:21:00.496 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:21:00.496 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-02-15T18:21:00.496 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:21:00.496 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:21:00.674 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:21:00.676 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:21:00.676 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-02-15T18:21:00.676 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:21:00.676 INFO:teuthology.orchestra.run.smithi038.stdout:Resetting modules: 2024-02-15T18:21:00.676 INFO:teuthology.orchestra.run.smithi038.stdout: container-tools 2024-02-15T18:21:00.676 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:21:00.676 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-02-15T18:21:00.676 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:21:00.676 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:21:00.753 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:21:00.960 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:21:01.610 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:38 ago on Thu 15 Feb 2024 06:19:23 PM UTC. 2024-02-15T18:21:01.846 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:31 ago on Thu 15 Feb 2024 06:19:30 PM UTC. 2024-02-15T18:21:03.659 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:21:03.694 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:21:03.694 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-02-15T18:21:03.694 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:21:03.694 INFO:teuthology.orchestra.run.smithi121.stdout:Installing group/module packages: 2024-02-15T18:21:03.695 INFO:teuthology.orchestra.run.smithi121.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-02-15T18:21:03.695 INFO:teuthology.orchestra.run.smithi121.stdout: buildah x86_64 2:1.33.5-1.module_el8+885+7da147f3 appstream 9.7 M 2024-02-15T18:21:03.695 INFO:teuthology.orchestra.run.smithi121.stdout: cockpit-podman noarch 83-1.module_el8+847+7863d4e6 appstream 688 k 2024-02-15T18:21:03.695 INFO:teuthology.orchestra.run.smithi121.stdout: crun x86_64 1.14-1.module_el8+871+552be958 appstream 257 k 2024-02-15T18:21:03.695 INFO:teuthology.orchestra.run.smithi121.stdout: netavark x86_64 2:1.10.2-1.module_el8+885+7da147f3 appstream 4.3 M 2024-02-15T18:21:03.695 INFO:teuthology.orchestra.run.smithi121.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-02-15T18:21:03.695 INFO:teuthology.orchestra.run.smithi121.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-02-15T18:21:03.695 INFO:teuthology.orchestra.run.smithi121.stdout: toolbox x86_64 0.0.99.5-1.module_el8+847+7863d4e6 appstream 2.5 M 2024-02-15T18:21:03.695 INFO:teuthology.orchestra.run.smithi121.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-02-15T18:21:03.695 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-02-15T18:21:03.695 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-02-15T18:21:03.695 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-02-15T18:21:03.695 INFO:teuthology.orchestra.run.smithi121.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-02-15T18:21:03.695 INFO:teuthology.orchestra.run.smithi121.stdout:Installing weak dependencies: 2024-02-15T18:21:03.696 INFO:teuthology.orchestra.run.smithi121.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-02-15T18:21:03.696 INFO:teuthology.orchestra.run.smithi121.stdout:Installing module profiles: 2024-02-15T18:21:03.696 INFO:teuthology.orchestra.run.smithi121.stdout: container-tools/common 2024-02-15T18:21:03.696 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:21:03.696 INFO:teuthology.orchestra.run.smithi121.stdout:Enabling module streams: 2024-02-15T18:21:03.696 INFO:teuthology.orchestra.run.smithi121.stdout: container-tools rhel8 2024-02-15T18:21:03.696 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:21:03.696 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-02-15T18:21:03.696 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:21:03.696 INFO:teuthology.orchestra.run.smithi121.stdout:Install 13 Packages 2024-02-15T18:21:03.696 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:21:03.697 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 28 M 2024-02-15T18:21:03.697 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 92 M 2024-02-15T18:21:03.697 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-02-15T18:21:04.023 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:21:04.061 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:21:04.062 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-02-15T18:21:04.062 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:21:04.062 INFO:teuthology.orchestra.run.smithi038.stdout:Installing group/module packages: 2024-02-15T18:21:04.062 INFO:teuthology.orchestra.run.smithi038.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-02-15T18:21:04.062 INFO:teuthology.orchestra.run.smithi038.stdout: buildah x86_64 2:1.33.5-1.module_el8+885+7da147f3 appstream 9.7 M 2024-02-15T18:21:04.062 INFO:teuthology.orchestra.run.smithi038.stdout: cockpit-podman noarch 83-1.module_el8+847+7863d4e6 appstream 688 k 2024-02-15T18:21:04.062 INFO:teuthology.orchestra.run.smithi038.stdout: crun x86_64 1.14-1.module_el8+871+552be958 appstream 257 k 2024-02-15T18:21:04.062 INFO:teuthology.orchestra.run.smithi038.stdout: netavark x86_64 2:1.10.2-1.module_el8+885+7da147f3 appstream 4.3 M 2024-02-15T18:21:04.062 INFO:teuthology.orchestra.run.smithi038.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-02-15T18:21:04.062 INFO:teuthology.orchestra.run.smithi038.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-02-15T18:21:04.062 INFO:teuthology.orchestra.run.smithi038.stdout: toolbox x86_64 0.0.99.5-1.module_el8+847+7863d4e6 appstream 2.5 M 2024-02-15T18:21:04.062 INFO:teuthology.orchestra.run.smithi038.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-02-15T18:21:04.063 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-02-15T18:21:04.063 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-02-15T18:21:04.063 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-02-15T18:21:04.063 INFO:teuthology.orchestra.run.smithi038.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-02-15T18:21:04.063 INFO:teuthology.orchestra.run.smithi038.stdout:Installing weak dependencies: 2024-02-15T18:21:04.063 INFO:teuthology.orchestra.run.smithi038.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-02-15T18:21:04.063 INFO:teuthology.orchestra.run.smithi038.stdout:Installing module profiles: 2024-02-15T18:21:04.063 INFO:teuthology.orchestra.run.smithi038.stdout: container-tools/common 2024-02-15T18:21:04.063 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:21:04.063 INFO:teuthology.orchestra.run.smithi038.stdout:Enabling module streams: 2024-02-15T18:21:04.063 INFO:teuthology.orchestra.run.smithi038.stdout: container-tools rhel8 2024-02-15T18:21:04.063 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:21:04.063 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-02-15T18:21:04.063 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:21:04.063 INFO:teuthology.orchestra.run.smithi038.stdout:Install 13 Packages 2024-02-15T18:21:04.064 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:21:04.064 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 28 M 2024-02-15T18:21:04.064 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 92 M 2024-02-15T18:21:04.064 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-02-15T18:21:04.219 INFO:teuthology.orchestra.run.smithi121.stdout:(1/13): aardvark-dns-1.10.0-1.module_el8+871+55 2.3 MB/s | 1.0 MB 00:00 2024-02-15T18:21:04.244 INFO:teuthology.orchestra.run.smithi121.stdout:(2/13): cockpit-podman-83-1.module_el8+847+7863 1.4 MB/s | 688 kB 00:00 2024-02-15T18:21:04.323 INFO:teuthology.orchestra.run.smithi121.stdout:(3/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 365 kB/s | 38 kB 00:00 2024-02-15T18:21:04.386 INFO:teuthology.orchestra.run.smithi121.stdout:(4/13): crun-1.14-1.module_el8+871+552be958.x86 1.8 MB/s | 257 kB 00:00 2024-02-15T18:21:04.503 INFO:teuthology.orchestra.run.smithi121.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 1.5 MB/s | 180 kB 00:00 2024-02-15T18:21:04.570 INFO:teuthology.orchestra.run.smithi121.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 371 kB/s | 25 kB 00:00 2024-02-15T18:21:04.667 INFO:teuthology.orchestra.run.smithi121.stdout:(7/13): python3-pyxdg-0.25-16.el8.noarch.rpm 967 kB/s | 94 kB 00:00 2024-02-15T18:21:04.978 INFO:teuthology.orchestra.run.smithi038.stdout:(1/13): cockpit-podman-83-1.module_el8+847+7863 855 kB/s | 688 kB 00:00 2024-02-15T18:21:05.037 INFO:teuthology.orchestra.run.smithi121.stdout:(8/13): netavark-1.10.2-1.module_el8+885+7da147 6.0 MB/s | 4.3 MB 00:00 2024-02-15T18:21:05.053 INFO:teuthology.orchestra.run.smithi038.stdout:(2/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 506 kB/s | 38 kB 00:00 2024-02-15T18:21:05.162 INFO:teuthology.orchestra.run.smithi038.stdout:(3/13): aardvark-dns-1.10.0-1.module_el8+871+55 1.0 MB/s | 1.0 MB 00:00 2024-02-15T18:21:05.306 INFO:teuthology.orchestra.run.smithi038.stdout:(4/13): crun-1.14-1.module_el8+871+552be958.x86 1.0 MB/s | 257 kB 00:00 2024-02-15T18:21:05.446 INFO:teuthology.orchestra.run.smithi121.stdout:(9/13): buildah-1.33.5-1.module_el8+885+7da147f 5.8 MB/s | 9.7 MB 00:01 2024-02-15T18:21:05.479 INFO:teuthology.orchestra.run.smithi038.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 1.0 MB/s | 180 kB 00:00 2024-02-15T18:21:05.488 INFO:teuthology.orchestra.run.smithi121.stdout:(10/13): toolbox-0.0.99.5-1.module_el8+847+7863 5.6 MB/s | 2.5 MB 00:00 2024-02-15T18:21:05.513 INFO:teuthology.orchestra.run.smithi121.stdout:(11/13): udica-0.2.6-20.module_el8+804+f131391c 787 kB/s | 53 kB 00:00 2024-02-15T18:21:05.548 INFO:teuthology.orchestra.run.smithi038.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 360 kB/s | 25 kB 00:00 2024-02-15T18:21:05.563 INFO:teuthology.orchestra.run.smithi121.stdout:(12/13): yajl-2.1.0-12.el8.x86_64.rpm 545 kB/s | 41 kB 00:00 2024-02-15T18:21:05.674 INFO:teuthology.orchestra.run.smithi038.stdout:(7/13): python3-pyxdg-0.25-16.el8.noarch.rpm 751 kB/s | 94 kB 00:00 2024-02-15T18:21:06.097 INFO:teuthology.orchestra.run.smithi121.stdout:(13/13): skopeo-1.14.3-0.1.module_el8+885+7da14 6.2 MB/s | 8.8 MB 00:01 2024-02-15T18:21:06.097 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:21:06.097 INFO:teuthology.orchestra.run.smithi121.stdout:Total 12 MB/s | 28 MB 00:02 2024-02-15T18:21:06.366 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-02-15T18:21:06.387 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-02-15T18:21:06.387 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-02-15T18:21:06.641 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-02-15T18:21:06.644 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-02-15T18:21:07.385 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-02-15T18:21:08.302 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-02-15T18:21:08.505 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-02-15T18:21:08.700 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-02-15T18:21:08.732 INFO:teuthology.orchestra.run.smithi038.stdout:(8/13): netavark-1.10.2-1.module_el8+885+7da147 1.2 MB/s | 4.3 MB 00:03 2024-02-15T18:21:08.898 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-02-15T18:21:09.161 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-02-15T18:21:09.797 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-02-15T18:21:10.024 INFO:teuthology.orchestra.run.smithi038.stdout:(9/13): toolbox-0.0.99.5-1.module_el8+847+7863d 2.0 MB/s | 2.5 MB 00:01 2024-02-15T18:21:10.074 INFO:teuthology.orchestra.run.smithi038.stdout:(10/13): udica-0.2.6-20.module_el8+804+f131391c 1.0 MB/s | 53 kB 00:00 2024-02-15T18:21:10.124 INFO:teuthology.orchestra.run.smithi038.stdout:(11/13): yajl-2.1.0-12.el8.x86_64.rpm 820 kB/s | 41 kB 00:00 2024-02-15T18:21:10.324 INFO:teuthology.orchestra.run.smithi038.stdout:(12/13): buildah-1.33.5-1.module_el8+885+7da147 1.6 MB/s | 9.7 MB 00:06 2024-02-15T18:21:10.781 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 7/13 2024-02-15T18:21:11.032 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 8/13 2024-02-15T18:21:11.235 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : crun-1.14-1.module_el8+871+552be958.x86_64 9/13 2024-02-15T18:21:11.541 INFO:teuthology.orchestra.run.smithi038.stdout:(13/13): skopeo-1.14.3-0.1.module_el8+885+7da14 1.5 MB/s | 8.8 MB 00:05 2024-02-15T18:21:11.542 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:21:11.542 INFO:teuthology.orchestra.run.smithi038.stdout:Total 3.7 MB/s | 28 MB 00:07 2024-02-15T18:21:11.711 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-02-15T18:21:11.825 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-02-15T18:21:11.847 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-02-15T18:21:11.847 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-02-15T18:21:12.106 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-15T18:21:12.136 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-02-15T18:21:12.139 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-02-15T18:21:12.307 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-15T18:21:12.507 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-15T18:21:12.936 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-15T18:21:12.936 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-02-15T18:21:12.936 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2/13 2024-02-15T18:21:12.936 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 3/13 2024-02-15T18:21:12.936 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-02-15T18:21:12.936 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : crun-1.14-1.module_el8+871+552be958.x86_64 5/13 2024-02-15T18:21:12.936 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 6/13 2024-02-15T18:21:12.936 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-02-15T18:21:12.936 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-02-15T18:21:12.936 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-02-15T18:21:12.937 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-02-15T18:21:12.937 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-15T18:21:12.937 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-15T18:21:13.099 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-02-15T18:21:13.469 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-02-15T18:21:13.469 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:21:13.469 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-02-15T18:21:13.469 INFO:teuthology.orchestra.run.smithi121.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-02-15T18:21:13.469 INFO:teuthology.orchestra.run.smithi121.stdout: buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2024-02-15T18:21:13.469 INFO:teuthology.orchestra.run.smithi121.stdout: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 2024-02-15T18:21:13.469 INFO:teuthology.orchestra.run.smithi121.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-02-15T18:21:13.470 INFO:teuthology.orchestra.run.smithi121.stdout: crun-1.14-1.module_el8+871+552be958.x86_64 2024-02-15T18:21:13.470 INFO:teuthology.orchestra.run.smithi121.stdout: netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 2024-02-15T18:21:13.470 INFO:teuthology.orchestra.run.smithi121.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-02-15T18:21:13.470 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-02-15T18:21:13.470 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-02-15T18:21:13.470 INFO:teuthology.orchestra.run.smithi121.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-02-15T18:21:13.470 INFO:teuthology.orchestra.run.smithi121.stdout: toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 2024-02-15T18:21:13.470 INFO:teuthology.orchestra.run.smithi121.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-02-15T18:21:13.470 INFO:teuthology.orchestra.run.smithi121.stdout: yajl-2.1.0-12.el8.x86_64 2024-02-15T18:21:13.470 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:21:13.470 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:21:13.753 DEBUG:teuthology.parallel:result is None 2024-02-15T18:21:14.129 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-02-15T18:21:14.320 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-02-15T18:21:14.515 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-02-15T18:21:14.729 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-02-15T18:21:14.975 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-02-15T18:21:15.553 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-02-15T18:21:16.557 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 7/13 2024-02-15T18:21:16.806 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 8/13 2024-02-15T18:21:17.065 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : crun-1.14-1.module_el8+871+552be958.x86_64 9/13 2024-02-15T18:21:17.687 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-02-15T18:21:17.931 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-15T18:21:18.585 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-15T18:21:18.814 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-15T18:21:19.299 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-15T18:21:19.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-02-15T18:21:19.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2/13 2024-02-15T18:21:19.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 3/13 2024-02-15T18:21:19.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-02-15T18:21:19.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : crun-1.14-1.module_el8+871+552be958.x86_64 5/13 2024-02-15T18:21:19.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 6/13 2024-02-15T18:21:19.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-02-15T18:21:19.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-02-15T18:21:19.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-02-15T18:21:19.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-02-15T18:21:19.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-15T18:21:19.300 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-15T18:21:19.847 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-02-15T18:21:19.847 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:21:19.847 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-02-15T18:21:19.847 INFO:teuthology.orchestra.run.smithi038.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-02-15T18:21:19.847 INFO:teuthology.orchestra.run.smithi038.stdout: buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2024-02-15T18:21:19.847 INFO:teuthology.orchestra.run.smithi038.stdout: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 2024-02-15T18:21:19.847 INFO:teuthology.orchestra.run.smithi038.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-02-15T18:21:19.848 INFO:teuthology.orchestra.run.smithi038.stdout: crun-1.14-1.module_el8+871+552be958.x86_64 2024-02-15T18:21:19.848 INFO:teuthology.orchestra.run.smithi038.stdout: netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 2024-02-15T18:21:19.848 INFO:teuthology.orchestra.run.smithi038.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-02-15T18:21:19.848 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-02-15T18:21:19.848 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-02-15T18:21:19.848 INFO:teuthology.orchestra.run.smithi038.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-02-15T18:21:19.848 INFO:teuthology.orchestra.run.smithi038.stdout: toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 2024-02-15T18:21:19.848 INFO:teuthology.orchestra.run.smithi038.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-02-15T18:21:19.848 INFO:teuthology.orchestra.run.smithi038.stdout: yajl-2.1.0-12.el8.x86_64 2024-02-15T18:21:19.848 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:21:19.848 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:21:20.148 DEBUG:teuthology.parallel:result is None 2024-02-15T18:21:20.148 INFO:teuthology.run_tasks:Running task pexec... 2024-02-15T18:21:20.156 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-15T18:21:20.156 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi038.front.sepia.ceph.com 2024-02-15T18:21:20.157 DEBUG:teuthology.orchestra.run.smithi038:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-15T18:21:20.157 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi121.front.sepia.ceph.com 2024-02-15T18:21:20.157 DEBUG:teuthology.orchestra.run.smithi121:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-15T18:21:20.318 DEBUG:teuthology.parallel:result is None 2024-02-15T18:21:20.385 DEBUG:teuthology.parallel:result is None 2024-02-15T18:21:20.386 INFO:teuthology.run_tasks:Running task install... 2024-02-15T18:21:20.395 DEBUG:teuthology.task.install:project ceph 2024-02-15T18:21:20.395 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '0e714d9a4bd2a821113e6318adb87bd06cf81ec1'}} 2024-02-15T18:21:20.395 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v16.2.4', 'flavor': 'default', 'sha1': '0e714d9a4bd2a821113e6318adb87bd06cf81ec1'} 2024-02-15T18:21:20.395 INFO:teuthology.task.install:Using flavor: default 2024-02-15T18:21:20.404 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-15T18:21:20.405 INFO:teuthology.task.install:extra packages: [] 2024-02-15T18:21:20.405 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0e714d9a4bd2a821113e6318adb87bd06cf81ec1', 'tag': 'v16.2.4', 'wait_for_package': False} 2024-02-15T18:21:20.405 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-15T18:21:20.406 INFO:teuthology.packaging:ref: None 2024-02-15T18:21:20.406 INFO:teuthology.packaging:tag: v16.2.4 2024-02-15T18:21:20.406 INFO:teuthology.packaging:branch: None 2024-02-15T18:21:20.406 INFO:teuthology.packaging:sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:21:20.705 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph-ci v16.2.4^{} -> 3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2024-02-15T18:21:20.706 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2024-02-15T18:21:20.708 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0e714d9a4bd2a821113e6318adb87bd06cf81ec1', 'tag': 'v16.2.4', 'wait_for_package': False} 2024-02-15T18:21:20.709 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-15T18:21:20.709 INFO:teuthology.packaging:ref: None 2024-02-15T18:21:20.709 INFO:teuthology.packaging:tag: v16.2.4 2024-02-15T18:21:20.709 INFO:teuthology.packaging:branch: None 2024-02-15T18:21:20.709 INFO:teuthology.packaging:sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:21:20.709 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2024-02-15T18:21:20.853 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/ 2024-02-15T18:21:20.853 INFO:teuthology.task.install.rpm:Package version is 16.2.4-0 2024-02-15T18:21:20.886 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-15T18:21:20.886 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-02-15T18:21:20.886 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-15T18:21:20.920 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-15T18:21:20.920 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-15T18:21:20.920 INFO:teuthology.packaging:ref: None 2024-02-15T18:21:20.920 INFO:teuthology.packaging:tag: v16.2.4 2024-02-15T18:21:20.920 INFO:teuthology.packaging:branch: None 2024-02-15T18:21:20.920 INFO:teuthology.packaging:sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:21:20.920 DEBUG:teuthology.orchestra.run.smithi121:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v16.2.4/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-15T18:21:20.991 DEBUG:teuthology.orchestra.run.smithi121:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-15T18:21:21.022 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/ 2024-02-15T18:21:21.022 INFO:teuthology.task.install.rpm:Package version is 16.2.4-0 2024-02-15T18:21:21.048 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-15T18:21:21.048 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-02-15T18:21:21.048 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-15T18:21:21.074 DEBUG:teuthology.orchestra.run.smithi121:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-15T18:21:21.080 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-15T18:21:21.080 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-15T18:21:21.080 INFO:teuthology.packaging:ref: None 2024-02-15T18:21:21.081 INFO:teuthology.packaging:tag: v16.2.4 2024-02-15T18:21:21.081 INFO:teuthology.packaging:branch: None 2024-02-15T18:21:21.081 INFO:teuthology.packaging:sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:21:21.081 DEBUG:teuthology.orchestra.run.smithi038:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v16.2.4/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-15T18:21:21.141 INFO:teuthology.orchestra.run.smithi121.stdout:check_obsoletes = 1 2024-02-15T18:21:21.143 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum clean all 2024-02-15T18:21:21.152 DEBUG:teuthology.orchestra.run.smithi038:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-15T18:21:21.234 DEBUG:teuthology.orchestra.run.smithi038:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-15T18:21:21.304 INFO:teuthology.orchestra.run.smithi038.stdout:check_obsoletes = 1 2024-02-15T18:21:21.305 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum clean all 2024-02-15T18:21:21.411 INFO:teuthology.orchestra.run.smithi121.stdout:76 files removed 2024-02-15T18:21:21.439 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-radosgw 2024-02-15T18:21:21.584 INFO:teuthology.orchestra.run.smithi038.stdout:76 files removed 2024-02-15T18:21:21.613 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-radosgw 2024-02-15T18:21:21.912 INFO:teuthology.orchestra.run.smithi121.stdout:ceph packages for x86_64 1.0 MB/s | 74 kB 00:00 2024-02-15T18:21:22.013 INFO:teuthology.orchestra.run.smithi121.stdout:ceph noarch packages 240 kB/s | 15 kB 00:00 2024-02-15T18:21:22.114 INFO:teuthology.orchestra.run.smithi121.stdout:ceph source packages 24 kB/s | 1.7 kB 00:00 2024-02-15T18:21:22.117 INFO:teuthology.orchestra.run.smithi038.stdout:ceph packages for x86_64 979 kB/s | 74 kB 00:00 2024-02-15T18:21:22.243 INFO:teuthology.orchestra.run.smithi038.stdout:ceph noarch packages 180 kB/s | 15 kB 00:00 2024-02-15T18:21:22.343 INFO:teuthology.orchestra.run.smithi038.stdout:ceph source packages 25 kB/s | 1.7 kB 00:00 2024-02-15T18:21:33.029 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 8 - AppStream 2.5 MB/s | 27 MB 00:10 2024-02-15T18:21:33.668 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 8 - AppStream 2.4 MB/s | 27 MB 00:11 2024-02-15T18:21:38.296 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 8 - BaseOS 25 MB/s | 10 MB 00:00 2024-02-15T18:21:39.627 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 8 - BaseOS 17 MB/s | 10 MB 00:00 2024-02-15T18:21:40.421 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 8 - Extras 39 kB/s | 18 kB 00:00 2024-02-15T18:21:40.922 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 8 - Extras common packages 18 kB/s | 6.9 kB 00:00 2024-02-15T18:21:41.811 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 8 - Extras 39 kB/s | 18 kB 00:00 2024-02-15T18:21:41.831 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 8 - PowerTools 5.9 MB/s | 4.8 MB 00:00 2024-02-15T18:21:42.345 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 8 - Extras common packages 17 kB/s | 6.9 kB 00:00 2024-02-15T18:21:42.899 INFO:teuthology.orchestra.run.smithi121.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2024-02-15T18:21:43.279 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 8 - PowerTools 5.8 MB/s | 4.8 MB 00:00 2024-02-15T18:21:44.615 INFO:teuthology.orchestra.run.smithi038.stdout:Copr repo for python3-asyncssh owned by ceph 11 kB/s | 3.5 kB 00:00 2024-02-15T18:21:45.482 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2024-02-15T18:21:45.685 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 6.0 MB/s | 16 MB 00:02 2024-02-15T18:21:49.443 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 596 kB/s | 24 kB 00:00 2024-02-15T18:21:49.581 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 265 kB/s | 24 kB 00:00 2024-02-15T18:21:53.020 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:21:53.023 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:21:53.023 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-02-15T18:21:53.023 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:21:53.023 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-02-15T18:21:53.023 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw x86_64 2:16.2.4-0.el8 ceph 13 M 2024-02-15T18:21:53.023 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrading: 2024-02-15T18:21:53.023 INFO:teuthology.orchestra.run.smithi121.stdout: librados2 x86_64 2:16.2.4-0.el8 ceph 3.7 M 2024-02-15T18:21:53.024 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1 x86_64 2:16.2.4-0.el8 ceph 4.0 M 2024-02-15T18:21:53.024 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-02-15T18:21:53.024 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base x86_64 2:16.2.4-0.el8 ceph 6.0 M 2024-02-15T18:21:53.024 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common x86_64 2:16.2.4-0.el8 ceph 24 M 2024-02-15T18:21:53.024 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux x86_64 2:16.2.4-0.el8 ceph 25 k 2024-02-15T18:21:53.024 INFO:teuthology.orchestra.run.smithi121.stdout: daxctl-libs x86_64 71.1-7.el8 baseos 42 k 2024-02-15T18:21:53.024 INFO:teuthology.orchestra.run.smithi121.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2024-02-15T18:21:53.024 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-02-15T18:21:53.024 INFO:teuthology.orchestra.run.smithi121.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2024-02-15T18:21:53.024 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2 x86_64 2:16.2.4-0.el8 ceph 801 k 2024-02-15T18:21:53.024 INFO:teuthology.orchestra.run.smithi121.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-02-15T18:21:53.024 INFO:teuthology.orchestra.run.smithi121.stdout: libpmemobj x86_64 1.6.1-1.el8 appstream 145 k 2024-02-15T18:21:53.025 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-02-15T18:21:53.025 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1 x86_64 2:16.2.4-0.el8 ceph 486 k 2024-02-15T18:21:53.025 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-02-15T18:21:53.025 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2 x86_64 2:16.2.4-0.el8 ceph 4.1 M 2024-02-15T18:21:53.025 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-02-15T18:21:53.025 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-02-15T18:21:53.025 INFO:teuthology.orchestra.run.smithi121.stdout: ndctl-libs x86_64 71.1-7.el8 baseos 80 k 2024-02-15T18:21:53.025 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse x86_64 2:16.2.4-0.el8 ceph 44 k 2024-02-15T18:21:53.025 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common x86_64 2:16.2.4-0.el8 ceph 80 k 2024-02-15T18:21:53.025 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs x86_64 2:16.2.4-0.el8 ceph 213 k 2024-02-15T18:21:53.025 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-02-15T18:21:53.025 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados x86_64 2:16.2.4-0.el8 ceph 387 k 2024-02-15T18:21:53.025 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd x86_64 2:16.2.4-0.el8 ceph 368 k 2024-02-15T18:21:53.026 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw x86_64 2:16.2.4-0.el8 ceph 114 k 2024-02-15T18:21:53.026 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:21:53.026 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-02-15T18:21:53.026 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:21:53.026 INFO:teuthology.orchestra.run.smithi121.stdout:Install 25 Packages 2024-02-15T18:21:53.026 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrade 2 Packages 2024-02-15T18:21:53.026 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:21:53.026 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 59 M 2024-02-15T18:21:53.027 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-02-15T18:21:53.358 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:21:53.361 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:21:53.361 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-02-15T18:21:53.361 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:21:53.361 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-02-15T18:21:53.361 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw x86_64 2:16.2.4-0.el8 ceph 13 M 2024-02-15T18:21:53.361 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrading: 2024-02-15T18:21:53.361 INFO:teuthology.orchestra.run.smithi038.stdout: librados2 x86_64 2:16.2.4-0.el8 ceph 3.7 M 2024-02-15T18:21:53.361 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1 x86_64 2:16.2.4-0.el8 ceph 4.0 M 2024-02-15T18:21:53.361 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-02-15T18:21:53.361 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base x86_64 2:16.2.4-0.el8 ceph 6.0 M 2024-02-15T18:21:53.361 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common x86_64 2:16.2.4-0.el8 ceph 24 M 2024-02-15T18:21:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux x86_64 2:16.2.4-0.el8 ceph 25 k 2024-02-15T18:21:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: daxctl-libs x86_64 71.1-7.el8 baseos 42 k 2024-02-15T18:21:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2024-02-15T18:21:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-02-15T18:21:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2024-02-15T18:21:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2 x86_64 2:16.2.4-0.el8 ceph 801 k 2024-02-15T18:21:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-02-15T18:21:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: libpmemobj x86_64 1.6.1-1.el8 appstream 145 k 2024-02-15T18:21:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-02-15T18:21:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1 x86_64 2:16.2.4-0.el8 ceph 486 k 2024-02-15T18:21:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-02-15T18:21:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2 x86_64 2:16.2.4-0.el8 ceph 4.1 M 2024-02-15T18:21:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-02-15T18:21:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-02-15T18:21:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: ndctl-libs x86_64 71.1-7.el8 baseos 80 k 2024-02-15T18:21:53.362 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse x86_64 2:16.2.4-0.el8 ceph 44 k 2024-02-15T18:21:53.363 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common x86_64 2:16.2.4-0.el8 ceph 80 k 2024-02-15T18:21:53.363 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs x86_64 2:16.2.4-0.el8 ceph 213 k 2024-02-15T18:21:53.363 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-02-15T18:21:53.363 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados x86_64 2:16.2.4-0.el8 ceph 387 k 2024-02-15T18:21:53.363 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd x86_64 2:16.2.4-0.el8 ceph 368 k 2024-02-15T18:21:53.363 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw x86_64 2:16.2.4-0.el8 ceph 114 k 2024-02-15T18:21:53.363 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:21:53.363 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-02-15T18:21:53.363 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:21:53.363 INFO:teuthology.orchestra.run.smithi038.stdout:Install 25 Packages 2024-02-15T18:21:53.363 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrade 2 Packages 2024-02-15T18:21:53.363 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:21:53.364 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 59 M 2024-02-15T18:21:53.364 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-02-15T18:21:53.576 INFO:teuthology.orchestra.run.smithi121.stdout:(1/27): ceph-base-16.2.4-0.el8.x86_64.rpm 46 MB/s | 6.0 MB 00:00 2024-02-15T18:21:53.601 INFO:teuthology.orchestra.run.smithi121.stdout:(2/27): ceph-selinux-16.2.4-0.el8.x86_64.rpm 994 kB/s | 25 kB 00:00 2024-02-15T18:21:53.643 INFO:teuthology.orchestra.run.smithi121.stdout:(3/27): libcephfs2-16.2.4-0.el8.x86_64.rpm 19 MB/s | 801 kB 00:00 2024-02-15T18:21:53.677 INFO:teuthology.orchestra.run.smithi121.stdout:(4/27): libradosstriper1-16.2.4-0.el8.x86_64.rp 14 MB/s | 486 kB 00:00 2024-02-15T18:21:53.769 INFO:teuthology.orchestra.run.smithi121.stdout:(5/27): librgw2-16.2.4-0.el8.x86_64.rpm 44 MB/s | 4.1 MB 00:00 2024-02-15T18:21:53.794 INFO:teuthology.orchestra.run.smithi121.stdout:(6/27): python3-ceph-argparse-16.2.4-0.el8.x86_ 1.7 MB/s | 44 kB 00:00 2024-02-15T18:21:53.819 INFO:teuthology.orchestra.run.smithi121.stdout:(7/27): python3-ceph-common-16.2.4-0.el8.x86_64 3.1 MB/s | 80 kB 00:00 2024-02-15T18:21:53.845 INFO:teuthology.orchestra.run.smithi121.stdout:(8/27): python3-cephfs-16.2.4-0.el8.x86_64.rpm 8.3 MB/s | 213 kB 00:00 2024-02-15T18:21:53.870 INFO:teuthology.orchestra.run.smithi121.stdout:(9/27): python3-rados-16.2.4-0.el8.x86_64.rpm 15 MB/s | 387 kB 00:00 2024-02-15T18:21:53.895 INFO:teuthology.orchestra.run.smithi121.stdout:(10/27): python3-rbd-16.2.4-0.el8.x86_64.rpm 14 MB/s | 368 kB 00:00 2024-02-15T18:21:53.921 INFO:teuthology.orchestra.run.smithi121.stdout:(11/27): python3-rgw-16.2.4-0.el8.x86_64.rpm 4.5 MB/s | 114 kB 00:00 2024-02-15T18:21:54.088 INFO:teuthology.orchestra.run.smithi121.stdout:(12/27): ceph-radosgw-16.2.4-0.el8.x86_64.rpm 20 MB/s | 13 MB 00:00 2024-02-15T18:21:54.355 INFO:teuthology.orchestra.run.smithi121.stdout:(13/27): ceph-common-16.2.4-0.el8.x86_64.rpm 26 MB/s | 24 MB 00:00 2024-02-15T18:21:54.375 INFO:teuthology.orchestra.run.smithi038.stdout:(1/27): ceph-base-16.2.4-0.el8.x86_64.rpm 29 MB/s | 6.0 MB 00:00 2024-02-15T18:21:54.400 INFO:teuthology.orchestra.run.smithi038.stdout:(2/27): ceph-selinux-16.2.4-0.el8.x86_64.rpm 996 kB/s | 25 kB 00:00 2024-02-15T18:21:54.413 INFO:teuthology.orchestra.run.smithi121.stdout:(14/27): libpmemobj-1.6.1-1.el8.x86_64.rpm 295 kB/s | 145 kB 00:00 2024-02-15T18:21:54.442 INFO:teuthology.orchestra.run.smithi038.stdout:(3/27): libcephfs2-16.2.4-0.el8.x86_64.rpm 19 MB/s | 801 kB 00:00 2024-02-15T18:21:54.476 INFO:teuthology.orchestra.run.smithi038.stdout:(4/27): libradosstriper1-16.2.4-0.el8.x86_64.rp 14 MB/s | 486 kB 00:00 2024-02-15T18:21:54.480 INFO:teuthology.orchestra.run.smithi121.stdout:(15/27): python3-prettytable-0.7.2-14.el8.noarc 659 kB/s | 44 kB 00:00 2024-02-15T18:21:54.580 INFO:teuthology.orchestra.run.smithi121.stdout:(16/27): daxctl-libs-71.1-7.el8.x86_64.rpm 422 kB/s | 42 kB 00:00 2024-02-15T18:21:54.610 INFO:teuthology.orchestra.run.smithi038.stdout:(5/27): librgw2-16.2.4-0.el8.x86_64.rpm 30 MB/s | 4.1 MB 00:00 2024-02-15T18:21:54.644 INFO:teuthology.orchestra.run.smithi038.stdout:(6/27): python3-ceph-argparse-16.2.4-0.el8.x86_ 1.3 MB/s | 44 kB 00:00 2024-02-15T18:21:54.648 INFO:teuthology.orchestra.run.smithi121.stdout:(17/27): librabbitmq-0.9.0-5.el8.x86_64.rpm 702 kB/s | 47 kB 00:00 2024-02-15T18:21:54.669 INFO:teuthology.orchestra.run.smithi038.stdout:(7/27): python3-ceph-common-16.2.4-0.el8.x86_64 3.1 MB/s | 80 kB 00:00 2024-02-15T18:21:54.686 INFO:teuthology.orchestra.run.smithi038.stdout:(8/27): python3-cephfs-16.2.4-0.el8.x86_64.rpm 13 MB/s | 213 kB 00:00 2024-02-15T18:21:54.690 INFO:teuthology.orchestra.run.smithi121.stdout:(18/27): ndctl-libs-71.1-7.el8.x86_64.rpm 1.9 MB/s | 80 kB 00:00 2024-02-15T18:21:54.719 INFO:teuthology.orchestra.run.smithi038.stdout:(9/27): python3-rados-16.2.4-0.el8.x86_64.rpm 11 MB/s | 387 kB 00:00 2024-02-15T18:21:54.753 INFO:teuthology.orchestra.run.smithi038.stdout:(10/27): python3-rbd-16.2.4-0.el8.x86_64.rpm 11 MB/s | 368 kB 00:00 2024-02-15T18:21:54.778 INFO:teuthology.orchestra.run.smithi038.stdout:(11/27): python3-rgw-16.2.4-0.el8.x86_64.rpm 4.5 MB/s | 114 kB 00:00 2024-02-15T18:21:54.782 INFO:teuthology.orchestra.run.smithi121.stdout:(19/27): librdkafka-1.6.1-1.el8.x86_64.rpm 976 kB/s | 677 kB 00:00 2024-02-15T18:21:54.946 INFO:teuthology.orchestra.run.smithi038.stdout:(12/27): ceph-radosgw-16.2.4-0.el8.x86_64.rpm 16 MB/s | 13 MB 00:00 2024-02-15T18:21:54.982 INFO:teuthology.orchestra.run.smithi121.stdout:(20/27): lttng-ust-2.8.1-11.el8.x86_64.rpm 414 kB/s | 259 kB 00:00 2024-02-15T18:21:55.078 INFO:teuthology.orchestra.run.smithi121.stdout:(21/27): fmt-6.2.1-1.el8.x86_64.rpm 299 kB/s | 116 kB 00:00 2024-02-15T18:21:55.174 INFO:teuthology.orchestra.run.smithi121.stdout:(22/27): liboath-2.6.2-3.el8.x86_64.rpm 617 kB/s | 59 kB 00:00 2024-02-15T18:21:55.199 INFO:teuthology.orchestra.run.smithi121.stdout:(23/27): gperftools-libs-2.7-9.el8.x86_64.rpm 743 kB/s | 306 kB 00:00 2024-02-15T18:21:55.271 INFO:teuthology.orchestra.run.smithi038.stdout:(13/27): ceph-common-16.2.4-0.el8.x86_64.rpm 22 MB/s | 24 MB 00:01 2024-02-15T18:21:55.275 INFO:teuthology.orchestra.run.smithi121.stdout:(24/27): librados2-16.2.4-0.el8.x86_64.rpm 50 MB/s | 3.7 MB 00:00 2024-02-15T18:21:55.296 INFO:teuthology.orchestra.run.smithi038.stdout:(14/27): libpmemobj-1.6.1-1.el8.x86_64.rpm 281 kB/s | 145 kB 00:00 2024-02-15T18:21:55.300 INFO:teuthology.orchestra.run.smithi121.stdout:(25/27): libunwind-1.3.1-3.el8.x86_64.rpm 596 kB/s | 75 kB 00:00 2024-02-15T18:21:55.384 INFO:teuthology.orchestra.run.smithi121.stdout:(26/27): librbd1-16.2.4-0.el8.x86_64.rpm 37 MB/s | 4.0 MB 00:00 2024-02-15T18:21:55.434 INFO:teuthology.orchestra.run.smithi121.stdout:(27/27): leveldb-1.22-1.el8.x86_64.rpm 400 kB/s | 181 kB 00:00 2024-02-15T18:21:55.438 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:21:55.438 INFO:teuthology.orchestra.run.smithi121.stdout:Total 24 MB/s | 59 MB 00:02 2024-02-15T18:21:55.498 INFO:teuthology.orchestra.run.smithi038.stdout:(15/27): python3-prettytable-0.7.2-14.el8.noarc 217 kB/s | 44 kB 00:00 2024-02-15T18:21:55.506 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-02-15T18:21:55.555 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-02-15T18:21:55.555 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-02-15T18:21:55.589 INFO:teuthology.orchestra.run.smithi038.stdout:(16/27): daxctl-libs-71.1-7.el8.x86_64.rpm 470 kB/s | 42 kB 00:00 2024-02-15T18:21:55.630 INFO:teuthology.orchestra.run.smithi038.stdout:(17/27): librabbitmq-0.9.0-5.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-02-15T18:21:55.681 INFO:teuthology.orchestra.run.smithi038.stdout:(18/27): ndctl-libs-71.1-7.el8.x86_64.rpm 1.6 MB/s | 80 kB 00:00 2024-02-15T18:21:55.789 INFO:teuthology.orchestra.run.smithi038.stdout:(19/27): lttng-ust-2.8.1-11.el8.x86_64.rpm 501 kB/s | 259 kB 00:00 2024-02-15T18:21:55.856 INFO:teuthology.orchestra.run.smithi038.stdout:(20/27): librdkafka-1.6.1-1.el8.x86_64.rpm 744 kB/s | 677 kB 00:00 2024-02-15T18:21:55.896 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-02-15T18:21:55.899 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-02-15T18:21:55.915 INFO:teuthology.orchestra.run.smithi038.stdout:(21/27): fmt-6.2.1-1.el8.x86_64.rpm 497 kB/s | 116 kB 00:00 2024-02-15T18:21:55.974 INFO:teuthology.orchestra.run.smithi038.stdout:(22/27): liboath-2.6.2-3.el8.x86_64.rpm 1.0 MB/s | 59 kB 00:00 2024-02-15T18:21:56.047 INFO:teuthology.orchestra.run.smithi038.stdout:(23/27): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2024-02-15T18:21:56.074 INFO:teuthology.orchestra.run.smithi038.stdout:(24/27): libunwind-1.3.1-3.el8.x86_64.rpm 751 kB/s | 75 kB 00:00 2024-02-15T18:21:56.108 INFO:teuthology.orchestra.run.smithi038.stdout:(25/27): leveldb-1.22-1.el8.x86_64.rpm 723 kB/s | 181 kB 00:00 2024-02-15T18:21:56.200 INFO:teuthology.orchestra.run.smithi038.stdout:(26/27): librados2-16.2.4-0.el8.x86_64.rpm 25 MB/s | 3.7 MB 00:00 2024-02-15T18:21:56.266 INFO:teuthology.orchestra.run.smithi038.stdout:(27/27): librbd1-16.2.4-0.el8.x86_64.rpm 21 MB/s | 4.0 MB 00:00 2024-02-15T18:21:56.271 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:21:56.271 INFO:teuthology.orchestra.run.smithi038.stdout:Total 20 MB/s | 59 MB 00:02 2024-02-15T18:21:56.340 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-02-15T18:21:56.392 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-02-15T18:21:56.392 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-02-15T18:21:56.768 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-02-15T18:21:56.772 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-02-15T18:21:57.027 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-02-15T18:21:57.265 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/29 2024-02-15T18:21:57.485 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/29 2024-02-15T18:21:57.632 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/29 2024-02-15T18:21:57.987 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-02-15T18:21:58.044 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/29 2024-02-15T18:21:58.158 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/29 2024-02-15T18:21:58.240 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librados2-2:16.2.4-0.el8.x86_64 4/29 2024-02-15T18:21:58.333 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:16.2.4-0.el8.x86_64 4/29 2024-02-15T18:21:58.398 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/29 2024-02-15T18:21:58.583 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/29 2024-02-15T18:21:58.590 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rados-2:16.2.4-0.el8.x86_64 5/29 2024-02-15T18:21:58.731 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs2-2:16.2.4-0.el8.x86_64 6/29 2024-02-15T18:21:58.821 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephfs2-2:16.2.4-0.el8.x86_64 6/29 2024-02-15T18:21:58.999 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/29 2024-02-15T18:21:59.049 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 7/29 2024-02-15T18:21:59.208 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librados2-2:16.2.4-0.el8.x86_64 4/29 2024-02-15T18:21:59.310 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:16.2.4-0.el8.x86_64 4/29 2024-02-15T18:21:59.580 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rados-2:16.2.4-0.el8.x86_64 5/29 2024-02-15T18:21:59.606 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 8/29 2024-02-15T18:21:59.748 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librgw2-2:16.2.4-0.el8.x86_64 9/29 2024-02-15T18:21:59.758 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs2-2:16.2.4-0.el8.x86_64 6/29 2024-02-15T18:21:59.805 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephfs2-2:16.2.4-0.el8.x86_64 6/29 2024-02-15T18:21:59.852 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librgw2-2:16.2.4-0.el8.x86_64 9/29 2024-02-15T18:22:00.047 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 7/29 2024-02-15T18:22:00.061 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : leveldb-1.22-1.el8.x86_64 10/29 2024-02-15T18:22:00.239 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : daxctl-libs-71.1-7.el8.x86_64 11/29 2024-02-15T18:22:00.301 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: daxctl-libs-71.1-7.el8.x86_64 11/29 2024-02-15T18:22:00.480 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-argparse-2:16.2.4-0.el8.x86_64 12/29 2024-02-15T18:22:00.574 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 8/29 2024-02-15T18:22:00.632 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cephfs-2:16.2.4-0.el8.x86_64 13/29 2024-02-15T18:22:00.700 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librgw2-2:16.2.4-0.el8.x86_64 9/29 2024-02-15T18:22:00.809 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librgw2-2:16.2.4-0.el8.x86_64 9/29 2024-02-15T18:22:00.831 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ndctl-libs-71.1-7.el8.x86_64 14/29 2024-02-15T18:22:00.902 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ndctl-libs-71.1-7.el8.x86_64 14/29 2024-02-15T18:22:01.005 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : leveldb-1.22-1.el8.x86_64 10/29 2024-02-15T18:22:01.022 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libpmemobj-1.6.1-1.el8.x86_64 15/29 2024-02-15T18:22:01.200 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : daxctl-libs-71.1-7.el8.x86_64 11/29 2024-02-15T18:22:01.269 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: daxctl-libs-71.1-7.el8.x86_64 11/29 2024-02-15T18:22:01.457 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libpmemobj-1.6.1-1.el8.x86_64 15/29 2024-02-15T18:22:01.464 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-argparse-2:16.2.4-0.el8.x86_64 12/29 2024-02-15T18:22:01.656 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librbd1-2:16.2.4-0.el8.x86_64 16/29 2024-02-15T18:22:01.684 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cephfs-2:16.2.4-0.el8.x86_64 13/29 2024-02-15T18:22:01.757 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:16.2.4-0.el8.x86_64 16/29 2024-02-15T18:22:01.850 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ndctl-libs-71.1-7.el8.x86_64 14/29 2024-02-15T18:22:01.963 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ndctl-libs-71.1-7.el8.x86_64 14/29 2024-02-15T18:22:01.976 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rbd-2:16.2.4-0.el8.x86_64 17/29 2024-02-15T18:22:02.100 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libpmemobj-1.6.1-1.el8.x86_64 15/29 2024-02-15T18:22:02.199 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rgw-2:16.2.4-0.el8.x86_64 18/29 2024-02-15T18:22:02.356 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libradosstriper1-2:16.2.4-0.el8.x86_64 19/29 2024-02-15T18:22:02.434 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libradosstriper1-2:16.2.4-0.el8.x86_64 19/29 2024-02-15T18:22:02.594 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libpmemobj-1.6.1-1.el8.x86_64 15/29 2024-02-15T18:22:02.648 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 20/29 2024-02-15T18:22:02.808 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librbd1-2:16.2.4-0.el8.x86_64 16/29 2024-02-15T18:22:02.891 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:16.2.4-0.el8.x86_64 16/29 2024-02-15T18:22:02.902 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 21/29 2024-02-15T18:22:03.063 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 22/29 2024-02-15T18:22:03.112 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rbd-2:16.2.4-0.el8.x86_64 17/29 2024-02-15T18:22:03.272 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-common-2:16.2.4-0.el8.x86_64 23/29 2024-02-15T18:22:03.362 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rgw-2:16.2.4-0.el8.x86_64 18/29 2024-02-15T18:22:03.499 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libradosstriper1-2:16.2.4-0.el8.x86_64 19/29 2024-02-15T18:22:03.626 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libradosstriper1-2:16.2.4-0.el8.x86_64 19/29 2024-02-15T18:22:03.819 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 20/29 2024-02-15T18:22:04.046 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 21/29 2024-02-15T18:22:04.255 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 22/29 2024-02-15T18:22:04.466 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-common-2:16.2.4-0.el8.x86_64 23/29 2024-02-15T18:22:05.867 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:16.2.4-0.el8.x86_64 24/29 2024-02-15T18:22:06.089 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-common-2:16.2.4-0.el8.x86_64 24/29 2024-02-15T18:22:06.798 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:16.2.4-0.el8.x86_64 24/29 2024-02-15T18:22:07.099 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-base-2:16.2.4-0.el8.x86_64 25/29 2024-02-15T18:22:07.229 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:16.2.4-0.el8.x86_64 24/29 2024-02-15T18:22:07.237 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-base-2:16.2.4-0.el8.x86_64 25/29 2024-02-15T18:22:07.379 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-selinux-2:16.2.4-0.el8.x86_64 26/29 2024-02-15T18:22:07.508 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-common-2:16.2.4-0.el8.x86_64 24/29 2024-02-15T18:22:08.256 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:16.2.4-0.el8.x86_64 24/29 2024-02-15T18:22:08.561 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-base-2:16.2.4-0.el8.x86_64 25/29 2024-02-15T18:22:08.687 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-base-2:16.2.4-0.el8.x86_64 25/29 2024-02-15T18:22:08.982 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-selinux-2:16.2.4-0.el8.x86_64 26/29 2024-02-15T18:22:26.255 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-selinux-2:16.2.4-0.el8.x86_64 26/29 2024-02-15T18:22:26.256 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /sys 2024-02-15T18:22:26.256 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /proc 2024-02-15T18:22:26.256 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /dev 2024-02-15T18:22:26.256 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /run 2024-02-15T18:22:26.256 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /mnt 2024-02-15T18:22:26.256 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /var/tmp 2024-02-15T18:22:26.256 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /home 2024-02-15T18:22:26.256 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /tmp 2024-02-15T18:22:26.256 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /dev 2024-02-15T18:22:26.256 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:22:27.611 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-radosgw-2:16.2.4-0.el8.x86_64 27/29 2024-02-15T18:22:27.701 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-radosgw-2:16.2.4-0.el8.x86_64 27/29 2024-02-15T18:22:27.702 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 28/29 2024-02-15T18:22:27.894 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 28/29 2024-02-15T18:22:27.894 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 29/29 2024-02-15T18:22:28.566 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 29/29 2024-02-15T18:22:28.566 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-base-2:16.2.4-0.el8.x86_64 1/29 2024-02-15T18:22:28.566 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-common-2:16.2.4-0.el8.x86_64 2/29 2024-02-15T18:22:28.566 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-radosgw-2:16.2.4-0.el8.x86_64 3/29 2024-02-15T18:22:28.566 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-selinux-2:16.2.4-0.el8.x86_64 4/29 2024-02-15T18:22:28.566 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs2-2:16.2.4-0.el8.x86_64 5/29 2024-02-15T18:22:28.567 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libradosstriper1-2:16.2.4-0.el8.x86_64 6/29 2024-02-15T18:22:28.567 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librgw2-2:16.2.4-0.el8.x86_64 7/29 2024-02-15T18:22:28.567 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-argparse-2:16.2.4-0.el8.x86_64 8/29 2024-02-15T18:22:28.567 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-common-2:16.2.4-0.el8.x86_64 9/29 2024-02-15T18:22:28.567 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cephfs-2:16.2.4-0.el8.x86_64 10/29 2024-02-15T18:22:28.567 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rados-2:16.2.4-0.el8.x86_64 11/29 2024-02-15T18:22:28.567 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rbd-2:16.2.4-0.el8.x86_64 12/29 2024-02-15T18:22:28.567 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rgw-2:16.2.4-0.el8.x86_64 13/29 2024-02-15T18:22:28.567 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libpmemobj-1.6.1-1.el8.x86_64 14/29 2024-02-15T18:22:28.567 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 15/29 2024-02-15T18:22:28.567 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 16/29 2024-02-15T18:22:28.567 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 17/29 2024-02-15T18:22:28.567 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : daxctl-libs-71.1-7.el8.x86_64 18/29 2024-02-15T18:22:28.567 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 19/29 2024-02-15T18:22:28.567 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ndctl-libs-71.1-7.el8.x86_64 20/29 2024-02-15T18:22:28.567 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 21/29 2024-02-15T18:22:28.568 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 22/29 2024-02-15T18:22:28.568 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : leveldb-1.22-1.el8.x86_64 23/29 2024-02-15T18:22:28.568 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 24/29 2024-02-15T18:22:28.568 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 25/29 2024-02-15T18:22:28.568 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:16.2.4-0.el8.x86_64 26/29 2024-02-15T18:22:28.568 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 27/29 2024-02-15T18:22:28.568 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:16.2.4-0.el8.x86_64 28/29 2024-02-15T18:22:29.130 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-selinux-2:16.2.4-0.el8.x86_64 26/29 2024-02-15T18:22:29.130 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /sys 2024-02-15T18:22:29.130 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /proc 2024-02-15T18:22:29.131 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /dev 2024-02-15T18:22:29.131 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /run 2024-02-15T18:22:29.131 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /mnt 2024-02-15T18:22:29.131 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /var/tmp 2024-02-15T18:22:29.131 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /home 2024-02-15T18:22:29.131 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /tmp 2024-02-15T18:22:29.131 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /dev 2024-02-15T18:22:29.131 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:22:29.541 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 29/29 2024-02-15T18:22:29.541 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:22:29.541 INFO:teuthology.orchestra.run.smithi121.stdout:Upgraded: 2024-02-15T18:22:29.541 INFO:teuthology.orchestra.run.smithi121.stdout: librados2-2:16.2.4-0.el8.x86_64 librbd1-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:29.541 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-02-15T18:22:29.541 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:29.541 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:29.541 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:29.542 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:29.542 INFO:teuthology.orchestra.run.smithi121.stdout: daxctl-libs-71.1-7.el8.x86_64 2024-02-15T18:22:29.542 INFO:teuthology.orchestra.run.smithi121.stdout: fmt-6.2.1-1.el8.x86_64 2024-02-15T18:22:29.542 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-02-15T18:22:29.542 INFO:teuthology.orchestra.run.smithi121.stdout: leveldb-1.22-1.el8.x86_64 2024-02-15T18:22:29.542 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:29.542 INFO:teuthology.orchestra.run.smithi121.stdout: liboath-2.6.2-3.el8.x86_64 2024-02-15T18:22:29.542 INFO:teuthology.orchestra.run.smithi121.stdout: libpmemobj-1.6.1-1.el8.x86_64 2024-02-15T18:22:29.542 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-02-15T18:22:29.542 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:29.542 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-02-15T18:22:29.542 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:29.543 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind-1.3.1-3.el8.x86_64 2024-02-15T18:22:29.543 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-02-15T18:22:29.543 INFO:teuthology.orchestra.run.smithi121.stdout: ndctl-libs-71.1-7.el8.x86_64 2024-02-15T18:22:29.543 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:29.543 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:29.543 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:29.543 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-02-15T18:22:29.543 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:29.543 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:29.543 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:29.543 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:22:29.544 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:22:29.867 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-test 2024-02-15T18:22:30.574 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-radosgw-2:16.2.4-0.el8.x86_64 27/29 2024-02-15T18:22:30.625 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:41 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:22:30.653 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-radosgw-2:16.2.4-0.el8.x86_64 27/29 2024-02-15T18:22:30.653 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 28/29 2024-02-15T18:22:30.873 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 28/29 2024-02-15T18:22:30.874 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 29/29 2024-02-15T18:22:31.601 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 29/29 2024-02-15T18:22:31.602 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-base-2:16.2.4-0.el8.x86_64 1/29 2024-02-15T18:22:31.602 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-common-2:16.2.4-0.el8.x86_64 2/29 2024-02-15T18:22:31.602 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-radosgw-2:16.2.4-0.el8.x86_64 3/29 2024-02-15T18:22:31.602 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-selinux-2:16.2.4-0.el8.x86_64 4/29 2024-02-15T18:22:31.602 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs2-2:16.2.4-0.el8.x86_64 5/29 2024-02-15T18:22:31.602 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libradosstriper1-2:16.2.4-0.el8.x86_64 6/29 2024-02-15T18:22:31.602 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librgw2-2:16.2.4-0.el8.x86_64 7/29 2024-02-15T18:22:31.602 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-argparse-2:16.2.4-0.el8.x86_64 8/29 2024-02-15T18:22:31.602 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-common-2:16.2.4-0.el8.x86_64 9/29 2024-02-15T18:22:31.602 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cephfs-2:16.2.4-0.el8.x86_64 10/29 2024-02-15T18:22:31.602 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rados-2:16.2.4-0.el8.x86_64 11/29 2024-02-15T18:22:31.602 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rbd-2:16.2.4-0.el8.x86_64 12/29 2024-02-15T18:22:31.602 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rgw-2:16.2.4-0.el8.x86_64 13/29 2024-02-15T18:22:31.603 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libpmemobj-1.6.1-1.el8.x86_64 14/29 2024-02-15T18:22:31.603 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 15/29 2024-02-15T18:22:31.603 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 16/29 2024-02-15T18:22:31.603 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 17/29 2024-02-15T18:22:31.603 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : daxctl-libs-71.1-7.el8.x86_64 18/29 2024-02-15T18:22:31.603 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 19/29 2024-02-15T18:22:31.603 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ndctl-libs-71.1-7.el8.x86_64 20/29 2024-02-15T18:22:31.603 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 21/29 2024-02-15T18:22:31.603 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 22/29 2024-02-15T18:22:31.603 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : leveldb-1.22-1.el8.x86_64 23/29 2024-02-15T18:22:31.603 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 24/29 2024-02-15T18:22:31.603 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 25/29 2024-02-15T18:22:31.604 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:16.2.4-0.el8.x86_64 26/29 2024-02-15T18:22:31.604 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 27/29 2024-02-15T18:22:31.604 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:16.2.4-0.el8.x86_64 28/29 2024-02-15T18:22:32.176 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:22:32.177 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:22:32.177 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2024-02-15T18:22:32.178 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:22:32.178 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-02-15T18:22:32.178 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test x86_64 2:16.2.4-0.el8 ceph 52 M 2024-02-15T18:22:32.178 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-02-15T18:22:32.178 INFO:teuthology.orchestra.run.smithi121.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-02-15T18:22:32.178 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite x86_64 2:16.2.4-0.el8 ceph 169 k 2024-02-15T18:22:32.178 INFO:teuthology.orchestra.run.smithi121.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-02-15T18:22:32.178 INFO:teuthology.orchestra.run.smithi121.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-02-15T18:22:32.178 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-02-15T18:22:32.178 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:22:32.178 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-02-15T18:22:32.179 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:22:32.179 INFO:teuthology.orchestra.run.smithi121.stdout:Install 6 Packages 2024-02-15T18:22:32.179 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:22:32.179 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 53 M 2024-02-15T18:22:32.179 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 229 M 2024-02-15T18:22:32.179 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-02-15T18:22:32.286 INFO:teuthology.orchestra.run.smithi121.stdout:(1/6): libcephsqlite-16.2.4-0.el8.x86_64.rpm 2.5 MB/s | 169 kB 00:00 2024-02-15T18:22:32.428 INFO:teuthology.orchestra.run.smithi121.stdout:(2/6): jq-1.6-8.el8.x86_64.rpm 982 kB/s | 203 kB 00:00 2024-02-15T18:22:32.478 INFO:teuthology.orchestra.run.smithi121.stdout:(3/6): oniguruma-6.8.2-3.el8.x86_64.rpm 980 kB/s | 188 kB 00:00 2024-02-15T18:22:32.537 INFO:teuthology.orchestra.run.smithi121.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-02-15T18:22:32.626 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 29/29 2024-02-15T18:22:32.626 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:22:32.626 INFO:teuthology.orchestra.run.smithi038.stdout:Upgraded: 2024-02-15T18:22:32.627 INFO:teuthology.orchestra.run.smithi038.stdout: librados2-2:16.2.4-0.el8.x86_64 librbd1-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:32.627 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-02-15T18:22:32.627 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:32.627 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:32.627 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:32.627 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:32.627 INFO:teuthology.orchestra.run.smithi038.stdout: daxctl-libs-71.1-7.el8.x86_64 2024-02-15T18:22:32.627 INFO:teuthology.orchestra.run.smithi038.stdout: fmt-6.2.1-1.el8.x86_64 2024-02-15T18:22:32.627 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-02-15T18:22:32.627 INFO:teuthology.orchestra.run.smithi038.stdout: leveldb-1.22-1.el8.x86_64 2024-02-15T18:22:32.627 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:32.627 INFO:teuthology.orchestra.run.smithi038.stdout: liboath-2.6.2-3.el8.x86_64 2024-02-15T18:22:32.627 INFO:teuthology.orchestra.run.smithi038.stdout: libpmemobj-1.6.1-1.el8.x86_64 2024-02-15T18:22:32.628 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-02-15T18:22:32.628 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:32.628 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-02-15T18:22:32.628 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:32.628 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind-1.3.1-3.el8.x86_64 2024-02-15T18:22:32.628 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-02-15T18:22:32.628 INFO:teuthology.orchestra.run.smithi038.stdout: ndctl-libs-71.1-7.el8.x86_64 2024-02-15T18:22:32.628 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:32.628 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:32.628 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:32.628 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-02-15T18:22:32.628 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:32.628 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:32.628 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:32.629 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:22:32.629 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:22:32.926 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-test 2024-02-15T18:22:33.037 INFO:teuthology.orchestra.run.smithi121.stdout:(5/6): ceph-test-16.2.4-0.el8.x86_64.rpm 64 MB/s | 52 MB 00:00 2024-02-15T18:22:33.062 INFO:teuthology.orchestra.run.smithi121.stdout:(6/6): socat-1.7.4.1-1.el8.x86_64.rpm 509 kB/s | 323 kB 00:00 2024-02-15T18:22:33.064 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:22:33.065 INFO:teuthology.orchestra.run.smithi121.stdout:Total 60 MB/s | 53 MB 00:00 2024-02-15T18:22:33.106 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-02-15T18:22:33.127 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-02-15T18:22:33.127 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-02-15T18:22:33.442 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-02-15T18:22:33.446 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-02-15T18:22:33.717 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:00:44 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:22:33.967 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-02-15T18:22:34.193 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-02-15T18:22:34.388 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-02-15T18:22:34.573 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-15T18:22:34.650 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-15T18:22:34.815 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-02-15T18:22:34.998 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephsqlite-2:16.2.4-0.el8.x86_64 5/6 2024-02-15T18:22:35.380 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:22:35.381 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:22:35.381 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-02-15T18:22:35.381 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:22:35.381 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-02-15T18:22:35.381 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test x86_64 2:16.2.4-0.el8 ceph 52 M 2024-02-15T18:22:35.382 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-02-15T18:22:35.382 INFO:teuthology.orchestra.run.smithi038.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-02-15T18:22:35.382 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite x86_64 2:16.2.4-0.el8 ceph 169 k 2024-02-15T18:22:35.382 INFO:teuthology.orchestra.run.smithi038.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-02-15T18:22:35.382 INFO:teuthology.orchestra.run.smithi038.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-02-15T18:22:35.382 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-02-15T18:22:35.382 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:22:35.382 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-02-15T18:22:35.382 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:22:35.382 INFO:teuthology.orchestra.run.smithi038.stdout:Install 6 Packages 2024-02-15T18:22:35.382 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:22:35.383 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 53 M 2024-02-15T18:22:35.383 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 229 M 2024-02-15T18:22:35.383 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-02-15T18:22:35.509 INFO:teuthology.orchestra.run.smithi038.stdout:(1/6): libcephsqlite-16.2.4-0.el8.x86_64.rpm 3.1 MB/s | 169 kB 00:00 2024-02-15T18:22:36.243 INFO:teuthology.orchestra.run.smithi038.stdout:(2/6): ceph-test-16.2.4-0.el8.x86_64.rpm 66 MB/s | 52 MB 00:00 2024-02-15T18:22:36.268 INFO:teuthology.orchestra.run.smithi038.stdout:(3/6): jq-1.6-8.el8.x86_64.rpm 250 kB/s | 203 kB 00:00 2024-02-15T18:22:36.327 INFO:teuthology.orchestra.run.smithi038.stdout:(4/6): oniguruma-6.8.2-3.el8.x86_64.rpm 230 kB/s | 188 kB 00:00 2024-02-15T18:22:36.411 INFO:teuthology.orchestra.run.smithi038.stdout:(5/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 497 kB/s | 71 kB 00:00 2024-02-15T18:22:36.619 INFO:teuthology.orchestra.run.smithi038.stdout:(6/6): socat-1.7.4.1-1.el8.x86_64.rpm 861 kB/s | 323 kB 00:00 2024-02-15T18:22:36.620 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:22:36.621 INFO:teuthology.orchestra.run.smithi038.stdout:Total 43 MB/s | 53 MB 00:01 2024-02-15T18:22:36.664 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-02-15T18:22:36.687 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-02-15T18:22:36.687 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-02-15T18:22:37.017 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-02-15T18:22:37.020 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-02-15T18:22:37.546 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-02-15T18:22:37.800 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-02-15T18:22:38.019 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-02-15T18:22:38.170 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-15T18:22:38.250 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-15T18:22:38.440 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-02-15T18:22:38.611 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephsqlite-2:16.2.4-0.el8.x86_64 5/6 2024-02-15T18:22:39.852 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephsqlite-2:16.2.4-0.el8.x86_64 5/6 2024-02-15T18:22:40.104 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-test-2:16.2.4-0.el8.x86_64 6/6 2024-02-15T18:22:40.374 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-test-2:16.2.4-0.el8.x86_64 6/6 2024-02-15T18:22:40.374 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-test-2:16.2.4-0.el8.x86_64 1/6 2024-02-15T18:22:40.374 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephsqlite-2:16.2.4-0.el8.x86_64 2/6 2024-02-15T18:22:40.374 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-02-15T18:22:40.375 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-02-15T18:22:40.375 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-02-15T18:22:40.827 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-02-15T18:22:40.827 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:22:40.827 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-02-15T18:22:40.827 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test-2:16.2.4-0.el8.x86_64 jq-1.6-8.el8.x86_64 2024-02-15T18:22:40.827 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite-2:16.2.4-0.el8.x86_64 oniguruma-6.8.2-3.el8.x86_64 2024-02-15T18:22:40.827 INFO:teuthology.orchestra.run.smithi121.stdout: socat-1.7.4.1-1.el8.x86_64 xmlstarlet-1.6.1-20.el8.x86_64 2024-02-15T18:22:40.828 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:22:40.828 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:22:41.083 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph 2024-02-15T18:22:41.842 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:52 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:22:43.369 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:22:43.372 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:22:43.373 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-02-15T18:22:43.373 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:22:43.373 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-02-15T18:22:43.373 INFO:teuthology.orchestra.run.smithi121.stdout: ceph x86_64 2:16.2.4-0.el8 ceph 6.3 k 2024-02-15T18:22:43.373 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-02-15T18:22:43.373 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds x86_64 2:16.2.4-0.el8 ceph 2.4 M 2024-02-15T18:22:43.373 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr x86_64 2:16.2.4-0.el8 ceph 1.6 M 2024-02-15T18:22:43.373 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core noarch 2:16.2.4-0.el8 ceph-noarch 220 k 2024-02-15T18:22:43.373 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon x86_64 2:16.2.4-0.el8 ceph 5.0 M 2024-02-15T18:22:43.373 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd x86_64 2:16.2.4-0.el8 ceph 18 M 2024-02-15T18:22:43.373 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-02-15T18:22:43.373 INFO:teuthology.orchestra.run.smithi121.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-02-15T18:22:43.373 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-02-15T18:22:43.374 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-02-15T18:22:43.374 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-02-15T18:22:43.374 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-02-15T18:22:43.374 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-02-15T18:22:43.374 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-02-15T18:22:43.374 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-02-15T18:22:43.374 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-02-15T18:22:43.374 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-02-15T18:22:43.374 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-02-15T18:22:43.374 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-02-15T18:22:43.374 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-02-15T18:22:43.374 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-02-15T18:22:43.374 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-02-15T18:22:43.374 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-02-15T18:22:43.375 INFO:teuthology.orchestra.run.smithi121.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-02-15T18:22:43.375 INFO:teuthology.orchestra.run.smithi121.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-02-15T18:22:43.375 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-02-15T18:22:43.375 INFO:teuthology.orchestra.run.smithi121.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-02-15T18:22:43.375 INFO:teuthology.orchestra.run.smithi121.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-02-15T18:22:43.375 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-02-15T18:22:43.375 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-02-15T18:22:43.375 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-02-15T18:22:43.375 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-02-15T18:22:43.375 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:22:43.375 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-02-15T18:22:43.375 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:22:43.375 INFO:teuthology.orchestra.run.smithi121.stdout:Install 32 Packages 2024-02-15T18:22:43.375 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:22:43.376 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 31 M 2024-02-15T18:22:43.376 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 111 M 2024-02-15T18:22:43.376 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-02-15T18:22:43.757 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephsqlite-2:16.2.4-0.el8.x86_64 5/6 2024-02-15T18:22:43.789 INFO:teuthology.orchestra.run.smithi121.stdout:(1/32): ceph-16.2.4-0.el8.x86_64.rpm 121 kB/s | 6.3 kB 00:00 2024-02-15T18:22:43.840 INFO:teuthology.orchestra.run.smithi121.stdout:(2/32): ceph-mgr-16.2.4-0.el8.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-02-15T18:22:43.890 INFO:teuthology.orchestra.run.smithi121.stdout:(3/32): ceph-mds-16.2.4-0.el8.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-02-15T18:22:43.915 INFO:teuthology.orchestra.run.smithi121.stdout:(4/32): ceph-mgr-modules-core-16.2.4-0.el8.noar 8.7 MB/s | 220 kB 00:00 2024-02-15T18:22:44.024 INFO:teuthology.orchestra.run.smithi121.stdout:(5/32): ceph-mon-16.2.4-0.el8.x86_64.rpm 21 MB/s | 5.0 MB 00:00 2024-02-15T18:22:44.035 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-test-2:16.2.4-0.el8.x86_64 6/6 2024-02-15T18:22:44.258 INFO:teuthology.orchestra.run.smithi121.stdout:(6/32): ceph-osd-16.2.4-0.el8.x86_64.rpm 44 MB/s | 18 MB 00:00 2024-02-15T18:22:44.299 INFO:teuthology.orchestra.run.smithi121.stdout:(7/32): python3-markupsafe-0.23-19.el8.x86_64.r 141 kB/s | 39 kB 00:00 2024-02-15T18:22:44.317 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-test-2:16.2.4-0.el8.x86_64 6/6 2024-02-15T18:22:44.318 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-test-2:16.2.4-0.el8.x86_64 1/6 2024-02-15T18:22:44.318 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephsqlite-2:16.2.4-0.el8.x86_64 2/6 2024-02-15T18:22:44.318 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-02-15T18:22:44.318 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-02-15T18:22:44.318 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-02-15T18:22:44.324 INFO:teuthology.orchestra.run.smithi121.stdout:(8/32): python3-mako-1.0.6-14.el8.noarch.rpm 384 kB/s | 157 kB 00:00 2024-02-15T18:22:44.384 INFO:teuthology.orchestra.run.smithi121.stdout:(9/32): python3-pytz-2017.2-11.el8.noarch.rpm 681 kB/s | 57 kB 00:00 2024-02-15T18:22:44.442 INFO:teuthology.orchestra.run.smithi121.stdout:(10/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 562 kB/s | 103 kB 00:00 2024-02-15T18:22:44.509 INFO:teuthology.orchestra.run.smithi121.stdout:(11/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.9 MB/s | 249 kB 00:00 2024-02-15T18:22:44.534 INFO:teuthology.orchestra.run.smithi121.stdout:(12/32): python3-werkzeug-0.12.2-4.el8.noarch.r 2.1 MB/s | 457 kB 00:00 2024-02-15T18:22:44.560 INFO:teuthology.orchestra.run.smithi121.stdout:(13/32): python3-pycparser-2.14-14.el8.noarch.r 2.1 MB/s | 109 kB 00:00 2024-02-15T18:22:44.652 INFO:teuthology.orchestra.run.smithi121.stdout:(14/32): python3-cryptography-3.2.1-7.el8.x86_6 3.0 MB/s | 641 kB 00:00 2024-02-15T18:22:44.773 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-02-15T18:22:44.773 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:22:44.773 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-02-15T18:22:44.773 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test-2:16.2.4-0.el8.x86_64 jq-1.6-8.el8.x86_64 2024-02-15T18:22:44.773 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite-2:16.2.4-0.el8.x86_64 oniguruma-6.8.2-3.el8.x86_64 2024-02-15T18:22:44.773 INFO:teuthology.orchestra.run.smithi038.stdout: socat-1.7.4.1-1.el8.x86_64 xmlstarlet-1.6.1-20.el8.x86_64 2024-02-15T18:22:44.773 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:22:44.773 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:22:44.919 INFO:teuthology.orchestra.run.smithi121.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 113 kB/s | 44 kB 00:00 2024-02-15T18:22:45.030 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph 2024-02-15T18:22:45.060 INFO:teuthology.orchestra.run.smithi121.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 370 kB/s | 185 kB 00:00 2024-02-15T18:22:45.136 INFO:teuthology.orchestra.run.smithi121.stdout:(17/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 356 kB/s | 173 kB 00:00 2024-02-15T18:22:45.178 INFO:teuthology.orchestra.run.smithi121.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2024-02-15T18:22:45.245 INFO:teuthology.orchestra.run.smithi121.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 163 kB/s | 18 kB 00:00 2024-02-15T18:22:45.294 INFO:teuthology.orchestra.run.smithi121.stdout:(20/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.0 MB/s | 384 kB 00:00 2024-02-15T18:22:45.329 INFO:teuthology.orchestra.run.smithi121.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 323 kB/s | 49 kB 00:00 2024-02-15T18:22:45.354 INFO:teuthology.orchestra.run.smithi121.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 539 kB/s | 59 kB 00:00 2024-02-15T18:22:45.439 INFO:teuthology.orchestra.run.smithi121.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 143 kB/s | 16 kB 00:00 2024-02-15T18:22:45.463 INFO:teuthology.orchestra.run.smithi121.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 172 kB/s | 19 kB 00:00 2024-02-15T18:22:45.497 INFO:teuthology.orchestra.run.smithi121.stdout:(25/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.4 MB/s | 283 kB 00:00 2024-02-15T18:22:45.538 INFO:teuthology.orchestra.run.smithi121.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 238 kB/s | 24 kB 00:00 2024-02-15T18:22:45.572 INFO:teuthology.orchestra.run.smithi121.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 268 kB/s | 29 kB 00:00 2024-02-15T18:22:45.597 INFO:teuthology.orchestra.run.smithi121.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 274 kB/s | 27 kB 00:00 2024-02-15T18:22:45.706 INFO:teuthology.orchestra.run.smithi121.stdout:(29/32): python3-webtest-2.0.33-1.el8.noarch.rp 788 kB/s | 85 kB 00:00 2024-02-15T18:22:45.731 INFO:teuthology.orchestra.run.smithi121.stdout:(30/32): python3-waitress-1.4.3-1.el8.noarch.rp 1.3 MB/s | 255 kB 00:00 2024-02-15T18:22:45.756 INFO:teuthology.orchestra.run.smithi121.stdout:(31/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.3 MB/s | 251 kB 00:00 2024-02-15T18:22:45.815 INFO:teuthology.orchestra.run.smithi121.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 211 kB/s | 23 kB 00:00 2024-02-15T18:22:45.817 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:22:45.818 INFO:teuthology.orchestra.run.smithi121.stdout:Total 13 MB/s | 31 MB 00:02 2024-02-15T18:22:45.821 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:00:56 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:22:45.898 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-02-15T18:22:45.937 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-02-15T18:22:45.937 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-02-15T18:22:46.254 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-02-15T18:22:46.256 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-02-15T18:22:47.434 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-02-15T18:22:47.491 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:22:47.495 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:22:47.495 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-02-15T18:22:47.495 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:22:47.495 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-02-15T18:22:47.495 INFO:teuthology.orchestra.run.smithi038.stdout: ceph x86_64 2:16.2.4-0.el8 ceph 6.3 k 2024-02-15T18:22:47.495 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-02-15T18:22:47.495 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds x86_64 2:16.2.4-0.el8 ceph 2.4 M 2024-02-15T18:22:47.495 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr x86_64 2:16.2.4-0.el8 ceph 1.6 M 2024-02-15T18:22:47.495 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core noarch 2:16.2.4-0.el8 ceph-noarch 220 k 2024-02-15T18:22:47.496 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon x86_64 2:16.2.4-0.el8 ceph 5.0 M 2024-02-15T18:22:47.496 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd x86_64 2:16.2.4-0.el8 ceph 18 M 2024-02-15T18:22:47.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-02-15T18:22:47.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-02-15T18:22:47.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-02-15T18:22:47.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-02-15T18:22:47.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-02-15T18:22:47.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-02-15T18:22:47.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-02-15T18:22:47.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-02-15T18:22:47.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-02-15T18:22:47.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-02-15T18:22:47.496 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-02-15T18:22:47.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-02-15T18:22:47.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-02-15T18:22:47.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-02-15T18:22:47.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-02-15T18:22:47.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-02-15T18:22:47.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-02-15T18:22:47.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-02-15T18:22:47.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-02-15T18:22:47.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-02-15T18:22:47.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-02-15T18:22:47.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-02-15T18:22:47.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-02-15T18:22:47.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-02-15T18:22:47.497 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-02-15T18:22:47.498 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-02-15T18:22:47.498 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:22:47.498 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-02-15T18:22:47.498 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:22:47.498 INFO:teuthology.orchestra.run.smithi038.stdout:Install 32 Packages 2024-02-15T18:22:47.498 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:22:47.498 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 31 M 2024-02-15T18:22:47.498 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 111 M 2024-02-15T18:22:47.499 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-02-15T18:22:47.644 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-02-15T18:22:47.830 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-02-15T18:22:47.938 INFO:teuthology.orchestra.run.smithi038.stdout:(1/32): ceph-16.2.4-0.el8.x86_64.rpm 86 kB/s | 6.3 kB 00:00 2024-02-15T18:22:47.989 INFO:teuthology.orchestra.run.smithi038.stdout:(2/32): ceph-mgr-16.2.4-0.el8.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-02-15T18:22:48.048 INFO:teuthology.orchestra.run.smithi038.stdout:(3/32): ceph-mds-16.2.4-0.el8.x86_64.rpm 13 MB/s | 2.4 MB 00:00 2024-02-15T18:22:48.081 INFO:teuthology.orchestra.run.smithi038.stdout:(4/32): ceph-mgr-modules-core-16.2.4-0.el8.noar 6.5 MB/s | 220 kB 00:00 2024-02-15T18:22:48.094 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-02-15T18:22:48.190 INFO:teuthology.orchestra.run.smithi038.stdout:(5/32): ceph-mon-16.2.4-0.el8.x86_64.rpm 20 MB/s | 5.0 MB 00:00 2024-02-15T18:22:48.285 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-02-15T18:22:48.432 INFO:teuthology.orchestra.run.smithi038.stdout:(6/32): ceph-osd-16.2.4-0.el8.x86_64.rpm 41 MB/s | 18 MB 00:00 2024-02-15T18:22:48.482 INFO:teuthology.orchestra.run.smithi038.stdout:(7/32): python3-markupsafe-0.23-19.el8.x86_64.r 133 kB/s | 39 kB 00:00 2024-02-15T18:22:48.483 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-02-15T18:22:48.507 INFO:teuthology.orchestra.run.smithi038.stdout:(8/32): python3-mako-1.0.6-14.el8.noarch.rpm 369 kB/s | 157 kB 00:00 2024-02-15T18:22:48.558 INFO:teuthology.orchestra.run.smithi038.stdout:(9/32): python3-pytz-2017.2-11.el8.noarch.rpm 759 kB/s | 57 kB 00:00 2024-02-15T18:22:48.625 INFO:teuthology.orchestra.run.smithi038.stdout:(10/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 538 kB/s | 103 kB 00:00 2024-02-15T18:22:48.671 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-02-15T18:22:48.692 INFO:teuthology.orchestra.run.smithi038.stdout:(11/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.8 MB/s | 249 kB 00:00 2024-02-15T18:22:48.748 INFO:teuthology.orchestra.run.smithi038.stdout:(12/32): python3-werkzeug-0.12.2-4.el8.noarch.r 1.9 MB/s | 457 kB 00:00 2024-02-15T18:22:48.775 INFO:teuthology.orchestra.run.smithi038.stdout:(13/32): python3-pycparser-2.14-14.el8.noarch.r 1.3 MB/s | 109 kB 00:00 2024-02-15T18:22:48.831 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-02-15T18:22:48.834 INFO:teuthology.orchestra.run.smithi038.stdout:(14/32): python3-cryptography-3.2.1-7.el8.x86_6 3.0 MB/s | 641 kB 00:00 2024-02-15T18:22:48.901 INFO:teuthology.orchestra.run.smithi038.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 287 kB/s | 44 kB 00:00 2024-02-15T18:22:48.926 INFO:teuthology.orchestra.run.smithi038.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.2 MB/s | 185 kB 00:00 2024-02-15T18:22:48.977 INFO:teuthology.orchestra.run.smithi038.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 223 kB/s | 11 kB 00:00 2024-02-15T18:22:49.009 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-02-15T18:22:49.027 INFO:teuthology.orchestra.run.smithi038.stdout:(18/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 898 kB/s | 173 kB 00:00 2024-02-15T18:22:49.061 INFO:teuthology.orchestra.run.smithi038.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 210 kB/s | 18 kB 00:00 2024-02-15T18:22:49.086 INFO:teuthology.orchestra.run.smithi038.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 835 kB/s | 49 kB 00:00 2024-02-15T18:22:49.111 INFO:teuthology.orchestra.run.smithi038.stdout:(21/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.8 MB/s | 384 kB 00:00 2024-02-15T18:22:49.136 INFO:teuthology.orchestra.run.smithi038.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 779 kB/s | 59 kB 00:00 2024-02-15T18:22:49.170 INFO:teuthology.orchestra.run.smithi038.stdout:(23/32): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2024-02-15T18:22:49.182 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-02-15T18:22:49.195 INFO:teuthology.orchestra.run.smithi038.stdout:(24/32): python3-portend-2.6-1.el8.noarch.rpm 186 kB/s | 16 kB 00:00 2024-02-15T18:22:49.221 INFO:teuthology.orchestra.run.smithi038.stdout:(25/32): python3-simplegeneric-0.8.1-17.el8.noa 223 kB/s | 19 kB 00:00 2024-02-15T18:22:49.246 INFO:teuthology.orchestra.run.smithi038.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 316 kB/s | 24 kB 00:00 2024-02-15T18:22:49.271 INFO:teuthology.orchestra.run.smithi038.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 387 kB/s | 29 kB 00:00 2024-02-15T18:22:49.329 INFO:teuthology.orchestra.run.smithi038.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 253 kB/s | 27 kB 00:00 2024-02-15T18:22:49.355 INFO:teuthology.orchestra.run.smithi038.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 2.3 MB/s | 255 kB 00:00 2024-02-15T18:22:49.380 INFO:teuthology.orchestra.run.smithi038.stdout:(30/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.3 MB/s | 251 kB 00:00 2024-02-15T18:22:49.406 INFO:teuthology.orchestra.run.smithi038.stdout:(31/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.1 MB/s | 85 kB 00:00 2024-02-15T18:22:49.412 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-02-15T18:22:49.439 INFO:teuthology.orchestra.run.smithi038.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 392 kB/s | 23 kB 00:00 2024-02-15T18:22:49.440 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:22:49.441 INFO:teuthology.orchestra.run.smithi038.stdout:Total 16 MB/s | 31 MB 00:01 2024-02-15T18:22:49.523 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-02-15T18:22:49.565 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-02-15T18:22:49.565 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-02-15T18:22:49.600 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-02-15T18:22:49.857 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-02-15T18:22:49.888 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-02-15T18:22:49.892 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-02-15T18:22:50.246 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-02-15T18:22:50.506 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-02-15T18:22:50.694 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-02-15T18:22:50.938 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-02-15T18:22:51.093 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-02-15T18:22:51.129 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-15T18:22:51.446 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-02-15T18:22:51.452 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-02-15T18:22:51.634 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-02-15T18:22:51.668 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-02-15T18:22:51.822 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-02-15T18:22:51.958 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-02-15T18:22:52.025 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-02-15T18:22:52.192 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-02-15T18:22:52.267 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-02-15T18:22:52.373 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-02-15T18:22:52.482 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-02-15T18:22:52.589 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-02-15T18:22:52.718 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-02-15T18:22:52.838 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-02-15T18:22:53.038 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-02-15T18:22:53.049 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-02-15T18:22:53.261 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-02-15T18:22:53.325 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-02-15T18:22:53.506 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-02-15T18:22:53.726 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 27/32 2024-02-15T18:22:53.760 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-02-15T18:22:53.939 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-2:16.2.4-0.el8.x86_64 28/32 2024-02-15T18:22:54.035 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-02-15T18:22:54.440 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-02-15T18:22:54.780 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-02-15T18:22:55.001 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-02-15T18:22:55.248 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-02-15T18:22:55.453 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-15T18:22:55.474 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-2:16.2.4-0.el8.x86_64 28/32 2024-02-15T18:22:55.721 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-osd-2:16.2.4-0.el8.x86_64 29/32 2024-02-15T18:22:55.779 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-02-15T18:22:56.026 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-02-15T18:22:56.154 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-osd-2:16.2.4-0.el8.x86_64 29/32 2024-02-15T18:22:56.238 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-02-15T18:22:56.396 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mon-2:16.2.4-0.el8.x86_64 30/32 2024-02-15T18:22:56.457 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-02-15T18:22:56.618 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mon-2:16.2.4-0.el8.x86_64 30/32 2024-02-15T18:22:56.752 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-02-15T18:22:56.827 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mds-2:16.2.4-0.el8.x86_64 31/32 2024-02-15T18:22:56.844 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mds-2:16.2.4-0.el8.x86_64 31/32 2024-02-15T18:22:56.977 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-2:16.2.4-0.el8.x86_64 32/32 2024-02-15T18:22:57.006 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-02-15T18:22:57.273 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-02-15T18:22:57.415 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-2:16.2.4-0.el8.x86_64 32/32 2024-02-15T18:22:57.415 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-2:16.2.4-0.el8.x86_64 1/32 2024-02-15T18:22:57.415 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mds-2:16.2.4-0.el8.x86_64 2/32 2024-02-15T18:22:57.415 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-2:16.2.4-0.el8.x86_64 3/32 2024-02-15T18:22:57.415 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mon-2:16.2.4-0.el8.x86_64 4/32 2024-02-15T18:22:57.415 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-osd-2:16.2.4-0.el8.x86_64 5/32 2024-02-15T18:22:57.415 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 6/32 2024-02-15T18:22:57.416 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-02-15T18:22:57.416 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-02-15T18:22:57.416 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-02-15T18:22:57.416 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-02-15T18:22:57.416 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-02-15T18:22:57.416 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-02-15T18:22:57.416 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-02-15T18:22:57.416 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-02-15T18:22:57.416 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-02-15T18:22:57.416 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-02-15T18:22:57.416 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-15T18:22:57.417 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-02-15T18:22:57.417 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-02-15T18:22:57.417 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-02-15T18:22:57.417 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-02-15T18:22:57.417 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-02-15T18:22:57.417 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-02-15T18:22:57.417 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-02-15T18:22:57.417 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-02-15T18:22:57.417 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-02-15T18:22:57.417 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-02-15T18:22:57.417 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-02-15T18:22:57.417 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-02-15T18:22:57.418 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-02-15T18:22:57.418 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-02-15T18:22:57.671 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-02-15T18:22:57.998 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-02-15T18:22:58.272 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-02-15T18:22:58.272 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:22:58.273 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-02-15T18:22:58.273 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:58.273 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:58.273 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:58.273 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 2024-02-15T18:22:58.273 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:58.273 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd-2:16.2.4-0.el8.x86_64 2024-02-15T18:22:58.273 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-02-15T18:22:58.273 INFO:teuthology.orchestra.run.smithi121.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-02-15T18:22:58.273 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-02-15T18:22:58.273 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-02-15T18:22:58.273 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-02-15T18:22:58.274 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-02-15T18:22:58.274 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-6.2-6.el8.noarch 2024-02-15T18:22:58.274 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-02-15T18:22:58.274 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-02-15T18:22:58.274 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako-1.0.6-14.el8.noarch 2024-02-15T18:22:58.274 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-02-15T18:22:58.274 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-02-15T18:22:58.274 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-02-15T18:22:58.274 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend-2.6-1.el8.noarch 2024-02-15T18:22:58.274 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-02-15T18:22:58.274 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser-2.14-14.el8.noarch 2024-02-15T18:22:58.274 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz-2017.2-11.el8.noarch 2024-02-15T18:22:58.275 INFO:teuthology.orchestra.run.smithi121.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-02-15T18:22:58.275 INFO:teuthology.orchestra.run.smithi121.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-02-15T18:22:58.275 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-02-15T18:22:58.275 INFO:teuthology.orchestra.run.smithi121.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-02-15T18:22:58.275 INFO:teuthology.orchestra.run.smithi121.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-02-15T18:22:58.275 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-02-15T18:22:58.275 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-02-15T18:22:58.275 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-02-15T18:22:58.275 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-02-15T18:22:58.275 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:22:58.275 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:22:58.418 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 27/32 2024-02-15T18:22:58.616 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-base 2024-02-15T18:22:58.721 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-2:16.2.4-0.el8.x86_64 28/32 2024-02-15T18:22:59.375 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:10 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:23:00.352 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-2:16.2.4-0.el8.x86_64 28/32 2024-02-15T18:23:00.546 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-osd-2:16.2.4-0.el8.x86_64 29/32 2024-02-15T18:23:00.877 INFO:teuthology.orchestra.run.smithi121.stdout:Package ceph-base-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:23:00.923 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:23:00.924 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-02-15T18:23:00.924 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:23:01.005 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-osd-2:16.2.4-0.el8.x86_64 29/32 2024-02-15T18:23:01.254 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mon-2:16.2.4-0.el8.x86_64 30/32 2024-02-15T18:23:01.288 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install cephadm 2024-02-15T18:23:01.490 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mon-2:16.2.4-0.el8.x86_64 30/32 2024-02-15T18:23:01.675 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mds-2:16.2.4-0.el8.x86_64 31/32 2024-02-15T18:23:01.693 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mds-2:16.2.4-0.el8.x86_64 31/32 2024-02-15T18:23:01.867 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-2:16.2.4-0.el8.x86_64 32/32 2024-02-15T18:23:02.034 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:13 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:23:02.286 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-2:16.2.4-0.el8.x86_64 32/32 2024-02-15T18:23:02.286 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-2:16.2.4-0.el8.x86_64 1/32 2024-02-15T18:23:02.286 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mds-2:16.2.4-0.el8.x86_64 2/32 2024-02-15T18:23:02.287 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-2:16.2.4-0.el8.x86_64 3/32 2024-02-15T18:23:02.287 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mon-2:16.2.4-0.el8.x86_64 4/32 2024-02-15T18:23:02.287 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-osd-2:16.2.4-0.el8.x86_64 5/32 2024-02-15T18:23:02.287 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 6/32 2024-02-15T18:23:02.287 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-02-15T18:23:02.287 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-02-15T18:23:02.287 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-02-15T18:23:02.287 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-02-15T18:23:02.287 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-02-15T18:23:02.287 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-02-15T18:23:02.287 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-02-15T18:23:02.287 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-02-15T18:23:02.287 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-02-15T18:23:02.288 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-02-15T18:23:02.288 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-15T18:23:02.288 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-02-15T18:23:02.288 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-02-15T18:23:02.288 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-02-15T18:23:02.288 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-02-15T18:23:02.288 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-02-15T18:23:02.288 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-02-15T18:23:02.288 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-02-15T18:23:02.288 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-02-15T18:23:02.288 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-02-15T18:23:02.288 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-02-15T18:23:02.288 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-02-15T18:23:02.289 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-02-15T18:23:02.289 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-02-15T18:23:02.289 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-02-15T18:23:03.137 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-02-15T18:23:03.138 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:03.138 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-02-15T18:23:03.138 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-2:16.2.4-0.el8.x86_64 2024-02-15T18:23:03.138 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds-2:16.2.4-0.el8.x86_64 2024-02-15T18:23:03.138 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-2:16.2.4-0.el8.x86_64 2024-02-15T18:23:03.138 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 2024-02-15T18:23:03.138 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon-2:16.2.4-0.el8.x86_64 2024-02-15T18:23:03.138 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd-2:16.2.4-0.el8.x86_64 2024-02-15T18:23:03.138 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-02-15T18:23:03.138 INFO:teuthology.orchestra.run.smithi038.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-02-15T18:23:03.138 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-02-15T18:23:03.138 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-02-15T18:23:03.138 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-02-15T18:23:03.138 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-02-15T18:23:03.139 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-6.2-6.el8.noarch 2024-02-15T18:23:03.139 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-02-15T18:23:03.139 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-02-15T18:23:03.139 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako-1.0.6-14.el8.noarch 2024-02-15T18:23:03.139 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-02-15T18:23:03.139 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-02-15T18:23:03.139 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-02-15T18:23:03.139 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend-2.6-1.el8.noarch 2024-02-15T18:23:03.139 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-02-15T18:23:03.139 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser-2.14-14.el8.noarch 2024-02-15T18:23:03.139 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz-2017.2-11.el8.noarch 2024-02-15T18:23:03.139 INFO:teuthology.orchestra.run.smithi038.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-02-15T18:23:03.139 INFO:teuthology.orchestra.run.smithi038.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-02-15T18:23:03.140 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-02-15T18:23:03.140 INFO:teuthology.orchestra.run.smithi038.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-02-15T18:23:03.140 INFO:teuthology.orchestra.run.smithi038.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-02-15T18:23:03.140 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-02-15T18:23:03.140 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-02-15T18:23:03.140 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-02-15T18:23:03.140 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-02-15T18:23:03.140 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:03.140 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:23:03.401 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-base 2024-02-15T18:23:03.581 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:23:03.581 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:03.582 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2024-02-15T18:23:03.582 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:03.582 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-02-15T18:23:03.582 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm noarch 2:16.2.4-0.el8 ceph-noarch 71 k 2024-02-15T18:23:03.582 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:03.582 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-02-15T18:23:03.582 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:03.582 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-02-15T18:23:03.582 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:03.582 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 71 k 2024-02-15T18:23:03.582 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 290 k 2024-02-15T18:23:03.583 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-02-15T18:23:03.640 INFO:teuthology.orchestra.run.smithi121.stdout:cephadm-16.2.4-0.el8.noarch.rpm 1.3 MB/s | 71 kB 00:00 2024-02-15T18:23:03.640 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:23:03.640 INFO:teuthology.orchestra.run.smithi121.stdout:Total 1.3 MB/s | 71 kB 00:00 2024-02-15T18:23:03.640 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-02-15T18:23:03.644 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-02-15T18:23:03.644 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-02-15T18:23:03.646 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-02-15T18:23:03.649 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-02-15T18:23:03.690 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-02-15T18:23:04.191 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:15 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:23:04.242 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephadm-2:16.2.4-0.el8.noarch 1/1 2024-02-15T18:23:04.399 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephadm-2:16.2.4-0.el8.noarch 1/1 2024-02-15T18:23:04.625 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephadm-2:16.2.4-0.el8.noarch 1/1 2024-02-15T18:23:04.996 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephadm-2:16.2.4-0.el8.noarch 1/1 2024-02-15T18:23:04.996 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:04.997 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-02-15T18:23:04.997 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm-2:16.2.4-0.el8.noarch 2024-02-15T18:23:04.997 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:04.997 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:23:05.225 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-immutable-object-cache 2024-02-15T18:23:05.810 INFO:teuthology.orchestra.run.smithi038.stdout:Package ceph-base-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:23:05.861 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:23:05.862 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2024-02-15T18:23:05.862 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:23:05.982 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:16 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:23:05.999 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install cephadm 2024-02-15T18:23:06.791 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:17 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:23:07.542 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:23:07.543 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:07.543 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repo Size 2024-02-15T18:23:07.543 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:07.543 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-02-15T18:23:07.543 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache x86_64 2:16.2.4-0.el8 ceph 163 k 2024-02-15T18:23:07.544 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:07.544 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-02-15T18:23:07.544 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:07.544 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-02-15T18:23:07.544 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:07.544 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 163 k 2024-02-15T18:23:07.544 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 587 k 2024-02-15T18:23:07.544 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-02-15T18:23:07.600 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-immutable-object-cache-16.2.4-0.el8.x86_64 2.8 MB/s | 163 kB 00:00 2024-02-15T18:23:07.601 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:23:07.601 INFO:teuthology.orchestra.run.smithi121.stdout:Total 2.8 MB/s | 163 kB 00:00 2024-02-15T18:23:07.601 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-02-15T18:23:07.611 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-02-15T18:23:07.611 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-02-15T18:23:07.658 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-02-15T18:23:07.661 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-02-15T18:23:07.771 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-02-15T18:23:07.975 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:23:08.385 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:23:08.462 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:23:08.462 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:23:08.462 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-02-15T18:23:08.462 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:23:08.462 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-02-15T18:23:08.462 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm noarch 2:16.2.4-0.el8 ceph-noarch 71 k 2024-02-15T18:23:08.463 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:08.463 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-02-15T18:23:08.463 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:23:08.463 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2024-02-15T18:23:08.463 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:08.463 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 71 k 2024-02-15T18:23:08.463 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 290 k 2024-02-15T18:23:08.463 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-02-15T18:23:08.512 INFO:teuthology.orchestra.run.smithi038.stdout:cephadm-16.2.4-0.el8.noarch.rpm 1.4 MB/s | 71 kB 00:00 2024-02-15T18:23:08.512 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:23:08.513 INFO:teuthology.orchestra.run.smithi038.stdout:Total 1.4 MB/s | 71 kB 00:00 2024-02-15T18:23:08.513 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-02-15T18:23:08.520 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-02-15T18:23:08.520 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-02-15T18:23:08.522 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-02-15T18:23:08.526 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-02-15T18:23:08.566 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-02-15T18:23:08.776 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:23:08.777 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:08.777 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-02-15T18:23:08.777 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 2024-02-15T18:23:08.777 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:08.777 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:23:08.983 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr 2024-02-15T18:23:09.088 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephadm-2:16.2.4-0.el8.noarch 1/1 2024-02-15T18:23:09.299 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephadm-2:16.2.4-0.el8.noarch 1/1 2024-02-15T18:23:09.537 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephadm-2:16.2.4-0.el8.noarch 1/1 2024-02-15T18:23:09.750 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:20 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:23:10.537 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephadm-2:16.2.4-0.el8.noarch 1/1 2024-02-15T18:23:10.537 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:10.537 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-02-15T18:23:10.537 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm-2:16.2.4-0.el8.noarch 2024-02-15T18:23:10.537 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:10.537 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:23:10.938 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-immutable-object-cache 2024-02-15T18:23:11.249 INFO:teuthology.orchestra.run.smithi121.stdout:Package ceph-mgr-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:23:11.297 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:23:11.298 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-02-15T18:23:11.298 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:23:11.414 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr-dashboard 2024-02-15T18:23:11.738 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:22 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:23:12.163 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:23 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:23:13.414 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:23:13.415 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:23:13.415 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repo Size 2024-02-15T18:23:13.415 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:23:13.416 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-02-15T18:23:13.416 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache x86_64 2:16.2.4-0.el8 ceph 163 k 2024-02-15T18:23:13.416 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:13.416 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-02-15T18:23:13.416 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:23:13.416 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2024-02-15T18:23:13.416 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:13.416 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 163 k 2024-02-15T18:23:13.417 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 587 k 2024-02-15T18:23:13.417 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-02-15T18:23:13.468 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-immutable-object-cache-16.2.4-0.el8.x86_64 3.1 MB/s | 163 kB 00:00 2024-02-15T18:23:13.468 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:23:13.468 INFO:teuthology.orchestra.run.smithi038.stdout:Total 3.0 MB/s | 163 kB 00:00 2024-02-15T18:23:13.469 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-02-15T18:23:13.480 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-02-15T18:23:13.481 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-02-15T18:23:13.530 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-02-15T18:23:13.535 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-02-15T18:23:13.683 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-02-15T18:23:13.714 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:23:13.715 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:13.715 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-02-15T18:23:13.715 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:13.715 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-02-15T18:23:13.715 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard noarch 2:16.2.4-0.el8 ceph-noarch 2.7 M 2024-02-15T18:23:13.715 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-02-15T18:23:13.715 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards noarch 2:16.2.4-0.el8 ceph-noarch 24 k 2024-02-15T18:23:13.715 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts noarch 2:16.2.4-0.el8 ceph-noarch 8.9 k 2024-02-15T18:23:13.715 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-02-15T18:23:13.715 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-02-15T18:23:13.715 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-02-15T18:23:13.716 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:13.716 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-02-15T18:23:13.716 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:13.716 INFO:teuthology.orchestra.run.smithi121.stdout:Install 6 Packages 2024-02-15T18:23:13.716 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:13.717 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 3.0 M 2024-02-15T18:23:13.717 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 15 M 2024-02-15T18:23:13.717 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-02-15T18:23:13.865 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:23:14.212 INFO:teuthology.orchestra.run.smithi121.stdout:(1/6): ceph-grafana-dashboards-16.2.4-0.el8.noa 507 kB/s | 24 kB 00:00 2024-02-15T18:23:14.237 INFO:teuthology.orchestra.run.smithi121.stdout:(2/6): ceph-prometheus-alerts-16.2.4-0.el8.noar 123 kB/s | 8.9 kB 00:00 2024-02-15T18:23:14.296 INFO:teuthology.orchestra.run.smithi121.stdout:(3/6): ceph-mgr-dashboard-16.2.4-0.el8.noarch.r 20 MB/s | 2.7 MB 00:00 2024-02-15T18:23:14.297 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:23:14.404 INFO:teuthology.orchestra.run.smithi121.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 221 kB/s | 43 kB 00:00 2024-02-15T18:23:14.493 INFO:teuthology.orchestra.run.smithi121.stdout:(5/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 132 kB/s | 34 kB 00:00 2024-02-15T18:23:14.554 INFO:teuthology.orchestra.run.smithi121.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 773 kB/s | 196 kB 00:00 2024-02-15T18:23:14.557 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:23:14.558 INFO:teuthology.orchestra.run.smithi121.stdout:Total 3.5 MB/s | 3.0 MB 00:00 2024-02-15T18:23:14.567 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-02-15T18:23:14.577 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-02-15T18:23:14.577 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-02-15T18:23:14.691 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:23:14.692 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:14.692 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-02-15T18:23:14.692 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 2024-02-15T18:23:14.692 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:14.692 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:23:14.745 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-02-15T18:23:14.748 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-02-15T18:23:14.906 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-mgr 2024-02-15T18:23:15.039 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-02-15T18:23:15.270 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-02-15T18:23:15.499 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-02-15T18:23:15.703 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:26 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:23:15.725 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-02-15T18:23:15.862 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 4/6 2024-02-15T18:23:16.827 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 5/6 2024-02-15T18:23:17.108 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 6/6 2024-02-15T18:23:17.313 INFO:teuthology.orchestra.run.smithi038.stdout:Package ceph-mgr-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:23:17.363 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:23:17.365 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2024-02-15T18:23:17.365 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:23:17.437 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 6/6 2024-02-15T18:23:17.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 1/6 2024-02-15T18:23:17.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 2/6 2024-02-15T18:23:17.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 3/6 2024-02-15T18:23:17.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-02-15T18:23:17.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-02-15T18:23:17.497 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-mgr-dashboard 2024-02-15T18:23:17.826 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-02-15T18:23:17.826 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:17.826 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-02-15T18:23:17.827 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 2024-02-15T18:23:17.827 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 2024-02-15T18:23:17.827 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 2024-02-15T18:23:17.827 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-02-15T18:23:17.827 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-02-15T18:23:17.827 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes-2.4.1-12.el8.noarch 2024-02-15T18:23:17.827 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:17.827 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:23:17.980 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-15T18:23:18.299 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:29 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:23:18.735 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:29 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:23:19.968 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:23:19.970 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:23:19.970 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-02-15T18:23:19.970 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:23:19.970 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-02-15T18:23:19.970 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard noarch 2:16.2.4-0.el8 ceph-noarch 2.7 M 2024-02-15T18:23:19.970 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-02-15T18:23:19.970 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards noarch 2:16.2.4-0.el8 ceph-noarch 24 k 2024-02-15T18:23:19.970 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts noarch 2:16.2.4-0.el8 ceph-noarch 8.9 k 2024-02-15T18:23:19.970 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-02-15T18:23:19.970 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-02-15T18:23:19.970 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-02-15T18:23:19.970 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:19.971 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-02-15T18:23:19.971 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:23:19.971 INFO:teuthology.orchestra.run.smithi038.stdout:Install 6 Packages 2024-02-15T18:23:19.971 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:19.971 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 3.0 M 2024-02-15T18:23:19.971 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 15 M 2024-02-15T18:23:19.971 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-02-15T18:23:20.263 INFO:teuthology.orchestra.run.smithi038.stdout:(1/6): ceph-grafana-dashboards-16.2.4-0.el8.noa 348 kB/s | 24 kB 00:00 2024-02-15T18:23:20.274 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:23:20.275 INFO:teuthology.orchestra.run.smithi121.stdout:============================================================================================ 2024-02-15T18:23:20.275 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-02-15T18:23:20.275 INFO:teuthology.orchestra.run.smithi121.stdout:============================================================================================ 2024-02-15T18:23:20.275 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-02-15T18:23:20.276 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.4-0.el8 ceph-noarch 8.6 M 2024-02-15T18:23:20.276 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-02-15T18:23:20.276 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-02-15T18:23:20.276 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-02-15T18:23:20.276 INFO:teuthology.orchestra.run.smithi121.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-02-15T18:23:20.276 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-02-15T18:23:20.276 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-02-15T18:23:20.276 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-02-15T18:23:20.276 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-02-15T18:23:20.276 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:20.276 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-02-15T18:23:20.276 INFO:teuthology.orchestra.run.smithi121.stdout:============================================================================================ 2024-02-15T18:23:20.276 INFO:teuthology.orchestra.run.smithi121.stdout:Install 8 Packages 2024-02-15T18:23:20.276 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:20.277 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 37 M 2024-02-15T18:23:20.277 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 210 M 2024-02-15T18:23:20.277 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-02-15T18:23:20.288 INFO:teuthology.orchestra.run.smithi038.stdout:(2/6): ceph-prometheus-alerts-16.2.4-0.el8.noar 95 kB/s | 8.9 kB 00:00 2024-02-15T18:23:20.389 INFO:teuthology.orchestra.run.smithi038.stdout:(3/6): ceph-mgr-dashboard-16.2.4-0.el8.noarch.r 14 MB/s | 2.7 MB 00:00 2024-02-15T18:23:20.489 INFO:teuthology.orchestra.run.smithi038.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 168 kB/s | 34 kB 00:00 2024-02-15T18:23:20.510 INFO:teuthology.orchestra.run.smithi121.stdout:(1/8): ceph-mgr-diskprediction-local-16.2.4-0.e 56 MB/s | 8.6 MB 00:00 2024-02-15T18:23:20.556 INFO:teuthology.orchestra.run.smithi038.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.2 MB/s | 196 kB 00:00 2024-02-15T18:23:20.656 INFO:teuthology.orchestra.run.smithi038.stdout:(6/6): python3-jwt-1.6.1-2.el8.noarch.rpm 109 kB/s | 43 kB 00:00 2024-02-15T18:23:20.660 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:23:20.660 INFO:teuthology.orchestra.run.smithi038.stdout:Total 4.3 MB/s | 3.0 MB 00:00 2024-02-15T18:23:20.671 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-02-15T18:23:20.682 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-02-15T18:23:20.682 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-02-15T18:23:20.847 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-02-15T18:23:20.850 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-02-15T18:23:21.251 INFO:teuthology.orchestra.run.smithi121.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 5.0 MB/s | 3.7 MB 00:00 2024-02-15T18:23:21.315 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-02-15T18:23:21.318 INFO:teuthology.orchestra.run.smithi121.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 3.3 MB/s | 225 kB 00:00 2024-02-15T18:23:21.560 INFO:teuthology.orchestra.run.smithi121.stdout:(4/8): openblas-0.3.15-4.el8.x86_64.rpm 3.9 MB/s | 4.7 MB 00:01 2024-02-15T18:23:21.591 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-02-15T18:23:21.652 INFO:teuthology.orchestra.run.smithi121.stdout:(5/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 3.8 MB/s | 4.9 MB 00:01 2024-02-15T18:23:21.861 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-02-15T18:23:22.119 INFO:teuthology.orchestra.run.smithi121.stdout:(6/8): libquadmath-8.5.0-21.el8.x86_64.rpm 370 kB/s | 172 kB 00:00 2024-02-15T18:23:22.167 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-02-15T18:23:22.263 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 4/6 2024-02-15T18:23:22.519 INFO:teuthology.orchestra.run.smithi121.stdout:(7/8): python3-scipy-1.0.0-21.module_el8+762+77 12 MB/s | 14 MB 00:01 2024-02-15T18:23:22.544 INFO:teuthology.orchestra.run.smithi121.stdout:(8/8): libgfortran-8.5.0-21.el8.x86_64.rpm 656 kB/s | 645 kB 00:00 2024-02-15T18:23:22.545 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:23:22.545 INFO:teuthology.orchestra.run.smithi121.stdout:Total 16 MB/s | 37 MB 00:02 2024-02-15T18:23:22.775 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-02-15T18:23:22.793 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-02-15T18:23:22.793 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-02-15T18:23:23.118 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-02-15T18:23:23.121 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-02-15T18:23:23.355 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 5/6 2024-02-15T18:23:23.617 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 6/6 2024-02-15T18:23:23.902 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 6/6 2024-02-15T18:23:23.902 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 1/6 2024-02-15T18:23:23.903 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 2/6 2024-02-15T18:23:23.903 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 3/6 2024-02-15T18:23:23.903 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-02-15T18:23:23.903 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-02-15T18:23:24.308 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-02-15T18:23:24.308 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:24.309 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-02-15T18:23:24.309 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 2024-02-15T18:23:24.309 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 2024-02-15T18:23:24.309 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 2024-02-15T18:23:24.309 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-02-15T18:23:24.309 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-02-15T18:23:24.309 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes-2.4.1-12.el8.noarch 2024-02-15T18:23:24.309 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:24.309 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:23:24.437 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-15T18:23:24.859 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-02-15T18:23:25.004 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-15T18:23:25.172 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-15T18:23:25.230 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:36 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:23:25.340 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-15T18:23:25.967 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-15T18:23:26.187 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-02-15T18:23:26.804 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-02-15T18:23:26.891 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:23:26.892 INFO:teuthology.orchestra.run.smithi038.stdout:============================================================================================ 2024-02-15T18:23:26.892 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-02-15T18:23:26.892 INFO:teuthology.orchestra.run.smithi038.stdout:============================================================================================ 2024-02-15T18:23:26.892 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-02-15T18:23:26.893 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.4-0.el8 ceph-noarch 8.6 M 2024-02-15T18:23:26.893 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-02-15T18:23:26.893 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-02-15T18:23:26.893 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-02-15T18:23:26.893 INFO:teuthology.orchestra.run.smithi038.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-02-15T18:23:26.893 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-02-15T18:23:26.893 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-02-15T18:23:26.893 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-02-15T18:23:26.893 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-02-15T18:23:26.893 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:26.893 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-02-15T18:23:26.893 INFO:teuthology.orchestra.run.smithi038.stdout:============================================================================================ 2024-02-15T18:23:26.893 INFO:teuthology.orchestra.run.smithi038.stdout:Install 8 Packages 2024-02-15T18:23:26.893 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:26.894 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 37 M 2024-02-15T18:23:26.894 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 210 M 2024-02-15T18:23:26.894 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-02-15T18:23:26.956 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-15T18:23:27.226 INFO:teuthology.orchestra.run.smithi038.stdout:(1/8): ceph-mgr-diskprediction-local-16.2.4-0.e 34 MB/s | 8.6 MB 00:00 2024-02-15T18:23:27.641 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-15T18:23:27.655 INFO:teuthology.orchestra.run.smithi038.stdout:(2/8): openblas-0.3.15-4.el8.x86_64.rpm 7.0 MB/s | 4.7 MB 00:00 2024-02-15T18:23:27.722 INFO:teuthology.orchestra.run.smithi038.stdout:(3/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 6.5 MB/s | 4.9 MB 00:00 2024-02-15T18:23:27.747 INFO:teuthology.orchestra.run.smithi038.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 2.4 MB/s | 225 kB 00:00 2024-02-15T18:23:28.006 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-02-15T18:23:28.297 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-15T18:23:28.531 INFO:teuthology.orchestra.run.smithi038.stdout:(5/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 2.9 MB/s | 3.7 MB 00:01 2024-02-15T18:23:28.623 INFO:teuthology.orchestra.run.smithi038.stdout:(6/8): libgfortran-8.5.0-21.el8.x86_64.rpm 737 kB/s | 645 kB 00:00 2024-02-15T18:23:29.007 INFO:teuthology.orchestra.run.smithi038.stdout:(7/8): python3-scipy-1.0.0-21.module_el8+762+77 11 MB/s | 14 MB 00:01 2024-02-15T18:23:29.031 INFO:teuthology.orchestra.run.smithi038.stdout:(8/8): libquadmath-8.5.0-21.el8.x86_64.rpm 345 kB/s | 172 kB 00:00 2024-02-15T18:23:29.033 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:23:29.033 INFO:teuthology.orchestra.run.smithi038.stdout:Total 17 MB/s | 37 MB 00:02 2024-02-15T18:23:29.274 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-02-15T18:23:29.293 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-02-15T18:23:29.293 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-02-15T18:23:29.640 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-02-15T18:23:29.644 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-02-15T18:23:30.045 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-15T18:23:30.288 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-02-15T18:23:30.454 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-15T18:23:30.600 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-15T18:23:30.783 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-15T18:23:31.396 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-15T18:23:31.553 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-02-15T18:23:31.579 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-02-15T18:23:31.743 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 8/8 2024-02-15T18:23:32.087 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 8/8 2024-02-15T18:23:32.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 1/8 2024-02-15T18:23:32.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-02-15T18:23:32.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-02-15T18:23:32.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-02-15T18:23:32.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-02-15T18:23:32.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-02-15T18:23:32.087 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-02-15T18:23:32.232 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-02-15T18:23:32.429 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-15T18:23:32.573 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-02-15T18:23:32.573 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:32.573 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-02-15T18:23:32.573 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 2024-02-15T18:23:32.573 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-02-15T18:23:32.573 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-02-15T18:23:32.573 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-0.3.15-4.el8.x86_64 2024-02-15T18:23:32.573 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-02-15T18:23:32.573 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-02-15T18:23:32.573 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-02-15T18:23:32.573 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-02-15T18:23:32.573 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:32.574 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:23:32.696 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr-rook 2024-02-15T18:23:33.177 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-15T18:23:33.453 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:44 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:23:33.878 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-02-15T18:23:34.121 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-15T18:23:34.999 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:23:35.000 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:35.000 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-02-15T18:23:35.000 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:35.001 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-02-15T18:23:35.001 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook noarch 2:16.2.4-0.el8 ceph-noarch 27 k 2024-02-15T18:23:35.001 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-02-15T18:23:35.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-02-15T18:23:35.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-02-15T18:23:35.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-02-15T18:23:35.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-02-15T18:23:35.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-02-15T18:23:35.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-02-15T18:23:35.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-02-15T18:23:35.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-02-15T18:23:35.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-02-15T18:23:35.002 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-02-15T18:23:35.002 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-02-15T18:23:35.002 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-02-15T18:23:35.002 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:35.002 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-02-15T18:23:35.002 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:35.002 INFO:teuthology.orchestra.run.smithi121.stdout:Install 13 Packages 2024-02-15T18:23:35.002 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:35.003 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 1.9 M 2024-02-15T18:23:35.003 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 24 M 2024-02-15T18:23:35.003 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-02-15T18:23:35.846 INFO:teuthology.orchestra.run.smithi121.stdout:(1/13): ceph-mgr-rook-16.2.4-0.el8.noarch.rpm 549 kB/s | 27 kB 00:00 2024-02-15T18:23:36.001 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-15T18:23:36.012 INFO:teuthology.orchestra.run.smithi121.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 92 kB/s | 20 kB 00:00 2024-02-15T18:23:36.029 INFO:teuthology.orchestra.run.smithi121.stdout:(3/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 114 kB/s | 27 kB 00:00 2024-02-15T18:23:36.213 INFO:teuthology.orchestra.run.smithi121.stdout:(4/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 344 kB/s | 126 kB 00:00 2024-02-15T18:23:36.238 INFO:teuthology.orchestra.run.smithi121.stdout:(5/13): python3-pyasn1-modules-0.3.7-6.el8.noar 488 kB/s | 110 kB 00:00 2024-02-15T18:23:36.330 INFO:teuthology.orchestra.run.smithi121.stdout:(6/13): python3-cachetools-3.1.1-4.el8.noarch.r 359 kB/s | 33 kB 00:00 2024-02-15T18:23:36.380 INFO:teuthology.orchestra.run.smithi121.stdout:(7/13): python3-certifi-2018.10.15-7.el8.noarch 326 kB/s | 16 kB 00:00 2024-02-15T18:23:36.487 INFO:teuthology.orchestra.run.smithi121.stdout:(8/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 339 kB/s | 155 kB 00:00 2024-02-15T18:23:36.523 INFO:teuthology.orchestra.run.smithi121.stdout:(9/13): python3-google-auth-1.1.1-10.el8.noarch 572 kB/s | 81 kB 00:00 2024-02-15T18:23:36.565 INFO:teuthology.orchestra.run.smithi121.stdout:(10/13): python3-requests-oauthlib-1.0.0-1.el8. 121 kB/s | 43 kB 00:00 2024-02-15T18:23:36.649 INFO:teuthology.orchestra.run.smithi121.stdout:(11/13): python3-kubernetes-11.0.0-6.el8.noarch 7.2 MB/s | 1.2 MB 00:00 2024-02-15T18:23:36.673 INFO:teuthology.orchestra.run.smithi121.stdout:(12/13): python3-rsa-4.9-2.el8.noarch.rpm 427 kB/s | 64 kB 00:00 2024-02-15T18:23:36.699 INFO:teuthology.orchestra.run.smithi121.stdout:(13/13): python3-websocket-client-0.56.0-5.el8. 457 kB/s | 61 kB 00:00 2024-02-15T18:23:36.699 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:23:36.699 INFO:teuthology.orchestra.run.smithi121.stdout:Total 1.1 MB/s | 1.9 MB 00:01 2024-02-15T18:23:36.757 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-02-15T18:23:36.773 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-02-15T18:23:36.773 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-02-15T18:23:36.860 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-02-15T18:23:36.863 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-02-15T18:23:37.374 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-02-15T18:23:37.414 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-02-15T18:23:37.620 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-02-15T18:23:37.647 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 8/8 2024-02-15T18:23:37.787 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-02-15T18:23:37.977 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 8/8 2024-02-15T18:23:37.977 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 1/8 2024-02-15T18:23:37.977 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-02-15T18:23:37.977 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-02-15T18:23:37.977 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-02-15T18:23:37.978 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-02-15T18:23:37.978 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-02-15T18:23:37.978 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-02-15T18:23:38.020 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-02-15T18:23:38.266 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-02-15T18:23:38.459 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-02-15T18:23:38.459 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:38.459 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-02-15T18:23:38.459 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 2024-02-15T18:23:38.459 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-02-15T18:23:38.459 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-02-15T18:23:38.459 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-0.3.15-4.el8.x86_64 2024-02-15T18:23:38.459 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-02-15T18:23:38.459 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-02-15T18:23:38.460 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-02-15T18:23:38.460 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-02-15T18:23:38.460 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:38.460 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:23:38.473 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-02-15T18:23:38.586 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-mgr-rook 2024-02-15T18:23:38.712 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-02-15T18:23:39.021 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-02-15T18:23:39.280 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-02-15T18:23:39.381 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:01:50 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:23:40.276 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-02-15T18:23:40.574 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-02-15T18:23:40.785 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-02-15T18:23:41.000 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-02-15T18:23:41.034 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:23:41.036 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:23:41.036 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-02-15T18:23:41.036 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:23:41.036 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-02-15T18:23:41.036 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook noarch 2:16.2.4-0.el8 ceph-noarch 27 k 2024-02-15T18:23:41.036 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-02-15T18:23:41.036 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-02-15T18:23:41.036 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-02-15T18:23:41.036 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-02-15T18:23:41.037 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-02-15T18:23:41.037 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-02-15T18:23:41.037 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-02-15T18:23:41.037 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-02-15T18:23:41.037 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-02-15T18:23:41.037 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-02-15T18:23:41.037 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-02-15T18:23:41.037 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-02-15T18:23:41.037 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-02-15T18:23:41.037 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:41.037 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-02-15T18:23:41.037 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:23:41.037 INFO:teuthology.orchestra.run.smithi038.stdout:Install 13 Packages 2024-02-15T18:23:41.037 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:41.038 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 1.9 M 2024-02-15T18:23:41.038 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 24 M 2024-02-15T18:23:41.038 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-02-15T18:23:41.150 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-rook-2:16.2.4-0.el8.noarch 13/13 2024-02-15T18:23:41.486 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.4-0.el8.noarch 13/13 2024-02-15T18:23:41.486 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-rook-2:16.2.4-0.el8.noarch 1/13 2024-02-15T18:23:41.486 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-02-15T18:23:41.486 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-02-15T18:23:41.486 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-02-15T18:23:41.487 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-02-15T18:23:41.487 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-02-15T18:23:41.487 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-02-15T18:23:41.487 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-02-15T18:23:41.487 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-02-15T18:23:41.487 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-02-15T18:23:41.487 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-02-15T18:23:41.487 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-02-15T18:23:41.963 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-02-15T18:23:41.963 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:41.963 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-02-15T18:23:41.963 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook-2:16.2.4-0.el8.noarch 2024-02-15T18:23:41.964 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-02-15T18:23:41.964 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-02-15T18:23:41.964 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-02-15T18:23:41.964 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-02-15T18:23:41.964 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-02-15T18:23:41.964 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-02-15T18:23:41.964 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-02-15T18:23:41.964 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-02-15T18:23:41.964 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-02-15T18:23:41.964 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-02-15T18:23:41.964 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa-4.9-2.el8.noarch 2024-02-15T18:23:41.964 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-02-15T18:23:41.964 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:41.964 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:23:42.091 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr-cephadm 2024-02-15T18:23:42.454 INFO:teuthology.orchestra.run.smithi038.stdout:(1/13): ceph-mgr-rook-16.2.4-0.el8.noarch.rpm 336 kB/s | 27 kB 00:00 2024-02-15T18:23:42.582 INFO:teuthology.orchestra.run.smithi038.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 127 kB/s | 27 kB 00:00 2024-02-15T18:23:42.607 INFO:teuthology.orchestra.run.smithi038.stdout:(3/13): python3-jsonpointer-1.10-11.el8.noarch. 85 kB/s | 20 kB 00:00 2024-02-15T18:23:42.732 INFO:teuthology.orchestra.run.smithi038.stdout:(4/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 455 kB/s | 126 kB 00:00 2024-02-15T18:23:42.783 INFO:teuthology.orchestra.run.smithi038.stdout:(5/13): python3-pyasn1-modules-0.3.7-6.el8.noar 628 kB/s | 110 kB 00:00 2024-02-15T18:23:42.840 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:53 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:23:42.975 INFO:teuthology.orchestra.run.smithi038.stdout:(6/13): python3-cachetools-3.1.1-4.el8.noarch.r 172 kB/s | 33 kB 00:00 2024-02-15T18:23:43.075 INFO:teuthology.orchestra.run.smithi038.stdout:(7/13): python3-certifi-2018.10.15-7.el8.noarch 163 kB/s | 16 kB 00:00 2024-02-15T18:23:43.101 INFO:teuthology.orchestra.run.smithi038.stdout:(8/13): python3-requests-oauthlib-1.0.0-1.el8.n 116 kB/s | 43 kB 00:00 2024-02-15T18:23:43.126 INFO:teuthology.orchestra.run.smithi038.stdout:(9/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 299 kB/s | 155 kB 00:00 2024-02-15T18:23:43.235 INFO:teuthology.orchestra.run.smithi038.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 512 kB/s | 81 kB 00:00 2024-02-15T18:23:43.335 INFO:teuthology.orchestra.run.smithi038.stdout:(11/13): python3-websocket-client-0.56.0-5.el8. 612 kB/s | 61 kB 00:00 2024-02-15T18:23:43.368 INFO:teuthology.orchestra.run.smithi038.stdout:(12/13): python3-rsa-4.9-2.el8.noarch.rpm 265 kB/s | 64 kB 00:00 2024-02-15T18:23:43.594 INFO:teuthology.orchestra.run.smithi038.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 2.4 MB/s | 1.2 MB 00:00 2024-02-15T18:23:43.594 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:23:43.594 INFO:teuthology.orchestra.run.smithi038.stdout:Total 765 kB/s | 1.9 MB 00:02 2024-02-15T18:23:43.655 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-02-15T18:23:43.672 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-02-15T18:23:43.672 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-02-15T18:23:43.768 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-02-15T18:23:43.772 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-02-15T18:23:44.383 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-02-15T18:23:44.392 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:23:44.393 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:44.393 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-02-15T18:23:44.393 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:44.393 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-02-15T18:23:44.393 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm noarch 2:16.2.4-0.el8 ceph-noarch 89 k 2024-02-15T18:23:44.393 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-02-15T18:23:44.393 INFO:teuthology.orchestra.run.smithi121.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2024-02-15T18:23:44.393 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-02-15T18:23:44.393 INFO:teuthology.orchestra.run.smithi121.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2024-02-15T18:23:44.394 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-02-15T18:23:44.394 INFO:teuthology.orchestra.run.smithi121.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2024-02-15T18:23:44.394 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:44.394 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-02-15T18:23:44.394 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:44.394 INFO:teuthology.orchestra.run.smithi121.stdout:Install 6 Packages 2024-02-15T18:23:44.394 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:44.394 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 5.9 M 2024-02-15T18:23:44.395 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 24 M 2024-02-15T18:23:44.395 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-02-15T18:23:44.641 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-02-15T18:23:44.864 INFO:teuthology.orchestra.run.smithi121.stdout:(1/6): ceph-mgr-cephadm-16.2.4-0.el8.noarch.rpm 1.8 MB/s | 89 kB 00:00 2024-02-15T18:23:44.871 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-02-15T18:23:45.123 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-02-15T18:23:45.164 INFO:teuthology.orchestra.run.smithi121.stdout:(2/6): python3-apipkg-1.5-6.el8.noarch.rpm 71 kB/s | 21 kB 00:00 2024-02-15T18:23:45.197 INFO:teuthology.orchestra.run.smithi121.stdout:(3/6): python3-jinja2-2.10.1-4.el8.noarch.rpm 1.5 MB/s | 578 kB 00:00 2024-02-15T18:23:45.410 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-02-15T18:23:45.564 INFO:teuthology.orchestra.run.smithi121.stdout:(4/6): python3-execnet-1.7.1-1.el8.noarch.rpm 888 kB/s | 355 kB 00:00 2024-02-15T18:23:45.623 INFO:teuthology.orchestra.run.smithi121.stdout:(5/6): python3-remoto-1.2.1-1.el8.noarch.rpm 115 kB/s | 49 kB 00:00 2024-02-15T18:23:45.635 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-02-15T18:23:45.887 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-02-15T18:23:45.974 INFO:teuthology.orchestra.run.smithi121.stdout:(6/6): python3-babel-2.5.1-7.el8.noarch.rpm 4.1 MB/s | 4.8 MB 00:01 2024-02-15T18:23:45.974 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:23:45.974 INFO:teuthology.orchestra.run.smithi121.stdout:Total 3.7 MB/s | 5.9 MB 00:01 2024-02-15T18:23:46.027 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-02-15T18:23:46.037 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-02-15T18:23:46.037 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-02-15T18:23:46.143 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-02-15T18:23:46.147 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-02-15T18:23:46.208 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-02-15T18:23:46.447 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-02-15T18:23:46.494 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-02-15T18:23:46.731 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/6 2024-02-15T18:23:47.062 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 2/6 2024-02-15T18:23:47.636 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-02-15T18:23:47.917 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-02-15T18:23:47.961 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 3/6 2024-02-15T18:23:48.151 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-02-15T18:23:48.322 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 4/6 2024-02-15T18:23:48.360 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-02-15T18:23:48.510 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-rook-2:16.2.4-0.el8.noarch 13/13 2024-02-15T18:23:48.553 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 5/6 2024-02-15T18:23:48.765 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 6/6 2024-02-15T18:23:48.855 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.4-0.el8.noarch 13/13 2024-02-15T18:23:48.856 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-rook-2:16.2.4-0.el8.noarch 1/13 2024-02-15T18:23:48.858 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-02-15T18:23:48.858 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-02-15T18:23:48.858 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-02-15T18:23:48.858 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-02-15T18:23:48.859 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-02-15T18:23:48.859 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-02-15T18:23:48.859 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-02-15T18:23:48.859 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-02-15T18:23:48.859 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-02-15T18:23:48.859 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-02-15T18:23:48.860 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-02-15T18:23:49.047 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 6/6 2024-02-15T18:23:49.047 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 1/6 2024-02-15T18:23:49.047 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/6 2024-02-15T18:23:49.047 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/6 2024-02-15T18:23:49.047 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/6 2024-02-15T18:23:49.047 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 5/6 2024-02-15T18:23:49.381 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-02-15T18:23:49.381 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:49.381 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-02-15T18:23:49.381 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook-2:16.2.4-0.el8.noarch 2024-02-15T18:23:49.382 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-02-15T18:23:49.382 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-02-15T18:23:49.382 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-02-15T18:23:49.382 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-02-15T18:23:49.382 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-02-15T18:23:49.382 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-02-15T18:23:49.382 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-02-15T18:23:49.382 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-02-15T18:23:49.382 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-02-15T18:23:49.382 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-02-15T18:23:49.382 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa-4.9-2.el8.noarch 2024-02-15T18:23:49.382 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-02-15T18:23:49.383 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:49.383 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:23:49.450 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 6/6 2024-02-15T18:23:49.450 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:49.450 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-02-15T18:23:49.450 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm-2:16.2.4-0.el8.noarch python3-apipkg-1.5-6.el8.noarch 2024-02-15T18:23:49.450 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel-2.5.1-7.el8.noarch python3-execnet-1.7.1-1.el8.noarch 2024-02-15T18:23:49.450 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2-2.10.1-4.el8.noarch python3-remoto-1.2.1-1.el8.noarch 2024-02-15T18:23:49.450 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:49.451 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:23:49.530 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-mgr-cephadm 2024-02-15T18:23:49.600 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-fuse 2024-02-15T18:23:50.316 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:02:01 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:23:50.354 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:01 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:23:51.915 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:23:51.916 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:51.916 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2024-02-15T18:23:51.916 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:51.916 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-02-15T18:23:51.916 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse x86_64 2:16.2.4-0.el8 ceph 885 k 2024-02-15T18:23:51.916 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-02-15T18:23:51.916 INFO:teuthology.orchestra.run.smithi121.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2024-02-15T18:23:51.916 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:51.917 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-02-15T18:23:51.917 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:51.917 INFO:teuthology.orchestra.run.smithi121.stdout:Install 2 Packages 2024-02-15T18:23:51.917 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:51.917 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 968 k 2024-02-15T18:23:51.917 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 3.2 M 2024-02-15T18:23:51.918 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-02-15T18:23:51.979 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:23:51.980 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:23:51.980 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-02-15T18:23:51.980 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:23:51.980 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-02-15T18:23:51.980 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm noarch 2:16.2.4-0.el8 ceph-noarch 89 k 2024-02-15T18:23:51.980 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-02-15T18:23:51.981 INFO:teuthology.orchestra.run.smithi038.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2024-02-15T18:23:51.981 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-02-15T18:23:51.981 INFO:teuthology.orchestra.run.smithi038.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2024-02-15T18:23:51.981 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-02-15T18:23:51.981 INFO:teuthology.orchestra.run.smithi038.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2024-02-15T18:23:51.981 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:51.981 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-02-15T18:23:51.981 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:23:51.981 INFO:teuthology.orchestra.run.smithi038.stdout:Install 6 Packages 2024-02-15T18:23:51.981 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:51.982 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 5.9 M 2024-02-15T18:23:51.982 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 24 M 2024-02-15T18:23:51.982 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-02-15T18:23:52.068 INFO:teuthology.orchestra.run.smithi121.stdout:(1/2): ceph-fuse-16.2.4-0.el8.x86_64.rpm 10 MB/s | 885 kB 00:00 2024-02-15T18:23:52.093 INFO:teuthology.orchestra.run.smithi121.stdout:(2/2): fuse-2.9.7-17.el8.x86_64.rpm 753 kB/s | 83 kB 00:00 2024-02-15T18:23:52.094 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:23:52.094 INFO:teuthology.orchestra.run.smithi121.stdout:Total 5.3 MB/s | 968 kB 00:00 2024-02-15T18:23:52.104 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-02-15T18:23:52.124 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-02-15T18:23:52.124 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-02-15T18:23:52.209 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-02-15T18:23:52.212 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-02-15T18:23:52.399 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-02-15T18:23:52.618 INFO:teuthology.orchestra.run.smithi038.stdout:(1/6): ceph-mgr-cephadm-16.2.4-0.el8.noarch.rpm 1.6 MB/s | 89 kB 00:00 2024-02-15T18:23:52.647 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-02-15T18:23:52.777 INFO:teuthology.orchestra.run.smithi038.stdout:(2/6): python3-apipkg-1.5-6.el8.noarch.rpm 134 kB/s | 21 kB 00:00 2024-02-15T18:23:52.848 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-fuse-2:16.2.4-0.el8.x86_64 2/2 2024-02-15T18:23:53.035 INFO:teuthology.orchestra.run.smithi038.stdout:(3/6): python3-execnet-1.7.1-1.el8.noarch.rpm 1.3 MB/s | 355 kB 00:00 2024-02-15T18:23:53.102 INFO:teuthology.orchestra.run.smithi038.stdout:(4/6): python3-remoto-1.2.1-1.el8.noarch.rpm 735 kB/s | 49 kB 00:00 2024-02-15T18:23:53.136 INFO:teuthology.orchestra.run.smithi038.stdout:(5/6): python3-jinja2-2.10.1-4.el8.noarch.rpm 1.0 MB/s | 578 kB 00:00 2024-02-15T18:23:53.242 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-fuse-2:16.2.4-0.el8.x86_64 2/2 2024-02-15T18:23:53.242 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-fuse-2:16.2.4-0.el8.x86_64 1/2 2024-02-15T18:23:53.621 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-02-15T18:23:53.621 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:53.621 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-02-15T18:23:53.621 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse-2:16.2.4-0.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-02-15T18:23:53.621 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:53.621 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:23:53.862 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install librados-devel 2024-02-15T18:23:54.402 INFO:teuthology.orchestra.run.smithi038.stdout:(6/6): python3-babel-2.5.1-7.el8.noarch.rpm 2.6 MB/s | 4.8 MB 00:01 2024-02-15T18:23:54.403 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:23:54.403 INFO:teuthology.orchestra.run.smithi038.stdout:Total 2.4 MB/s | 5.9 MB 00:02 2024-02-15T18:23:54.459 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-02-15T18:23:54.470 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-02-15T18:23:54.470 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-02-15T18:23:54.584 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-02-15T18:23:54.588 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-02-15T18:23:54.632 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:05 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:23:55.161 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-02-15T18:23:55.466 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/6 2024-02-15T18:23:55.789 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 2/6 2024-02-15T18:23:56.184 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:23:56.185 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:56.185 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2024-02-15T18:23:56.185 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:56.185 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-02-15T18:23:56.185 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel x86_64 2:16.2.4-0.el8 ceph 131 k 2024-02-15T18:23:56.186 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:56.186 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-02-15T18:23:56.186 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:23:56.186 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-02-15T18:23:56.186 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:56.186 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 131 k 2024-02-15T18:23:56.186 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 439 k 2024-02-15T18:23:56.186 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-02-15T18:23:56.655 INFO:teuthology.orchestra.run.smithi121.stdout:librados-devel-16.2.4-0.el8.x86_64.rpm 280 kB/s | 131 kB 00:00 2024-02-15T18:23:56.656 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:23:56.656 INFO:teuthology.orchestra.run.smithi121.stdout:Total 279 kB/s | 131 kB 00:00 2024-02-15T18:23:56.657 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-02-15T18:23:56.663 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-02-15T18:23:56.663 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-02-15T18:23:56.718 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-02-15T18:23:56.721 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-02-15T18:23:56.801 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 3/6 2024-02-15T18:23:56.952 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-02-15T18:23:57.198 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 4/6 2024-02-15T18:23:57.221 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librados-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:23:57.456 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 5/6 2024-02-15T18:23:57.529 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:23:57.683 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 6/6 2024-02-15T18:23:57.909 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:23:57.909 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:57.909 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-02-15T18:23:57.909 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel-2:16.2.4-0.el8.x86_64 2024-02-15T18:23:57.909 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:23:57.909 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:23:57.977 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 6/6 2024-02-15T18:23:57.977 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 1/6 2024-02-15T18:23:57.977 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/6 2024-02-15T18:23:57.977 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/6 2024-02-15T18:23:57.977 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/6 2024-02-15T18:23:57.977 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 5/6 2024-02-15T18:23:58.170 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install libcephfs2 2024-02-15T18:23:58.388 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 6/6 2024-02-15T18:23:58.388 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:58.388 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-02-15T18:23:58.388 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm-2:16.2.4-0.el8.noarch python3-apipkg-1.5-6.el8.noarch 2024-02-15T18:23:58.388 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel-2.5.1-7.el8.noarch python3-execnet-1.7.1-1.el8.noarch 2024-02-15T18:23:58.388 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2-2.10.1-4.el8.noarch python3-remoto-1.2.1-1.el8.noarch 2024-02-15T18:23:58.388 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:23:58.388 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:23:58.534 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-fuse 2024-02-15T18:23:58.940 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:09 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:23:59.335 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:02:10 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:00.447 INFO:teuthology.orchestra.run.smithi121.stdout:Package libcephfs2-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:24:00.494 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:24:00.495 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-02-15T18:24:00.495 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:24:00.604 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install libcephfs-devel 2024-02-15T18:24:00.988 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:24:00.988 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:24:00.989 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-02-15T18:24:00.989 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:24:00.989 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-02-15T18:24:00.989 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse x86_64 2:16.2.4-0.el8 ceph 885 k 2024-02-15T18:24:00.989 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-02-15T18:24:00.989 INFO:teuthology.orchestra.run.smithi038.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2024-02-15T18:24:00.989 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:00.989 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-02-15T18:24:00.989 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:24:00.989 INFO:teuthology.orchestra.run.smithi038.stdout:Install 2 Packages 2024-02-15T18:24:00.989 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:00.990 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 968 k 2024-02-15T18:24:00.990 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 3.2 M 2024-02-15T18:24:00.990 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-02-15T18:24:01.121 INFO:teuthology.orchestra.run.smithi038.stdout:(1/2): ceph-fuse-16.2.4-0.el8.x86_64.rpm 9.5 MB/s | 885 kB 00:00 2024-02-15T18:24:01.156 INFO:teuthology.orchestra.run.smithi038.stdout:(2/2): fuse-2.9.7-17.el8.x86_64.rpm 663 kB/s | 83 kB 00:00 2024-02-15T18:24:01.157 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:24:01.157 INFO:teuthology.orchestra.run.smithi038.stdout:Total 5.7 MB/s | 968 kB 00:00 2024-02-15T18:24:01.167 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-02-15T18:24:01.189 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-02-15T18:24:01.189 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-02-15T18:24:01.282 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-02-15T18:24:01.285 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-02-15T18:24:01.373 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:12 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:01.468 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-02-15T18:24:01.780 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-02-15T18:24:02.000 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-fuse-2:16.2.4-0.el8.x86_64 2/2 2024-02-15T18:24:02.401 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-fuse-2:16.2.4-0.el8.x86_64 2/2 2024-02-15T18:24:02.401 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-fuse-2:16.2.4-0.el8.x86_64 1/2 2024-02-15T18:24:02.811 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-02-15T18:24:02.812 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:02.812 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-02-15T18:24:02.812 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse-2:16.2.4-0.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-02-15T18:24:02.812 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:02.812 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:24:02.935 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:24:02.936 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:24:02.936 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2024-02-15T18:24:02.936 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:24:02.937 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-02-15T18:24:02.937 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel x86_64 2:16.2.4-0.el8 ceph 24 k 2024-02-15T18:24:02.937 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:24:02.937 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-02-15T18:24:02.937 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:24:02.937 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-02-15T18:24:02.937 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:24:02.937 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 24 k 2024-02-15T18:24:02.937 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 94 k 2024-02-15T18:24:02.937 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-02-15T18:24:03.074 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install librados-devel 2024-02-15T18:24:03.478 INFO:teuthology.orchestra.run.smithi121.stdout:libcephfs-devel-16.2.4-0.el8.x86_64.rpm 45 kB/s | 24 kB 00:00 2024-02-15T18:24:03.479 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:24:03.479 INFO:teuthology.orchestra.run.smithi121.stdout:Total 45 kB/s | 24 kB 00:00 2024-02-15T18:24:03.480 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-02-15T18:24:03.483 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-02-15T18:24:03.484 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-02-15T18:24:03.486 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-02-15T18:24:03.489 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-02-15T18:24:03.537 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-02-15T18:24:03.745 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:03.874 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:02:14 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:04.035 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:04.387 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:04.387 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:24:04.387 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-02-15T18:24:04.387 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel-2:16.2.4-0.el8.x86_64 2024-02-15T18:24:04.388 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:24:04.388 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:24:04.507 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install librados2 2024-02-15T18:24:05.260 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:16 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:05.539 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:24:05.539 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:24:05.539 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-02-15T18:24:05.539 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:24:05.539 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-02-15T18:24:05.540 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel x86_64 2:16.2.4-0.el8 ceph 131 k 2024-02-15T18:24:05.540 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:05.540 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-02-15T18:24:05.540 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:24:05.540 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2024-02-15T18:24:05.540 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:05.541 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 131 k 2024-02-15T18:24:05.541 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 439 k 2024-02-15T18:24:05.541 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-02-15T18:24:05.777 INFO:teuthology.orchestra.run.smithi038.stdout:librados-devel-16.2.4-0.el8.x86_64.rpm 555 kB/s | 131 kB 00:00 2024-02-15T18:24:05.778 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:24:05.778 INFO:teuthology.orchestra.run.smithi038.stdout:Total 552 kB/s | 131 kB 00:00 2024-02-15T18:24:05.778 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-02-15T18:24:05.784 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-02-15T18:24:05.784 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-02-15T18:24:05.838 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-02-15T18:24:05.841 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-02-15T18:24:06.362 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-02-15T18:24:06.742 INFO:teuthology.orchestra.run.smithi121.stdout:Package librados2-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:24:06.788 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:24:06.789 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-02-15T18:24:06.789 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:24:06.900 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install librbd1 2024-02-15T18:24:07.668 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:18 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:07.991 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librados-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:08.316 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:08.727 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:08.727 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:08.727 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-02-15T18:24:08.727 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel-2:16.2.4-0.el8.x86_64 2024-02-15T18:24:08.727 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:08.727 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:24:08.952 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install libcephfs2 2024-02-15T18:24:09.158 INFO:teuthology.orchestra.run.smithi121.stdout:Package librbd1-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:24:09.204 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:24:09.205 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-02-15T18:24:09.205 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:24:09.315 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install python3-rados 2024-02-15T18:24:09.751 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:02:20 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:10.072 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:21 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:11.361 INFO:teuthology.orchestra.run.smithi038.stdout:Package libcephfs2-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:24:11.411 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:24:11.413 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2024-02-15T18:24:11.413 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:24:11.533 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install libcephfs-devel 2024-02-15T18:24:11.570 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-rados-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:24:11.617 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:24:11.618 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-02-15T18:24:11.618 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:24:11.735 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install python3-rgw 2024-02-15T18:24:12.328 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:02:23 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:12.486 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:23 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:13.983 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-rgw-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:24:14.015 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:24:14.016 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:24:14.016 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-02-15T18:24:14.016 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:24:14.016 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-02-15T18:24:14.017 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel x86_64 2:16.2.4-0.el8 ceph 24 k 2024-02-15T18:24:14.017 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:14.017 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-02-15T18:24:14.017 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:24:14.017 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2024-02-15T18:24:14.017 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:14.017 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 24 k 2024-02-15T18:24:14.017 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 94 k 2024-02-15T18:24:14.017 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-02-15T18:24:14.030 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:24:14.031 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-02-15T18:24:14.031 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:24:14.075 INFO:teuthology.orchestra.run.smithi038.stdout:libcephfs-devel-16.2.4-0.el8.x86_64.rpm 418 kB/s | 24 kB 00:00 2024-02-15T18:24:14.076 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:24:14.076 INFO:teuthology.orchestra.run.smithi038.stdout:Total 410 kB/s | 24 kB 00:00 2024-02-15T18:24:14.076 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-02-15T18:24:14.080 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-02-15T18:24:14.080 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-02-15T18:24:14.082 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-02-15T18:24:14.085 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-02-15T18:24:14.131 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install python3-cephfs 2024-02-15T18:24:14.135 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-02-15T18:24:14.368 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:14.680 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:14.882 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:25 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:15.110 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:15.111 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:15.111 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-02-15T18:24:15.111 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel-2:16.2.4-0.el8.x86_64 2024-02-15T18:24:15.111 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:15.111 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:24:15.237 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install librados2 2024-02-15T18:24:16.031 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:02:27 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:16.380 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-cephfs-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:24:16.426 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:24:16.427 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-02-15T18:24:16.428 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:24:16.529 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install python3-rbd 2024-02-15T18:24:17.281 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:28 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:17.645 INFO:teuthology.orchestra.run.smithi038.stdout:Package librados2-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:24:17.696 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:24:17.697 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2024-02-15T18:24:17.697 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:24:17.802 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install librbd1 2024-02-15T18:24:18.594 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:02:29 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:18.770 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-rbd-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:24:18.817 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:24:18.818 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-02-15T18:24:18.818 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:24:18.929 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install rbd-fuse 2024-02-15T18:24:19.682 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:30 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:20.206 INFO:teuthology.orchestra.run.smithi038.stdout:Package librbd1-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:24:20.256 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:24:20.258 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2024-02-15T18:24:20.258 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:24:20.379 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install python3-rados 2024-02-15T18:24:21.171 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:02:32 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:21.241 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:24:21.242 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:24:21.242 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2024-02-15T18:24:21.242 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:24:21.242 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-02-15T18:24:21.242 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse x86_64 2:16.2.4-0.el8 ceph 89 k 2024-02-15T18:24:21.242 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:24:21.242 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-02-15T18:24:21.242 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:24:21.242 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-02-15T18:24:21.242 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:24:21.243 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 89 k 2024-02-15T18:24:21.243 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 298 k 2024-02-15T18:24:21.243 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-02-15T18:24:21.308 INFO:teuthology.orchestra.run.smithi121.stdout:rbd-fuse-16.2.4-0.el8.x86_64.rpm 1.3 MB/s | 89 kB 00:00 2024-02-15T18:24:21.309 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:24:21.309 INFO:teuthology.orchestra.run.smithi121.stdout:Total 1.3 MB/s | 89 kB 00:00 2024-02-15T18:24:21.310 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-02-15T18:24:21.319 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-02-15T18:24:21.319 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-02-15T18:24:21.369 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-02-15T18:24:21.372 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-02-15T18:24:21.477 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-02-15T18:24:21.673 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:21.950 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:22.349 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:22.349 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:24:22.349 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-02-15T18:24:22.349 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse-2:16.2.4-0.el8.x86_64 2024-02-15T18:24:22.350 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:24:22.350 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:24:22.552 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install rbd-mirror 2024-02-15T18:24:22.791 INFO:teuthology.orchestra.run.smithi038.stdout:Package python3-rados-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:24:22.842 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:24:22.843 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2024-02-15T18:24:22.843 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:24:22.937 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install python3-rgw 2024-02-15T18:24:23.315 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:34 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:23.724 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:02:34 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:24.883 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:24:24.883 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:24:24.883 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2024-02-15T18:24:24.884 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:24:24.884 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-02-15T18:24:24.884 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror x86_64 2:16.2.4-0.el8 ceph 3.7 M 2024-02-15T18:24:24.884 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:24:24.884 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-02-15T18:24:24.884 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:24:24.884 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-02-15T18:24:24.884 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:24:24.884 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 3.7 M 2024-02-15T18:24:24.884 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 18 M 2024-02-15T18:24:24.884 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-02-15T18:24:25.001 INFO:teuthology.orchestra.run.smithi121.stdout:rbd-mirror-16.2.4-0.el8.x86_64.rpm 32 MB/s | 3.7 MB 00:00 2024-02-15T18:24:25.002 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:24:25.002 INFO:teuthology.orchestra.run.smithi121.stdout:Total 31 MB/s | 3.7 MB 00:00 2024-02-15T18:24:25.002 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-02-15T18:24:25.014 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-02-15T18:24:25.014 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-02-15T18:24:25.076 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-02-15T18:24:25.079 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-02-15T18:24:25.363 INFO:teuthology.orchestra.run.smithi038.stdout:Package python3-rgw-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:24:25.414 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:24:25.415 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2024-02-15T18:24:25.415 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:24:25.515 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install python3-cephfs 2024-02-15T18:24:25.551 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-02-15T18:24:25.757 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:26.159 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:26.306 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:02:37 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:26.525 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:26.525 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:24:26.526 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-02-15T18:24:26.526 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror-2:16.2.4-0.el8.x86_64 2024-02-15T18:24:26.526 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:24:26.526 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:24:26.747 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install rbd-nbd 2024-02-15T18:24:27.521 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:38 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:27.932 INFO:teuthology.orchestra.run.smithi038.stdout:Package python3-cephfs-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:24:27.983 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:24:27.984 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2024-02-15T18:24:27.984 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:24:28.108 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install python3-rbd 2024-02-15T18:24:28.902 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:02:39 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:29.068 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-02-15T18:24:29.069 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:24:29.069 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2024-02-15T18:24:29.069 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:24:29.069 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-02-15T18:24:29.069 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd x86_64 2:16.2.4-0.el8 ceph 173 k 2024-02-15T18:24:29.069 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:24:29.069 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-02-15T18:24:29.070 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-02-15T18:24:29.070 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-02-15T18:24:29.070 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:24:29.070 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 173 k 2024-02-15T18:24:29.070 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 649 k 2024-02-15T18:24:29.070 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-02-15T18:24:29.116 INFO:teuthology.orchestra.run.smithi121.stdout:rbd-nbd-16.2.4-0.el8.x86_64.rpm 3.6 MB/s | 173 kB 00:00 2024-02-15T18:24:29.117 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:24:29.117 INFO:teuthology.orchestra.run.smithi121.stdout:Total 3.5 MB/s | 173 kB 00:00 2024-02-15T18:24:29.117 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-02-15T18:24:29.128 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-02-15T18:24:29.128 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-02-15T18:24:29.179 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-02-15T18:24:29.182 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-02-15T18:24:29.301 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-02-15T18:24:29.542 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:29.829 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:30.326 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:30.326 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:24:30.326 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-02-15T18:24:30.326 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd-2:16.2.4-0.el8.x86_64 2024-02-15T18:24:30.326 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:24:30.326 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-02-15T18:24:30.511 INFO:teuthology.orchestra.run.smithi038.stdout:Package python3-rbd-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:24:30.560 DEBUG:teuthology.parallel:result is None 2024-02-15T18:24:30.562 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:24:30.563 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2024-02-15T18:24:30.563 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:24:30.667 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install rbd-fuse 2024-02-15T18:24:31.469 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:02:42 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:33.141 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:24:33.141 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:24:33.141 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-02-15T18:24:33.142 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:24:33.142 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-02-15T18:24:33.142 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse x86_64 2:16.2.4-0.el8 ceph 89 k 2024-02-15T18:24:33.142 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:33.142 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-02-15T18:24:33.142 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:24:33.142 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2024-02-15T18:24:33.142 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:33.143 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 89 k 2024-02-15T18:24:33.143 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 298 k 2024-02-15T18:24:33.143 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-02-15T18:24:33.198 INFO:teuthology.orchestra.run.smithi038.stdout:rbd-fuse-16.2.4-0.el8.x86_64.rpm 1.6 MB/s | 89 kB 00:00 2024-02-15T18:24:33.199 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:24:33.199 INFO:teuthology.orchestra.run.smithi038.stdout:Total 1.5 MB/s | 89 kB 00:00 2024-02-15T18:24:33.199 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-02-15T18:24:33.210 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-02-15T18:24:33.210 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-02-15T18:24:33.262 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-02-15T18:24:33.265 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-02-15T18:24:33.371 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-02-15T18:24:33.736 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:34.043 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:34.447 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:34.448 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:34.448 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-02-15T18:24:34.448 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse-2:16.2.4-0.el8.x86_64 2024-02-15T18:24:34.448 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:34.448 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:24:34.710 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install rbd-mirror 2024-02-15T18:24:35.510 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:02:46 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:37.207 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:24:37.207 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:24:37.208 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-02-15T18:24:37.208 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:24:37.208 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-02-15T18:24:37.208 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror x86_64 2:16.2.4-0.el8 ceph 3.7 M 2024-02-15T18:24:37.208 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:37.208 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-02-15T18:24:37.208 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:24:37.208 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2024-02-15T18:24:37.208 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:37.209 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 3.7 M 2024-02-15T18:24:37.209 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 18 M 2024-02-15T18:24:37.209 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-02-15T18:24:37.318 INFO:teuthology.orchestra.run.smithi038.stdout:rbd-mirror-16.2.4-0.el8.x86_64.rpm 34 MB/s | 3.7 MB 00:00 2024-02-15T18:24:37.319 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:24:37.319 INFO:teuthology.orchestra.run.smithi038.stdout:Total 34 MB/s | 3.7 MB 00:00 2024-02-15T18:24:37.320 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-02-15T18:24:37.332 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-02-15T18:24:37.333 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-02-15T18:24:37.400 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-02-15T18:24:37.403 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-02-15T18:24:37.895 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-02-15T18:24:38.086 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:38.519 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:38.940 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:38.940 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:38.940 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-02-15T18:24:38.940 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror-2:16.2.4-0.el8.x86_64 2024-02-15T18:24:38.941 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:38.941 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:24:39.152 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install rbd-nbd 2024-02-15T18:24:39.948 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:02:50 ago on Thu 15 Feb 2024 06:21:49 PM UTC. 2024-02-15T18:24:41.601 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-02-15T18:24:41.602 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:24:41.602 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-02-15T18:24:41.602 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:24:41.602 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-02-15T18:24:41.602 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd x86_64 2:16.2.4-0.el8 ceph 173 k 2024-02-15T18:24:41.602 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:41.602 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-02-15T18:24:41.602 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-02-15T18:24:41.602 INFO:teuthology.orchestra.run.smithi038.stdout:Install 1 Package 2024-02-15T18:24:41.603 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:41.603 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 173 k 2024-02-15T18:24:41.603 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 649 k 2024-02-15T18:24:41.603 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-02-15T18:24:41.655 INFO:teuthology.orchestra.run.smithi038.stdout:rbd-nbd-16.2.4-0.el8.x86_64.rpm 3.3 MB/s | 173 kB 00:00 2024-02-15T18:24:41.656 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:24:41.656 INFO:teuthology.orchestra.run.smithi038.stdout:Total 3.2 MB/s | 173 kB 00:00 2024-02-15T18:24:41.656 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-02-15T18:24:41.668 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-02-15T18:24:41.668 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-02-15T18:24:41.723 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-02-15T18:24:41.726 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-02-15T18:24:41.848 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-02-15T18:24:42.064 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:42.364 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:42.773 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:24:42.773 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:42.773 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-02-15T18:24:42.773 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd-2:16.2.4-0.el8.x86_64 2024-02-15T18:24:42.773 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:24:42.773 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-02-15T18:24:43.046 DEBUG:teuthology.parallel:result is None 2024-02-15T18:24:43.046 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-15T18:24:43.046 INFO:teuthology.packaging:ref: None 2024-02-15T18:24:43.046 INFO:teuthology.packaging:tag: v16.2.4 2024-02-15T18:24:43.046 INFO:teuthology.packaging:branch: None 2024-02-15T18:24:43.046 INFO:teuthology.packaging:sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:24:43.047 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2024-02-15T18:24:43.353 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-15T18:24:43.377 INFO:teuthology.orchestra.run.smithi038.stdout:16.2.4-0.el8 2024-02-15T18:24:43.378 INFO:teuthology.packaging:The installed version of ceph is 16.2.4-0.el8 2024-02-15T18:24:43.378 INFO:teuthology.task.install:The correct ceph version 16.2.4-0 is installed. 2024-02-15T18:24:43.380 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-15T18:24:43.380 INFO:teuthology.packaging:ref: None 2024-02-15T18:24:43.380 INFO:teuthology.packaging:tag: v16.2.4 2024-02-15T18:24:43.380 INFO:teuthology.packaging:branch: None 2024-02-15T18:24:43.380 INFO:teuthology.packaging:sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:24:43.380 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2024-02-15T18:24:43.683 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-15T18:24:43.710 INFO:teuthology.orchestra.run.smithi121.stdout:16.2.4-0.el8 2024-02-15T18:24:43.710 INFO:teuthology.packaging:The installed version of ceph is 16.2.4-0.el8 2024-02-15T18:24:43.710 INFO:teuthology.task.install:The correct ceph version 16.2.4-0 is installed. 2024-02-15T18:24:43.712 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-15T18:24:43.713 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-02-15T18:24:43.713 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-15T18:24:43.746 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-02-15T18:24:43.746 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-15T18:24:43.782 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-15T18:24:43.783 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-02-15T18:24:43.783 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/daemon-helper 2024-02-15T18:24:43.818 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-15T18:24:43.889 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-02-15T18:24:43.889 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/daemon-helper 2024-02-15T18:24:43.919 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-15T18:24:43.988 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-15T18:24:43.989 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-02-15T18:24:43.989 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-15T18:24:44.020 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-15T18:24:44.090 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-02-15T18:24:44.090 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-15T18:24:44.120 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-15T18:24:44.189 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-15T18:24:44.190 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-02-15T18:24:44.190 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/stdin-killer 2024-02-15T18:24:44.221 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-15T18:24:44.291 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-02-15T18:24:44.291 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/stdin-killer 2024-02-15T18:24:44.321 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-15T18:24:44.391 INFO:teuthology.run_tasks:Running task print... 2024-02-15T18:24:44.401 INFO:teuthology.task.print:**** done install task... 2024-02-15T18:24:44.401 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-15T18:24:44.487 INFO:tasks.cephadm:Config: {'avoid_pacific_features': True, 'cephadm_branch': 'v16.2.4', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'image': 'docker.io/ceph/ceph:v16.2.4', 'roleless': True, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(PG_DEGRADED\\)', 'Degraded data redundancy', '\\(PG_', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '0e714d9a4bd2a821113e6318adb87bd06cf81ec1'} 2024-02-15T18:24:44.487 INFO:tasks.cephadm:Cluster image is docker.io/ceph/ceph:v16.2.4 2024-02-15T18:24:44.488 INFO:tasks.cephadm:Cluster fsid is 7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:24:44.488 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-15T18:24:44.488 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-02-15T18:24:44.488 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi038': '172.21.15.38', 'mon.smithi121': '172.21.15.121'} 2024-02-15T18:24:44.488 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-15T18:24:44.488 DEBUG:teuthology.orchestra.run.smithi038:> sudo hostname $(hostname -s) 2024-02-15T18:24:44.521 DEBUG:teuthology.orchestra.run.smithi121:> sudo hostname $(hostname -s) 2024-02-15T18:24:44.551 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref v16.2.4)... 2024-02-15T18:24:44.551 DEBUG:teuthology.orchestra.run.smithi038:> curl --silent https://raw.githubusercontent.com/ceph/ceph/v16.2.4/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-15T18:24:44.816 INFO:teuthology.orchestra.run.smithi038.stdout:-rw-rw-r--. 1 ubuntu ubuntu 292005 Feb 15 18:24 /home/ubuntu/cephtest/cephadm 2024-02-15T18:24:44.817 DEBUG:teuthology.orchestra.run.smithi121:> curl --silent https://raw.githubusercontent.com/ceph/ceph/v16.2.4/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-15T18:24:45.086 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-rw-r--. 1 ubuntu ubuntu 292005 Feb 15 18:24 /home/ubuntu/cephtest/cephadm 2024-02-15T18:24:45.087 DEBUG:teuthology.orchestra.run.smithi038:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-15T18:24:45.110 DEBUG:teuthology.orchestra.run.smithi121:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-15T18:24:45.143 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /etc/ceph 2024-02-15T18:24:45.183 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /etc/ceph 2024-02-15T18:24:45.215 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 777 /etc/ceph 2024-02-15T18:24:45.255 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 777 /etc/ceph 2024-02-15T18:24:45.284 INFO:tasks.cephadm:Writing seed config... 2024-02-15T18:24:45.285 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-02-15T18:24:45.286 INFO:tasks.cephadm: override: [client] debug client = 20 2024-02-15T18:24:45.286 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-15T18:24:45.286 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-02-15T18:24:45.286 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-02-15T18:24:45.286 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-02-15T18:24:45.286 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-02-15T18:24:45.286 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-02-15T18:24:45.286 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-02-15T18:24:45.286 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-02-15T18:24:45.286 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-02-15T18:24:45.286 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-02-15T18:24:45.286 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-02-15T18:24:45.286 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-02-15T18:24:45.286 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-02-15T18:24:45.286 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-02-15T18:24:45.286 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-15T18:24:45.287 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-15T18:24:45.287 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-15T18:24:45.287 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-15T18:24:45.287 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-15T18:24:45.287 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-02-15T18:24:45.287 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-02-15T18:24:45.287 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-02-15T18:24:45.287 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-02-15T18:24:45.287 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-02-15T18:24:45.287 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-02-15T18:24:45.287 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-02-15T18:24:45.287 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-02-15T18:24:45.287 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-15T18:24:45.287 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-15T18:24:45.287 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-02-15T18:24:45.288 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-02-15T18:24:45.288 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-02-15T18:24:45.288 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-02-15T18:24:45.288 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-02-15T18:24:45.288 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-02-15T18:24:45.288 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-02-15T18:24:45.289 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-02-15T18:24:45.289 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-15T18:24:45.316 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 7e05753e-cc2f-11ee-95ba-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-02-15T18:24:45.317 DEBUG:teuthology.orchestra.run.smithi038:mon.smithi038> sudo journalctl -f -n 0 -u ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mon.smithi038.service 2024-02-15T18:24:45.359 INFO:tasks.cephadm:Bootstrapping... 2024-02-15T18:24:45.359 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 -v bootstrap --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.38 && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-15T18:24:45.389 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:-- Logs begin at Thu 2024-02-15 18:12:54 UTC. -- 2024-02-15T18:24:45.643 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman version --format {{.Client.Version}} 2024-02-15T18:24:46.531 INFO:teuthology.orchestra.run.smithi038.stderr:/bin/podman: 4.9.3-dev 2024-02-15T18:24:46.541 INFO:teuthology.orchestra.run.smithi038.stderr:Verifying podman|docker is present... 2024-02-15T18:24:46.541 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman version --format {{.Client.Version}} 2024-02-15T18:24:46.565 INFO:teuthology.orchestra.run.smithi038.stderr:/bin/podman: 4.9.3-dev 2024-02-15T18:24:46.576 INFO:teuthology.orchestra.run.smithi038.stderr:Verifying lvm2 is present... 2024-02-15T18:24:46.577 INFO:teuthology.orchestra.run.smithi038.stderr:Verifying time synchronization is in place... 2024-02-15T18:24:46.577 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl is-enabled chrony.service 2024-02-15T18:24:46.584 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-02-15T18:24:46.585 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl is-active chrony.service 2024-02-15T18:24:46.591 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: inactive 2024-02-15T18:24:46.592 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl is-enabled chronyd.service 2024-02-15T18:24:46.598 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: enabled 2024-02-15T18:24:46.599 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl is-active chronyd.service 2024-02-15T18:24:46.605 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: active 2024-02-15T18:24:46.605 INFO:teuthology.orchestra.run.smithi038.stderr:Unit chronyd.service is enabled and running 2024-02-15T18:24:46.606 INFO:teuthology.orchestra.run.smithi038.stderr:Repeating the final host check... 2024-02-15T18:24:46.606 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman version --format {{.Client.Version}} 2024-02-15T18:24:46.627 INFO:teuthology.orchestra.run.smithi038.stderr:/bin/podman: 4.9.3-dev 2024-02-15T18:24:46.638 INFO:teuthology.orchestra.run.smithi038.stderr:podman|docker (/bin/podman) is present 2024-02-15T18:24:46.638 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl is present 2024-02-15T18:24:46.638 INFO:teuthology.orchestra.run.smithi038.stderr:lvcreate is present 2024-02-15T18:24:46.638 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl is-enabled chrony.service 2024-02-15T18:24:46.644 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-02-15T18:24:46.644 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl is-active chrony.service 2024-02-15T18:24:46.650 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: inactive 2024-02-15T18:24:46.650 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl is-enabled chronyd.service 2024-02-15T18:24:46.656 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: enabled 2024-02-15T18:24:46.657 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl is-active chronyd.service 2024-02-15T18:24:46.662 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: active 2024-02-15T18:24:46.663 INFO:teuthology.orchestra.run.smithi038.stderr:Unit chronyd.service is enabled and running 2024-02-15T18:24:46.663 INFO:teuthology.orchestra.run.smithi038.stderr:Host looks OK 2024-02-15T18:24:46.663 INFO:teuthology.orchestra.run.smithi038.stderr:Cluster fsid: 7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:24:46.663 INFO:teuthology.orchestra.run.smithi038.stderr:Acquiring lock 140714512923112 on /run/cephadm/7e05753e-cc2f-11ee-95ba-87774f69a715.lock 2024-02-15T18:24:46.664 INFO:teuthology.orchestra.run.smithi038.stderr:Lock 140714512923112 acquired on /run/cephadm/7e05753e-cc2f-11ee-95ba-87774f69a715.lock 2024-02-15T18:24:46.664 INFO:teuthology.orchestra.run.smithi038.stderr:Verifying IP 172.21.15.38 port 3300 ... 2024-02-15T18:24:46.664 INFO:teuthology.orchestra.run.smithi038.stderr:Verifying IP 172.21.15.38 port 6789 ... 2024-02-15T18:24:46.664 INFO:teuthology.orchestra.run.smithi038.stderr:Base mon IP is 172.21.15.38, final addrv is [v2:172.21.15.38:3300,v1:172.21.15.38:6789] 2024-02-15T18:24:46.665 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /sbin/ip route ls 2024-02-15T18:24:46.667 INFO:teuthology.orchestra.run.smithi038.stderr:/sbin/ip: default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.38 metric 100 2024-02-15T18:24:46.667 INFO:teuthology.orchestra.run.smithi038.stderr:/sbin/ip: 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.38 metric 100 2024-02-15T18:24:46.667 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /sbin/ip -6 route ls 2024-02-15T18:24:46.670 INFO:teuthology.orchestra.run.smithi038.stderr:/sbin/ip: ::1 dev lo proto kernel metric 256 pref medium 2024-02-15T18:24:46.670 INFO:teuthology.orchestra.run.smithi038.stderr:/sbin/ip: fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-02-15T18:24:46.670 INFO:teuthology.orchestra.run.smithi038.stderr:/sbin/ip: default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1624sec hoplimit 64 pref medium 2024-02-15T18:24:46.670 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /sbin/ip -6 addr ls 2024-02-15T18:24:46.673 INFO:teuthology.orchestra.run.smithi038.stderr:/sbin/ip: 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-15T18:24:46.673 INFO:teuthology.orchestra.run.smithi038.stderr:/sbin/ip: inet6 ::1/128 scope host 2024-02-15T18:24:46.673 INFO:teuthology.orchestra.run.smithi038.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2024-02-15T18:24:46.673 INFO:teuthology.orchestra.run.smithi038.stderr:/sbin/ip: 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-15T18:24:46.673 INFO:teuthology.orchestra.run.smithi038.stderr:/sbin/ip: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-02-15T18:24:46.673 INFO:teuthology.orchestra.run.smithi038.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2024-02-15T18:24:46.674 INFO:teuthology.orchestra.run.smithi038.stderr:Mon IP 172.21.15.38 is in CIDR network 172.21.0.0/20 2024-02-15T18:24:46.674 INFO:teuthology.orchestra.run.smithi038.stderr:- internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-15T18:24:46.675 INFO:teuthology.orchestra.run.smithi038.stderr:Pulling container image docker.io/ceph/ceph:v16.2.4... 2024-02-15T18:24:46.675 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman pull docker.io/ceph/ceph:v16.2.4 2024-02-15T18:24:46.767 INFO:teuthology.orchestra.run.smithi038.stderr:/bin/podman: Trying to pull docker.io/ceph/ceph:v16.2.4... 2024-02-15T18:24:53.723 INFO:teuthology.orchestra.run.smithi038.stderr:/bin/podman: Getting image source signatures 2024-02-15T18:24:53.729 INFO:teuthology.orchestra.run.smithi038.stderr:/bin/podman: Copying blob sha256:551101690791055467b1840be133b2e6b029d263771bc6c46a96bb7acdacac39 2024-02-15T18:24:53.730 INFO:teuthology.orchestra.run.smithi038.stderr:/bin/podman: Copying blob sha256:7a0437f04f83f084b7ed68ad9c4a4947e12fc4e1b006b38129bac89114ec3621 2024-02-15T18:25:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:/bin/podman: Copying config sha256:8d91d370c2b86c07de46146aba8d36718eaefa69b1880c77fa312fda6efd7d29 2024-02-15T18:25:21.730 INFO:teuthology.orchestra.run.smithi038.stderr:/bin/podman: Writing manifest to image destination 2024-02-15T18:25:23.904 INFO:teuthology.orchestra.run.smithi038.stderr:/bin/podman: 8d91d370c2b86c07de46146aba8d36718eaefa69b1880c77fa312fda6efd7d29 2024-02-15T18:25:23.915 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 docker.io/ceph/ceph:v16.2.4 --version 2024-02-15T18:25:24.626 INFO:teuthology.orchestra.run.smithi038.stderr:ceph: ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable) 2024-02-15T18:25:25.468 INFO:teuthology.orchestra.run.smithi038.stderr:Ceph version: ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable) 2024-02-15T18:25:25.468 INFO:teuthology.orchestra.run.smithi038.stderr:Extracting ceph user uid/gid from container image... 2024-02-15T18:25:25.468 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 docker.io/ceph/ceph:v16.2.4 -c %u %g /var/lib/ceph 2024-02-15T18:25:26.051 INFO:teuthology.orchestra.run.smithi038.stderr:stat: 167 167 2024-02-15T18:25:26.622 INFO:teuthology.orchestra.run.smithi038.stderr:Creating initial keys... 2024-02-15T18:25:26.623 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph-authtool --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 docker.io/ceph/ceph:v16.2.4 --gen-print-key 2024-02-15T18:25:27.307 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-authtool: AQAXV85lbyezDBAAGn7yTB5//a21o78JDqvwqw== 2024-02-15T18:25:28.062 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph-authtool --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 docker.io/ceph/ceph:v16.2.4 --gen-print-key 2024-02-15T18:25:28.697 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-authtool: AQAYV85ly8frIxAAT2TK3pXGyC65sHC4npbzhQ== 2024-02-15T18:25:29.246 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph-authtool --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 docker.io/ceph/ceph:v16.2.4 --gen-print-key 2024-02-15T18:25:29.828 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-authtool: AQAZV85l9trgKhAAOxpoQPtjEvyC5AKv77C/7w== 2024-02-15T18:25:30.395 INFO:teuthology.orchestra.run.smithi038.stderr:Creating initial monmap... 2024-02-15T18:25:30.395 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/monmaptool --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /tmp/ceph-tmppb7froll:/tmp/monmap:z docker.io/ceph/ceph:v16.2.4 --create --clobber --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 --addv smithi038 [v2:172.21.15.38:3300,v1:172.21.15.38:6789] /tmp/monmap 2024-02-15T18:25:30.993 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-15T18:25:30.994 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: set fsid to 7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:25:30.994 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-15T18:25:31.607 INFO:teuthology.orchestra.run.smithi038.stderr:monmaptool for smithi038 [v2:172.21.15.38:3300,v1:172.21.15.38:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-15T18:25:31.608 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/monmaptool: set fsid to 7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:25:31.608 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-15T18:25:31.608 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-02-15T18:25:31.608 INFO:teuthology.orchestra.run.smithi038.stderr:Creating mon... 2024-02-15T18:25:31.608 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph-mon --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038:/var/lib/ceph/mon/ceph-smithi038:z -v /tmp/ceph-tmp_57x0wmc:/tmp/keyring:z -v /tmp/ceph-tmppb7froll:/tmp/monmap:z docker.io/ceph/ceph:v16.2.4 --mkfs -i smithi038 --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -c /dev/null --monmap /tmp/monmap --keyring /tmp/keyring --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true 2024-02-15T18:25:32.257 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.107+0000 7fd37e056700 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-15T18:25:32.257 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.107+0000 7fd37e056700 1 imported monmap: 2024-02-15T18:25:32.257 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: epoch 0 2024-02-15T18:25:32.257 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:25:32.257 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: last_changed 2024-02-15T18:25:30.869198+0000 2024-02-15T18:25:32.257 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: created 2024-02-15T18:25:30.869198+0000 2024-02-15T18:25:32.257 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: min_mon_release 0 (unknown) 2024-02-15T18:25:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: election_strategy: 1 2024-02-15T18:25:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.smithi038 2024-02-15T18:25:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: 2024-02-15T18:25:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.108+0000 7fd37e056700 0 /usr/bin/ceph-mon: set fsid to 7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:25:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: RocksDB version: 6.8.1 2024-02-15T18:25:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: 2024-02-15T18:25:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-15T18:25:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Compile date May 13 2021 2024-02-15T18:25:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: DB SUMMARY 2024-02-15T18:25:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: 2024-02-15T18:25:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi038/store.db dir, Total Num: 0, files: 2024-02-15T18:25:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: 2024-02-15T18:25:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi038/store.db: 2024-02-15T18:25:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: 2024-02-15T18:25:32.259 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.error_if_exists: 0 2024-02-15T18:25:32.259 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.create_if_missing: 1 2024-02-15T18:25:32.259 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.paranoid_checks: 1 2024-02-15T18:25:32.259 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.env: 0x560aae1791c0 2024-02-15T18:25:32.259 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.fs: Posix File System 2024-02-15T18:25:32.259 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.info_log: 0x560aaf83aec0 2024-02-15T18:25:32.259 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-15T18:25:32.259 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.statistics: (nil) 2024-02-15T18:25:32.259 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.use_fsync: 0 2024-02-15T18:25:32.259 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.max_log_file_size: 0 2024-02-15T18:25:32.259 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-15T18:25:32.259 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-15T18:25:32.259 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-15T18:25:32.259 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-15T18:25:32.260 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.allow_fallocate: 1 2024-02-15T18:25:32.260 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-15T18:25:32.260 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-15T18:25:32.260 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.use_direct_reads: 0 2024-02-15T18:25:32.260 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-15T18:25:32.260 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.create_missing_column_families: 0 2024-02-15T18:25:32.260 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.db_log_dir: 2024-02-15T18:25:32.260 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi038/store.db 2024-02-15T18:25:32.260 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-15T18:25:32.260 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.max_subcompactions: 1 2024-02-15T18:25:32.260 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.max_background_flushes: -1 2024-02-15T18:25:32.260 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-15T18:25:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-15T18:25:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-15T18:25:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-15T18:25:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-15T18:25:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.advise_random_on_open: 1 2024-02-15T18:25:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-15T18:25:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.write_buffer_manager: 0x560aaf841530 2024-02-15T18:25:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-15T18:25:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-15T18:25:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-15T18:25:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-15T18:25:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.rate_limiter: (nil) 2024-02-15T18:25:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-15T18:25:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-15T18:25:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-15T18:25:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-15T18:25:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.unordered_write: 0 2024-02-15T18:25:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-15T18:25:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-15T18:25:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-15T18:25:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-15T18:25:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.row_cache: None 2024-02-15T18:25:32.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.wal_filter: None 2024-02-15T18:25:32.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-15T18:25:32.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-15T18:25:32.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.preserve_deletes: 0 2024-02-15T18:25:32.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.two_write_queues: 0 2024-02-15T18:25:32.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.manual_wal_flush: 0 2024-02-15T18:25:32.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.atomic_flush: 0 2024-02-15T18:25:32.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-15T18:25:32.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-15T18:25:32.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-15T18:25:32.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.log_readahead_size: 0 2024-02-15T18:25:32.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-15T18:25:32.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.max_background_jobs: 2 2024-02-15T18:25:32.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.max_background_compactions: -1 2024-02-15T18:25:32.264 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-15T18:25:32.264 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-15T18:25:32.264 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-15T18:25:32.264 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.max_total_wal_size: 0 2024-02-15T18:25:32.264 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-15T18:25:32.264 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-15T18:25:32.264 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-15T18:25:32.264 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-15T18:25:32.264 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.max_open_files: -1 2024-02-15T18:25:32.264 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.bytes_per_sync: 0 2024-02-15T18:25:32.264 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-15T18:25:32.264 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-15T18:25:32.264 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-15T18:25:32.264 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Compression algorithms supported: 2024-02-15T18:25:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-15T18:25:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: kZSTD supported: 0 2024-02-15T18:25:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: kXpressCompression supported: 0 2024-02-15T18:25:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-15T18:25:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: kLZ4Compression supported: 1 2024-02-15T18:25:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: kBZip2Compression supported: 0 2024-02-15T18:25:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: kZlibCompression supported: 1 2024-02-15T18:25:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: kSnappyCompression supported: 1 2024-02-15T18:25:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-15T18:25:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.109+0000 7fd37e056700 4 rocksdb: [db_impl/db_impl_open.cc:273] Creating manifest 1 2024-02-15T18:25:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: 2024-02-15T18:25:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi038/store.db/MANIFEST-000001 2024-02-15T18:25:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: 2024-02-15T18:25:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-15T18:25:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: 2024-02-15T18:25:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-15T18:25:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.merge_operator: 2024-02-15T18:25:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.compaction_filter: None 2024-02-15T18:25:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.compaction_filter_factory: None 2024-02-15T18:25:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-15T18:25:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-15T18:25:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560aaf753cb8) 2024-02-15T18:25:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks: 1 2024-02-15T18:25:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-15T18:25:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-15T18:25:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: pin_top_level_index_and_filter: 1 2024-02-15T18:25:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: index_type: 0 2024-02-15T18:25:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: data_block_index_type: 0 2024-02-15T18:25:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: index_shortening: 1 2024-02-15T18:25:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: data_block_hash_table_util_ratio: 0.750000 2024-02-15T18:25:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: hash_index_allow_collision: 1 2024-02-15T18:25:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: checksum: 1 2024-02-15T18:25:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: no_block_cache: 0 2024-02-15T18:25:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: block_cache: 0x560aaf788d10 2024-02-15T18:25:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: block_cache_name: BinnedLRUCache 2024-02-15T18:25:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: block_cache_options: 2024-02-15T18:25:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: capacity : 536870912 2024-02-15T18:25:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: num_shard_bits : 4 2024-02-15T18:25:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: strict_capacity_limit : 0 2024-02-15T18:25:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: high_pri_pool_ratio: 0.000 2024-02-15T18:25:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: block_cache_compressed: (nil) 2024-02-15T18:25:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: persistent_cache: (nil) 2024-02-15T18:25:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: block_size: 4096 2024-02-15T18:25:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: block_size_deviation: 10 2024-02-15T18:25:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: block_restart_interval: 16 2024-02-15T18:25:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: index_block_restart_interval: 1 2024-02-15T18:25:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: metadata_block_size: 4096 2024-02-15T18:25:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: partition_filters: 0 2024-02-15T18:25:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: use_delta_encoding: 1 2024-02-15T18:25:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-15T18:25:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: whole_key_filtering: 1 2024-02-15T18:25:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: verify_compression: 0 2024-02-15T18:25:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: read_amp_bytes_per_bit: 0 2024-02-15T18:25:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: format_version: 2 2024-02-15T18:25:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: enable_index_compression: 1 2024-02-15T18:25:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: block_align: 0 2024-02-15T18:25:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: 2024-02-15T18:25:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-15T18:25:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-15T18:25:32.269 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.compression: NoCompression 2024-02-15T18:25:32.269 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-15T18:25:32.269 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-15T18:25:32.269 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-15T18:25:32.269 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.num_levels: 7 2024-02-15T18:25:32.269 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-15T18:25:32.269 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-15T18:25:32.269 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-15T18:25:32.269 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-15T18:25:32.269 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-15T18:25:32.269 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-15T18:25:32.269 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-15T18:25:32.270 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-15T18:25:32.270 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-15T18:25:32.270 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-15T18:25:32.270 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.compression_opts.level: 32767 2024-02-15T18:25:32.270 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-15T18:25:32.270 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-15T18:25:32.270 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-15T18:25:32.270 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.compression_opts.enabled: false 2024-02-15T18:25:32.270 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-15T18:25:32.270 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-15T18:25:32.270 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-15T18:25:32.270 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-15T18:25:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-15T18:25:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-15T18:25:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-15T18:25:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-15T18:25:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-15T18:25:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-15T18:25:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-15T18:25:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-15T18:25:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-15T18:25:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-15T18:25:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-15T18:25:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-15T18:25:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-15T18:25:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.arena_block_size: 4194304 2024-02-15T18:25:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-15T18:25:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-15T18:25:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-15T18:25:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-15T18:25:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-15T18:25:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-15T18:25:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-15T18:25:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-15T18:25:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-15T18:25:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-15T18:25:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-15T18:25:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-15T18:25:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-15T18:25:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-15T18:25:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.table_properties_collectors: 2024-02-15T18:25:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.inplace_update_support: 0 2024-02-15T18:25:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-15T18:25:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-15T18:25:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-15T18:25:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-15T18:25:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.bloom_locality: 0 2024-02-15T18:25:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.max_successive_merges: 0 2024-02-15T18:25:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-15T18:25:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-15T18:25:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.force_consistency_checks: 0 2024-02-15T18:25:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-15T18:25:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.ttl: 2592000 2024-02-15T18:25:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-15T18:25:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi038/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-15T18:25:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: 2024-02-15T18:25:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.112+0000 7fd37e056700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-02-15T18:25:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: 2024-02-15T18:25:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.113+0000 7fd37e056700 4 rocksdb: DB pointer 0x560aaf84d800 2024-02-15T18:25:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.113+0000 7fd366eef700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-15T18:25:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.113+0000 7fd366eef700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-15T18:25:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: ** DB Stats ** 2024-02-15T18:25:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:25:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-15T18:25:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-15T18:25:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-15T18:25:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-15T18:25:32.275 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-15T18:25:32.275 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-15T18:25:32.275 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: 2024-02-15T18:25:32.275 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-15T18:25:32.275 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:25:32.275 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:25:32.275 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:25:32.275 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:25:32.275 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: 2024-02-15T18:25:32.275 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-15T18:25:32.275 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:25:32.275 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:25:32.276 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:25:32.276 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2024-02-15T18:25:32.276 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-15T18:25:32.276 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2024-02-15T18:25:32.276 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2024-02-15T18:25:32.276 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2024-02-15T18:25:32.276 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:25:32.276 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:25:32.276 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-15T18:25:32.276 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: 2024-02-15T18:25:32.276 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2024-02-15T18:25:32.276 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: 2024-02-15T18:25:32.277 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-15T18:25:32.277 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:25:32.277 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:25:32.277 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:25:32.277 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:25:32.277 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: 2024-02-15T18:25:32.277 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-15T18:25:32.277 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:25:32.277 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:25:32.277 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:25:32.277 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2024-02-15T18:25:32.277 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-15T18:25:32.277 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2024-02-15T18:25:32.277 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2024-02-15T18:25:32.278 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2024-02-15T18:25:32.278 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:25:32.278 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:25:32.278 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-15T18:25:32.278 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: 2024-02-15T18:25:32.278 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2024-02-15T18:25:32.278 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: 2024-02-15T18:25:32.278 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.115+0000 7fd37e056700 4 rocksdb: [db_impl/db_impl.cc:397] Shutdown: canceling all background work 2024-02-15T18:25:32.278 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.115+0000 7fd37e056700 4 rocksdb: [db_impl/db_impl.cc:573] Shutdown complete 2024-02-15T18:25:32.278 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:25:32.115+0000 7fd37e056700 0 /usr/bin/ceph-mon: created monfs at /var/lib/ceph/mon/ceph-smithi038 for mon.smithi038 2024-02-15T18:25:32.804 INFO:teuthology.orchestra.run.smithi038.stderr:create mon.smithi038 on 2024-02-15T18:25:32.805 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: install -d -m0770 -o 167 -g 167 /var/run/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:25:32.811 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl disable ceph.target 2024-02-15T18:25:32.819 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-02-15T18:25:32.913 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl enable ceph.target 2024-02-15T18:25:32.920 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-15T18:25:33.007 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl start ceph.target 2024-02-15T18:25:33.014 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl enable ceph-7e05753e-cc2f-11ee-95ba-87774f69a715.target 2024-02-15T18:25:33.020 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-7e05753e-cc2f-11ee-95ba-87774f69a715.target → /etc/systemd/system/ceph-7e05753e-cc2f-11ee-95ba-87774f69a715.target. 2024-02-15T18:25:33.020 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: Created symlink /etc/systemd/system/ceph.target.wants/ceph-7e05753e-cc2f-11ee-95ba-87774f69a715.target → /etc/systemd/system/ceph-7e05753e-cc2f-11ee-95ba-87774f69a715.target. 2024-02-15T18:25:33.104 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl start ceph-7e05753e-cc2f-11ee-95ba-87774f69a715.target 2024-02-15T18:25:33.112 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl daemon-reload 2024-02-15T18:25:33.201 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl stop ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mon.smithi038 2024-02-15T18:25:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl reset-failed ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mon.smithi038 2024-02-15T18:25:33.235 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: Failed to reset failed state of unit ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mon.smithi038.service: Unit ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mon.smithi038.service not loaded. 2024-02-15T18:25:33.236 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl enable ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mon.smithi038 2024-02-15T18:25:33.242 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: Created symlink /etc/systemd/system/ceph-7e05753e-cc2f-11ee-95ba-87774f69a715.target.wants/ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mon.smithi038.service → /etc/systemd/system/ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@.service. 2024-02-15T18:25:33.331 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl start ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mon.smithi038 2024-02-15T18:25:33.372 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:33 smithi038 systemd[1]: Starting Ceph mon.smithi038 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:25:33.919 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl is-enabled firewalld.service 2024-02-15T18:25:33.927 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: disabled 2024-02-15T18:25:33.928 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl is-active firewalld.service 2024-02-15T18:25:33.935 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: inactive 2024-02-15T18:25:33.936 INFO:teuthology.orchestra.run.smithi038.stderr:firewalld.service is not enabled 2024-02-15T18:25:33.936 INFO:teuthology.orchestra.run.smithi038.stderr:Not possible to enable service . firewalld.service is not available 2024-02-15T18:25:33.936 INFO:teuthology.orchestra.run.smithi038.stderr:Not possible to open ports <[]>. firewalld.service is not available 2024-02-15T18:25:33.937 INFO:teuthology.orchestra.run.smithi038.stderr:Waiting for mon to start... 2024-02-15T18:25:33.937 INFO:teuthology.orchestra.run.smithi038.stderr:Waiting for mon... 2024-02-15T18:25:33.937 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038:/var/lib/ceph/mon/ceph-smithi038:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 status 2024-02-15T18:25:34.168 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[37727]: cluster 2024-02-15T18:25:33.849672+0000 mon.smithi038 (mon.0) 0 : [INF] mkfs 7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:25:34.169 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[37727]: cluster 2024-02-15T18:25:33.834147+0000 mon.smithi038 (mon.0) 1 : cluster [INF] mon.smithi038 is new leader, mons smithi038 in quorum (ranks 0) 2024-02-15T18:25:34.169 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:33 smithi038 bash[37712]: d6c53a7512d91859fac4e86944ec8ff2547e37a4beead5202c372193628844a5 2024-02-15T18:25:34.169 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:33 smithi038 systemd[1]: Started Ceph mon.smithi038 for 7e05753e-cc2f-11ee-95ba-87774f69a715. 2024-02-15T18:25:34.494 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.496+0000 7fd131b4a700 1 Processor -- start 2024-02-15T18:25:34.496 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.496+0000 7fd131b4a700 1 -- start start 2024-02-15T18:25:34.496 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.496+0000 7fd131b4a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c070370 0x7fd12c0feb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:34.496 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.496+0000 7fd131b4a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c070790 con 0x7fd12c070370 2024-02-15T18:25:34.497 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.497+0000 7fd12b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c070370 0x7fd12c0feb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:34.497 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.497+0000 7fd12b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c070370 0x7fd12c0feb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47626/0 (socket says 172.21.15.38:47626) 2024-02-15T18:25:34.497 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.497+0000 7fd12b7fe700 1 -- 172.21.15.38:0/3530265828 learned_addr learned my addr 172.21.15.38:0/3530265828 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:25:34.497 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.497+0000 7fd12b7fe700 1 -- 172.21.15.38:0/3530265828 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c0ff000 con 0x7fd12c070370 2024-02-15T18:25:34.497 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.497+0000 7fd12b7fe700 1 --2- 172.21.15.38:0/3530265828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c070370 0x7fd12c0feb10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fd11c00b5f0 tx=0x7fd11c006ee0).ready entity=mon.0 client_cookie=a5a4623bffcbd071 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:34.497 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.497+0000 7fd12a7fc700 1 -- 172.21.15.38:0/3530265828 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd11c00d600 con 0x7fd12c070370 2024-02-15T18:25:34.497 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.497+0000 7fd12a7fc700 1 -- 172.21.15.38:0/3530265828 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11c00d760 con 0x7fd12c070370 2024-02-15T18:25:34.497 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.497+0000 7fd12a7fc700 1 -- 172.21.15.38:0/3530265828 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd11c00da30 con 0x7fd12c070370 2024-02-15T18:25:34.497 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.498+0000 7fd131b4a700 1 -- 172.21.15.38:0/3530265828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c070370 msgr2=0x7fd12c0feb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:34.497 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.498+0000 7fd131b4a700 1 --2- 172.21.15.38:0/3530265828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c070370 0x7fd12c0feb10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fd11c00b5f0 tx=0x7fd11c006ee0).stop 2024-02-15T18:25:34.497 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.498+0000 7fd131b4a700 1 -- 172.21.15.38:0/3530265828 shutdown_connections 2024-02-15T18:25:34.497 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.498+0000 7fd131b4a700 1 --2- 172.21.15.38:0/3530265828 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c070370 0x7fd12c0feb10 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:34.497 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.498+0000 7fd131b4a700 1 -- 172.21.15.38:0/3530265828 >> 172.21.15.38:0/3530265828 conn(0x7fd12c0f6530 msgr2=0x7fd12c0f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:34.497 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.498+0000 7fd131b4a700 1 -- 172.21.15.38:0/3530265828 shutdown_connections 2024-02-15T18:25:34.498 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.498+0000 7fd131b4a700 1 -- 172.21.15.38:0/3530265828 wait complete. 2024-02-15T18:25:34.499 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.499+0000 7fd131b4a700 1 Processor -- start 2024-02-15T18:25:34.499 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.499+0000 7fd131b4a700 1 -- start start 2024-02-15T18:25:34.499 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.499+0000 7fd131b4a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c070370 0x7fd12c0fc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:34.500 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.499+0000 7fd131b4a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c0faa10 con 0x7fd12c070370 2024-02-15T18:25:34.500 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.499+0000 7fd12b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c070370 0x7fd12c0fc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:34.500 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.499+0000 7fd12b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c070370 0x7fd12c0fc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47632/0 (socket says 172.21.15.38:47632) 2024-02-15T18:25:34.500 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.499+0000 7fd12b7fe700 1 -- 172.21.15.38:0/2813999442 learned_addr learned my addr 172.21.15.38:0/2813999442 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:25:34.500 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.500+0000 7fd12b7fe700 1 -- 172.21.15.38:0/2813999442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11c00b200 con 0x7fd12c070370 2024-02-15T18:25:34.500 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.500+0000 7fd12b7fe700 1 --2- 172.21.15.38:0/2813999442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c070370 0x7fd12c0fc590 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7fd11c006c70 tx=0x7fd11c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:34.500 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.500+0000 7fd113fff700 1 -- 172.21.15.38:0/2813999442 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd11c011b40 con 0x7fd12c070370 2024-02-15T18:25:34.500 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.500+0000 7fd113fff700 1 -- 172.21.15.38:0/2813999442 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11c011ca0 con 0x7fd12c070370 2024-02-15T18:25:34.500 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.500+0000 7fd113fff700 1 -- 172.21.15.38:0/2813999442 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd11c01d430 con 0x7fd12c070370 2024-02-15T18:25:34.500 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.500+0000 7fd131b4a700 1 -- 172.21.15.38:0/2813999442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12c0fabb0 con 0x7fd12c070370 2024-02-15T18:25:34.500 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.500+0000 7fd131b4a700 1 -- 172.21.15.38:0/2813999442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12c0fb040 con 0x7fd12c070370 2024-02-15T18:25:34.500 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.500+0000 7fd113fff700 1 -- 172.21.15.38:0/2813999442 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fd11c00f070 con 0x7fd12c070370 2024-02-15T18:25:34.500 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.500+0000 7fd113fff700 1 -- 172.21.15.38:0/2813999442 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fd11c027460 con 0x7fd12c070370 2024-02-15T18:25:34.501 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.501+0000 7fd131b4a700 1 -- 172.21.15.38:0/2813999442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12c045ad0 con 0x7fd12c070370 2024-02-15T18:25:34.501 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.504+0000 7fd113fff700 1 -- 172.21.15.38:0/2813999442 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7fd11c016070 con 0x7fd12c070370 2024-02-15T18:25:34.574 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.573+0000 7fd131b4a700 1 -- 172.21.15.38:0/2813999442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fd12c104090 con 0x7fd12c070370 2024-02-15T18:25:34.574 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.573+0000 7fd113fff700 1 -- 172.21.15.38:0/2813999442 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7fd11c036070 con 0x7fd12c070370 2024-02-15T18:25:34.574 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: cluster: 2024-02-15T18:25:34.574 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: id: 7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:25:34.574 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: health: HEALTH_OK 2024-02-15T18:25:34.574 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.574 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: services: 2024-02-15T18:25:34.574 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: mon: 1 daemons, quorum smithi038 (age 0.725248s) 2024-02-15T18:25:34.574 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: mgr: no daemons active 2024-02-15T18:25:34.575 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: osd: 0 osds: 0 up, 0 in 2024-02-15T18:25:34.575 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.575 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: data: 2024-02-15T18:25:34.575 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: pools: 0 pools, 0 pgs 2024-02-15T18:25:34.575 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: objects: 0 objects, 0 B 2024-02-15T18:25:34.575 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: usage: 0 B used, 0 B / 0 B avail 2024-02-15T18:25:34.575 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: pgs: 2024-02-15T18:25:34.575 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.575 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.574+0000 7fd131b4a700 1 -- 172.21.15.38:0/2813999442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c070370 msgr2=0x7fd12c0fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:34.575 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.574+0000 7fd131b4a700 1 --2- 172.21.15.38:0/2813999442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c070370 0x7fd12c0fc590 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7fd11c006c70 tx=0x7fd11c016000).stop 2024-02-15T18:25:34.575 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.574+0000 7fd131b4a700 1 -- 172.21.15.38:0/2813999442 shutdown_connections 2024-02-15T18:25:34.575 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.574+0000 7fd131b4a700 1 --2- 172.21.15.38:0/2813999442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd12c070370 0x7fd12c0fc590 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:34.575 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.574+0000 7fd131b4a700 1 -- 172.21.15.38:0/2813999442 >> 172.21.15.38:0/2813999442 conn(0x7fd12c0f6530 msgr2=0x7fd12c0f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:34.575 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.574+0000 7fd131b4a700 1 -- 172.21.15.38:0/2813999442 shutdown_connections 2024-02-15T18:25:34.576 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:34.574+0000 7fd131b4a700 1 -- 172.21.15.38:0/2813999442 wait complete. 2024-02-15T18:25:35.168 INFO:teuthology.orchestra.run.smithi038.stderr:mon is available 2024-02-15T18:25:35.168 INFO:teuthology.orchestra.run.smithi038.stderr:Assimilating anything we can from ceph.conf... 2024-02-15T18:25:35.168 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z -v /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038:/var/lib/ceph/mon/ceph-smithi038:z docker.io/ceph/ceph:v16.2.4 config assimilate-conf -i /var/lib/ceph/mon/ceph-smithi038/config 2024-02-15T18:25:35.170 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[37727]: cluster 2024-02-15T18:25:33.849023+0000 mon.smithi038 (mon.0) 2 : cluster [INF] mon.smithi038 is new leader, mons smithi038 in quorum (ranks 0) 2024-02-15T18:25:35.170 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[37727]: cluster 2024-02-15T18:25:33.849255+0000 mon.smithi038 (mon.0) 3 : cluster [DBG] monmap e1: 1 mons at {smithi038=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0]} 2024-02-15T18:25:35.170 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[37727]: cluster 2024-02-15T18:25:33.850957+0000 mon.smithi038 (mon.0) 4 : cluster [DBG] fsmap 2024-02-15T18:25:35.170 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[37727]: cluster 2024-02-15T18:25:33.855595+0000 mon.smithi038 (mon.0) 5 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-02-15T18:25:35.170 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[37727]: cluster 2024-02-15T18:25:33.857283+0000 mon.smithi038 (mon.0) 6 : cluster [DBG] mgrmap e1: no daemons active 2024-02-15T18:25:35.170 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[37727]: audit 2024-02-15T18:25:34.574242+0000 mon.smithi038 (mon.0) 7 : audit [DBG] from='client.? 172.21.15.38:0/2813999442' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-15T18:25:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.690+0000 7fc368315700 1 Processor -- start 2024-02-15T18:25:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.690+0000 7fc368315700 1 -- start start 2024-02-15T18:25:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.691+0000 7fc368315700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3600703f0 0x7fc3600fea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:35.692 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.691+0000 7fc368315700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3600fef80 con 0x7fc3600703f0 2024-02-15T18:25:35.692 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.691+0000 7fc3660b1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3600703f0 0x7fc3600fea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:35.692 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.691+0000 7fc3660b1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3600703f0 0x7fc3600fea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47648/0 (socket says 172.21.15.38:47648) 2024-02-15T18:25:35.692 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.691+0000 7fc3660b1700 1 -- 172.21.15.38:0/4089896194 learned_addr learned my addr 172.21.15.38:0/4089896194 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:25:35.692 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.691+0000 7fc3660b1700 1 -- 172.21.15.38:0/4089896194 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3600ff0c0 con 0x7fc3600703f0 2024-02-15T18:25:35.692 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.692+0000 7fc3660b1700 1 --2- 172.21.15.38:0/4089896194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3600703f0 0x7fc3600fea90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7fc35c00b5f0 tx=0x7fc35c006ee0).ready entity=mon.0 client_cookie=de6cd7e87df5d759 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:35.692 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.692+0000 7fc3650af700 1 -- 172.21.15.38:0/4089896194 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc35c00d600 con 0x7fc3600703f0 2024-02-15T18:25:35.692 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.692+0000 7fc3650af700 1 -- 172.21.15.38:0/4089896194 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35c00d760 con 0x7fc3600703f0 2024-02-15T18:25:35.692 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.692+0000 7fc3650af700 1 -- 172.21.15.38:0/4089896194 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc35c00da30 con 0x7fc3600703f0 2024-02-15T18:25:35.692 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.692+0000 7fc368315700 1 -- 172.21.15.38:0/4089896194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3600703f0 msgr2=0x7fc3600fea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:35.692 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.692+0000 7fc368315700 1 --2- 172.21.15.38:0/4089896194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3600703f0 0x7fc3600fea90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7fc35c00b5f0 tx=0x7fc35c006ee0).stop 2024-02-15T18:25:35.692 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.692+0000 7fc368315700 1 -- 172.21.15.38:0/4089896194 shutdown_connections 2024-02-15T18:25:35.692 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.692+0000 7fc368315700 1 --2- 172.21.15.38:0/4089896194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3600703f0 0x7fc3600fea90 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:35.693 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.692+0000 7fc368315700 1 -- 172.21.15.38:0/4089896194 >> 172.21.15.38:0/4089896194 conn(0x7fc3600f6530 msgr2=0x7fc3600f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:35.693 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.692+0000 7fc368315700 1 -- 172.21.15.38:0/4089896194 shutdown_connections 2024-02-15T18:25:35.693 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.692+0000 7fc368315700 1 -- 172.21.15.38:0/4089896194 wait complete. 2024-02-15T18:25:35.693 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.693+0000 7fc368315700 1 Processor -- start 2024-02-15T18:25:35.693 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.693+0000 7fc368315700 1 -- start start 2024-02-15T18:25:35.693 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.693+0000 7fc368315700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3600703f0 0x7fc36010a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:35.693 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.693+0000 7fc368315700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36010a860 con 0x7fc3600703f0 2024-02-15T18:25:35.693 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.694+0000 7fc3660b1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3600703f0 0x7fc36010a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:35.693 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.694+0000 7fc3660b1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3600703f0 0x7fc36010a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47664/0 (socket says 172.21.15.38:47664) 2024-02-15T18:25:35.693 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.694+0000 7fc3660b1700 1 -- 172.21.15.38:0/2313978676 learned_addr learned my addr 172.21.15.38:0/2313978676 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:25:35.695 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.694+0000 7fc3660b1700 1 -- 172.21.15.38:0/2313978676 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c00b200 con 0x7fc3600703f0 2024-02-15T18:25:35.695 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.694+0000 7fc3660b1700 1 --2- 172.21.15.38:0/2313978676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3600703f0 0x7fc36010a370 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7fc35c006c70 tx=0x7fc35c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:35.695 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.694+0000 7fc352ffd700 1 -- 172.21.15.38:0/2313978676 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc35c011c80 con 0x7fc3600703f0 2024-02-15T18:25:35.695 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.694+0000 7fc352ffd700 1 -- 172.21.15.38:0/2313978676 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35c011de0 con 0x7fc3600703f0 2024-02-15T18:25:35.695 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.694+0000 7fc368315700 1 -- 172.21.15.38:0/2313978676 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36010aa60 con 0x7fc3600703f0 2024-02-15T18:25:35.695 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.694+0000 7fc352ffd700 1 -- 172.21.15.38:0/2313978676 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc35c01d5d0 con 0x7fc3600703f0 2024-02-15T18:25:35.695 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.694+0000 7fc368315700 1 -- 172.21.15.38:0/2313978676 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36010aef0 con 0x7fc3600703f0 2024-02-15T18:25:35.695 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.695+0000 7fc352ffd700 1 -- 172.21.15.38:0/2313978676 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fc35c00f070 con 0x7fc3600703f0 2024-02-15T18:25:35.695 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.695+0000 7fc352ffd700 1 -- 172.21.15.38:0/2313978676 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fc35c027460 con 0x7fc3600703f0 2024-02-15T18:25:35.695 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.696+0000 7fc368315700 1 -- 172.21.15.38:0/2313978676 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc360045ad0 con 0x7fc3600703f0 2024-02-15T18:25:35.696 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.698+0000 7fc352ffd700 1 -- 172.21.15.38:0/2313978676 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7fc35c016070 con 0x7fc3600703f0 2024-02-15T18:25:35.762 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.765+0000 7fc368315700 1 -- 172.21.15.38:0/2313978676 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fc3601040a0 con 0x7fc3600703f0 2024-02-15T18:25:35.771 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.771+0000 7fc352ffd700 1 -- 172.21.15.38:0/2313978676 <== mon.0 v2:172.21.15.38:3300/0 7 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fc35c0318c0 con 0x7fc3600703f0 2024-02-15T18:25:35.771 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.771+0000 7fc352ffd700 1 -- 172.21.15.38:0/2313978676 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+489 (secure 0 0 0) 0x7fc35c030790 con 0x7fc3600703f0 2024-02-15T18:25:35.771 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.771 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: [global] 2024-02-15T18:25:35.771 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: fsid = 7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:25:35.771 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: log_to_journald = false 2024-02-15T18:25:35.771 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: mon_host = [v2:172.21.15.38:3300,v1:172.21.15.38:6789] 2024-02-15T18:25:35.771 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-02-15T18:25:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-02-15T18:25:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-02-15T18:25:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-02-15T18:25:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: [mgr] 2024-02-15T18:25:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-02-15T18:25:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: [osd] 2024-02-15T18:25:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: mon_osd_backfillfull_ratio = 0.85 2024-02-15T18:25:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: mon_osd_full_ratio = 0.9 2024-02-15T18:25:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: mon_osd_nearfull_ratio = 0.8 2024-02-15T18:25:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-02-15T18:25:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: osd_memory_target_autotune = true 2024-02-15T18:25:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-02-15T18:25:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.772+0000 7fc368315700 1 -- 172.21.15.38:0/2313978676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3600703f0 msgr2=0x7fc36010a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.772+0000 7fc368315700 1 --2- 172.21.15.38:0/2313978676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3600703f0 0x7fc36010a370 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7fc35c006c70 tx=0x7fc35c016000).stop 2024-02-15T18:25:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.772+0000 7fc368315700 1 -- 172.21.15.38:0/2313978676 shutdown_connections 2024-02-15T18:25:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.772+0000 7fc368315700 1 --2- 172.21.15.38:0/2313978676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3600703f0 0x7fc36010a370 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.772+0000 7fc368315700 1 -- 172.21.15.38:0/2313978676 >> 172.21.15.38:0/2313978676 conn(0x7fc3600f6530 msgr2=0x7fc3600f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.772+0000 7fc368315700 1 -- 172.21.15.38:0/2313978676 shutdown_connections 2024-02-15T18:25:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:35.772+0000 7fc368315700 1 -- 172.21.15.38:0/2313978676 wait complete. 2024-02-15T18:25:36.103 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[37727]: audit 2024-02-15T18:25:35.766290+0000 mon.smithi038 (mon.0) 8 : audit [INF] from='client.? 172.21.15.38:0/2313978676' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-15T18:25:36.103 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[37727]: audit 2024-02-15T18:25:35.771984+0000 mon.smithi038 (mon.0) 9 : audit [INF] from='client.? 172.21.15.38:0/2313978676' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-15T18:25:36.399 INFO:teuthology.orchestra.run.smithi038.stderr:Generating new minimal ceph.conf... 2024-02-15T18:25:36.399 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z -v /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038:/var/lib/ceph/mon/ceph-smithi038:z docker.io/ceph/ceph:v16.2.4 config generate-minimal-conf -o /var/lib/ceph/mon/ceph-smithi038/config 2024-02-15T18:25:36.977 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.966+0000 7f1f6ef88700 1 Processor -- start 2024-02-15T18:25:36.977 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.966+0000 7f1f6ef88700 1 -- start start 2024-02-15T18:25:36.977 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.966+0000 7f1f6ef88700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f68070430 0x7f1f680fea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.966+0000 7f1f6ef88700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f680fef40 con 0x7f1f68070430 2024-02-15T18:25:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.966+0000 7f1f6cd24700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f68070430 0x7f1f680fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.966+0000 7f1f6cd24700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f68070430 0x7f1f680fea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47680/0 (socket says 172.21.15.38:47680) 2024-02-15T18:25:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.966+0000 7f1f6cd24700 1 -- 172.21.15.38:0/788214415 learned_addr learned my addr 172.21.15.38:0/788214415 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:25:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.967+0000 7f1f6cd24700 1 -- 172.21.15.38:0/788214415 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f680ff080 con 0x7f1f68070430 2024-02-15T18:25:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.967+0000 7f1f6cd24700 1 --2- 172.21.15.38:0/788214415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f68070430 0x7f1f680fea50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f1f5c00b5f0 tx=0x7f1f5c006ee0).ready entity=mon.0 client_cookie=3387f95c79e92265 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.967+0000 7f1f677fe700 1 -- 172.21.15.38:0/788214415 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f5c00d600 con 0x7f1f68070430 2024-02-15T18:25:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.967+0000 7f1f677fe700 1 -- 172.21.15.38:0/788214415 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f1f5c00dbf0 con 0x7f1f68070430 2024-02-15T18:25:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.967+0000 7f1f677fe700 1 -- 172.21.15.38:0/788214415 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f5c011cb0 con 0x7f1f68070430 2024-02-15T18:25:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.967+0000 7f1f6ef88700 1 -- 172.21.15.38:0/788214415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f68070430 msgr2=0x7f1f680fea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.967+0000 7f1f6ef88700 1 --2- 172.21.15.38:0/788214415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f68070430 0x7f1f680fea50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f1f5c00b5f0 tx=0x7f1f5c006ee0).stop 2024-02-15T18:25:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.967+0000 7f1f6ef88700 1 -- 172.21.15.38:0/788214415 shutdown_connections 2024-02-15T18:25:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.967+0000 7f1f6ef88700 1 --2- 172.21.15.38:0/788214415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f68070430 0x7f1f680fea50 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:36.978 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.967+0000 7f1f6ef88700 1 -- 172.21.15.38:0/788214415 >> 172.21.15.38:0/788214415 conn(0x7f1f680f6530 msgr2=0x7f1f680f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:36.979 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.968+0000 7f1f6ef88700 1 -- 172.21.15.38:0/788214415 shutdown_connections 2024-02-15T18:25:36.979 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.968+0000 7f1f6ef88700 1 -- 172.21.15.38:0/788214415 wait complete. 2024-02-15T18:25:36.979 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.968+0000 7f1f6ef88700 1 Processor -- start 2024-02-15T18:25:36.979 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.968+0000 7f1f6ef88700 1 -- start start 2024-02-15T18:25:36.979 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.968+0000 7f1f6ef88700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f68070430 0x7f1f6810ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:36.979 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.968+0000 7f1f6ef88700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6810b030 con 0x7f1f68070430 2024-02-15T18:25:36.979 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.969+0000 7f1f6cd24700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f68070430 0x7f1f6810ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:36.979 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.969+0000 7f1f6cd24700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f68070430 0x7f1f6810ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47692/0 (socket says 172.21.15.38:47692) 2024-02-15T18:25:36.979 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.969+0000 7f1f6cd24700 1 -- 172.21.15.38:0/1488605796 learned_addr learned my addr 172.21.15.38:0/1488605796 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:25:36.979 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.969+0000 7f1f6cd24700 1 -- 172.21.15.38:0/1488605796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c00b200 con 0x7f1f68070430 2024-02-15T18:25:36.979 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.969+0000 7f1f6cd24700 1 --2- 172.21.15.38:0/1488605796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f68070430 0x7f1f6810ab40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f1f5c006800 tx=0x7f1f5c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:36.979 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.969+0000 7f1f657fa700 1 -- 172.21.15.38:0/1488605796 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f5c01b570 con 0x7f1f68070430 2024-02-15T18:25:36.979 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.969+0000 7f1f657fa700 1 -- 172.21.15.38:0/1488605796 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f1f5c01bb60 con 0x7f1f68070430 2024-02-15T18:25:36.980 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.969+0000 7f1f657fa700 1 -- 172.21.15.38:0/1488605796 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f5c01a9f0 con 0x7f1f68070430 2024-02-15T18:25:36.980 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.969+0000 7f1f6ef88700 1 -- 172.21.15.38:0/1488605796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6810b230 con 0x7f1f68070430 2024-02-15T18:25:36.980 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.969+0000 7f1f6ef88700 1 -- 172.21.15.38:0/1488605796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6810b6c0 con 0x7f1f68070430 2024-02-15T18:25:36.980 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.969+0000 7f1f657fa700 1 -- 172.21.15.38:0/1488605796 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f1f5c00f070 con 0x7f1f68070430 2024-02-15T18:25:36.980 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.969+0000 7f1f657fa700 1 -- 172.21.15.38:0/1488605796 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f1f5c025cb0 con 0x7f1f68070430 2024-02-15T18:25:36.980 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.970+0000 7f1f6ef88700 1 -- 172.21.15.38:0/1488605796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f68045ad0 con 0x7f1f68070430 2024-02-15T18:25:36.980 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:36.973+0000 7f1f657fa700 1 -- 172.21.15.38:0/1488605796 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7f1f5c016070 con 0x7f1f68070430 2024-02-15T18:25:37.038 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:37.041+0000 7f1f6ef88700 1 -- 172.21.15.38:0/1488605796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f1f680ffdc0 con 0x7f1f68070430 2024-02-15T18:25:37.038 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:37.041+0000 7f1f657fa700 1 -- 172.21.15.38:0/1488605796 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f1f5c01aea0 con 0x7f1f68070430 2024-02-15T18:25:37.039 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:37.042+0000 7f1f6ef88700 1 -- 172.21.15.38:0/1488605796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f68070430 msgr2=0x7f1f6810ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:37.039 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:37.042+0000 7f1f6ef88700 1 --2- 172.21.15.38:0/1488605796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f68070430 0x7f1f6810ab40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f1f5c006800 tx=0x7f1f5c006ee0).stop 2024-02-15T18:25:37.040 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:37.042+0000 7f1f6ef88700 1 -- 172.21.15.38:0/1488605796 shutdown_connections 2024-02-15T18:25:37.040 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:37.042+0000 7f1f6ef88700 1 --2- 172.21.15.38:0/1488605796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f68070430 0x7f1f6810ab40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:37.040 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:37.042+0000 7f1f6ef88700 1 -- 172.21.15.38:0/1488605796 >> 172.21.15.38:0/1488605796 conn(0x7f1f680f6530 msgr2=0x7f1f680f7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:37.040 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:37.042+0000 7f1f6ef88700 1 -- 172.21.15.38:0/1488605796 shutdown_connections 2024-02-15T18:25:37.040 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:37.042+0000 7f1f6ef88700 1 -- 172.21.15.38:0/1488605796 wait complete. 2024-02-15T18:25:37.468 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[37727]: audit 2024-02-15T18:25:37.042046+0000 mon.smithi038 (mon.0) 10 : audit [DBG] from='client.? 172.21.15.38:0/1488605796' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:25:37.765 INFO:teuthology.orchestra.run.smithi038.stderr:Restarting the monitor... 2024-02-15T18:25:37.765 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl restart ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mon.smithi038 2024-02-15T18:25:38.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:37 smithi038 systemd[1]: Stopping Ceph mon.smithi038 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:25:38.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[37727]: debug 2024-02-15T18:25:37.908+0000 7f7b7a7fb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi038 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-15T18:25:38.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[37727]: debug 2024-02-15T18:25:37.908+0000 7f7b7a7fb700 -1 mon.smithi038@0(leader) e1 *** Got Signal Terminated *** 2024-02-15T18:25:38.640 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:38 smithi038 podman[38051]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038 2024-02-15T18:25:39.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:38 smithi038 systemd[1]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mon.smithi038.service: Succeeded. 2024-02-15T18:25:39.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:38 smithi038 systemd[1]: Stopped Ceph mon.smithi038 for 7e05753e-cc2f-11ee-95ba-87774f69a715. 2024-02-15T18:25:39.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:38 smithi038 systemd[1]: Starting Ceph mon.smithi038 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:25:39.222 INFO:teuthology.orchestra.run.smithi038.stderr:Setting mon public_network to 172.21.0.0/20 2024-02-15T18:25:39.222 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 config set mon public_network 172.21.0.0/20 2024-02-15T18:25:39.488 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.173+0000 7f5ae7d94700 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-15T18:25:39.488 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.173+0000 7f5ae7d94700 0 ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable), process ceph-mon, pid 7 2024-02-15T18:25:39.489 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.173+0000 7f5ae7d94700 0 pidfile_write: ignore empty --pid-file 2024-02-15T18:25:39.489 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.177+0000 7f5ae7d94700 0 load: jerasure load: lrc load: isa 2024-02-15T18:25:39.489 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: RocksDB version: 6.8.1 2024-02-15T18:25:39.489 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.489 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-15T18:25:39.489 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Compile date May 13 2021 2024-02-15T18:25:39.489 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: DB SUMMARY 2024-02-15T18:25:39.489 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.489 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: CURRENT file: CURRENT 2024-02-15T18:25:39.489 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.489 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: IDENTITY file: IDENTITY 2024-02-15T18:25:39.489 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.490 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 131 Bytes 2024-02-15T18:25:39.490 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.490 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi038/store.db dir, Total Num: 1, files: 000004.sst 2024-02-15T18:25:39.490 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.490 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi038/store.db: 000006.log size: 98708 ; 2024-02-15T18:25:39.490 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.490 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.error_if_exists: 0 2024-02-15T18:25:39.490 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.create_if_missing: 0 2024-02-15T18:25:39.490 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.paranoid_checks: 1 2024-02-15T18:25:39.490 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.env: 0x55ea467e51c0 2024-02-15T18:25:39.490 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.fs: Posix File System 2024-02-15T18:25:39.490 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.info_log: 0x55ea471a5960 2024-02-15T18:25:39.490 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-15T18:25:39.491 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.statistics: (nil) 2024-02-15T18:25:39.491 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.use_fsync: 0 2024-02-15T18:25:39.491 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_log_file_size: 0 2024-02-15T18:25:39.491 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-15T18:25:39.491 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-15T18:25:39.491 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-15T18:25:39.491 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-15T18:25:39.491 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.allow_fallocate: 1 2024-02-15T18:25:39.491 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-15T18:25:39.491 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-15T18:25:39.491 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.use_direct_reads: 0 2024-02-15T18:25:39.491 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-15T18:25:39.491 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.create_missing_column_families: 0 2024-02-15T18:25:39.492 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.db_log_dir: 2024-02-15T18:25:39.492 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi038/store.db 2024-02-15T18:25:39.492 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-15T18:25:39.492 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_subcompactions: 1 2024-02-15T18:25:39.492 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_background_flushes: -1 2024-02-15T18:25:39.492 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-15T18:25:39.492 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-15T18:25:39.492 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-15T18:25:39.492 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-15T18:25:39.492 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-15T18:25:39.492 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.advise_random_on_open: 1 2024-02-15T18:25:39.492 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-15T18:25:39.493 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.write_buffer_manager: 0x55ea47e1a030 2024-02-15T18:25:39.493 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-15T18:25:39.493 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-15T18:25:39.493 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-15T18:25:39.493 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-15T18:25:39.493 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.rate_limiter: (nil) 2024-02-15T18:25:39.493 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-15T18:25:39.493 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-15T18:25:39.493 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-15T18:25:39.493 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-15T18:25:39.493 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.unordered_write: 0 2024-02-15T18:25:39.493 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-15T18:25:39.495 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-15T18:25:39.495 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-15T18:25:39.495 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-15T18:25:39.495 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.row_cache: None 2024-02-15T18:25:39.495 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.wal_filter: None 2024-02-15T18:25:39.495 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-15T18:25:39.496 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-15T18:25:39.496 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.preserve_deletes: 0 2024-02-15T18:25:39.496 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.two_write_queues: 0 2024-02-15T18:25:39.496 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.manual_wal_flush: 0 2024-02-15T18:25:39.496 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.atomic_flush: 0 2024-02-15T18:25:39.496 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-15T18:25:39.496 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-15T18:25:39.496 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-15T18:25:39.496 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.log_readahead_size: 0 2024-02-15T18:25:39.496 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-15T18:25:39.496 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_background_jobs: 2 2024-02-15T18:25:39.496 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_background_compactions: -1 2024-02-15T18:25:39.496 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-15T18:25:39.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-15T18:25:39.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-15T18:25:39.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_total_wal_size: 0 2024-02-15T18:25:39.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-15T18:25:39.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-15T18:25:39.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-15T18:25:39.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-15T18:25:39.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_open_files: -1 2024-02-15T18:25:39.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.bytes_per_sync: 0 2024-02-15T18:25:39.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-15T18:25:39.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-15T18:25:39.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-15T18:25:39.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Compression algorithms supported: 2024-02-15T18:25:39.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-15T18:25:39.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: kZSTD supported: 0 2024-02-15T18:25:39.498 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: kXpressCompression supported: 0 2024-02-15T18:25:39.498 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-15T18:25:39.498 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: kLZ4Compression supported: 1 2024-02-15T18:25:39.498 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: kBZip2Compression supported: 0 2024-02-15T18:25:39.498 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: kZlibCompression supported: 1 2024-02-15T18:25:39.498 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: kSnappyCompression supported: 1 2024-02-15T18:25:39.498 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-15T18:25:39.498 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi038/store.db/MANIFEST-000005 2024-02-15T18:25:39.498 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.498 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-15T18:25:39.498 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.498 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-15T18:25:39.498 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.merge_operator: 2024-02-15T18:25:39.498 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.compaction_filter: None 2024-02-15T18:25:39.498 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.compaction_filter_factory: None 2024-02-15T18:25:39.499 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-15T18:25:39.499 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-15T18:25:39.499 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ea470bdcb0) 2024-02-15T18:25:39.499 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cache_index_and_filter_blocks: 1 2024-02-15T18:25:39.499 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-15T18:25:39.499 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-15T18:25:39.499 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: pin_top_level_index_and_filter: 1 2024-02-15T18:25:39.499 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: index_type: 0 2024-02-15T18:25:39.499 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: data_block_index_type: 0 2024-02-15T18:25:39.499 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: index_shortening: 1 2024-02-15T18:25:39.499 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: data_block_hash_table_util_ratio: 0.750000 2024-02-15T18:25:39.499 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: hash_index_allow_collision: 1 2024-02-15T18:25:39.499 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: checksum: 1 2024-02-15T18:25:39.499 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: no_block_cache: 0 2024-02-15T18:25:39.500 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: block_cache: 0x55ea470f2f10 2024-02-15T18:25:39.500 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: block_cache_name: BinnedLRUCache 2024-02-15T18:25:39.500 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: block_cache_options: 2024-02-15T18:25:39.500 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: capacity : 536870912 2024-02-15T18:25:39.500 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: num_shard_bits : 4 2024-02-15T18:25:39.500 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: strict_capacity_limit : 0 2024-02-15T18:25:39.500 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: high_pri_pool_ratio: 0.000 2024-02-15T18:25:39.500 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: block_cache_compressed: (nil) 2024-02-15T18:25:39.500 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: persistent_cache: (nil) 2024-02-15T18:25:39.500 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: block_size: 4096 2024-02-15T18:25:39.500 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: block_size_deviation: 10 2024-02-15T18:25:39.500 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: block_restart_interval: 16 2024-02-15T18:25:39.500 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: index_block_restart_interval: 1 2024-02-15T18:25:39.501 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: metadata_block_size: 4096 2024-02-15T18:25:39.501 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: partition_filters: 0 2024-02-15T18:25:39.501 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: use_delta_encoding: 1 2024-02-15T18:25:39.501 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-15T18:25:39.501 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: whole_key_filtering: 1 2024-02-15T18:25:39.501 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: verify_compression: 0 2024-02-15T18:25:39.501 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: read_amp_bytes_per_bit: 0 2024-02-15T18:25:39.501 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: format_version: 2 2024-02-15T18:25:39.501 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: enable_index_compression: 1 2024-02-15T18:25:39.501 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: block_align: 0 2024-02-15T18:25:39.501 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.501 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-15T18:25:39.501 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-15T18:25:39.501 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.compression: NoCompression 2024-02-15T18:25:39.501 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-15T18:25:39.501 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-15T18:25:39.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-15T18:25:39.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.num_levels: 7 2024-02-15T18:25:39.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-15T18:25:39.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-15T18:25:39.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-15T18:25:39.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-15T18:25:39.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-15T18:25:39.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-15T18:25:39.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-15T18:25:39.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-15T18:25:39.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-15T18:25:39.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-15T18:25:39.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.compression_opts.level: 32767 2024-02-15T18:25:39.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-15T18:25:39.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-15T18:25:39.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-15T18:25:39.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.compression_opts.enabled: false 2024-02-15T18:25:39.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-15T18:25:39.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-15T18:25:39.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-15T18:25:39.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-15T18:25:39.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-15T18:25:39.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-15T18:25:39.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-15T18:25:39.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-15T18:25:39.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-15T18:25:39.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-15T18:25:39.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-15T18:25:39.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-15T18:25:39.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-15T18:25:39.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-15T18:25:39.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-15T18:25:39.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-15T18:25:39.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-15T18:25:39.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.arena_block_size: 4194304 2024-02-15T18:25:39.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-15T18:25:39.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-15T18:25:39.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-15T18:25:39.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-15T18:25:39.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-15T18:25:39.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-15T18:25:39.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-15T18:25:39.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-15T18:25:39.505 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-15T18:25:39.505 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-15T18:25:39.505 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-15T18:25:39.505 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-15T18:25:39.505 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.178+0000 7f5ae7d94700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-15T18:25:39.505 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.179+0000 7f5ae7d94700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-15T18:25:39.505 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.179+0000 7f5ae7d94700 4 rocksdb: Options.table_properties_collectors: 2024-02-15T18:25:39.505 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.179+0000 7f5ae7d94700 4 rocksdb: Options.inplace_update_support: 0 2024-02-15T18:25:39.505 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.179+0000 7f5ae7d94700 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-15T18:25:39.505 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.179+0000 7f5ae7d94700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-15T18:25:39.505 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.179+0000 7f5ae7d94700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-15T18:25:39.505 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.179+0000 7f5ae7d94700 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-15T18:25:39.506 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.179+0000 7f5ae7d94700 4 rocksdb: Options.bloom_locality: 0 2024-02-15T18:25:39.506 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.179+0000 7f5ae7d94700 4 rocksdb: Options.max_successive_merges: 0 2024-02-15T18:25:39.506 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.179+0000 7f5ae7d94700 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-15T18:25:39.506 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.179+0000 7f5ae7d94700 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-15T18:25:39.506 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.179+0000 7f5ae7d94700 4 rocksdb: Options.force_consistency_checks: 0 2024-02-15T18:25:39.506 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.179+0000 7f5ae7d94700 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-15T18:25:39.506 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.179+0000 7f5ae7d94700 4 rocksdb: Options.ttl: 2592000 2024-02-15T18:25:39.506 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.179+0000 7f5ae7d94700 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-15T18:25:39.506 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.179+0000 7f5ae7d94700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi038/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 5, log_number is 4,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-15T18:25:39.506 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.506 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.179+0000 7f5ae7d94700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 4 2024-02-15T18:25:39.506 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.506 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.179+0000 7f5ae7d94700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708021539180384, "job": 1, "event": "recovery_started", "log_files": [6]} 2024-02-15T18:25:39.507 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.179+0000 7f5ae7d94700 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #6 mode 2 2024-02-15T18:25:39.507 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.180+0000 7f5ae7d94700 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-15T18:25:39.507 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.180+0000 7f5ae7d94700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708021539181472, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 7, "file_size": 94282, "table_properties": {"data_size": 92370, "index_size": 389, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 709, "raw_key_size": 12563, "raw_average_key_size": 49, "raw_value_size": 85401, "raw_average_value_size": 336, "num_data_blocks": 12, "num_entries": 254, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708021539, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-15T18:25:39.507 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.181+0000 7f5ae7d94700 4 rocksdb: [version_set.cc:3826] Creating manifest 8 2024-02-15T18:25:39.507 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.507 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.182+0000 7f5ae7d94700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708021539182509, "job": 1, "event": "recovery_finished"} 2024-02-15T18:25:39.507 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.185+0000 7f5ae7d94700 4 rocksdb: DB pointer 0x55ea471b7800 2024-02-15T18:25:39.507 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.185+0000 7f5acebac700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-15T18:25:39.507 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.185+0000 7f5acebac700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-15T18:25:39.507 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: ** DB Stats ** 2024-02-15T18:25:39.507 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:25:39.507 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-15T18:25:39.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-15T18:25:39.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-15T18:25:39.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-15T18:25:39.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-15T18:25:39.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-15T18:25:39.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: ** Compaction Stats [default] ** 2024-02-15T18:25:39.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:25:39.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:25:39.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: L0 2/0 93.76 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 124.9 0.00 0.00 1 0.001 0 0 2024-02-15T18:25:39.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Sum 2/0 93.76 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 124.9 0.00 0.00 1 0.001 0 0 2024-02-15T18:25:39.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 124.9 0.00 0.00 1 0.001 0 0 2024-02-15T18:25:39.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: ** Compaction Stats [default] ** 2024-02-15T18:25:39.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:25:39.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:25:39.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 124.9 0.00 0.00 1 0.001 0 0 2024-02-15T18:25:39.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:25:39.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-15T18:25:39.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-15T18:25:39.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: AddFile(Total Files): cumulative 0, interval 0 2024-02-15T18:25:39.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-15T18:25:39.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: AddFile(Keys): cumulative 0, interval 0 2024-02-15T18:25:39.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Cumulative compaction: 0.00 GB write, 13.97 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:25:39.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Interval compaction: 0.00 GB write, 13.97 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:25:39.510 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-15T18:25:39.510 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.510 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: ** File Read Latency Histogram By Level [default] ** 2024-02-15T18:25:39.510 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.510 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: ** Compaction Stats [default] ** 2024-02-15T18:25:39.510 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:25:39.510 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:25:39.510 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: L0 2/0 93.76 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 124.9 0.00 0.00 1 0.001 0 0 2024-02-15T18:25:39.510 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Sum 2/0 93.76 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 124.9 0.00 0.00 1 0.001 0 0 2024-02-15T18:25:39.510 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:25:39.510 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.510 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: ** Compaction Stats [default] ** 2024-02-15T18:25:39.511 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:25:39.511 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:25:39.511 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 124.9 0.00 0.00 1 0.001 0 0 2024-02-15T18:25:39.511 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:25:39.511 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-15T18:25:39.511 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-15T18:25:39.511 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: AddFile(Total Files): cumulative 0, interval 0 2024-02-15T18:25:39.511 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-15T18:25:39.511 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: AddFile(Keys): cumulative 0, interval 0 2024-02-15T18:25:39.511 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Cumulative compaction: 0.00 GB write, 13.83 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:25:39.511 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:25:39.511 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-15T18:25:39.511 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.511 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: ** File Read Latency Histogram By Level [default] ** 2024-02-15T18:25:39.512 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.512 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.186+0000 7f5ae7d94700 0 starting mon.smithi038 rank 0 at public addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] at bind addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi038 fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:25:39.512 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.187+0000 7f5ae7d94700 1 mon.smithi038@-1(???) e1 preinit fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:25:39.512 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.187+0000 7f5ae7d94700 0 mon.smithi038@-1(???).mds e1 new map 2024-02-15T18:25:39.512 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.187+0000 7f5ae7d94700 0 mon.smithi038@-1(???).mds e1 print_map 2024-02-15T18:25:39.512 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: e1 2024-02-15T18:25:39.512 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:25:39.512 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:25:39.512 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: legacy client fscid: -1 2024-02-15T18:25:39.512 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.512 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: No filesystems configured 2024-02-15T18:25:39.512 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: 2024-02-15T18:25:39.512 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.187+0000 7f5ae7d94700 0 mon.smithi038@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-15T18:25:39.512 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.187+0000 7f5ae7d94700 0 mon.smithi038@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-15T18:25:39.512 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.187+0000 7f5ae7d94700 0 mon.smithi038@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-15T18:25:39.513 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.187+0000 7f5ae7d94700 0 mon.smithi038@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-15T18:25:39.513 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:25:37.042046+0000 mon.smithi038 (mon.0) 10 : audit [DBG] from='client.? 172.21.15.38:0/1488605796' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:25:39.513 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:25:39.187+0000 7f5ae7d94700 1 mon.smithi038@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-15T18:25:39.513 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 bash[38122]: 01b30eaebcfe918f21c00b471bb91fed022ecde1cb168b6b936307d3a0a70e36 2024-02-15T18:25:39.513 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 systemd[1]: Started Ceph mon.smithi038 for 7e05753e-cc2f-11ee-95ba-87774f69a715. 2024-02-15T18:25:39.513 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:25:39.203301+0000 mon.smithi038 (mon.0) 1 : cluster [INF] mon.smithi038 is new leader, mons smithi038 in quorum (ranks 0) 2024-02-15T18:25:39.513 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:25:39.203347+0000 mon.smithi038 (mon.0) 2 : cluster [DBG] monmap e1: 1 mons at {smithi038=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0]} 2024-02-15T18:25:39.513 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:25:39.203378+0000 mon.smithi038 (mon.0) 3 : cluster [DBG] fsmap 2024-02-15T18:25:39.513 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:25:39.203404+0000 mon.smithi038 (mon.0) 4 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-02-15T18:25:39.513 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:25:39.203891+0000 mon.smithi038 (mon.0) 5 : cluster [DBG] mgrmap e1: no daemons active 2024-02-15T18:25:39.845 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.846+0000 7efd31550700 1 Processor -- start 2024-02-15T18:25:39.845 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.846+0000 7efd31550700 1 -- start start 2024-02-15T18:25:39.845 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.846+0000 7efd31550700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd2c0703d0 0x7efd2c100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:39.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.846+0000 7efd31550700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c0fc8a0 con 0x7efd2c0703d0 2024-02-15T18:25:39.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.847+0000 7efd2affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd2c0703d0 0x7efd2c100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:39.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.847+0000 7efd2affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd2c0703d0 0x7efd2c100eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47700/0 (socket says 172.21.15.38:47700) 2024-02-15T18:25:39.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.847+0000 7efd2affd700 1 -- 172.21.15.38:0/144487694 learned_addr learned my addr 172.21.15.38:0/144487694 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:25:39.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.847+0000 7efd2affd700 1 -- 172.21.15.38:0/144487694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2c1013a0 con 0x7efd2c0703d0 2024-02-15T18:25:39.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.847+0000 7efd2affd700 1 --2- 172.21.15.38:0/144487694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd2c0703d0 0x7efd2c100eb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7efd1c00b5f0 tx=0x7efd1c006ee0).ready entity=mon.0 client_cookie=93105840dfe5282f server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:39.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.848+0000 7efd29ffb700 1 -- 172.21.15.38:0/144487694 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd1c00d600 con 0x7efd2c0703d0 2024-02-15T18:25:39.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.848+0000 7efd29ffb700 1 -- 172.21.15.38:0/144487694 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7efd1c00dbf0 con 0x7efd2c0703d0 2024-02-15T18:25:39.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.848+0000 7efd29ffb700 1 -- 172.21.15.38:0/144487694 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd1c011cb0 con 0x7efd2c0703d0 2024-02-15T18:25:39.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.848+0000 7efd31550700 1 -- 172.21.15.38:0/144487694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd2c0703d0 msgr2=0x7efd2c100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:39.849 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.848+0000 7efd31550700 1 --2- 172.21.15.38:0/144487694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd2c0703d0 0x7efd2c100eb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7efd1c00b5f0 tx=0x7efd1c006ee0).stop 2024-02-15T18:25:39.849 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.848+0000 7efd31550700 1 -- 172.21.15.38:0/144487694 shutdown_connections 2024-02-15T18:25:39.849 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.848+0000 7efd31550700 1 --2- 172.21.15.38:0/144487694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd2c0703d0 0x7efd2c100eb0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:39.849 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.848+0000 7efd31550700 1 -- 172.21.15.38:0/144487694 >> 172.21.15.38:0/144487694 conn(0x7efd2c0f6530 msgr2=0x7efd2c0f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:39.849 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.848+0000 7efd31550700 1 -- 172.21.15.38:0/144487694 shutdown_connections 2024-02-15T18:25:39.849 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.848+0000 7efd31550700 1 -- 172.21.15.38:0/144487694 wait complete. 2024-02-15T18:25:39.849 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.849+0000 7efd31550700 1 Processor -- start 2024-02-15T18:25:39.849 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.849+0000 7efd31550700 1 -- start start 2024-02-15T18:25:39.849 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.849+0000 7efd31550700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd2c0703d0 0x7efd2c10ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.849+0000 7efd31550700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c10d330 con 0x7efd2c0703d0 2024-02-15T18:25:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.849+0000 7efd2affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd2c0703d0 0x7efd2c10ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.849+0000 7efd2affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd2c0703d0 0x7efd2c10ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47712/0 (socket says 172.21.15.38:47712) 2024-02-15T18:25:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.849+0000 7efd2affd700 1 -- 172.21.15.38:0/3769302343 learned_addr learned my addr 172.21.15.38:0/3769302343 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:25:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.849+0000 7efd2affd700 1 -- 172.21.15.38:0/3769302343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c00b200 con 0x7efd2c0703d0 2024-02-15T18:25:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.850+0000 7efd2affd700 1 --2- 172.21.15.38:0/3769302343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd2c0703d0 0x7efd2c10ce40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7efd1c006800 tx=0x7efd1c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.850+0000 7efd237fe700 1 -- 172.21.15.38:0/3769302343 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd1c01b570 con 0x7efd2c0703d0 2024-02-15T18:25:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.850+0000 7efd237fe700 1 -- 172.21.15.38:0/3769302343 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7efd1c01bb60 con 0x7efd2c0703d0 2024-02-15T18:25:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.850+0000 7efd31550700 1 -- 172.21.15.38:0/3769302343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd2c10d530 con 0x7efd2c0703d0 2024-02-15T18:25:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.850+0000 7efd237fe700 1 -- 172.21.15.38:0/3769302343 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd1c01a940 con 0x7efd2c0703d0 2024-02-15T18:25:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.850+0000 7efd31550700 1 -- 172.21.15.38:0/3769302343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd2c10e140 con 0x7efd2c0703d0 2024-02-15T18:25:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.850+0000 7efd237fe700 1 -- 172.21.15.38:0/3769302343 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7efd1c00f070 con 0x7efd2c0703d0 2024-02-15T18:25:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.850+0000 7efd237fe700 1 -- 172.21.15.38:0/3769302343 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7efd1c025de0 con 0x7efd2c0703d0 2024-02-15T18:25:39.850 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.852+0000 7efd31550700 1 -- 172.21.15.38:0/3769302343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd2c059100 con 0x7efd2c0703d0 2024-02-15T18:25:39.851 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.854+0000 7efd237fe700 1 -- 172.21.15.38:0/3769302343 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7efd1c016070 con 0x7efd2c0703d0 2024-02-15T18:25:39.918 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.921+0000 7efd31550700 1 -- 172.21.15.38:0/3769302343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7efd2c10ef70 con 0x7efd2c0703d0 2024-02-15T18:25:39.923 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.926+0000 7efd237fe700 1 -- 172.21.15.38:0/3769302343 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7efd1c025550 con 0x7efd2c0703d0 2024-02-15T18:25:39.924 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.927+0000 7efd31550700 1 -- 172.21.15.38:0/3769302343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd2c0703d0 msgr2=0x7efd2c10ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:39.925 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.927+0000 7efd31550700 1 --2- 172.21.15.38:0/3769302343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd2c0703d0 0x7efd2c10ce40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7efd1c006800 tx=0x7efd1c006ee0).stop 2024-02-15T18:25:39.925 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.927+0000 7efd31550700 1 -- 172.21.15.38:0/3769302343 shutdown_connections 2024-02-15T18:25:39.925 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.927+0000 7efd31550700 1 --2- 172.21.15.38:0/3769302343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd2c0703d0 0x7efd2c10ce40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:39.925 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.927+0000 7efd31550700 1 -- 172.21.15.38:0/3769302343 >> 172.21.15.38:0/3769302343 conn(0x7efd2c0f6530 msgr2=0x7efd2c0f7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:39.925 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.927+0000 7efd31550700 1 -- 172.21.15.38:0/3769302343 shutdown_connections 2024-02-15T18:25:39.925 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:39.927+0000 7efd31550700 1 -- 172.21.15.38:0/3769302343 wait complete. 2024-02-15T18:25:40.627 INFO:teuthology.orchestra.run.smithi038.stderr:Wrote config to /etc/ceph/ceph.conf 2024-02-15T18:25:40.627 INFO:teuthology.orchestra.run.smithi038.stderr:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-15T18:25:40.627 INFO:teuthology.orchestra.run.smithi038.stderr:Creating mgr... 2024-02-15T18:25:40.628 INFO:teuthology.orchestra.run.smithi038.stderr:Verifying port 9283 ... 2024-02-15T18:25:40.630 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl daemon-reload 2024-02-15T18:25:40.733 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl stop ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mgr.smithi038.khesbl 2024-02-15T18:25:40.760 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl reset-failed ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mgr.smithi038.khesbl 2024-02-15T18:25:40.766 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: Failed to reset failed state of unit ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mgr.smithi038.khesbl.service: Unit ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mgr.smithi038.khesbl.service not loaded. 2024-02-15T18:25:40.767 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl enable ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mgr.smithi038.khesbl 2024-02-15T18:25:40.773 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: Created symlink /etc/systemd/system/ceph-7e05753e-cc2f-11ee-95ba-87774f69a715.target.wants/ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mgr.smithi038.khesbl.service → /etc/systemd/system/ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@.service. 2024-02-15T18:25:40.861 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl start ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mgr.smithi038.khesbl 2024-02-15T18:25:41.183 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:25:39.926765+0000 mon.smithi038 (mon.0) 6 : audit [INF] from='client.? 172.21.15.38:0/3769302343' entity='client.admin' 2024-02-15T18:25:41.465 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl is-enabled firewalld.service 2024-02-15T18:25:41.474 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: disabled 2024-02-15T18:25:41.474 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl is-active firewalld.service 2024-02-15T18:25:41.482 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: inactive 2024-02-15T18:25:41.483 INFO:teuthology.orchestra.run.smithi038.stderr:firewalld.service is not enabled 2024-02-15T18:25:41.483 INFO:teuthology.orchestra.run.smithi038.stderr:Not possible to enable service . firewalld.service is not available 2024-02-15T18:25:41.483 INFO:teuthology.orchestra.run.smithi038.stderr:Not possible to open ports <[]>. firewalld.service is not available 2024-02-15T18:25:41.483 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl is-enabled firewalld.service 2024-02-15T18:25:41.490 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: disabled 2024-02-15T18:25:41.491 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl is-active firewalld.service 2024-02-15T18:25:41.499 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: inactive 2024-02-15T18:25:41.500 INFO:teuthology.orchestra.run.smithi038.stderr:firewalld.service is not enabled 2024-02-15T18:25:41.500 INFO:teuthology.orchestra.run.smithi038.stderr:Not possible to open ports <[9283]>. firewalld.service is not available 2024-02-15T18:25:41.500 INFO:teuthology.orchestra.run.smithi038.stderr:Waiting for mgr to start... 2024-02-15T18:25:41.500 INFO:teuthology.orchestra.run.smithi038.stderr:Waiting for mgr... 2024-02-15T18:25:41.500 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 status -f json-pretty 2024-02-15T18:25:42.127 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.120+0000 7ff3e8282700 1 Processor -- start 2024-02-15T18:25:42.127 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.120+0000 7ff3e8282700 1 -- start start 2024-02-15T18:25:42.127 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.120+0000 7ff3e8282700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3e00724b0 0x7ff3e0070a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:42.127 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.120+0000 7ff3e8282700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e00728d0 con 0x7ff3e00724b0 2024-02-15T18:25:42.127 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.121+0000 7ff3e601e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3e00724b0 0x7ff3e0070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:42.127 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.121+0000 7ff3e601e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3e00724b0 0x7ff3e0070a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47738/0 (socket says 172.21.15.38:47738) 2024-02-15T18:25:42.127 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.121+0000 7ff3e601e700 1 -- 172.21.15.38:0/580515570 learned_addr learned my addr 172.21.15.38:0/580515570 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:25:42.127 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.121+0000 7ff3e601e700 1 -- 172.21.15.38:0/580515570 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e0070f40 con 0x7ff3e00724b0 2024-02-15T18:25:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.121+0000 7ff3e601e700 1 --2- 172.21.15.38:0/580515570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3e00724b0 0x7ff3e0070a50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7ff3dc00b5f0 tx=0x7ff3dc006ee0).ready entity=mon.0 client_cookie=f45bf37ab0301f58 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.121+0000 7ff3e501c700 1 -- 172.21.15.38:0/580515570 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3dc00d600 con 0x7ff3e00724b0 2024-02-15T18:25:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.121+0000 7ff3e501c700 1 -- 172.21.15.38:0/580515570 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7ff3dc00dbf0 con 0x7ff3e00724b0 2024-02-15T18:25:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.121+0000 7ff3e501c700 1 -- 172.21.15.38:0/580515570 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3dc011cb0 con 0x7ff3e00724b0 2024-02-15T18:25:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.122+0000 7ff3e8282700 1 -- 172.21.15.38:0/580515570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3e00724b0 msgr2=0x7ff3e0070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.122+0000 7ff3e8282700 1 --2- 172.21.15.38:0/580515570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3e00724b0 0x7ff3e0070a50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7ff3dc00b5f0 tx=0x7ff3dc006ee0).stop 2024-02-15T18:25:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.122+0000 7ff3e8282700 1 -- 172.21.15.38:0/580515570 shutdown_connections 2024-02-15T18:25:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.122+0000 7ff3e8282700 1 --2- 172.21.15.38:0/580515570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3e00724b0 0x7ff3e0070a50 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.122+0000 7ff3e8282700 1 -- 172.21.15.38:0/580515570 >> 172.21.15.38:0/580515570 conn(0x7ff3e00f7640 msgr2=0x7ff3e00f9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.122+0000 7ff3e8282700 1 -- 172.21.15.38:0/580515570 shutdown_connections 2024-02-15T18:25:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.122+0000 7ff3e8282700 1 -- 172.21.15.38:0/580515570 wait complete. 2024-02-15T18:25:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.122+0000 7ff3e8282700 1 Processor -- start 2024-02-15T18:25:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.122+0000 7ff3e8282700 1 -- start start 2024-02-15T18:25:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.122+0000 7ff3e8282700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3e00724b0 0x7ff3e010cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:42.128 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.122+0000 7ff3e8282700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e010d2a0 con 0x7ff3e00724b0 2024-02-15T18:25:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.123+0000 7ff3e601e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3e00724b0 0x7ff3e010cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.123+0000 7ff3e601e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3e00724b0 0x7ff3e010cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47752/0 (socket says 172.21.15.38:47752) 2024-02-15T18:25:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.123+0000 7ff3e601e700 1 -- 172.21.15.38:0/3860902279 learned_addr learned my addr 172.21.15.38:0/3860902279 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:25:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.123+0000 7ff3e601e700 1 -- 172.21.15.38:0/3860902279 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc00b200 con 0x7ff3e00724b0 2024-02-15T18:25:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.123+0000 7ff3e601e700 1 --2- 172.21.15.38:0/3860902279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3e00724b0 0x7ff3e010cdb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7ff3dc00df90 tx=0x7ff3dc006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.123+0000 7ff3ceffd700 1 -- 172.21.15.38:0/3860902279 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3dc01b410 con 0x7ff3e00724b0 2024-02-15T18:25:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.123+0000 7ff3e8282700 1 -- 172.21.15.38:0/3860902279 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e010d4a0 con 0x7ff3e00724b0 2024-02-15T18:25:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.123+0000 7ff3ceffd700 1 -- 172.21.15.38:0/3860902279 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7ff3dc01ba00 con 0x7ff3e00724b0 2024-02-15T18:25:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.123+0000 7ff3ceffd700 1 -- 172.21.15.38:0/3860902279 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3dc01aa30 con 0x7ff3e00724b0 2024-02-15T18:25:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.123+0000 7ff3e8282700 1 -- 172.21.15.38:0/3860902279 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e010e0e0 con 0x7ff3e00724b0 2024-02-15T18:25:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.123+0000 7ff3ceffd700 1 -- 172.21.15.38:0/3860902279 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7ff3dc00f070 con 0x7ff3e00724b0 2024-02-15T18:25:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.124+0000 7ff3ceffd700 1 -- 172.21.15.38:0/3860902279 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7ff3dc025c80 con 0x7ff3e00724b0 2024-02-15T18:25:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.124+0000 7ff3e8282700 1 -- 172.21.15.38:0/3860902279 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e010d930 con 0x7ff3e00724b0 2024-02-15T18:25:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.127+0000 7ff3ceffd700 1 -- 172.21.15.38:0/3860902279 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7ff3dc016070 con 0x7ff3e00724b0 2024-02-15T18:25:42.200 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.197+0000 7ff3e8282700 1 -- 172.21.15.38:0/3860902279 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff3e01020d0 con 0x7ff3e00724b0 2024-02-15T18:25:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.197+0000 7ff3ceffd700 1 -- 172.21.15.38:0/3860902279 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1230 (secure 0 0 0) 0x7ff3dc025460 con 0x7ff3e00724b0 2024-02-15T18:25:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: { 2024-02-15T18:25:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "fsid": "7e05753e-cc2f-11ee-95ba-87774f69a715", 2024-02-15T18:25:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "health": { 2024-02-15T18:25:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-15T18:25:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "checks": {}, 2024-02-15T18:25:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "mutes": [] 2024-02-15T18:25:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-15T18:25:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "quorum": [ 2024-02-15T18:25:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 0 2024-02-15T18:25:42.201 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: ], 2024-02-15T18:25:42.202 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-15T18:25:42.202 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "smithi038" 2024-02-15T18:25:42.202 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: ], 2024-02-15T18:25:42.202 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "quorum_age": 2, 2024-02-15T18:25:42.202 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "monmap": { 2024-02-15T18:25:42.202 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:25:42.202 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-15T18:25:42.202 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-15T18:25:42.202 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:42.202 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "osdmap": { 2024-02-15T18:25:42.202 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:25:42.202 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-15T18:25:42.202 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-15T18:25:42.202 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-15T18:25:42.203 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-15T18:25:42.203 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-15T18:25:42.203 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-15T18:25:42.203 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:42.203 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "pgmap": { 2024-02-15T18:25:42.203 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-15T18:25:42.203 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-15T18:25:42.203 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-15T18:25:42.203 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-15T18:25:42.203 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-15T18:25:42.203 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-15T18:25:42.203 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-15T18:25:42.203 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-15T18:25:42.203 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:42.203 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "fsmap": { 2024-02-15T18:25:42.204 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:25:42.204 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-15T18:25:42.204 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-15T18:25:42.204 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:42.204 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-15T18:25:42.204 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "available": false, 2024-02-15T18:25:42.204 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-15T18:25:42.204 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "modules": [ 2024-02-15T18:25:42.204 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "iostat", 2024-02-15T18:25:42.204 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "restful" 2024-02-15T18:25:42.204 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: ], 2024-02-15T18:25:42.204 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "services": {} 2024-02-15T18:25:42.204 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:42.205 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "servicemap": { 2024-02-15T18:25:42.205 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:25:42.205 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "modified": "2024-02-15T18:25:33.850022+0000", 2024-02-15T18:25:42.205 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "services": {} 2024-02-15T18:25:42.205 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:42.205 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-15T18:25:42.205 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: } 2024-02-15T18:25:42.205 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.198+0000 7ff3e8282700 1 -- 172.21.15.38:0/3860902279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3e00724b0 msgr2=0x7ff3e010cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:42.205 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.198+0000 7ff3e8282700 1 --2- 172.21.15.38:0/3860902279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3e00724b0 0x7ff3e010cdb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7ff3dc00df90 tx=0x7ff3dc006ee0).stop 2024-02-15T18:25:42.205 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.198+0000 7ff3e8282700 1 -- 172.21.15.38:0/3860902279 shutdown_connections 2024-02-15T18:25:42.205 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.198+0000 7ff3e8282700 1 --2- 172.21.15.38:0/3860902279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3e00724b0 0x7ff3e010cdb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:42.205 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.198+0000 7ff3e8282700 1 -- 172.21.15.38:0/3860902279 >> 172.21.15.38:0/3860902279 conn(0x7ff3e00f7640 msgr2=0x7ff3e00f9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:42.205 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.198+0000 7ff3e8282700 1 -- 172.21.15.38:0/3860902279 shutdown_connections 2024-02-15T18:25:42.205 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:42.198+0000 7ff3e8282700 1 -- 172.21.15.38:0/3860902279 wait complete. 2024-02-15T18:25:42.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:25:42.198006+0000 mon.smithi038 (mon.0) 7 : audit [DBG] from='client.? 172.21.15.38:0/3860902279' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-15T18:25:42.879 INFO:teuthology.orchestra.run.smithi038.stderr:mgr not available, waiting (1/15)... 2024-02-15T18:25:44.882 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 status -f json-pretty 2024-02-15T18:25:47.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:25:46.483410+0000 mon.smithi038 (mon.0) 8 : cluster [INF] Activating manager daemon smithi038.khesbl 2024-02-15T18:25:47.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:25:46.488309+0000 mon.smithi038 (mon.0) 9 : cluster [DBG] mgrmap e2: smithi038.khesbl(active, starting, since 0.00501172s) 2024-02-15T18:25:47.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:25:46.489359+0000 mon.smithi038 (mon.0) 10 : audit [DBG] from='mgr.14100 172.21.15.38:0/897478749' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-15T18:25:47.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:25:46.489500+0000 mon.smithi038 (mon.0) 11 : audit [DBG] from='mgr.14100 172.21.15.38:0/897478749' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-15T18:25:47.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:25:46.489624+0000 mon.smithi038 (mon.0) 12 : audit [DBG] from='mgr.14100 172.21.15.38:0/897478749' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-15T18:25:47.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:25:46.489738+0000 mon.smithi038 (mon.0) 13 : audit [DBG] from='mgr.14100 172.21.15.38:0/897478749' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-02-15T18:25:47.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:25:46.489991+0000 mon.smithi038 (mon.0) 14 : audit [DBG] from='mgr.14100 172.21.15.38:0/897478749' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mgr metadata", "who": "smithi038.khesbl", "id": "smithi038.khesbl"}]: dispatch 2024-02-15T18:25:47.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:25:46.504072+0000 mon.smithi038 (mon.0) 15 : cluster [INF] Manager daemon smithi038.khesbl is now available 2024-02-15T18:25:47.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:25:46.511732+0000 mon.smithi038 (mon.0) 16 : audit [INF] from='mgr.14100 172.21.15.38:0/897478749' entity='mgr.smithi038.khesbl' 2024-02-15T18:25:47.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:25:46.516930+0000 mon.smithi038 (mon.0) 17 : audit [INF] from='mgr.14100 172.21.15.38:0/897478749' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:25:47.031 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:25:46.518082+0000 mon.smithi038 (mon.0) 18 : audit [INF] from='mgr.14100 172.21.15.38:0/897478749' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:25:47.031 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:25:46.523468+0000 mon.smithi038 (mon.0) 19 : audit [INF] from='mgr.14100 172.21.15.38:0/897478749' entity='mgr.smithi038.khesbl' 2024-02-15T18:25:47.031 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:25:46.525373+0000 mon.smithi038 (mon.0) 20 : audit [INF] from='mgr.14100 172.21.15.38:0/897478749' entity='mgr.smithi038.khesbl' 2024-02-15T18:25:47.780 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:25:47.319446+0000 mon.smithi038 (mon.0) 21 : audit [DBG] from='client.? 172.21.15.38:0/1900859960' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-15T18:25:47.780 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:25:47.491812+0000 mon.smithi038 (mon.0) 22 : cluster [DBG] mgrmap e3: smithi038.khesbl(active, since 1.00851s) 2024-02-15T18:25:48.178 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.243+0000 7f40a5f7d700 1 Processor -- start 2024-02-15T18:25:48.178 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.243+0000 7f40a5f7d700 1 -- start start 2024-02-15T18:25:48.178 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.243+0000 7f40a5f7d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40a0070370 0x7f40a00fe950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:48.178 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.243+0000 7f40a5f7d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a0070790 con 0x7f40a0070370 2024-02-15T18:25:48.179 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.243+0000 7f40a4f7b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40a0070370 0x7f40a00fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:48.179 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.243+0000 7f40a4f7b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40a0070370 0x7f40a00fe950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44900/0 (socket says 172.21.15.38:44900) 2024-02-15T18:25:48.179 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.243+0000 7f40a4f7b700 1 -- 172.21.15.38:0/1645534208 learned_addr learned my addr 172.21.15.38:0/1645534208 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:25:48.179 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.244+0000 7f40a4f7b700 1 -- 172.21.15.38:0/1645534208 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a00fee40 con 0x7f40a0070370 2024-02-15T18:25:48.179 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.244+0000 7f40a4f7b700 1 --2- 172.21.15.38:0/1645534208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40a0070370 0x7f40a00fe950 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f409400b5f0 tx=0x7f4094006ee0).ready entity=mon.0 client_cookie=e1cb1756b1a9bc3f server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:48.179 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.244+0000 7f409f7fe700 1 -- 172.21.15.38:0/1645534208 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f409400d600 con 0x7f40a0070370 2024-02-15T18:25:48.179 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.244+0000 7f409f7fe700 1 -- 172.21.15.38:0/1645534208 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f409400dbf0 con 0x7f40a0070370 2024-02-15T18:25:48.179 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.244+0000 7f409f7fe700 1 -- 172.21.15.38:0/1645534208 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4094011cb0 con 0x7f40a0070370 2024-02-15T18:25:48.179 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.244+0000 7f40a5f7d700 1 -- 172.21.15.38:0/1645534208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40a0070370 msgr2=0x7f40a00fe950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:48.179 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.244+0000 7f40a5f7d700 1 --2- 172.21.15.38:0/1645534208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40a0070370 0x7f40a00fe950 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f409400b5f0 tx=0x7f4094006ee0).stop 2024-02-15T18:25:48.179 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.245+0000 7f40a5f7d700 1 -- 172.21.15.38:0/1645534208 shutdown_connections 2024-02-15T18:25:48.179 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.245+0000 7f40a5f7d700 1 --2- 172.21.15.38:0/1645534208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40a0070370 0x7f40a00fe950 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:48.179 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.245+0000 7f40a5f7d700 1 -- 172.21.15.38:0/1645534208 >> 172.21.15.38:0/1645534208 conn(0x7f40a00f6390 msgr2=0x7f40a00f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:48.179 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.245+0000 7f40a5f7d700 1 -- 172.21.15.38:0/1645534208 shutdown_connections 2024-02-15T18:25:48.180 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.245+0000 7f40a5f7d700 1 -- 172.21.15.38:0/1645534208 wait complete. 2024-02-15T18:25:48.180 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.245+0000 7f40a5f7d700 1 Processor -- start 2024-02-15T18:25:48.180 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.245+0000 7f40a5f7d700 1 -- start start 2024-02-15T18:25:48.180 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.246+0000 7f40a5f7d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40a0070370 0x7f40a010a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:48.180 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.246+0000 7f40a5f7d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a010ae90 con 0x7f40a0070370 2024-02-15T18:25:48.180 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.246+0000 7f40a4f7b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40a0070370 0x7f40a010a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:48.180 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.246+0000 7f40a4f7b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40a0070370 0x7f40a010a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44912/0 (socket says 172.21.15.38:44912) 2024-02-15T18:25:48.180 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.246+0000 7f40a4f7b700 1 -- 172.21.15.38:0/1900859960 learned_addr learned my addr 172.21.15.38:0/1900859960 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:25:48.180 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.246+0000 7f40a4f7b700 1 -- 172.21.15.38:0/1900859960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409400b200 con 0x7f40a0070370 2024-02-15T18:25:48.180 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.246+0000 7f40a4f7b700 1 --2- 172.21.15.38:0/1900859960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40a0070370 0x7f40a010a9a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f409400df90 tx=0x7f4094006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:48.180 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.246+0000 7f409d7fa700 1 -- 172.21.15.38:0/1900859960 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f409401b410 con 0x7f40a0070370 2024-02-15T18:25:48.180 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.246+0000 7f409d7fa700 1 -- 172.21.15.38:0/1900859960 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f409401ba00 con 0x7f40a0070370 2024-02-15T18:25:48.180 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.246+0000 7f40a5f7d700 1 -- 172.21.15.38:0/1900859960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a010b090 con 0x7f40a0070370 2024-02-15T18:25:48.180 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.246+0000 7f409d7fa700 1 -- 172.21.15.38:0/1900859960 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f409401aa30 con 0x7f40a0070370 2024-02-15T18:25:48.181 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.246+0000 7f40a5f7d700 1 -- 172.21.15.38:0/1900859960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a010b520 con 0x7f40a0070370 2024-02-15T18:25:48.181 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.247+0000 7f409d7fa700 1 -- 172.21.15.38:0/1900859960 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 2) v1 ==== 35959+0+0 (secure 0 0 0) 0x7f409400f070 con 0x7f40a0070370 2024-02-15T18:25:48.181 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.247+0000 7f409d7fa700 1 -- 172.21.15.38:0/1900859960 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f4094040fb0 con 0x7f40a0070370 2024-02-15T18:25:48.181 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.247+0000 7f40a5f7d700 1 -- 172.21.15.38:0/1900859960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a0045ad0 con 0x7f40a0070370 2024-02-15T18:25:48.181 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.249+0000 7f409d7fa700 1 -- 172.21.15.38:0/1900859960 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7f4094025460 con 0x7f40a0070370 2024-02-15T18:25:48.181 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.318+0000 7f40a5f7d700 1 -- 172.21.15.38:0/1900859960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f40a0104960 con 0x7f40a0070370 2024-02-15T18:25:48.181 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.319+0000 7f409d7fa700 1 -- 172.21.15.38:0/1900859960 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1230 (secure 0 0 0) 0x7f4094022310 con 0x7f40a0070370 2024-02-15T18:25:48.181 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.319+0000 7f40a5f7d700 1 -- 172.21.15.38:0/1900859960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40a0070370 msgr2=0x7f40a010a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:48.181 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.319+0000 7f40a5f7d700 1 --2- 172.21.15.38:0/1900859960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40a0070370 0x7f40a010a9a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f409400df90 tx=0x7f4094006ee0).stop 2024-02-15T18:25:48.181 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.319+0000 7f40a5f7d700 1 -- 172.21.15.38:0/1900859960 shutdown_connections 2024-02-15T18:25:48.181 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.319+0000 7f40a5f7d700 1 --2- 172.21.15.38:0/1900859960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40a0070370 0x7f40a010a9a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:48.181 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.319+0000 7f40a5f7d700 1 -- 172.21.15.38:0/1900859960 >> 172.21.15.38:0/1900859960 conn(0x7f40a00f6390 msgr2=0x7f40a00f7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:48.181 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.319+0000 7f40a5f7d700 1 -- 172.21.15.38:0/1900859960 shutdown_connections 2024-02-15T18:25:48.181 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:47.319+0000 7f40a5f7d700 1 -- 172.21.15.38:0/1900859960 wait complete. 2024-02-15T18:25:48.181 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:48.182 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: { 2024-02-15T18:25:48.182 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "fsid": "7e05753e-cc2f-11ee-95ba-87774f69a715", 2024-02-15T18:25:48.182 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "health": { 2024-02-15T18:25:48.182 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-15T18:25:48.182 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "checks": {}, 2024-02-15T18:25:48.182 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "mutes": [] 2024-02-15T18:25:48.182 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:48.182 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-15T18:25:48.182 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "quorum": [ 2024-02-15T18:25:48.182 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 0 2024-02-15T18:25:48.182 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: ], 2024-02-15T18:25:48.182 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-15T18:25:48.182 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "smithi038" 2024-02-15T18:25:48.182 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: ], 2024-02-15T18:25:48.182 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "quorum_age": 8, 2024-02-15T18:25:48.183 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "monmap": { 2024-02-15T18:25:48.183 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:25:48.183 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-15T18:25:48.183 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-15T18:25:48.183 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:48.183 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "osdmap": { 2024-02-15T18:25:48.183 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:25:48.183 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-15T18:25:48.183 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-15T18:25:48.183 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-15T18:25:48.183 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-15T18:25:48.183 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-15T18:25:48.183 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-15T18:25:48.183 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:48.183 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "pgmap": { 2024-02-15T18:25:48.183 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-15T18:25:48.184 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-15T18:25:48.184 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-15T18:25:48.184 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-15T18:25:48.184 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-15T18:25:48.184 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-15T18:25:48.184 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-15T18:25:48.184 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-15T18:25:48.184 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:48.184 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "fsmap": { 2024-02-15T18:25:48.184 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:25:48.184 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-15T18:25:48.184 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-15T18:25:48.184 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:48.184 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-15T18:25:48.185 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "available": false, 2024-02-15T18:25:48.185 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-15T18:25:48.185 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "modules": [ 2024-02-15T18:25:48.185 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "iostat", 2024-02-15T18:25:48.185 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "restful" 2024-02-15T18:25:48.185 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: ], 2024-02-15T18:25:48.185 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "services": {} 2024-02-15T18:25:48.185 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:48.185 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "servicemap": { 2024-02-15T18:25:48.185 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:25:48.185 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "modified": "2024-02-15T18:25:33.850022+0000", 2024-02-15T18:25:48.185 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "services": {} 2024-02-15T18:25:48.185 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:48.185 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-15T18:25:48.185 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: } 2024-02-15T18:25:50.030 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:25:48.539095+0000 mon.smithi038 (mon.0) 23 : cluster [DBG] mgrmap e4: smithi038.khesbl(active, since 2s) 2024-02-15T18:25:53.353 INFO:teuthology.orchestra.run.smithi038.stderr:mgr not available, waiting (2/15)... 2024-02-15T18:25:55.356 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 status -f json-pretty 2024-02-15T18:25:55.886 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.884+0000 7f956f59e700 1 Processor -- start 2024-02-15T18:25:55.886 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.884+0000 7f956f59e700 1 -- start start 2024-02-15T18:25:55.886 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.884+0000 7f956f59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9570070430 0x7f95700fe890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:55.886 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.884+0000 7f956f59e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95700fed80 con 0x7f9570070430 2024-02-15T18:25:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.884+0000 7f956e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9570070430 0x7f95700fe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.885+0000 7f956e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9570070430 0x7f95700fe890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43456/0 (socket says 172.21.15.38:43456) 2024-02-15T18:25:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.885+0000 7f956e59c700 1 -- 172.21.15.38:0/1546862312 learned_addr learned my addr 172.21.15.38:0/1546862312 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:25:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.885+0000 7f956e59c700 1 -- 172.21.15.38:0/1546862312 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95700feec0 con 0x7f9570070430 2024-02-15T18:25:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.885+0000 7f956e59c700 1 --2- 172.21.15.38:0/1546862312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9570070430 0x7f95700fe890 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f956000b5f0 tx=0x7f9560006ee0).ready entity=mon.0 client_cookie=413bf7968b08de84 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.885+0000 7f956d59a700 1 -- 172.21.15.38:0/1546862312 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f956000d600 con 0x7f9570070430 2024-02-15T18:25:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.885+0000 7f956d59a700 1 -- 172.21.15.38:0/1546862312 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f956000dbf0 con 0x7f9570070430 2024-02-15T18:25:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.885+0000 7f956d59a700 1 -- 172.21.15.38:0/1546862312 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9560011cb0 con 0x7f9570070430 2024-02-15T18:25:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.886+0000 7f956f59e700 1 -- 172.21.15.38:0/1546862312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9570070430 msgr2=0x7f95700fe890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.886+0000 7f956f59e700 1 --2- 172.21.15.38:0/1546862312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9570070430 0x7f95700fe890 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f956000b5f0 tx=0x7f9560006ee0).stop 2024-02-15T18:25:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.886+0000 7f956f59e700 1 -- 172.21.15.38:0/1546862312 shutdown_connections 2024-02-15T18:25:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.886+0000 7f956f59e700 1 --2- 172.21.15.38:0/1546862312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9570070430 0x7f95700fe890 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.886+0000 7f956f59e700 1 -- 172.21.15.38:0/1546862312 >> 172.21.15.38:0/1546862312 conn(0x7f95700f6390 msgr2=0x7f95700f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:55.888 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.886+0000 7f956f59e700 1 -- 172.21.15.38:0/1546862312 shutdown_connections 2024-02-15T18:25:55.888 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.886+0000 7f956f59e700 1 -- 172.21.15.38:0/1546862312 wait complete. 2024-02-15T18:25:55.888 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.886+0000 7f956f59e700 1 Processor -- start 2024-02-15T18:25:55.888 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.886+0000 7f956f59e700 1 -- start start 2024-02-15T18:25:55.888 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.887+0000 7f956f59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9570070430 0x7f957010a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:55.888 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.887+0000 7f956f59e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957010ae80 con 0x7f9570070430 2024-02-15T18:25:55.888 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.887+0000 7f956e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9570070430 0x7f957010a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:55.888 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.887+0000 7f956e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9570070430 0x7f957010a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43458/0 (socket says 172.21.15.38:43458) 2024-02-15T18:25:55.888 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.887+0000 7f956e59c700 1 -- 172.21.15.38:0/2496449003 learned_addr learned my addr 172.21.15.38:0/2496449003 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:25:55.888 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.887+0000 7f956e59c700 1 -- 172.21.15.38:0/2496449003 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956000b200 con 0x7f9570070430 2024-02-15T18:25:55.888 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.887+0000 7f956e59c700 1 --2- 172.21.15.38:0/2496449003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9570070430 0x7f957010a990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f9560006800 tx=0x7f9560006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:55.889 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.887+0000 7f955effd700 1 -- 172.21.15.38:0/2496449003 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f956001b570 con 0x7f9570070430 2024-02-15T18:25:55.889 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.887+0000 7f955effd700 1 -- 172.21.15.38:0/2496449003 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f956001bb60 con 0x7f9570070430 2024-02-15T18:25:55.889 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.887+0000 7f956f59e700 1 -- 172.21.15.38:0/2496449003 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957010b080 con 0x7f9570070430 2024-02-15T18:25:55.889 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.887+0000 7f955effd700 1 -- 172.21.15.38:0/2496449003 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f956001a9f0 con 0x7f9570070430 2024-02-15T18:25:55.889 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.887+0000 7f956f59e700 1 -- 172.21.15.38:0/2496449003 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957010b510 con 0x7f9570070430 2024-02-15T18:25:55.889 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.888+0000 7f955effd700 1 -- 172.21.15.38:0/2496449003 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 36109+0+0 (secure 0 0 0) 0x7f956000f070 con 0x7f9570070430 2024-02-15T18:25:55.889 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.888+0000 7f956f59e700 1 -- 172.21.15.38:0/2496449003 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9570045ad0 con 0x7f9570070430 2024-02-15T18:25:55.890 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.888+0000 7f955effd700 1 --2- 172.21.15.38:0/2496449003 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7f955402fe10 0x7f95540322d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:55.890 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.888+0000 7f955effd700 1 -- 172.21.15.38:0/2496449003 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f95600412a0 con 0x7f9570070430 2024-02-15T18:25:55.890 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.889+0000 7f956dd9b700 1 --2- 172.21.15.38:0/2496449003 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7f955402fe10 0x7f95540322d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:55.890 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.889+0000 7f956dd9b700 1 --2- 172.21.15.38:0/2496449003 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7f955402fe10 0x7f95540322d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f95640096a0 tx=0x7f9564006b40).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:55.891 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:55.893+0000 7f955effd700 1 -- 172.21.15.38:0/2496449003 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9560019cd0 con 0x7f9570070430 2024-02-15T18:25:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:56.108+0000 7f956f59e700 1 -- 172.21.15.38:0/2496449003 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f95700ffc00 con 0x7f9570070430 2024-02-15T18:25:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:56.109+0000 7f955effd700 1 -- 172.21.15.38:0/2496449003 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1230 (secure 0 0 0) 0x7f9560016070 con 0x7f9570070430 2024-02-15T18:25:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:56.108 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: { 2024-02-15T18:25:56.108 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "fsid": "7e05753e-cc2f-11ee-95ba-87774f69a715", 2024-02-15T18:25:56.108 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "health": { 2024-02-15T18:25:56.108 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-15T18:25:56.108 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "checks": {}, 2024-02-15T18:25:56.108 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "mutes": [] 2024-02-15T18:25:56.108 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:56.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-15T18:25:56.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "quorum": [ 2024-02-15T18:25:56.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 0 2024-02-15T18:25:56.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: ], 2024-02-15T18:25:56.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-15T18:25:56.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "smithi038" 2024-02-15T18:25:56.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: ], 2024-02-15T18:25:56.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "quorum_age": 16, 2024-02-15T18:25:56.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "monmap": { 2024-02-15T18:25:56.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:25:56.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-15T18:25:56.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-15T18:25:56.115 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:56.115 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "osdmap": { 2024-02-15T18:25:56.115 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:25:56.115 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-15T18:25:56.115 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-15T18:25:56.115 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-15T18:25:56.115 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-15T18:25:56.115 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-15T18:25:56.115 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-15T18:25:56.115 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:56.115 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "pgmap": { 2024-02-15T18:25:56.115 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-15T18:25:56.115 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-15T18:25:56.115 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-15T18:25:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-15T18:25:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-15T18:25:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-15T18:25:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-15T18:25:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-15T18:25:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "fsmap": { 2024-02-15T18:25:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:25:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-15T18:25:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-15T18:25:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-15T18:25:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "available": true, 2024-02-15T18:25:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-15T18:25:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "modules": [ 2024-02-15T18:25:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "iostat", 2024-02-15T18:25:56.117 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "restful" 2024-02-15T18:25:56.117 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: ], 2024-02-15T18:25:56.117 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "services": {} 2024-02-15T18:25:56.117 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:56.117 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "servicemap": { 2024-02-15T18:25:56.117 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:25:56.117 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "modified": "2024-02-15T18:25:33.850022+0000", 2024-02-15T18:25:56.117 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "services": {} 2024-02-15T18:25:56.117 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: }, 2024-02-15T18:25:56.117 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-15T18:25:56.117 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: } 2024-02-15T18:25:56.117 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:56.110+0000 7f956f59e700 1 -- 172.21.15.38:0/2496449003 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7f955402fe10 msgr2=0x7f95540322d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:56.117 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:56.110+0000 7f956f59e700 1 --2- 172.21.15.38:0/2496449003 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7f955402fe10 0x7f95540322d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f95640096a0 tx=0x7f9564006b40).stop 2024-02-15T18:25:56.117 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:56.110+0000 7f956f59e700 1 -- 172.21.15.38:0/2496449003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9570070430 msgr2=0x7f957010a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:56.117 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:56.110+0000 7f956f59e700 1 --2- 172.21.15.38:0/2496449003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9570070430 0x7f957010a990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f9560006800 tx=0x7f9560006ee0).stop 2024-02-15T18:25:56.117 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:56.111+0000 7f956f59e700 1 -- 172.21.15.38:0/2496449003 shutdown_connections 2024-02-15T18:25:56.118 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:56.111+0000 7f956f59e700 1 --2- 172.21.15.38:0/2496449003 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7f955402fe10 0x7f95540322d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:56.118 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:56.111+0000 7f956f59e700 1 --2- 172.21.15.38:0/2496449003 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9570070430 0x7f957010a990 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:56.118 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:56.111+0000 7f956f59e700 1 -- 172.21.15.38:0/2496449003 >> 172.21.15.38:0/2496449003 conn(0x7f95700f6390 msgr2=0x7f95700f7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:56.118 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:56.111+0000 7f956f59e700 1 -- 172.21.15.38:0/2496449003 shutdown_connections 2024-02-15T18:25:56.118 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:56.111+0000 7f956f59e700 1 -- 172.21.15.38:0/2496449003 wait complete. 2024-02-15T18:25:56.458 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:25:56.110419+0000 mon.smithi038 (mon.0) 24 : audit [DBG] from='client.? 172.21.15.38:0/2496449003' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-15T18:25:56.756 INFO:teuthology.orchestra.run.smithi038.stderr:mgr is available 2024-02-15T18:25:56.757 INFO:teuthology.orchestra.run.smithi038.stderr:Enabling cephadm module... 2024-02-15T18:25:56.757 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 mgr module enable cephadm 2024-02-15T18:25:57.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.260+0000 7fc604f90700 1 Processor -- start 2024-02-15T18:25:57.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.260+0000 7fc604f90700 1 -- start start 2024-02-15T18:25:57.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.261+0000 7fc604f90700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc600070420 0x7fc600100f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:57.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.261+0000 7fc604f90700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6000fc970 con 0x7fc600070420 2024-02-15T18:25:57.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.261+0000 7fc5fe59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc600070420 0x7fc600100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:57.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.261+0000 7fc5fe59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc600070420 0x7fc600100f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43460/0 (socket says 172.21.15.38:43460) 2024-02-15T18:25:57.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.261+0000 7fc5fe59c700 1 -- 172.21.15.38:0/3341532917 learned_addr learned my addr 172.21.15.38:0/3341532917 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:25:57.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.261+0000 7fc5fe59c700 1 -- 172.21.15.38:0/3341532917 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc600101470 con 0x7fc600070420 2024-02-15T18:25:57.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.261+0000 7fc5fe59c700 1 --2- 172.21.15.38:0/3341532917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc600070420 0x7fc600100f80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fc5f000b5f0 tx=0x7fc5f0006ee0).ready entity=mon.0 client_cookie=cecd73f069baf9a3 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:57.262 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.262+0000 7fc5fd59a700 1 -- 172.21.15.38:0/3341532917 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc5f000d600 con 0x7fc600070420 2024-02-15T18:25:57.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.262+0000 7fc5fd59a700 1 -- 172.21.15.38:0/3341532917 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fc5f000dbf0 con 0x7fc600070420 2024-02-15T18:25:57.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.262+0000 7fc5fd59a700 1 -- 172.21.15.38:0/3341532917 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc5f0011cb0 con 0x7fc600070420 2024-02-15T18:25:57.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.262+0000 7fc604f90700 1 -- 172.21.15.38:0/3341532917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc600070420 msgr2=0x7fc600100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:57.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.262+0000 7fc604f90700 1 --2- 172.21.15.38:0/3341532917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc600070420 0x7fc600100f80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fc5f000b5f0 tx=0x7fc5f0006ee0).stop 2024-02-15T18:25:57.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.263+0000 7fc604f90700 1 -- 172.21.15.38:0/3341532917 shutdown_connections 2024-02-15T18:25:57.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.263+0000 7fc604f90700 1 --2- 172.21.15.38:0/3341532917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc600070420 0x7fc600100f80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:57.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.263+0000 7fc604f90700 1 -- 172.21.15.38:0/3341532917 >> 172.21.15.38:0/3341532917 conn(0x7fc6000f65c0 msgr2=0x7fc6000f89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:57.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.263+0000 7fc604f90700 1 -- 172.21.15.38:0/3341532917 shutdown_connections 2024-02-15T18:25:57.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.263+0000 7fc604f90700 1 -- 172.21.15.38:0/3341532917 wait complete. 2024-02-15T18:25:57.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.263+0000 7fc604f90700 1 Processor -- start 2024-02-15T18:25:57.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.263+0000 7fc604f90700 1 -- start start 2024-02-15T18:25:57.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.263+0000 7fc604f90700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc600070420 0x7fc60010cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:57.263 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.263+0000 7fc604f90700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60010d2b0 con 0x7fc600070420 2024-02-15T18:25:57.264 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.264+0000 7fc5fe59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc600070420 0x7fc60010cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:57.264 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.264+0000 7fc5fe59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc600070420 0x7fc60010cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43466/0 (socket says 172.21.15.38:43466) 2024-02-15T18:25:57.264 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.264+0000 7fc5fe59c700 1 -- 172.21.15.38:0/2380009442 learned_addr learned my addr 172.21.15.38:0/2380009442 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:25:57.265 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.264+0000 7fc5fe59c700 1 -- 172.21.15.38:0/2380009442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f000b200 con 0x7fc600070420 2024-02-15T18:25:57.265 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.264+0000 7fc5fe59c700 1 --2- 172.21.15.38:0/2380009442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc600070420 0x7fc60010cdc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fc5f0006800 tx=0x7fc5f0006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:57.265 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.264+0000 7fc5f6ffd700 1 -- 172.21.15.38:0/2380009442 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc5f001b570 con 0x7fc600070420 2024-02-15T18:25:57.265 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.264+0000 7fc604f90700 1 -- 172.21.15.38:0/2380009442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60010d4b0 con 0x7fc600070420 2024-02-15T18:25:57.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.264+0000 7fc5f6ffd700 1 -- 172.21.15.38:0/2380009442 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fc5f001bb60 con 0x7fc600070420 2024-02-15T18:25:57.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.264+0000 7fc5f6ffd700 1 -- 172.21.15.38:0/2380009442 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc5f001a9f0 con 0x7fc600070420 2024-02-15T18:25:57.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.264+0000 7fc604f90700 1 -- 172.21.15.38:0/2380009442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60010e0f0 con 0x7fc600070420 2024-02-15T18:25:57.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.265+0000 7fc5f6ffd700 1 -- 172.21.15.38:0/2380009442 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 36109+0+0 (secure 0 0 0) 0x7fc5f000f070 con 0x7fc600070420 2024-02-15T18:25:57.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.265+0000 7fc5f6ffd700 1 --2- 172.21.15.38:0/2380009442 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fc5e002fdc0 0x7fc5e0032280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:57.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.265+0000 7fc5f6ffd700 1 -- 172.21.15.38:0/2380009442 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fc5f00414b0 con 0x7fc600070420 2024-02-15T18:25:57.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.265+0000 7fc5fdd9b700 1 --2- 172.21.15.38:0/2380009442 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fc5e002fdc0 0x7fc5e0032280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:57.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.265+0000 7fc604f90700 1 -- 172.21.15.38:0/2380009442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc60010d940 con 0x7fc600070420 2024-02-15T18:25:57.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.266+0000 7fc5fdd9b700 1 --2- 172.21.15.38:0/2380009442 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fc5e002fdc0 0x7fc5e0032280 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fc5e80096a0 tx=0x7fc5e8006b40).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:57.268 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.270+0000 7fc5f6ffd700 1 -- 172.21.15.38:0/2380009442 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc5f0016070 con 0x7fc600070420 2024-02-15T18:25:57.465 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.467+0000 7fc604f90700 1 -- 172.21.15.38:0/2380009442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fc600102120 con 0x7fc600070420 2024-02-15T18:25:57.519 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.521+0000 7fc5f6ffd700 1 -- 172.21.15.38:0/2380009442 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 5) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fc5f001bcd0 con 0x7fc600070420 2024-02-15T18:25:57.521 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.523+0000 7fc5f6ffd700 1 -- 172.21.15.38:0/2380009442 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fc5f0019b70 con 0x7fc600070420 2024-02-15T18:25:57.523 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.524+0000 7fc604f90700 1 -- 172.21.15.38:0/2380009442 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fc5e002fdc0 msgr2=0x7fc5e0032280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.524+0000 7fc604f90700 1 --2- 172.21.15.38:0/2380009442 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fc5e002fdc0 0x7fc5e0032280 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fc5e80096a0 tx=0x7fc5e8006b40).stop 2024-02-15T18:25:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.524+0000 7fc604f90700 1 -- 172.21.15.38:0/2380009442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc600070420 msgr2=0x7fc60010cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.524+0000 7fc604f90700 1 --2- 172.21.15.38:0/2380009442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc600070420 0x7fc60010cdc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fc5f0006800 tx=0x7fc5f0006ee0).stop 2024-02-15T18:25:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.524+0000 7fc604f90700 1 -- 172.21.15.38:0/2380009442 shutdown_connections 2024-02-15T18:25:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.525+0000 7fc604f90700 1 --2- 172.21.15.38:0/2380009442 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fc5e002fdc0 0x7fc5e0032280 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.525+0000 7fc604f90700 1 --2- 172.21.15.38:0/2380009442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc600070420 0x7fc60010cdc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.525+0000 7fc604f90700 1 -- 172.21.15.38:0/2380009442 >> 172.21.15.38:0/2380009442 conn(0x7fc6000f65c0 msgr2=0x7fc6000f7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.525+0000 7fc604f90700 1 -- 172.21.15.38:0/2380009442 shutdown_connections 2024-02-15T18:25:57.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:57.525+0000 7fc604f90700 1 -- 172.21.15.38:0/2380009442 wait complete. 2024-02-15T18:25:57.779 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:25:57.468837+0000 mon.smithi038 (mon.0) 25 : audit [INF] from='client.? 172.21.15.38:0/2380009442' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-15T18:25:58.222 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 mgr stat 2024-02-15T18:25:58.786 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:25:57.524423+0000 mon.smithi038 (mon.0) 26 : audit [INF] from='client.? 172.21.15.38:0/2380009442' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-15T18:25:58.786 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:25:57.524468+0000 mon.smithi038 (mon.0) 27 : cluster [DBG] mgrmap e5: smithi038.khesbl(active, since 11s) 2024-02-15T18:25:58.915 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.915+0000 7fbe3135f700 1 Processor -- start 2024-02-15T18:25:58.915 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.915+0000 7fbe3135f700 1 -- start start 2024-02-15T18:25:58.915 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.915+0000 7fbe3135f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe2c0fadf0 0x7fbe2c0fb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:58.915 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.915+0000 7fbe3135f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c0fb700 con 0x7fbe2c0fadf0 2024-02-15T18:25:58.915 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.916+0000 7fbe2affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe2c0fadf0 0x7fbe2c0fb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:58.915 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.916+0000 7fbe2affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe2c0fadf0 0x7fbe2c0fb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43490/0 (socket says 172.21.15.38:43490) 2024-02-15T18:25:58.915 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.916+0000 7fbe2affd700 1 -- 172.21.15.38:0/1354607953 learned_addr learned my addr 172.21.15.38:0/1354607953 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:25:58.915 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.916+0000 7fbe2affd700 1 -- 172.21.15.38:0/1354607953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2c0fb840 con 0x7fbe2c0fadf0 2024-02-15T18:25:58.915 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.916+0000 7fbe2affd700 1 --2- 172.21.15.38:0/1354607953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe2c0fadf0 0x7fbe2c0fb210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fbe1c00b870 tx=0x7fbe1c006f70).ready entity=mon.0 client_cookie=6b7f760a4c40f366 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:58.915 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.916+0000 7fbe29ffb700 1 -- 172.21.15.38:0/1354607953 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe1c00da20 con 0x7fbe2c0fadf0 2024-02-15T18:25:58.915 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.916+0000 7fbe29ffb700 1 -- 172.21.15.38:0/1354607953 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fbe1c010470 con 0x7fbe2c0fadf0 2024-02-15T18:25:58.915 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.917+0000 7fbe29ffb700 1 -- 172.21.15.38:0/1354607953 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe1c00f5b0 con 0x7fbe2c0fadf0 2024-02-15T18:25:58.917 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.917+0000 7fbe3135f700 1 -- 172.21.15.38:0/1354607953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe2c0fadf0 msgr2=0x7fbe2c0fb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:58.917 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.917+0000 7fbe3135f700 1 --2- 172.21.15.38:0/1354607953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe2c0fadf0 0x7fbe2c0fb210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fbe1c00b870 tx=0x7fbe1c006f70).stop 2024-02-15T18:25:58.918 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.917+0000 7fbe3135f700 1 -- 172.21.15.38:0/1354607953 shutdown_connections 2024-02-15T18:25:58.918 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.917+0000 7fbe3135f700 1 --2- 172.21.15.38:0/1354607953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe2c0fadf0 0x7fbe2c0fb210 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:58.918 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.917+0000 7fbe3135f700 1 -- 172.21.15.38:0/1354607953 >> 172.21.15.38:0/1354607953 conn(0x7fbe2c0f65c0 msgr2=0x7fbe2c0f89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:58.918 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.917+0000 7fbe3135f700 1 -- 172.21.15.38:0/1354607953 shutdown_connections 2024-02-15T18:25:58.918 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.918+0000 7fbe3135f700 1 -- 172.21.15.38:0/1354607953 wait complete. 2024-02-15T18:25:58.918 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.918+0000 7fbe3135f700 1 Processor -- start 2024-02-15T18:25:58.918 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.918+0000 7fbe3135f700 1 -- start start 2024-02-15T18:25:58.918 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.919+0000 7fbe3135f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe2c0fadf0 0x7fbe2c108960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:58.918 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.919+0000 7fbe3135f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c108e50 con 0x7fbe2c0fadf0 2024-02-15T18:25:58.919 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.919+0000 7fbe2affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe2c0fadf0 0x7fbe2c108960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:58.919 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.919+0000 7fbe2affd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe2c0fadf0 0x7fbe2c108960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43498/0 (socket says 172.21.15.38:43498) 2024-02-15T18:25:58.919 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.919+0000 7fbe2affd700 1 -- 172.21.15.38:0/2616636252 learned_addr learned my addr 172.21.15.38:0/2616636252 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:25:58.920 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.919+0000 7fbe2affd700 1 -- 172.21.15.38:0/2616636252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1c00b480 con 0x7fbe2c0fadf0 2024-02-15T18:25:58.920 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.919+0000 7fbe2affd700 1 --2- 172.21.15.38:0/2616636252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe2c0fadf0 0x7fbe2c108960 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fbe1c00daf0 tx=0x7fbe1c006f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:58.920 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.919+0000 7fbe137fe700 1 -- 172.21.15.38:0/2616636252 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe1c00f3d0 con 0x7fbe2c0fadf0 2024-02-15T18:25:58.920 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.919+0000 7fbe3135f700 1 -- 172.21.15.38:0/2616636252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe2c109050 con 0x7fbe2c0fadf0 2024-02-15T18:25:58.920 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.920+0000 7fbe137fe700 1 -- 172.21.15.38:0/2616636252 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fbe1c00de30 con 0x7fbe2c0fadf0 2024-02-15T18:25:58.920 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.920+0000 7fbe137fe700 1 -- 172.21.15.38:0/2616636252 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe1c019dc0 con 0x7fbe2c0fadf0 2024-02-15T18:25:58.920 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.920+0000 7fbe3135f700 1 -- 172.21.15.38:0/2616636252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe2c1094e0 con 0x7fbe2c0fadf0 2024-02-15T18:25:58.920 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.920+0000 7fbe137fe700 1 -- 172.21.15.38:0/2616636252 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fbe1c016030 con 0x7fbe2c0fadf0 2024-02-15T18:25:58.920 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.921+0000 7fbe137fe700 1 --2- 172.21.15.38:0/2616636252 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fbe1802fe10 0x7fbe180322d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:58.920 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.921+0000 7fbe137fe700 1 -- 172.21.15.38:0/2616636252 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fbe1c043ed0 con 0x7fbe2c0fadf0 2024-02-15T18:25:58.921 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.921+0000 7fbe2a7fc700 1 -- 172.21.15.38:0/2616636252 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fbe1802fe10 msgr2=0x7fbe180322d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/4189944318 2024-02-15T18:25:58.921 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.921+0000 7fbe2a7fc700 1 --2- 172.21.15.38:0/2616636252 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fbe1802fe10 0x7fbe180322d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-15T18:25:58.921 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.921+0000 7fbe3135f700 1 -- 172.21.15.38:0/2616636252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe2c045ad0 con 0x7fbe2c0fadf0 2024-02-15T18:25:58.923 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:58.925+0000 7fbe137fe700 1 -- 172.21.15.38:0/2616636252 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fbe1c014080 con 0x7fbe2c0fadf0 2024-02-15T18:25:59.120 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:59.122+0000 7fbe2a7fc700 1 -- 172.21.15.38:0/2616636252 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fbe1802fe10 msgr2=0x7fbe180322d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/4189944318 2024-02-15T18:25:59.120 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:59.122+0000 7fbe2a7fc700 1 --2- 172.21.15.38:0/2616636252 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fbe1802fe10 0x7fbe180322d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-15T18:25:59.121 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:59.123+0000 7fbe3135f700 1 -- 172.21.15.38:0/2616636252 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fbe2c102830 con 0x7fbe2c0fadf0 2024-02-15T18:25:59.122 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:59.123+0000 7fbe137fe700 1 -- 172.21.15.38:0/2616636252 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7fbe1c022320 con 0x7fbe2c0fadf0 2024-02-15T18:25:59.122 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: { 2024-02-15T18:25:59.122 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "epoch": 5, 2024-02-15T18:25:59.122 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "available": true, 2024-02-15T18:25:59.122 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "active_name": "smithi038.khesbl", 2024-02-15T18:25:59.122 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_standby": 0 2024-02-15T18:25:59.122 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: } 2024-02-15T18:25:59.124 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:59.125+0000 7fbe3135f700 1 -- 172.21.15.38:0/2616636252 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fbe1802fe10 msgr2=0x7fbe180322d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-15T18:25:59.124 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:59.125+0000 7fbe3135f700 1 --2- 172.21.15.38:0/2616636252 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fbe1802fe10 0x7fbe180322d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:25:59.124 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:59.125+0000 7fbe3135f700 1 -- 172.21.15.38:0/2616636252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe2c0fadf0 msgr2=0x7fbe2c108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:59.124 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:59.125+0000 7fbe3135f700 1 --2- 172.21.15.38:0/2616636252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe2c0fadf0 0x7fbe2c108960 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fbe1c00daf0 tx=0x7fbe1c006f70).stop 2024-02-15T18:25:59.124 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:59.125+0000 7fbe3135f700 1 -- 172.21.15.38:0/2616636252 shutdown_connections 2024-02-15T18:25:59.124 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:59.125+0000 7fbe3135f700 1 --2- 172.21.15.38:0/2616636252 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fbe1802fe10 0x7fbe180322d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:25:59.124 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:59.125+0000 7fbe3135f700 1 --2- 172.21.15.38:0/2616636252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe2c0fadf0 0x7fbe2c108960 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:59.124 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:59.125+0000 7fbe3135f700 1 -- 172.21.15.38:0/2616636252 >> 172.21.15.38:0/2616636252 conn(0x7fbe2c0f65c0 msgr2=0x7fbe2c0ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:59.124 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:59.126+0000 7fbe3135f700 1 -- 172.21.15.38:0/2616636252 shutdown_connections 2024-02-15T18:25:59.124 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:25:59.126+0000 7fbe3135f700 1 -- 172.21.15.38:0/2616636252 wait complete. 2024-02-15T18:25:59.729 INFO:teuthology.orchestra.run.smithi038.stderr:Waiting for the mgr to restart... 2024-02-15T18:25:59.729 INFO:teuthology.orchestra.run.smithi038.stderr:Waiting for mgr epoch 5... 2024-02-15T18:25:59.729 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 tell mgr mgr_status 2024-02-15T18:25:59.732 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:25:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:25:59.124876+0000 mon.smithi038 (mon.0) 28 : audit [DBG] from='client.? 172.21.15.38:0/2616636252' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-15T18:26:00.296 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.297+0000 7fe266725700 1 Processor -- start 2024-02-15T18:26:00.296 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.297+0000 7fe266725700 1 -- start start 2024-02-15T18:26:00.297 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.297+0000 7fe266725700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2600fae00 0x7fe2600fb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:00.297 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.297+0000 7fe266725700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2600fb710 con 0x7fe2600fae00 2024-02-15T18:26:00.297 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.298+0000 7fe25ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2600fae00 0x7fe2600fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:00.297 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.298+0000 7fe25ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2600fae00 0x7fe2600fb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43502/0 (socket says 172.21.15.38:43502) 2024-02-15T18:26:00.297 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.298+0000 7fe25ffff700 1 -- 172.21.15.38:0/3173910053 learned_addr learned my addr 172.21.15.38:0/3173910053 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:00.297 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.298+0000 7fe25ffff700 1 -- 172.21.15.38:0/3173910053 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2600fb850 con 0x7fe2600fae00 2024-02-15T18:26:00.298 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.298+0000 7fe25ffff700 1 --2- 172.21.15.38:0/3173910053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2600fae00 0x7fe2600fb220 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fe25000b5f0 tx=0x7fe250006ee0).ready entity=mon.0 client_cookie=30a0428d106218ef server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:00.298 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.298+0000 7fe25effd700 1 -- 172.21.15.38:0/3173910053 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe25000d600 con 0x7fe2600fae00 2024-02-15T18:26:00.298 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.298+0000 7fe25effd700 1 -- 172.21.15.38:0/3173910053 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fe25000dbf0 con 0x7fe2600fae00 2024-02-15T18:26:00.299 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.298+0000 7fe25effd700 1 -- 172.21.15.38:0/3173910053 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe250011cb0 con 0x7fe2600fae00 2024-02-15T18:26:00.299 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.299+0000 7fe266725700 1 -- 172.21.15.38:0/3173910053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2600fae00 msgr2=0x7fe2600fb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:00.299 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.299+0000 7fe266725700 1 --2- 172.21.15.38:0/3173910053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2600fae00 0x7fe2600fb220 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fe25000b5f0 tx=0x7fe250006ee0).stop 2024-02-15T18:26:00.299 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.299+0000 7fe266725700 1 -- 172.21.15.38:0/3173910053 shutdown_connections 2024-02-15T18:26:00.299 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.299+0000 7fe266725700 1 --2- 172.21.15.38:0/3173910053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2600fae00 0x7fe2600fb220 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:00.299 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.299+0000 7fe266725700 1 -- 172.21.15.38:0/3173910053 >> 172.21.15.38:0/3173910053 conn(0x7fe2600f65b0 msgr2=0x7fe2600f89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:00.299 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.299+0000 7fe266725700 1 -- 172.21.15.38:0/3173910053 shutdown_connections 2024-02-15T18:26:00.299 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.299+0000 7fe266725700 1 -- 172.21.15.38:0/3173910053 wait complete. 2024-02-15T18:26:00.301 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.299+0000 7fe266725700 1 Processor -- start 2024-02-15T18:26:00.301 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.300+0000 7fe266725700 1 -- start start 2024-02-15T18:26:00.301 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.300+0000 7fe266725700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2600fae00 0x7fe26010abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:00.301 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.300+0000 7fe266725700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26010b0b0 con 0x7fe2600fae00 2024-02-15T18:26:00.301 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.300+0000 7fe25ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2600fae00 0x7fe26010abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:00.301 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.300+0000 7fe25ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2600fae00 0x7fe26010abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43518/0 (socket says 172.21.15.38:43518) 2024-02-15T18:26:00.301 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.300+0000 7fe25ffff700 1 -- 172.21.15.38:0/1289462239 learned_addr learned my addr 172.21.15.38:0/1289462239 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:00.301 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.300+0000 7fe25ffff700 1 -- 172.21.15.38:0/1289462239 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25000b200 con 0x7fe2600fae00 2024-02-15T18:26:00.301 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.300+0000 7fe25ffff700 1 --2- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2600fae00 0x7fe26010abc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fe25000dfc0 tx=0x7fe250006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:00.301 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.300+0000 7fe25cff9700 1 -- 172.21.15.38:0/1289462239 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe25001b410 con 0x7fe2600fae00 2024-02-15T18:26:00.302 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.301+0000 7fe25cff9700 1 -- 172.21.15.38:0/1289462239 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fe25001ba00 con 0x7fe2600fae00 2024-02-15T18:26:00.302 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.301+0000 7fe266725700 1 -- 172.21.15.38:0/1289462239 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26010b2b0 con 0x7fe2600fae00 2024-02-15T18:26:00.302 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.301+0000 7fe25cff9700 1 -- 172.21.15.38:0/1289462239 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe25001a990 con 0x7fe2600fae00 2024-02-15T18:26:00.302 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.301+0000 7fe266725700 1 -- 172.21.15.38:0/1289462239 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe26010b740 con 0x7fe2600fae00 2024-02-15T18:26:00.302 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.301+0000 7fe25cff9700 1 -- 172.21.15.38:0/1289462239 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fe25000f070 con 0x7fe2600fae00 2024-02-15T18:26:00.302 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.302+0000 7fe25cff9700 1 --2- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fe24c02fe10 0x7fe24c0322d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:00.302 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.302+0000 7fe25cff9700 1 -- 172.21.15.38:0/1289462239 --> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe24c032990 con 0x7fe24c02fe10 2024-02-15T18:26:00.302 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.302+0000 7fe25cff9700 1 -- 172.21.15.38:0/1289462239 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fe250041270 con 0x7fe2600fae00 2024-02-15T18:26:00.302 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.302+0000 7fe25f7fe700 1 -- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fe24c02fe10 msgr2=0x7fe24c0322d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/4189944318 2024-02-15T18:26:00.302 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.302+0000 7fe25f7fe700 1 --2- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fe24c02fe10 0x7fe24c0322d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-15T18:26:00.501 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.503+0000 7fe25f7fe700 1 -- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fe24c02fe10 msgr2=0x7fe24c0322d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/4189944318 2024-02-15T18:26:00.501 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.503+0000 7fe25f7fe700 1 --2- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fe24c02fe10 0x7fe24c0322d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-15T18:26:00.902 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.904+0000 7fe25f7fe700 1 -- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fe24c02fe10 msgr2=0x7fe24c0322d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/4189944318 2024-02-15T18:26:00.923 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:00.904+0000 7fe25f7fe700 1 --2- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fe24c02fe10 0x7fe24c0322d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-02-15T18:26:01.703 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:01.705+0000 7fe25f7fe700 1 -- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fe24c02fe10 msgr2=0x7fe24c0322d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/4189944318 2024-02-15T18:26:01.703 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:01.705+0000 7fe25f7fe700 1 --2- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fe24c02fe10 0x7fe24c0322d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-02-15T18:26:02.822 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:02.823+0000 7fe25cff9700 1 -- 172.21.15.38:0/1289462239 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mgrmap(e 6) v1 ==== 35970+0+0 (secure 0 0 0) 0x7fe250025980 con 0x7fe2600fae00 2024-02-15T18:26:02.822 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:02.823+0000 7fe25cff9700 1 -- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fe24c02fe10 msgr2=0x7fe24c0322d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-15T18:26:02.822 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:02.823+0000 7fe25cff9700 1 --2- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fe24c02fe10 0x7fe24c0322d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:26:03.279 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:02.772376+0000 mon.smithi038 (mon.0) 29 : cluster [INF] Active manager daemon smithi038.khesbl restarted 2024-02-15T18:26:03.279 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:02.772884+0000 mon.smithi038 (mon.0) 30 : cluster [INF] Activating manager daemon smithi038.khesbl 2024-02-15T18:26:03.280 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:02.779040+0000 mon.smithi038 (mon.0) 31 : cluster [DBG] osdmap e2: 0 total, 0 up, 0 in 2024-02-15T18:26:03.830 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:03.831+0000 7fe25cff9700 1 -- 172.21.15.38:0/1289462239 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 7) v1 ==== 36040+0+0 (secure 0 0 0) 0x7fe250022310 con 0x7fe2600fae00 2024-02-15T18:26:03.831 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:03.832+0000 7fe25cff9700 1 --2- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fe24c033930 0x7fe24c035d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:03.831 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:03.832+0000 7fe25cff9700 1 -- 172.21.15.38:0/1289462239 --> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe24c032990 con 0x7fe24c033930 2024-02-15T18:26:03.831 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:03.832+0000 7fe25f7fe700 1 --2- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fe24c033930 0x7fe24c035d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:03.831 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:03.833+0000 7fe25f7fe700 1 --2- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fe24c033930 0x7fe24c035d20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7fe2540036d0 tx=0x7fe254006ca0).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:03.832 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:03.834+0000 7fe25cff9700 1 -- 172.21.15.38:0/1289462239 <== mgr.14114 v2:172.21.15.38:6800/3009942445 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6438 (secure 0 0 0) 0x7fe24c032990 con 0x7fe24c033930 2024-02-15T18:26:03.838 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:03.840+0000 7fe266725700 1 -- 172.21.15.38:0/1289462239 --> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fe260059100 con 0x7fe24c033930 2024-02-15T18:26:03.839 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:03.841+0000 7fe25cff9700 1 -- 172.21.15.38:0/1289462239 <== mgr.14114 v2:172.21.15.38:6800/3009942445 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fe260059100 con 0x7fe24c033930 2024-02-15T18:26:03.839 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: { 2024-02-15T18:26:03.839 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "mgrmap_epoch": 7, 2024-02-15T18:26:03.839 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "initialized": true 2024-02-15T18:26:03.839 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: } 2024-02-15T18:26:03.841 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:03.841+0000 7fe266725700 1 -- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fe24c033930 msgr2=0x7fe24c035d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:03.841 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:03.841+0000 7fe266725700 1 --2- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fe24c033930 0x7fe24c035d20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7fe2540036d0 tx=0x7fe254006ca0).stop 2024-02-15T18:26:03.841 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:03.842+0000 7fe266725700 1 -- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2600fae00 msgr2=0x7fe26010abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:03.841 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:03.842+0000 7fe266725700 1 --2- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2600fae00 0x7fe26010abc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fe25000dfc0 tx=0x7fe250006ee0).stop 2024-02-15T18:26:03.842 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:03.842+0000 7fe266725700 1 -- 172.21.15.38:0/1289462239 shutdown_connections 2024-02-15T18:26:03.842 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:03.842+0000 7fe266725700 1 --2- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fe24c033930 0x7fe24c035d20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:03.842 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:03.842+0000 7fe266725700 1 --2- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2600fae00 0x7fe26010abc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:03.842 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:03.842+0000 7fe266725700 1 --2- 172.21.15.38:0/1289462239 >> [v2:172.21.15.38:6800/4189944318,v1:172.21.15.38:6801/4189944318] conn(0x7fe24c02fe10 0x7fe24c0322d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:26:03.842 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:03.842+0000 7fe266725700 1 -- 172.21.15.38:0/1289462239 >> 172.21.15.38:0/1289462239 conn(0x7fe2600f65b0 msgr2=0x7fe2600f71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:03.842 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:03.842+0000 7fe266725700 1 -- 172.21.15.38:0/1289462239 shutdown_connections 2024-02-15T18:26:03.842 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:03.842+0000 7fe266725700 1 -- 172.21.15.38:0/1289462239 wait complete. 2024-02-15T18:26:04.250 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:02.825675+0000 mon.smithi038 (mon.0) 32 : cluster [DBG] mgrmap e6: smithi038.khesbl(active, starting, since 0.052866s) 2024-02-15T18:26:04.250 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:02.828328+0000 mon.smithi038 (mon.0) 33 : audit [DBG] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-02-15T18:26:04.250 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:02.828949+0000 mon.smithi038 (mon.0) 34 : audit [DBG] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mgr metadata", "who": "smithi038.khesbl", "id": "smithi038.khesbl"}]: dispatch 2024-02-15T18:26:04.250 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:02.829590+0000 mon.smithi038 (mon.0) 35 : audit [DBG] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-15T18:26:04.250 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:02.829774+0000 mon.smithi038 (mon.0) 36 : audit [DBG] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-15T18:26:04.250 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:02.829936+0000 mon.smithi038 (mon.0) 37 : audit [DBG] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-15T18:26:04.250 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:02.843859+0000 mon.smithi038 (mon.0) 38 : cluster [INF] Manager daemon smithi038.khesbl is now available 2024-02-15T18:26:04.250 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:02.857359+0000 mon.smithi038 (mon.0) 39 : audit [INF] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:04.250 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:02.860076+0000 mon.smithi038 (mon.0) 40 : audit [INF] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:04.250 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:02.862556+0000 mon.smithi038 (mon.0) 41 : audit [INF] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:04.250 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:02.963451+0000 mon.smithi038 (mon.0) 42 : audit [INF] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:04.251 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:02.964132+0000 mon.smithi038 (mon.0) 43 : audit [DBG] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:04.251 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:02.965266+0000 mon.smithi038 (mon.0) 44 : audit [DBG] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:04.251 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:02.966843+0000 mon.smithi038 (mon.0) 45 : audit [DBG] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:04.251 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:02.977857+0000 mon.smithi038 (mon.0) 46 : audit [INF] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:26:04.251 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:02.979127+0000 mon.smithi038 (mon.0) 47 : audit [INF] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:26:04.488 INFO:teuthology.orchestra.run.smithi038.stderr:mgr epoch 5 is available 2024-02-15T18:26:04.488 INFO:teuthology.orchestra.run.smithi038.stderr:Setting orchestrator backend to cephadm... 2024-02-15T18:26:04.488 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch set backend cephadm 2024-02-15T18:26:05.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:04 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:03.835267+0000 mgr.smithi038.khesbl (mgr.14114) 1 : audit [DBG] from='client.14118 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-15T18:26:05.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:04 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:03.836121+0000 mon.smithi038 (mon.0) 48 : cluster [DBG] mgrmap e7: smithi038.khesbl(active, since 1.0633s) 2024-02-15T18:26:05.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:04 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:03.842391+0000 mgr.smithi038.khesbl (mgr.14114) 2 : audit [DBG] from='client.14118 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-15T18:26:05.046 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.046+0000 7fdeeb895700 1 Processor -- start 2024-02-15T18:26:05.046 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.046+0000 7fdeeb895700 1 -- start start 2024-02-15T18:26:05.046 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.046+0000 7fdeeb895700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4070430 0x7fdee40fea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:05.046 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.046+0000 7fdeeb895700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee40fef40 con 0x7fdee4070430 2024-02-15T18:26:05.046 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.047+0000 7fdee9631700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4070430 0x7fdee40fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:05.046 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.047+0000 7fdee9631700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4070430 0x7fdee40fea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37400/0 (socket says 172.21.15.38:37400) 2024-02-15T18:26:05.046 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.047+0000 7fdee9631700 1 -- 172.21.15.38:0/2432697368 learned_addr learned my addr 172.21.15.38:0/2432697368 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:05.046 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.047+0000 7fdee9631700 1 -- 172.21.15.38:0/2432697368 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee40ff080 con 0x7fdee4070430 2024-02-15T18:26:05.046 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.047+0000 7fdee9631700 1 --2- 172.21.15.38:0/2432697368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4070430 0x7fdee40fea50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fded400b5f0 tx=0x7fded4006ee0).ready entity=mon.0 client_cookie=73ded0e5834f91f0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:05.046 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.047+0000 7fded3fff700 1 -- 172.21.15.38:0/2432697368 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fded400d600 con 0x7fdee4070430 2024-02-15T18:26:05.046 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.047+0000 7fded3fff700 1 -- 172.21.15.38:0/2432697368 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fded400dbf0 con 0x7fdee4070430 2024-02-15T18:26:05.046 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.047+0000 7fded3fff700 1 -- 172.21.15.38:0/2432697368 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fded4011cb0 con 0x7fdee4070430 2024-02-15T18:26:05.046 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.048+0000 7fdeeb895700 1 -- 172.21.15.38:0/2432697368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4070430 msgr2=0x7fdee40fea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:05.047 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.048+0000 7fdeeb895700 1 --2- 172.21.15.38:0/2432697368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4070430 0x7fdee40fea50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fded400b5f0 tx=0x7fded4006ee0).stop 2024-02-15T18:26:05.049 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.048+0000 7fdeeb895700 1 -- 172.21.15.38:0/2432697368 shutdown_connections 2024-02-15T18:26:05.050 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.048+0000 7fdeeb895700 1 --2- 172.21.15.38:0/2432697368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4070430 0x7fdee40fea50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:05.050 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.048+0000 7fdeeb895700 1 -- 172.21.15.38:0/2432697368 >> 172.21.15.38:0/2432697368 conn(0x7fdee40f6530 msgr2=0x7fdee40f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:05.050 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.048+0000 7fdeeb895700 1 -- 172.21.15.38:0/2432697368 shutdown_connections 2024-02-15T18:26:05.050 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.048+0000 7fdeeb895700 1 -- 172.21.15.38:0/2432697368 wait complete. 2024-02-15T18:26:05.050 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.049+0000 7fdeeb895700 1 Processor -- start 2024-02-15T18:26:05.050 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.049+0000 7fdeeb895700 1 -- start start 2024-02-15T18:26:05.050 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.049+0000 7fdeeb895700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4070430 0x7fdee410ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:05.050 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.049+0000 7fdeeb895700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee410b080 con 0x7fdee4070430 2024-02-15T18:26:05.050 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.049+0000 7fdee9631700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4070430 0x7fdee410ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:05.050 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.049+0000 7fdee9631700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4070430 0x7fdee410ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37406/0 (socket says 172.21.15.38:37406) 2024-02-15T18:26:05.050 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.049+0000 7fdee9631700 1 -- 172.21.15.38:0/2184028210 learned_addr learned my addr 172.21.15.38:0/2184028210 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:05.051 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.050+0000 7fdee9631700 1 -- 172.21.15.38:0/2184028210 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded400b200 con 0x7fdee4070430 2024-02-15T18:26:05.051 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.050+0000 7fdee9631700 1 --2- 172.21.15.38:0/2184028210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4070430 0x7fdee410ab90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fded400df90 tx=0x7fded4006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:05.051 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.050+0000 7fded1ffb700 1 -- 172.21.15.38:0/2184028210 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fded401b410 con 0x7fdee4070430 2024-02-15T18:26:05.051 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.050+0000 7fded1ffb700 1 -- 172.21.15.38:0/2184028210 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fded401ba00 con 0x7fdee4070430 2024-02-15T18:26:05.051 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.050+0000 7fdeeb895700 1 -- 172.21.15.38:0/2184028210 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee410b280 con 0x7fdee4070430 2024-02-15T18:26:05.051 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.050+0000 7fded1ffb700 1 -- 172.21.15.38:0/2184028210 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fded401aa30 con 0x7fdee4070430 2024-02-15T18:26:05.051 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.050+0000 7fdeeb895700 1 -- 172.21.15.38:0/2184028210 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee410b710 con 0x7fdee4070430 2024-02-15T18:26:05.051 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.051+0000 7fded1ffb700 1 -- 172.21.15.38:0/2184028210 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 7) v1 ==== 36040+0+0 (secure 0 0 0) 0x7fded400f070 con 0x7fdee4070430 2024-02-15T18:26:05.051 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.051+0000 7fdeeb895700 1 -- 172.21.15.38:0/2184028210 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee4045ad0 con 0x7fdee4070430 2024-02-15T18:26:05.051 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.051+0000 7fded1ffb700 1 --2- 172.21.15.38:0/2184028210 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fdecc02fdd0 0x7fdecc032290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:05.051 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.051+0000 7fded1ffb700 1 -- 172.21.15.38:0/2184028210 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fded4041070 con 0x7fdee4070430 2024-02-15T18:26:05.051 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.052+0000 7fdee8e30700 1 --2- 172.21.15.38:0/2184028210 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fdecc02fdd0 0x7fdecc032290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:05.051 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.052+0000 7fdee8e30700 1 --2- 172.21.15.38:0/2184028210 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fdecc02fdd0 0x7fdecc032290 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fdee00096a0 tx=0x7fdee0006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:05.054 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.056+0000 7fded1ffb700 1 -- 172.21.15.38:0/2184028210 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fded4022720 con 0x7fdee4070430 2024-02-15T18:26:05.198 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.200+0000 7fded1ffb700 1 -- 172.21.15.38:0/2184028210 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fded401abb0 con 0x7fdee4070430 2024-02-15T18:26:05.244 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.246+0000 7fdeeb895700 1 -- 172.21.15.38:0/2184028210 --> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fdee410c030 con 0x7fdecc02fdd0 2024-02-15T18:26:05.249 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.251+0000 7fded1ffb700 1 -- 172.21.15.38:0/2184028210 <== mgr.14114 v2:172.21.15.38:6800/3009942445 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdee410c030 con 0x7fdecc02fdd0 2024-02-15T18:26:05.251 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.252+0000 7fdeeb895700 1 -- 172.21.15.38:0/2184028210 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fdecc02fdd0 msgr2=0x7fdecc032290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:05.252 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.252+0000 7fdeeb895700 1 --2- 172.21.15.38:0/2184028210 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fdecc02fdd0 0x7fdecc032290 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fdee00096a0 tx=0x7fdee0006b40).stop 2024-02-15T18:26:05.252 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.252+0000 7fdeeb895700 1 -- 172.21.15.38:0/2184028210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4070430 msgr2=0x7fdee410ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:05.252 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.252+0000 7fdeeb895700 1 --2- 172.21.15.38:0/2184028210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4070430 0x7fdee410ab90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fded400df90 tx=0x7fded4006ee0).stop 2024-02-15T18:26:05.252 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.253+0000 7fdeeb895700 1 -- 172.21.15.38:0/2184028210 shutdown_connections 2024-02-15T18:26:05.252 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.253+0000 7fdeeb895700 1 --2- 172.21.15.38:0/2184028210 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fdecc02fdd0 0x7fdecc032290 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:05.252 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.253+0000 7fdeeb895700 1 --2- 172.21.15.38:0/2184028210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4070430 0x7fdee410ab90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:05.252 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.253+0000 7fdeeb895700 1 -- 172.21.15.38:0/2184028210 >> 172.21.15.38:0/2184028210 conn(0x7fdee40f6530 msgr2=0x7fdee40f7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:05.252 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.253+0000 7fdeeb895700 1 -- 172.21.15.38:0/2184028210 shutdown_connections 2024-02-15T18:26:05.252 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:05.253+0000 7fdeeb895700 1 -- 172.21.15.38:0/2184028210 wait complete. 2024-02-15T18:26:05.968 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 cephadm set-user root 2024-02-15T18:26:06.248 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:05.203185+0000 mon.smithi038 (mon.0) 49 : cluster [DBG] mgrmap e8: smithi038.khesbl(active, since 2s) 2024-02-15T18:26:06.248 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:05.248607+0000 mgr.smithi038.khesbl (mgr.14114) 3 : audit [DBG] from='client.14124 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:06.248 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:05.252306+0000 mon.smithi038 (mon.0) 50 : audit [INF] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:06.248 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:05.255689+0000 mon.smithi038 (mon.0) 51 : audit [DBG] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:06.523 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.492+0000 7ffbb469d700 1 Processor -- start 2024-02-15T18:26:06.523 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.493+0000 7ffbb469d700 1 -- start start 2024-02-15T18:26:06.523 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.493+0000 7ffbb469d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbac0703d0 0x7ffbac100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:06.523 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.493+0000 7ffbb469d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac0fc8a0 con 0x7ffbac0703d0 2024-02-15T18:26:06.523 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.493+0000 7ffbb2439700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbac0703d0 0x7ffbac100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:06.523 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.493+0000 7ffbb2439700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbac0703d0 0x7ffbac100eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37410/0 (socket says 172.21.15.38:37410) 2024-02-15T18:26:06.523 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.493+0000 7ffbb2439700 1 -- 172.21.15.38:0/89252950 learned_addr learned my addr 172.21.15.38:0/89252950 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:06.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.493+0000 7ffbb2439700 1 -- 172.21.15.38:0/89252950 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac1013a0 con 0x7ffbac0703d0 2024-02-15T18:26:06.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.493+0000 7ffbb2439700 1 --2- 172.21.15.38:0/89252950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbac0703d0 0x7ffbac100eb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7ffba000b5f0 tx=0x7ffba0006ee0).ready entity=mon.0 client_cookie=789104650f2cb81c server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:06.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.494+0000 7ffbb1437700 1 -- 172.21.15.38:0/89252950 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffba000d600 con 0x7ffbac0703d0 2024-02-15T18:26:06.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.494+0000 7ffbb1437700 1 -- 172.21.15.38:0/89252950 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7ffba000dbf0 con 0x7ffbac0703d0 2024-02-15T18:26:06.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.494+0000 7ffbb1437700 1 -- 172.21.15.38:0/89252950 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffba0011cb0 con 0x7ffbac0703d0 2024-02-15T18:26:06.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.494+0000 7ffbb469d700 1 -- 172.21.15.38:0/89252950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbac0703d0 msgr2=0x7ffbac100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:06.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.494+0000 7ffbb469d700 1 --2- 172.21.15.38:0/89252950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbac0703d0 0x7ffbac100eb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7ffba000b5f0 tx=0x7ffba0006ee0).stop 2024-02-15T18:26:06.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.494+0000 7ffbb469d700 1 -- 172.21.15.38:0/89252950 shutdown_connections 2024-02-15T18:26:06.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.494+0000 7ffbb469d700 1 --2- 172.21.15.38:0/89252950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbac0703d0 0x7ffbac100eb0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.494+0000 7ffbb469d700 1 -- 172.21.15.38:0/89252950 >> 172.21.15.38:0/89252950 conn(0x7ffbac0f6530 msgr2=0x7ffbac0f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:06.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.494+0000 7ffbb469d700 1 -- 172.21.15.38:0/89252950 shutdown_connections 2024-02-15T18:26:06.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.494+0000 7ffbb469d700 1 -- 172.21.15.38:0/89252950 wait complete. 2024-02-15T18:26:06.524 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.495+0000 7ffbb469d700 1 Processor -- start 2024-02-15T18:26:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.495+0000 7ffbb469d700 1 -- start start 2024-02-15T18:26:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.495+0000 7ffbb469d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbac0703d0 0x7ffbac10cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.495+0000 7ffbb469d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac10d2a0 con 0x7ffbac0703d0 2024-02-15T18:26:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.495+0000 7ffbb2439700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbac0703d0 0x7ffbac10cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.495+0000 7ffbb2439700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbac0703d0 0x7ffbac10cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37414/0 (socket says 172.21.15.38:37414) 2024-02-15T18:26:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.495+0000 7ffbb2439700 1 -- 172.21.15.38:0/3286501560 learned_addr learned my addr 172.21.15.38:0/3286501560 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.495+0000 7ffbb2439700 1 -- 172.21.15.38:0/3286501560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba000b200 con 0x7ffbac0703d0 2024-02-15T18:26:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.495+0000 7ffbb2439700 1 --2- 172.21.15.38:0/3286501560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbac0703d0 0x7ffbac10cdb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ffba0006a20 tx=0x7ffba0006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.496+0000 7ffb9effd700 1 -- 172.21.15.38:0/3286501560 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffba001b570 con 0x7ffbac0703d0 2024-02-15T18:26:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.496+0000 7ffb9effd700 1 -- 172.21.15.38:0/3286501560 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7ffba001bb60 con 0x7ffbac0703d0 2024-02-15T18:26:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.496+0000 7ffbb469d700 1 -- 172.21.15.38:0/3286501560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbac10d4a0 con 0x7ffbac0703d0 2024-02-15T18:26:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.496+0000 7ffb9effd700 1 -- 172.21.15.38:0/3286501560 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffba001aa20 con 0x7ffbac0703d0 2024-02-15T18:26:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.496+0000 7ffbb469d700 1 -- 172.21.15.38:0/3286501560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbac10e0e0 con 0x7ffbac0703d0 2024-02-15T18:26:06.525 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.496+0000 7ffb9effd700 1 -- 172.21.15.38:0/3286501560 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7ffba000f070 con 0x7ffbac0703d0 2024-02-15T18:26:06.526 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.497+0000 7ffb9effd700 1 --2- 172.21.15.38:0/3286501560 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7ffb9402fe70 0x7ffb94032330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:06.526 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.497+0000 7ffbb469d700 1 -- 172.21.15.38:0/3286501560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbac10d930 con 0x7ffbac0703d0 2024-02-15T18:26:06.526 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.497+0000 7ffb9effd700 1 -- 172.21.15.38:0/3286501560 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7ffba0041230 con 0x7ffbac0703d0 2024-02-15T18:26:06.526 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.497+0000 7ffbb1c38700 1 --2- 172.21.15.38:0/3286501560 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7ffb9402fe70 0x7ffb94032330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:06.526 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.497+0000 7ffbb1c38700 1 --2- 172.21.15.38:0/3286501560 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7ffb9402fe70 0x7ffb94032330 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7ffba80096a0 tx=0x7ffba8006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:06.526 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.501+0000 7ffb9effd700 1 -- 172.21.15.38:0/3286501560 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ffba0016070 con 0x7ffbac0703d0 2024-02-15T18:26:06.661 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.663+0000 7ffbb469d700 1 -- 172.21.15.38:0/3286501560 --> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7ffbac10e3e0 con 0x7ffb9402fe70 2024-02-15T18:26:06.662 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.664+0000 7ffb9effd700 1 -- 172.21.15.38:0/3286501560 <== mgr.14114 v2:172.21.15.38:6800/3009942445 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7ffbac10e3e0 con 0x7ffb9402fe70 2024-02-15T18:26:06.662 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: value unchanged 2024-02-15T18:26:06.664 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.665+0000 7ffbb469d700 1 -- 172.21.15.38:0/3286501560 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7ffb9402fe70 msgr2=0x7ffb94032330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:06.664 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.665+0000 7ffbb469d700 1 --2- 172.21.15.38:0/3286501560 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7ffb9402fe70 0x7ffb94032330 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7ffba80096a0 tx=0x7ffba8006b40).stop 2024-02-15T18:26:06.664 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.665+0000 7ffbb469d700 1 -- 172.21.15.38:0/3286501560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbac0703d0 msgr2=0x7ffbac10cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:06.665 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.665+0000 7ffbb469d700 1 --2- 172.21.15.38:0/3286501560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbac0703d0 0x7ffbac10cdb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ffba0006a20 tx=0x7ffba0006ee0).stop 2024-02-15T18:26:06.665 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.665+0000 7ffbb469d700 1 -- 172.21.15.38:0/3286501560 shutdown_connections 2024-02-15T18:26:06.665 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.665+0000 7ffbb469d700 1 --2- 172.21.15.38:0/3286501560 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7ffb9402fe70 0x7ffb94032330 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.665 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.665+0000 7ffbb469d700 1 --2- 172.21.15.38:0/3286501560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbac0703d0 0x7ffbac10cdb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.665 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.665+0000 7ffbb469d700 1 -- 172.21.15.38:0/3286501560 >> 172.21.15.38:0/3286501560 conn(0x7ffbac0f6530 msgr2=0x7ffbac0f7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:06.665 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.666+0000 7ffbb469d700 1 -- 172.21.15.38:0/3286501560 shutdown_connections 2024-02-15T18:26:06.665 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:06.666+0000 7ffbb469d700 1 -- 172.21.15.38:0/3286501560 wait complete. 2024-02-15T18:26:07.266 INFO:teuthology.orchestra.run.smithi038.stderr:Generating ssh key... 2024-02-15T18:26:07.266 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 cephadm generate-key 2024-02-15T18:26:07.786 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.786+0000 7fa19fe24700 1 Processor -- start 2024-02-15T18:26:07.786 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.786+0000 7fa19fe24700 1 -- start start 2024-02-15T18:26:07.786 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.786+0000 7fa19fe24700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1980fad90 0x7fa1980fb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:07.786 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.786+0000 7fa19fe24700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980fb6a0 con 0x7fa1980fad90 2024-02-15T18:26:07.786 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.787+0000 7fa19dbc0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1980fad90 0x7fa1980fb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:07.786 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.787+0000 7fa19dbc0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1980fad90 0x7fa1980fb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37430/0 (socket says 172.21.15.38:37430) 2024-02-15T18:26:07.786 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.787+0000 7fa19dbc0700 1 -- 172.21.15.38:0/2906553353 learned_addr learned my addr 172.21.15.38:0/2906553353 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:07.786 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.787+0000 7fa19dbc0700 1 -- 172.21.15.38:0/2906553353 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1980fb7e0 con 0x7fa1980fad90 2024-02-15T18:26:07.786 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.787+0000 7fa19dbc0700 1 --2- 172.21.15.38:0/2906553353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1980fad90 0x7fa1980fb1b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fa18800b5f0 tx=0x7fa188006ee0).ready entity=mon.0 client_cookie=4acfe67b030ae3e4 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:07.786 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.787+0000 7fa19cbbe700 1 -- 172.21.15.38:0/2906553353 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa18800d600 con 0x7fa1980fad90 2024-02-15T18:26:07.786 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.787+0000 7fa19cbbe700 1 -- 172.21.15.38:0/2906553353 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fa18800dbf0 con 0x7fa1980fad90 2024-02-15T18:26:07.787 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.787+0000 7fa19cbbe700 1 -- 172.21.15.38:0/2906553353 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa188011cb0 con 0x7fa1980fad90 2024-02-15T18:26:07.787 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.788+0000 7fa19fe24700 1 -- 172.21.15.38:0/2906553353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1980fad90 msgr2=0x7fa1980fb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:07.787 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.788+0000 7fa19fe24700 1 --2- 172.21.15.38:0/2906553353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1980fad90 0x7fa1980fb1b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fa18800b5f0 tx=0x7fa188006ee0).stop 2024-02-15T18:26:07.787 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.788+0000 7fa19fe24700 1 -- 172.21.15.38:0/2906553353 shutdown_connections 2024-02-15T18:26:07.789 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.788+0000 7fa19fe24700 1 --2- 172.21.15.38:0/2906553353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1980fad90 0x7fa1980fb1b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:07.789 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.788+0000 7fa19fe24700 1 -- 172.21.15.38:0/2906553353 >> 172.21.15.38:0/2906553353 conn(0x7fa1980f6530 msgr2=0x7fa1980f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:07.790 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.788+0000 7fa19fe24700 1 -- 172.21.15.38:0/2906553353 shutdown_connections 2024-02-15T18:26:07.790 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.788+0000 7fa19fe24700 1 -- 172.21.15.38:0/2906553353 wait complete. 2024-02-15T18:26:07.790 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.789+0000 7fa19fe24700 1 Processor -- start 2024-02-15T18:26:07.790 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.789+0000 7fa19fe24700 1 -- start start 2024-02-15T18:26:07.790 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.789+0000 7fa19fe24700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1980fad90 0x7fa198108920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:07.790 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.789+0000 7fa19fe24700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa198108e10 con 0x7fa1980fad90 2024-02-15T18:26:07.790 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.789+0000 7fa19dbc0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1980fad90 0x7fa198108920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:07.790 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.789+0000 7fa19dbc0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1980fad90 0x7fa198108920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37436/0 (socket says 172.21.15.38:37436) 2024-02-15T18:26:07.790 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.789+0000 7fa19dbc0700 1 -- 172.21.15.38:0/985610394 learned_addr learned my addr 172.21.15.38:0/985610394 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:07.790 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.789+0000 7fa19dbc0700 1 -- 172.21.15.38:0/985610394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18800b200 con 0x7fa1980fad90 2024-02-15T18:26:07.790 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.789+0000 7fa19dbc0700 1 --2- 172.21.15.38:0/985610394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1980fad90 0x7fa198108920 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fa188011ea0 tx=0x7fa188006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:07.790 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.789+0000 7fa1867fc700 1 -- 172.21.15.38:0/985610394 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa18801b570 con 0x7fa1980fad90 2024-02-15T18:26:07.790 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.790+0000 7fa1867fc700 1 -- 172.21.15.38:0/985610394 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fa18801bb60 con 0x7fa1980fad90 2024-02-15T18:26:07.790 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.790+0000 7fa1867fc700 1 -- 172.21.15.38:0/985610394 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa18801a8d0 con 0x7fa1980fad90 2024-02-15T18:26:07.790 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.790+0000 7fa19fe24700 1 -- 172.21.15.38:0/985610394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa198109010 con 0x7fa1980fad90 2024-02-15T18:26:07.791 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.790+0000 7fa19fe24700 1 -- 172.21.15.38:0/985610394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1981094a0 con 0x7fa1980fad90 2024-02-15T18:26:07.791 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.790+0000 7fa1867fc700 1 -- 172.21.15.38:0/985610394 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fa18800f070 con 0x7fa1980fad90 2024-02-15T18:26:07.791 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.791+0000 7fa1867fc700 1 --2- 172.21.15.38:0/985610394 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fa1800320b0 0x7fa180034570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:07.791 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.791+0000 7fa1867fc700 1 -- 172.21.15.38:0/985610394 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fa188041350 con 0x7fa1980fad90 2024-02-15T18:26:07.791 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.791+0000 7fa19d3bf700 1 --2- 172.21.15.38:0/985610394 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fa1800320b0 0x7fa180034570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:07.791 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.791+0000 7fa19fe24700 1 -- 172.21.15.38:0/985610394 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa198045ad0 con 0x7fa1980fad90 2024-02-15T18:26:07.791 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.791+0000 7fa19d3bf700 1 --2- 172.21.15.38:0/985610394 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fa1800320b0 0x7fa180034570 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fa1940096a0 tx=0x7fa194006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:07.793 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.796+0000 7fa1867fc700 1 -- 172.21.15.38:0/985610394 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fa188016070 con 0x7fa1980fad90 2024-02-15T18:26:07.953 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:07.955+0000 7fa19fe24700 1 -- 172.21.15.38:0/985610394 --> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1980008d0 con 0x7fa1800320b0 2024-02-15T18:26:08.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:06.665642+0000 mgr.smithi038.khesbl (mgr.14114) 4 : audit [DBG] from='client.14126 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:08.042 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:08.044+0000 7fa1867fc700 1 -- 172.21.15.38:0/985610394 <== mgr.14114 v2:172.21.15.38:6800/3009942445 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa1980008d0 con 0x7fa1800320b0 2024-02-15T18:26:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:08.046+0000 7fa19fe24700 1 -- 172.21.15.38:0/985610394 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fa1800320b0 msgr2=0x7fa180034570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:08.046+0000 7fa19fe24700 1 --2- 172.21.15.38:0/985610394 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fa1800320b0 0x7fa180034570 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fa1940096a0 tx=0x7fa194006b40).stop 2024-02-15T18:26:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:08.046+0000 7fa19fe24700 1 -- 172.21.15.38:0/985610394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1980fad90 msgr2=0x7fa198108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:08.046+0000 7fa19fe24700 1 --2- 172.21.15.38:0/985610394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1980fad90 0x7fa198108920 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fa188011ea0 tx=0x7fa188006ee0).stop 2024-02-15T18:26:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:08.046+0000 7fa19fe24700 1 -- 172.21.15.38:0/985610394 shutdown_connections 2024-02-15T18:26:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:08.046+0000 7fa19fe24700 1 --2- 172.21.15.38:0/985610394 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fa1800320b0 0x7fa180034570 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:08.046+0000 7fa19fe24700 1 --2- 172.21.15.38:0/985610394 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1980fad90 0x7fa198108920 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:08.046+0000 7fa19fe24700 1 -- 172.21.15.38:0/985610394 >> 172.21.15.38:0/985610394 conn(0x7fa1980f6530 msgr2=0x7fa1980ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:08.046+0000 7fa19fe24700 1 -- 172.21.15.38:0/985610394 shutdown_connections 2024-02-15T18:26:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:08.046+0000 7fa19fe24700 1 -- 172.21.15.38:0/985610394 wait complete. 2024-02-15T18:26:08.707 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 cephadm get-pub-key 2024-02-15T18:26:09.279 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:26:07.957716+0000 mgr.smithi038.khesbl (mgr.14114) 6 : cephadm [INF] Generating ssh key... 2024-02-15T18:26:09.279 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:08.038433+0000 mon.smithi038 (mon.0) 52 : audit [INF] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:09.279 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:08.045334+0000 mon.smithi038 (mon.0) 53 : audit [INF] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:09.361 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.363+0000 7f1f1022a700 1 Processor -- start 2024-02-15T18:26:09.361 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.363+0000 7f1f1022a700 1 -- start start 2024-02-15T18:26:09.361 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.363+0000 7f1f1022a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f08070390 0x7f1f08100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:09.362 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.363+0000 7f1f1022a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080fc8e0 con 0x7f1f08070390 2024-02-15T18:26:09.362 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.363+0000 7f1f0dfc6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f08070390 0x7f1f08100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:09.362 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.363+0000 7f1f0dfc6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f08070390 0x7f1f08100ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37438/0 (socket says 172.21.15.38:37438) 2024-02-15T18:26:09.362 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.363+0000 7f1f0dfc6700 1 -- 172.21.15.38:0/1591159490 learned_addr learned my addr 172.21.15.38:0/1591159490 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:09.362 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.363+0000 7f1f0dfc6700 1 -- 172.21.15.38:0/1591159490 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f081013e0 con 0x7f1f08070390 2024-02-15T18:26:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.364+0000 7f1f0dfc6700 1 --2- 172.21.15.38:0/1591159490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f08070390 0x7f1f08100ef0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f1f0400b5f0 tx=0x7f1f04006ee0).ready entity=mon.0 client_cookie=bbf4cd3b19cb93b0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.364+0000 7f1f0cfc4700 1 -- 172.21.15.38:0/1591159490 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f0400d600 con 0x7f1f08070390 2024-02-15T18:26:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.364+0000 7f1f0cfc4700 1 -- 172.21.15.38:0/1591159490 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f1f0400dbf0 con 0x7f1f08070390 2024-02-15T18:26:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.364+0000 7f1f0cfc4700 1 -- 172.21.15.38:0/1591159490 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f04011cb0 con 0x7f1f08070390 2024-02-15T18:26:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.364+0000 7f1f1022a700 1 -- 172.21.15.38:0/1591159490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f08070390 msgr2=0x7f1f08100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.364+0000 7f1f1022a700 1 --2- 172.21.15.38:0/1591159490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f08070390 0x7f1f08100ef0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f1f0400b5f0 tx=0x7f1f04006ee0).stop 2024-02-15T18:26:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.364+0000 7f1f1022a700 1 -- 172.21.15.38:0/1591159490 shutdown_connections 2024-02-15T18:26:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.364+0000 7f1f1022a700 1 --2- 172.21.15.38:0/1591159490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f08070390 0x7f1f08100ef0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.364+0000 7f1f1022a700 1 -- 172.21.15.38:0/1591159490 >> 172.21.15.38:0/1591159490 conn(0x7f1f080f6530 msgr2=0x7f1f080f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.365+0000 7f1f1022a700 1 -- 172.21.15.38:0/1591159490 shutdown_connections 2024-02-15T18:26:09.365 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.365+0000 7f1f1022a700 1 -- 172.21.15.38:0/1591159490 wait complete. 2024-02-15T18:26:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.365+0000 7f1f1022a700 1 Processor -- start 2024-02-15T18:26:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.365+0000 7f1f1022a700 1 -- start start 2024-02-15T18:26:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.366+0000 7f1f1022a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f08070390 0x7f1f0810cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.366+0000 7f1f1022a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0810d290 con 0x7f1f08070390 2024-02-15T18:26:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.366+0000 7f1f0dfc6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f08070390 0x7f1f0810cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.366+0000 7f1f0dfc6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f08070390 0x7f1f0810cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37452/0 (socket says 172.21.15.38:37452) 2024-02-15T18:26:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.366+0000 7f1f0dfc6700 1 -- 172.21.15.38:0/1798341961 learned_addr learned my addr 172.21.15.38:0/1798341961 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.366+0000 7f1f0dfc6700 1 -- 172.21.15.38:0/1798341961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0400b200 con 0x7f1f08070390 2024-02-15T18:26:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.366+0000 7f1f0dfc6700 1 --2- 172.21.15.38:0/1798341961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f08070390 0x7f1f0810cda0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f1f0400df90 tx=0x7f1f04006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.366+0000 7f1efa7fc700 1 -- 172.21.15.38:0/1798341961 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f0401b410 con 0x7f1f08070390 2024-02-15T18:26:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.366+0000 7f1efa7fc700 1 -- 172.21.15.38:0/1798341961 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f1f0401ba00 con 0x7f1f08070390 2024-02-15T18:26:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.366+0000 7f1f1022a700 1 -- 172.21.15.38:0/1798341961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0810d490 con 0x7f1f08070390 2024-02-15T18:26:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.366+0000 7f1efa7fc700 1 -- 172.21.15.38:0/1798341961 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f0401aa30 con 0x7f1f08070390 2024-02-15T18:26:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.367+0000 7f1f1022a700 1 -- 172.21.15.38:0/1798341961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0810e0d0 con 0x7f1f08070390 2024-02-15T18:26:09.367 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.367+0000 7f1efa7fc700 1 -- 172.21.15.38:0/1798341961 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f1f0400f070 con 0x7f1f08070390 2024-02-15T18:26:09.367 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.368+0000 7f1efa7fc700 1 --2- 172.21.15.38:0/1798341961 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7f1ef002fe70 0x7f1ef0032330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:09.368 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.368+0000 7f1efa7fc700 1 -- 172.21.15.38:0/1798341961 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f1f04041120 con 0x7f1f08070390 2024-02-15T18:26:09.368 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.368+0000 7f1f1022a700 1 -- 172.21.15.38:0/1798341961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0810d920 con 0x7f1f08070390 2024-02-15T18:26:09.368 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.368+0000 7f1f0d7c5700 1 --2- 172.21.15.38:0/1798341961 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7f1ef002fe70 0x7f1ef0032330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:09.368 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.368+0000 7f1f0d7c5700 1 --2- 172.21.15.38:0/1798341961 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7f1ef002fe70 0x7f1ef0032330 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f1efc0096a0 tx=0x7f1efc006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:09.370 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.372+0000 7f1efa7fc700 1 -- 172.21.15.38:0/1798341961 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f1f04016070 con 0x7f1f08070390 2024-02-15T18:26:09.531 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.533+0000 7f1f1022a700 1 -- 172.21.15.38:0/1798341961 --> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f080008d0 con 0x7f1ef002fe70 2024-02-15T18:26:09.531 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.533+0000 7f1efa7fc700 1 -- 172.21.15.38:0/1798341961 <== mgr.14114 v2:172.21.15.38:6800/3009942445 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f1f080008d0 con 0x7f1ef002fe70 2024-02-15T18:26:09.531 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDX6uAM6XgBdhMfcotW1fV787J7VAuvY/lUylyipkOEtsG2fDfD4emNwcfHv/e4+q+yGOQ0eHlz5CLZcT0/oLEKq45LDsYfFsSzY7Hto2hnuZZWM0L3zgyYXC+WnSBCBSKsFabpkZ0gINwVfhpxAfBbgC7efO79phGZr79x24mtPkx8PunuwHQ8vNxB9UqGMqnLi9S8DFjtATugamb5Ru5EYX94mBH964Gtz9CvdHzZn3JXj/cFyQfdP9rpW4lo4dXIgBfqlSjDc2MFa5bD8aJbgGpyp/Yfx9kAZTZGzEWIJKHTpNHzaFsjJBBILIqj+jwZi1rVq+dh8iwnmqIRU5MtCGbxVuVbQ4i6U8fLwt4+v0uJuz5H1KPiwD6dzQ0YjNyVgXYuEJOq9DU9eZKX+6S23k65aUs0PQLS4TYQeSSVj/le12qTOY3I98ZAWfB8Abgg/6is7kKiOZMVySboZTpoc0XVhofFFs0R2XYLBl3u142kOMBoy5Eb6VXTMvfyKJU= ceph-7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:26:09.534 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.535+0000 7f1f1022a700 1 -- 172.21.15.38:0/1798341961 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7f1ef002fe70 msgr2=0x7f1ef0032330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:09.534 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.535+0000 7f1f1022a700 1 --2- 172.21.15.38:0/1798341961 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7f1ef002fe70 0x7f1ef0032330 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f1efc0096a0 tx=0x7f1efc006b40).stop 2024-02-15T18:26:09.534 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.535+0000 7f1f1022a700 1 -- 172.21.15.38:0/1798341961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f08070390 msgr2=0x7f1f0810cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:09.534 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.535+0000 7f1f1022a700 1 --2- 172.21.15.38:0/1798341961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f08070390 0x7f1f0810cda0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f1f0400df90 tx=0x7f1f04006ee0).stop 2024-02-15T18:26:09.534 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.535+0000 7f1f1022a700 1 -- 172.21.15.38:0/1798341961 shutdown_connections 2024-02-15T18:26:09.534 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.535+0000 7f1f1022a700 1 --2- 172.21.15.38:0/1798341961 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7f1ef002fe70 0x7f1ef0032330 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:09.534 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.535+0000 7f1f1022a700 1 --2- 172.21.15.38:0/1798341961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f08070390 0x7f1f0810cda0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:09.534 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.535+0000 7f1f1022a700 1 -- 172.21.15.38:0/1798341961 >> 172.21.15.38:0/1798341961 conn(0x7f1f080f6530 msgr2=0x7f1f080f7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:09.534 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.535+0000 7f1f1022a700 1 -- 172.21.15.38:0/1798341961 shutdown_connections 2024-02-15T18:26:09.534 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:09.535+0000 7f1f1022a700 1 -- 172.21.15.38:0/1798341961 wait complete. 2024-02-15T18:26:10.181 INFO:teuthology.orchestra.run.smithi038.stderr:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-15T18:26:10.181 INFO:teuthology.orchestra.run.smithi038.stderr:Adding key to root@localhost authorized_keys... 2024-02-15T18:26:10.182 INFO:teuthology.orchestra.run.smithi038.stderr:Adding host smithi038... 2024-02-15T18:26:10.182 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch host add smithi038 172.21.15.38 2024-02-15T18:26:10.456 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:09.535154+0000 mgr.smithi038.khesbl (mgr.14114) 7 : audit [DBG] from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:10.824 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.826+0000 7fb66d318700 1 Processor -- start 2024-02-15T18:26:10.827 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.826+0000 7fb66d318700 1 -- start start 2024-02-15T18:26:10.827 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.826+0000 7fb66d318700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6680703b0 0x7fb668100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:10.827 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.826+0000 7fb66d318700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6680fc8c0 con 0x7fb6680703b0 2024-02-15T18:26:10.827 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.827+0000 7fb666ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6680703b0 0x7fb668100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:10.827 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.827+0000 7fb666ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6680703b0 0x7fb668100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37456/0 (socket says 172.21.15.38:37456) 2024-02-15T18:26:10.827 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.827+0000 7fb666ffd700 1 -- 172.21.15.38:0/4137522509 learned_addr learned my addr 172.21.15.38:0/4137522509 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:10.827 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.827+0000 7fb666ffd700 1 -- 172.21.15.38:0/4137522509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6681013c0 con 0x7fb6680703b0 2024-02-15T18:26:10.827 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.827+0000 7fb666ffd700 1 --2- 172.21.15.38:0/4137522509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6680703b0 0x7fb668100ed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fb65c00b5f0 tx=0x7fb65c006ee0).ready entity=mon.0 client_cookie=97233b5bb791a32b server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.827+0000 7fb665ffb700 1 -- 172.21.15.38:0/4137522509 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb65c00d600 con 0x7fb6680703b0 2024-02-15T18:26:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.827+0000 7fb665ffb700 1 -- 172.21.15.38:0/4137522509 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fb65c00dbf0 con 0x7fb6680703b0 2024-02-15T18:26:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.827+0000 7fb665ffb700 1 -- 172.21.15.38:0/4137522509 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb65c011cb0 con 0x7fb6680703b0 2024-02-15T18:26:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.828+0000 7fb66d318700 1 -- 172.21.15.38:0/4137522509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6680703b0 msgr2=0x7fb668100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.828+0000 7fb66d318700 1 --2- 172.21.15.38:0/4137522509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6680703b0 0x7fb668100ed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fb65c00b5f0 tx=0x7fb65c006ee0).stop 2024-02-15T18:26:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.828+0000 7fb66d318700 1 -- 172.21.15.38:0/4137522509 shutdown_connections 2024-02-15T18:26:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.828+0000 7fb66d318700 1 --2- 172.21.15.38:0/4137522509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6680703b0 0x7fb668100ed0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.828+0000 7fb66d318700 1 -- 172.21.15.38:0/4137522509 >> 172.21.15.38:0/4137522509 conn(0x7fb6680f6530 msgr2=0x7fb6680f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.828+0000 7fb66d318700 1 -- 172.21.15.38:0/4137522509 shutdown_connections 2024-02-15T18:26:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.828+0000 7fb66d318700 1 -- 172.21.15.38:0/4137522509 wait complete. 2024-02-15T18:26:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.829+0000 7fb66d318700 1 Processor -- start 2024-02-15T18:26:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.829+0000 7fb66d318700 1 -- start start 2024-02-15T18:26:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.829+0000 7fb66d318700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6680703b0 0x7fb66810cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.829+0000 7fb66d318700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66810d270 con 0x7fb6680703b0 2024-02-15T18:26:10.829 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.829+0000 7fb666ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6680703b0 0x7fb66810cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:10.829 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.829+0000 7fb666ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6680703b0 0x7fb66810cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37458/0 (socket says 172.21.15.38:37458) 2024-02-15T18:26:10.829 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.829+0000 7fb666ffd700 1 -- 172.21.15.38:0/2508405717 learned_addr learned my addr 172.21.15.38:0/2508405717 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:10.829 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.829+0000 7fb666ffd700 1 -- 172.21.15.38:0/2508405717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c00b200 con 0x7fb6680703b0 2024-02-15T18:26:10.829 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.829+0000 7fb666ffd700 1 --2- 172.21.15.38:0/2508405717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6680703b0 0x7fb66810cd80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fb65c006800 tx=0x7fb65c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:10.829 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.830+0000 7fb64f7fe700 1 -- 172.21.15.38:0/2508405717 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb65c01b570 con 0x7fb6680703b0 2024-02-15T18:26:10.829 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.830+0000 7fb66d318700 1 -- 172.21.15.38:0/2508405717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb66810d470 con 0x7fb6680703b0 2024-02-15T18:26:10.829 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.830+0000 7fb64f7fe700 1 -- 172.21.15.38:0/2508405717 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fb65c01bb60 con 0x7fb6680703b0 2024-02-15T18:26:10.829 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.830+0000 7fb64f7fe700 1 -- 172.21.15.38:0/2508405717 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb65c01a9f0 con 0x7fb6680703b0 2024-02-15T18:26:10.829 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.830+0000 7fb66d318700 1 -- 172.21.15.38:0/2508405717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66810e080 con 0x7fb6680703b0 2024-02-15T18:26:10.829 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.830+0000 7fb64f7fe700 1 -- 172.21.15.38:0/2508405717 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fb65c00f070 con 0x7fb6680703b0 2024-02-15T18:26:10.829 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.830+0000 7fb64f7fe700 1 --2- 172.21.15.38:0/2508405717 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fb65402fe10 0x7fb6540322d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:10.830 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.831+0000 7fb64f7fe700 1 -- 172.21.15.38:0/2508405717 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fb65c0422b0 con 0x7fb6680703b0 2024-02-15T18:26:10.830 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.831+0000 7fb66d318700 1 -- 172.21.15.38:0/2508405717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb668059100 con 0x7fb6680703b0 2024-02-15T18:26:10.830 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.831+0000 7fb6667fc700 1 --2- 172.21.15.38:0/2508405717 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fb65402fe10 0x7fb6540322d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:10.830 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.831+0000 7fb6667fc700 1 --2- 172.21.15.38:0/2508405717 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fb65402fe10 0x7fb6540322d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fb6580096a0 tx=0x7fb658006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:10.832 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:10.835+0000 7fb64f7fe700 1 -- 172.21.15.38:0/2508405717 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb65c016070 con 0x7fb6680703b0 2024-02-15T18:26:11.017 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:11.019+0000 7fb66d318700 1 -- 172.21.15.38:0/2508405717 --> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi038", "addr": "172.21.15.38", "target": ["mon-mgr", ""]}) v1 -- 0x7fb668102150 con 0x7fb65402fe10 2024-02-15T18:26:11.636 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: Added host 'smithi038' 2024-02-15T18:26:11.636 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:11.637+0000 7fb64f7fe700 1 -- 172.21.15.38:0/2508405717 <== mgr.14114 v2:172.21.15.38:6800/3009942445 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+23 (secure 0 0 0) 0x7fb668102150 con 0x7fb65402fe10 2024-02-15T18:26:11.637 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:11.639+0000 7fb66d318700 1 -- 172.21.15.38:0/2508405717 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fb65402fe10 msgr2=0x7fb6540322d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:11.638 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:11.639+0000 7fb66d318700 1 --2- 172.21.15.38:0/2508405717 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fb65402fe10 0x7fb6540322d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fb6580096a0 tx=0x7fb658006b40).stop 2024-02-15T18:26:11.638 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:11.640+0000 7fb66d318700 1 -- 172.21.15.38:0/2508405717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6680703b0 msgr2=0x7fb66810cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:11.638 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:11.640+0000 7fb66d318700 1 --2- 172.21.15.38:0/2508405717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6680703b0 0x7fb66810cd80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fb65c006800 tx=0x7fb65c006ee0).stop 2024-02-15T18:26:11.638 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:11.640+0000 7fb66d318700 1 -- 172.21.15.38:0/2508405717 shutdown_connections 2024-02-15T18:26:11.638 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:11.640+0000 7fb66d318700 1 --2- 172.21.15.38:0/2508405717 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fb65402fe10 0x7fb6540322d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:11.640+0000 7fb66d318700 1 --2- 172.21.15.38:0/2508405717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6680703b0 0x7fb66810cd80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:11.640+0000 7fb66d318700 1 -- 172.21.15.38:0/2508405717 >> 172.21.15.38:0/2508405717 conn(0x7fb6680f6530 msgr2=0x7fb6680f7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:11.640+0000 7fb66d318700 1 -- 172.21.15.38:0/2508405717 shutdown_connections 2024-02-15T18:26:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:11.640+0000 7fb66d318700 1 -- 172.21.15.38:0/2508405717 wait complete. 2024-02-15T18:26:11.960 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:11.021385+0000 mgr.smithi038.khesbl (mgr.14114) 8 : audit [DBG] from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi038", "addr": "172.21.15.38", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:12.263 INFO:teuthology.orchestra.run.smithi038.stderr:Deploying mon service with default placement... 2024-02-15T18:26:12.263 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply mon 2024-02-15T18:26:12.844 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.831+0000 7f51ec8c6700 1 Processor -- start 2024-02-15T18:26:12.844 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.831+0000 7f51ec8c6700 1 -- start start 2024-02-15T18:26:12.844 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.831+0000 7f51ec8c6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e8070390 0x7f51e8100d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:12.844 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.831+0000 7f51ec8c6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e80fc720 con 0x7f51e8070390 2024-02-15T18:26:12.844 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.831+0000 7f51e77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e8070390 0x7f51e8100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:12.844 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.832+0000 7f51e77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e8070390 0x7f51e8100d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37472/0 (socket says 172.21.15.38:37472) 2024-02-15T18:26:12.844 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.832+0000 7f51e77fe700 1 -- 172.21.15.38:0/1845176497 learned_addr learned my addr 172.21.15.38:0/1845176497 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:12.844 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.832+0000 7f51e77fe700 1 -- 172.21.15.38:0/1845176497 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e8101220 con 0x7f51e8070390 2024-02-15T18:26:12.845 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.832+0000 7f51e77fe700 1 --2- 172.21.15.38:0/1845176497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e8070390 0x7f51e8100d30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f51dc00b5f0 tx=0x7f51dc006ee0).ready entity=mon.0 client_cookie=a10f2c86522e2321 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:12.845 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.832+0000 7f51e67fc700 1 -- 172.21.15.38:0/1845176497 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51dc00d600 con 0x7f51e8070390 2024-02-15T18:26:12.845 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.832+0000 7f51e67fc700 1 -- 172.21.15.38:0/1845176497 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f51dc00dbf0 con 0x7f51e8070390 2024-02-15T18:26:12.845 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.832+0000 7f51e67fc700 1 -- 172.21.15.38:0/1845176497 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51dc011cb0 con 0x7f51e8070390 2024-02-15T18:26:12.845 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.833+0000 7f51ec8c6700 1 -- 172.21.15.38:0/1845176497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e8070390 msgr2=0x7f51e8100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:12.845 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.833+0000 7f51ec8c6700 1 --2- 172.21.15.38:0/1845176497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e8070390 0x7f51e8100d30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f51dc00b5f0 tx=0x7f51dc006ee0).stop 2024-02-15T18:26:12.845 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.833+0000 7f51ec8c6700 1 -- 172.21.15.38:0/1845176497 shutdown_connections 2024-02-15T18:26:12.845 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.833+0000 7f51ec8c6700 1 --2- 172.21.15.38:0/1845176497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e8070390 0x7f51e8100d30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:12.845 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.833+0000 7f51ec8c6700 1 -- 172.21.15.38:0/1845176497 >> 172.21.15.38:0/1845176497 conn(0x7f51e80f6390 msgr2=0x7f51e80f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:12.845 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.833+0000 7f51ec8c6700 1 -- 172.21.15.38:0/1845176497 shutdown_connections 2024-02-15T18:26:12.845 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.833+0000 7f51ec8c6700 1 -- 172.21.15.38:0/1845176497 wait complete. 2024-02-15T18:26:12.845 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.833+0000 7f51ec8c6700 1 Processor -- start 2024-02-15T18:26:12.845 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.834+0000 7f51ec8c6700 1 -- start start 2024-02-15T18:26:12.845 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.834+0000 7f51ec8c6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e8070390 0x7f51e810cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:12.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.834+0000 7f51ec8c6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e810d060 con 0x7f51e8070390 2024-02-15T18:26:12.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.834+0000 7f51e77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e8070390 0x7f51e810cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:12.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.834+0000 7f51e77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e8070390 0x7f51e810cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37480/0 (socket says 172.21.15.38:37480) 2024-02-15T18:26:12.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.834+0000 7f51e77fe700 1 -- 172.21.15.38:0/4174521927 learned_addr learned my addr 172.21.15.38:0/4174521927 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:12.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.834+0000 7f51e77fe700 1 -- 172.21.15.38:0/4174521927 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc00b200 con 0x7f51e8070390 2024-02-15T18:26:12.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.834+0000 7f51e77fe700 1 --2- 172.21.15.38:0/4174521927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e8070390 0x7f51e810cb70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f51dc006800 tx=0x7f51dc006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:12.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.834+0000 7f51cffff700 1 -- 172.21.15.38:0/4174521927 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51dc01b570 con 0x7f51e8070390 2024-02-15T18:26:12.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.834+0000 7f51ec8c6700 1 -- 172.21.15.38:0/4174521927 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e810d260 con 0x7f51e8070390 2024-02-15T18:26:12.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.835+0000 7f51cffff700 1 -- 172.21.15.38:0/4174521927 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f51dc01bb60 con 0x7f51e8070390 2024-02-15T18:26:12.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.835+0000 7f51cffff700 1 -- 172.21.15.38:0/4174521927 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51dc01a9f0 con 0x7f51e8070390 2024-02-15T18:26:12.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.835+0000 7f51ec8c6700 1 -- 172.21.15.38:0/4174521927 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e810d6f0 con 0x7f51e8070390 2024-02-15T18:26:12.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.835+0000 7f51cffff700 1 -- 172.21.15.38:0/4174521927 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f51dc00f070 con 0x7f51e8070390 2024-02-15T18:26:12.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.836+0000 7f51ec8c6700 1 -- 172.21.15.38:0/4174521927 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e8045ad0 con 0x7f51e8070390 2024-02-15T18:26:12.846 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.836+0000 7f51cffff700 1 --2- 172.21.15.38:0/4174521927 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7f51d402fe10 0x7f51d40322d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:12.847 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.836+0000 7f51cffff700 1 -- 172.21.15.38:0/4174521927 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f51dc041450 con 0x7f51e8070390 2024-02-15T18:26:12.847 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.836+0000 7f51e6ffd700 1 --2- 172.21.15.38:0/4174521927 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7f51d402fe10 0x7f51d40322d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:12.847 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.836+0000 7f51e6ffd700 1 --2- 172.21.15.38:0/4174521927 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7f51d402fe10 0x7f51d40322d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f51d80096a0 tx=0x7f51d8006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:12.847 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:12.840+0000 7f51cffff700 1 -- 172.21.15.38:0/4174521927 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f51dc010360 con 0x7f51e8070390 2024-02-15T18:26:13.026 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:13.028+0000 7f51ec8c6700 1 -- 172.21.15.38:0/4174521927 --> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f51e810e010 con 0x7f51d402fe10 2024-02-15T18:26:13.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:11.638025+0000 mon.smithi038 (mon.0) 54 : audit [INF] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:13.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:26:11.638344+0000 mgr.smithi038.khesbl (mgr.14114) 9 : cephadm [INF] Added host smithi038 2024-02-15T18:26:13.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:11.638608+0000 mon.smithi038 (mon.0) 55 : audit [DBG] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:13.030 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:13.032+0000 7f51cffff700 1 -- 172.21.15.38:0/4174521927 <== mgr.14114 v2:172.21.15.38:6800/3009942445 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f51e810e010 con 0x7f51d402fe10 2024-02-15T18:26:13.030 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: Scheduled mon update... 2024-02-15T18:26:13.032 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:13.034+0000 7f51ec8c6700 1 -- 172.21.15.38:0/4174521927 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7f51d402fe10 msgr2=0x7f51d40322d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:13.032 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:13.034+0000 7f51ec8c6700 1 --2- 172.21.15.38:0/4174521927 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7f51d402fe10 0x7f51d40322d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f51d80096a0 tx=0x7f51d8006b40).stop 2024-02-15T18:26:13.032 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:13.034+0000 7f51ec8c6700 1 -- 172.21.15.38:0/4174521927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e8070390 msgr2=0x7f51e810cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:13.032 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:13.034+0000 7f51ec8c6700 1 --2- 172.21.15.38:0/4174521927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e8070390 0x7f51e810cb70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f51dc006800 tx=0x7f51dc006ee0).stop 2024-02-15T18:26:13.033 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:13.034+0000 7f51ec8c6700 1 -- 172.21.15.38:0/4174521927 shutdown_connections 2024-02-15T18:26:13.033 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:13.034+0000 7f51ec8c6700 1 --2- 172.21.15.38:0/4174521927 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7f51d402fe10 0x7f51d40322d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:13.033 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:13.034+0000 7f51ec8c6700 1 --2- 172.21.15.38:0/4174521927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51e8070390 0x7f51e810cb70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:13.033 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:13.034+0000 7f51ec8c6700 1 -- 172.21.15.38:0/4174521927 >> 172.21.15.38:0/4174521927 conn(0x7f51e80f6390 msgr2=0x7f51e80f7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:13.033 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:13.035+0000 7f51ec8c6700 1 -- 172.21.15.38:0/4174521927 shutdown_connections 2024-02-15T18:26:13.033 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:13.035+0000 7f51ec8c6700 1 -- 172.21.15.38:0/4174521927 wait complete. 2024-02-15T18:26:13.668 INFO:teuthology.orchestra.run.smithi038.stderr:Deploying mgr service with default placement... 2024-02-15T18:26:13.668 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply mgr 2024-02-15T18:26:14.235 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.235+0000 7f7b949cc700 1 Processor -- start 2024-02-15T18:26:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.235+0000 7f7b949cc700 1 -- start start 2024-02-15T18:26:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.236+0000 7f7b949cc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b900703f0 0x7f7b90100e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.236+0000 7f7b949cc700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b900fc880 con 0x7f7b900703f0 2024-02-15T18:26:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.236+0000 7f7b8e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b900703f0 0x7f7b90100e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.236+0000 7f7b8e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b900703f0 0x7f7b90100e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33152/0 (socket says 172.21.15.38:33152) 2024-02-15T18:26:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.236+0000 7f7b8e59c700 1 -- 172.21.15.38:0/1731017259 learned_addr learned my addr 172.21.15.38:0/1731017259 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.236+0000 7f7b8e59c700 1 -- 172.21.15.38:0/1731017259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b90101380 con 0x7f7b900703f0 2024-02-15T18:26:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.236+0000 7f7b8e59c700 1 --2- 172.21.15.38:0/1731017259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b900703f0 0x7f7b90100e90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f7b8400b5f0 tx=0x7f7b84006ee0).ready entity=mon.0 client_cookie=ab58c9a0f6da687d server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.236+0000 7f7b8d59a700 1 -- 172.21.15.38:0/1731017259 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7b8400d600 con 0x7f7b900703f0 2024-02-15T18:26:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.236+0000 7f7b8d59a700 1 -- 172.21.15.38:0/1731017259 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f7b8400dbf0 con 0x7f7b900703f0 2024-02-15T18:26:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.236+0000 7f7b8d59a700 1 -- 172.21.15.38:0/1731017259 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7b84011ce0 con 0x7f7b900703f0 2024-02-15T18:26:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.237+0000 7f7b949cc700 1 -- 172.21.15.38:0/1731017259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b900703f0 msgr2=0x7f7b90100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:14.236 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.237+0000 7f7b949cc700 1 --2- 172.21.15.38:0/1731017259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b900703f0 0x7f7b90100e90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f7b8400b5f0 tx=0x7f7b84006ee0).stop 2024-02-15T18:26:14.237 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.237+0000 7f7b949cc700 1 -- 172.21.15.38:0/1731017259 shutdown_connections 2024-02-15T18:26:14.237 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.237+0000 7f7b949cc700 1 --2- 172.21.15.38:0/1731017259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b900703f0 0x7f7b90100e90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:14.237 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.237+0000 7f7b949cc700 1 -- 172.21.15.38:0/1731017259 >> 172.21.15.38:0/1731017259 conn(0x7f7b900f6530 msgr2=0x7f7b900f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:14.237 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.237+0000 7f7b949cc700 1 -- 172.21.15.38:0/1731017259 shutdown_connections 2024-02-15T18:26:14.237 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.237+0000 7f7b949cc700 1 -- 172.21.15.38:0/1731017259 wait complete. 2024-02-15T18:26:14.237 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.238+0000 7f7b949cc700 1 Processor -- start 2024-02-15T18:26:14.237 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.238+0000 7f7b949cc700 1 -- start start 2024-02-15T18:26:14.240 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.238+0000 7f7b949cc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b900703f0 0x7f7b9010cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:14.240 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.238+0000 7f7b949cc700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9010d290 con 0x7f7b900703f0 2024-02-15T18:26:14.240 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.238+0000 7f7b8e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b900703f0 0x7f7b9010cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:14.240 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.238+0000 7f7b8e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b900703f0 0x7f7b9010cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33162/0 (socket says 172.21.15.38:33162) 2024-02-15T18:26:14.240 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.238+0000 7f7b8e59c700 1 -- 172.21.15.38:0/2053537600 learned_addr learned my addr 172.21.15.38:0/2053537600 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:14.240 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.238+0000 7f7b8e59c700 1 -- 172.21.15.38:0/2053537600 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8400b200 con 0x7f7b900703f0 2024-02-15T18:26:14.240 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.238+0000 7f7b8e59c700 1 --2- 172.21.15.38:0/2053537600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b900703f0 0x7f7b9010cda0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f7b84011ed0 tx=0x7f7b84006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:14.241 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.239+0000 7f7b7effd700 1 -- 172.21.15.38:0/2053537600 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7b8401b570 con 0x7f7b900703f0 2024-02-15T18:26:14.241 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.239+0000 7f7b7effd700 1 -- 172.21.15.38:0/2053537600 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f7b8401bb60 con 0x7f7b900703f0 2024-02-15T18:26:14.241 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.239+0000 7f7b949cc700 1 -- 172.21.15.38:0/2053537600 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9010d490 con 0x7f7b900703f0 2024-02-15T18:26:14.241 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.239+0000 7f7b7effd700 1 -- 172.21.15.38:0/2053537600 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7b8401aad0 con 0x7f7b900703f0 2024-02-15T18:26:14.241 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.239+0000 7f7b949cc700 1 -- 172.21.15.38:0/2053537600 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9010e0d0 con 0x7f7b900703f0 2024-02-15T18:26:14.241 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.240+0000 7f7b7effd700 1 -- 172.21.15.38:0/2053537600 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f7b8400f070 con 0x7f7b900703f0 2024-02-15T18:26:14.241 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.240+0000 7f7b7effd700 1 --2- 172.21.15.38:0/2053537600 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7f7b7402fe70 0x7f7b74032330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:14.241 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.240+0000 7f7b7effd700 1 -- 172.21.15.38:0/2053537600 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f7b84041430 con 0x7f7b900703f0 2024-02-15T18:26:14.241 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.240+0000 7f7b949cc700 1 -- 172.21.15.38:0/2053537600 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b9010d920 con 0x7f7b900703f0 2024-02-15T18:26:14.241 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.240+0000 7f7b8dd9b700 1 --2- 172.21.15.38:0/2053537600 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7f7b7402fe70 0x7f7b74032330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:14.241 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.240+0000 7f7b8dd9b700 1 --2- 172.21.15.38:0/2053537600 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7f7b7402fe70 0x7f7b74032330 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f7b800096a0 tx=0x7f7b80006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:14.242 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.245+0000 7f7b7effd700 1 -- 172.21.15.38:0/2053537600 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f7b84016070 con 0x7f7b900703f0 2024-02-15T18:26:14.428 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.432+0000 7f7b949cc700 1 -- 172.21.15.38:0/2053537600 --> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b90102180 con 0x7f7b7402fe70 2024-02-15T18:26:14.437 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.440+0000 7f7b7effd700 1 -- 172.21.15.38:0/2053537600 <== mgr.14114 v2:172.21.15.38:6800/3009942445 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7b90102180 con 0x7f7b7402fe70 2024-02-15T18:26:14.437 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: Scheduled mgr update... 2024-02-15T18:26:14.439 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.442+0000 7f7b949cc700 1 -- 172.21.15.38:0/2053537600 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7f7b7402fe70 msgr2=0x7f7b74032330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:14.439 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.442+0000 7f7b949cc700 1 --2- 172.21.15.38:0/2053537600 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7f7b7402fe70 0x7f7b74032330 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f7b800096a0 tx=0x7f7b80006b40).stop 2024-02-15T18:26:14.440 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.442+0000 7f7b949cc700 1 -- 172.21.15.38:0/2053537600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b900703f0 msgr2=0x7f7b9010cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:14.440 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.442+0000 7f7b949cc700 1 --2- 172.21.15.38:0/2053537600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b900703f0 0x7f7b9010cda0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f7b84011ed0 tx=0x7f7b84006ee0).stop 2024-02-15T18:26:14.440 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.442+0000 7f7b949cc700 1 -- 172.21.15.38:0/2053537600 shutdown_connections 2024-02-15T18:26:14.440 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.442+0000 7f7b949cc700 1 --2- 172.21.15.38:0/2053537600 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7f7b7402fe70 0x7f7b74032330 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:14.440 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.442+0000 7f7b949cc700 1 --2- 172.21.15.38:0/2053537600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b900703f0 0x7f7b9010cda0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:14.440 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.442+0000 7f7b949cc700 1 -- 172.21.15.38:0/2053537600 >> 172.21.15.38:0/2053537600 conn(0x7f7b900f6530 msgr2=0x7f7b900f7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:14.440 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.442+0000 7f7b949cc700 1 -- 172.21.15.38:0/2053537600 shutdown_connections 2024-02-15T18:26:14.440 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:14.442+0000 7f7b949cc700 1 -- 172.21.15.38:0/2053537600 wait complete. 2024-02-15T18:26:14.459 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:13.030436+0000 mgr.smithi038.khesbl (mgr.14114) 10 : audit [DBG] from='client.14134 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:14.459 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:26:13.031148+0000 mgr.smithi038.khesbl (mgr.14114) 11 : cephadm [INF] Saving service mon spec with placement count:5 2024-02-15T18:26:14.459 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:13.033409+0000 mon.smithi038 (mon.0) 56 : audit [INF] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:15.127 INFO:teuthology.orchestra.run.smithi038.stderr:Deploying crash service with default placement... 2024-02-15T18:26:15.127 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply crash 2024-02-15T18:26:15.681 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.669+0000 7fdad91f7700 1 Processor -- start 2024-02-15T18:26:15.681 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.669+0000 7fdad91f7700 1 -- start start 2024-02-15T18:26:15.681 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.670+0000 7fdad91f7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdad4070430 0x7fdad40fea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:15.682 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.670+0000 7fdad91f7700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad40fef40 con 0x7fdad4070430 2024-02-15T18:26:15.682 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.670+0000 7fdad2d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdad4070430 0x7fdad40fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:15.682 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.670+0000 7fdad2d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdad4070430 0x7fdad40fea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33170/0 (socket says 172.21.15.38:33170) 2024-02-15T18:26:15.682 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.670+0000 7fdad2d9d700 1 -- 172.21.15.38:0/2174312153 learned_addr learned my addr 172.21.15.38:0/2174312153 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:15.682 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.670+0000 7fdad2d9d700 1 -- 172.21.15.38:0/2174312153 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad40ff080 con 0x7fdad4070430 2024-02-15T18:26:15.682 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.670+0000 7fdad2d9d700 1 --2- 172.21.15.38:0/2174312153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdad4070430 0x7fdad40fea50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fdac800b5f0 tx=0x7fdac8006ee0).ready entity=mon.0 client_cookie=214110bf6861399e server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:15.682 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.670+0000 7fdad1d9b700 1 -- 172.21.15.38:0/2174312153 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdac800d600 con 0x7fdad4070430 2024-02-15T18:26:15.682 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.670+0000 7fdad1d9b700 1 -- 172.21.15.38:0/2174312153 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fdac800dbf0 con 0x7fdad4070430 2024-02-15T18:26:15.682 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.670+0000 7fdad1d9b700 1 -- 172.21.15.38:0/2174312153 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdac8011cb0 con 0x7fdad4070430 2024-02-15T18:26:15.682 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.671+0000 7fdad91f7700 1 -- 172.21.15.38:0/2174312153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdad4070430 msgr2=0x7fdad40fea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:15.682 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.671+0000 7fdad91f7700 1 --2- 172.21.15.38:0/2174312153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdad4070430 0x7fdad40fea50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fdac800b5f0 tx=0x7fdac8006ee0).stop 2024-02-15T18:26:15.682 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.671+0000 7fdad91f7700 1 -- 172.21.15.38:0/2174312153 shutdown_connections 2024-02-15T18:26:15.682 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.671+0000 7fdad91f7700 1 --2- 172.21.15.38:0/2174312153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdad4070430 0x7fdad40fea50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:15.682 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.671+0000 7fdad91f7700 1 -- 172.21.15.38:0/2174312153 >> 172.21.15.38:0/2174312153 conn(0x7fdad40f6530 msgr2=0x7fdad40f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:15.682 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.671+0000 7fdad91f7700 1 -- 172.21.15.38:0/2174312153 shutdown_connections 2024-02-15T18:26:15.682 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.671+0000 7fdad91f7700 1 -- 172.21.15.38:0/2174312153 wait complete. 2024-02-15T18:26:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.672+0000 7fdad91f7700 1 Processor -- start 2024-02-15T18:26:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.672+0000 7fdad91f7700 1 -- start start 2024-02-15T18:26:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.672+0000 7fdad91f7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdad410ad40 0x7fdad410b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.672+0000 7fdad91f7700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad410b650 con 0x7fdad410ad40 2024-02-15T18:26:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.672+0000 7fdad2d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdad410ad40 0x7fdad410b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.672+0000 7fdad2d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdad410ad40 0x7fdad410b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33184/0 (socket says 172.21.15.38:33184) 2024-02-15T18:26:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.672+0000 7fdad2d9d700 1 -- 172.21.15.38:0/2881207681 learned_addr learned my addr 172.21.15.38:0/2881207681 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.672+0000 7fdad2d9d700 1 -- 172.21.15.38:0/2881207681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac800b200 con 0x7fdad410ad40 2024-02-15T18:26:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.672+0000 7fdad2d9d700 1 --2- 172.21.15.38:0/2881207681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdad410ad40 0x7fdad410b160 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fdac800b5c0 tx=0x7fdac8006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.672+0000 7fdabb7fe700 1 -- 172.21.15.38:0/2881207681 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdac8011400 con 0x7fdad410ad40 2024-02-15T18:26:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.672+0000 7fdad91f7700 1 -- 172.21.15.38:0/2881207681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad410b850 con 0x7fdad410ad40 2024-02-15T18:26:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.672+0000 7fdabb7fe700 1 -- 172.21.15.38:0/2881207681 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fdac801b8a0 con 0x7fdad410ad40 2024-02-15T18:26:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.672+0000 7fdabb7fe700 1 -- 172.21.15.38:0/2881207681 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdac801a7c0 con 0x7fdad410ad40 2024-02-15T18:26:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.672+0000 7fdad91f7700 1 -- 172.21.15.38:0/2881207681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad410c2f0 con 0x7fdad410ad40 2024-02-15T18:26:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.673+0000 7fdabb7fe700 1 -- 172.21.15.38:0/2881207681 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fdac800f070 con 0x7fdad410ad40 2024-02-15T18:26:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.673+0000 7fdabb7fe700 1 --2- 172.21.15.38:0/2881207681 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fdac002fe10 0x7fdac00322d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:15.684 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.673+0000 7fdad91f7700 1 -- 172.21.15.38:0/2881207681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad4059100 con 0x7fdad410ad40 2024-02-15T18:26:15.684 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.673+0000 7fdabb7fe700 1 -- 172.21.15.38:0/2881207681 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fdac8041200 con 0x7fdad410ad40 2024-02-15T18:26:15.684 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.673+0000 7fdad259c700 1 --2- 172.21.15.38:0/2881207681 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fdac002fe10 0x7fdac00322d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:15.684 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.674+0000 7fdad259c700 1 --2- 172.21.15.38:0/2881207681 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fdac002fe10 0x7fdac00322d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fdac40096a0 tx=0x7fdac4006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:15.684 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.678+0000 7fdabb7fe700 1 -- 172.21.15.38:0/2881207681 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fdac8022310 con 0x7fdad410ad40 2024-02-15T18:26:15.779 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:14.433211+0000 mgr.smithi038.khesbl (mgr.14114) 12 : audit [DBG] from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:15.779 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:26:14.433711+0000 mgr.smithi038.khesbl (mgr.14114) 13 : cephadm [INF] Saving service mgr spec with placement count:2 2024-02-15T18:26:15.779 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:14.440893+0000 mon.smithi038 (mon.0) 57 : audit [INF] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:15.858 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.861+0000 7fdad91f7700 1 -- 172.21.15.38:0/2881207681 --> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7fdad4104d00 con 0x7fdac002fe10 2024-02-15T18:26:15.867 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.870+0000 7fdabb7fe700 1 -- 172.21.15.38:0/2881207681 <== mgr.14114 v2:172.21.15.38:6800/3009942445 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fdac40092a0 con 0x7fdac002fe10 2024-02-15T18:26:15.867 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: Scheduled crash update... 2024-02-15T18:26:15.870 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.872+0000 7fdad91f7700 1 -- 172.21.15.38:0/2881207681 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fdac002fe10 msgr2=0x7fdac00322d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:15.870 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.872+0000 7fdad91f7700 1 --2- 172.21.15.38:0/2881207681 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fdac002fe10 0x7fdac00322d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fdac40096a0 tx=0x7fdac4006b40).stop 2024-02-15T18:26:15.870 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.872+0000 7fdad91f7700 1 -- 172.21.15.38:0/2881207681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdad410ad40 msgr2=0x7fdad410b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:15.870 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.872+0000 7fdad91f7700 1 --2- 172.21.15.38:0/2881207681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdad410ad40 0x7fdad410b160 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fdac800b5c0 tx=0x7fdac8006ee0).stop 2024-02-15T18:26:15.870 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.872+0000 7fdad91f7700 1 -- 172.21.15.38:0/2881207681 shutdown_connections 2024-02-15T18:26:15.870 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.872+0000 7fdad91f7700 1 --2- 172.21.15.38:0/2881207681 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fdac002fe10 0x7fdac00322d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:15.870 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.872+0000 7fdad91f7700 1 --2- 172.21.15.38:0/2881207681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdad410ad40 0x7fdad410b160 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:15.870 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.872+0000 7fdad91f7700 1 -- 172.21.15.38:0/2881207681 >> 172.21.15.38:0/2881207681 conn(0x7fdad40f6530 msgr2=0x7fdad40f7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:15.870 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.872+0000 7fdad91f7700 1 -- 172.21.15.38:0/2881207681 shutdown_connections 2024-02-15T18:26:15.870 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:15.872+0000 7fdad91f7700 1 -- 172.21.15.38:0/2881207681 wait complete. 2024-02-15T18:26:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:Enabling mgr prometheus module... 2024-02-15T18:26:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 mgr module enable prometheus 2024-02-15T18:26:17.191 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.194+0000 7efdc4468700 1 Processor -- start 2024-02-15T18:26:17.192 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.194+0000 7efdc4468700 1 -- start start 2024-02-15T18:26:17.192 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.194+0000 7efdc4468700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdbc0fbea0 0x7efdbc0fc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:17.192 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.194+0000 7efdc4468700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc0fc7b0 con 0x7efdbc0fbea0 2024-02-15T18:26:17.193 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.195+0000 7efdc2204700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdbc0fbea0 0x7efdbc0fc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:17.193 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.195+0000 7efdc2204700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdbc0fbea0 0x7efdbc0fc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33192/0 (socket says 172.21.15.38:33192) 2024-02-15T18:26:17.193 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.195+0000 7efdc2204700 1 -- 172.21.15.38:0/735364431 learned_addr learned my addr 172.21.15.38:0/735364431 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:17.193 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.195+0000 7efdc2204700 1 -- 172.21.15.38:0/735364431 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc0fc8f0 con 0x7efdbc0fbea0 2024-02-15T18:26:17.193 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.195+0000 7efdc2204700 1 --2- 172.21.15.38:0/735364431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdbc0fbea0 0x7efdbc0fc2c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7efdac00b5f0 tx=0x7efdac006ee0).ready entity=mon.0 client_cookie=f89e7ebc78aee925 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:17.194 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.196+0000 7efdc1202700 1 -- 172.21.15.38:0/735364431 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efdac00d600 con 0x7efdbc0fbea0 2024-02-15T18:26:17.194 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.196+0000 7efdc1202700 1 -- 172.21.15.38:0/735364431 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7efdac00dbf0 con 0x7efdbc0fbea0 2024-02-15T18:26:17.194 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.196+0000 7efdc1202700 1 -- 172.21.15.38:0/735364431 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efdac011cb0 con 0x7efdbc0fbea0 2024-02-15T18:26:17.194 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.197+0000 7efdc4468700 1 -- 172.21.15.38:0/735364431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdbc0fbea0 msgr2=0x7efdbc0fc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:17.194 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.197+0000 7efdc4468700 1 --2- 172.21.15.38:0/735364431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdbc0fbea0 0x7efdbc0fc2c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7efdac00b5f0 tx=0x7efdac006ee0).stop 2024-02-15T18:26:17.196 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.197+0000 7efdc4468700 1 -- 172.21.15.38:0/735364431 shutdown_connections 2024-02-15T18:26:17.196 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.197+0000 7efdc4468700 1 --2- 172.21.15.38:0/735364431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdbc0fbea0 0x7efdbc0fc2c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:17.196 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.197+0000 7efdc4468700 1 -- 172.21.15.38:0/735364431 >> 172.21.15.38:0/735364431 conn(0x7efdbc0f76a0 msgr2=0x7efdbc0f9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:17.197 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.197+0000 7efdc4468700 1 -- 172.21.15.38:0/735364431 shutdown_connections 2024-02-15T18:26:17.197 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.197+0000 7efdc4468700 1 -- 172.21.15.38:0/735364431 wait complete. 2024-02-15T18:26:17.197 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.198+0000 7efdc4468700 1 Processor -- start 2024-02-15T18:26:17.197 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.198+0000 7efdc4468700 1 -- start start 2024-02-15T18:26:17.197 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.198+0000 7efdc4468700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdbc0fbea0 0x7efdbc10ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:17.197 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.198+0000 7efdc4468700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc10b080 con 0x7efdbc0fbea0 2024-02-15T18:26:17.197 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.198+0000 7efdc2204700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdbc0fbea0 0x7efdbc10ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:17.197 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.198+0000 7efdc2204700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdbc0fbea0 0x7efdbc10ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33194/0 (socket says 172.21.15.38:33194) 2024-02-15T18:26:17.197 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.198+0000 7efdc2204700 1 -- 172.21.15.38:0/1851720389 learned_addr learned my addr 172.21.15.38:0/1851720389 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:17.197 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.198+0000 7efdc2204700 1 -- 172.21.15.38:0/1851720389 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdac00b200 con 0x7efdbc0fbea0 2024-02-15T18:26:17.197 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.198+0000 7efdc2204700 1 --2- 172.21.15.38:0/1851720389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdbc0fbea0 0x7efdbc10ab90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7efdac00bf40 tx=0x7efdac006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:17.197 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.199+0000 7efdaaffd700 1 -- 172.21.15.38:0/1851720389 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efdac01b570 con 0x7efdbc0fbea0 2024-02-15T18:26:17.197 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.199+0000 7efdc4468700 1 -- 172.21.15.38:0/1851720389 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdbc10b280 con 0x7efdbc0fbea0 2024-02-15T18:26:17.197 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.199+0000 7efdaaffd700 1 -- 172.21.15.38:0/1851720389 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7efdac01bb60 con 0x7efdbc0fbea0 2024-02-15T18:26:17.197 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.199+0000 7efdaaffd700 1 -- 172.21.15.38:0/1851720389 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efdac01aa20 con 0x7efdbc0fbea0 2024-02-15T18:26:17.198 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.199+0000 7efdc4468700 1 -- 172.21.15.38:0/1851720389 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdbc10b710 con 0x7efdbc0fbea0 2024-02-15T18:26:17.198 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.199+0000 7efdaaffd700 1 -- 172.21.15.38:0/1851720389 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7efdac00f070 con 0x7efdbc0fbea0 2024-02-15T18:26:17.198 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.200+0000 7efdaaffd700 1 --2- 172.21.15.38:0/1851720389 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7efda402fe10 0x7efda40322d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:17.198 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.200+0000 7efdaaffd700 1 -- 172.21.15.38:0/1851720389 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7efdac042320 con 0x7efdbc0fbea0 2024-02-15T18:26:17.198 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.200+0000 7efdc1a03700 1 --2- 172.21.15.38:0/1851720389 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7efda402fe10 0x7efda40322d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:17.198 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.200+0000 7efdc4468700 1 -- 172.21.15.38:0/1851720389 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdbc045ad0 con 0x7efdbc0fbea0 2024-02-15T18:26:17.198 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.200+0000 7efdc1a03700 1 --2- 172.21.15.38:0/1851720389 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7efda402fe10 0x7efda40322d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7efdb80096a0 tx=0x7efdb8006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:17.201 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.205+0000 7efdaaffd700 1 -- 172.21.15.38:0/1851720389 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7efdac016070 con 0x7efdbc0fbea0 2024-02-15T18:26:17.279 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:15.862982+0000 mgr.smithi038.khesbl (mgr.14114) 14 : audit [DBG] from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:17.279 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:26:15.863641+0000 mgr.smithi038.khesbl (mgr.14114) 15 : cephadm [INF] Saving service crash spec with placement * 2024-02-15T18:26:17.279 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:15.870959+0000 mon.smithi038 (mon.0) 58 : audit [INF] from='mgr.14114 172.21.15.38:0/1128835090' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:17.398 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.401+0000 7efdc4468700 1 -- 172.21.15.38:0/1851720389 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "prometheus"} v 0) v1 -- 0x7efdbc104bb0 con 0x7efdbc0fbea0 2024-02-15T18:26:17.876 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.878+0000 7efdaaffd700 1 -- 172.21.15.38:0/1851720389 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 9) v1 ==== 36134+0+0 (secure 0 0 0) 0x7efdac01bcd0 con 0x7efdbc0fbea0 2024-02-15T18:26:17.876 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.879+0000 7efdaaffd700 1 -- 172.21.15.38:0/1851720389 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "prometheus"}]=0 v9) v1 ==== 89+0+0 (secure 0 0 0) 0x7efdac022ac0 con 0x7efdbc0fbea0 2024-02-15T18:26:17.879 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.881+0000 7efdc4468700 1 -- 172.21.15.38:0/1851720389 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7efda402fe10 msgr2=0x7efda40322d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:17.879 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.881+0000 7efdc4468700 1 --2- 172.21.15.38:0/1851720389 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7efda402fe10 0x7efda40322d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7efdb80096a0 tx=0x7efdb8006b40).stop 2024-02-15T18:26:17.879 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.881+0000 7efdc4468700 1 -- 172.21.15.38:0/1851720389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdbc0fbea0 msgr2=0x7efdbc10ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:17.879 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.881+0000 7efdc4468700 1 --2- 172.21.15.38:0/1851720389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdbc0fbea0 0x7efdbc10ab90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7efdac00bf40 tx=0x7efdac006ee0).stop 2024-02-15T18:26:17.879 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.882+0000 7efdc4468700 1 -- 172.21.15.38:0/1851720389 shutdown_connections 2024-02-15T18:26:17.880 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.882+0000 7efdc4468700 1 --2- 172.21.15.38:0/1851720389 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7efda402fe10 0x7efda40322d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:17.880 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.882+0000 7efdc4468700 1 --2- 172.21.15.38:0/1851720389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdbc0fbea0 0x7efdbc10ab90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:17.880 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.882+0000 7efdc4468700 1 -- 172.21.15.38:0/1851720389 >> 172.21.15.38:0/1851720389 conn(0x7efdbc0f76a0 msgr2=0x7efdbc0f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:17.880 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.882+0000 7efdc4468700 1 -- 172.21.15.38:0/1851720389 shutdown_connections 2024-02-15T18:26:17.880 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:17.882+0000 7efdc4468700 1 -- 172.21.15.38:0/1851720389 wait complete. 2024-02-15T18:26:18.279 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:17.402243+0000 mon.smithi038 (mon.0) 59 : audit [INF] from='client.? 172.21.15.38:0/1851720389' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-15T18:26:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:Deploying prometheus service with default placement... 2024-02-15T18:26:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply prometheus 2024-02-15T18:26:19.279 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:17.880213+0000 mon.smithi038 (mon.0) 60 : audit [INF] from='client.? 172.21.15.38:0/1851720389' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-15T18:26:19.279 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:17.880301+0000 mon.smithi038 (mon.0) 61 : cluster [DBG] mgrmap e9: smithi038.khesbl(active, since 15s) 2024-02-15T18:26:19.716 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.244+0000 7fed37bec700 1 Processor -- start 2024-02-15T18:26:19.716 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.244+0000 7fed37bec700 1 -- start start 2024-02-15T18:26:19.716 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.244+0000 7fed37bec700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed300fc4b0 0x7fed300fc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:19.717 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.244+0000 7fed37bec700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed300fcdc0 con 0x7fed300fc4b0 2024-02-15T18:26:19.717 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.245+0000 7fed35988700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed300fc4b0 0x7fed300fc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:19.717 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.245+0000 7fed35988700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed300fc4b0 0x7fed300fc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33234/0 (socket says 172.21.15.38:33234) 2024-02-15T18:26:19.717 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.245+0000 7fed35988700 1 -- 172.21.15.38:0/3931154306 learned_addr learned my addr 172.21.15.38:0/3931154306 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:19.717 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.245+0000 7fed35988700 1 -- 172.21.15.38:0/3931154306 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed300fcf00 con 0x7fed300fc4b0 2024-02-15T18:26:19.717 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.245+0000 7fed35988700 1 --2- 172.21.15.38:0/3931154306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed300fc4b0 0x7fed300fc8d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fed2c00b5f0 tx=0x7fed2c006ee0).ready entity=mon.0 client_cookie=41c4ace2a7532b08 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:19.717 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.245+0000 7fed34986700 1 -- 172.21.15.38:0/3931154306 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed2c00d600 con 0x7fed300fc4b0 2024-02-15T18:26:19.717 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.245+0000 7fed34986700 1 -- 172.21.15.38:0/3931154306 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fed2c00dbf0 con 0x7fed300fc4b0 2024-02-15T18:26:19.717 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.245+0000 7fed34986700 1 -- 172.21.15.38:0/3931154306 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed2c011ce0 con 0x7fed300fc4b0 2024-02-15T18:26:19.717 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.246+0000 7fed37bec700 1 -- 172.21.15.38:0/3931154306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed300fc4b0 msgr2=0x7fed300fc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:19.717 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.246+0000 7fed37bec700 1 --2- 172.21.15.38:0/3931154306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed300fc4b0 0x7fed300fc8d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fed2c00b5f0 tx=0x7fed2c006ee0).stop 2024-02-15T18:26:19.717 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.246+0000 7fed37bec700 1 -- 172.21.15.38:0/3931154306 shutdown_connections 2024-02-15T18:26:19.717 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.246+0000 7fed37bec700 1 --2- 172.21.15.38:0/3931154306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed300fc4b0 0x7fed300fc8d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:19.717 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.246+0000 7fed37bec700 1 -- 172.21.15.38:0/3931154306 >> 172.21.15.38:0/3931154306 conn(0x7fed300f5d20 msgr2=0x7fed300f8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:19.718 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.246+0000 7fed37bec700 1 -- 172.21.15.38:0/3931154306 shutdown_connections 2024-02-15T18:26:19.718 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.246+0000 7fed37bec700 1 -- 172.21.15.38:0/3931154306 wait complete. 2024-02-15T18:26:19.718 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.246+0000 7fed37bec700 1 Processor -- start 2024-02-15T18:26:19.718 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.246+0000 7fed37bec700 1 -- start start 2024-02-15T18:26:19.718 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.247+0000 7fed37bec700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed300fc4b0 0x7fed3010ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:19.718 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.247+0000 7fed37bec700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3010d310 con 0x7fed300fc4b0 2024-02-15T18:26:19.718 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.247+0000 7fed35988700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed300fc4b0 0x7fed3010ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:19.718 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.247+0000 7fed35988700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed300fc4b0 0x7fed3010ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33250/0 (socket says 172.21.15.38:33250) 2024-02-15T18:26:19.718 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.247+0000 7fed35988700 1 -- 172.21.15.38:0/1442816312 learned_addr learned my addr 172.21.15.38:0/1442816312 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:19.718 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.247+0000 7fed35988700 1 -- 172.21.15.38:0/1442816312 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c00b200 con 0x7fed300fc4b0 2024-02-15T18:26:19.718 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.247+0000 7fed35988700 1 --2- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed300fc4b0 0x7fed3010ce20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fed2c002f40 tx=0x7fed2c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:19.718 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.247+0000 7fed1e7fc700 1 -- 172.21.15.38:0/1442816312 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed2c01b570 con 0x7fed300fc4b0 2024-02-15T18:26:19.718 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.247+0000 7fed1e7fc700 1 -- 172.21.15.38:0/1442816312 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fed2c01bb60 con 0x7fed300fc4b0 2024-02-15T18:26:19.718 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.247+0000 7fed37bec700 1 -- 172.21.15.38:0/1442816312 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3010d510 con 0x7fed300fc4b0 2024-02-15T18:26:19.719 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.247+0000 7fed1e7fc700 1 -- 172.21.15.38:0/1442816312 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed2c01ab10 con 0x7fed300fc4b0 2024-02-15T18:26:19.719 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.247+0000 7fed37bec700 1 -- 172.21.15.38:0/1442816312 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3010e120 con 0x7fed300fc4b0 2024-02-15T18:26:19.719 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.248+0000 7fed1e7fc700 1 -- 172.21.15.38:0/1442816312 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 9) v1 ==== 36134+0+0 (secure 0 0 0) 0x7fed2c00f070 con 0x7fed300fc4b0 2024-02-15T18:26:19.719 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.248+0000 7fed37bec700 1 -- 172.21.15.38:0/1442816312 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed30059100 con 0x7fed300fc4b0 2024-02-15T18:26:19.719 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.249+0000 7fed1e7fc700 1 --2- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fed1802ff10 0x7fed180323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:19.719 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.249+0000 7fed1e7fc700 1 -- 172.21.15.38:0/1442816312 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fed2c0411e0 con 0x7fed300fc4b0 2024-02-15T18:26:19.719 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.249+0000 7fed35187700 1 -- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fed1802ff10 msgr2=0x7fed180323d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3009942445 2024-02-15T18:26:19.719 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.249+0000 7fed35187700 1 --2- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fed1802ff10 0x7fed180323d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-15T18:26:19.719 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.253+0000 7fed1e7fc700 1 -- 172.21.15.38:0/1442816312 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fed2c019cd0 con 0x7fed300fc4b0 2024-02-15T18:26:19.719 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.442+0000 7fed37bec700 1 -- 172.21.15.38:0/1442816312 --> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fed3010e360 con 0x7fed1802ff10 2024-02-15T18:26:19.719 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.450+0000 7fed35187700 1 -- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fed1802ff10 msgr2=0x7fed180323d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3009942445 2024-02-15T18:26:19.719 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.450+0000 7fed35187700 1 --2- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fed1802ff10 0x7fed180323d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-15T18:26:19.847 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.851+0000 7fed35187700 1 -- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fed1802ff10 msgr2=0x7fed180323d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3009942445 2024-02-15T18:26:19.848 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:19.851+0000 7fed35187700 1 --2- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fed1802ff10 0x7fed180323d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-02-15T18:26:20.648 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:20.652+0000 7fed35187700 1 -- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fed1802ff10 msgr2=0x7fed180323d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3009942445 2024-02-15T18:26:20.649 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:20.652+0000 7fed35187700 1 --2- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fed1802ff10 0x7fed180323d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-02-15T18:26:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:22.253+0000 7fed35187700 1 -- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fed1802ff10 msgr2=0x7fed180323d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/3009942445 2024-02-15T18:26:22.251 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:22.253+0000 7fed35187700 1 --2- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fed1802ff10 0x7fed180323d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2024-02-15T18:26:23.151 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:23.154+0000 7fed1e7fc700 1 -- 172.21.15.38:0/1442816312 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 10) v1 ==== 35984+0+0 (secure 0 0 0) 0x7fed2c019300 con 0x7fed300fc4b0 2024-02-15T18:26:23.151 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:23.154+0000 7fed1e7fc700 1 -- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fed1802ff10 msgr2=0x7fed180323d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-15T18:26:23.151 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:23.154+0000 7fed1e7fc700 1 --2- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fed1802ff10 0x7fed180323d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:26:23.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:23.102268+0000 mon.smithi038 (mon.0) 62 : cluster [INF] Active manager daemon smithi038.khesbl restarted 2024-02-15T18:26:23.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:23.102843+0000 mon.smithi038 (mon.0) 63 : cluster [INF] Activating manager daemon smithi038.khesbl 2024-02-15T18:26:23.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:23.108970+0000 mon.smithi038 (mon.0) 64 : cluster [DBG] osdmap e3: 0 total, 0 up, 0 in 2024-02-15T18:26:24.165 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:24.167+0000 7fed1e7fc700 1 -- 172.21.15.38:0/1442816312 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mgrmap(e 11) v1 ==== 36054+0+0 (secure 0 0 0) 0x7fed2c03d250 con 0x7fed300fc4b0 2024-02-15T18:26:24.165 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:24.167+0000 7fed1e7fc700 1 --2- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fed18033d30 0x7fed18036120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:24.165 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:24.167+0000 7fed1e7fc700 1 -- 172.21.15.38:0/1442816312 --> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fed3010e360 con 0x7fed18033d30 2024-02-15T18:26:24.165 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:24.167+0000 7fed35187700 1 --2- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fed18033d30 0x7fed18036120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:24.165 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:24.168+0000 7fed35187700 1 --2- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fed18033d30 0x7fed18036120 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fed20006fd0 tx=0x7fed20006db0).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:24.171 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:24.174+0000 7fed1e7fc700 1 -- 172.21.15.38:0/1442816312 <== mgr.14142 v2:172.21.15.38:6800/2095530547 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fed3010e360 con 0x7fed18033d30 2024-02-15T18:26:24.171 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: Scheduled prometheus update... 2024-02-15T18:26:24.174 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:24.177+0000 7fed37bec700 1 -- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fed18033d30 msgr2=0x7fed18036120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:24.174 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:24.177+0000 7fed37bec700 1 --2- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fed18033d30 0x7fed18036120 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fed20006fd0 tx=0x7fed20006db0).stop 2024-02-15T18:26:24.174 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:24.177+0000 7fed37bec700 1 -- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed300fc4b0 msgr2=0x7fed3010ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:24.175 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:24.177+0000 7fed37bec700 1 --2- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed300fc4b0 0x7fed3010ce20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fed2c002f40 tx=0x7fed2c006ee0).stop 2024-02-15T18:26:24.175 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:24.177+0000 7fed37bec700 1 -- 172.21.15.38:0/1442816312 shutdown_connections 2024-02-15T18:26:24.175 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:24.177+0000 7fed37bec700 1 --2- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fed18033d30 0x7fed18036120 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:24.175 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:24.177+0000 7fed37bec700 1 --2- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed300fc4b0 0x7fed3010ce20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:24.175 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:24.177+0000 7fed37bec700 1 --2- 172.21.15.38:0/1442816312 >> [v2:172.21.15.38:6800/3009942445,v1:172.21.15.38:6801/3009942445] conn(0x7fed1802ff10 0x7fed180323d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:26:24.175 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:24.177+0000 7fed37bec700 1 -- 172.21.15.38:0/1442816312 >> 172.21.15.38:0/1442816312 conn(0x7fed300f5d20 msgr2=0x7fed300f7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:24.175 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:24.177+0000 7fed37bec700 1 -- 172.21.15.38:0/1442816312 shutdown_connections 2024-02-15T18:26:24.175 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:24.177+0000 7fed37bec700 1 -- 172.21.15.38:0/1442816312 wait complete. 2024-02-15T18:26:24.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:23.155746+0000 mon.smithi038 (mon.0) 65 : cluster [DBG] mgrmap e10: smithi038.khesbl(active, starting, since 0.0530212s) 2024-02-15T18:26:24.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:23.158393+0000 mon.smithi038 (mon.0) 66 : audit [DBG] from='mgr.14142 172.21.15.38:0/1079131341' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-02-15T18:26:24.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:23.158923+0000 mon.smithi038 (mon.0) 67 : audit [DBG] from='mgr.14142 172.21.15.38:0/1079131341' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mgr metadata", "who": "smithi038.khesbl", "id": "smithi038.khesbl"}]: dispatch 2024-02-15T18:26:24.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:23.159616+0000 mon.smithi038 (mon.0) 68 : audit [DBG] from='mgr.14142 172.21.15.38:0/1079131341' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-15T18:26:24.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:23.159798+0000 mon.smithi038 (mon.0) 69 : audit [DBG] from='mgr.14142 172.21.15.38:0/1079131341' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-15T18:26:24.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:23.159946+0000 mon.smithi038 (mon.0) 70 : audit [DBG] from='mgr.14142 172.21.15.38:0/1079131341' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-15T18:26:24.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:23.161995+0000 mon.smithi038 (mon.0) 71 : cluster [INF] Manager daemon smithi038.khesbl is now available 2024-02-15T18:26:24.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:23.265962+0000 mon.smithi038 (mon.0) 72 : audit [DBG] from='mgr.14142 172.21.15.38:0/1079131341' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:24.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:23.267054+0000 mon.smithi038 (mon.0) 73 : audit [DBG] from='mgr.14142 172.21.15.38:0/1079131341' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:24.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:23.289736+0000 mon.smithi038 (mon.0) 74 : audit [INF] from='mgr.14142 172.21.15.38:0/1079131341' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:26:24.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:23.292883+0000 mon.smithi038 (mon.0) 75 : audit [INF] from='mgr.14142 172.21.15.38:0/1079131341' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:26:24.754 INFO:teuthology.orchestra.run.smithi038.stderr:Deploying grafana service with default placement... 2024-02-15T18:26:24.754 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply grafana 2024-02-15T18:26:25.279 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:24.169683+0000 mon.smithi038 (mon.0) 76 : cluster [DBG] mgrmap e11: smithi038.khesbl(active, since 1.06696s) 2024-02-15T18:26:25.279 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:26:24.170792+0000 mgr.smithi038.khesbl (mgr.14142) 2 : cephadm [INF] Saving service prometheus spec with placement count:1 2024-02-15T18:26:25.279 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:24.174910+0000 mon.smithi038 (mon.0) 77 : audit [INF] from='mgr.14142 172.21.15.38:0/1079131341' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:25.365 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.359+0000 7f980f85f700 1 Processor -- start 2024-02-15T18:26:25.365 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.359+0000 7f980f85f700 1 -- start start 2024-02-15T18:26:25.366 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.360+0000 7f980f85f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98080fae00 0x7f98080fb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:25.366 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.360+0000 7f980f85f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98080fb710 con 0x7f98080fae00 2024-02-15T18:26:25.366 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.360+0000 7f980d5fb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98080fae00 0x7f98080fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:25.366 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.360+0000 7f980d5fb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98080fae00 0x7f98080fb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58370/0 (socket says 172.21.15.38:58370) 2024-02-15T18:26:25.366 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.360+0000 7f980d5fb700 1 -- 172.21.15.38:0/3348909269 learned_addr learned my addr 172.21.15.38:0/3348909269 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:25.366 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.360+0000 7f980d5fb700 1 -- 172.21.15.38:0/3348909269 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98080fb850 con 0x7f98080fae00 2024-02-15T18:26:25.367 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.360+0000 7f980d5fb700 1 --2- 172.21.15.38:0/3348909269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98080fae00 0x7f98080fb220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f97f800b5f0 tx=0x7f97f8006ee0).ready entity=mon.0 client_cookie=886bd8bf1f18bbb0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:25.367 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.360+0000 7f97f7fff700 1 -- 172.21.15.38:0/3348909269 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97f800d600 con 0x7f98080fae00 2024-02-15T18:26:25.367 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.361+0000 7f97f7fff700 1 -- 172.21.15.38:0/3348909269 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f97f800dbf0 con 0x7f98080fae00 2024-02-15T18:26:25.367 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.361+0000 7f97f7fff700 1 -- 172.21.15.38:0/3348909269 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97f8011cb0 con 0x7f98080fae00 2024-02-15T18:26:25.367 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.361+0000 7f980f85f700 1 -- 172.21.15.38:0/3348909269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98080fae00 msgr2=0x7f98080fb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:25.367 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.361+0000 7f980f85f700 1 --2- 172.21.15.38:0/3348909269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98080fae00 0x7f98080fb220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f97f800b5f0 tx=0x7f97f8006ee0).stop 2024-02-15T18:26:25.367 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.361+0000 7f980f85f700 1 -- 172.21.15.38:0/3348909269 shutdown_connections 2024-02-15T18:26:25.367 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.361+0000 7f980f85f700 1 --2- 172.21.15.38:0/3348909269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98080fae00 0x7f98080fb220 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:25.367 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.361+0000 7f980f85f700 1 -- 172.21.15.38:0/3348909269 >> 172.21.15.38:0/3348909269 conn(0x7f98080f65b0 msgr2=0x7f98080f89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:25.367 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.361+0000 7f980f85f700 1 -- 172.21.15.38:0/3348909269 shutdown_connections 2024-02-15T18:26:25.367 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.361+0000 7f980f85f700 1 -- 172.21.15.38:0/3348909269 wait complete. 2024-02-15T18:26:25.367 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.362+0000 7f980f85f700 1 Processor -- start 2024-02-15T18:26:25.367 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.362+0000 7f980f85f700 1 -- start start 2024-02-15T18:26:25.368 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.362+0000 7f980f85f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98080fae00 0x7f980810d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:25.368 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.362+0000 7f980f85f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980806fc30 con 0x7f98080fae00 2024-02-15T18:26:25.368 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.362+0000 7f980d5fb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98080fae00 0x7f980810d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:25.368 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.362+0000 7f980d5fb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98080fae00 0x7f980810d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58384/0 (socket says 172.21.15.38:58384) 2024-02-15T18:26:25.368 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.362+0000 7f980d5fb700 1 -- 172.21.15.38:0/2538877399 learned_addr learned my addr 172.21.15.38:0/2538877399 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:25.368 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.362+0000 7f980d5fb700 1 -- 172.21.15.38:0/2538877399 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f800b200 con 0x7f98080fae00 2024-02-15T18:26:25.368 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.362+0000 7f980d5fb700 1 --2- 172.21.15.38:0/2538877399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98080fae00 0x7f980810d8f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f97f800dfc0 tx=0x7f97f8006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:25.368 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.363+0000 7f97f5ffb700 1 -- 172.21.15.38:0/2538877399 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97f801b410 con 0x7f98080fae00 2024-02-15T18:26:25.368 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.363+0000 7f97f5ffb700 1 -- 172.21.15.38:0/2538877399 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f97f801ba00 con 0x7f98080fae00 2024-02-15T18:26:25.368 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.363+0000 7f980f85f700 1 -- 172.21.15.38:0/2538877399 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980806fe30 con 0x7f98080fae00 2024-02-15T18:26:25.368 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.363+0000 7f97f5ffb700 1 -- 172.21.15.38:0/2538877399 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97f801a990 con 0x7f98080fae00 2024-02-15T18:26:25.368 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.363+0000 7f980f85f700 1 -- 172.21.15.38:0/2538877399 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980810dfa0 con 0x7f98080fae00 2024-02-15T18:26:25.368 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.364+0000 7f97f5ffb700 1 -- 172.21.15.38:0/2538877399 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7f97f800f070 con 0x7f98080fae00 2024-02-15T18:26:25.369 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.364+0000 7f97f5ffb700 1 --2- 172.21.15.38:0/2538877399 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f97f002ff70 0x7f97f0032430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:25.369 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.364+0000 7f97f5ffb700 1 -- 172.21.15.38:0/2538877399 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f97f801bb70 con 0x7f98080fae00 2024-02-15T18:26:25.369 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.364+0000 7f980cdfa700 1 --2- 172.21.15.38:0/2538877399 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f97f002ff70 0x7f97f0032430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:25.369 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.364+0000 7f980f85f700 1 -- 172.21.15.38:0/2538877399 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9808059100 con 0x7f98080fae00 2024-02-15T18:26:25.369 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.364+0000 7f980cdfa700 1 --2- 172.21.15.38:0/2538877399 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f97f002ff70 0x7f97f0032430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f98040096a0 tx=0x7f9804006b40).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:25.369 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.369+0000 7f97f5ffb700 1 -- 172.21.15.38:0/2538877399 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f97f8016070 con 0x7f98080fae00 2024-02-15T18:26:25.551 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.554+0000 7f980f85f700 1 -- 172.21.15.38:0/2538877399 --> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f98080fcb80 con 0x7f97f002ff70 2024-02-15T18:26:25.556 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.560+0000 7f97f5ffb700 1 -- 172.21.15.38:0/2538877399 <== mgr.14142 v2:172.21.15.38:6800/2095530547 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f98080fcb80 con 0x7f97f002ff70 2024-02-15T18:26:25.556 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: Scheduled grafana update... 2024-02-15T18:26:25.559 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.561+0000 7f980f85f700 1 -- 172.21.15.38:0/2538877399 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f97f002ff70 msgr2=0x7f97f0032430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:25.559 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.561+0000 7f980f85f700 1 --2- 172.21.15.38:0/2538877399 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f97f002ff70 0x7f97f0032430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f98040096a0 tx=0x7f9804006b40).stop 2024-02-15T18:26:25.559 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.562+0000 7f980f85f700 1 -- 172.21.15.38:0/2538877399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98080fae00 msgr2=0x7f980810d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:25.559 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.562+0000 7f980f85f700 1 --2- 172.21.15.38:0/2538877399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98080fae00 0x7f980810d8f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f97f800dfc0 tx=0x7f97f8006ee0).stop 2024-02-15T18:26:25.559 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.562+0000 7f980f85f700 1 -- 172.21.15.38:0/2538877399 shutdown_connections 2024-02-15T18:26:25.559 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.562+0000 7f980f85f700 1 --2- 172.21.15.38:0/2538877399 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f97f002ff70 0x7f97f0032430 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:25.559 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.562+0000 7f980f85f700 1 --2- 172.21.15.38:0/2538877399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98080fae00 0x7f980810d8f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:25.559 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.562+0000 7f980f85f700 1 -- 172.21.15.38:0/2538877399 >> 172.21.15.38:0/2538877399 conn(0x7f98080f65b0 msgr2=0x7f98080f7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:25.560 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.562+0000 7f980f85f700 1 -- 172.21.15.38:0/2538877399 shutdown_connections 2024-02-15T18:26:25.560 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:25.562+0000 7f980f85f700 1 -- 172.21.15.38:0/2538877399 wait complete. 2024-02-15T18:26:26.270 INFO:teuthology.orchestra.run.smithi038.stderr:Deploying node-exporter service with default placement... 2024-02-15T18:26:26.270 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply node-exporter 2024-02-15T18:26:26.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:25.205654+0000 mon.smithi038 (mon.0) 78 : cluster [DBG] mgrmap e12: smithi038.khesbl(active, since 2s) 2024-02-15T18:26:26.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:25.555771+0000 mgr.smithi038.khesbl (mgr.14142) 3 : audit [DBG] from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:26.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:26:25.556286+0000 mgr.smithi038.khesbl (mgr.14142) 4 : cephadm [INF] Saving service grafana spec with placement count:1 2024-02-15T18:26:26.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:25.560252+0000 mon.smithi038 (mon.0) 79 : audit [INF] from='mgr.14142 172.21.15.38:0/1079131341' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:26.856 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.825+0000 7fe80dc75700 1 Processor -- start 2024-02-15T18:26:26.857 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.826+0000 7fe80dc75700 1 -- start start 2024-02-15T18:26:26.857 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.826+0000 7fe80dc75700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8080703b0 0x7fe808100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:26.857 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.826+0000 7fe80dc75700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8080fc8c0 con 0x7fe8080703b0 2024-02-15T18:26:26.857 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.826+0000 7fe8077fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8080703b0 0x7fe808100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:26.857 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.826+0000 7fe8077fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8080703b0 0x7fe808100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58398/0 (socket says 172.21.15.38:58398) 2024-02-15T18:26:26.857 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.826+0000 7fe8077fe700 1 -- 172.21.15.38:0/3547191763 learned_addr learned my addr 172.21.15.38:0/3547191763 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:26.857 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.826+0000 7fe8077fe700 1 -- 172.21.15.38:0/3547191763 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8081013c0 con 0x7fe8080703b0 2024-02-15T18:26:26.857 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.826+0000 7fe8077fe700 1 --2- 172.21.15.38:0/3547191763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8080703b0 0x7fe808100ed0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fe800009960 tx=0x7fe8000092d0).ready entity=mon.0 client_cookie=d1b9380e673d074e server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:26.857 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.827+0000 7fe8067fc700 1 -- 172.21.15.38:0/3547191763 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe80000b9c0 con 0x7fe8080703b0 2024-02-15T18:26:26.857 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.827+0000 7fe8067fc700 1 -- 172.21.15.38:0/3547191763 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fe800011400 con 0x7fe8080703b0 2024-02-15T18:26:26.857 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.827+0000 7fe8067fc700 1 -- 172.21.15.38:0/3547191763 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe800010580 con 0x7fe8080703b0 2024-02-15T18:26:26.857 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.827+0000 7fe80dc75700 1 -- 172.21.15.38:0/3547191763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8080703b0 msgr2=0x7fe808100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:26.857 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.827+0000 7fe80dc75700 1 --2- 172.21.15.38:0/3547191763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8080703b0 0x7fe808100ed0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fe800009960 tx=0x7fe8000092d0).stop 2024-02-15T18:26:26.857 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.827+0000 7fe80dc75700 1 -- 172.21.15.38:0/3547191763 shutdown_connections 2024-02-15T18:26:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.827+0000 7fe80dc75700 1 --2- 172.21.15.38:0/3547191763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8080703b0 0x7fe808100ed0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.827+0000 7fe80dc75700 1 -- 172.21.15.38:0/3547191763 >> 172.21.15.38:0/3547191763 conn(0x7fe8080f6530 msgr2=0x7fe8080f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.827+0000 7fe80dc75700 1 -- 172.21.15.38:0/3547191763 shutdown_connections 2024-02-15T18:26:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.827+0000 7fe80dc75700 1 -- 172.21.15.38:0/3547191763 wait complete. 2024-02-15T18:26:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.828+0000 7fe80dc75700 1 Processor -- start 2024-02-15T18:26:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.828+0000 7fe80dc75700 1 -- start start 2024-02-15T18:26:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.828+0000 7fe80dc75700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8080703b0 0x7fe80810cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.828+0000 7fe80dc75700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80810d270 con 0x7fe8080703b0 2024-02-15T18:26:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.828+0000 7fe8077fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8080703b0 0x7fe80810cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.828+0000 7fe8077fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8080703b0 0x7fe80810cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58404/0 (socket says 172.21.15.38:58404) 2024-02-15T18:26:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.828+0000 7fe8077fe700 1 -- 172.21.15.38:0/329002098 learned_addr learned my addr 172.21.15.38:0/329002098 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.829+0000 7fe8077fe700 1 -- 172.21.15.38:0/329002098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe800009570 con 0x7fe8080703b0 2024-02-15T18:26:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.829+0000 7fe8077fe700 1 --2- 172.21.15.38:0/329002098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8080703b0 0x7fe80810cd80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fe80000bd90 tx=0x7fe800011f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:26.859 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.829+0000 7fe7fffff700 1 -- 172.21.15.38:0/329002098 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8000108d0 con 0x7fe8080703b0 2024-02-15T18:26:26.859 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.829+0000 7fe7fffff700 1 -- 172.21.15.38:0/329002098 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fe800010ec0 con 0x7fe8080703b0 2024-02-15T18:26:26.859 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.829+0000 7fe80dc75700 1 -- 172.21.15.38:0/329002098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe80810d470 con 0x7fe8080703b0 2024-02-15T18:26:26.859 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.829+0000 7fe7fffff700 1 -- 172.21.15.38:0/329002098 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe80001ada0 con 0x7fe8080703b0 2024-02-15T18:26:26.859 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.829+0000 7fe80dc75700 1 -- 172.21.15.38:0/329002098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe80810e080 con 0x7fe8080703b0 2024-02-15T18:26:26.859 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.830+0000 7fe7fffff700 1 -- 172.21.15.38:0/329002098 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7fe80000f070 con 0x7fe8080703b0 2024-02-15T18:26:26.859 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.830+0000 7fe7fffff700 1 --2- 172.21.15.38:0/329002098 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fe7e802ff20 0x7fe7e80323e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:26.859 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.830+0000 7fe80dc75700 1 -- 172.21.15.38:0/329002098 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe808059100 con 0x7fe8080703b0 2024-02-15T18:26:26.859 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.830+0000 7fe7fffff700 1 -- 172.21.15.38:0/329002098 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7fe800041c60 con 0x7fe8080703b0 2024-02-15T18:26:26.859 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.830+0000 7fe806ffd700 1 --2- 172.21.15.38:0/329002098 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fe7e802ff20 0x7fe7e80323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:26.859 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.830+0000 7fe806ffd700 1 --2- 172.21.15.38:0/329002098 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fe7e802ff20 0x7fe7e80323e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fe7f00096a0 tx=0x7fe7f0006b40).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:26.859 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:26.834+0000 7fe7fffff700 1 -- 172.21.15.38:0/329002098 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe800021320 con 0x7fe8080703b0 2024-02-15T18:26:27.017 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:27.020+0000 7fe80dc75700 1 -- 172.21.15.38:0/329002098 --> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fe80810e2c0 con 0x7fe7e802ff20 2024-02-15T18:26:27.023 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:27.026+0000 7fe7fffff700 1 -- 172.21.15.38:0/329002098 <== mgr.14142 v2:172.21.15.38:6800/2095530547 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fe80810e2c0 con 0x7fe7e802ff20 2024-02-15T18:26:27.023 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: Scheduled node-exporter update... 2024-02-15T18:26:27.025 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:27.028+0000 7fe80dc75700 1 -- 172.21.15.38:0/329002098 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fe7e802ff20 msgr2=0x7fe7e80323e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:27.026 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:27.028+0000 7fe80dc75700 1 --2- 172.21.15.38:0/329002098 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fe7e802ff20 0x7fe7e80323e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fe7f00096a0 tx=0x7fe7f0006b40).stop 2024-02-15T18:26:27.026 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:27.028+0000 7fe80dc75700 1 -- 172.21.15.38:0/329002098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8080703b0 msgr2=0x7fe80810cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:27.026 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:27.028+0000 7fe80dc75700 1 --2- 172.21.15.38:0/329002098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8080703b0 0x7fe80810cd80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fe80000bd90 tx=0x7fe800011f90).stop 2024-02-15T18:26:27.026 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:27.028+0000 7fe80dc75700 1 -- 172.21.15.38:0/329002098 shutdown_connections 2024-02-15T18:26:27.026 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:27.028+0000 7fe80dc75700 1 --2- 172.21.15.38:0/329002098 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fe7e802ff20 0x7fe7e80323e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:27.026 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:27.028+0000 7fe80dc75700 1 --2- 172.21.15.38:0/329002098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8080703b0 0x7fe80810cd80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:27.026 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:27.028+0000 7fe80dc75700 1 -- 172.21.15.38:0/329002098 >> 172.21.15.38:0/329002098 conn(0x7fe8080f6530 msgr2=0x7fe8080f7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:27.026 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:27.028+0000 7fe80dc75700 1 -- 172.21.15.38:0/329002098 shutdown_connections 2024-02-15T18:26:27.026 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:27.028+0000 7fe80dc75700 1 -- 172.21.15.38:0/329002098 wait complete. 2024-02-15T18:26:27.709 INFO:teuthology.orchestra.run.smithi038.stderr:Deploying alertmanager service with default placement... 2024-02-15T18:26:27.709 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply alertmanager 2024-02-15T18:26:28.279 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:27.022074+0000 mgr.smithi038.khesbl (mgr.14142) 5 : audit [DBG] from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:28.279 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:26:27.022792+0000 mgr.smithi038.khesbl (mgr.14142) 6 : cephadm [INF] Saving service node-exporter spec with placement * 2024-02-15T18:26:28.279 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:27.027054+0000 mon.smithi038 (mon.0) 80 : audit [INF] from='mgr.14142 172.21.15.38:0/1079131341' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:28.357 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.358+0000 7f16caa60700 1 Processor -- start 2024-02-15T18:26:28.358 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.358+0000 7f16caa60700 1 -- start start 2024-02-15T18:26:28.358 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.358+0000 7f16caa60700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16c406fc10 0x7f16c40feb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:28.358 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.358+0000 7f16caa60700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c40ff000 con 0x7f16c406fc10 2024-02-15T18:26:28.358 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.358+0000 7f16c3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16c406fc10 0x7f16c40feb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:28.358 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.358+0000 7f16c3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16c406fc10 0x7f16c40feb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58418/0 (socket says 172.21.15.38:58418) 2024-02-15T18:26:28.358 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.358+0000 7f16c3fff700 1 -- 172.21.15.38:0/329790131 learned_addr learned my addr 172.21.15.38:0/329790131 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:28.358 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.359+0000 7f16c3fff700 1 -- 172.21.15.38:0/329790131 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c40ff140 con 0x7f16c406fc10 2024-02-15T18:26:28.358 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.359+0000 7f16c3fff700 1 --2- 172.21.15.38:0/329790131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16c406fc10 0x7f16c40feb10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f16b400b5f0 tx=0x7f16b4006ee0).ready entity=mon.0 client_cookie=637e2e14540f3c14 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:28.358 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.359+0000 7f16c2ffd700 1 -- 172.21.15.38:0/329790131 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16b400d600 con 0x7f16c406fc10 2024-02-15T18:26:28.358 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.359+0000 7f16c2ffd700 1 -- 172.21.15.38:0/329790131 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f16b400dbf0 con 0x7f16c406fc10 2024-02-15T18:26:28.358 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.359+0000 7f16c2ffd700 1 -- 172.21.15.38:0/329790131 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16b4011cb0 con 0x7f16c406fc10 2024-02-15T18:26:28.358 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.359+0000 7f16caa60700 1 -- 172.21.15.38:0/329790131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16c406fc10 msgr2=0x7f16c40feb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:28.359 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.359+0000 7f16caa60700 1 --2- 172.21.15.38:0/329790131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16c406fc10 0x7f16c40feb10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f16b400b5f0 tx=0x7f16b4006ee0).stop 2024-02-15T18:26:28.359 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.360+0000 7f16caa60700 1 -- 172.21.15.38:0/329790131 shutdown_connections 2024-02-15T18:26:28.359 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.360+0000 7f16caa60700 1 --2- 172.21.15.38:0/329790131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16c406fc10 0x7f16c40feb10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:28.359 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.360+0000 7f16caa60700 1 -- 172.21.15.38:0/329790131 >> 172.21.15.38:0/329790131 conn(0x7f16c40f65b0 msgr2=0x7f16c40f89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:28.359 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.360+0000 7f16caa60700 1 -- 172.21.15.38:0/329790131 shutdown_connections 2024-02-15T18:26:28.359 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.360+0000 7f16caa60700 1 -- 172.21.15.38:0/329790131 wait complete. 2024-02-15T18:26:28.359 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.360+0000 7f16caa60700 1 Processor -- start 2024-02-15T18:26:28.359 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.360+0000 7f16caa60700 1 -- start start 2024-02-15T18:26:28.359 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.361+0000 7f16caa60700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16c406fc10 0x7f16c40fc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:28.359 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.361+0000 7f16caa60700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c40faa10 con 0x7f16c406fc10 2024-02-15T18:26:28.359 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.361+0000 7f16c3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16c406fc10 0x7f16c40fc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:28.359 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.361+0000 7f16c3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16c406fc10 0x7f16c40fc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58422/0 (socket says 172.21.15.38:58422) 2024-02-15T18:26:28.359 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.361+0000 7f16c3fff700 1 -- 172.21.15.38:0/1143835891 learned_addr learned my addr 172.21.15.38:0/1143835891 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:28.359 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.361+0000 7f16c3fff700 1 -- 172.21.15.38:0/1143835891 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b400b200 con 0x7f16c406fc10 2024-02-15T18:26:28.360 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.361+0000 7f16c3fff700 1 --2- 172.21.15.38:0/1143835891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16c406fc10 0x7f16c40fc590 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f16b4006800 tx=0x7f16b4006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:28.361 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.361+0000 7f16c0ff9700 1 -- 172.21.15.38:0/1143835891 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16b401b570 con 0x7f16c406fc10 2024-02-15T18:26:28.361 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.361+0000 7f16c0ff9700 1 -- 172.21.15.38:0/1143835891 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f16b401bb60 con 0x7f16c406fc10 2024-02-15T18:26:28.361 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.361+0000 7f16caa60700 1 -- 172.21.15.38:0/1143835891 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c40fabb0 con 0x7f16c406fc10 2024-02-15T18:26:28.361 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.361+0000 7f16c0ff9700 1 -- 172.21.15.38:0/1143835891 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16b401a9f0 con 0x7f16c406fc10 2024-02-15T18:26:28.361 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.361+0000 7f16caa60700 1 -- 172.21.15.38:0/1143835891 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c40fb040 con 0x7f16c406fc10 2024-02-15T18:26:28.361 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.362+0000 7f16c0ff9700 1 -- 172.21.15.38:0/1143835891 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7f16b400f070 con 0x7f16c406fc10 2024-02-15T18:26:28.361 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.362+0000 7f16c0ff9700 1 --2- 172.21.15.38:0/1143835891 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f16b002ff20 0x7f16b00323e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:28.361 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.362+0000 7f16caa60700 1 -- 172.21.15.38:0/1143835891 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c4045ad0 con 0x7f16c406fc10 2024-02-15T18:26:28.361 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.362+0000 7f16c0ff9700 1 -- 172.21.15.38:0/1143835891 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f16b4041630 con 0x7f16c406fc10 2024-02-15T18:26:28.361 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.363+0000 7f16c37fe700 1 --2- 172.21.15.38:0/1143835891 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f16b002ff20 0x7f16b00323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:28.361 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.363+0000 7f16c37fe700 1 --2- 172.21.15.38:0/1143835891 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f16b002ff20 0x7f16b00323e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f16b80096a0 tx=0x7f16b8006b40).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:28.363 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.367+0000 7f16c0ff9700 1 -- 172.21.15.38:0/1143835891 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f16b4022b00 con 0x7f16c406fc10 2024-02-15T18:26:28.548 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.551+0000 7f16caa60700 1 -- 172.21.15.38:0/1143835891 --> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f16c40fff90 con 0x7f16b002ff20 2024-02-15T18:26:28.554 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.557+0000 7f16c0ff9700 1 -- 172.21.15.38:0/1143835891 <== mgr.14142 v2:172.21.15.38:6800/2095530547 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f16c40fff90 con 0x7f16b002ff20 2024-02-15T18:26:28.554 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: Scheduled alertmanager update... 2024-02-15T18:26:28.556 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.559+0000 7f16caa60700 1 -- 172.21.15.38:0/1143835891 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f16b002ff20 msgr2=0x7f16b00323e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:28.556 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.559+0000 7f16caa60700 1 --2- 172.21.15.38:0/1143835891 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f16b002ff20 0x7f16b00323e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f16b80096a0 tx=0x7f16b8006b40).stop 2024-02-15T18:26:28.557 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.559+0000 7f16caa60700 1 -- 172.21.15.38:0/1143835891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16c406fc10 msgr2=0x7f16c40fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:28.557 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.559+0000 7f16caa60700 1 --2- 172.21.15.38:0/1143835891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16c406fc10 0x7f16c40fc590 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f16b4006800 tx=0x7f16b4006ee0).stop 2024-02-15T18:26:28.557 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.559+0000 7f16caa60700 1 -- 172.21.15.38:0/1143835891 shutdown_connections 2024-02-15T18:26:28.557 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.559+0000 7f16caa60700 1 --2- 172.21.15.38:0/1143835891 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f16b002ff20 0x7f16b00323e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:28.557 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.559+0000 7f16caa60700 1 --2- 172.21.15.38:0/1143835891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16c406fc10 0x7f16c40fc590 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:28.557 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.559+0000 7f16caa60700 1 -- 172.21.15.38:0/1143835891 >> 172.21.15.38:0/1143835891 conn(0x7f16c40f65b0 msgr2=0x7f16c40f7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:28.557 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.559+0000 7f16caa60700 1 -- 172.21.15.38:0/1143835891 shutdown_connections 2024-02-15T18:26:28.557 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:28.559+0000 7f16caa60700 1 -- 172.21.15.38:0/1143835891 wait complete. 2024-02-15T18:26:29.175 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 config set mgr mgr/cephadm/container_init True --force 2024-02-15T18:26:29.756 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.759+0000 7f99eeece700 1 Processor -- start 2024-02-15T18:26:29.756 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.759+0000 7f99eeece700 1 -- start start 2024-02-15T18:26:29.757 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.759+0000 7f99eeece700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e80703f0 0x7f99e80fea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:29.757 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.759+0000 7f99eeece700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e80fef80 con 0x7f99e80703f0 2024-02-15T18:26:29.757 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.759+0000 7f99ecc6a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e80703f0 0x7f99e80fea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:29.757 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.759+0000 7f99ecc6a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e80703f0 0x7f99e80fea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58436/0 (socket says 172.21.15.38:58436) 2024-02-15T18:26:29.757 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.759+0000 7f99ecc6a700 1 -- 172.21.15.38:0/1483709371 learned_addr learned my addr 172.21.15.38:0/1483709371 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:29.757 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.759+0000 7f99ecc6a700 1 -- 172.21.15.38:0/1483709371 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e80ff0c0 con 0x7f99e80703f0 2024-02-15T18:26:29.759 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.759+0000 7f99ecc6a700 1 --2- 172.21.15.38:0/1483709371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e80703f0 0x7f99e80fea90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f99dc013e00 tx=0x7f99dc006ee0).ready entity=mon.0 client_cookie=a0091874ea276085 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:29.759 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.760+0000 7f99e77fe700 1 -- 172.21.15.38:0/1483709371 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99dc009e90 con 0x7f99e80703f0 2024-02-15T18:26:29.759 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.760+0000 7f99e77fe700 1 -- 172.21.15.38:0/1483709371 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f99dc012d70 con 0x7f99e80703f0 2024-02-15T18:26:29.759 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.760+0000 7f99e77fe700 1 -- 172.21.15.38:0/1483709371 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99dc01c620 con 0x7f99e80703f0 2024-02-15T18:26:29.759 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.760+0000 7f99eeece700 1 -- 172.21.15.38:0/1483709371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e80703f0 msgr2=0x7f99e80fea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:29.759 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.760+0000 7f99eeece700 1 --2- 172.21.15.38:0/1483709371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e80703f0 0x7f99e80fea90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f99dc013e00 tx=0x7f99dc006ee0).stop 2024-02-15T18:26:29.760 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.760+0000 7f99eeece700 1 -- 172.21.15.38:0/1483709371 shutdown_connections 2024-02-15T18:26:29.760 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.760+0000 7f99eeece700 1 --2- 172.21.15.38:0/1483709371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e80703f0 0x7f99e80fea90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:29.760 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.760+0000 7f99eeece700 1 -- 172.21.15.38:0/1483709371 >> 172.21.15.38:0/1483709371 conn(0x7f99e80f6530 msgr2=0x7f99e80f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:29.760 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.761+0000 7f99eeece700 1 -- 172.21.15.38:0/1483709371 shutdown_connections 2024-02-15T18:26:29.760 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.761+0000 7f99eeece700 1 -- 172.21.15.38:0/1483709371 wait complete. 2024-02-15T18:26:29.760 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.762+0000 7f99eeece700 1 Processor -- start 2024-02-15T18:26:29.760 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.762+0000 7f99eeece700 1 -- start start 2024-02-15T18:26:29.760 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.762+0000 7f99eeece700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e80703f0 0x7f99e8108920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:29.760 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.762+0000 7f99eeece700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e8108e10 con 0x7f99e80703f0 2024-02-15T18:26:29.760 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.762+0000 7f99ecc6a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e80703f0 0x7f99e8108920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:29.760 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.762+0000 7f99ecc6a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e80703f0 0x7f99e8108920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58448/0 (socket says 172.21.15.38:58448) 2024-02-15T18:26:29.760 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.762+0000 7f99ecc6a700 1 -- 172.21.15.38:0/2209777497 learned_addr learned my addr 172.21.15.38:0/2209777497 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:29.761 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.763+0000 7f99ecc6a700 1 -- 172.21.15.38:0/2209777497 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99dc013a10 con 0x7f99e80703f0 2024-02-15T18:26:29.762 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.763+0000 7f99ecc6a700 1 --2- 172.21.15.38:0/2209777497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e80703f0 0x7f99e8108920 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f99dc0135d0 tx=0x7f99dc006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:29.762 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.763+0000 7f99e57fa700 1 -- 172.21.15.38:0/2209777497 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99dc01c7f0 con 0x7f99e80703f0 2024-02-15T18:26:29.762 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.763+0000 7f99eeece700 1 -- 172.21.15.38:0/2209777497 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e8109010 con 0x7f99e80703f0 2024-02-15T18:26:29.762 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.763+0000 7f99e57fa700 1 -- 172.21.15.38:0/2209777497 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f99dc01cde0 con 0x7f99e80703f0 2024-02-15T18:26:29.762 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.763+0000 7f99e57fa700 1 -- 172.21.15.38:0/2209777497 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99dc025de0 con 0x7f99e80703f0 2024-02-15T18:26:29.762 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.763+0000 7f99eeece700 1 -- 172.21.15.38:0/2209777497 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e81094a0 con 0x7f99e80703f0 2024-02-15T18:26:29.762 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.764+0000 7f99eeece700 1 -- 172.21.15.38:0/2209777497 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e8045ad0 con 0x7f99e80703f0 2024-02-15T18:26:29.762 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.764+0000 7f99e57fa700 1 -- 172.21.15.38:0/2209777497 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7f99dc01c950 con 0x7f99e80703f0 2024-02-15T18:26:29.762 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.764+0000 7f99e57fa700 1 --2- 172.21.15.38:0/2209777497 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f99cc02ff20 0x7f99cc0323e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:29.762 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.764+0000 7f99e57fa700 1 -- 172.21.15.38:0/2209777497 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f99dc049500 con 0x7f99e80703f0 2024-02-15T18:26:29.762 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.764+0000 7f99e7fff700 1 --2- 172.21.15.38:0/2209777497 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f99cc02ff20 0x7f99cc0323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:29.762 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.765+0000 7f99e7fff700 1 --2- 172.21.15.38:0/2209777497 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f99cc02ff20 0x7f99cc0323e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f99d40096a0 tx=0x7f99d4006b40).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:29.765 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.769+0000 7f99e57fa700 1 -- 172.21.15.38:0/2209777497 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f99dc01cbe0 con 0x7f99e80703f0 2024-02-15T18:26:29.778 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:28.553204+0000 mgr.smithi038.khesbl (mgr.14142) 7 : audit [DBG] from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:29.779 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:26:28.553743+0000 mgr.smithi038.khesbl (mgr.14142) 8 : cephadm [INF] Saving service alertmanager spec with placement count:1 2024-02-15T18:26:29.779 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:28.557994+0000 mon.smithi038 (mon.0) 81 : audit [INF] from='mgr.14142 172.21.15.38:0/1079131341' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:29.924 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.927+0000 7f99eeece700 1 -- 172.21.15.38:0/2209777497 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f99e8102a00 con 0x7f99e80703f0 2024-02-15T18:26:29.927 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.930+0000 7f99e57fa700 1 -- 172.21.15.38:0/2209777497 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v8)=0 v8) v1 ==== 142+0+0 (secure 0 0 0) 0x7f99dc0237b0 con 0x7f99e80703f0 2024-02-15T18:26:29.930 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.932+0000 7f99eeece700 1 -- 172.21.15.38:0/2209777497 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f99cc02ff20 msgr2=0x7f99cc0323e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:29.930 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.932+0000 7f99eeece700 1 --2- 172.21.15.38:0/2209777497 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f99cc02ff20 0x7f99cc0323e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f99d40096a0 tx=0x7f99d4006b40).stop 2024-02-15T18:26:29.930 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.932+0000 7f99eeece700 1 -- 172.21.15.38:0/2209777497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e80703f0 msgr2=0x7f99e8108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:29.930 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.932+0000 7f99eeece700 1 --2- 172.21.15.38:0/2209777497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e80703f0 0x7f99e8108920 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f99dc0135d0 tx=0x7f99dc006ee0).stop 2024-02-15T18:26:29.930 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.932+0000 7f99eeece700 1 -- 172.21.15.38:0/2209777497 shutdown_connections 2024-02-15T18:26:29.930 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.932+0000 7f99eeece700 1 --2- 172.21.15.38:0/2209777497 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f99cc02ff20 0x7f99cc0323e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:29.930 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.932+0000 7f99eeece700 1 --2- 172.21.15.38:0/2209777497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e80703f0 0x7f99e8108920 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:29.930 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.932+0000 7f99eeece700 1 -- 172.21.15.38:0/2209777497 >> 172.21.15.38:0/2209777497 conn(0x7f99e80f6530 msgr2=0x7f99e80fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:29.930 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.933+0000 7f99eeece700 1 -- 172.21.15.38:0/2209777497 shutdown_connections 2024-02-15T18:26:29.930 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:29.933+0000 7f99eeece700 1 -- 172.21.15.38:0/2209777497 wait complete. 2024-02-15T18:26:30.816 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 config set mgr mgr/dashboard/ssl_server_port 8443 2024-02-15T18:26:31.160 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:29.931177+0000 mon.smithi038 (mon.0) 82 : audit [INF] from='client.? 172.21.15.38:0/2209777497' entity='client.admin' 2024-02-15T18:26:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.522+0000 7f846ca1a700 1 Processor -- start 2024-02-15T18:26:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.522+0000 7f846ca1a700 1 -- start start 2024-02-15T18:26:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.523+0000 7f846ca1a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84680fc4e0 0x7f84680fc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.523+0000 7f846ca1a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84680fcdf0 con 0x7f84680fc4e0 2024-02-15T18:26:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.523+0000 7f846659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84680fc4e0 0x7f84680fc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.523+0000 7f846659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84680fc4e0 0x7f84680fc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58450/0 (socket says 172.21.15.38:58450) 2024-02-15T18:26:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.523+0000 7f846659c700 1 -- 172.21.15.38:0/4029929571 learned_addr learned my addr 172.21.15.38:0/4029929571 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.523+0000 7f846659c700 1 -- 172.21.15.38:0/4029929571 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84680fcf30 con 0x7f84680fc4e0 2024-02-15T18:26:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.523+0000 7f846659c700 1 --2- 172.21.15.38:0/4029929571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84680fc4e0 0x7f84680fc900 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f8460009960 tx=0x7f84600092d0).ready entity=mon.0 client_cookie=b6ad394ef268c668 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.523+0000 7f846559a700 1 -- 172.21.15.38:0/4029929571 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f846000b9c0 con 0x7f84680fc4e0 2024-02-15T18:26:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.523+0000 7f846559a700 1 -- 172.21.15.38:0/4029929571 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f8460011400 con 0x7f84680fc4e0 2024-02-15T18:26:31.635 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.523+0000 7f846559a700 1 -- 172.21.15.38:0/4029929571 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8460010580 con 0x7f84680fc4e0 2024-02-15T18:26:31.635 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.524+0000 7f846ca1a700 1 -- 172.21.15.38:0/4029929571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84680fc4e0 msgr2=0x7f84680fc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:31.635 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.524+0000 7f846ca1a700 1 --2- 172.21.15.38:0/4029929571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84680fc4e0 0x7f84680fc900 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f8460009960 tx=0x7f84600092d0).stop 2024-02-15T18:26:31.635 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.524+0000 7f846ca1a700 1 -- 172.21.15.38:0/4029929571 shutdown_connections 2024-02-15T18:26:31.635 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.524+0000 7f846ca1a700 1 --2- 172.21.15.38:0/4029929571 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84680fc4e0 0x7f84680fc900 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:31.635 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.524+0000 7f846ca1a700 1 -- 172.21.15.38:0/4029929571 >> 172.21.15.38:0/4029929571 conn(0x7f84680f5d30 msgr2=0x7f84680f8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:31.635 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.524+0000 7f846ca1a700 1 -- 172.21.15.38:0/4029929571 shutdown_connections 2024-02-15T18:26:31.635 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.524+0000 7f846ca1a700 1 -- 172.21.15.38:0/4029929571 wait complete. 2024-02-15T18:26:31.635 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.525+0000 7f846ca1a700 1 Processor -- start 2024-02-15T18:26:31.635 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.525+0000 7f846ca1a700 1 -- start start 2024-02-15T18:26:31.635 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.525+0000 7f846ca1a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84680fc4e0 0x7f846810cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:31.636 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.525+0000 7f846ca1a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846810d290 con 0x7f84680fc4e0 2024-02-15T18:26:31.636 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.525+0000 7f846659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84680fc4e0 0x7f846810cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:31.636 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.525+0000 7f846659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84680fc4e0 0x7f846810cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58462/0 (socket says 172.21.15.38:58462) 2024-02-15T18:26:31.636 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.525+0000 7f846659c700 1 -- 172.21.15.38:0/2673918563 learned_addr learned my addr 172.21.15.38:0/2673918563 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:31.636 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.525+0000 7f846659c700 1 -- 172.21.15.38:0/2673918563 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8460009570 con 0x7f84680fc4e0 2024-02-15T18:26:31.636 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.525+0000 7f846659c700 1 --2- 172.21.15.38:0/2673918563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84680fc4e0 0x7f846810cda0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f846000bac0 tx=0x7f8460011f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:31.636 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.525+0000 7f845effd700 1 -- 172.21.15.38:0/2673918563 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84600107f0 con 0x7f84680fc4e0 2024-02-15T18:26:31.636 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.525+0000 7f846ca1a700 1 -- 172.21.15.38:0/2673918563 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f846810d490 con 0x7f84680fc4e0 2024-02-15T18:26:31.636 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.525+0000 7f845effd700 1 -- 172.21.15.38:0/2673918563 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f8460010de0 con 0x7f84680fc4e0 2024-02-15T18:26:31.636 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.525+0000 7f845effd700 1 -- 172.21.15.38:0/2673918563 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f846001ae70 con 0x7f84680fc4e0 2024-02-15T18:26:31.636 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.525+0000 7f846ca1a700 1 -- 172.21.15.38:0/2673918563 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846810e0d0 con 0x7f84680fc4e0 2024-02-15T18:26:31.636 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.526+0000 7f845effd700 1 -- 172.21.15.38:0/2673918563 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7f846000f070 con 0x7f84680fc4e0 2024-02-15T18:26:31.637 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.526+0000 7f845effd700 1 --2- 172.21.15.38:0/2673918563 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f844802fec0 0x7f8448032380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:31.637 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.526+0000 7f845effd700 1 -- 172.21.15.38:0/2673918563 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f8460041d50 con 0x7f84680fc4e0 2024-02-15T18:26:31.637 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.526+0000 7f846ca1a700 1 -- 172.21.15.38:0/2673918563 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f846810d920 con 0x7f84680fc4e0 2024-02-15T18:26:31.637 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.526+0000 7f8465d9b700 1 --2- 172.21.15.38:0/2673918563 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f844802fec0 0x7f8448032380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:31.637 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.527+0000 7f8465d9b700 1 --2- 172.21.15.38:0/2673918563 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f844802fec0 0x7f8448032380 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f84500096a0 tx=0x7f8450006b40).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:31.637 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.535+0000 7f845effd700 1 -- 172.21.15.38:0/2673918563 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f846001e020 con 0x7f84680fc4e0 2024-02-15T18:26:31.696 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.700+0000 7f846ca1a700 1 -- 172.21.15.38:0/2673918563 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f8468059100 con 0x7f84680fc4e0 2024-02-15T18:26:31.700 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.703+0000 7f845effd700 1 -- 172.21.15.38:0/2673918563 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v9)=0 v9) v1 ==== 130+0+0 (secure 0 0 0) 0x7f846002c3f0 con 0x7f84680fc4e0 2024-02-15T18:26:31.703 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.705+0000 7f846ca1a700 1 -- 172.21.15.38:0/2673918563 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f844802fec0 msgr2=0x7f8448032380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:31.703 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.705+0000 7f846ca1a700 1 --2- 172.21.15.38:0/2673918563 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f844802fec0 0x7f8448032380 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f84500096a0 tx=0x7f8450006b40).stop 2024-02-15T18:26:31.703 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.705+0000 7f846ca1a700 1 -- 172.21.15.38:0/2673918563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84680fc4e0 msgr2=0x7f846810cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:31.703 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.705+0000 7f846ca1a700 1 --2- 172.21.15.38:0/2673918563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84680fc4e0 0x7f846810cda0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f846000bac0 tx=0x7f8460011f90).stop 2024-02-15T18:26:31.703 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.706+0000 7f846ca1a700 1 -- 172.21.15.38:0/2673918563 shutdown_connections 2024-02-15T18:26:31.703 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.706+0000 7f846ca1a700 1 --2- 172.21.15.38:0/2673918563 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f844802fec0 0x7f8448032380 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:31.703 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.706+0000 7f846ca1a700 1 --2- 172.21.15.38:0/2673918563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84680fc4e0 0x7f846810cda0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:31.704 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.706+0000 7f846ca1a700 1 -- 172.21.15.38:0/2673918563 >> 172.21.15.38:0/2673918563 conn(0x7f84680f5d30 msgr2=0x7f84680f76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:31.704 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.706+0000 7f846ca1a700 1 -- 172.21.15.38:0/2673918563 shutdown_connections 2024-02-15T18:26:31.704 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:31.706+0000 7f846ca1a700 1 -- 172.21.15.38:0/2673918563 wait complete. 2024-02-15T18:26:32.699 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:31.704209+0000 mon.smithi038 (mon.0) 83 : audit [INF] from='client.? 172.21.15.38:0/2673918563' entity='client.admin' 2024-02-15T18:26:32.699 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:32.480045+0000 mon.smithi038 (mon.0) 84 : audit [INF] from='mgr.14142 172.21.15.38:0/1079131341' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:33.141 INFO:teuthology.orchestra.run.smithi038.stderr:Enabling the dashboard module... 2024-02-15T18:26:33.141 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 mgr module enable dashboard 2024-02-15T18:26:33.647 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.649+0000 7fe83759e700 1 Processor -- start 2024-02-15T18:26:33.647 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.649+0000 7fe83759e700 1 -- start start 2024-02-15T18:26:33.647 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.649+0000 7fe83759e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8380703b0 0x7fe838100d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:33.647 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.649+0000 7fe83759e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8380fc700 con 0x7fe8380703b0 2024-02-15T18:26:33.647 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.649+0000 7fe83659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8380703b0 0x7fe838100d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:33.647 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.649+0000 7fe83659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8380703b0 0x7fe838100d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49824/0 (socket says 172.21.15.38:49824) 2024-02-15T18:26:33.647 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.649+0000 7fe83659c700 1 -- 172.21.15.38:0/1676547928 learned_addr learned my addr 172.21.15.38:0/1676547928 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:33.647 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.650+0000 7fe83659c700 1 -- 172.21.15.38:0/1676547928 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe838101200 con 0x7fe8380703b0 2024-02-15T18:26:33.647 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.650+0000 7fe83659c700 1 --2- 172.21.15.38:0/1676547928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8380703b0 0x7fe838100d10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fe82800b5f0 tx=0x7fe828006ee0).ready entity=mon.0 client_cookie=3d8a6f812114425c server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:33.647 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.650+0000 7fe83559a700 1 -- 172.21.15.38:0/1676547928 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe82800d600 con 0x7fe8380703b0 2024-02-15T18:26:33.648 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.650+0000 7fe83559a700 1 -- 172.21.15.38:0/1676547928 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe82800dc30 con 0x7fe8380703b0 2024-02-15T18:26:33.648 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.650+0000 7fe83559a700 1 -- 172.21.15.38:0/1676547928 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe828011d30 con 0x7fe8380703b0 2024-02-15T18:26:33.648 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.651+0000 7fe83759e700 1 -- 172.21.15.38:0/1676547928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8380703b0 msgr2=0x7fe838100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:33.649 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.651+0000 7fe83759e700 1 --2- 172.21.15.38:0/1676547928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8380703b0 0x7fe838100d10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fe82800b5f0 tx=0x7fe828006ee0).stop 2024-02-15T18:26:33.649 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.651+0000 7fe83759e700 1 -- 172.21.15.38:0/1676547928 shutdown_connections 2024-02-15T18:26:33.649 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.651+0000 7fe83759e700 1 --2- 172.21.15.38:0/1676547928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8380703b0 0x7fe838100d10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:33.649 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.651+0000 7fe83759e700 1 -- 172.21.15.38:0/1676547928 >> 172.21.15.38:0/1676547928 conn(0x7fe8380f6390 msgr2=0x7fe8380f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:33.649 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.651+0000 7fe83759e700 1 -- 172.21.15.38:0/1676547928 shutdown_connections 2024-02-15T18:26:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.651+0000 7fe83759e700 1 -- 172.21.15.38:0/1676547928 wait complete. 2024-02-15T18:26:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.651+0000 7fe83759e700 1 Processor -- start 2024-02-15T18:26:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.651+0000 7fe83759e700 1 -- start start 2024-02-15T18:26:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.652+0000 7fe83759e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8380703b0 0x7fe83810cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.652+0000 7fe83759e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83810d0b0 con 0x7fe8380703b0 2024-02-15T18:26:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.652+0000 7fe83659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8380703b0 0x7fe83810cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.652+0000 7fe83659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8380703b0 0x7fe83810cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49826/0 (socket says 172.21.15.38:49826) 2024-02-15T18:26:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.652+0000 7fe83659c700 1 -- 172.21.15.38:0/422691 learned_addr learned my addr 172.21.15.38:0/422691 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.652+0000 7fe83659c700 1 -- 172.21.15.38:0/422691 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82800b200 con 0x7fe8380703b0 2024-02-15T18:26:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.652+0000 7fe83659c700 1 --2- 172.21.15.38:0/422691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8380703b0 0x7fe83810cbc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fe82800e040 tx=0x7fe828006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.652+0000 7fe826ffd700 1 -- 172.21.15.38:0/422691 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe82800da60 con 0x7fe8380703b0 2024-02-15T18:26:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.653+0000 7fe826ffd700 1 -- 172.21.15.38:0/422691 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe82801b8e0 con 0x7fe8380703b0 2024-02-15T18:26:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.653+0000 7fe83759e700 1 -- 172.21.15.38:0/422691 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83810d2b0 con 0x7fe8380703b0 2024-02-15T18:26:33.651 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.653+0000 7fe826ffd700 1 -- 172.21.15.38:0/422691 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe82801a900 con 0x7fe8380703b0 2024-02-15T18:26:33.651 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.653+0000 7fe83759e700 1 -- 172.21.15.38:0/422691 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83810d740 con 0x7fe8380703b0 2024-02-15T18:26:33.652 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.653+0000 7fe83759e700 1 -- 172.21.15.38:0/422691 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe838045ad0 con 0x7fe8380703b0 2024-02-15T18:26:33.652 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.654+0000 7fe826ffd700 1 -- 172.21.15.38:0/422691 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7fe82800f070 con 0x7fe8380703b0 2024-02-15T18:26:33.652 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.654+0000 7fe826ffd700 1 --2- 172.21.15.38:0/422691 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fe81802ff70 0x7fe818032430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:33.652 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.654+0000 7fe826ffd700 1 -- 172.21.15.38:0/422691 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7fe8280418c0 con 0x7fe8380703b0 2024-02-15T18:26:33.652 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.655+0000 7fe835d9b700 1 --2- 172.21.15.38:0/422691 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fe81802ff70 0x7fe818032430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:33.652 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.655+0000 7fe835d9b700 1 --2- 172.21.15.38:0/422691 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fe81802ff70 0x7fe818032430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fe8200096a0 tx=0x7fe820006b40).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:33.654 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.658+0000 7fe826ffd700 1 -- 172.21.15.38:0/422691 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe828010360 con 0x7fe8380703b0 2024-02-15T18:26:33.858 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:33.861+0000 7fe83759e700 1 -- 172.21.15.38:0/422691 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fe838101ef0 con 0x7fe8380703b0 2024-02-15T18:26:34.279 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:32.789215+0000 mon.smithi038 (mon.0) 85 : audit [INF] from='mgr.14142 172.21.15.38:0/1079131341' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:34.792 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:34.796+0000 7fe826ffd700 1 -- 172.21.15.38:0/422691 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 13) v1 ==== 36208+0+0 (secure 0 0 0) 0x7fe82801ba50 con 0x7fe8380703b0 2024-02-15T18:26:34.793 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:34.797+0000 7fe826ffd700 1 -- 172.21.15.38:0/422691 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v13) v1 ==== 88+0+0 (secure 0 0 0) 0x7fe82802d960 con 0x7fe8380703b0 2024-02-15T18:26:34.796 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:34.800+0000 7fe83759e700 1 -- 172.21.15.38:0/422691 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fe81802ff70 msgr2=0x7fe818032430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:34.796 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:34.800+0000 7fe83759e700 1 --2- 172.21.15.38:0/422691 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fe81802ff70 0x7fe818032430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fe8200096a0 tx=0x7fe820006b40).stop 2024-02-15T18:26:34.797 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:34.800+0000 7fe83759e700 1 -- 172.21.15.38:0/422691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8380703b0 msgr2=0x7fe83810cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:34.797 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:34.800+0000 7fe83759e700 1 --2- 172.21.15.38:0/422691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8380703b0 0x7fe83810cbc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fe82800e040 tx=0x7fe828006ee0).stop 2024-02-15T18:26:34.797 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:34.800+0000 7fe83759e700 1 -- 172.21.15.38:0/422691 shutdown_connections 2024-02-15T18:26:34.798 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:34.800+0000 7fe83759e700 1 --2- 172.21.15.38:0/422691 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fe81802ff70 0x7fe818032430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:34.798 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:34.800+0000 7fe83759e700 1 --2- 172.21.15.38:0/422691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8380703b0 0x7fe83810cbc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:34.798 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:34.800+0000 7fe83759e700 1 -- 172.21.15.38:0/422691 >> 172.21.15.38:0/422691 conn(0x7fe8380f6390 msgr2=0x7fe8380f7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:34.798 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:34.800+0000 7fe83759e700 1 -- 172.21.15.38:0/422691 shutdown_connections 2024-02-15T18:26:34.798 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:34.800+0000 7fe83759e700 1 -- 172.21.15.38:0/422691 wait complete. 2024-02-15T18:26:35.209 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:33.861943+0000 mon.smithi038 (mon.0) 86 : audit [INF] from='client.? 172.21.15.38:0/422691' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-15T18:26:35.209 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:34.432483+0000 mon.smithi038 (mon.0) 87 : audit [INF] from='mgr.14142 172.21.15.38:0/1079131341' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:35.580 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 mgr stat 2024-02-15T18:26:35.848 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:34.797972+0000 mon.smithi038 (mon.0) 88 : audit [INF] from='client.? 172.21.15.38:0/422691' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-15T18:26:35.848 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:34.798017+0000 mon.smithi038 (mon.0) 89 : cluster [DBG] mgrmap e13: smithi038.khesbl(active, since 11s) 2024-02-15T18:26:36.423 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.426+0000 7f60b00b6700 1 Processor -- start 2024-02-15T18:26:36.424 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.426+0000 7f60b00b6700 1 -- start start 2024-02-15T18:26:36.424 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.426+0000 7f60b00b6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60a80703f0 0x7f60a80fea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:36.424 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.426+0000 7f60b00b6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a80fef80 con 0x7f60a80703f0 2024-02-15T18:26:36.424 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.426+0000 7f60ade52700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60a80703f0 0x7f60a80fea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:36.424 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.426+0000 7f60ade52700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60a80703f0 0x7f60a80fea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49862/0 (socket says 172.21.15.38:49862) 2024-02-15T18:26:36.424 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.426+0000 7f60ade52700 1 -- 172.21.15.38:0/1559239035 learned_addr learned my addr 172.21.15.38:0/1559239035 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:36.424 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.427+0000 7f60ade52700 1 -- 172.21.15.38:0/1559239035 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a80ff0c0 con 0x7f60a80703f0 2024-02-15T18:26:36.424 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.427+0000 7f60ade52700 1 --2- 172.21.15.38:0/1559239035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60a80703f0 0x7f60a80fea90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f60a400b5f0 tx=0x7f60a4006ee0).ready entity=mon.0 client_cookie=b2812b48327f1f1a server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:36.424 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.427+0000 7f60ace50700 1 -- 172.21.15.38:0/1559239035 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60a400d600 con 0x7f60a80703f0 2024-02-15T18:26:36.424 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.427+0000 7f60ace50700 1 -- 172.21.15.38:0/1559239035 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60a400dc30 con 0x7f60a80703f0 2024-02-15T18:26:36.424 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.427+0000 7f60ace50700 1 -- 172.21.15.38:0/1559239035 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60a4011d60 con 0x7f60a80703f0 2024-02-15T18:26:36.426 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.428+0000 7f60b00b6700 1 -- 172.21.15.38:0/1559239035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60a80703f0 msgr2=0x7f60a80fea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:36.426 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.428+0000 7f60b00b6700 1 --2- 172.21.15.38:0/1559239035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60a80703f0 0x7f60a80fea90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f60a400b5f0 tx=0x7f60a4006ee0).stop 2024-02-15T18:26:36.426 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.428+0000 7f60b00b6700 1 -- 172.21.15.38:0/1559239035 shutdown_connections 2024-02-15T18:26:36.426 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.428+0000 7f60b00b6700 1 --2- 172.21.15.38:0/1559239035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60a80703f0 0x7f60a80fea90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:36.426 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.428+0000 7f60b00b6700 1 -- 172.21.15.38:0/1559239035 >> 172.21.15.38:0/1559239035 conn(0x7f60a80f6530 msgr2=0x7f60a80f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:36.427 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.428+0000 7f60b00b6700 1 -- 172.21.15.38:0/1559239035 shutdown_connections 2024-02-15T18:26:36.427 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.428+0000 7f60b00b6700 1 -- 172.21.15.38:0/1559239035 wait complete. 2024-02-15T18:26:36.427 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.428+0000 7f60b00b6700 1 Processor -- start 2024-02-15T18:26:36.427 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.428+0000 7f60b00b6700 1 -- start start 2024-02-15T18:26:36.427 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.429+0000 7f60b00b6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60a80703f0 0x7f60a80fc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:36.427 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.429+0000 7f60b00b6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a80fa990 con 0x7f60a80703f0 2024-02-15T18:26:36.427 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.429+0000 7f60ade52700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60a80703f0 0x7f60a80fc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:36.427 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.429+0000 7f60ade52700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60a80703f0 0x7f60a80fc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49878/0 (socket says 172.21.15.38:49878) 2024-02-15T18:26:36.427 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.429+0000 7f60ade52700 1 -- 172.21.15.38:0/185044079 learned_addr learned my addr 172.21.15.38:0/185044079 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:36.427 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.429+0000 7f60ade52700 1 -- 172.21.15.38:0/185044079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a400b200 con 0x7f60a80703f0 2024-02-15T18:26:36.428 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.429+0000 7f60ade52700 1 --2- 172.21.15.38:0/185044079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60a80703f0 0x7f60a80fc510 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f60a4006b80 tx=0x7f60a4006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:36.428 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.429+0000 7f60967fc700 1 -- 172.21.15.38:0/185044079 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60a4011e10 con 0x7f60a80703f0 2024-02-15T18:26:36.428 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.429+0000 7f60967fc700 1 -- 172.21.15.38:0/185044079 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60a4011610 con 0x7f60a80703f0 2024-02-15T18:26:36.428 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.429+0000 7f60967fc700 1 -- 172.21.15.38:0/185044079 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60a401a7b0 con 0x7f60a80703f0 2024-02-15T18:26:36.429 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.430+0000 7f60b00b6700 1 -- 172.21.15.38:0/185044079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a80fab30 con 0x7f60a80703f0 2024-02-15T18:26:36.429 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.430+0000 7f60b00b6700 1 -- 172.21.15.38:0/185044079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a80fafc0 con 0x7f60a80703f0 2024-02-15T18:26:36.429 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.431+0000 7f60967fc700 1 -- 172.21.15.38:0/185044079 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 36208+0+0 (secure 0 0 0) 0x7f60a400f070 con 0x7f60a80703f0 2024-02-15T18:26:36.429 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.431+0000 7f60967fc700 1 --2- 172.21.15.38:0/185044079 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f609002ff70 0x7f6090032430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:36.429 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.431+0000 7f60967fc700 1 -- 172.21.15.38:0/185044079 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f60a4040550 con 0x7f60a80703f0 2024-02-15T18:26:36.429 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.431+0000 7f60ad651700 1 -- 172.21.15.38:0/185044079 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f609002ff70 msgr2=0x7f6090032430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/2095530547 2024-02-15T18:26:36.429 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.431+0000 7f60ad651700 1 --2- 172.21.15.38:0/185044079 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f609002ff70 0x7f6090032430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-15T18:26:36.429 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.431+0000 7f60b00b6700 1 -- 172.21.15.38:0/185044079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a8045ad0 con 0x7f60a80703f0 2024-02-15T18:26:36.432 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.435+0000 7f60967fc700 1 -- 172.21.15.38:0/185044079 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f60a4016070 con 0x7f60a80703f0 2024-02-15T18:26:36.628 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.632+0000 7f60ad651700 1 -- 172.21.15.38:0/185044079 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f609002ff70 msgr2=0x7f6090032430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/2095530547 2024-02-15T18:26:36.628 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.632+0000 7f60ad651700 1 --2- 172.21.15.38:0/185044079 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f609002ff70 0x7f6090032430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-15T18:26:36.631 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.635+0000 7f60b00b6700 1 -- 172.21.15.38:0/185044079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f60a8104b70 con 0x7f60a80703f0 2024-02-15T18:26:36.632 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.636+0000 7f60967fc700 1 -- 172.21.15.38:0/185044079 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v13) v1 ==== 56+0+104 (secure 0 0 0) 0x7f60a401b410 con 0x7f60a80703f0 2024-02-15T18:26:36.633 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: { 2024-02-15T18:26:36.633 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "epoch": 13, 2024-02-15T18:26:36.633 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "available": true, 2024-02-15T18:26:36.633 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "active_name": "smithi038.khesbl", 2024-02-15T18:26:36.633 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "num_standby": 0 2024-02-15T18:26:36.633 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: } 2024-02-15T18:26:36.634 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.637+0000 7f60b00b6700 1 -- 172.21.15.38:0/185044079 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f609002ff70 msgr2=0x7f6090032430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-15T18:26:36.634 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.637+0000 7f60b00b6700 1 --2- 172.21.15.38:0/185044079 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f609002ff70 0x7f6090032430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:26:36.634 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.637+0000 7f60b00b6700 1 -- 172.21.15.38:0/185044079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60a80703f0 msgr2=0x7f60a80fc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:36.634 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.637+0000 7f60b00b6700 1 --2- 172.21.15.38:0/185044079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60a80703f0 0x7f60a80fc510 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f60a4006b80 tx=0x7f60a4006ee0).stop 2024-02-15T18:26:36.634 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.637+0000 7f60b00b6700 1 -- 172.21.15.38:0/185044079 shutdown_connections 2024-02-15T18:26:36.635 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.637+0000 7f60b00b6700 1 --2- 172.21.15.38:0/185044079 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7f609002ff70 0x7f6090032430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:26:36.635 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.637+0000 7f60b00b6700 1 --2- 172.21.15.38:0/185044079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60a80703f0 0x7f60a80fc510 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:36.635 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.637+0000 7f60b00b6700 1 -- 172.21.15.38:0/185044079 >> 172.21.15.38:0/185044079 conn(0x7f60a80f6530 msgr2=0x7f60a80f7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:36.635 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.638+0000 7f60b00b6700 1 -- 172.21.15.38:0/185044079 shutdown_connections 2024-02-15T18:26:36.635 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:36.638+0000 7f60b00b6700 1 -- 172.21.15.38:0/185044079 wait complete. 2024-02-15T18:26:36.923 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:36.636379+0000 mon.smithi038 (mon.0) 90 : audit [DBG] from='client.? 172.21.15.38:0/185044079' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-15T18:26:37.683 INFO:teuthology.orchestra.run.smithi038.stderr:Waiting for the mgr to restart... 2024-02-15T18:26:37.683 INFO:teuthology.orchestra.run.smithi038.stderr:Waiting for mgr epoch 13... 2024-02-15T18:26:37.683 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 tell mgr mgr_status 2024-02-15T18:26:38.321 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.315+0000 7fb95f348700 1 Processor -- start 2024-02-15T18:26:38.321 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.315+0000 7fb95f348700 1 -- start start 2024-02-15T18:26:38.321 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.315+0000 7fb95f348700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb958070430 0x7fb9580fea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:38.322 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.315+0000 7fb95f348700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9580fef40 con 0x7fb958070430 2024-02-15T18:26:38.322 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.316+0000 7fb95d0e4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb958070430 0x7fb9580fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:38.322 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.316+0000 7fb95d0e4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb958070430 0x7fb9580fea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49882/0 (socket says 172.21.15.38:49882) 2024-02-15T18:26:38.322 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.316+0000 7fb95d0e4700 1 -- 172.21.15.38:0/3499152889 learned_addr learned my addr 172.21.15.38:0/3499152889 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:38.322 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.316+0000 7fb95d0e4700 1 -- 172.21.15.38:0/3499152889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9580ff080 con 0x7fb958070430 2024-02-15T18:26:38.322 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.316+0000 7fb95d0e4700 1 --2- 172.21.15.38:0/3499152889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb958070430 0x7fb9580fea50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fb95400b5f0 tx=0x7fb954006ee0).ready entity=mon.0 client_cookie=d721c78bf8e3bfa1 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:38.322 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.316+0000 7fb947fff700 1 -- 172.21.15.38:0/3499152889 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb95400d600 con 0x7fb958070430 2024-02-15T18:26:38.322 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.316+0000 7fb947fff700 1 -- 172.21.15.38:0/3499152889 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb95400dc30 con 0x7fb958070430 2024-02-15T18:26:38.322 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.316+0000 7fb947fff700 1 -- 172.21.15.38:0/3499152889 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb954011d30 con 0x7fb958070430 2024-02-15T18:26:38.322 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.317+0000 7fb95f348700 1 -- 172.21.15.38:0/3499152889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb958070430 msgr2=0x7fb9580fea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:38.322 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.317+0000 7fb95f348700 1 --2- 172.21.15.38:0/3499152889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb958070430 0x7fb9580fea50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fb95400b5f0 tx=0x7fb954006ee0).stop 2024-02-15T18:26:38.322 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.317+0000 7fb95f348700 1 -- 172.21.15.38:0/3499152889 shutdown_connections 2024-02-15T18:26:38.322 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.317+0000 7fb95f348700 1 --2- 172.21.15.38:0/3499152889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb958070430 0x7fb9580fea50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:38.323 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.317+0000 7fb95f348700 1 -- 172.21.15.38:0/3499152889 >> 172.21.15.38:0/3499152889 conn(0x7fb9580f6530 msgr2=0x7fb9580f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:38.323 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.317+0000 7fb95f348700 1 -- 172.21.15.38:0/3499152889 shutdown_connections 2024-02-15T18:26:38.323 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.317+0000 7fb95f348700 1 -- 172.21.15.38:0/3499152889 wait complete. 2024-02-15T18:26:38.323 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.317+0000 7fb95f348700 1 Processor -- start 2024-02-15T18:26:38.323 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.317+0000 7fb95f348700 1 -- start start 2024-02-15T18:26:38.323 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.318+0000 7fb95f348700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb958070430 0x7fb9580fc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:38.323 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.318+0000 7fb95f348700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9580fa950 con 0x7fb958070430 2024-02-15T18:26:38.323 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.318+0000 7fb95d0e4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb958070430 0x7fb9580fc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:38.323 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.318+0000 7fb95d0e4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb958070430 0x7fb9580fc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49896/0 (socket says 172.21.15.38:49896) 2024-02-15T18:26:38.323 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.318+0000 7fb95d0e4700 1 -- 172.21.15.38:0/1794674368 learned_addr learned my addr 172.21.15.38:0/1794674368 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:38.323 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.318+0000 7fb95d0e4700 1 -- 172.21.15.38:0/1794674368 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95400b200 con 0x7fb958070430 2024-02-15T18:26:38.323 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.318+0000 7fb95d0e4700 1 --2- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb958070430 0x7fb9580fc4d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fb95400e040 tx=0x7fb954006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:38.323 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.318+0000 7fb945ffb700 1 -- 172.21.15.38:0/1794674368 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb95400da60 con 0x7fb958070430 2024-02-15T18:26:38.323 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.318+0000 7fb95f348700 1 -- 172.21.15.38:0/1794674368 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9580faaf0 con 0x7fb958070430 2024-02-15T18:26:38.323 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.318+0000 7fb945ffb700 1 -- 172.21.15.38:0/1794674368 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb95401b8e0 con 0x7fb958070430 2024-02-15T18:26:38.324 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.318+0000 7fb945ffb700 1 -- 172.21.15.38:0/1794674368 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb95401a900 con 0x7fb958070430 2024-02-15T18:26:38.324 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.319+0000 7fb95f348700 1 -- 172.21.15.38:0/1794674368 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9580faf80 con 0x7fb958070430 2024-02-15T18:26:38.324 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.319+0000 7fb945ffb700 1 -- 172.21.15.38:0/1794674368 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) v1 ==== 36208+0+0 (secure 0 0 0) 0x7fb95400f070 con 0x7fb958070430 2024-02-15T18:26:38.324 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.319+0000 7fb945ffb700 1 --2- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fb94002ff70 0x7fb940032430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:38.324 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.319+0000 7fb945ffb700 1 -- 172.21.15.38:0/1794674368 --> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb940032af0 con 0x7fb94002ff70 2024-02-15T18:26:38.324 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.319+0000 7fb945ffb700 1 -- 172.21.15.38:0/1794674368 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7fb954041470 con 0x7fb958070430 2024-02-15T18:26:38.324 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.319+0000 7fb95c8e3700 1 -- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fb94002ff70 msgr2=0x7fb940032430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/2095530547 2024-02-15T18:26:38.324 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.320+0000 7fb95c8e3700 1 --2- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fb94002ff70 0x7fb940032430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-15T18:26:38.517 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.521+0000 7fb95c8e3700 1 -- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fb94002ff70 msgr2=0x7fb940032430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/2095530547 2024-02-15T18:26:38.517 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.521+0000 7fb95c8e3700 1 --2- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fb94002ff70 0x7fb940032430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-15T18:26:38.918 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.922+0000 7fb95c8e3700 1 -- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fb94002ff70 msgr2=0x7fb940032430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/2095530547 2024-02-15T18:26:38.918 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:38.922+0000 7fb95c8e3700 1 --2- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fb94002ff70 0x7fb940032430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-02-15T18:26:39.719 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:39.723+0000 7fb95c8e3700 1 -- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fb94002ff70 msgr2=0x7fb940032430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/2095530547 2024-02-15T18:26:39.719 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:39.723+0000 7fb95c8e3700 1 --2- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fb94002ff70 0x7fb940032430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-02-15T18:26:40.208 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:40.209+0000 7fb945ffb700 1 -- 172.21.15.38:0/1794674368 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mgrmap(e 14) v1 ==== 35997+0+0 (secure 0 0 0) 0x7fb95401abc0 con 0x7fb958070430 2024-02-15T18:26:40.208 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:40.209+0000 7fb945ffb700 1 -- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fb94002ff70 msgr2=0x7fb940032430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-15T18:26:40.208 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:40.209+0000 7fb945ffb700 1 --2- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fb94002ff70 0x7fb940032430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:26:40.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:40.033275+0000 mon.smithi038 (mon.0) 91 : cluster [INF] Active manager daemon smithi038.khesbl restarted 2024-02-15T18:26:40.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:40.033783+0000 mon.smithi038 (mon.0) 92 : cluster [INF] Activating manager daemon smithi038.khesbl 2024-02-15T18:26:40.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:40.039149+0000 mon.smithi038 (mon.0) 93 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-15T18:26:41.213 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:41.215+0000 7fb945ffb700 1 -- 172.21.15.38:0/1794674368 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 15) v1 ==== 36067+0+0 (secure 0 0 0) 0x7fb954041180 con 0x7fb958070430 2024-02-15T18:26:41.213 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:41.215+0000 7fb945ffb700 1 --2- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb940033c00 0x7fb940035ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:41.213 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:41.215+0000 7fb945ffb700 1 -- 172.21.15.38:0/1794674368 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb940032af0 con 0x7fb940033c00 2024-02-15T18:26:41.214 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:41.216+0000 7fb95c8e3700 1 --2- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb940033c00 0x7fb940035ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:41.214 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:41.216+0000 7fb95c8e3700 1 --2- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb940033c00 0x7fb940035ff0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7fb9480036d0 tx=0x7fb948006ca0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:41.214 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:41.217+0000 7fb945ffb700 1 -- 172.21.15.38:0/1794674368 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6438 (secure 0 0 0) 0x7fb940032af0 con 0x7fb940033c00 2024-02-15T18:26:41.220 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:41.223+0000 7fb95f348700 1 -- 172.21.15.38:0/1794674368 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fb958059100 con 0x7fb940033c00 2024-02-15T18:26:41.221 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:41.223+0000 7fb945ffb700 1 -- 172.21.15.38:0/1794674368 <== mgr.14162 v2:172.21.15.38:6800/2900497610 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fb958059100 con 0x7fb940033c00 2024-02-15T18:26:41.221 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: { 2024-02-15T18:26:41.221 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "mgrmap_epoch": 15, 2024-02-15T18:26:41.221 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: "initialized": true 2024-02-15T18:26:41.222 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: } 2024-02-15T18:26:41.222 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:41.224+0000 7fb95f348700 1 -- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb940033c00 msgr2=0x7fb940035ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:41.222 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:41.224+0000 7fb95f348700 1 --2- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb940033c00 0x7fb940035ff0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7fb9480036d0 tx=0x7fb948006ca0).stop 2024-02-15T18:26:41.222 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:41.224+0000 7fb95f348700 1 -- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb958070430 msgr2=0x7fb9580fc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:41.222 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:41.224+0000 7fb95f348700 1 --2- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb958070430 0x7fb9580fc4d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fb95400e040 tx=0x7fb954006ee0).stop 2024-02-15T18:26:41.222 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:41.224+0000 7fb95f348700 1 -- 172.21.15.38:0/1794674368 shutdown_connections 2024-02-15T18:26:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:41.224+0000 7fb95f348700 1 --2- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb940033c00 0x7fb940035ff0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:41.224+0000 7fb95f348700 1 --2- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb958070430 0x7fb9580fc4d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:41.224+0000 7fb95f348700 1 --2- 172.21.15.38:0/1794674368 >> [v2:172.21.15.38:6800/2095530547,v1:172.21.15.38:6801/2095530547] conn(0x7fb94002ff70 0x7fb940032430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:26:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:41.224+0000 7fb95f348700 1 -- 172.21.15.38:0/1794674368 >> 172.21.15.38:0/1794674368 conn(0x7fb9580f6530 msgr2=0x7fb9580f7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:41.224+0000 7fb95f348700 1 -- 172.21.15.38:0/1794674368 shutdown_connections 2024-02-15T18:26:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:41.224+0000 7fb95f348700 1 -- 172.21.15.38:0/1794674368 wait complete. 2024-02-15T18:26:41.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:40.212441+0000 mon.smithi038 (mon.0) 94 : cluster [DBG] mgrmap e14: smithi038.khesbl(active, starting, since 0.17874s) 2024-02-15T18:26:41.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:40.213637+0000 mon.smithi038 (mon.0) 95 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-02-15T18:26:41.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:40.214246+0000 mon.smithi038 (mon.0) 96 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mgr metadata", "who": "smithi038.khesbl", "id": "smithi038.khesbl"}]: dispatch 2024-02-15T18:26:41.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:40.214989+0000 mon.smithi038 (mon.0) 97 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-15T18:26:41.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:40.215129+0000 mon.smithi038 (mon.0) 98 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-15T18:26:41.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:40.215259+0000 mon.smithi038 (mon.0) 99 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-15T18:26:41.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:40.217212+0000 mon.smithi038 (mon.0) 100 : cluster [INF] Manager daemon smithi038.khesbl is now available 2024-02-15T18:26:41.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:40.321562+0000 mon.smithi038 (mon.0) 101 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:41.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:40.324108+0000 mon.smithi038 (mon.0) 102 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:41.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:40.346627+0000 mon.smithi038 (mon.0) 103 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:26:41.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:40.350216+0000 mon.smithi038 (mon.0) 104 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:26:42.261 INFO:teuthology.orchestra.run.smithi038.stderr:mgr epoch 13 is available 2024-02-15T18:26:42.261 INFO:teuthology.orchestra.run.smithi038.stderr:Generating a dashboard self-signed certificate... 2024-02-15T18:26:42.262 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 dashboard create-self-signed-cert 2024-02-15T18:26:42.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:41.218649+0000 mon.smithi038 (mon.0) 105 : cluster [DBG] mgrmap e15: smithi038.khesbl(active, since 1.18495s) 2024-02-15T18:26:42.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:41.218724+0000 mgr.smithi038.khesbl (mgr.14162) 1 : audit [DBG] from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-15T18:26:42.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:41.225094+0000 mgr.smithi038.khesbl (mgr.14162) 2 : audit [DBG] from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-15T18:26:42.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:41.872598+0000 mon.smithi038 (mon.0) 106 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:43.112 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.988+0000 7ff60682a700 1 Processor -- start 2024-02-15T18:26:43.112 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.988+0000 7ff60682a700 1 -- start start 2024-02-15T18:26:43.112 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.988+0000 7ff60682a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6000703b0 0x7ff600100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:43.112 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.988+0000 7ff60682a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6000fc8c0 con 0x7ff6000703b0 2024-02-15T18:26:43.112 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.988+0000 7ff5fffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6000703b0 0x7ff600100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:43.112 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.988+0000 7ff5fffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6000703b0 0x7ff600100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49936/0 (socket says 172.21.15.38:49936) 2024-02-15T18:26:43.112 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.988+0000 7ff5fffff700 1 -- 172.21.15.38:0/321246993 learned_addr learned my addr 172.21.15.38:0/321246993 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:43.112 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.988+0000 7ff5fffff700 1 -- 172.21.15.38:0/321246993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6001013c0 con 0x7ff6000703b0 2024-02-15T18:26:43.112 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.989+0000 7ff5fffff700 1 --2- 172.21.15.38:0/321246993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6000703b0 0x7ff600100ed0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7ff5f000b5f0 tx=0x7ff5f0006ee0).ready entity=mon.0 client_cookie=59fb9d91445e6375 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:43.112 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.989+0000 7ff5feffd700 1 -- 172.21.15.38:0/321246993 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5f000d600 con 0x7ff6000703b0 2024-02-15T18:26:43.112 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.989+0000 7ff5feffd700 1 -- 172.21.15.38:0/321246993 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5f000dc30 con 0x7ff6000703b0 2024-02-15T18:26:43.112 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.989+0000 7ff5feffd700 1 -- 172.21.15.38:0/321246993 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5f0011d30 con 0x7ff6000703b0 2024-02-15T18:26:43.113 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.989+0000 7ff60682a700 1 -- 172.21.15.38:0/321246993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6000703b0 msgr2=0x7ff600100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:43.113 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.989+0000 7ff60682a700 1 --2- 172.21.15.38:0/321246993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6000703b0 0x7ff600100ed0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7ff5f000b5f0 tx=0x7ff5f0006ee0).stop 2024-02-15T18:26:43.113 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.989+0000 7ff60682a700 1 -- 172.21.15.38:0/321246993 shutdown_connections 2024-02-15T18:26:43.113 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.989+0000 7ff60682a700 1 --2- 172.21.15.38:0/321246993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6000703b0 0x7ff600100ed0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:43.113 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.989+0000 7ff60682a700 1 -- 172.21.15.38:0/321246993 >> 172.21.15.38:0/321246993 conn(0x7ff6000f6530 msgr2=0x7ff6000f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:43.113 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.989+0000 7ff60682a700 1 -- 172.21.15.38:0/321246993 shutdown_connections 2024-02-15T18:26:43.113 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.989+0000 7ff60682a700 1 -- 172.21.15.38:0/321246993 wait complete. 2024-02-15T18:26:43.113 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.990+0000 7ff60682a700 1 Processor -- start 2024-02-15T18:26:43.113 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.990+0000 7ff60682a700 1 -- start start 2024-02-15T18:26:43.113 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.990+0000 7ff60682a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6000703b0 0x7ff60010cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:43.113 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.990+0000 7ff60682a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60010d290 con 0x7ff6000703b0 2024-02-15T18:26:43.113 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.990+0000 7ff5fffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6000703b0 0x7ff60010cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:43.113 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.990+0000 7ff5fffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6000703b0 0x7ff60010cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49952/0 (socket says 172.21.15.38:49952) 2024-02-15T18:26:43.113 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.990+0000 7ff5fffff700 1 -- 172.21.15.38:0/2397276054 learned_addr learned my addr 172.21.15.38:0/2397276054 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:43.113 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.990+0000 7ff5fffff700 1 -- 172.21.15.38:0/2397276054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f000b200 con 0x7ff6000703b0 2024-02-15T18:26:43.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.991+0000 7ff5fffff700 1 --2- 172.21.15.38:0/2397276054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6000703b0 0x7ff60010cda0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7ff5f000e040 tx=0x7ff5f0006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:43.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.991+0000 7ff5fcff9700 1 -- 172.21.15.38:0/2397276054 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5f000da60 con 0x7ff6000703b0 2024-02-15T18:26:43.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.991+0000 7ff5fcff9700 1 -- 172.21.15.38:0/2397276054 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5f001b8e0 con 0x7ff6000703b0 2024-02-15T18:26:43.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.991+0000 7ff60682a700 1 -- 172.21.15.38:0/2397276054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60010d490 con 0x7ff6000703b0 2024-02-15T18:26:43.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.991+0000 7ff5fcff9700 1 -- 172.21.15.38:0/2397276054 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5f001a900 con 0x7ff6000703b0 2024-02-15T18:26:43.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.991+0000 7ff60682a700 1 -- 172.21.15.38:0/2397276054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60010e0d0 con 0x7ff6000703b0 2024-02-15T18:26:43.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.992+0000 7ff5fcff9700 1 -- 172.21.15.38:0/2397276054 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) v1 ==== 36208+0+0 (secure 0 0 0) 0x7ff5f000f070 con 0x7ff6000703b0 2024-02-15T18:26:43.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.992+0000 7ff60682a700 1 -- 172.21.15.38:0/2397276054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff60010d920 con 0x7ff6000703b0 2024-02-15T18:26:43.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.992+0000 7ff5fcff9700 1 --2- 172.21.15.38:0/2397276054 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff5ec02ffc0 0x7ff5ec032480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:43.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.992+0000 7ff5fcff9700 1 -- 172.21.15.38:0/2397276054 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7ff5f0041490 con 0x7ff6000703b0 2024-02-15T18:26:43.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.992+0000 7ff5ff7fe700 1 --2- 172.21.15.38:0/2397276054 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff5ec02ffc0 0x7ff5ec032480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:43.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.992+0000 7ff5ff7fe700 1 --2- 172.21.15.38:0/2397276054 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff5ec02ffc0 0x7ff5ec032480 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7ff5f40096a0 tx=0x7ff5f4006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:43.114 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:42.996+0000 7ff5fcff9700 1 -- 172.21.15.38:0/2397276054 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ff5f0022ac0 con 0x7ff6000703b0 2024-02-15T18:26:43.160 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:43.163+0000 7ff60682a700 1 -- 172.21.15.38:0/2397276054 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7ff60010e3d0 con 0x7ff5ec02ffc0 2024-02-15T18:26:43.235 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:43.237+0000 7ff5fcff9700 1 -- 172.21.15.38:0/2397276054 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7ff60010e3d0 con 0x7ff5ec02ffc0 2024-02-15T18:26:43.235 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: Self-signed certificate created 2024-02-15T18:26:43.237 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:43.239+0000 7ff60682a700 1 -- 172.21.15.38:0/2397276054 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff5ec02ffc0 msgr2=0x7ff5ec032480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:43.237 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:43.239+0000 7ff60682a700 1 --2- 172.21.15.38:0/2397276054 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff5ec02ffc0 0x7ff5ec032480 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7ff5f40096a0 tx=0x7ff5f4006b40).stop 2024-02-15T18:26:43.237 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:43.239+0000 7ff60682a700 1 -- 172.21.15.38:0/2397276054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6000703b0 msgr2=0x7ff60010cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:43.237 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:43.239+0000 7ff60682a700 1 --2- 172.21.15.38:0/2397276054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6000703b0 0x7ff60010cda0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7ff5f000e040 tx=0x7ff5f0006ee0).stop 2024-02-15T18:26:43.238 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:43.239+0000 7ff60682a700 1 -- 172.21.15.38:0/2397276054 shutdown_connections 2024-02-15T18:26:43.238 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:43.239+0000 7ff60682a700 1 --2- 172.21.15.38:0/2397276054 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff5ec02ffc0 0x7ff5ec032480 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:43.238 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:43.239+0000 7ff60682a700 1 --2- 172.21.15.38:0/2397276054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6000703b0 0x7ff60010cda0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:43.238 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:43.239+0000 7ff60682a700 1 -- 172.21.15.38:0/2397276054 >> 172.21.15.38:0/2397276054 conn(0x7ff6000f6530 msgr2=0x7ff6000f7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:43.238 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:43.239+0000 7ff60682a700 1 -- 172.21.15.38:0/2397276054 shutdown_connections 2024-02-15T18:26:43.238 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:43.239+0000 7ff60682a700 1 -- 172.21.15.38:0/2397276054 wait complete. 2024-02-15T18:26:44.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:42.875737+0000 mon.smithi038 (mon.0) 107 : cluster [DBG] mgrmap e16: smithi038.khesbl(active, since 2s) 2024-02-15T18:26:44.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:43.165578+0000 mgr.smithi038.khesbl (mgr.14162) 3 : audit [DBG] from='client.14172 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:44.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:43.234034+0000 mon.smithi038 (mon.0) 108 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:44.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:43.238962+0000 mon.smithi038 (mon.0) 109 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:44.452 INFO:teuthology.orchestra.run.smithi038.stderr:Creating initial admin user... 2024-02-15T18:26:44.453 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmp9qq0qoip:/tmp/dashboard.pw:z docker.io/ceph/ceph:v16.2.4 dashboard ac-user-create admin -i /tmp/dashboard.pw administrator --force-password --pwd-update-required 2024-02-15T18:26:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.084+0000 7f1c659d6700 1 Processor -- start 2024-02-15T18:26:45.083 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.084+0000 7f1c659d6700 1 -- start start 2024-02-15T18:26:45.083 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.084+0000 7f1c659d6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c600fbea0 0x7f1c600fc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:45.083 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.084+0000 7f1c659d6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c600fc7b0 con 0x7f1c600fbea0 2024-02-15T18:26:45.083 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.085+0000 7f1c5effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c600fbea0 0x7f1c600fc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:45.083 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.085+0000 7f1c5effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c600fbea0 0x7f1c600fc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52578/0 (socket says 172.21.15.38:52578) 2024-02-15T18:26:45.083 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.085+0000 7f1c5effd700 1 -- 172.21.15.38:0/2173053191 learned_addr learned my addr 172.21.15.38:0/2173053191 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:45.086 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.085+0000 7f1c5effd700 1 -- 172.21.15.38:0/2173053191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c600fc8f0 con 0x7f1c600fbea0 2024-02-15T18:26:45.086 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.085+0000 7f1c5effd700 1 --2- 172.21.15.38:0/2173053191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c600fbea0 0x7f1c600fc2c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f1c4800b5f0 tx=0x7f1c48006ee0).ready entity=mon.0 client_cookie=9ceabc60fa42838a server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:45.086 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.086+0000 7f1c5dffb700 1 -- 172.21.15.38:0/2173053191 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c4800d600 con 0x7f1c600fbea0 2024-02-15T18:26:45.086 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.086+0000 7f1c5dffb700 1 -- 172.21.15.38:0/2173053191 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c4800dc30 con 0x7f1c600fbea0 2024-02-15T18:26:45.087 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.086+0000 7f1c5dffb700 1 -- 172.21.15.38:0/2173053191 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c48011d30 con 0x7f1c600fbea0 2024-02-15T18:26:45.087 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.087+0000 7f1c659d6700 1 -- 172.21.15.38:0/2173053191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c600fbea0 msgr2=0x7f1c600fc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:45.087 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.087+0000 7f1c659d6700 1 --2- 172.21.15.38:0/2173053191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c600fbea0 0x7f1c600fc2c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f1c4800b5f0 tx=0x7f1c48006ee0).stop 2024-02-15T18:26:45.087 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.087+0000 7f1c659d6700 1 -- 172.21.15.38:0/2173053191 shutdown_connections 2024-02-15T18:26:45.087 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.087+0000 7f1c659d6700 1 --2- 172.21.15.38:0/2173053191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c600fbea0 0x7f1c600fc2c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:45.087 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.087+0000 7f1c659d6700 1 -- 172.21.15.38:0/2173053191 >> 172.21.15.38:0/2173053191 conn(0x7f1c600f76a0 msgr2=0x7f1c600f9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:45.087 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.087+0000 7f1c659d6700 1 -- 172.21.15.38:0/2173053191 shutdown_connections 2024-02-15T18:26:45.087 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.087+0000 7f1c659d6700 1 -- 172.21.15.38:0/2173053191 wait complete. 2024-02-15T18:26:45.087 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.087+0000 7f1c659d6700 1 Processor -- start 2024-02-15T18:26:45.087 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.087+0000 7f1c659d6700 1 -- start start 2024-02-15T18:26:45.087 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.088+0000 7f1c659d6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c600fbea0 0x7f1c600725e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:45.087 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.088+0000 7f1c659d6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c60070a60 con 0x7f1c600fbea0 2024-02-15T18:26:45.087 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.088+0000 7f1c5effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c600fbea0 0x7f1c600725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:45.087 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.088+0000 7f1c5effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c600fbea0 0x7f1c600725e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52590/0 (socket says 172.21.15.38:52590) 2024-02-15T18:26:45.088 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.088+0000 7f1c5effd700 1 -- 172.21.15.38:0/4244387312 learned_addr learned my addr 172.21.15.38:0/4244387312 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:45.088 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.088+0000 7f1c5effd700 1 -- 172.21.15.38:0/4244387312 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4800b200 con 0x7f1c600fbea0 2024-02-15T18:26:45.088 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.088+0000 7f1c5effd700 1 --2- 172.21.15.38:0/4244387312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c600fbea0 0x7f1c600725e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f1c48006b80 tx=0x7f1c48006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:45.088 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.088+0000 7f1c577fe700 1 -- 172.21.15.38:0/4244387312 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c4800d9e0 con 0x7f1c600fbea0 2024-02-15T18:26:45.088 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.088+0000 7f1c577fe700 1 -- 172.21.15.38:0/4244387312 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c4801b8e0 con 0x7f1c600fbea0 2024-02-15T18:26:45.088 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.088+0000 7f1c659d6700 1 -- 172.21.15.38:0/4244387312 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c60070c00 con 0x7f1c600fbea0 2024-02-15T18:26:45.088 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.088+0000 7f1c577fe700 1 -- 172.21.15.38:0/4244387312 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c4801a870 con 0x7f1c600fbea0 2024-02-15T18:26:45.088 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.088+0000 7f1c659d6700 1 -- 172.21.15.38:0/4244387312 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c60071090 con 0x7f1c600fbea0 2024-02-15T18:26:45.088 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.089+0000 7f1c577fe700 1 -- 172.21.15.38:0/4244387312 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) v1 ==== 36208+0+0 (secure 0 0 0) 0x7f1c4800f070 con 0x7f1c600fbea0 2024-02-15T18:26:45.089 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.089+0000 7f1c577fe700 1 --2- 172.21.15.38:0/4244387312 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1c4002ff70 0x7f1c40032430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:45.089 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.089+0000 7f1c577fe700 1 -- 172.21.15.38:0/4244387312 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f1c48041530 con 0x7f1c600fbea0 2024-02-15T18:26:45.089 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.090+0000 7f1c5e7fc700 1 --2- 172.21.15.38:0/4244387312 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1c4002ff70 0x7f1c40032430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:45.089 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.090+0000 7f1c5e7fc700 1 --2- 172.21.15.38:0/4244387312 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1c4002ff70 0x7f1c40032430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f1c500096a0 tx=0x7f1c50006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:45.089 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.090+0000 7f1c659d6700 1 -- 172.21.15.38:0/4244387312 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c60059100 con 0x7f1c600fbea0 2024-02-15T18:26:45.092 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.095+0000 7f1c577fe700 1 -- 172.21.15.38:0/4244387312 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f1c48022d50 con 0x7f1c600fbea0 2024-02-15T18:26:45.266 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.268+0000 7f1c659d6700 1 -- 172.21.15.38:0/4244387312 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f1c60045ad0 con 0x7f1c4002ff70 2024-02-15T18:26:45.514 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.516+0000 7f1c577fe700 1 -- 172.21.15.38:0/4244387312 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f1c60045ad0 con 0x7f1c4002ff70 2024-02-15T18:26:45.514 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: {"username": "admin", "password": "$2b$12$cFk6PsX02NwXV7YR0y/RFOVYJFJiJfnr52Ob23DAp/N6Xo1897gRe", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708021605, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-15T18:26:45.516 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.518+0000 7f1c659d6700 1 -- 172.21.15.38:0/4244387312 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1c4002ff70 msgr2=0x7f1c40032430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:45.516 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.518+0000 7f1c659d6700 1 --2- 172.21.15.38:0/4244387312 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1c4002ff70 0x7f1c40032430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f1c500096a0 tx=0x7f1c50006b40).stop 2024-02-15T18:26:45.517 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.518+0000 7f1c659d6700 1 -- 172.21.15.38:0/4244387312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c600fbea0 msgr2=0x7f1c600725e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:45.517 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.518+0000 7f1c659d6700 1 --2- 172.21.15.38:0/4244387312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c600fbea0 0x7f1c600725e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f1c48006b80 tx=0x7f1c48006ee0).stop 2024-02-15T18:26:45.517 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.518+0000 7f1c659d6700 1 -- 172.21.15.38:0/4244387312 shutdown_connections 2024-02-15T18:26:45.517 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.518+0000 7f1c659d6700 1 --2- 172.21.15.38:0/4244387312 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1c4002ff70 0x7f1c40032430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:45.517 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.518+0000 7f1c659d6700 1 --2- 172.21.15.38:0/4244387312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c600fbea0 0x7f1c600725e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:45.517 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.518+0000 7f1c659d6700 1 -- 172.21.15.38:0/4244387312 >> 172.21.15.38:0/4244387312 conn(0x7f1c600f76a0 msgr2=0x7f1c600f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:45.517 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.518+0000 7f1c659d6700 1 -- 172.21.15.38:0/4244387312 shutdown_connections 2024-02-15T18:26:45.517 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:45.519+0000 7f1c659d6700 1 -- 172.21.15.38:0/4244387312 wait complete. 2024-02-15T18:26:46.260 INFO:teuthology.orchestra.run.smithi038.stderr:Fetching dashboard port number... 2024-02-15T18:26:46.260 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi038 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpn76yqex2:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp824yk_k1:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 config get mgr mgr/dashboard/ssl_server_port 2024-02-15T18:26:46.778 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:45.270887+0000 mgr.smithi038.khesbl (mgr.14162) 4 : audit [DBG] from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:46.779 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:45.518092+0000 mon.smithi038 (mon.0) 110 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:46.920 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.921+0000 7f9383c8f700 1 Processor -- start 2024-02-15T18:26:46.920 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.921+0000 7f9383c8f700 1 -- start start 2024-02-15T18:26:46.920 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.921+0000 7f9383c8f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f937c0703d0 0x7f937c100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:46.920 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.921+0000 7f9383c8f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937c0fc8a0 con 0x7f937c0703d0 2024-02-15T18:26:46.920 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.922+0000 7f9381a2b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f937c0703d0 0x7f937c100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:46.921 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.922+0000 7f9381a2b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f937c0703d0 0x7f937c100eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52606/0 (socket says 172.21.15.38:52606) 2024-02-15T18:26:46.921 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.922+0000 7f9381a2b700 1 -- 172.21.15.38:0/1854878994 learned_addr learned my addr 172.21.15.38:0/1854878994 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:46.921 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.922+0000 7f9381a2b700 1 -- 172.21.15.38:0/1854878994 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c1013a0 con 0x7f937c0703d0 2024-02-15T18:26:46.921 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.922+0000 7f9381a2b700 1 --2- 172.21.15.38:0/1854878994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f937c0703d0 0x7f937c100eb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f936c00b5f0 tx=0x7f936c006ee0).ready entity=mon.0 client_cookie=6f29e2d9113257ff server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:46.921 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.922+0000 7f9380a29700 1 -- 172.21.15.38:0/1854878994 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f936c00d600 con 0x7f937c0703d0 2024-02-15T18:26:46.921 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.922+0000 7f9380a29700 1 -- 172.21.15.38:0/1854878994 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f936c00dc30 con 0x7f937c0703d0 2024-02-15T18:26:46.922 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.922+0000 7f9380a29700 1 -- 172.21.15.38:0/1854878994 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f936c011d30 con 0x7f937c0703d0 2024-02-15T18:26:46.922 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.923+0000 7f9383c8f700 1 -- 172.21.15.38:0/1854878994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f937c0703d0 msgr2=0x7f937c100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:46.922 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.923+0000 7f9383c8f700 1 --2- 172.21.15.38:0/1854878994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f937c0703d0 0x7f937c100eb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f936c00b5f0 tx=0x7f936c006ee0).stop 2024-02-15T18:26:46.926 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.923+0000 7f9383c8f700 1 -- 172.21.15.38:0/1854878994 shutdown_connections 2024-02-15T18:26:46.926 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.923+0000 7f9383c8f700 1 --2- 172.21.15.38:0/1854878994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f937c0703d0 0x7f937c100eb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:46.926 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.923+0000 7f9383c8f700 1 -- 172.21.15.38:0/1854878994 >> 172.21.15.38:0/1854878994 conn(0x7f937c0f6530 msgr2=0x7f937c0f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:46.927 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.923+0000 7f9383c8f700 1 -- 172.21.15.38:0/1854878994 shutdown_connections 2024-02-15T18:26:46.927 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.923+0000 7f9383c8f700 1 -- 172.21.15.38:0/1854878994 wait complete. 2024-02-15T18:26:46.927 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.924+0000 7f9383c8f700 1 Processor -- start 2024-02-15T18:26:46.927 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.924+0000 7f9383c8f700 1 -- start start 2024-02-15T18:26:46.927 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.924+0000 7f9383c8f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f937c0703d0 0x7f937c10ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:46.927 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.924+0000 7f9383c8f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937c10d3a0 con 0x7f937c0703d0 2024-02-15T18:26:46.927 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.924+0000 7f9381a2b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f937c0703d0 0x7f937c10ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:46.927 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.924+0000 7f9381a2b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f937c0703d0 0x7f937c10ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52620/0 (socket says 172.21.15.38:52620) 2024-02-15T18:26:46.927 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.924+0000 7f9381a2b700 1 -- 172.21.15.38:0/1299081255 learned_addr learned my addr 172.21.15.38:0/1299081255 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:46.927 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.924+0000 7f9381a2b700 1 -- 172.21.15.38:0/1299081255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c00b200 con 0x7f937c0703d0 2024-02-15T18:26:46.927 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.924+0000 7f9381a2b700 1 --2- 172.21.15.38:0/1299081255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f937c0703d0 0x7f937c10ceb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f936c00e040 tx=0x7f936c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:46.927 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.924+0000 7f936a7fc700 1 -- 172.21.15.38:0/1299081255 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f936c00d9e0 con 0x7f937c0703d0 2024-02-15T18:26:46.927 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.925+0000 7f936a7fc700 1 -- 172.21.15.38:0/1299081255 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f936c01b8e0 con 0x7f937c0703d0 2024-02-15T18:26:46.927 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.925+0000 7f9383c8f700 1 -- 172.21.15.38:0/1299081255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f937c10d5a0 con 0x7f937c0703d0 2024-02-15T18:26:46.928 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.925+0000 7f936a7fc700 1 -- 172.21.15.38:0/1299081255 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f936c01a820 con 0x7f937c0703d0 2024-02-15T18:26:46.928 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.925+0000 7f9383c8f700 1 -- 172.21.15.38:0/1299081255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f937c10e1b0 con 0x7f937c0703d0 2024-02-15T18:26:46.928 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.925+0000 7f936a7fc700 1 -- 172.21.15.38:0/1299081255 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f936c00f070 con 0x7f937c0703d0 2024-02-15T18:26:46.928 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.926+0000 7f936a7fc700 1 --2- 172.21.15.38:0/1299081255 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9364030070 0x7f9364032530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:46.928 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.926+0000 7f9383c8f700 1 -- 172.21.15.38:0/1299081255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f937c059100 con 0x7f937c0703d0 2024-02-15T18:26:46.928 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.926+0000 7f936a7fc700 1 -- 172.21.15.38:0/1299081255 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f936c041890 con 0x7f937c0703d0 2024-02-15T18:26:46.928 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.926+0000 7f938122a700 1 --2- 172.21.15.38:0/1299081255 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9364030070 0x7f9364032530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:46.928 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.926+0000 7f938122a700 1 --2- 172.21.15.38:0/1299081255 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9364030070 0x7f9364032530 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f93780096a0 tx=0x7f9378006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:46.928 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:46.930+0000 7f936a7fc700 1 -- 172.21.15.38:0/1299081255 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f936c022d20 con 0x7f937c0703d0 2024-02-15T18:26:47.086 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:47.088+0000 7f9383c8f700 1 -- 172.21.15.38:0/1299081255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f937c10efe0 con 0x7f937c0703d0 2024-02-15T18:26:47.086 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:47.088+0000 7f936a7fc700 1 -- 172.21.15.38:0/1299081255 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v10) v1 ==== 112+0+5 (secure 0 0 0) 0x7f936c022910 con 0x7f937c0703d0 2024-02-15T18:26:47.086 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 8443 2024-02-15T18:26:47.088 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:47.090+0000 7f9383c8f700 1 -- 172.21.15.38:0/1299081255 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9364030070 msgr2=0x7f9364032530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:47.088 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:47.090+0000 7f9383c8f700 1 --2- 172.21.15.38:0/1299081255 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9364030070 0x7f9364032530 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f93780096a0 tx=0x7f9378006b40).stop 2024-02-15T18:26:47.088 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:47.090+0000 7f9383c8f700 1 -- 172.21.15.38:0/1299081255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f937c0703d0 msgr2=0x7f937c10ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:47.088 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:47.090+0000 7f9383c8f700 1 --2- 172.21.15.38:0/1299081255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f937c0703d0 0x7f937c10ceb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f936c00e040 tx=0x7f936c006ee0).stop 2024-02-15T18:26:47.088 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:47.090+0000 7f9383c8f700 1 -- 172.21.15.38:0/1299081255 shutdown_connections 2024-02-15T18:26:47.088 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:47.090+0000 7f9383c8f700 1 --2- 172.21.15.38:0/1299081255 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9364030070 0x7f9364032530 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:47.089 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:47.090+0000 7f9383c8f700 1 --2- 172.21.15.38:0/1299081255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f937c0703d0 0x7f937c10ceb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:47.089 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:47.090+0000 7f9383c8f700 1 -- 172.21.15.38:0/1299081255 >> 172.21.15.38:0/1299081255 conn(0x7f937c0f6530 msgr2=0x7f937c0f7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:47.089 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:47.091+0000 7f9383c8f700 1 -- 172.21.15.38:0/1299081255 shutdown_connections 2024-02-15T18:26:47.089 INFO:teuthology.orchestra.run.smithi038.stderr:/usr/bin/ceph: 2024-02-15T18:26:47.091+0000 7f9383c8f700 1 -- 172.21.15.38:0/1299081255 wait complete. 2024-02-15T18:26:47.779 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:26:46.524040+0000 mon.smithi038 (mon.0) 111 : cluster [DBG] mgrmap e17: smithi038.khesbl(active, since 6s) 2024-02-15T18:26:47.779 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:47.090165+0000 mon.smithi038 (mon.0) 112 : audit [DBG] from='client.? 172.21.15.38:0/1299081255' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-15T18:26:47.852 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl is-enabled firewalld.service 2024-02-15T18:26:47.863 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: disabled 2024-02-15T18:26:47.864 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: systemctl is-active firewalld.service 2024-02-15T18:26:47.872 INFO:teuthology.orchestra.run.smithi038.stderr:systemctl: inactive 2024-02-15T18:26:47.873 INFO:teuthology.orchestra.run.smithi038.stderr:firewalld.service is not enabled 2024-02-15T18:26:47.873 INFO:teuthology.orchestra.run.smithi038.stderr:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-15T18:26:47.876 INFO:teuthology.orchestra.run.smithi038.stderr:Ceph Dashboard is now available at: 2024-02-15T18:26:47.876 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-02-15T18:26:47.877 INFO:teuthology.orchestra.run.smithi038.stderr: URL: https://smithi038.front.sepia.ceph.com:8443/ 2024-02-15T18:26:47.877 INFO:teuthology.orchestra.run.smithi038.stderr: User: admin 2024-02-15T18:26:47.877 INFO:teuthology.orchestra.run.smithi038.stderr: Password: xmbjfop359 2024-02-15T18:26:47.877 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-02-15T18:26:47.877 INFO:teuthology.orchestra.run.smithi038.stderr:You can access the Ceph CLI with: 2024-02-15T18:26:47.877 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-02-15T18:26:47.877 INFO:teuthology.orchestra.run.smithi038.stderr: sudo /home/ubuntu/cephtest/cephadm shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-15T18:26:47.877 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-02-15T18:26:47.877 INFO:teuthology.orchestra.run.smithi038.stderr:Please consider enabling telemetry to help improve Ceph: 2024-02-15T18:26:47.877 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-02-15T18:26:47.877 INFO:teuthology.orchestra.run.smithi038.stderr: ceph telemetry on 2024-02-15T18:26:47.877 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-02-15T18:26:47.877 INFO:teuthology.orchestra.run.smithi038.stderr:For more information see: 2024-02-15T18:26:47.877 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-02-15T18:26:47.878 INFO:teuthology.orchestra.run.smithi038.stderr: https://docs.ceph.com/docs/pacific/mgr/telemetry/ 2024-02-15T18:26:47.878 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-02-15T18:26:47.878 INFO:teuthology.orchestra.run.smithi038.stderr:Bootstrap complete. 2024-02-15T18:26:47.878 INFO:teuthology.orchestra.run.smithi038.stderr:Releasing lock 140714512923112 on /run/cephadm/7e05753e-cc2f-11ee-95ba-87774f69a715.lock 2024-02-15T18:26:47.878 INFO:teuthology.orchestra.run.smithi038.stderr:Lock 140714512923112 released on /run/cephadm/7e05753e-cc2f-11ee-95ba-87774f69a715.lock 2024-02-15T18:26:47.914 INFO:tasks.cephadm:Fetching config... 2024-02-15T18:26:47.915 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-02-15T18:26:47.915 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-15T18:26:47.975 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-15T18:26:47.975 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-02-15T18:26:47.975 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-15T18:26:48.033 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-15T18:26:48.034 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-02-15T18:26:48.034 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/keyring of=/dev/stdout 2024-02-15T18:26:48.103 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-15T18:26:48.103 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-02-15T18:26:48.103 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-15T18:26:48.161 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-15T18:26:48.162 DEBUG:teuthology.orchestra.run.smithi038:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDX6uAM6XgBdhMfcotW1fV787J7VAuvY/lUylyipkOEtsG2fDfD4emNwcfHv/e4+q+yGOQ0eHlz5CLZcT0/oLEKq45LDsYfFsSzY7Hto2hnuZZWM0L3zgyYXC+WnSBCBSKsFabpkZ0gINwVfhpxAfBbgC7efO79phGZr79x24mtPkx8PunuwHQ8vNxB9UqGMqnLi9S8DFjtATugamb5Ru5EYX94mBH964Gtz9CvdHzZn3JXj/cFyQfdP9rpW4lo4dXIgBfqlSjDc2MFa5bD8aJbgGpyp/Yfx9kAZTZGzEWIJKHTpNHzaFsjJBBILIqj+jwZi1rVq+dh8iwnmqIRU5MtCGbxVuVbQ4i6U8fLwt4+v0uJuz5H1KPiwD6dzQ0YjNyVgXYuEJOq9DU9eZKX+6S23k65aUs0PQLS4TYQeSSVj/le12qTOY3I98ZAWfB8Abgg/6is7kKiOZMVySboZTpoc0XVhofFFs0R2XYLBl3u142kOMBoy5Eb6VXTMvfyKJU= ceph-7e05753e-cc2f-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-15T18:26:48.245 INFO:teuthology.orchestra.run.smithi038.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDX6uAM6XgBdhMfcotW1fV787J7VAuvY/lUylyipkOEtsG2fDfD4emNwcfHv/e4+q+yGOQ0eHlz5CLZcT0/oLEKq45LDsYfFsSzY7Hto2hnuZZWM0L3zgyYXC+WnSBCBSKsFabpkZ0gINwVfhpxAfBbgC7efO79phGZr79x24mtPkx8PunuwHQ8vNxB9UqGMqnLi9S8DFjtATugamb5Ru5EYX94mBH964Gtz9CvdHzZn3JXj/cFyQfdP9rpW4lo4dXIgBfqlSjDc2MFa5bD8aJbgGpyp/Yfx9kAZTZGzEWIJKHTpNHzaFsjJBBILIqj+jwZi1rVq+dh8iwnmqIRU5MtCGbxVuVbQ4i6U8fLwt4+v0uJuz5H1KPiwD6dzQ0YjNyVgXYuEJOq9DU9eZKX+6S23k65aUs0PQLS4TYQeSSVj/le12qTOY3I98ZAWfB8Abgg/6is7kKiOZMVySboZTpoc0XVhofFFs0R2XYLBl3u142kOMBoy5Eb6VXTMvfyKJU= ceph-7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:26:48.256 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDX6uAM6XgBdhMfcotW1fV787J7VAuvY/lUylyipkOEtsG2fDfD4emNwcfHv/e4+q+yGOQ0eHlz5CLZcT0/oLEKq45LDsYfFsSzY7Hto2hnuZZWM0L3zgyYXC+WnSBCBSKsFabpkZ0gINwVfhpxAfBbgC7efO79phGZr79x24mtPkx8PunuwHQ8vNxB9UqGMqnLi9S8DFjtATugamb5Ru5EYX94mBH964Gtz9CvdHzZn3JXj/cFyQfdP9rpW4lo4dXIgBfqlSjDc2MFa5bD8aJbgGpyp/Yfx9kAZTZGzEWIJKHTpNHzaFsjJBBILIqj+jwZi1rVq+dh8iwnmqIRU5MtCGbxVuVbQ4i6U8fLwt4+v0uJuz5H1KPiwD6dzQ0YjNyVgXYuEJOq9DU9eZKX+6S23k65aUs0PQLS4TYQeSSVj/le12qTOY3I98ZAWfB8Abgg/6is7kKiOZMVySboZTpoc0XVhofFFs0R2XYLBl3u142kOMBoy5Eb6VXTMvfyKJU= ceph-7e05753e-cc2f-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-15T18:26:48.300 INFO:teuthology.orchestra.run.smithi121.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDX6uAM6XgBdhMfcotW1fV787J7VAuvY/lUylyipkOEtsG2fDfD4emNwcfHv/e4+q+yGOQ0eHlz5CLZcT0/oLEKq45LDsYfFsSzY7Hto2hnuZZWM0L3zgyYXC+WnSBCBSKsFabpkZ0gINwVfhpxAfBbgC7efO79phGZr79x24mtPkx8PunuwHQ8vNxB9UqGMqnLi9S8DFjtATugamb5Ru5EYX94mBH964Gtz9CvdHzZn3JXj/cFyQfdP9rpW4lo4dXIgBfqlSjDc2MFa5bD8aJbgGpyp/Yfx9kAZTZGzEWIJKHTpNHzaFsjJBBILIqj+jwZi1rVq+dh8iwnmqIRU5MtCGbxVuVbQ4i6U8fLwt4+v0uJuz5H1KPiwD6dzQ0YjNyVgXYuEJOq9DU9eZKX+6S23k65aUs0PQLS4TYQeSSVj/le12qTOY3I98ZAWfB8Abgg/6is7kKiOZMVySboZTpoc0XVhofFFs0R2XYLBl3u142kOMBoy5Eb6VXTMvfyKJU= ceph-7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:26:48.311 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-15T18:26:51.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.499+0000 7f603049d700 1 -- 172.21.15.38:0/1909133595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6028060280 msgr2=0x7f60280f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:51.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.499+0000 7f603049d700 1 --2- 172.21.15.38:0/1909133595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6028060280 0x7f60280f90f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f6024009fb0 tx=0x7f6024009b30).stop 2024-02-15T18:26:51.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.499+0000 7f603049d700 1 -- 172.21.15.38:0/1909133595 shutdown_connections 2024-02-15T18:26:51.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.499+0000 7f603049d700 1 --2- 172.21.15.38:0/1909133595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6028060280 0x7f60280f90f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:51.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.499+0000 7f603049d700 1 -- 172.21.15.38:0/1909133595 >> 172.21.15.38:0/1909133595 conn(0x7f60280f0b70 msgr2=0x7f60280f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:51.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.499+0000 7f603049d700 1 -- 172.21.15.38:0/1909133595 shutdown_connections 2024-02-15T18:26:51.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.499+0000 7f603049d700 1 -- 172.21.15.38:0/1909133595 wait complete. 2024-02-15T18:26:51.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.499+0000 7f603049d700 1 Processor -- start 2024-02-15T18:26:51.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.500+0000 7f603049d700 1 -- start start 2024-02-15T18:26:51.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.500+0000 7f603049d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6028060280 0x7f60280f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:51.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.500+0000 7f603049d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60280f4ff0 con 0x7f6028060280 2024-02-15T18:26:51.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.500+0000 7f602e239700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6028060280 0x7f60280f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:51.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.500+0000 7f602e239700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6028060280 0x7f60280f6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52634/0 (socket says 172.21.15.38:52634) 2024-02-15T18:26:51.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.500+0000 7f602e239700 1 -- 172.21.15.38:0/1877115081 learned_addr learned my addr 172.21.15.38:0/1877115081 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:26:51.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.500+0000 7f602e239700 1 -- 172.21.15.38:0/1877115081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6024005040 con 0x7f6028060280 2024-02-15T18:26:51.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.500+0000 7f602e239700 1 --2- 172.21.15.38:0/1877115081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6028060280 0x7f60280f6b70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f6024009830 tx=0x7f602400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:51.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.500+0000 7f601effd700 1 -- 172.21.15.38:0/1877115081 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6024009320 con 0x7f6028060280 2024-02-15T18:26:51.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.500+0000 7f601effd700 1 -- 172.21.15.38:0/1877115081 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6024013380 con 0x7f6028060280 2024-02-15T18:26:51.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.500+0000 7f603049d700 1 -- 172.21.15.38:0/1877115081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60280f5190 con 0x7f6028060280 2024-02-15T18:26:51.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.500+0000 7f601effd700 1 -- 172.21.15.38:0/1877115081 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6024014420 con 0x7f6028060280 2024-02-15T18:26:51.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.500+0000 7f603049d700 1 -- 172.21.15.38:0/1877115081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60280f5620 con 0x7f6028060280 2024-02-15T18:26:51.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.501+0000 7f601effd700 1 -- 172.21.15.38:0/1877115081 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f6024014580 con 0x7f6028060280 2024-02-15T18:26:51.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.501+0000 7f603049d700 1 -- 172.21.15.38:0/1877115081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6028059100 con 0x7f6028060280 2024-02-15T18:26:51.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.501+0000 7f601effd700 1 --2- 172.21.15.38:0/1877115081 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f601402ff60 0x7f6014032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:51.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.502+0000 7f601effd700 1 -- 172.21.15.38:0/1877115081 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f6024040e30 con 0x7f6028060280 2024-02-15T18:26:51.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.502+0000 7f602da38700 1 --2- 172.21.15.38:0/1877115081 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f601402ff60 0x7f6014032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:51.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.502+0000 7f602da38700 1 --2- 172.21.15.38:0/1877115081 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f601402ff60 0x7f6014032420 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f60180096a0 tx=0x7f6018006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:51.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.506+0000 7f601effd700 1 -- 172.21.15.38:0/1877115081 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f602401c890 con 0x7f6028060280 2024-02-15T18:26:51.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.670+0000 7f603049d700 1 -- 172.21.15.38:0/1877115081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f6028045ad0 con 0x7f6028060280 2024-02-15T18:26:51.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.673+0000 7f601effd700 1 -- 172.21.15.38:0/1877115081 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v11)=0 v11) v1 ==== 125+0+0 (secure 0 0 0) 0x7f602401c890 con 0x7f6028060280 2024-02-15T18:26:51.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.675+0000 7f603049d700 1 -- 172.21.15.38:0/1877115081 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f601402ff60 msgr2=0x7f6014032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:51.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.675+0000 7f603049d700 1 --2- 172.21.15.38:0/1877115081 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f601402ff60 0x7f6014032420 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f60180096a0 tx=0x7f6018006b40).stop 2024-02-15T18:26:51.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.675+0000 7f603049d700 1 -- 172.21.15.38:0/1877115081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6028060280 msgr2=0x7f60280f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:51.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.675+0000 7f603049d700 1 --2- 172.21.15.38:0/1877115081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6028060280 0x7f60280f6b70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f6024009830 tx=0x7f602400b040).stop 2024-02-15T18:26:51.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.675+0000 7f603049d700 1 -- 172.21.15.38:0/1877115081 shutdown_connections 2024-02-15T18:26:51.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.675+0000 7f603049d700 1 --2- 172.21.15.38:0/1877115081 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f601402ff60 0x7f6014032420 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:51.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.675+0000 7f603049d700 1 --2- 172.21.15.38:0/1877115081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6028060280 0x7f60280f6b70 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0x7f6024009830 tx=0x7f602400b040).stop 2024-02-15T18:26:51.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.675+0000 7f603049d700 1 -- 172.21.15.38:0/1877115081 >> 172.21.15.38:0/1877115081 conn(0x7f60280f0b70 msgr2=0x7f60280f1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:51.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.675+0000 7f603049d700 1 -- 172.21.15.38:0/1877115081 shutdown_connections 2024-02-15T18:26:51.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:26:51.675+0000 7f603049d700 1 -- 172.21.15.38:0/1877115081 wait complete. 2024-02-15T18:26:51.779 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:50.474531+0000 mon.smithi038 (mon.0) 113 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:51.779 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:50.879014+0000 mon.smithi038 (mon.0) 114 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:26:52.969 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:52 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:26:50.885441+0000 mgr.smithi038.khesbl (mgr.14162) 5 : cephadm [INF] Deploying daemon alertmanager.smithi038 on smithi038 2024-02-15T18:26:52.969 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:26:52 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:26:51.675188+0000 mon.smithi038 (mon.0) 115 : audit [INF] from='client.? 172.21.15.38:0/1877115081' entity='client.admin' 2024-02-15T18:26:52.990 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi121 2024-02-15T18:26:52.990 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-02-15T18:26:52.990 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/etc/ceph/ceph.conf 2024-02-15T18:26:53.012 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-02-15T18:26:53.012 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-15T18:26:53.074 INFO:tasks.cephadm:Adding host smithi121 to orchestrator... 2024-02-15T18:26:53.074 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph orch host add smithi121 2024-02-15T18:27:01.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:00.216457+0000 mgr.smithi038.khesbl (mgr.14162) 6 : cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:03.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:02.216771+0000 mgr.smithi038.khesbl (mgr.14162) 7 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:05.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:04.217012+0000 mgr.smithi038.khesbl (mgr.14162) 8 : cluster [DBG] pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:07.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:06.217362+0000 mgr.smithi038.khesbl (mgr.14162) 9 : cluster [DBG] pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:09.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:08.217595+0000 mgr.smithi038.khesbl (mgr.14162) 10 : cluster [DBG] pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:09.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:09.192229+0000 mon.smithi038 (mon.0) 116 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:27:09.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:09.193225+0000 mon.smithi038 (mon.0) 117 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi038", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-15T18:27:09.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:09.202753+0000 mon.smithi038 (mon.0) 118 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi038", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-15T18:27:09.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:09.203339+0000 mon.smithi038 (mon.0) 119 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:27:10.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:27:09.203927+0000 mgr.smithi038.khesbl (mgr.14162) 11 : cephadm [INF] Deploying daemon crash.smithi038 on smithi038 2024-02-15T18:27:11.778 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:10.217788+0000 mgr.smithi038.khesbl (mgr.14162) 12 : cluster [DBG] pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:13.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:11.568834+0000 mon.smithi038 (mon.0) 120 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:27:13.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:11.645983+0000 mon.smithi038 (mon.0) 121 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:27:13.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:11.647843+0000 mon.smithi038 (mon.0) 122 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:27:13.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:11.648240+0000 mon.smithi038 (mon.0) 123 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-15T18:27:13.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:11.648673+0000 mgr.smithi038.khesbl (mgr.14162) 13 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-15T18:27:13.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:11.651081+0000 mon.smithi038 (mon.0) 124 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:27:13.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:27:11.654270+0000 mgr.smithi038.khesbl (mgr.14162) 14 : cephadm [INF] Deploying daemon grafana.smithi038 on smithi038 2024-02-15T18:27:14.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:12.218063+0000 mgr.smithi038.khesbl (mgr.14162) 15 : cluster [DBG] pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:15.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:14.218347+0000 mgr.smithi038.khesbl (mgr.14162) 16 : cluster [DBG] pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:16.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.951+0000 7f31c2fc8700 1 -- 172.21.15.121:0/1849008595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31bc0f47c0 msgr2=0x7f31bc0f4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:16.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.951+0000 7f31c2fc8700 1 --2- 172.21.15.121:0/1849008595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31bc0f47c0 0x7f31bc0f4be0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f31b0009fb0 tx=0x7f31b0009b30).stop 2024-02-15T18:27:16.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.951+0000 7f31c2fc8700 1 -- 172.21.15.121:0/1849008595 shutdown_connections 2024-02-15T18:27:16.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.951+0000 7f31c2fc8700 1 --2- 172.21.15.121:0/1849008595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31bc0f47c0 0x7f31bc0f4be0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:16.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.951+0000 7f31c2fc8700 1 -- 172.21.15.121:0/1849008595 >> 172.21.15.121:0/1849008595 conn(0x7f31bc0f02e0 msgr2=0x7f31bc0f2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:16.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.951+0000 7f31c2fc8700 1 -- 172.21.15.121:0/1849008595 shutdown_connections 2024-02-15T18:27:16.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.951+0000 7f31c2fc8700 1 -- 172.21.15.121:0/1849008595 wait complete. 2024-02-15T18:27:16.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.951+0000 7f31c2fc8700 1 Processor -- start 2024-02-15T18:27:16.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.951+0000 7f31c2fc8700 1 -- start start 2024-02-15T18:27:16.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.952+0000 7f31c2fc8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31bc1068d0 0x7f31bc106cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:16.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.952+0000 7f31c2fc8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc1071e0 con 0x7f31bc1068d0 2024-02-15T18:27:16.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.952+0000 7f31c0d64700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31bc1068d0 0x7f31bc106cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:16.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.952+0000 7f31c0d64700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31bc1068d0 0x7f31bc106cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.121:51772/0 (socket says 172.21.15.121:51772) 2024-02-15T18:27:16.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.952+0000 7f31c0d64700 1 -- 172.21.15.121:0/1406924044 learned_addr learned my addr 172.21.15.121:0/1406924044 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-02-15T18:27:16.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.952+0000 7f31c0d64700 1 -- 172.21.15.121:0/1406924044 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b0005040 con 0x7f31bc1068d0 2024-02-15T18:27:16.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.952+0000 7f31c0d64700 1 --2- 172.21.15.121:0/1406924044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31bc1068d0 0x7f31bc106cf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f31b0009f80 tx=0x7f31b0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:16.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.953+0000 7f31b97fa700 1 -- 172.21.15.121:0/1406924044 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f31b0013380 con 0x7f31bc1068d0 2024-02-15T18:27:16.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.953+0000 7f31c2fc8700 1 -- 172.21.15.121:0/1406924044 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31bc1073e0 con 0x7f31bc1068d0 2024-02-15T18:27:16.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.953+0000 7f31b97fa700 1 -- 172.21.15.121:0/1406924044 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f31b0013790 con 0x7f31bc1068d0 2024-02-15T18:27:16.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.953+0000 7f31b97fa700 1 -- 172.21.15.121:0/1406924044 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f31b0014420 con 0x7f31bc1068d0 2024-02-15T18:27:16.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.953+0000 7f31c2fc8700 1 -- 172.21.15.121:0/1406924044 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31bc107f30 con 0x7f31bc1068d0 2024-02-15T18:27:16.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.953+0000 7f31b97fa700 1 -- 172.21.15.121:0/1406924044 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f31b0025060 con 0x7f31bc1068d0 2024-02-15T18:27:16.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.953+0000 7f31c2fc8700 1 -- 172.21.15.121:0/1406924044 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31bc059100 con 0x7f31bc1068d0 2024-02-15T18:27:16.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.954+0000 7f31b97fa700 1 --2- 172.21.15.121:0/1406924044 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f31ac02ff60 0x7f31ac032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:16.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.954+0000 7f31b97fa700 1 -- 172.21.15.121:0/1406924044 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f31b0040e30 con 0x7f31bc1068d0 2024-02-15T18:27:16.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.954+0000 7f31bbfff700 1 --2- 172.21.15.121:0/1406924044 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f31ac02ff60 0x7f31ac032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:16.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.954+0000 7f31bbfff700 1 --2- 172.21.15.121:0/1406924044 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f31ac02ff60 0x7f31ac032420 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f31a80096a0 tx=0x7f31a8006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:16.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:16.959+0000 7f31b97fa700 1 -- 172.21.15.121:0/1406924044 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f31b000f280 con 0x7f31bc1068d0 2024-02-15T18:27:17.123 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:17.128+0000 7f31c2fc8700 1 -- 172.21.15.121:0/1406924044 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi121", "target": ["mon-mgr", ""]}) v1 -- 0x7f31bc1086d0 con 0x7f31ac02ff60 2024-02-15T18:27:17.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:16.218654+0000 mgr.smithi038.khesbl (mgr.14162) 17 : cluster [DBG] pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:17.718 INFO:teuthology.orchestra.run.smithi121.stdout:Added host 'smithi121' 2024-02-15T18:27:17.719 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:17.722+0000 7f31b97fa700 1 -- 172.21.15.121:0/1406924044 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+23 (secure 0 0 0) 0x7f31bc1086d0 con 0x7f31ac02ff60 2024-02-15T18:27:17.720 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:17.725+0000 7f31c2fc8700 1 -- 172.21.15.121:0/1406924044 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f31ac02ff60 msgr2=0x7f31ac032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:17.720 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:17.725+0000 7f31c2fc8700 1 --2- 172.21.15.121:0/1406924044 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f31ac02ff60 0x7f31ac032420 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f31a80096a0 tx=0x7f31a8006b40).stop 2024-02-15T18:27:17.720 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:17.725+0000 7f31c2fc8700 1 -- 172.21.15.121:0/1406924044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31bc1068d0 msgr2=0x7f31bc106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:17.720 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:17.725+0000 7f31c2fc8700 1 --2- 172.21.15.121:0/1406924044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31bc1068d0 0x7f31bc106cf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f31b0009f80 tx=0x7f31b0009b30).stop 2024-02-15T18:27:17.720 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:17.725+0000 7f31c2fc8700 1 -- 172.21.15.121:0/1406924044 shutdown_connections 2024-02-15T18:27:17.720 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:17.725+0000 7f31c2fc8700 1 --2- 172.21.15.121:0/1406924044 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f31ac02ff60 0x7f31ac032420 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:17.721 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:17.725+0000 7f31c2fc8700 1 --2- 172.21.15.121:0/1406924044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31bc1068d0 0x7f31bc106cf0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:17.721 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:17.725+0000 7f31c2fc8700 1 -- 172.21.15.121:0/1406924044 >> 172.21.15.121:0/1406924044 conn(0x7f31bc0f02e0 msgr2=0x7f31bc0f0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:17.721 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:17.725+0000 7f31c2fc8700 1 -- 172.21.15.121:0/1406924044 shutdown_connections 2024-02-15T18:27:17.721 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:17.725+0000 7f31c2fc8700 1 -- 172.21.15.121:0/1406924044 wait complete. 2024-02-15T18:27:18.410 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph orch host ls --format=json 2024-02-15T18:27:19.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:17.129575+0000 mgr.smithi038.khesbl (mgr.14162) 18 : audit [DBG] from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi121", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:27:19.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:17.722040+0000 mon.smithi038 (mon.0) 125 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:27:19.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:27:17.722453+0000 mgr.smithi038.khesbl (mgr.14162) 19 : cephadm [INF] Added host smithi121 2024-02-15T18:27:20.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:18.219009+0000 mgr.smithi038.khesbl (mgr.14162) 20 : cluster [DBG] pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:20.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.173+0000 7fb4ee3c9700 1 -- 172.21.15.121:0/2947628922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e80f6510 msgr2=0x7fb4e80f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:20.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.173+0000 7fb4ee3c9700 1 --2- 172.21.15.121:0/2947628922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e80f6510 0x7fb4e80f6930 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fb4d0009fb0 tx=0x7fb4d0009b30).stop 2024-02-15T18:27:20.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.173+0000 7fb4ee3c9700 1 -- 172.21.15.121:0/2947628922 shutdown_connections 2024-02-15T18:27:20.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.173+0000 7fb4ee3c9700 1 --2- 172.21.15.121:0/2947628922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e80f6510 0x7fb4e80f6930 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:20.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.173+0000 7fb4ee3c9700 1 -- 172.21.15.121:0/2947628922 >> 172.21.15.121:0/2947628922 conn(0x7fb4e80f1c70 msgr2=0x7fb4e80f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:20.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.173+0000 7fb4ee3c9700 1 -- 172.21.15.121:0/2947628922 shutdown_connections 2024-02-15T18:27:20.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.173+0000 7fb4ee3c9700 1 -- 172.21.15.121:0/2947628922 wait complete. 2024-02-15T18:27:20.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.173+0000 7fb4ee3c9700 1 Processor -- start 2024-02-15T18:27:20.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.174+0000 7fb4ee3c9700 1 -- start start 2024-02-15T18:27:20.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.174+0000 7fb4ee3c9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e80f6510 0x7fb4e81067e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:20.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.174+0000 7fb4ee3c9700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e8106cd0 con 0x7fb4e80f6510 2024-02-15T18:27:20.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.174+0000 7fb4e7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e80f6510 0x7fb4e81067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:20.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.174+0000 7fb4e7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e80f6510 0x7fb4e81067e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.121:40118/0 (socket says 172.21.15.121:40118) 2024-02-15T18:27:20.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.174+0000 7fb4e7fff700 1 -- 172.21.15.121:0/2784829263 learned_addr learned my addr 172.21.15.121:0/2784829263 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-02-15T18:27:20.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.174+0000 7fb4e7fff700 1 -- 172.21.15.121:0/2784829263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d0005040 con 0x7fb4e80f6510 2024-02-15T18:27:20.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.174+0000 7fb4e7fff700 1 --2- 172.21.15.121:0/2784829263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e80f6510 0x7fb4e81067e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fb4d0005010 tx=0x7fb4d000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:20.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.174+0000 7fb4e4ff9700 1 -- 172.21.15.121:0/2784829263 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb4d0009320 con 0x7fb4e80f6510 2024-02-15T18:27:20.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.174+0000 7fb4ee3c9700 1 -- 172.21.15.121:0/2784829263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e8106ed0 con 0x7fb4e80f6510 2024-02-15T18:27:20.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.175+0000 7fb4e4ff9700 1 -- 172.21.15.121:0/2784829263 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb4d0013380 con 0x7fb4e80f6510 2024-02-15T18:27:20.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.175+0000 7fb4e4ff9700 1 -- 172.21.15.121:0/2784829263 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb4d0014e50 con 0x7fb4e80f6510 2024-02-15T18:27:20.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.175+0000 7fb4ee3c9700 1 -- 172.21.15.121:0/2784829263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e8107360 con 0x7fb4e80f6510 2024-02-15T18:27:20.172 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.175+0000 7fb4e4ff9700 1 -- 172.21.15.121:0/2784829263 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fb4d00095f0 con 0x7fb4e80f6510 2024-02-15T18:27:20.172 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.175+0000 7fb4ee3c9700 1 -- 172.21.15.121:0/2784829263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e8045ad0 con 0x7fb4e80f6510 2024-02-15T18:27:20.172 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.175+0000 7fb4e4ff9700 1 --2- 172.21.15.121:0/2784829263 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb4d402ff10 0x7fb4d40323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:20.172 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.175+0000 7fb4e4ff9700 1 -- 172.21.15.121:0/2784829263 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fb4d0040fa0 con 0x7fb4e80f6510 2024-02-15T18:27:20.172 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.176+0000 7fb4e77fe700 1 --2- 172.21.15.121:0/2784829263 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb4d402ff10 0x7fb4d40323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:20.172 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.176+0000 7fb4e77fe700 1 --2- 172.21.15.121:0/2784829263 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb4d402ff10 0x7fb4d40323d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fb4d80096a0 tx=0x7fb4d8006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:20.175 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.180+0000 7fb4e4ff9700 1 -- 172.21.15.121:0/2784829263 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb4d0014420 con 0x7fb4e80f6510 2024-02-15T18:27:20.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.350+0000 7fb4ee3c9700 1 -- 172.21.15.121:0/2784829263 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4e80f2e20 con 0x7fb4d402ff10 2024-02-15T18:27:20.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.352+0000 7fb4e4ff9700 1 -- 172.21.15.121:0/2784829263 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+156 (secure 0 0 0) 0x7fb4e80f2e20 con 0x7fb4d402ff10 2024-02-15T18:27:20.347 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:27:20.347 INFO:teuthology.orchestra.run.smithi121.stdout:[{"addr": "172.21.15.38", "hostname": "smithi038", "labels": [], "status": ""}, {"addr": "smithi121", "hostname": "smithi121", "labels": [], "status": ""}] 2024-02-15T18:27:20.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.353+0000 7fb4ee3c9700 1 -- 172.21.15.121:0/2784829263 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb4d402ff10 msgr2=0x7fb4d40323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:20.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.353+0000 7fb4ee3c9700 1 --2- 172.21.15.121:0/2784829263 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb4d402ff10 0x7fb4d40323d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fb4d80096a0 tx=0x7fb4d8006b40).stop 2024-02-15T18:27:20.352 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.353+0000 7fb4ee3c9700 1 -- 172.21.15.121:0/2784829263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e80f6510 msgr2=0x7fb4e81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:20.352 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.353+0000 7fb4ee3c9700 1 --2- 172.21.15.121:0/2784829263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e80f6510 0x7fb4e81067e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fb4d0005010 tx=0x7fb4d000b040).stop 2024-02-15T18:27:20.352 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.353+0000 7fb4ee3c9700 1 -- 172.21.15.121:0/2784829263 shutdown_connections 2024-02-15T18:27:20.352 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.353+0000 7fb4ee3c9700 1 --2- 172.21.15.121:0/2784829263 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb4d402ff10 0x7fb4d40323d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:20.352 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.353+0000 7fb4ee3c9700 1 --2- 172.21.15.121:0/2784829263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e80f6510 0x7fb4e81067e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:20.352 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.353+0000 7fb4ee3c9700 1 -- 172.21.15.121:0/2784829263 >> 172.21.15.121:0/2784829263 conn(0x7fb4e80f1c70 msgr2=0x7fb4e80f2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:20.352 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.353+0000 7fb4ee3c9700 1 -- 172.21.15.121:0/2784829263 shutdown_connections 2024-02-15T18:27:20.352 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:20.353+0000 7fb4ee3c9700 1 -- 172.21.15.121:0/2784829263 wait complete. 2024-02-15T18:27:21.025 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-15T18:27:21.026 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph osd crush tunables default 2024-02-15T18:27:21.841 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:20.219243+0000 mgr.smithi038.khesbl (mgr.14162) 21 : cluster [DBG] pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:21.842 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:20.351441+0000 mgr.smithi038.khesbl (mgr.14162) 22 : audit [DBG] from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-15T18:27:23.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.308+0000 7f4678b88700 1 -- 172.21.15.38:0/2284193096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46740f7ae0 msgr2=0x7f46740f7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:23.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.308+0000 7f4678b88700 1 --2- 172.21.15.38:0/2284193096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46740f7ae0 0x7f46740f7f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f4664009fb0 tx=0x7f4664009b30).stop 2024-02-15T18:27:23.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.309+0000 7f4678b88700 1 -- 172.21.15.38:0/2284193096 shutdown_connections 2024-02-15T18:27:23.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.309+0000 7f4678b88700 1 --2- 172.21.15.38:0/2284193096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46740f7ae0 0x7f46740f7f00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:23.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.309+0000 7f4678b88700 1 -- 172.21.15.38:0/2284193096 >> 172.21.15.38:0/2284193096 conn(0x7f46740f0260 msgr2=0x7f46740f2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:23.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.309+0000 7f4678b88700 1 -- 172.21.15.38:0/2284193096 shutdown_connections 2024-02-15T18:27:23.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.309+0000 7f4678b88700 1 -- 172.21.15.38:0/2284193096 wait complete. 2024-02-15T18:27:23.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.309+0000 7f4678b88700 1 Processor -- start 2024-02-15T18:27:23.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.309+0000 7f4678b88700 1 -- start start 2024-02-15T18:27:23.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.309+0000 7f4678b88700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46740f7ae0 0x7f46740f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:23.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.309+0000 7f4678b88700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46740f5830 con 0x7f46740f7ae0 2024-02-15T18:27:23.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.310+0000 7f467259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46740f7ae0 0x7f46740f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:23.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.310+0000 7f467259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46740f7ae0 0x7f46740f73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36684/0 (socket says 172.21.15.38:36684) 2024-02-15T18:27:23.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.310+0000 7f467259c700 1 -- 172.21.15.38:0/1605329543 learned_addr learned my addr 172.21.15.38:0/1605329543 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:27:23.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.310+0000 7f467259c700 1 -- 172.21.15.38:0/1605329543 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4664005040 con 0x7f46740f7ae0 2024-02-15T18:27:23.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.310+0000 7f467259c700 1 --2- 172.21.15.38:0/1605329543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46740f7ae0 0x7f46740f73b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f4664005010 tx=0x7f466400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:23.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.310+0000 7f4662ffd700 1 -- 172.21.15.38:0/1605329543 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4664009320 con 0x7f46740f7ae0 2024-02-15T18:27:23.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.310+0000 7f4662ffd700 1 -- 172.21.15.38:0/1605329543 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4664013380 con 0x7f46740f7ae0 2024-02-15T18:27:23.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.310+0000 7f4678b88700 1 -- 172.21.15.38:0/1605329543 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46740f59d0 con 0x7f46740f7ae0 2024-02-15T18:27:23.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.310+0000 7f4662ffd700 1 -- 172.21.15.38:0/1605329543 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4664014e50 con 0x7f46740f7ae0 2024-02-15T18:27:23.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.310+0000 7f4678b88700 1 -- 172.21.15.38:0/1605329543 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46740f5e60 con 0x7f46740f7ae0 2024-02-15T18:27:23.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.311+0000 7f4662ffd700 1 -- 172.21.15.38:0/1605329543 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f46640263a0 con 0x7f46740f7ae0 2024-02-15T18:27:23.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.311+0000 7f4662ffd700 1 --2- 172.21.15.38:0/1605329543 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f465c02ff10 0x7f465c0323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:23.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.311+0000 7f4662ffd700 1 -- 172.21.15.38:0/1605329543 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f4664040e90 con 0x7f46740f7ae0 2024-02-15T18:27:23.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.311+0000 7f4671d9b700 1 --2- 172.21.15.38:0/1605329543 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f465c02ff10 0x7f465c0323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:23.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.312+0000 7f4678b88700 1 -- 172.21.15.38:0/1605329543 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4674059100 con 0x7f46740f7ae0 2024-02-15T18:27:23.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.312+0000 7f4671d9b700 1 --2- 172.21.15.38:0/1605329543 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f465c02ff10 0x7f465c0323d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f46680096a0 tx=0x7f4668006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:23.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.319+0000 7f4662ffd700 1 -- 172.21.15.38:0/1605329543 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f46640672b0 con 0x7f46740f7ae0 2024-02-15T18:27:23.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.529+0000 7f4678b88700 1 -- 172.21.15.38:0/1605329543 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f46740f64c0 con 0x7f46740f7ae0 2024-02-15T18:27:23.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.758+0000 7f4662ffd700 1 -- 172.21.15.38:0/1605329543 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v5) v1 ==== 124+0+0 (secure 0 0 0) 0x7f466402b030 con 0x7f46740f7ae0 2024-02-15T18:27:23.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.760+0000 7f4678b88700 1 -- 172.21.15.38:0/1605329543 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f465c02ff10 msgr2=0x7f465c0323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:23.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.760+0000 7f4678b88700 1 --2- 172.21.15.38:0/1605329543 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f465c02ff10 0x7f465c0323d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f46680096a0 tx=0x7f4668006b40).stop 2024-02-15T18:27:23.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.760+0000 7f4678b88700 1 -- 172.21.15.38:0/1605329543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46740f7ae0 msgr2=0x7f46740f73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:23.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.760+0000 7f4678b88700 1 --2- 172.21.15.38:0/1605329543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46740f7ae0 0x7f46740f73b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f4664005010 tx=0x7f466400b040).stop 2024-02-15T18:27:23.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.760+0000 7f4678b88700 1 -- 172.21.15.38:0/1605329543 shutdown_connections 2024-02-15T18:27:23.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.760+0000 7f4678b88700 1 --2- 172.21.15.38:0/1605329543 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f465c02ff10 0x7f465c0323d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:23.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.760+0000 7f4678b88700 1 --2- 172.21.15.38:0/1605329543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46740f7ae0 0x7f46740f73b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:23.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.760+0000 7f4678b88700 1 -- 172.21.15.38:0/1605329543 >> 172.21.15.38:0/1605329543 conn(0x7f46740f0260 msgr2=0x7f46740f0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:23.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.760+0000 7f4678b88700 1 -- 172.21.15.38:0/1605329543 shutdown_connections 2024-02-15T18:27:23.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:27:23.760+0000 7f4678b88700 1 -- 172.21.15.38:0/1605329543 wait complete. 2024-02-15T18:27:23.756 INFO:teuthology.orchestra.run.smithi038.stderr:adjusted tunables profile to default 2024-02-15T18:27:24.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:22.219464+0000 mgr.smithi038.khesbl (mgr.14162) 23 : cluster [DBG] pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:24.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:23.530711+0000 mon.smithi038 (mon.0) 126 : audit [INF] from='client.? 172.21.15.38:0/1605329543' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-15T18:27:25.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:23.759015+0000 mon.smithi038 (mon.0) 127 : audit [INF] from='client.? 172.21.15.38:0/1605329543' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-15T18:27:25.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:23.759090+0000 mon.smithi038 (mon.0) 128 : cluster [DBG] osdmap e5: 0 total, 0 up, 0 in 2024-02-15T18:27:26.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:24.219703+0000 mgr.smithi038.khesbl (mgr.14162) 24 : cluster [DBG] pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:27.946 INFO:tasks.cephadm:Adding mon.smithi038 on smithi038 2024-02-15T18:27:27.946 INFO:tasks.cephadm:Adding mon.smithi121 on smithi121 2024-02-15T18:27:27.946 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph orch apply mon '2;smithi038:172.21.15.38=smithi038;smithi121:172.21.15.121=smithi121' 2024-02-15T18:27:28.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:26.219945+0000 mgr.smithi038.khesbl (mgr.14162) 25 : cluster [DBG] pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:29.673 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.666+0000 7faff5fee700 1 -- 172.21.15.121:0/2720763531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faff00f6510 msgr2=0x7faff00f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:29.674 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.666+0000 7faff5fee700 1 --2- 172.21.15.121:0/2720763531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faff00f6510 0x7faff00f6930 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fafe4009fb0 tx=0x7fafe4009b30).stop 2024-02-15T18:27:29.674 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.666+0000 7faff5fee700 1 -- 172.21.15.121:0/2720763531 shutdown_connections 2024-02-15T18:27:29.674 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.666+0000 7faff5fee700 1 --2- 172.21.15.121:0/2720763531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faff00f6510 0x7faff00f6930 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:29.674 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.666+0000 7faff5fee700 1 -- 172.21.15.121:0/2720763531 >> 172.21.15.121:0/2720763531 conn(0x7faff00f1c70 msgr2=0x7faff00f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:29.674 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.666+0000 7faff5fee700 1 -- 172.21.15.121:0/2720763531 shutdown_connections 2024-02-15T18:27:29.674 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.666+0000 7faff5fee700 1 -- 172.21.15.121:0/2720763531 wait complete. 2024-02-15T18:27:29.674 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.667+0000 7faff5fee700 1 Processor -- start 2024-02-15T18:27:29.674 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.667+0000 7faff5fee700 1 -- start start 2024-02-15T18:27:29.674 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.667+0000 7faff5fee700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faff00f6510 0x7faff01067e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:29.674 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.667+0000 7faff5fee700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff0106cd0 con 0x7faff00f6510 2024-02-15T18:27:29.674 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.667+0000 7fafef7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faff00f6510 0x7faff01067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:29.674 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.667+0000 7fafef7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faff00f6510 0x7faff01067e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.121:37306/0 (socket says 172.21.15.121:37306) 2024-02-15T18:27:29.674 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.667+0000 7fafef7fe700 1 -- 172.21.15.121:0/265582402 learned_addr learned my addr 172.21.15.121:0/265582402 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-02-15T18:27:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.668+0000 7fafef7fe700 1 -- 172.21.15.121:0/265582402 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe4005040 con 0x7faff00f6510 2024-02-15T18:27:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.668+0000 7fafef7fe700 1 --2- 172.21.15.121:0/265582402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faff00f6510 0x7faff01067e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fafe4005010 tx=0x7fafe400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.668+0000 7fafd7fff700 1 -- 172.21.15.121:0/265582402 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fafe4009320 con 0x7faff00f6510 2024-02-15T18:27:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.668+0000 7faff5fee700 1 -- 172.21.15.121:0/265582402 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff0106ed0 con 0x7faff00f6510 2024-02-15T18:27:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.668+0000 7fafd7fff700 1 -- 172.21.15.121:0/265582402 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fafe4013380 con 0x7faff00f6510 2024-02-15T18:27:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.668+0000 7faff5fee700 1 -- 172.21.15.121:0/265582402 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff0107360 con 0x7faff00f6510 2024-02-15T18:27:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.668+0000 7fafd7fff700 1 -- 172.21.15.121:0/265582402 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fafe4014e50 con 0x7faff00f6510 2024-02-15T18:27:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.669+0000 7fafd7fff700 1 -- 172.21.15.121:0/265582402 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fafe4014420 con 0x7faff00f6510 2024-02-15T18:27:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.669+0000 7faff5fee700 1 -- 172.21.15.121:0/265582402 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff0045ad0 con 0x7faff00f6510 2024-02-15T18:27:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.669+0000 7fafd7fff700 1 --2- 172.21.15.121:0/265582402 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fafd802ff10 0x7fafd80323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.669+0000 7fafd7fff700 1 -- 172.21.15.121:0/265582402 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fafe4040e60 con 0x7faff00f6510 2024-02-15T18:27:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.669+0000 7fafeeffd700 1 --2- 172.21.15.121:0/265582402 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fafd802ff10 0x7fafd80323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.669+0000 7fafeeffd700 1 --2- 172.21.15.121:0/265582402 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fafd802ff10 0x7fafd80323d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fafe00096a0 tx=0x7fafe0006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.673+0000 7fafd7fff700 1 -- 172.21.15.121:0/265582402 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fafe4014740 con 0x7faff00f6510 2024-02-15T18:27:29.843 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.847+0000 7faff5fee700 1 -- 172.21.15.121:0/265582402 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi038:172.21.15.38=smithi038;smithi121:172.21.15.121=smithi121", "target": ["mon-mgr", ""]}) v1 -- 0x7faff00f2ed0 con 0x7fafd802ff10 2024-02-15T18:27:29.848 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.852+0000 7fafd7fff700 1 -- 172.21.15.121:0/265582402 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7faff00f2ed0 con 0x7fafd802ff10 2024-02-15T18:27:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:Scheduled mon update... 2024-02-15T18:27:29.850 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.853+0000 7faff5fee700 1 -- 172.21.15.121:0/265582402 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fafd802ff10 msgr2=0x7fafd80323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:29.850 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.853+0000 7faff5fee700 1 --2- 172.21.15.121:0/265582402 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fafd802ff10 0x7fafd80323d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fafe00096a0 tx=0x7fafe0006b40).stop 2024-02-15T18:27:29.850 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.853+0000 7faff5fee700 1 -- 172.21.15.121:0/265582402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faff00f6510 msgr2=0x7faff01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:29.850 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.853+0000 7faff5fee700 1 --2- 172.21.15.121:0/265582402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faff00f6510 0x7faff01067e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fafe4005010 tx=0x7fafe400b040).stop 2024-02-15T18:27:29.850 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.854+0000 7faff5fee700 1 -- 172.21.15.121:0/265582402 shutdown_connections 2024-02-15T18:27:29.850 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.854+0000 7faff5fee700 1 --2- 172.21.15.121:0/265582402 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fafd802ff10 0x7fafd80323d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:29.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.854+0000 7faff5fee700 1 --2- 172.21.15.121:0/265582402 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faff00f6510 0x7faff01067e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:29.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.854+0000 7faff5fee700 1 -- 172.21.15.121:0/265582402 >> 172.21.15.121:0/265582402 conn(0x7faff00f1c70 msgr2=0x7faff00f2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:29.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.854+0000 7faff5fee700 1 -- 172.21.15.121:0/265582402 shutdown_connections 2024-02-15T18:27:29.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:29.854+0000 7faff5fee700 1 -- 172.21.15.121:0/265582402 wait complete. 2024-02-15T18:27:30.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:28.220326+0000 mgr.smithi038.khesbl (mgr.14162) 26 : cluster [DBG] pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:30.486 DEBUG:teuthology.orchestra.run.smithi121:mon.smithi121> sudo journalctl -f -n 0 -u ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mon.smithi121.service 2024-02-15T18:27:30.488 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:27:30.488 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:27:30.517 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:-- Logs begin at Thu 2024-02-15 18:14:30 UTC. -- 2024-02-15T18:27:30.989 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:29.849009+0000 mgr.smithi038.khesbl (mgr.14162) 27 : audit [DBG] from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi038:172.21.15.38=smithi038;smithi121:172.21.15.121=smithi121", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:27:30.989 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:27:29.850532+0000 mgr.smithi038.khesbl (mgr.14162) 28 : cephadm [INF] Saving service mon spec with placement smithi038:172.21.15.38=smithi038;smithi121:172.21.15.121=smithi121;count:2 2024-02-15T18:27:30.989 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:29.852947+0000 mon.smithi038 (mon.0) 129 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:27:30.989 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:30.221019+0000 mgr.smithi038.khesbl (mgr.14162) 29 : cluster [DBG] pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:32.275 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.279+0000 7f739a77c700 1 -- 172.21.15.121:0/858096535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7394060180 msgr2=0x7f73940f8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:32.275 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.279+0000 7f739a77c700 1 --2- 172.21.15.121:0/858096535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7394060180 0x7f73940f8f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f7390009fb0 tx=0x7f7390009b30).stop 2024-02-15T18:27:32.275 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.279+0000 7f739a77c700 1 -- 172.21.15.121:0/858096535 shutdown_connections 2024-02-15T18:27:32.275 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.279+0000 7f739a77c700 1 --2- 172.21.15.121:0/858096535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7394060180 0x7f73940f8f80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:32.275 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.279+0000 7f739a77c700 1 -- 172.21.15.121:0/858096535 >> 172.21.15.121:0/858096535 conn(0x7f73940f09a0 msgr2=0x7f73940f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:32.275 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.279+0000 7f739a77c700 1 -- 172.21.15.121:0/858096535 shutdown_connections 2024-02-15T18:27:32.275 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.279+0000 7f739a77c700 1 -- 172.21.15.121:0/858096535 wait complete. 2024-02-15T18:27:32.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.279+0000 7f739a77c700 1 Processor -- start 2024-02-15T18:27:32.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.279+0000 7f739a77c700 1 -- start start 2024-02-15T18:27:32.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.279+0000 7f739a77c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7394060180 0x7f73940f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:32.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.279+0000 7f739a77c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73940f4e80 con 0x7f7394060180 2024-02-15T18:27:32.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.280+0000 7f739977a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7394060180 0x7f73940f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:32.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.280+0000 7f739977a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7394060180 0x7f73940f6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.121:37314/0 (socket says 172.21.15.121:37314) 2024-02-15T18:27:32.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.280+0000 7f739977a700 1 -- 172.21.15.121:0/1151696864 learned_addr learned my addr 172.21.15.121:0/1151696864 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-02-15T18:27:32.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.280+0000 7f739977a700 1 -- 172.21.15.121:0/1151696864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7390005040 con 0x7f7394060180 2024-02-15T18:27:32.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.280+0000 7f739977a700 1 --2- 172.21.15.121:0/1151696864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7394060180 0x7f73940f6a00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f7390005010 tx=0x7f739000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:32.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.280+0000 7f7389ffb700 1 -- 172.21.15.121:0/1151696864 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7390009320 con 0x7f7394060180 2024-02-15T18:27:32.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.280+0000 7f7389ffb700 1 -- 172.21.15.121:0/1151696864 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7390013380 con 0x7f7394060180 2024-02-15T18:27:32.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.280+0000 7f739a77c700 1 -- 172.21.15.121:0/1151696864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73940f5020 con 0x7f7394060180 2024-02-15T18:27:32.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.280+0000 7f7389ffb700 1 -- 172.21.15.121:0/1151696864 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7390014e50 con 0x7f7394060180 2024-02-15T18:27:32.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.281+0000 7f739a77c700 1 -- 172.21.15.121:0/1151696864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73940f54b0 con 0x7f7394060180 2024-02-15T18:27:32.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.281+0000 7f739a77c700 1 -- 172.21.15.121:0/1151696864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7394059100 con 0x7f7394060180 2024-02-15T18:27:32.279 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.283+0000 7f7389ffb700 1 -- 172.21.15.121:0/1151696864 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f73900095f0 con 0x7f7394060180 2024-02-15T18:27:32.279 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.283+0000 7f7389ffb700 1 --2- 172.21.15.121:0/1151696864 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f738002ff10 0x7f73800323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:32.279 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.283+0000 7f7389ffb700 1 -- 172.21.15.121:0/1151696864 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f7390041070 con 0x7f7394060180 2024-02-15T18:27:32.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.283+0000 7f7398f79700 1 --2- 172.21.15.121:0/1151696864 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f738002ff10 0x7f73800323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:32.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.284+0000 7f7398f79700 1 --2- 172.21.15.121:0/1151696864 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f738002ff10 0x7f73800323d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f73840096a0 tx=0x7f7384006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:32.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.287+0000 7f7389ffb700 1 -- 172.21.15.121:0/1151696864 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f7390014420 con 0x7f7394060180 2024-02-15T18:27:32.485 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.489+0000 7f739a77c700 1 -- 172.21.15.121:0/1151696864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f73940fbe20 con 0x7f7394060180 2024-02-15T18:27:32.486 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.490+0000 7f7389ffb700 1 -- 172.21.15.121:0/1151696864 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f739000f7b0 con 0x7f7394060180 2024-02-15T18:27:32.487 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:27:32.487 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":1,"fsid":"7e05753e-cc2f-11ee-95ba-87774f69a715","modified":"2024-02-15T18:25:30.869198Z","created":"2024-02-15T18:25:30.869198Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-15T18:27:32.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.492+0000 7f739a77c700 1 -- 172.21.15.121:0/1151696864 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f738002ff10 msgr2=0x7f73800323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:32.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.492+0000 7f739a77c700 1 --2- 172.21.15.121:0/1151696864 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f738002ff10 0x7f73800323d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f73840096a0 tx=0x7f7384006b40).stop 2024-02-15T18:27:32.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.492+0000 7f739a77c700 1 -- 172.21.15.121:0/1151696864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7394060180 msgr2=0x7f73940f6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:32.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.492+0000 7f739a77c700 1 --2- 172.21.15.121:0/1151696864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7394060180 0x7f73940f6a00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f7390005010 tx=0x7f739000b040).stop 2024-02-15T18:27:32.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.492+0000 7f739a77c700 1 -- 172.21.15.121:0/1151696864 shutdown_connections 2024-02-15T18:27:32.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.492+0000 7f739a77c700 1 --2- 172.21.15.121:0/1151696864 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f738002ff10 0x7f73800323d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:32.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.492+0000 7f739a77c700 1 --2- 172.21.15.121:0/1151696864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7394060180 0x7f73940f6a00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:32.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.492+0000 7f739a77c700 1 -- 172.21.15.121:0/1151696864 >> 172.21.15.121:0/1151696864 conn(0x7f73940f09a0 msgr2=0x7f73940f1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:32.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.492+0000 7f739a77c700 1 -- 172.21.15.121:0/1151696864 shutdown_connections 2024-02-15T18:27:32.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:32.492+0000 7f739a77c700 1 -- 172.21.15.121:0/1151696864 wait complete. 2024-02-15T18:27:32.489 INFO:teuthology.orchestra.run.smithi121.stderr:dumped monmap epoch 1 2024-02-15T18:27:32.778 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:31.282961+0000 mon.smithi038 (mon.0) 130 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:27:32.779 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:27:31.284395+0000 mgr.smithi038.khesbl (mgr.14162) 30 : cephadm [INF] Deploying daemon node-exporter.smithi038 on smithi038 2024-02-15T18:27:33.778 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:32.221339+0000 mgr.smithi038.khesbl (mgr.14162) 31 : cluster [DBG] pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:33.778 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:32.491500+0000 mon.smithi038 (mon.0) 131 : audit [DBG] from='client.? 172.21.15.121:0/1151696864' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:27:34.132 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:27:34.132 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:27:35.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:34.221565+0000 mgr.smithi038.khesbl (mgr.14162) 32 : cluster [DBG] pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:35.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.869+0000 7f99eba0e700 1 -- 172.21.15.121:0/3498180186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e40f5410 msgr2=0x7f99e40f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:35.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.869+0000 7f99eba0e700 1 --2- 172.21.15.121:0/3498180186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e40f5410 0x7f99e40f5830 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f99d4009fb0 tx=0x7f99d4009b30).stop 2024-02-15T18:27:35.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.869+0000 7f99eba0e700 1 -- 172.21.15.121:0/3498180186 shutdown_connections 2024-02-15T18:27:35.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.869+0000 7f99eba0e700 1 --2- 172.21.15.121:0/3498180186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e40f5410 0x7f99e40f5830 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:35.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.869+0000 7f99eba0e700 1 -- 172.21.15.121:0/3498180186 >> 172.21.15.121:0/3498180186 conn(0x7f99e40f0b70 msgr2=0x7f99e40f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:35.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.869+0000 7f99eba0e700 1 -- 172.21.15.121:0/3498180186 shutdown_connections 2024-02-15T18:27:35.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.869+0000 7f99eba0e700 1 -- 172.21.15.121:0/3498180186 wait complete. 2024-02-15T18:27:35.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.870+0000 7f99eba0e700 1 Processor -- start 2024-02-15T18:27:35.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.870+0000 7f99eba0e700 1 -- start start 2024-02-15T18:27:35.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.870+0000 7f99eba0e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e40f5410 0x7f99e4109540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:35.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.870+0000 7f99eba0e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e4060220 con 0x7f99e40f5410 2024-02-15T18:27:35.868 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.871+0000 7f99e97aa700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e40f5410 0x7f99e4109540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:35.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.871+0000 7f99e97aa700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e40f5410 0x7f99e4109540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.121:37322/0 (socket says 172.21.15.121:37322) 2024-02-15T18:27:35.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.871+0000 7f99e97aa700 1 -- 172.21.15.121:0/2110456836 learned_addr learned my addr 172.21.15.121:0/2110456836 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-02-15T18:27:35.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.871+0000 7f99e97aa700 1 -- 172.21.15.121:0/2110456836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d4005040 con 0x7f99e40f5410 2024-02-15T18:27:35.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.871+0000 7f99e97aa700 1 --2- 172.21.15.121:0/2110456836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e40f5410 0x7f99e4109540 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f99d4005010 tx=0x7f99d400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:35.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.871+0000 7f99d9ffb700 1 -- 172.21.15.121:0/2110456836 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99d4009320 con 0x7f99e40f5410 2024-02-15T18:27:35.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.871+0000 7f99d9ffb700 1 -- 172.21.15.121:0/2110456836 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99d4013380 con 0x7f99e40f5410 2024-02-15T18:27:35.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.871+0000 7f99eba0e700 1 -- 172.21.15.121:0/2110456836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e4060420 con 0x7f99e40f5410 2024-02-15T18:27:35.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.871+0000 7f99d9ffb700 1 -- 172.21.15.121:0/2110456836 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99d4014e50 con 0x7f99e40f5410 2024-02-15T18:27:35.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.871+0000 7f99eba0e700 1 -- 172.21.15.121:0/2110456836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e4109bf0 con 0x7f99e40f5410 2024-02-15T18:27:35.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.872+0000 7f99d9ffb700 1 -- 172.21.15.121:0/2110456836 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f99d40095f0 con 0x7f99e40f5410 2024-02-15T18:27:35.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.872+0000 7f99eba0e700 1 -- 172.21.15.121:0/2110456836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e4059100 con 0x7f99e40f5410 2024-02-15T18:27:35.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.873+0000 7f99d9ffb700 1 --2- 172.21.15.121:0/2110456836 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f99d002ff10 0x7f99d00323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:35.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.873+0000 7f99d9ffb700 1 -- 172.21.15.121:0/2110456836 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f99d4040e70 con 0x7f99e40f5410 2024-02-15T18:27:35.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.873+0000 7f99e8fa9700 1 --2- 172.21.15.121:0/2110456836 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f99d002ff10 0x7f99d00323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:35.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.873+0000 7f99e8fa9700 1 --2- 172.21.15.121:0/2110456836 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f99d002ff10 0x7f99d00323d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f99e00096a0 tx=0x7f99e0006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:35.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:35.877+0000 7f99d9ffb700 1 -- 172.21.15.121:0/2110456836 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f99d4014420 con 0x7f99e40f5410 2024-02-15T18:27:36.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:36.084+0000 7f99eba0e700 1 -- 172.21.15.121:0/2110456836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f99e40fafa0 con 0x7f99e40f5410 2024-02-15T18:27:36.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:36.085+0000 7f99d9ffb700 1 -- 172.21.15.121:0/2110456836 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f99d4010230 con 0x7f99e40f5410 2024-02-15T18:27:36.081 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:27:36.081 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":1,"fsid":"7e05753e-cc2f-11ee-95ba-87774f69a715","modified":"2024-02-15T18:25:30.869198Z","created":"2024-02-15T18:25:30.869198Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-15T18:27:36.083 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:36.086+0000 7f99eba0e700 1 -- 172.21.15.121:0/2110456836 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f99d002ff10 msgr2=0x7f99d00323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:36.083 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:36.086+0000 7f99eba0e700 1 --2- 172.21.15.121:0/2110456836 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f99d002ff10 0x7f99d00323d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f99e00096a0 tx=0x7f99e0006b40).stop 2024-02-15T18:27:36.083 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:36.086+0000 7f99eba0e700 1 -- 172.21.15.121:0/2110456836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e40f5410 msgr2=0x7f99e4109540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:36.083 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:36.086+0000 7f99eba0e700 1 --2- 172.21.15.121:0/2110456836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e40f5410 0x7f99e4109540 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f99d4005010 tx=0x7f99d400b040).stop 2024-02-15T18:27:36.083 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:36.087+0000 7f99eba0e700 1 -- 172.21.15.121:0/2110456836 shutdown_connections 2024-02-15T18:27:36.083 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:36.087+0000 7f99eba0e700 1 --2- 172.21.15.121:0/2110456836 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f99d002ff10 0x7f99d00323d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:36.083 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:36.087+0000 7f99eba0e700 1 --2- 172.21.15.121:0/2110456836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e40f5410 0x7f99e4109540 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:36.083 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:36.087+0000 7f99eba0e700 1 -- 172.21.15.121:0/2110456836 >> 172.21.15.121:0/2110456836 conn(0x7f99e40f0b70 msgr2=0x7f99e40f1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:36.083 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:36.087+0000 7f99eba0e700 1 -- 172.21.15.121:0/2110456836 shutdown_connections 2024-02-15T18:27:36.083 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:36.087+0000 7f99eba0e700 1 -- 172.21.15.121:0/2110456836 wait complete. 2024-02-15T18:27:36.084 INFO:teuthology.orchestra.run.smithi121.stderr:dumped monmap epoch 1 2024-02-15T18:27:36.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:36.086156+0000 mon.smithi038 (mon.0) 132 : audit [DBG] from='client.? 172.21.15.121:0/2110456836' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:27:37.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:36.221923+0000 mgr.smithi038.khesbl (mgr.14162) 33 : cluster [DBG] pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:37.768 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:27:37.768 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:27:39.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:38.222317+0000 mgr.smithi038.khesbl (mgr.14162) 34 : cluster [DBG] pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:41.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:40.222587+0000 mgr.smithi038.khesbl (mgr.14162) 35 : cluster [DBG] pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:41.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:40.348748+0000 mon.smithi038 (mon.0) 133 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:27:41.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:40.351831+0000 mon.smithi038 (mon.0) 134 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:27:41.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:40.989282+0000 mon.smithi038 (mon.0) 135 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:27:42.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:27:40.994752+0000 mgr.smithi038.khesbl (mgr.14162) 36 : cephadm [INF] Deploying daemon prometheus.smithi038 on smithi038 2024-02-15T18:27:43.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:42.222922+0000 mgr.smithi038.khesbl (mgr.14162) 37 : cluster [DBG] pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:45.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:44.223135+0000 mgr.smithi038.khesbl (mgr.14162) 38 : cluster [DBG] pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:47.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:46.223397+0000 mgr.smithi038.khesbl (mgr.14162) 39 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:48.577 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.580+0000 7f8e6db79700 1 -- 172.21.15.121:0/2618846672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e680f6510 msgr2=0x7f8e680f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:48.577 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.580+0000 7f8e6db79700 1 --2- 172.21.15.121:0/2618846672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e680f6510 0x7f8e680f6930 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f8e58009fb0 tx=0x7f8e58009b30).stop 2024-02-15T18:27:48.577 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.580+0000 7f8e6db79700 1 -- 172.21.15.121:0/2618846672 shutdown_connections 2024-02-15T18:27:48.577 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.580+0000 7f8e6db79700 1 --2- 172.21.15.121:0/2618846672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e680f6510 0x7f8e680f6930 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:48.577 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.580+0000 7f8e6db79700 1 -- 172.21.15.121:0/2618846672 >> 172.21.15.121:0/2618846672 conn(0x7f8e680f1c70 msgr2=0x7f8e680f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:48.577 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.580+0000 7f8e6db79700 1 -- 172.21.15.121:0/2618846672 shutdown_connections 2024-02-15T18:27:48.577 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.580+0000 7f8e6db79700 1 -- 172.21.15.121:0/2618846672 wait complete. 2024-02-15T18:27:48.577 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.580+0000 7f8e6db79700 1 Processor -- start 2024-02-15T18:27:48.577 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.581+0000 7f8e6db79700 1 -- start start 2024-02-15T18:27:48.578 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.581+0000 7f8e6db79700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e680f6510 0x7f8e68108a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:48.578 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.581+0000 7f8e6db79700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e68108ef0 con 0x7f8e680f6510 2024-02-15T18:27:48.578 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.581+0000 7f8e677fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e680f6510 0x7f8e68108a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:48.578 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.581+0000 7f8e677fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e680f6510 0x7f8e68108a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.121:41678/0 (socket says 172.21.15.121:41678) 2024-02-15T18:27:48.578 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.581+0000 7f8e677fe700 1 -- 172.21.15.121:0/3956349014 learned_addr learned my addr 172.21.15.121:0/3956349014 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-02-15T18:27:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.581+0000 7f8e677fe700 1 -- 172.21.15.121:0/3956349014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e58005040 con 0x7f8e680f6510 2024-02-15T18:27:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.581+0000 7f8e677fe700 1 --2- 172.21.15.121:0/3956349014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e680f6510 0x7f8e68108a00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f8e58005010 tx=0x7f8e5800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.582+0000 7f8e4ffff700 1 -- 172.21.15.121:0/3956349014 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e58009320 con 0x7f8e680f6510 2024-02-15T18:27:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.582+0000 7f8e4ffff700 1 -- 172.21.15.121:0/3956349014 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e58013380 con 0x7f8e680f6510 2024-02-15T18:27:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.582+0000 7f8e6db79700 1 -- 172.21.15.121:0/3956349014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e681090f0 con 0x7f8e680f6510 2024-02-15T18:27:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.582+0000 7f8e4ffff700 1 -- 172.21.15.121:0/3956349014 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e58014e50 con 0x7f8e680f6510 2024-02-15T18:27:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.582+0000 7f8e6db79700 1 -- 172.21.15.121:0/3956349014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e68109580 con 0x7f8e680f6510 2024-02-15T18:27:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.582+0000 7f8e4ffff700 1 -- 172.21.15.121:0/3956349014 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f8e58014420 con 0x7f8e680f6510 2024-02-15T18:27:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.583+0000 7f8e6db79700 1 -- 172.21.15.121:0/3956349014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e68045ad0 con 0x7f8e680f6510 2024-02-15T18:27:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.583+0000 7f8e4ffff700 1 --2- 172.21.15.121:0/3956349014 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f8e5002ff10 0x7f8e500323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.583+0000 7f8e4ffff700 1 -- 172.21.15.121:0/3956349014 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f8e58040e30 con 0x7f8e680f6510 2024-02-15T18:27:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.583+0000 7f8e66ffd700 1 --2- 172.21.15.121:0/3956349014 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f8e5002ff10 0x7f8e500323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:48.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.583+0000 7f8e66ffd700 1 --2- 172.21.15.121:0/3956349014 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f8e5002ff10 0x7f8e500323d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f8e5c0096a0 tx=0x7f8e5c006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:48.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.588+0000 7f8e4ffff700 1 -- 172.21.15.121:0/3956349014 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f8e580095f0 con 0x7f8e680f6510 2024-02-15T18:27:48.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.798+0000 7f8e6db79700 1 -- 172.21.15.121:0/3956349014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8e6810a430 con 0x7f8e680f6510 2024-02-15T18:27:48.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.798+0000 7f8e4ffff700 1 -- 172.21.15.121:0/3956349014 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f8e5801db70 con 0x7f8e680f6510 2024-02-15T18:27:48.794 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:27:48.794 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":1,"fsid":"7e05753e-cc2f-11ee-95ba-87774f69a715","modified":"2024-02-15T18:25:30.869198Z","created":"2024-02-15T18:25:30.869198Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-15T18:27:48.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.800+0000 7f8e6db79700 1 -- 172.21.15.121:0/3956349014 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f8e5002ff10 msgr2=0x7f8e500323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:48.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.800+0000 7f8e6db79700 1 --2- 172.21.15.121:0/3956349014 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f8e5002ff10 0x7f8e500323d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f8e5c0096a0 tx=0x7f8e5c006b40).stop 2024-02-15T18:27:48.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.800+0000 7f8e6db79700 1 -- 172.21.15.121:0/3956349014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e680f6510 msgr2=0x7f8e68108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:48.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.800+0000 7f8e6db79700 1 --2- 172.21.15.121:0/3956349014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e680f6510 0x7f8e68108a00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f8e58005010 tx=0x7f8e5800b040).stop 2024-02-15T18:27:48.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.800+0000 7f8e6db79700 1 -- 172.21.15.121:0/3956349014 shutdown_connections 2024-02-15T18:27:48.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.800+0000 7f8e6db79700 1 --2- 172.21.15.121:0/3956349014 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f8e5002ff10 0x7f8e500323d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:48.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.800+0000 7f8e6db79700 1 --2- 172.21.15.121:0/3956349014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e680f6510 0x7f8e68108a00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:48.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.800+0000 7f8e6db79700 1 -- 172.21.15.121:0/3956349014 >> 172.21.15.121:0/3956349014 conn(0x7f8e680f1c70 msgr2=0x7f8e680f26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:48.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.800+0000 7f8e6db79700 1 -- 172.21.15.121:0/3956349014 shutdown_connections 2024-02-15T18:27:48.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:48.800+0000 7f8e6db79700 1 -- 172.21.15.121:0/3956349014 wait complete. 2024-02-15T18:27:48.797 INFO:teuthology.orchestra.run.smithi121.stderr:dumped monmap epoch 1 2024-02-15T18:27:49.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:48.223764+0000 mgr.smithi038.khesbl (mgr.14162) 40 : cluster [DBG] pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:49.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:48.799761+0000 mon.smithi038 (mon.0) 136 : audit [DBG] from='client.? 172.21.15.121:0/3956349014' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:27:50.429 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:27:50.429 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:27:51.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:50.224026+0000 mgr.smithi038.khesbl (mgr.14162) 41 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:52.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.316+0000 7fb0a3876700 1 -- 172.21.15.121:0/4273827415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb09c0f6510 msgr2=0x7fb09c0f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:52.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.316+0000 7fb0a3876700 1 --2- 172.21.15.121:0/4273827415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb09c0f6510 0x7fb09c0f6930 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fb08c009fb0 tx=0x7fb08c009b30).stop 2024-02-15T18:27:52.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.316+0000 7fb0a3876700 1 -- 172.21.15.121:0/4273827415 shutdown_connections 2024-02-15T18:27:52.314 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.316+0000 7fb0a3876700 1 --2- 172.21.15.121:0/4273827415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb09c0f6510 0x7fb09c0f6930 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:52.314 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.316+0000 7fb0a3876700 1 -- 172.21.15.121:0/4273827415 >> 172.21.15.121:0/4273827415 conn(0x7fb09c0f1c70 msgr2=0x7fb09c0f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:52.314 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.316+0000 7fb0a3876700 1 -- 172.21.15.121:0/4273827415 shutdown_connections 2024-02-15T18:27:52.314 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.316+0000 7fb0a3876700 1 -- 172.21.15.121:0/4273827415 wait complete. 2024-02-15T18:27:52.314 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.316+0000 7fb0a3876700 1 Processor -- start 2024-02-15T18:27:52.314 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.316+0000 7fb0a3876700 1 -- start start 2024-02-15T18:27:52.314 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.317+0000 7fb0a3876700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb09c0f6510 0x7fb09c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:52.314 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.317+0000 7fb0a3876700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09c069e20 con 0x7fb09c0f6510 2024-02-15T18:27:52.314 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.317+0000 7fb0a1612700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb09c0f6510 0x7fb09c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:52.314 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.317+0000 7fb0a1612700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb09c0f6510 0x7fb09c06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.121:54044/0 (socket says 172.21.15.121:54044) 2024-02-15T18:27:52.314 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.317+0000 7fb0a1612700 1 -- 172.21.15.121:0/1396347601 learned_addr learned my addr 172.21.15.121:0/1396347601 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-02-15T18:27:52.314 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.317+0000 7fb0a1612700 1 -- 172.21.15.121:0/1396347601 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c005040 con 0x7fb09c0f6510 2024-02-15T18:27:52.315 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.317+0000 7fb0a1612700 1 --2- 172.21.15.121:0/1396347601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb09c0f6510 0x7fb09c06b9a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fb08c005010 tx=0x7fb08c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:52.315 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.317+0000 7fb091ffb700 1 -- 172.21.15.121:0/1396347601 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb08c009320 con 0x7fb09c0f6510 2024-02-15T18:27:52.315 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.317+0000 7fb0a3876700 1 -- 172.21.15.121:0/1396347601 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09c069fc0 con 0x7fb09c0f6510 2024-02-15T18:27:52.315 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.317+0000 7fb091ffb700 1 -- 172.21.15.121:0/1396347601 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb08c013380 con 0x7fb09c0f6510 2024-02-15T18:27:52.315 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.317+0000 7fb091ffb700 1 -- 172.21.15.121:0/1396347601 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb08c014e50 con 0x7fb09c0f6510 2024-02-15T18:27:52.316 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.318+0000 7fb0a3876700 1 -- 172.21.15.121:0/1396347601 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09c06a450 con 0x7fb09c0f6510 2024-02-15T18:27:52.316 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.318+0000 7fb091ffb700 1 -- 172.21.15.121:0/1396347601 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fb08c0263a0 con 0x7fb09c0f6510 2024-02-15T18:27:52.316 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.318+0000 7fb091ffb700 1 --2- 172.21.15.121:0/1396347601 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb08802ff10 0x7fb0880323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:52.316 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.319+0000 7fb091ffb700 1 -- 172.21.15.121:0/1396347601 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fb08c040ed0 con 0x7fb09c0f6510 2024-02-15T18:27:52.316 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.319+0000 7fb0a3876700 1 -- 172.21.15.121:0/1396347601 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb09c059100 con 0x7fb09c0f6510 2024-02-15T18:27:52.316 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.319+0000 7fb0a0e11700 1 --2- 172.21.15.121:0/1396347601 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb08802ff10 0x7fb0880323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:52.316 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.319+0000 7fb0a0e11700 1 --2- 172.21.15.121:0/1396347601 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb08802ff10 0x7fb0880323d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fb0980096a0 tx=0x7fb098006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:52.319 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.324+0000 7fb091ffb700 1 -- 172.21.15.121:0/1396347601 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb08c012920 con 0x7fb09c0f6510 2024-02-15T18:27:52.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.527+0000 7fb0a3876700 1 -- 172.21.15.121:0/1396347601 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb09c045ad0 con 0x7fb09c0f6510 2024-02-15T18:27:52.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.527+0000 7fb091ffb700 1 -- 172.21.15.121:0/1396347601 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fb09c045ad0 con 0x7fb09c0f6510 2024-02-15T18:27:52.523 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:27:52.523 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":1,"fsid":"7e05753e-cc2f-11ee-95ba-87774f69a715","modified":"2024-02-15T18:25:30.869198Z","created":"2024-02-15T18:25:30.869198Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-15T18:27:52.525 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.529+0000 7fb0a3876700 1 -- 172.21.15.121:0/1396347601 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb08802ff10 msgr2=0x7fb0880323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:52.525 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.529+0000 7fb0a3876700 1 --2- 172.21.15.121:0/1396347601 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb08802ff10 0x7fb0880323d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fb0980096a0 tx=0x7fb098006b40).stop 2024-02-15T18:27:52.525 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.529+0000 7fb0a3876700 1 -- 172.21.15.121:0/1396347601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb09c0f6510 msgr2=0x7fb09c06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:52.525 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.529+0000 7fb0a3876700 1 --2- 172.21.15.121:0/1396347601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb09c0f6510 0x7fb09c06b9a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fb08c005010 tx=0x7fb08c00b040).stop 2024-02-15T18:27:52.525 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.529+0000 7fb0a3876700 1 -- 172.21.15.121:0/1396347601 shutdown_connections 2024-02-15T18:27:52.525 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.529+0000 7fb0a3876700 1 --2- 172.21.15.121:0/1396347601 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb08802ff10 0x7fb0880323d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:52.525 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.529+0000 7fb0a3876700 1 --2- 172.21.15.121:0/1396347601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb09c0f6510 0x7fb09c06b9a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:52.525 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.529+0000 7fb0a3876700 1 -- 172.21.15.121:0/1396347601 >> 172.21.15.121:0/1396347601 conn(0x7fb09c0f1c70 msgr2=0x7fb09c0f26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:52.526 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.529+0000 7fb0a3876700 1 -- 172.21.15.121:0/1396347601 shutdown_connections 2024-02-15T18:27:52.526 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:52.529+0000 7fb0a3876700 1 -- 172.21.15.121:0/1396347601 wait complete. 2024-02-15T18:27:52.526 INFO:teuthology.orchestra.run.smithi121.stderr:dumped monmap epoch 1 2024-02-15T18:27:53.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:52.224391+0000 mgr.smithi038.khesbl (mgr.14162) 42 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:53.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:52.528728+0000 mon.smithi038 (mon.0) 137 : audit [DBG] from='client.? 172.21.15.121:0/1396347601' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:27:54.192 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:27:54.192 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:27:55.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:54.224697+0000 mgr.smithi038.khesbl (mgr.14162) 43 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:56.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.018+0000 7f977955f700 1 -- 172.21.15.121:0/1394276192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97740f6510 msgr2=0x7f97740f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:56.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.018+0000 7f977955f700 1 --2- 172.21.15.121:0/1394276192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97740f6510 0x7f97740f6930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f975c009fb0 tx=0x7f975c009b30).stop 2024-02-15T18:27:56.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.018+0000 7f977955f700 1 -- 172.21.15.121:0/1394276192 shutdown_connections 2024-02-15T18:27:56.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.018+0000 7f977955f700 1 --2- 172.21.15.121:0/1394276192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97740f6510 0x7f97740f6930 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:56.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.018+0000 7f977955f700 1 -- 172.21.15.121:0/1394276192 >> 172.21.15.121:0/1394276192 conn(0x7f97740f1c70 msgr2=0x7f97740f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:56.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.018+0000 7f977955f700 1 -- 172.21.15.121:0/1394276192 shutdown_connections 2024-02-15T18:27:56.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.018+0000 7f977955f700 1 -- 172.21.15.121:0/1394276192 wait complete. 2024-02-15T18:27:56.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.019+0000 7f977955f700 1 Processor -- start 2024-02-15T18:27:56.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.019+0000 7f977955f700 1 -- start start 2024-02-15T18:27:56.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.019+0000 7f977955f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97740f6510 0x7f97741067e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:56.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.019+0000 7f977955f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9774106cd0 con 0x7f97740f6510 2024-02-15T18:27:56.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.019+0000 7f9772ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97740f6510 0x7f97741067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:56.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.019+0000 7f9772ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97740f6510 0x7f97741067e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.121:54056/0 (socket says 172.21.15.121:54056) 2024-02-15T18:27:56.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.019+0000 7f9772ffd700 1 -- 172.21.15.121:0/3188102933 learned_addr learned my addr 172.21.15.121:0/3188102933 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-02-15T18:27:56.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.019+0000 7f9772ffd700 1 -- 172.21.15.121:0/3188102933 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975c005040 con 0x7f97740f6510 2024-02-15T18:27:56.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.020+0000 7f9772ffd700 1 --2- 172.21.15.121:0/3188102933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97740f6510 0x7f97741067e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f975c005010 tx=0x7f975c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:56.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.020+0000 7f976b7fe700 1 -- 172.21.15.121:0/3188102933 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f975c009320 con 0x7f97740f6510 2024-02-15T18:27:56.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.020+0000 7f976b7fe700 1 -- 172.21.15.121:0/3188102933 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f975c013380 con 0x7f97740f6510 2024-02-15T18:27:56.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.020+0000 7f977955f700 1 -- 172.21.15.121:0/3188102933 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9774106ed0 con 0x7f97740f6510 2024-02-15T18:27:56.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.020+0000 7f976b7fe700 1 -- 172.21.15.121:0/3188102933 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f975c014e50 con 0x7f97740f6510 2024-02-15T18:27:56.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.020+0000 7f977955f700 1 -- 172.21.15.121:0/3188102933 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9774107360 con 0x7f97740f6510 2024-02-15T18:27:56.018 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.020+0000 7f976b7fe700 1 -- 172.21.15.121:0/3188102933 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f975c0095f0 con 0x7f97740f6510 2024-02-15T18:27:56.018 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.021+0000 7f977955f700 1 -- 172.21.15.121:0/3188102933 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9774045ad0 con 0x7f97740f6510 2024-02-15T18:27:56.018 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.021+0000 7f976b7fe700 1 --2- 172.21.15.121:0/3188102933 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f976002ff10 0x7f97600323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:56.018 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.021+0000 7f976b7fe700 1 -- 172.21.15.121:0/3188102933 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f975c040fa0 con 0x7f97740f6510 2024-02-15T18:27:56.018 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.021+0000 7f97727fc700 1 --2- 172.21.15.121:0/3188102933 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f976002ff10 0x7f97600323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:56.019 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.021+0000 7f97727fc700 1 --2- 172.21.15.121:0/3188102933 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f976002ff10 0x7f97600323d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f97640096a0 tx=0x7f9764006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:56.021 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.025+0000 7f976b7fe700 1 -- 172.21.15.121:0/3188102933 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f975c014420 con 0x7f97740f6510 2024-02-15T18:27:56.230 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.234+0000 7f977955f700 1 -- 172.21.15.121:0/3188102933 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9774108210 con 0x7f97740f6510 2024-02-15T18:27:56.230 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.235+0000 7f976b7fe700 1 -- 172.21.15.121:0/3188102933 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f975c010230 con 0x7f97740f6510 2024-02-15T18:27:56.231 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:27:56.231 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":1,"fsid":"7e05753e-cc2f-11ee-95ba-87774f69a715","modified":"2024-02-15T18:25:30.869198Z","created":"2024-02-15T18:25:30.869198Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-15T18:27:56.232 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.236+0000 7f977955f700 1 -- 172.21.15.121:0/3188102933 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f976002ff10 msgr2=0x7f97600323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:56.232 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.236+0000 7f977955f700 1 --2- 172.21.15.121:0/3188102933 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f976002ff10 0x7f97600323d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f97640096a0 tx=0x7f9764006b40).stop 2024-02-15T18:27:56.233 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.236+0000 7f977955f700 1 -- 172.21.15.121:0/3188102933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97740f6510 msgr2=0x7f97741067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:56.233 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.236+0000 7f977955f700 1 --2- 172.21.15.121:0/3188102933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97740f6510 0x7f97741067e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f975c005010 tx=0x7f975c00b040).stop 2024-02-15T18:27:56.233 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.236+0000 7f977955f700 1 -- 172.21.15.121:0/3188102933 shutdown_connections 2024-02-15T18:27:56.233 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.236+0000 7f977955f700 1 --2- 172.21.15.121:0/3188102933 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f976002ff10 0x7f97600323d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:56.233 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.236+0000 7f977955f700 1 --2- 172.21.15.121:0/3188102933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97740f6510 0x7f97741067e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:56.233 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.236+0000 7f977955f700 1 -- 172.21.15.121:0/3188102933 >> 172.21.15.121:0/3188102933 conn(0x7f97740f1c70 msgr2=0x7f97740f26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:56.233 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.236+0000 7f977955f700 1 -- 172.21.15.121:0/3188102933 shutdown_connections 2024-02-15T18:27:56.233 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:56.236+0000 7f977955f700 1 -- 172.21.15.121:0/3188102933 wait complete. 2024-02-15T18:27:56.233 INFO:teuthology.orchestra.run.smithi121.stderr:dumped monmap epoch 1 2024-02-15T18:27:56.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:56.236132+0000 mon.smithi038 (mon.0) 138 : audit [DBG] from='client.? 172.21.15.121:0/3188102933' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:27:57.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:56.224958+0000 mgr.smithi038.khesbl (mgr.14162) 44 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:57.881 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:27:57.882 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:27:59.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:27:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:27:58.225294+0000 mgr.smithi038.khesbl (mgr.14162) 45 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:27:59.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.729+0000 7f1bad330700 1 -- 172.21.15.121:0/4135610870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ba80f5410 msgr2=0x7f1ba80f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:59.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.729+0000 7f1bad330700 1 --2- 172.21.15.121:0/4135610870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ba80f5410 0x7f1ba80f5830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f1b90009fb0 tx=0x7f1b90009b30).stop 2024-02-15T18:27:59.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.729+0000 7f1bad330700 1 -- 172.21.15.121:0/4135610870 shutdown_connections 2024-02-15T18:27:59.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.729+0000 7f1bad330700 1 --2- 172.21.15.121:0/4135610870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ba80f5410 0x7f1ba80f5830 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:59.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.729+0000 7f1bad330700 1 -- 172.21.15.121:0/4135610870 >> 172.21.15.121:0/4135610870 conn(0x7f1ba80f0b70 msgr2=0x7f1ba80f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:59.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.729+0000 7f1bad330700 1 -- 172.21.15.121:0/4135610870 shutdown_connections 2024-02-15T18:27:59.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.729+0000 7f1bad330700 1 -- 172.21.15.121:0/4135610870 wait complete. 2024-02-15T18:27:59.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.730+0000 7f1bad330700 1 Processor -- start 2024-02-15T18:27:59.728 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.730+0000 7f1bad330700 1 -- start start 2024-02-15T18:27:59.728 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.730+0000 7f1bad330700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ba80f5410 0x7f1ba810b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:59.728 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.730+0000 7f1bad330700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba81072d0 con 0x7f1ba80f5410 2024-02-15T18:27:59.728 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.730+0000 7f1ba6ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ba80f5410 0x7f1ba810b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:59.728 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.731+0000 7f1ba6ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ba80f5410 0x7f1ba810b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.121:48232/0 (socket says 172.21.15.121:48232) 2024-02-15T18:27:59.728 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.731+0000 7f1ba6ffd700 1 -- 172.21.15.121:0/1310812044 learned_addr learned my addr 172.21.15.121:0/1310812044 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-02-15T18:27:59.728 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.731+0000 7f1ba6ffd700 1 -- 172.21.15.121:0/1310812044 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b90005040 con 0x7f1ba80f5410 2024-02-15T18:27:59.728 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.731+0000 7f1ba6ffd700 1 --2- 172.21.15.121:0/1310812044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ba80f5410 0x7f1ba810b8e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f1b90005010 tx=0x7f1b9000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:59.728 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.731+0000 7f1b9f7fe700 1 -- 172.21.15.121:0/1310812044 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1b90009320 con 0x7f1ba80f5410 2024-02-15T18:27:59.728 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.731+0000 7f1bad330700 1 -- 172.21.15.121:0/1310812044 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba8060280 con 0x7f1ba80f5410 2024-02-15T18:27:59.728 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.731+0000 7f1b9f7fe700 1 -- 172.21.15.121:0/1310812044 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b90013380 con 0x7f1ba80f5410 2024-02-15T18:27:59.728 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.731+0000 7f1b9f7fe700 1 -- 172.21.15.121:0/1310812044 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1b90014e50 con 0x7f1ba80f5410 2024-02-15T18:27:59.728 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.731+0000 7f1bad330700 1 -- 172.21.15.121:0/1310812044 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba810bdd0 con 0x7f1ba80f5410 2024-02-15T18:27:59.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.732+0000 7f1b9f7fe700 1 -- 172.21.15.121:0/1310812044 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f1b90014420 con 0x7f1ba80f5410 2024-02-15T18:27:59.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.732+0000 7f1bad330700 1 -- 172.21.15.121:0/1310812044 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba8045ad0 con 0x7f1ba80f5410 2024-02-15T18:27:59.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.732+0000 7f1b9f7fe700 1 --2- 172.21.15.121:0/1310812044 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1b9402ff10 0x7f1b940323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:59.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.732+0000 7f1b9f7fe700 1 -- 172.21.15.121:0/1310812044 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f1b90040e60 con 0x7f1ba80f5410 2024-02-15T18:27:59.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.732+0000 7f1ba67fc700 1 --2- 172.21.15.121:0/1310812044 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1b9402ff10 0x7f1b940323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:59.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.733+0000 7f1ba67fc700 1 --2- 172.21.15.121:0/1310812044 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1b9402ff10 0x7f1b940323d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f1b980096a0 tx=0x7f1b98006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:59.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.737+0000 7f1b9f7fe700 1 -- 172.21.15.121:0/1310812044 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f1b900095f0 con 0x7f1ba80f5410 2024-02-15T18:27:59.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.943+0000 7f1bad330700 1 -- 172.21.15.121:0/1310812044 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1ba810c5f0 con 0x7f1ba80f5410 2024-02-15T18:27:59.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.944+0000 7f1b9f7fe700 1 -- 172.21.15.121:0/1310812044 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f1b9001db70 con 0x7f1ba80f5410 2024-02-15T18:27:59.940 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:27:59.940 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":1,"fsid":"7e05753e-cc2f-11ee-95ba-87774f69a715","modified":"2024-02-15T18:25:30.869198Z","created":"2024-02-15T18:25:30.869198Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-15T18:27:59.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.945+0000 7f1bad330700 1 -- 172.21.15.121:0/1310812044 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1b9402ff10 msgr2=0x7f1b940323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:59.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.945+0000 7f1bad330700 1 --2- 172.21.15.121:0/1310812044 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1b9402ff10 0x7f1b940323d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f1b980096a0 tx=0x7f1b98006b40).stop 2024-02-15T18:27:59.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.945+0000 7f1bad330700 1 -- 172.21.15.121:0/1310812044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ba80f5410 msgr2=0x7f1ba810b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:59.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.945+0000 7f1bad330700 1 --2- 172.21.15.121:0/1310812044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ba80f5410 0x7f1ba810b8e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f1b90005010 tx=0x7f1b9000b040).stop 2024-02-15T18:27:59.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.945+0000 7f1bad330700 1 -- 172.21.15.121:0/1310812044 shutdown_connections 2024-02-15T18:27:59.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.945+0000 7f1bad330700 1 --2- 172.21.15.121:0/1310812044 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1b9402ff10 0x7f1b940323d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:59.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.946+0000 7f1bad330700 1 --2- 172.21.15.121:0/1310812044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ba80f5410 0x7f1ba810b8e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:59.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.946+0000 7f1bad330700 1 -- 172.21.15.121:0/1310812044 >> 172.21.15.121:0/1310812044 conn(0x7f1ba80f0b70 msgr2=0x7f1ba80f1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:59.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.946+0000 7f1bad330700 1 -- 172.21.15.121:0/1310812044 shutdown_connections 2024-02-15T18:27:59.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:27:59.946+0000 7f1bad330700 1 -- 172.21.15.121:0/1310812044 wait complete. 2024-02-15T18:27:59.942 INFO:teuthology.orchestra.run.smithi121.stderr:dumped monmap epoch 1 2024-02-15T18:28:00.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:27:59.945260+0000 mon.smithi038 (mon.0) 139 : audit [DBG] from='client.? 172.21.15.121:0/1310812044' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:28:01.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:00.225677+0000 mgr.smithi038.khesbl (mgr.14162) 46 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:01.664 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:28:01.665 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:28:03.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.468+0000 7fb4b2f38700 1 -- 172.21.15.121:0/924116744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4ac0f5410 msgr2=0x7fb4ac0f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:03.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.468+0000 7fb4b2f38700 1 --2- 172.21.15.121:0/924116744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4ac0f5410 0x7fb4ac0f5830 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fb4a0009fb0 tx=0x7fb4a0009b30).stop 2024-02-15T18:28:03.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.468+0000 7fb4b2f38700 1 -- 172.21.15.121:0/924116744 shutdown_connections 2024-02-15T18:28:03.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.468+0000 7fb4b2f38700 1 --2- 172.21.15.121:0/924116744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4ac0f5410 0x7fb4ac0f5830 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:03.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.468+0000 7fb4b2f38700 1 -- 172.21.15.121:0/924116744 >> 172.21.15.121:0/924116744 conn(0x7fb4ac0f0b70 msgr2=0x7fb4ac0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:03.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.468+0000 7fb4b2f38700 1 -- 172.21.15.121:0/924116744 shutdown_connections 2024-02-15T18:28:03.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.468+0000 7fb4b2f38700 1 -- 172.21.15.121:0/924116744 wait complete. 2024-02-15T18:28:03.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.469+0000 7fb4b2f38700 1 Processor -- start 2024-02-15T18:28:03.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.469+0000 7fb4b2f38700 1 -- start start 2024-02-15T18:28:03.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.469+0000 7fb4b2f38700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4ac0f5410 0x7fb4ac1067b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:03.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.469+0000 7fb4b2f38700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac106ca0 con 0x7fb4ac0f5410 2024-02-15T18:28:03.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.469+0000 7fb4b0cd4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4ac0f5410 0x7fb4ac1067b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:03.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.470+0000 7fb4b0cd4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4ac0f5410 0x7fb4ac1067b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.121:48252/0 (socket says 172.21.15.121:48252) 2024-02-15T18:28:03.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.470+0000 7fb4b0cd4700 1 -- 172.21.15.121:0/634575283 learned_addr learned my addr 172.21.15.121:0/634575283 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-02-15T18:28:03.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.470+0000 7fb4b0cd4700 1 -- 172.21.15.121:0/634575283 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a0005040 con 0x7fb4ac0f5410 2024-02-15T18:28:03.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.470+0000 7fb4b0cd4700 1 --2- 172.21.15.121:0/634575283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4ac0f5410 0x7fb4ac1067b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fb4a0005010 tx=0x7fb4a000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:03.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.470+0000 7fb4a97fa700 1 -- 172.21.15.121:0/634575283 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb4a0009320 con 0x7fb4ac0f5410 2024-02-15T18:28:03.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.470+0000 7fb4b2f38700 1 -- 172.21.15.121:0/634575283 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4ac106ea0 con 0x7fb4ac0f5410 2024-02-15T18:28:03.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.470+0000 7fb4a97fa700 1 -- 172.21.15.121:0/634575283 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb4a0013380 con 0x7fb4ac0f5410 2024-02-15T18:28:03.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.470+0000 7fb4a97fa700 1 -- 172.21.15.121:0/634575283 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb4a0014e50 con 0x7fb4ac0f5410 2024-02-15T18:28:03.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.470+0000 7fb4b2f38700 1 -- 172.21.15.121:0/634575283 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4ac107330 con 0x7fb4ac0f5410 2024-02-15T18:28:03.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.471+0000 7fb4a97fa700 1 -- 172.21.15.121:0/634575283 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fb4a0014420 con 0x7fb4ac0f5410 2024-02-15T18:28:03.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.471+0000 7fb4b2f38700 1 -- 172.21.15.121:0/634575283 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4ac045ad0 con 0x7fb4ac0f5410 2024-02-15T18:28:03.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.471+0000 7fb4a97fa700 1 --2- 172.21.15.121:0/634575283 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb49c02ff10 0x7fb49c0323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:03.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.471+0000 7fb4a97fa700 1 -- 172.21.15.121:0/634575283 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fb4a0040f40 con 0x7fb4ac0f5410 2024-02-15T18:28:03.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.471+0000 7fb4abfff700 1 --2- 172.21.15.121:0/634575283 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb49c02ff10 0x7fb49c0323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:03.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.472+0000 7fb4abfff700 1 --2- 172.21.15.121:0/634575283 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb49c02ff10 0x7fb49c0323d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fb4980096a0 tx=0x7fb498006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:03.472 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.476+0000 7fb4a97fa700 1 -- 172.21.15.121:0/634575283 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb4a00095f0 con 0x7fb4ac0f5410 2024-02-15T18:28:03.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:02.226034+0000 mgr.smithi038.khesbl (mgr.14162) 47 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:03.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.683+0000 7fb4b2f38700 1 -- 172.21.15.121:0/634575283 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb4ac1081e0 con 0x7fb4ac0f5410 2024-02-15T18:28:03.679 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.683+0000 7fb4a97fa700 1 -- 172.21.15.121:0/634575283 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fb4a001db70 con 0x7fb4ac0f5410 2024-02-15T18:28:03.679 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:28:03.679 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":1,"fsid":"7e05753e-cc2f-11ee-95ba-87774f69a715","modified":"2024-02-15T18:25:30.869198Z","created":"2024-02-15T18:25:30.869198Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-15T18:28:03.681 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.685+0000 7fb4b2f38700 1 -- 172.21.15.121:0/634575283 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb49c02ff10 msgr2=0x7fb49c0323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:03.681 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.685+0000 7fb4b2f38700 1 --2- 172.21.15.121:0/634575283 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb49c02ff10 0x7fb49c0323d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fb4980096a0 tx=0x7fb498006b40).stop 2024-02-15T18:28:03.681 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.685+0000 7fb4b2f38700 1 -- 172.21.15.121:0/634575283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4ac0f5410 msgr2=0x7fb4ac1067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:03.681 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.685+0000 7fb4b2f38700 1 --2- 172.21.15.121:0/634575283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4ac0f5410 0x7fb4ac1067b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fb4a0005010 tx=0x7fb4a000b040).stop 2024-02-15T18:28:03.681 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.685+0000 7fb4b2f38700 1 -- 172.21.15.121:0/634575283 shutdown_connections 2024-02-15T18:28:03.681 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.685+0000 7fb4b2f38700 1 --2- 172.21.15.121:0/634575283 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb49c02ff10 0x7fb49c0323d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:03.681 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.685+0000 7fb4b2f38700 1 --2- 172.21.15.121:0/634575283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4ac0f5410 0x7fb4ac1067b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:03.681 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.685+0000 7fb4b2f38700 1 -- 172.21.15.121:0/634575283 >> 172.21.15.121:0/634575283 conn(0x7fb4ac0f0b70 msgr2=0x7fb4ac05fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:03.681 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.685+0000 7fb4b2f38700 1 -- 172.21.15.121:0/634575283 shutdown_connections 2024-02-15T18:28:03.681 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:03.685+0000 7fb4b2f38700 1 -- 172.21.15.121:0/634575283 wait complete. 2024-02-15T18:28:03.682 INFO:teuthology.orchestra.run.smithi121.stderr:dumped monmap epoch 1 2024-02-15T18:28:04.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:04 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:03.684507+0000 mon.smithi038 (mon.0) 140 : audit [DBG] from='client.? 172.21.15.121:0/634575283' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:28:05.301 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:28:05.302 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:28:05.528 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:04.226409+0000 mgr.smithi038.khesbl (mgr.14162) 48 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:07.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.191+0000 7fa96adb1700 1 -- 172.21.15.121:0/3060588392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa964060280 msgr2=0x7fa9640f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:07.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.191+0000 7fa96adb1700 1 --2- 172.21.15.121:0/3060588392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa964060280 0x7fa9640f90f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fa958009fb0 tx=0x7fa958009b30).stop 2024-02-15T18:28:07.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.191+0000 7fa96adb1700 1 -- 172.21.15.121:0/3060588392 shutdown_connections 2024-02-15T18:28:07.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.191+0000 7fa96adb1700 1 --2- 172.21.15.121:0/3060588392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa964060280 0x7fa9640f90f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:07.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.191+0000 7fa96adb1700 1 -- 172.21.15.121:0/3060588392 >> 172.21.15.121:0/3060588392 conn(0x7fa9640f0b70 msgr2=0x7fa9640f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:07.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.191+0000 7fa96adb1700 1 -- 172.21.15.121:0/3060588392 shutdown_connections 2024-02-15T18:28:07.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.191+0000 7fa96adb1700 1 -- 172.21.15.121:0/3060588392 wait complete. 2024-02-15T18:28:07.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.192+0000 7fa96adb1700 1 Processor -- start 2024-02-15T18:28:07.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.192+0000 7fa96adb1700 1 -- start start 2024-02-15T18:28:07.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.192+0000 7fa96adb1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa964060280 0x7fa964106780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:07.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.192+0000 7fa96adb1700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa964106c70 con 0x7fa964060280 2024-02-15T18:28:07.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.192+0000 7fa968b4d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa964060280 0x7fa964106780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:07.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.192+0000 7fa968b4d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa964060280 0x7fa964106780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.121:48284/0 (socket says 172.21.15.121:48284) 2024-02-15T18:28:07.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.192+0000 7fa968b4d700 1 -- 172.21.15.121:0/1667999287 learned_addr learned my addr 172.21.15.121:0/1667999287 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-02-15T18:28:07.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.192+0000 7fa968b4d700 1 -- 172.21.15.121:0/1667999287 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa958005040 con 0x7fa964060280 2024-02-15T18:28:07.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.193+0000 7fa968b4d700 1 --2- 172.21.15.121:0/1667999287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa964060280 0x7fa964106780 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fa9580089f0 tx=0x7fa95800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:07.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.193+0000 7fa9617fa700 1 -- 172.21.15.121:0/1667999287 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa958009320 con 0x7fa964060280 2024-02-15T18:28:07.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.193+0000 7fa96adb1700 1 -- 172.21.15.121:0/1667999287 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa964106e70 con 0x7fa964060280 2024-02-15T18:28:07.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.193+0000 7fa9617fa700 1 -- 172.21.15.121:0/1667999287 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa958013380 con 0x7fa964060280 2024-02-15T18:28:07.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.193+0000 7fa9617fa700 1 -- 172.21.15.121:0/1667999287 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa958014e20 con 0x7fa964060280 2024-02-15T18:28:07.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.193+0000 7fa96adb1700 1 -- 172.21.15.121:0/1667999287 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa964107300 con 0x7fa964060280 2024-02-15T18:28:07.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.193+0000 7fa9617fa700 1 -- 172.21.15.121:0/1667999287 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fa958014420 con 0x7fa964060280 2024-02-15T18:28:07.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.194+0000 7fa96adb1700 1 -- 172.21.15.121:0/1667999287 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa964045ad0 con 0x7fa964060280 2024-02-15T18:28:07.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.194+0000 7fa9617fa700 1 --2- 172.21.15.121:0/1667999287 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fa95402ff10 0x7fa9540323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:07.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.194+0000 7fa9617fa700 1 -- 172.21.15.121:0/1667999287 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fa958041010 con 0x7fa964060280 2024-02-15T18:28:07.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.194+0000 7fa963fff700 1 --2- 172.21.15.121:0/1667999287 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fa95402ff10 0x7fa9540323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:07.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.194+0000 7fa963fff700 1 --2- 172.21.15.121:0/1667999287 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fa95402ff10 0x7fa9540323d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fa9500096a0 tx=0x7fa950006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:07.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.199+0000 7fa9617fa700 1 -- 172.21.15.121:0/1667999287 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fa95806d2b0 con 0x7fa964060280 2024-02-15T18:28:07.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.417+0000 7fa96adb1700 1 -- 172.21.15.121:0/1667999287 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa9641081b0 con 0x7fa964060280 2024-02-15T18:28:07.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.418+0000 7fa9617fa700 1 -- 172.21.15.121:0/1667999287 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fa9580095f0 con 0x7fa964060280 2024-02-15T18:28:07.414 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:28:07.414 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":1,"fsid":"7e05753e-cc2f-11ee-95ba-87774f69a715","modified":"2024-02-15T18:25:30.869198Z","created":"2024-02-15T18:25:30.869198Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-15T18:28:07.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.420+0000 7fa96adb1700 1 -- 172.21.15.121:0/1667999287 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fa95402ff10 msgr2=0x7fa9540323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:07.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.420+0000 7fa96adb1700 1 --2- 172.21.15.121:0/1667999287 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fa95402ff10 0x7fa9540323d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fa9500096a0 tx=0x7fa950006b40).stop 2024-02-15T18:28:07.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.420+0000 7fa96adb1700 1 -- 172.21.15.121:0/1667999287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa964060280 msgr2=0x7fa964106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:07.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.420+0000 7fa96adb1700 1 --2- 172.21.15.121:0/1667999287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa964060280 0x7fa964106780 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fa9580089f0 tx=0x7fa95800b040).stop 2024-02-15T18:28:07.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.420+0000 7fa96adb1700 1 -- 172.21.15.121:0/1667999287 shutdown_connections 2024-02-15T18:28:07.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.420+0000 7fa96adb1700 1 --2- 172.21.15.121:0/1667999287 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fa95402ff10 0x7fa9540323d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:07.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.420+0000 7fa96adb1700 1 --2- 172.21.15.121:0/1667999287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa964060280 0x7fa964106780 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:07.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.420+0000 7fa96adb1700 1 -- 172.21.15.121:0/1667999287 >> 172.21.15.121:0/1667999287 conn(0x7fa9640f0b70 msgr2=0x7fa9640f15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:07.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.420+0000 7fa96adb1700 1 -- 172.21.15.121:0/1667999287 shutdown_connections 2024-02-15T18:28:07.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:07.420+0000 7fa96adb1700 1 -- 172.21.15.121:0/1667999287 wait complete. 2024-02-15T18:28:07.417 INFO:teuthology.orchestra.run.smithi121.stderr:dumped monmap epoch 1 2024-02-15T18:28:07.576 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:06.226832+0000 mgr.smithi038.khesbl (mgr.14162) 49 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:07.576 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:06.947410+0000 mon.smithi038 (mon.0) 141 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:07.576 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:06.947930+0000 mon.smithi038 (mon.0) 142 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:28:07.576 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:07.276025+0000 mon.smithi038 (mon.0) 143 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:08.716 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:07.419632+0000 mon.smithi038 (mon.0) 144 : audit [DBG] from='client.? 172.21.15.121:0/1667999287' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:28:08.716 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:07.678959+0000 mon.smithi038 (mon.0) 145 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:09.031 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:28:09.031 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:28:10.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:08.227062+0000 mgr.smithi038.khesbl (mgr.14162) 50 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:12.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:10.227371+0000 mgr.smithi038.khesbl (mgr.14162) 51 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:12.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.631+0000 7f0b64a24700 1 -- 172.21.15.121:0/2719650187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b600f51a0 msgr2=0x7f0b600f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:12.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.631+0000 7f0b64a24700 1 --2- 172.21.15.121:0/2719650187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b600f51a0 0x7f0b600f55c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f0b50009fb0 tx=0x7f0b50009b30).stop 2024-02-15T18:28:12.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.631+0000 7f0b64a24700 1 -- 172.21.15.121:0/2719650187 shutdown_connections 2024-02-15T18:28:12.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.631+0000 7f0b64a24700 1 --2- 172.21.15.121:0/2719650187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b600f51a0 0x7f0b600f55c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:12.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.631+0000 7f0b64a24700 1 -- 172.21.15.121:0/2719650187 >> 172.21.15.121:0/2719650187 conn(0x7f0b600f09a0 msgr2=0x7f0b600f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:12.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.631+0000 7f0b64a24700 1 -- 172.21.15.121:0/2719650187 shutdown_connections 2024-02-15T18:28:12.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.631+0000 7f0b64a24700 1 -- 172.21.15.121:0/2719650187 wait complete. 2024-02-15T18:28:12.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.631+0000 7f0b64a24700 1 Processor -- start 2024-02-15T18:28:12.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.631+0000 7f0b64a24700 1 -- start start 2024-02-15T18:28:12.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.631+0000 7f0b64a24700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b600f51a0 0x7f0b60109290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:12.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.631+0000 7f0b64a24700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b60060280 con 0x7f0b600f51a0 2024-02-15T18:28:12.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.632+0000 7f0b5ed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b600f51a0 0x7f0b60109290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:12.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.632+0000 7f0b5ed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b600f51a0 0x7f0b60109290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.121:34732/0 (socket says 172.21.15.121:34732) 2024-02-15T18:28:12.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.632+0000 7f0b5ed9d700 1 -- 172.21.15.121:0/1855363293 learned_addr learned my addr 172.21.15.121:0/1855363293 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-02-15T18:28:12.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.632+0000 7f0b5ed9d700 1 -- 172.21.15.121:0/1855363293 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b50005040 con 0x7f0b600f51a0 2024-02-15T18:28:12.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.632+0000 7f0b5ed9d700 1 --2- 172.21.15.121:0/1855363293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b600f51a0 0x7f0b60109290 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f0b500089f0 tx=0x7f0b5000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:12.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.632+0000 7f0b577fe700 1 -- 172.21.15.121:0/1855363293 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0b50009320 con 0x7f0b600f51a0 2024-02-15T18:28:12.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.632+0000 7f0b64a24700 1 -- 172.21.15.121:0/1855363293 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b60060480 con 0x7f0b600f51a0 2024-02-15T18:28:12.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.632+0000 7f0b577fe700 1 -- 172.21.15.121:0/1855363293 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b50013380 con 0x7f0b600f51a0 2024-02-15T18:28:12.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.632+0000 7f0b577fe700 1 -- 172.21.15.121:0/1855363293 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0b50014e20 con 0x7f0b600f51a0 2024-02-15T18:28:12.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.632+0000 7f0b64a24700 1 -- 172.21.15.121:0/1855363293 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b601099c0 con 0x7f0b600f51a0 2024-02-15T18:28:12.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.633+0000 7f0b577fe700 1 -- 172.21.15.121:0/1855363293 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f0b50014420 con 0x7f0b600f51a0 2024-02-15T18:28:12.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.633+0000 7f0b64a24700 1 -- 172.21.15.121:0/1855363293 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b60045ad0 con 0x7f0b600f51a0 2024-02-15T18:28:12.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.633+0000 7f0b577fe700 1 --2- 172.21.15.121:0/1855363293 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f0b4c02ff10 0x7f0b4c0323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:12.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.633+0000 7f0b577fe700 1 -- 172.21.15.121:0/1855363293 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f0b50040ed0 con 0x7f0b600f51a0 2024-02-15T18:28:12.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.634+0000 7f0b5e59c700 1 --2- 172.21.15.121:0/1855363293 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f0b4c02ff10 0x7f0b4c0323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:12.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.634+0000 7f0b5e59c700 1 --2- 172.21.15.121:0/1855363293 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f0b4c02ff10 0x7f0b4c0323d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f0b480096a0 tx=0x7f0b48006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:12.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.638+0000 7f0b577fe700 1 -- 172.21.15.121:0/1855363293 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f0b500268c0 con 0x7f0b600f51a0 2024-02-15T18:28:12.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.844+0000 7f0b64a24700 1 -- 172.21.15.121:0/1855363293 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0b600fad30 con 0x7f0b600f51a0 2024-02-15T18:28:12.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.845+0000 7f0b577fe700 1 -- 172.21.15.121:0/1855363293 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0b5001d7f0 con 0x7f0b600f51a0 2024-02-15T18:28:12.049 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:28:12.049 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":1,"fsid":"7e05753e-cc2f-11ee-95ba-87774f69a715","modified":"2024-02-15T18:25:30.869198Z","created":"2024-02-15T18:25:30.869198Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-15T18:28:12.050 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.846+0000 7f0b64a24700 1 -- 172.21.15.121:0/1855363293 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f0b4c02ff10 msgr2=0x7f0b4c0323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:12.050 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.846+0000 7f0b64a24700 1 --2- 172.21.15.121:0/1855363293 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f0b4c02ff10 0x7f0b4c0323d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f0b480096a0 tx=0x7f0b48006b40).stop 2024-02-15T18:28:12.050 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.846+0000 7f0b64a24700 1 -- 172.21.15.121:0/1855363293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b600f51a0 msgr2=0x7f0b60109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:12.050 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.846+0000 7f0b64a24700 1 --2- 172.21.15.121:0/1855363293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b600f51a0 0x7f0b60109290 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f0b500089f0 tx=0x7f0b5000b040).stop 2024-02-15T18:28:12.050 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.846+0000 7f0b64a24700 1 -- 172.21.15.121:0/1855363293 shutdown_connections 2024-02-15T18:28:12.050 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.846+0000 7f0b64a24700 1 --2- 172.21.15.121:0/1855363293 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f0b4c02ff10 0x7f0b4c0323d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:12.050 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.846+0000 7f0b64a24700 1 --2- 172.21.15.121:0/1855363293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b600f51a0 0x7f0b60109290 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:12.050 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.846+0000 7f0b64a24700 1 -- 172.21.15.121:0/1855363293 >> 172.21.15.121:0/1855363293 conn(0x7f0b600f09a0 msgr2=0x7f0b600f13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:12.050 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.846+0000 7f0b64a24700 1 -- 172.21.15.121:0/1855363293 shutdown_connections 2024-02-15T18:28:12.050 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:11.846+0000 7f0b64a24700 1 -- 172.21.15.121:0/1855363293 wait complete. 2024-02-15T18:28:12.050 INFO:teuthology.orchestra.run.smithi121.stderr:dumped monmap epoch 1 2024-02-15T18:28:13.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:11.753766+0000 mon.smithi038 (mon.0) 146 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:13.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:11.846039+0000 mon.smithi038 (mon.0) 147 : audit [DBG] from='client.? 172.21.15.121:0/1855363293' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:28:13.668 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:28:13.669 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:28:14.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:12.227631+0000 mgr.smithi038.khesbl (mgr.14162) 52 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:14.074 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:12.763284+0000 mon.smithi038 (mon.0) 148 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:14.074 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:13.719147+0000 mon.smithi038 (mon.0) 149 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:14.074 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:13.722176+0000 mon.smithi038 (mon.0) 150 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:14.074 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:13.723567+0000 mon.smithi038 (mon.0) 151 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi121", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-15T18:28:14.074 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:13.725569+0000 mon.smithi038 (mon.0) 152 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi121", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-15T18:28:14.074 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:13.726436+0000 mon.smithi038 (mon.0) 153 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:28:15.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:28:13.727323+0000 mgr.smithi038.khesbl (mgr.14162) 53 : cephadm [INF] Deploying daemon crash.smithi121 on smithi121 2024-02-15T18:28:16.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:14.227853+0000 mgr.smithi038.khesbl (mgr.14162) 54 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:16.925 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.928+0000 7f3ef7124700 1 -- 172.21.15.121:0/3489205959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ef00f5410 msgr2=0x7f3ef00f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:16.925 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.928+0000 7f3ef7124700 1 --2- 172.21.15.121:0/3489205959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ef00f5410 0x7f3ef00f5830 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f3ee4009fb0 tx=0x7f3ee4009b30).stop 2024-02-15T18:28:16.925 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.928+0000 7f3ef7124700 1 -- 172.21.15.121:0/3489205959 shutdown_connections 2024-02-15T18:28:16.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.928+0000 7f3ef7124700 1 --2- 172.21.15.121:0/3489205959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ef00f5410 0x7f3ef00f5830 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:16.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.928+0000 7f3ef7124700 1 -- 172.21.15.121:0/3489205959 >> 172.21.15.121:0/3489205959 conn(0x7f3ef00f0b70 msgr2=0x7f3ef00f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:16.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.928+0000 7f3ef7124700 1 -- 172.21.15.121:0/3489205959 shutdown_connections 2024-02-15T18:28:16.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.928+0000 7f3ef7124700 1 -- 172.21.15.121:0/3489205959 wait complete. 2024-02-15T18:28:16.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.928+0000 7f3ef7124700 1 Processor -- start 2024-02-15T18:28:16.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.928+0000 7f3ef7124700 1 -- start start 2024-02-15T18:28:16.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.929+0000 7f3ef7124700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ef00f5410 0x7f3ef01094e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:16.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.929+0000 7f3ef7124700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef0060240 con 0x7f3ef00f5410 2024-02-15T18:28:16.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.929+0000 7f3ef4ec0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ef00f5410 0x7f3ef01094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:16.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.929+0000 7f3ef4ec0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ef00f5410 0x7f3ef01094e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.121:34752/0 (socket says 172.21.15.121:34752) 2024-02-15T18:28:16.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.929+0000 7f3ef4ec0700 1 -- 172.21.15.121:0/593501535 learned_addr learned my addr 172.21.15.121:0/593501535 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-02-15T18:28:16.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.929+0000 7f3ef4ec0700 1 -- 172.21.15.121:0/593501535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee4005040 con 0x7f3ef00f5410 2024-02-15T18:28:16.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.929+0000 7f3ef4ec0700 1 --2- 172.21.15.121:0/593501535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ef00f5410 0x7f3ef01094e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f3ee4005010 tx=0x7f3ee400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:16.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.929+0000 7f3eed7fa700 1 -- 172.21.15.121:0/593501535 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ee4009320 con 0x7f3ef00f5410 2024-02-15T18:28:16.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.929+0000 7f3ef7124700 1 -- 172.21.15.121:0/593501535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef0060440 con 0x7f3ef00f5410 2024-02-15T18:28:16.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.929+0000 7f3eed7fa700 1 -- 172.21.15.121:0/593501535 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ee4013380 con 0x7f3ef00f5410 2024-02-15T18:28:16.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.929+0000 7f3eed7fa700 1 -- 172.21.15.121:0/593501535 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ee4014e50 con 0x7f3ef00f5410 2024-02-15T18:28:16.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.929+0000 7f3ef7124700 1 -- 172.21.15.121:0/593501535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef0109b90 con 0x7f3ef00f5410 2024-02-15T18:28:16.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.930+0000 7f3eed7fa700 1 -- 172.21.15.121:0/593501535 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f3ee40263a0 con 0x7f3ef00f5410 2024-02-15T18:28:16.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.930+0000 7f3ef7124700 1 -- 172.21.15.121:0/593501535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef0045ad0 con 0x7f3ef00f5410 2024-02-15T18:28:16.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.930+0000 7f3eed7fa700 1 --2- 172.21.15.121:0/593501535 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f3ee002ff10 0x7f3ee00323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:16.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.930+0000 7f3eed7fa700 1 -- 172.21.15.121:0/593501535 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f3ee4040e80 con 0x7f3ef00f5410 2024-02-15T18:28:16.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.931+0000 7f3eeffff700 1 --2- 172.21.15.121:0/593501535 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f3ee002ff10 0x7f3ee00323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:16.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.931+0000 7f3eeffff700 1 --2- 172.21.15.121:0/593501535 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f3ee002ff10 0x7f3ee00323d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f3edc0096a0 tx=0x7f3edc006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:16.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:16.935+0000 7f3eed7fa700 1 -- 172.21.15.121:0/593501535 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f3ee40126f0 con 0x7f3ef00f5410 2024-02-15T18:28:17.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:17.163+0000 7f3ef7124700 1 -- 172.21.15.121:0/593501535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3ef010a3b0 con 0x7f3ef00f5410 2024-02-15T18:28:17.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:17.163+0000 7f3eed7fa700 1 -- 172.21.15.121:0/593501535 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f3ee4012330 con 0x7f3ef00f5410 2024-02-15T18:28:17.159 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:28:17.159 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":1,"fsid":"7e05753e-cc2f-11ee-95ba-87774f69a715","modified":"2024-02-15T18:25:30.869198Z","created":"2024-02-15T18:25:30.869198Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-15T18:28:17.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:17.165+0000 7f3ef7124700 1 -- 172.21.15.121:0/593501535 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f3ee002ff10 msgr2=0x7f3ee00323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:17.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:17.165+0000 7f3ef7124700 1 --2- 172.21.15.121:0/593501535 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f3ee002ff10 0x7f3ee00323d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f3edc0096a0 tx=0x7f3edc006b40).stop 2024-02-15T18:28:17.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:17.165+0000 7f3ef7124700 1 -- 172.21.15.121:0/593501535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ef00f5410 msgr2=0x7f3ef01094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:17.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:17.165+0000 7f3ef7124700 1 --2- 172.21.15.121:0/593501535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ef00f5410 0x7f3ef01094e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f3ee4005010 tx=0x7f3ee400b040).stop 2024-02-15T18:28:17.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:17.165+0000 7f3ef7124700 1 -- 172.21.15.121:0/593501535 shutdown_connections 2024-02-15T18:28:17.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:17.165+0000 7f3ef7124700 1 --2- 172.21.15.121:0/593501535 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f3ee002ff10 0x7f3ee00323d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:17.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:17.165+0000 7f3ef7124700 1 --2- 172.21.15.121:0/593501535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ef00f5410 0x7f3ef01094e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:17.162 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:17.165+0000 7f3ef7124700 1 -- 172.21.15.121:0/593501535 >> 172.21.15.121:0/593501535 conn(0x7f3ef00f0b70 msgr2=0x7f3ef00f1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:17.162 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:17.165+0000 7f3ef7124700 1 -- 172.21.15.121:0/593501535 shutdown_connections 2024-02-15T18:28:17.162 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:17.165+0000 7f3ef7124700 1 -- 172.21.15.121:0/593501535 wait complete. 2024-02-15T18:28:17.162 INFO:teuthology.orchestra.run.smithi121.stderr:dumped monmap epoch 1 2024-02-15T18:28:18.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:16.228178+0000 mgr.smithi038.khesbl (mgr.14162) 55 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:18.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:17.164946+0000 mon.smithi038 (mon.0) 154 : audit [DBG] from='client.? 172.21.15.121:0/593501535' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:28:18.793 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:28:18.794 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:28:19.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:18.228518+0000 mgr.smithi038.khesbl (mgr.14162) 56 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:19.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:18.304885+0000 mon.smithi038 (mon.0) 155 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:19.778 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:18.306317+0000 mon.smithi038 (mon.0) 156 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi121.ckiryi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:28:19.778 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:18.308650+0000 mon.smithi038 (mon.0) 157 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi121.ckiryi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-15T18:28:19.778 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:18.309430+0000 mon.smithi038 (mon.0) 158 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:28:19.778 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:18.310401+0000 mon.smithi038 (mon.0) 159 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:28:19.778 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:28:18.311208+0000 mgr.smithi038.khesbl (mgr.14162) 57 : cephadm [INF] Deploying daemon mgr.smithi121.ckiryi on smithi121 2024-02-15T18:28:21.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:20.228880+0000 mgr.smithi038.khesbl (mgr.14162) 58 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:23.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.340+0000 7f6c7d213700 1 -- 172.21.15.121:0/1329632674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c78060280 msgr2=0x7f6c780f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:23.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.340+0000 7f6c7d213700 1 --2- 172.21.15.121:0/1329632674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c78060280 0x7f6c780f90f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f6c68009fb0 tx=0x7f6c68009b30).stop 2024-02-15T18:28:23.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.340+0000 7f6c7d213700 1 -- 172.21.15.121:0/1329632674 shutdown_connections 2024-02-15T18:28:23.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.340+0000 7f6c7d213700 1 --2- 172.21.15.121:0/1329632674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c78060280 0x7f6c780f90f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:23.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.340+0000 7f6c7d213700 1 -- 172.21.15.121:0/1329632674 >> 172.21.15.121:0/1329632674 conn(0x7f6c780f0b70 msgr2=0x7f6c780f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:23.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.340+0000 7f6c7d213700 1 -- 172.21.15.121:0/1329632674 shutdown_connections 2024-02-15T18:28:23.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.340+0000 7f6c7d213700 1 -- 172.21.15.121:0/1329632674 wait complete. 2024-02-15T18:28:23.341 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.341+0000 7f6c7d213700 1 Processor -- start 2024-02-15T18:28:23.341 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.341+0000 7f6c7d213700 1 -- start start 2024-02-15T18:28:23.341 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.341+0000 7f6c7d213700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c78060280 0x7f6c781067a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:23.341 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.341+0000 7f6c7d213700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c78106c90 con 0x7f6c78060280 2024-02-15T18:28:23.341 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.342+0000 7f6c76d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c78060280 0x7f6c781067a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:23.341 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.342+0000 7f6c76d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c78060280 0x7f6c781067a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.121:43320/0 (socket says 172.21.15.121:43320) 2024-02-15T18:28:23.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.342+0000 7f6c76d9d700 1 -- 172.21.15.121:0/3125330498 learned_addr learned my addr 172.21.15.121:0/3125330498 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-02-15T18:28:23.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.342+0000 7f6c76d9d700 1 -- 172.21.15.121:0/3125330498 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c68005040 con 0x7f6c78060280 2024-02-15T18:28:23.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.342+0000 7f6c76d9d700 1 --2- 172.21.15.121:0/3125330498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c78060280 0x7f6c781067a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f6c680089f0 tx=0x7f6c6800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:23.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.342+0000 7f6c5f7fe700 1 -- 172.21.15.121:0/3125330498 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c68009320 con 0x7f6c78060280 2024-02-15T18:28:23.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.342+0000 7f6c7d213700 1 -- 172.21.15.121:0/3125330498 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c78106e90 con 0x7f6c78060280 2024-02-15T18:28:23.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.342+0000 7f6c5f7fe700 1 -- 172.21.15.121:0/3125330498 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c68013380 con 0x7f6c78060280 2024-02-15T18:28:23.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.342+0000 7f6c5f7fe700 1 -- 172.21.15.121:0/3125330498 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c68014e20 con 0x7f6c78060280 2024-02-15T18:28:23.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.342+0000 7f6c7d213700 1 -- 172.21.15.121:0/3125330498 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c78107320 con 0x7f6c78060280 2024-02-15T18:28:23.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.343+0000 7f6c5f7fe700 1 -- 172.21.15.121:0/3125330498 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f6c680263a0 con 0x7f6c78060280 2024-02-15T18:28:23.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.343+0000 7f6c7d213700 1 -- 172.21.15.121:0/3125330498 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c78045ad0 con 0x7f6c78060280 2024-02-15T18:28:23.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.343+0000 7f6c5f7fe700 1 --2- 172.21.15.121:0/3125330498 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6c6002ff10 0x7f6c600323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:23.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.343+0000 7f6c5f7fe700 1 -- 172.21.15.121:0/3125330498 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f6c68040f90 con 0x7f6c78060280 2024-02-15T18:28:23.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.344+0000 7f6c7659c700 1 --2- 172.21.15.121:0/3125330498 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6c6002ff10 0x7f6c600323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:23.343 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.344+0000 7f6c7659c700 1 --2- 172.21.15.121:0/3125330498 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6c6002ff10 0x7f6c600323d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f6c6c0096a0 tx=0x7f6c6c006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:23.344 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.348+0000 7f6c5f7fe700 1 -- 172.21.15.121:0/3125330498 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f6c6806d2b0 con 0x7f6c78060280 2024-02-15T18:28:23.596 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.600+0000 7f6c7d213700 1 -- 172.21.15.121:0/3125330498 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6c781081d0 con 0x7f6c78060280 2024-02-15T18:28:23.596 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.601+0000 7f6c5f7fe700 1 -- 172.21.15.121:0/3125330498 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f6c680126f0 con 0x7f6c78060280 2024-02-15T18:28:23.597 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:28:23.597 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":1,"fsid":"7e05753e-cc2f-11ee-95ba-87774f69a715","modified":"2024-02-15T18:25:30.869198Z","created":"2024-02-15T18:25:30.869198Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-15T18:28:23.598 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.603+0000 7f6c7d213700 1 -- 172.21.15.121:0/3125330498 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6c6002ff10 msgr2=0x7f6c600323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:23.598 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.603+0000 7f6c7d213700 1 --2- 172.21.15.121:0/3125330498 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6c6002ff10 0x7f6c600323d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f6c6c0096a0 tx=0x7f6c6c006b40).stop 2024-02-15T18:28:23.598 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.603+0000 7f6c7d213700 1 -- 172.21.15.121:0/3125330498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c78060280 msgr2=0x7f6c781067a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:23.598 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.603+0000 7f6c7d213700 1 --2- 172.21.15.121:0/3125330498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c78060280 0x7f6c781067a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f6c680089f0 tx=0x7f6c6800b040).stop 2024-02-15T18:28:23.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.603+0000 7f6c7d213700 1 -- 172.21.15.121:0/3125330498 shutdown_connections 2024-02-15T18:28:23.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.603+0000 7f6c7d213700 1 --2- 172.21.15.121:0/3125330498 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6c6002ff10 0x7f6c600323d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:23.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.603+0000 7f6c7d213700 1 --2- 172.21.15.121:0/3125330498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c78060280 0x7f6c781067a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:23.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.603+0000 7f6c7d213700 1 -- 172.21.15.121:0/3125330498 >> 172.21.15.121:0/3125330498 conn(0x7f6c780f0b70 msgr2=0x7f6c780f1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:23.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.603+0000 7f6c7d213700 1 -- 172.21.15.121:0/3125330498 shutdown_connections 2024-02-15T18:28:23.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:23.603+0000 7f6c7d213700 1 -- 172.21.15.121:0/3125330498 wait complete. 2024-02-15T18:28:23.599 INFO:teuthology.orchestra.run.smithi121.stderr:dumped monmap epoch 1 2024-02-15T18:28:23.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:22.229243+0000 mgr.smithi038.khesbl (mgr.14162) 59 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:23.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:22.484653+0000 mon.smithi038 (mon.0) 160 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:23.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:22.486001+0000 mon.smithi038 (mon.0) 161 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-15T18:28:23.778 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:22.486842+0000 mon.smithi038 (mon.0) 162 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:28:23.778 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:28:22.487550+0000 mgr.smithi038.khesbl (mgr.14162) 60 : cephadm [INF] Deploying daemon mon.smithi121 on smithi121 2024-02-15T18:28:24.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:23.602283+0000 mon.smithi038 (mon.0) 163 : audit [DBG] from='client.? 172.21.15.121:0/3125330498' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:28:25.307 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:28:25.307 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:28:25.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:24.229448+0000 mgr.smithi038.khesbl (mgr.14162) 61 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:27.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:26 smithi121 systemd[1]: Starting Ceph mon.smithi121 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:28:27.613 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.520+0000 7f0a58184700 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-15T18:28:27.614 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.520+0000 7f0a58184700 0 ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable), process ceph-mon, pid 7 2024-02-15T18:28:27.614 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.520+0000 7f0a58184700 0 pidfile_write: ignore empty --pid-file 2024-02-15T18:28:27.614 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 0 load: jerasure load: lrc load: isa 2024-02-15T18:28:27.614 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: RocksDB version: 6.8.1 2024-02-15T18:28:27.614 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.614 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-15T18:28:27.614 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Compile date May 13 2021 2024-02-15T18:28:27.614 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: DB SUMMARY 2024-02-15T18:28:27.614 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.614 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: CURRENT file: CURRENT 2024-02-15T18:28:27.614 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.615 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: IDENTITY file: IDENTITY 2024-02-15T18:28:27.615 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.615 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: MANIFEST file: MANIFEST-000001 size: 13 Bytes 2024-02-15T18:28:27.615 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.615 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi121/store.db dir, Total Num: 0, files: 2024-02-15T18:28:27.615 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.615 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi121/store.db: 000003.log size: 511 ; 2024-02-15T18:28:27.615 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.615 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.error_if_exists: 0 2024-02-15T18:28:27.615 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.create_if_missing: 0 2024-02-15T18:28:27.615 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.paranoid_checks: 1 2024-02-15T18:28:27.615 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.env: 0x55c94e7b11c0 2024-02-15T18:28:27.615 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.fs: Posix File System 2024-02-15T18:28:27.616 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.info_log: 0x55c94f3cd960 2024-02-15T18:28:27.616 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-15T18:28:27.616 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.statistics: (nil) 2024-02-15T18:28:27.616 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.use_fsync: 0 2024-02-15T18:28:27.616 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.max_log_file_size: 0 2024-02-15T18:28:27.616 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-15T18:28:27.616 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-15T18:28:27.616 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-15T18:28:27.616 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-15T18:28:27.616 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.allow_fallocate: 1 2024-02-15T18:28:27.616 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-15T18:28:27.616 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-15T18:28:27.617 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.use_direct_reads: 0 2024-02-15T18:28:27.617 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-15T18:28:27.617 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.create_missing_column_families: 0 2024-02-15T18:28:27.617 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.db_log_dir: 2024-02-15T18:28:27.617 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi121/store.db 2024-02-15T18:28:27.617 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-15T18:28:27.617 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.max_subcompactions: 1 2024-02-15T18:28:27.617 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.max_background_flushes: -1 2024-02-15T18:28:27.617 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-15T18:28:27.617 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-15T18:28:27.617 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-15T18:28:27.617 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-15T18:28:27.617 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-15T18:28:27.617 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.advise_random_on_open: 1 2024-02-15T18:28:27.618 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-15T18:28:27.618 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.write_buffer_manager: 0x55c950042030 2024-02-15T18:28:27.618 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-15T18:28:27.618 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-15T18:28:27.618 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-15T18:28:27.618 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-15T18:28:27.618 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.rate_limiter: (nil) 2024-02-15T18:28:27.618 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-15T18:28:27.618 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-15T18:28:27.618 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-15T18:28:27.618 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-15T18:28:27.618 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.unordered_write: 0 2024-02-15T18:28:27.618 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-15T18:28:27.618 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-15T18:28:27.618 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-15T18:28:27.619 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-15T18:28:27.619 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.row_cache: None 2024-02-15T18:28:27.619 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.wal_filter: None 2024-02-15T18:28:27.619 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-15T18:28:27.619 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-15T18:28:27.619 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.preserve_deletes: 0 2024-02-15T18:28:27.619 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.two_write_queues: 0 2024-02-15T18:28:27.619 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.manual_wal_flush: 0 2024-02-15T18:28:27.619 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.atomic_flush: 0 2024-02-15T18:28:27.619 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-15T18:28:27.619 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-15T18:28:27.619 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-15T18:28:27.619 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.log_readahead_size: 0 2024-02-15T18:28:27.620 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-15T18:28:27.620 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.max_background_jobs: 2 2024-02-15T18:28:27.620 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.max_background_compactions: -1 2024-02-15T18:28:27.620 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-15T18:28:27.620 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-15T18:28:27.620 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-15T18:28:27.620 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.max_total_wal_size: 0 2024-02-15T18:28:27.620 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-15T18:28:27.620 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-15T18:28:27.620 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-15T18:28:27.620 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-15T18:28:27.620 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.max_open_files: -1 2024-02-15T18:28:27.621 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.bytes_per_sync: 0 2024-02-15T18:28:27.621 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-15T18:28:27.621 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-15T18:28:27.621 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-15T18:28:27.621 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Compression algorithms supported: 2024-02-15T18:28:27.621 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-15T18:28:27.621 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: kZSTD supported: 0 2024-02-15T18:28:27.621 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: kXpressCompression supported: 0 2024-02-15T18:28:27.621 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-15T18:28:27.621 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: kLZ4Compression supported: 1 2024-02-15T18:28:27.621 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: kBZip2Compression supported: 0 2024-02-15T18:28:27.621 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: kZlibCompression supported: 1 2024-02-15T18:28:27.621 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: kSnappyCompression supported: 1 2024-02-15T18:28:27.622 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.525+0000 7f0a58184700 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-15T18:28:27.622 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi121/store.db/MANIFEST-000001 2024-02-15T18:28:27.622 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.622 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-15T18:28:27.622 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.622 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-15T18:28:27.622 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.merge_operator: 2024-02-15T18:28:27.622 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.compaction_filter: None 2024-02-15T18:28:27.622 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.compaction_filter_factory: None 2024-02-15T18:28:27.622 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-15T18:28:27.622 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-15T18:28:27.622 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c94f2e5cb0) 2024-02-15T18:28:27.622 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cache_index_and_filter_blocks: 1 2024-02-15T18:28:27.623 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-15T18:28:27.623 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-15T18:28:27.623 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: pin_top_level_index_and_filter: 1 2024-02-15T18:28:27.623 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: index_type: 0 2024-02-15T18:28:27.623 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: data_block_index_type: 0 2024-02-15T18:28:27.623 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: index_shortening: 1 2024-02-15T18:28:27.623 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: data_block_hash_table_util_ratio: 0.750000 2024-02-15T18:28:27.623 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: hash_index_allow_collision: 1 2024-02-15T18:28:27.623 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: checksum: 1 2024-02-15T18:28:27.623 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: no_block_cache: 0 2024-02-15T18:28:27.623 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: block_cache: 0x55c94f31af10 2024-02-15T18:28:27.623 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: block_cache_name: BinnedLRUCache 2024-02-15T18:28:27.623 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: block_cache_options: 2024-02-15T18:28:27.623 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: capacity : 536870912 2024-02-15T18:28:27.624 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: num_shard_bits : 4 2024-02-15T18:28:27.624 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: strict_capacity_limit : 0 2024-02-15T18:28:27.624 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: high_pri_pool_ratio: 0.000 2024-02-15T18:28:27.624 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: block_cache_compressed: (nil) 2024-02-15T18:28:27.624 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: persistent_cache: (nil) 2024-02-15T18:28:27.624 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: block_size: 4096 2024-02-15T18:28:27.624 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: block_size_deviation: 10 2024-02-15T18:28:27.624 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: block_restart_interval: 16 2024-02-15T18:28:27.624 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: index_block_restart_interval: 1 2024-02-15T18:28:27.624 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: metadata_block_size: 4096 2024-02-15T18:28:27.624 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: partition_filters: 0 2024-02-15T18:28:27.624 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: use_delta_encoding: 1 2024-02-15T18:28:27.624 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-15T18:28:27.625 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: whole_key_filtering: 1 2024-02-15T18:28:27.625 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: verify_compression: 0 2024-02-15T18:28:27.625 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: read_amp_bytes_per_bit: 0 2024-02-15T18:28:27.625 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: format_version: 2 2024-02-15T18:28:27.625 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: enable_index_compression: 1 2024-02-15T18:28:27.625 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: block_align: 0 2024-02-15T18:28:27.625 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.625 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-15T18:28:27.625 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-15T18:28:27.625 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.compression: NoCompression 2024-02-15T18:28:27.625 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-15T18:28:27.625 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-15T18:28:27.625 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-15T18:28:27.626 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.num_levels: 7 2024-02-15T18:28:27.626 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-15T18:28:27.626 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-15T18:28:27.626 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-15T18:28:27.626 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-15T18:28:27.626 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-15T18:28:27.626 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-15T18:28:27.626 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-15T18:28:27.626 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-15T18:28:27.626 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-15T18:28:27.626 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-15T18:28:27.626 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.compression_opts.level: 32767 2024-02-15T18:28:27.626 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-15T18:28:27.627 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-15T18:28:27.627 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-15T18:28:27.627 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.compression_opts.enabled: false 2024-02-15T18:28:27.627 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-15T18:28:27.627 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-15T18:28:27.627 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-15T18:28:27.627 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-15T18:28:27.627 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-15T18:28:27.627 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-15T18:28:27.627 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-15T18:28:27.627 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-15T18:28:27.627 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-15T18:28:27.627 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-15T18:28:27.628 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-15T18:28:27.628 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-15T18:28:27.628 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-15T18:28:27.628 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-15T18:28:27.628 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-15T18:28:27.628 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-15T18:28:27.628 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-15T18:28:27.628 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.arena_block_size: 4194304 2024-02-15T18:28:27.628 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-15T18:28:27.628 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-15T18:28:27.628 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-15T18:28:27.628 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-15T18:28:27.629 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-15T18:28:27.629 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-15T18:28:27.629 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-15T18:28:27.629 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-15T18:28:27.629 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-15T18:28:27.629 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-15T18:28:27.629 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-15T18:28:27.629 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-15T18:28:27.629 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-15T18:28:27.629 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-15T18:28:27.629 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.table_properties_collectors: 2024-02-15T18:28:27.629 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.inplace_update_support: 0 2024-02-15T18:28:27.629 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-15T18:28:27.629 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-15T18:28:27.630 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-15T18:28:27.630 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-15T18:28:27.630 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.bloom_locality: 0 2024-02-15T18:28:27.630 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.max_successive_merges: 0 2024-02-15T18:28:27.630 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-15T18:28:27.630 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-15T18:28:27.630 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.force_consistency_checks: 0 2024-02-15T18:28:27.630 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-15T18:28:27.630 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.ttl: 2592000 2024-02-15T18:28:27.630 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-15T18:28:27.630 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi121/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-15T18:28:27.630 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.630 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-02-15T18:28:27.631 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.631 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708021707526890, "job": 1, "event": "recovery_started", "log_files": [3]} 2024-02-15T18:28:27.631 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #3 mode 2 2024-02-15T18:28:27.631 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.526+0000 7f0a58184700 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-15T18:28:27.631 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.527+0000 7f0a58184700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708021707527333, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 4, "file_size": 1437, "table_properties": {"data_size": 523, "index_size": 40, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708021707, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-15T18:28:27.631 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.527+0000 7f0a58184700 4 rocksdb: [version_set.cc:3826] Creating manifest 5 2024-02-15T18:28:27.631 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.631 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.527+0000 7f0a58184700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708021707527699, "job": 1, "event": "recovery_finished"} 2024-02-15T18:28:27.631 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.528+0000 7f0a58184700 4 rocksdb: DB pointer 0x55c94f3df800 2024-02-15T18:28:27.631 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.528+0000 7f0a3ef9c700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-15T18:28:27.631 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.528+0000 7f0a3ef9c700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-15T18:28:27.631 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: ** DB Stats ** 2024-02-15T18:28:27.631 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:28:27.632 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-15T18:28:27.632 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-15T18:28:27.632 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-15T18:28:27.632 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-15T18:28:27.632 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-15T18:28:27.632 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-15T18:28:27.632 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.632 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: ** Compaction Stats [default] ** 2024-02-15T18:28:27.632 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:28:27.632 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:28:27.632 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: L0 1/0 1.40 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.000 0 0 2024-02-15T18:28:27.632 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Sum 1/0 1.40 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.000 0 0 2024-02-15T18:28:27.632 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.000 0 0 2024-02-15T18:28:27.633 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.633 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: ** Compaction Stats [default] ** 2024-02-15T18:28:27.633 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:28:27.633 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:28:27.633 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.4 0.00 0.00 1 0.000 0 0 2024-02-15T18:28:27.633 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:28:27.633 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-15T18:28:27.633 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-15T18:28:27.633 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: AddFile(Total Files): cumulative 0, interval 0 2024-02-15T18:28:27.633 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-15T18:28:27.633 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: AddFile(Keys): cumulative 0, interval 0 2024-02-15T18:28:27.633 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:28:27.633 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:28:27.633 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-15T18:28:27.633 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.634 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: ** File Read Latency Histogram By Level [default] ** 2024-02-15T18:28:27.634 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.634 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: ** Compaction Stats [default] ** 2024-02-15T18:28:27.634 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:28:27.634 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:28:27.634 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: L0 1/0 1.40 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.000 0 0 2024-02-15T18:28:27.634 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Sum 1/0 1.40 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.000 0 0 2024-02-15T18:28:27.634 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:28:27.634 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.634 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: ** Compaction Stats [default] ** 2024-02-15T18:28:27.634 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:28:27.634 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:28:27.634 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.4 0.00 0.00 1 0.000 0 0 2024-02-15T18:28:27.634 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:28:27.634 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-15T18:28:27.635 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-15T18:28:27.635 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: AddFile(Total Files): cumulative 0, interval 0 2024-02-15T18:28:27.635 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-15T18:28:27.635 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: AddFile(Keys): cumulative 0, interval 0 2024-02-15T18:28:27.635 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:28:27.635 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:28:27.636 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-15T18:28:27.636 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.636 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: ** File Read Latency Histogram By Level [default] ** 2024-02-15T18:28:27.636 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.636 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.528+0000 7f0a58184700 0 mon.smithi121 does not exist in monmap, will attempt to join an existing cluster 2024-02-15T18:28:27.636 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.528+0000 7f0a58184700 0 using public_addr v2:172.21.15.121:0/0 -> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] 2024-02-15T18:28:27.636 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.529+0000 7f0a58184700 0 starting mon.smithi121 rank -1 at public addrs [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] at bind addrs [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi121 fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:28:27.636 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.529+0000 7f0a58184700 1 mon.smithi121@-1(???) e0 preinit fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:28:27.636 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.562+0000 7f0a417a1700 0 mon.smithi121@-1(synchronizing).mds e1 new map 2024-02-15T18:28:27.636 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.562+0000 7f0a417a1700 0 mon.smithi121@-1(synchronizing).mds e1 print_map 2024-02-15T18:28:27.636 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: e1 2024-02-15T18:28:27.636 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:28:27.637 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:28:27.637 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: legacy client fscid: -1 2024-02-15T18:28:27.637 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.637 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: No filesystems configured 2024-02-15T18:28:27.637 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: 2024-02-15T18:28:27.637 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.562+0000 7f0a417a1700 1 mon.smithi121@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-15T18:28:27.637 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.562+0000 7f0a417a1700 1 mon.smithi121@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-15T18:28:27.637 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.562+0000 7f0a417a1700 1 mon.smithi121@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-15T18:28:27.637 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.562+0000 7f0a417a1700 1 mon.smithi121@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-15T18:28:27.637 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.562+0000 7f0a417a1700 1 mon.smithi121@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-15T18:28:27.637 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.562+0000 7f0a417a1700 1 mon.smithi121@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-15T18:28:27.637 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.562+0000 7f0a417a1700 1 mon.smithi121@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-02-15T18:28:27.637 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.562+0000 7f0a417a1700 0 mon.smithi121@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-15T18:28:27.637 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.562+0000 7f0a417a1700 0 mon.smithi121@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-02-15T18:28:27.638 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.562+0000 7f0a417a1700 0 mon.smithi121@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-02-15T18:28:27.638 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.562+0000 7f0a417a1700 0 mon.smithi121@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-02-15T18:28:27.638 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:26.229759+0000 mgr.smithi038.khesbl (mgr.14162) 62 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:27.638 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:27.362484+0000 mon.smithi038 (mon.0) 164 : cluster [DBG] Standby manager daemon smithi121.ckiryi started 2024-02-15T18:28:27.638 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:27.363801+0000 mon.smithi038 (mon.0) 165 : audit [DBG] from='mgr.? 172.21.15.121:0/3564052163' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi121.ckiryi/crt"}]: dispatch 2024-02-15T18:28:27.638 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:27.364706+0000 mon.smithi038 (mon.0) 166 : audit [DBG] from='mgr.? 172.21.15.121:0/3564052163' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:28:27.638 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:27.366640+0000 mon.smithi038 (mon.0) 167 : audit [DBG] from='mgr.? 172.21.15.121:0/3564052163' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi121.ckiryi/key"}]: dispatch 2024-02-15T18:28:27.638 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:27.367371+0000 mon.smithi038 (mon.0) 168 : audit [DBG] from='mgr.? 172.21.15.121:0/3564052163' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:28:27.638 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:28:27.563+0000 7f0a417a1700 1 mon.smithi121@-1(synchronizing).paxosservice(auth 1..7) refresh upgraded, format 0 -> 3 2024-02-15T18:28:27.638 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 bash[40974]: 8f588ecacb71dbd24574b9ba279faf6ae268f9e9bb3e62bf050d601684b514a4 2024-02-15T18:28:27.638 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:27 smithi121 systemd[1]: Started Ceph mon.smithi121 for 7e05753e-cc2f-11ee-95ba-87774f69a715. 2024-02-15T18:28:27.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:26.229759+0000 mgr.smithi038.khesbl (mgr.14162) 62 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:27.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:27.362484+0000 mon.smithi038 (mon.0) 164 : cluster [DBG] Standby manager daemon smithi121.ckiryi started 2024-02-15T18:28:27.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:27.363801+0000 mon.smithi038 (mon.0) 165 : audit [DBG] from='mgr.? 172.21.15.121:0/3564052163' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi121.ckiryi/crt"}]: dispatch 2024-02-15T18:28:27.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:27.364706+0000 mon.smithi038 (mon.0) 166 : audit [DBG] from='mgr.? 172.21.15.121:0/3564052163' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:28:27.778 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:27.366640+0000 mon.smithi038 (mon.0) 167 : audit [DBG] from='mgr.? 172.21.15.121:0/3564052163' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi121.ckiryi/key"}]: dispatch 2024-02-15T18:28:27.778 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:27.367371+0000 mon.smithi038 (mon.0) 168 : audit [DBG] from='mgr.? 172.21.15.121:0/3564052163' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:28:32.914 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.917+0000 7f48f09f3700 1 -- 172.21.15.121:0/2560541412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48d8003250 msgr2=0x7f48d80056e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:32.914 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.917+0000 7f48f09f3700 1 --2- 172.21.15.121:0/2560541412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48d8003250 0x7f48d80056e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f48ec0f61c0 tx=0x7f48d4002040).stop 2024-02-15T18:28:32.914 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.917+0000 7f48f09f3700 1 -- 172.21.15.121:0/2560541412 shutdown_connections 2024-02-15T18:28:32.914 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.917+0000 7f48f09f3700 1 --2- 172.21.15.121:0/2560541412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48d8003250 0x7f48d80056e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:32.914 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.917+0000 7f48f09f3700 1 -- 172.21.15.121:0/2560541412 >> 172.21.15.121:0/2560541412 conn(0x7f48ec0f0b70 msgr2=0x7f48ec0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:32.914 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.917+0000 7f48f09f3700 1 -- 172.21.15.121:0/2560541412 shutdown_connections 2024-02-15T18:28:32.914 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.917+0000 7f48f09f3700 1 -- 172.21.15.121:0/2560541412 wait complete. 2024-02-15T18:28:32.914 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.918+0000 7f48f09f3700 1 Processor -- start 2024-02-15T18:28:32.914 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.918+0000 7f48f09f3700 1 -- start start 2024-02-15T18:28:32.915 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.918+0000 7f48f09f3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48d8003250 0x7f48ec1094a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:32.915 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.918+0000 7f48f09f3700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48ec060220 0x7f48ec10b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:32.915 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.918+0000 7f48f09f3700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec10bf70 con 0x7f48ec060220 2024-02-15T18:28:32.915 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.918+0000 7f48f09f3700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec10c0b0 con 0x7f48d8003250 2024-02-15T18:28:32.915 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.918+0000 7f48ea59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48d8003250 0x7f48ec1094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:32.915 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.918+0000 7f48ea59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48d8003250 0x7f48ec1094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59340/0 (socket says 172.21.15.121:59340) 2024-02-15T18:28:32.915 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.918+0000 7f48ea59c700 1 -- 172.21.15.121:0/159672681 learned_addr learned my addr 172.21.15.121:0/159672681 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-02-15T18:28:32.915 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.918+0000 7f48e9d9b700 1 --2- 172.21.15.121:0/159672681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48ec060220 0x7f48ec10b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:32.915 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.918+0000 7f48ea59c700 1 -- 172.21.15.121:0/159672681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48d8003250 msgr2=0x7f48ec1094a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-15T18:28:32.915 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.918+0000 7f48ea59c700 1 -- 172.21.15.121:0/159672681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48d8003250 msgr2=0x7f48ec1094a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-15T18:28:32.915 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.918+0000 7f48ea59c700 1 --2- 172.21.15.121:0/159672681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48d8003250 0x7f48ec1094a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-15T18:28:32.915 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.918+0000 7f48ea59c700 1 --2- 172.21.15.121:0/159672681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48d8003250 0x7f48ec1094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-02-15T18:28:32.915 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.918+0000 7f48e9d9b700 1 -- 172.21.15.121:0/159672681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48d8003250 msgr2=0x7f48ec1094a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-15T18:28:32.915 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.918+0000 7f48e9d9b700 1 --2- 172.21.15.121:0/159672681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48d8003250 0x7f48ec1094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:32.916 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.918+0000 7f48e9d9b700 1 -- 172.21.15.121:0/159672681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d4005040 con 0x7f48ec060220 2024-02-15T18:28:32.916 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.919+0000 7f48e9d9b700 1 --2- 172.21.15.121:0/159672681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48ec060220 0x7f48ec10b9a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f48dc009390 tx=0x7f48dc006e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:32.916 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.919+0000 7f48e2ffd700 1 -- 172.21.15.121:0/159672681 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48dc00e650 con 0x7f48ec060220 2024-02-15T18:28:32.916 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.919+0000 7f48f09f3700 1 -- 172.21.15.121:0/159672681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ec10c390 con 0x7f48ec060220 2024-02-15T18:28:32.916 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.919+0000 7f48e2ffd700 1 -- 172.21.15.121:0/159672681 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48dc009e40 con 0x7f48ec060220 2024-02-15T18:28:32.916 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.919+0000 7f48e2ffd700 1 -- 172.21.15.121:0/159672681 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48dc011c30 con 0x7f48ec060220 2024-02-15T18:28:32.916 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.919+0000 7f48f09f3700 1 -- 172.21.15.121:0/159672681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ec10c8d0 con 0x7f48ec060220 2024-02-15T18:28:32.917 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.920+0000 7f48f09f3700 1 -- 172.21.15.121:0/159672681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ec045ad0 con 0x7f48ec060220 2024-02-15T18:28:32.917 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.920+0000 7f48e2ffd700 1 -- 172.21.15.121:0/159672681 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f48dc018030 con 0x7f48ec060220 2024-02-15T18:28:32.917 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.921+0000 7f48e2ffd700 1 --2- 172.21.15.121:0/159672681 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f48cc05c400 0x7f48cc05e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:32.917 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.921+0000 7f48e2ffd700 1 -- 172.21.15.121:0/159672681 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f48dc074870 con 0x7f48ec060220 2024-02-15T18:28:32.917 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.921+0000 7f48ea59c700 1 --2- 172.21.15.121:0/159672681 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f48cc05c400 0x7f48cc05e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:32.917 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.921+0000 7f48ea59c700 1 --2- 172.21.15.121:0/159672681 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f48cc05c400 0x7f48cc05e8c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f48d4004d50 tx=0x7f48d4002040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:32.919 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:32.925+0000 7f48e2ffd700 1 -- 172.21.15.121:0/159672681 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f48dc0452b0 con 0x7f48ec060220 2024-02-15T18:28:32.960 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:27.570591+0000 mon.smithi038 (mon.0) 172 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-02-15T18:28:32.960 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:27.570856+0000 mon.smithi038 (mon.0) 173 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:28:32.960 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:27.571013+0000 mon.smithi038 (mon.0) 174 : cluster [INF] mon.smithi038 calling monitor election 2024-02-15T18:28:32.960 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:28.230098+0000 mgr.smithi038.khesbl (mgr.14162) 63 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:32.960 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:28.567282+0000 mon.smithi038 (mon.0) 175 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:28:32.960 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:29.567385+0000 mon.smithi038 (mon.0) 176 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:28:32.960 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:29.568358+0000 mon.smithi121 (mon.1) 1 : cluster [INF] mon.smithi121 calling monitor election 2024-02-15T18:28:32.960 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:30.230458+0000 mgr.smithi038.khesbl (mgr.14162) 64 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:32.960 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:30.567417+0000 mon.smithi038 (mon.0) 177 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:28:32.960 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:31.567727+0000 mon.smithi038 (mon.0) 178 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:28:32.961 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:32.567547+0000 mon.smithi038 (mon.0) 179 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:28:32.961 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:32.588431+0000 mon.smithi038 (mon.0) 180 : cluster [INF] mon.smithi038 is new leader, mons smithi038,smithi121 in quorum (ranks 0,1) 2024-02-15T18:28:32.961 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:32.591641+0000 mon.smithi038 (mon.0) 181 : cluster [DBG] monmap e2: 2 mons at {smithi038=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0],smithi121=[v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0]} 2024-02-15T18:28:32.961 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:32.591688+0000 mon.smithi038 (mon.0) 182 : cluster [DBG] fsmap 2024-02-15T18:28:32.961 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:32.591717+0000 mon.smithi038 (mon.0) 183 : cluster [DBG] osdmap e5: 0 total, 0 up, 0 in 2024-02-15T18:28:32.961 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:32.592081+0000 mon.smithi038 (mon.0) 184 : cluster [DBG] mgrmap e18: smithi038.khesbl(active, since 112s), standbys: smithi121.ckiryi 2024-02-15T18:28:32.961 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:32.596248+0000 mon.smithi038 (mon.0) 185 : cluster [INF] overall HEALTH_OK 2024-02-15T18:28:32.961 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:32.598927+0000 mon.smithi038 (mon.0) 186 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:33.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:27.570591+0000 mon.smithi038 (mon.0) 172 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-02-15T18:28:33.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:27.570856+0000 mon.smithi038 (mon.0) 173 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:28:33.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:27.571013+0000 mon.smithi038 (mon.0) 174 : cluster [INF] mon.smithi038 calling monitor election 2024-02-15T18:28:33.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:28.230098+0000 mgr.smithi038.khesbl (mgr.14162) 63 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:33.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:28.567282+0000 mon.smithi038 (mon.0) 175 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:28:33.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:29.567385+0000 mon.smithi038 (mon.0) 176 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:28:33.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:29.568358+0000 mon.smithi121 (mon.1) 1 : cluster [INF] mon.smithi121 calling monitor election 2024-02-15T18:28:33.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:30.230458+0000 mgr.smithi038.khesbl (mgr.14162) 64 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:33.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:30.567417+0000 mon.smithi038 (mon.0) 177 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:28:33.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:31.567727+0000 mon.smithi038 (mon.0) 178 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:28:33.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:32.567547+0000 mon.smithi038 (mon.0) 179 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:28:33.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:32.588431+0000 mon.smithi038 (mon.0) 180 : cluster [INF] mon.smithi038 is new leader, mons smithi038,smithi121 in quorum (ranks 0,1) 2024-02-15T18:28:33.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:32.591641+0000 mon.smithi038 (mon.0) 181 : cluster [DBG] monmap e2: 2 mons at {smithi038=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0],smithi121=[v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0]} 2024-02-15T18:28:33.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:32.591688+0000 mon.smithi038 (mon.0) 182 : cluster [DBG] fsmap 2024-02-15T18:28:33.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:32.591717+0000 mon.smithi038 (mon.0) 183 : cluster [DBG] osdmap e5: 0 total, 0 up, 0 in 2024-02-15T18:28:33.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:32.592081+0000 mon.smithi038 (mon.0) 184 : cluster [DBG] mgrmap e18: smithi038.khesbl(active, since 112s), standbys: smithi121.ckiryi 2024-02-15T18:28:33.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:32.596248+0000 mon.smithi038 (mon.0) 185 : cluster [INF] overall HEALTH_OK 2024-02-15T18:28:33.029 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:32.598927+0000 mon.smithi038 (mon.0) 186 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:33.168 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:33.173+0000 7f48f09f3700 1 -- 172.21.15.121:0/159672681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f48ec027a00 con 0x7f48ec060220 2024-02-15T18:28:33.168 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:33.174+0000 7f48e2ffd700 1 -- 172.21.15.121:0/159672681 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+968 (secure 0 0 0) 0x7f48dc016030 con 0x7f48ec060220 2024-02-15T18:28:33.169 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-02-15T18:28:33.169 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":2,"fsid":"7e05753e-cc2f-11ee-95ba-87774f69a715","modified":"2024-02-15T18:28:27.567249Z","created":"2024-02-15T18:25:30.869198Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi038","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi121","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:3300","nonce":0},{"type":"v1","addr":"172.21.15.121:6789","nonce":0}]},"addr":"172.21.15.121:6789/0","public_addr":"172.21.15.121:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-15T18:28:33.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:33.176+0000 7f48f09f3700 1 -- 172.21.15.121:0/159672681 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f48cc05c400 msgr2=0x7f48cc05e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:33.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:33.176+0000 7f48f09f3700 1 --2- 172.21.15.121:0/159672681 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f48cc05c400 0x7f48cc05e8c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f48d4004d50 tx=0x7f48d4002040).stop 2024-02-15T18:28:33.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:33.176+0000 7f48f09f3700 1 -- 172.21.15.121:0/159672681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48ec060220 msgr2=0x7f48ec10b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:33.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:33.176+0000 7f48f09f3700 1 --2- 172.21.15.121:0/159672681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48ec060220 0x7f48ec10b9a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f48dc009390 tx=0x7f48dc006e20).stop 2024-02-15T18:28:33.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:33.176+0000 7f48f09f3700 1 -- 172.21.15.121:0/159672681 shutdown_connections 2024-02-15T18:28:33.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:33.176+0000 7f48f09f3700 1 --2- 172.21.15.121:0/159672681 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f48cc05c400 0x7f48cc05e8c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:33.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:33.176+0000 7f48f09f3700 1 --2- 172.21.15.121:0/159672681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48d8003250 0x7f48ec1094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:33.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:33.176+0000 7f48f09f3700 1 --2- 172.21.15.121:0/159672681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48ec060220 0x7f48ec10b9a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:33.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:33.176+0000 7f48f09f3700 1 -- 172.21.15.121:0/159672681 >> 172.21.15.121:0/159672681 conn(0x7f48ec0f0b70 msgr2=0x7f48ec0f9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:33.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:33.176+0000 7f48f09f3700 1 -- 172.21.15.121:0/159672681 shutdown_connections 2024-02-15T18:28:33.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:28:33.176+0000 7f48f09f3700 1 -- 172.21.15.121:0/159672681 wait complete. 2024-02-15T18:28:33.172 INFO:teuthology.orchestra.run.smithi121.stderr:dumped monmap epoch 2 2024-02-15T18:28:33.763 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:32.230807+0000 mgr.smithi038.khesbl (mgr.14162) 65 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:33.764 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:28:32.599985+0000 mgr.smithi038.khesbl (mgr.14162) 66 : cephadm [INF] Deploying daemon node-exporter.smithi121 on smithi121 2024-02-15T18:28:33.764 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:33.174484+0000 mon.smithi038 (mon.0) 187 : audit [DBG] from='client.? 172.21.15.121:0/159672681' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:28:33.764 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:33.567698+0000 mon.smithi038 (mon.0) 188 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:28:33.765 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-15T18:28:33.765 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph config generate-minimal-conf 2024-02-15T18:28:34.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:32.230807+0000 mgr.smithi038.khesbl (mgr.14162) 65 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:34.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:28:32.599985+0000 mgr.smithi038.khesbl (mgr.14162) 66 : cephadm [INF] Deploying daemon node-exporter.smithi121 on smithi121 2024-02-15T18:28:34.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:33.174484+0000 mon.smithi038 (mon.0) 187 : audit [DBG] from='client.? 172.21.15.121:0/159672681' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:28:34.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:33.567698+0000 mon.smithi038 (mon.0) 188 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:28:35.517 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:34.231037+0000 mgr.smithi038.khesbl (mgr.14162) 67 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:35.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:34.231037+0000 mgr.smithi038.khesbl (mgr.14162) 67 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:35.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.829+0000 7f8122ed4700 1 -- 172.21.15.38:0/3449194211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f811c0f5410 msgr2=0x7f811c0f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:35.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.829+0000 7f8122ed4700 1 --2- 172.21.15.38:0/3449194211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f811c0f5410 0x7f811c0f5830 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f8110009fb0 tx=0x7f8110009b30).stop 2024-02-15T18:28:35.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.829+0000 7f8122ed4700 1 -- 172.21.15.38:0/3449194211 shutdown_connections 2024-02-15T18:28:35.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.829+0000 7f8122ed4700 1 --2- 172.21.15.38:0/3449194211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f811c0f5410 0x7f811c0f5830 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.829+0000 7f8122ed4700 1 -- 172.21.15.38:0/3449194211 >> 172.21.15.38:0/3449194211 conn(0x7f811c0f0b70 msgr2=0x7f811c0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.830+0000 7f8122ed4700 1 -- 172.21.15.38:0/3449194211 shutdown_connections 2024-02-15T18:28:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.830+0000 7f8122ed4700 1 -- 172.21.15.38:0/3449194211 wait complete. 2024-02-15T18:28:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.830+0000 7f8122ed4700 1 Processor -- start 2024-02-15T18:28:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.830+0000 7f8122ed4700 1 -- start start 2024-02-15T18:28:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.830+0000 7f8122ed4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f811c0f5410 0x7f811c0fd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.830+0000 7f8122ed4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f811c0fb960 0x7f811c0fbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.830+0000 7f8122ed4700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c0fc360 con 0x7f811c0fb960 2024-02-15T18:28:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.830+0000 7f8122ed4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c0fc4a0 con 0x7f811c0f5410 2024-02-15T18:28:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.831+0000 7f811bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f811c0fb960 0x7f811c0fbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.831+0000 7f811bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f811c0fb960 0x7f811c0fbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37330/0 (socket says 172.21.15.38:37330) 2024-02-15T18:28:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.831+0000 7f811bfff700 1 -- 172.21.15.38:0/152591261 learned_addr learned my addr 172.21.15.38:0/152591261 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:28:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.831+0000 7f8120c70700 1 --2- 172.21.15.38:0/152591261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f811c0f5410 0x7f811c0fd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.831+0000 7f8120c70700 1 -- 172.21.15.38:0/152591261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f811c0f5410 msgr2=0x7f811c0fd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-15T18:28:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.831+0000 7f8120c70700 1 -- 172.21.15.38:0/152591261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f811c0f5410 msgr2=0x7f811c0fd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-15T18:28:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.831+0000 7f811bfff700 1 -- 172.21.15.38:0/152591261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f811c0f5410 msgr2=0x7f811c0fd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.831+0000 7f8120c70700 1 --2- 172.21.15.38:0/152591261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f811c0f5410 0x7f811c0fd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-15T18:28:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.831+0000 7f8120c70700 1 --2- 172.21.15.38:0/152591261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f811c0f5410 0x7f811c0fd4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-02-15T18:28:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.831+0000 7f811bfff700 1 --2- 172.21.15.38:0/152591261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f811c0f5410 0x7f811c0fd4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.831+0000 7f811bfff700 1 -- 172.21.15.38:0/152591261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8110005040 con 0x7f811c0fb960 2024-02-15T18:28:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.831+0000 7f811bfff700 1 --2- 172.21.15.38:0/152591261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f811c0fb960 0x7f811c0fbde0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f81080093c0 tx=0x7f8108006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.831+0000 7f81197fa700 1 -- 172.21.15.38:0/152591261 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f810800e650 con 0x7f811c0fb960 2024-02-15T18:28:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.831+0000 7f81197fa700 1 -- 172.21.15.38:0/152591261 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f810800ec80 con 0x7f811c0fb960 2024-02-15T18:28:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.831+0000 7f81197fa700 1 -- 172.21.15.38:0/152591261 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8108011cf0 con 0x7f811c0fb960 2024-02-15T18:28:35.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.831+0000 7f8122ed4700 1 -- 172.21.15.38:0/152591261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811c0fc750 con 0x7f811c0fb960 2024-02-15T18:28:35.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.831+0000 7f8122ed4700 1 -- 172.21.15.38:0/152591261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811c10ec80 con 0x7f811c0fb960 2024-02-15T18:28:35.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.832+0000 7f8122ed4700 1 -- 172.21.15.38:0/152591261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811c059100 con 0x7f811c0fb960 2024-02-15T18:28:35.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.833+0000 7f81197fa700 1 -- 172.21.15.38:0/152591261 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f8108018030 con 0x7f811c0fb960 2024-02-15T18:28:35.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.833+0000 7f81197fa700 1 --2- 172.21.15.38:0/152591261 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f810c05c400 0x7f810c05e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:35.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.833+0000 7f81197fa700 1 -- 172.21.15.38:0/152591261 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f81080748d0 con 0x7f811c0fb960 2024-02-15T18:28:35.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.833+0000 7f8120c70700 1 --2- 172.21.15.38:0/152591261 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f810c05c400 0x7f810c05e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:35.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.833+0000 7f8120c70700 1 --2- 172.21.15.38:0/152591261 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f810c05c400 0x7f810c05e8c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f8110000c00 tx=0x7f811000b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:35.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.837+0000 7f81197fa700 1 -- 172.21.15.38:0/152591261 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f81080452b0 con 0x7f811c0fb960 2024-02-15T18:28:35.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.999+0000 7f8122ed4700 1 -- 172.21.15.38:0/152591261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f811c045ad0 con 0x7f811c0fb960 2024-02-15T18:28:35.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:35.999+0000 7f81197fa700 1 -- 172.21.15.38:0/152591261 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v12) v1 ==== 76+0+225 (secure 0 0 0) 0x7f8108016030 con 0x7f811c0fb960 2024-02-15T18:28:35.995 INFO:teuthology.orchestra.run.smithi038.stdout:# minimal ceph.conf for 7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:28:35.995 INFO:teuthology.orchestra.run.smithi038.stdout:[global] 2024-02-15T18:28:35.995 INFO:teuthology.orchestra.run.smithi038.stdout: fsid = 7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:28:35.995 INFO:teuthology.orchestra.run.smithi038.stdout: mon_host = [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] 2024-02-15T18:28:35.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:36.001+0000 7f8122ed4700 1 -- 172.21.15.38:0/152591261 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f810c05c400 msgr2=0x7f810c05e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:35.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:36.001+0000 7f8122ed4700 1 --2- 172.21.15.38:0/152591261 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f810c05c400 0x7f810c05e8c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f8110000c00 tx=0x7f811000b040).stop 2024-02-15T18:28:35.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:36.001+0000 7f8122ed4700 1 -- 172.21.15.38:0/152591261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f811c0fb960 msgr2=0x7f811c0fbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:35.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:36.001+0000 7f8122ed4700 1 --2- 172.21.15.38:0/152591261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f811c0fb960 0x7f811c0fbde0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f81080093c0 tx=0x7f8108006ee0).stop 2024-02-15T18:28:35.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:36.001+0000 7f8122ed4700 1 -- 172.21.15.38:0/152591261 shutdown_connections 2024-02-15T18:28:35.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:36.001+0000 7f8122ed4700 1 --2- 172.21.15.38:0/152591261 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f810c05c400 0x7f810c05e8c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:35.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:36.001+0000 7f8122ed4700 1 --2- 172.21.15.38:0/152591261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f811c0f5410 0x7f811c0fd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:35.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:36.001+0000 7f8122ed4700 1 --2- 172.21.15.38:0/152591261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f811c0fb960 0x7f811c0fbde0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:35.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:36.001+0000 7f8122ed4700 1 -- 172.21.15.38:0/152591261 >> 172.21.15.38:0/152591261 conn(0x7f811c0f0b70 msgr2=0x7f811c05fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:35.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:36.001+0000 7f8122ed4700 1 -- 172.21.15.38:0/152591261 shutdown_connections 2024-02-15T18:28:35.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:36.001+0000 7f8122ed4700 1 -- 172.21.15.38:0/152591261 wait complete. 2024-02-15T18:28:36.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:36.000801+0000 mon.smithi038 (mon.0) 189 : audit [DBG] from='client.? 172.21.15.38:0/152591261' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:28:36.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:36 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:36.000801+0000 mon.smithi038 (mon.0) 189 : audit [DBG] from='client.? 172.21.15.38:0/152591261' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:28:36.632 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-15T18:28:36.632 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-02-15T18:28:36.633 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.conf 2024-02-15T18:28:36.667 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-02-15T18:28:36.667 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-15T18:28:36.736 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-02-15T18:28:36.736 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.conf 2024-02-15T18:28:36.775 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-02-15T18:28:36.775 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-15T18:28:36.847 INFO:tasks.cephadm:Deploying OSDs... 2024-02-15T18:28:36.847 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-02-15T18:28:36.847 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/scratch_devs of=/dev/stdout 2024-02-15T18:28:36.866 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-15T18:28:36.867 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_1 2024-02-15T18:28:36.925 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-15T18:28:36.926 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-15T18:28:36.926 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 6h/6d Inode: 165470 Links: 1 2024-02-15T18:28:36.926 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-15T18:28:36.926 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-02-15T18:28:36.926 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-02-15 18:28:06.196362441 +0000 2024-02-15T18:28:36.926 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-02-15 18:20:21.480282548 +0000 2024-02-15T18:28:36.926 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-02-15 18:20:21.480282548 +0000 2024-02-15T18:28:36.926 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: - 2024-02-15T18:28:36.926 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-15T18:28:36.993 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-02-15T18:28:36.994 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-02-15T18:28:36.994 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000438562 s, 1.2 MB/s 2024-02-15T18:28:36.995 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-15T18:28:37.055 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_2 2024-02-15T18:28:37.114 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-15T18:28:37.115 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-15T18:28:37.115 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 6h/6d Inode: 165514 Links: 1 2024-02-15T18:28:37.115 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-15T18:28:37.115 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-02-15T18:28:37.115 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-02-15 18:28:06.196362441 +0000 2024-02-15T18:28:37.115 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-02-15 18:20:21.859275276 +0000 2024-02-15T18:28:37.115 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-02-15 18:20:21.859275276 +0000 2024-02-15T18:28:37.115 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: - 2024-02-15T18:28:37.115 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-15T18:28:37.182 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-02-15T18:28:37.182 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-02-15T18:28:37.182 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000402244 s, 1.3 MB/s 2024-02-15T18:28:37.184 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-15T18:28:37.244 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_3 2024-02-15T18:28:37.303 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-15T18:28:37.303 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-15T18:28:37.303 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 6h/6d Inode: 165603 Links: 1 2024-02-15T18:28:37.304 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-15T18:28:37.304 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-02-15T18:28:37.304 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-02-15 18:28:06.196362441 +0000 2024-02-15T18:28:37.304 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-02-15 18:20:22.205268638 +0000 2024-02-15T18:28:37.304 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-02-15 18:20:22.205268638 +0000 2024-02-15T18:28:37.304 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: - 2024-02-15T18:28:37.304 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-15T18:28:37.371 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-02-15T18:28:37.371 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-02-15T18:28:37.371 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000279933 s, 1.8 MB/s 2024-02-15T18:28:37.372 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-15T18:28:37.432 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_4 2024-02-15T18:28:37.491 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-15T18:28:37.492 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-15T18:28:37.492 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 6h/6d Inode: 168986 Links: 1 2024-02-15T18:28:37.492 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-15T18:28:37.492 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-02-15T18:28:37.492 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-02-15 18:28:06.196362441 +0000 2024-02-15T18:28:37.492 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-02-15 18:20:22.562261788 +0000 2024-02-15T18:28:37.492 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-02-15 18:20:22.562261788 +0000 2024-02-15T18:28:37.492 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: - 2024-02-15T18:28:37.492 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-15T18:28:37.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:36.231376+0000 mgr.smithi038.khesbl (mgr.14162) 68 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:37.553 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-02-15T18:28:37.553 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-02-15T18:28:37.553 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000280585 s, 1.8 MB/s 2024-02-15T18:28:37.554 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-15T18:28:37.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:36.231376+0000 mgr.smithi038.khesbl (mgr.14162) 68 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:37.614 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-02-15T18:28:37.614 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/scratch_devs of=/dev/stdout 2024-02-15T18:28:37.637 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-15T18:28:37.637 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_1 2024-02-15T18:28:37.699 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-15T18:28:37.699 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-15T18:28:37.699 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 169160 Links: 1 2024-02-15T18:28:37.699 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-15T18:28:37.699 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-02-15T18:28:37.699 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-02-15 18:28:33.018108097 +0000 2024-02-15T18:28:37.699 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-02-15 18:20:15.995651950 +0000 2024-02-15T18:28:37.699 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-02-15 18:20:15.995651950 +0000 2024-02-15T18:28:37.699 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2024-02-15T18:28:37.699 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-15T18:28:37.770 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-02-15T18:28:37.770 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-02-15T18:28:37.770 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000282121 s, 1.8 MB/s 2024-02-15T18:28:37.772 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-15T18:28:37.834 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_2 2024-02-15T18:28:37.894 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-15T18:28:37.895 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-15T18:28:37.895 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 170195 Links: 1 2024-02-15T18:28:37.895 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-15T18:28:37.895 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-02-15T18:28:37.895 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-02-15 18:28:33.018108097 +0000 2024-02-15T18:28:37.895 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-02-15 18:20:16.339647418 +0000 2024-02-15T18:28:37.895 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-02-15 18:20:16.339647418 +0000 2024-02-15T18:28:37.895 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2024-02-15T18:28:37.895 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-15T18:28:37.963 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-02-15T18:28:37.963 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-02-15T18:28:37.963 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000226463 s, 2.3 MB/s 2024-02-15T18:28:37.964 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-15T18:28:38.024 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_3 2024-02-15T18:28:38.082 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-15T18:28:38.082 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-15T18:28:38.082 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 168228 Links: 1 2024-02-15T18:28:38.082 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-15T18:28:38.083 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-02-15T18:28:38.083 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-02-15 18:28:33.018108097 +0000 2024-02-15T18:28:38.083 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-02-15 18:20:16.674643004 +0000 2024-02-15T18:28:38.083 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-02-15 18:20:16.674643004 +0000 2024-02-15T18:28:38.083 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2024-02-15T18:28:38.083 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-15T18:28:38.149 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-02-15T18:28:38.149 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-02-15T18:28:38.149 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000209361 s, 2.4 MB/s 2024-02-15T18:28:38.150 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-15T18:28:38.210 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_4 2024-02-15T18:28:38.269 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-15T18:28:38.269 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-15T18:28:38.269 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 168328 Links: 1 2024-02-15T18:28:38.269 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-15T18:28:38.269 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-02-15T18:28:38.269 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-02-15 18:28:33.018108097 +0000 2024-02-15T18:28:38.269 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-02-15 18:20:17.011638563 +0000 2024-02-15T18:28:38.270 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-02-15 18:20:17.011638563 +0000 2024-02-15T18:28:38.270 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2024-02-15T18:28:38.270 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-15T18:28:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-02-15T18:28:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-02-15T18:28:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000230035 s, 2.2 MB/s 2024-02-15T18:28:38.336 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-15T18:28:38.394 INFO:tasks.cephadm:Deploying osd.0 on smithi038 with /dev/vg_nvme/lv_4... 2024-02-15T18:28:38.395 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-15T18:28:39.861 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:38.231828+0000 mgr.smithi038.khesbl (mgr.14162) 69 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:39.947 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:38.231828+0000 mgr.smithi038.khesbl (mgr.14162) 69 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:40.759 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-15T18:28:40.759 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-15T18:28:40.759 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-02-15T18:28:40.759 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-02-15T18:28:40.759 INFO:teuthology.orchestra.run.smithi038.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0147676 s, 710 MB/s 2024-02-15T18:28:40.759 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-02-15T18:28:40.861 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:40.349772+0000 mon.smithi038 (mon.0) 190 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:28:40.861 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:40.352584+0000 mon.smithi038 (mon.0) 191 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:28:41.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:40.349772+0000 mon.smithi038 (mon.0) 190 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:28:41.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:40.352584+0000 mon.smithi038 (mon.0) 191 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:28:41.704 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi038:vg_nvme/lv_4 2024-02-15T18:28:41.861 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:40.232224+0000 mgr.smithi038.khesbl (mgr.14162) 70 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:42.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:40.232224+0000 mgr.smithi038.khesbl (mgr.14162) 70 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:43.353 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:42.118561+0000 mon.smithi038 (mon.0) 192 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:43.353 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:42.119310+0000 mon.smithi038 (mon.0) 193 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:28:43.353 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:42.232504+0000 mgr.smithi038.khesbl (mgr.14162) 71 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:43.433 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:42.118561+0000 mon.smithi038 (mon.0) 192 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:43.433 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:42.119310+0000 mon.smithi038 (mon.0) 193 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:28:43.433 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:42.232504+0000 mgr.smithi038.khesbl (mgr.14162) 71 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:43.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.644+0000 7f1de81f7700 1 -- 172.21.15.38:0/984946273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1de00f66e0 msgr2=0x7f1de00f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:43.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.644+0000 7f1de81f7700 1 --2- 172.21.15.38:0/984946273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1de00f66e0 0x7f1de00f6b00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f1ddc004660 tx=0x7f1ddc009b30).stop 2024-02-15T18:28:43.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.644+0000 7f1de81f7700 1 -- 172.21.15.38:0/984946273 shutdown_connections 2024-02-15T18:28:43.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.644+0000 7f1de81f7700 1 --2- 172.21.15.38:0/984946273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1de00f6ff0 0x7f1de00f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:43.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.644+0000 7f1de81f7700 1 --2- 172.21.15.38:0/984946273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1de00f66e0 0x7f1de00f6b00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:43.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.644+0000 7f1de81f7700 1 -- 172.21.15.38:0/984946273 >> 172.21.15.38:0/984946273 conn(0x7f1de00f1f00 msgr2=0x7f1de00f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:43.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.645+0000 7f1de81f7700 1 -- 172.21.15.38:0/984946273 shutdown_connections 2024-02-15T18:28:43.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.645+0000 7f1de81f7700 1 -- 172.21.15.38:0/984946273 wait complete. 2024-02-15T18:28:43.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.645+0000 7f1de81f7700 1 Processor -- start 2024-02-15T18:28:43.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.645+0000 7f1de81f7700 1 -- start start 2024-02-15T18:28:43.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.645+0000 7f1de81f7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1de00f66e0 0x7f1de0107040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:43.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.645+0000 7f1de81f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1de00f6ff0 0x7f1de0107530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:43.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.645+0000 7f1de81f7700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de0107ae0 con 0x7f1de00f66e0 2024-02-15T18:28:43.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.645+0000 7f1de81f7700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de0107c20 con 0x7f1de00f6ff0 2024-02-15T18:28:43.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.646+0000 7f1de5f93700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1de00f66e0 0x7f1de0107040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:43.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.646+0000 7f1de5f93700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1de00f66e0 0x7f1de0107040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53004/0 (socket says 172.21.15.38:53004) 2024-02-15T18:28:43.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.646+0000 7f1de5f93700 1 -- 172.21.15.38:0/3577116995 learned_addr learned my addr 172.21.15.38:0/3577116995 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:28:43.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.646+0000 7f1de5792700 1 --2- 172.21.15.38:0/3577116995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1de00f6ff0 0x7f1de0107530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:43.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.646+0000 7f1de5f93700 1 -- 172.21.15.38:0/3577116995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1de00f6ff0 msgr2=0x7f1de0107530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:43.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.646+0000 7f1de5f93700 1 --2- 172.21.15.38:0/3577116995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1de00f6ff0 0x7f1de0107530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:43.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.646+0000 7f1de5f93700 1 -- 172.21.15.38:0/3577116995 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ddc005040 con 0x7f1de00f66e0 2024-02-15T18:28:43.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.646+0000 7f1de5f93700 1 --2- 172.21.15.38:0/3577116995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1de00f66e0 0x7f1de0107040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f1ddc009730 tx=0x7f1ddc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:43.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.646+0000 7f1dd67fc700 1 -- 172.21.15.38:0/3577116995 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ddc018070 con 0x7f1de00f66e0 2024-02-15T18:28:43.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.646+0000 7f1dd67fc700 1 -- 172.21.15.38:0/3577116995 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ddc009760 con 0x7f1de00f66e0 2024-02-15T18:28:43.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.646+0000 7f1dd67fc700 1 -- 172.21.15.38:0/3577116995 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ddc01c620 con 0x7f1de00f66e0 2024-02-15T18:28:43.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.646+0000 7f1de81f7700 1 -- 172.21.15.38:0/3577116995 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de0108560 con 0x7f1de00f66e0 2024-02-15T18:28:43.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.647+0000 7f1de81f7700 1 -- 172.21.15.38:0/3577116995 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de0108a70 con 0x7f1de00f66e0 2024-02-15T18:28:43.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.648+0000 7f1de81f7700 1 -- 172.21.15.38:0/3577116995 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de0045ad0 con 0x7f1de00f66e0 2024-02-15T18:28:43.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.648+0000 7f1dd67fc700 1 -- 172.21.15.38:0/3577116995 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f1ddc01d420 con 0x7f1de00f66e0 2024-02-15T18:28:43.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.648+0000 7f1dd67fc700 1 --2- 172.21.15.38:0/3577116995 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1dcc05c2f0 0x7f1dcc05e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:43.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.649+0000 7f1dd67fc700 1 -- 172.21.15.38:0/3577116995 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f1ddc077000 con 0x7f1de00f66e0 2024-02-15T18:28:43.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.649+0000 7f1de5792700 1 --2- 172.21.15.38:0/3577116995 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1dcc05c2f0 0x7f1dcc05e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:43.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.649+0000 7f1de5792700 1 --2- 172.21.15.38:0/3577116995 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1dcc05c2f0 0x7f1dcc05e7b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f1dd0007fd0 tx=0x7f1dd0009040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:43.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.653+0000 7f1dd67fc700 1 -- 172.21.15.38:0/3577116995 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f1ddc0a32b0 con 0x7f1de00f66e0 2024-02-15T18:28:43.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:28:43.847+0000 7f1de81f7700 1 -- 172.21.15.38:0/3577116995 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1de00ff0f0 con 0x7f1dcc05c2f0 2024-02-15T18:28:44.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:43.850026+0000 mgr.smithi038.khesbl (mgr.14162) 72 : audit [DBG] from='client.14222 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:28:44.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:43.850807+0000 mon.smithi038 (mon.0) 194 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:28:44.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:28:43.852030+0000 mgr.smithi038.khesbl (mgr.14162) 73 : cephadm [INF] Applying service osd.None on host smithi038... 2024-02-15T18:28:44.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:43.852273+0000 mon.smithi038 (mon.0) 195 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:28:44.278 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:43.852856+0000 mon.smithi038 (mon.0) 196 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:28:44.278 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:43.893318+0000 mon.smithi038 (mon.0) 197 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:44.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:43.850026+0000 mgr.smithi038.khesbl (mgr.14162) 72 : audit [DBG] from='client.14222 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:28:44.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:43.850807+0000 mon.smithi038 (mon.0) 194 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:28:44.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:28:43.852030+0000 mgr.smithi038.khesbl (mgr.14162) 73 : cephadm [INF] Applying service osd.None on host smithi038... 2024-02-15T18:28:44.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:43.852273+0000 mon.smithi038 (mon.0) 195 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:28:44.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:43.852856+0000 mon.smithi038 (mon.0) 196 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:28:44.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:43.893318+0000 mon.smithi038 (mon.0) 197 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:45.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:44.232724+0000 mgr.smithi038.khesbl (mgr.14162) 74 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:45.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:44.232724+0000 mgr.smithi038.khesbl (mgr.14162) 74 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:47.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:46.232914+0000 mgr.smithi038.khesbl (mgr.14162) 75 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:47.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:47.087238+0000 mon.smithi038 (mon.0) 198 : audit [INF] from='client.? 172.21.15.38:0/566621200' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eee9713b-ff79-4169-8c04-b1def80d2045"}]: dispatch 2024-02-15T18:28:47.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:47.092750+0000 mon.smithi038 (mon.0) 199 : audit [INF] from='client.? 172.21.15.38:0/566621200' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eee9713b-ff79-4169-8c04-b1def80d2045"}]': finished 2024-02-15T18:28:47.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:47.092811+0000 mon.smithi038 (mon.0) 200 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-15T18:28:47.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:47.092940+0000 mon.smithi038 (mon.0) 201 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:28:47.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:46.232914+0000 mgr.smithi038.khesbl (mgr.14162) 75 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:47.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:47.087238+0000 mon.smithi038 (mon.0) 198 : audit [INF] from='client.? 172.21.15.38:0/566621200' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eee9713b-ff79-4169-8c04-b1def80d2045"}]: dispatch 2024-02-15T18:28:47.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:47.092750+0000 mon.smithi038 (mon.0) 199 : audit [INF] from='client.? 172.21.15.38:0/566621200' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eee9713b-ff79-4169-8c04-b1def80d2045"}]': finished 2024-02-15T18:28:47.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:47.092811+0000 mon.smithi038 (mon.0) 200 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-15T18:28:47.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:47.092940+0000 mon.smithi038 (mon.0) 201 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:28:48.984 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:47.554843+0000 mon.smithi038 (mon.0) 202 : audit [DBG] from='client.? 172.21.15.38:0/1129477573' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:28:49.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:47.554843+0000 mon.smithi038 (mon.0) 202 : audit [DBG] from='client.? 172.21.15.38:0/1129477573' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:28:50.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:48.233217+0000 mgr.smithi038.khesbl (mgr.14162) 76 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:50.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:49.517871+0000 mon.smithi038 (mon.0) 203 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:50.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:49.522551+0000 mon.smithi038 (mon.0) 204 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:50.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:49.524449+0000 mon.smithi038 (mon.0) 205 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-15T18:28:50.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:49.525020+0000 mon.smithi038 (mon.0) 206 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-15T18:28:50.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:49.525540+0000 mon.smithi038 (mon.0) 207 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:28:50.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:48.233217+0000 mgr.smithi038.khesbl (mgr.14162) 76 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:50.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:49.517871+0000 mon.smithi038 (mon.0) 203 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:50.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:49.522551+0000 mon.smithi038 (mon.0) 204 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:50.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:49.524449+0000 mon.smithi038 (mon.0) 205 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-15T18:28:50.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:49.525020+0000 mon.smithi038 (mon.0) 206 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-15T18:28:50.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:49.525540+0000 mon.smithi038 (mon.0) 207 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:28:51.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:28:49.524205+0000 mgr.smithi038.khesbl (mgr.14162) 77 : cephadm [INF] Reconfiguring mon.smithi038 (unknown last config time)... 2024-02-15T18:28:51.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:28:49.525978+0000 mgr.smithi038.khesbl (mgr.14162) 78 : cephadm [INF] Reconfiguring daemon mon.smithi038 on smithi038 2024-02-15T18:28:51.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:28:49.524205+0000 mgr.smithi038.khesbl (mgr.14162) 77 : cephadm [INF] Reconfiguring mon.smithi038 (unknown last config time)... 2024-02-15T18:28:51.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:28:49.525978+0000 mgr.smithi038.khesbl (mgr.14162) 78 : cephadm [INF] Reconfiguring daemon mon.smithi038 on smithi038 2024-02-15T18:28:52.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:50.233503+0000 mgr.smithi038.khesbl (mgr.14162) 79 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:52.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:50.233503+0000 mgr.smithi038.khesbl (mgr.14162) 79 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:54.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:52.233994+0000 mgr.smithi038.khesbl (mgr.14162) 80 : cluster [DBG] pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:54.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:52.233994+0000 mgr.smithi038.khesbl (mgr.14162) 80 : cluster [DBG] pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:56.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:54.234302+0000 mgr.smithi038.khesbl (mgr.14162) 81 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:56.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:54.977459+0000 mon.smithi038 (mon.0) 208 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:56.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:54.978263+0000 mon.smithi038 (mon.0) 209 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi038.khesbl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:28:56.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:54.978977+0000 mon.smithi038 (mon.0) 210 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:28:56.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:54.979588+0000 mon.smithi038 (mon.0) 211 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:28:56.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:54.234302+0000 mgr.smithi038.khesbl (mgr.14162) 81 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:56.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:54.977459+0000 mon.smithi038 (mon.0) 208 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:56.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:54.978263+0000 mon.smithi038 (mon.0) 209 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi038.khesbl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:28:56.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:54.978977+0000 mon.smithi038 (mon.0) 210 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:28:56.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:54.979588+0000 mon.smithi038 (mon.0) 211 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:28:56.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:28:54.977987+0000 mgr.smithi038.khesbl (mgr.14162) 82 : cephadm [INF] Reconfiguring mgr.smithi038.khesbl (unknown last config time)... 2024-02-15T18:28:56.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:28:54.980102+0000 mgr.smithi038.khesbl (mgr.14162) 83 : cephadm [INF] Reconfiguring daemon mgr.smithi038.khesbl on smithi038 2024-02-15T18:28:56.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:56.561622+0000 mon.smithi038 (mon.0) 212 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-15T18:28:56.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:56.562379+0000 mon.smithi038 (mon.0) 213 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:28:57.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:28:54.977987+0000 mgr.smithi038.khesbl (mgr.14162) 82 : cephadm [INF] Reconfiguring mgr.smithi038.khesbl (unknown last config time)... 2024-02-15T18:28:57.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:28:54.980102+0000 mgr.smithi038.khesbl (mgr.14162) 83 : cephadm [INF] Reconfiguring daemon mgr.smithi038.khesbl on smithi038 2024-02-15T18:28:57.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:56.561622+0000 mon.smithi038 (mon.0) 212 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-15T18:28:57.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:56.562379+0000 mon.smithi038 (mon.0) 213 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:28:57.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:56.234606+0000 mgr.smithi038.khesbl (mgr.14162) 84 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:57.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:28:56.562991+0000 mgr.smithi038.khesbl (mgr.14162) 85 : cephadm [INF] Deploying daemon osd.0 on smithi038 2024-02-15T18:28:58.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:56.234606+0000 mgr.smithi038.khesbl (mgr.14162) 84 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:58.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:28:56.562991+0000 mgr.smithi038.khesbl (mgr.14162) 85 : cephadm [INF] Deploying daemon osd.0 on smithi038 2024-02-15T18:28:59.223 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:28:57.872888+0000 mon.smithi038 (mon.0) 214 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:59.223 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:28:57.873592+0000 mgr.smithi038.khesbl (mgr.14162) 86 : cephadm [INF] Reconfiguring alertmanager.smithi038 (dependencies changed)... 2024-02-15T18:28:59.223 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:28:57.875752+0000 mgr.smithi038.khesbl (mgr.14162) 87 : cephadm [INF] Reconfiguring daemon alertmanager.smithi038 on smithi038 2024-02-15T18:28:59.223 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:28:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:28:58.234820+0000 mgr.smithi038.khesbl (mgr.14162) 88 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:28:59.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:28:57.872888+0000 mon.smithi038 (mon.0) 214 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:28:59.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:28:57.873592+0000 mgr.smithi038.khesbl (mgr.14162) 86 : cephadm [INF] Reconfiguring alertmanager.smithi038 (dependencies changed)... 2024-02-15T18:28:59.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:28:57.875752+0000 mgr.smithi038.khesbl (mgr.14162) 87 : cephadm [INF] Reconfiguring daemon alertmanager.smithi038 on smithi038 2024-02-15T18:28:59.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:28:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:28:58.234820+0000 mgr.smithi038.khesbl (mgr.14162) 88 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:29:01.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:00.235037+0000 mgr.smithi038.khesbl (mgr.14162) 89 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:29:01.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:00.235037+0000 mgr.smithi038.khesbl (mgr.14162) 89 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:29:01.706 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 0 on host 'smithi038' 2024-02-15T18:29:01.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:01.710+0000 7f1dd67fc700 1 -- 172.21.15.38:0/3577116995 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1de00ff0f0 con 0x7f1dcc05c2f0 2024-02-15T18:29:01.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:01.713+0000 7f1de81f7700 1 -- 172.21.15.38:0/3577116995 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1dcc05c2f0 msgr2=0x7f1dcc05e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:01.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:01.713+0000 7f1de81f7700 1 --2- 172.21.15.38:0/3577116995 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1dcc05c2f0 0x7f1dcc05e7b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f1dd0007fd0 tx=0x7f1dd0009040).stop 2024-02-15T18:29:01.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:01.713+0000 7f1de81f7700 1 -- 172.21.15.38:0/3577116995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1de00f66e0 msgr2=0x7f1de0107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:01.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:01.713+0000 7f1de81f7700 1 --2- 172.21.15.38:0/3577116995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1de00f66e0 0x7f1de0107040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f1ddc009730 tx=0x7f1ddc009b30).stop 2024-02-15T18:29:01.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:01.713+0000 7f1de81f7700 1 -- 172.21.15.38:0/3577116995 shutdown_connections 2024-02-15T18:29:01.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:01.713+0000 7f1de81f7700 1 --2- 172.21.15.38:0/3577116995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1de00f66e0 0x7f1de0107040 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:01.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:01.713+0000 7f1de81f7700 1 --2- 172.21.15.38:0/3577116995 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1dcc05c2f0 0x7f1dcc05e7b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:01.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:01.713+0000 7f1de81f7700 1 --2- 172.21.15.38:0/3577116995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1de00f6ff0 0x7f1de0107530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:01.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:01.713+0000 7f1de81f7700 1 -- 172.21.15.38:0/3577116995 >> 172.21.15.38:0/3577116995 conn(0x7f1de00f1f00 msgr2=0x7f1de00fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:01.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:01.713+0000 7f1de81f7700 1 -- 172.21.15.38:0/3577116995 shutdown_connections 2024-02-15T18:29:01.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:01.713+0000 7f1de81f7700 1 -- 172.21.15.38:0/3577116995 wait complete. 2024-02-15T18:29:02.814 DEBUG:teuthology.orchestra.run.smithi038:osd.0> sudo journalctl -f -n 0 -u ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.0.service 2024-02-15T18:29:02.817 INFO:tasks.cephadm:Deploying osd.1 on smithi038 with /dev/vg_nvme/lv_3... 2024-02-15T18:29:02.817 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-15T18:29:02.850 INFO:journalctl@ceph.osd.0.smithi038.stdout:-- Logs begin at Thu 2024-02-15 18:12:54 UTC. -- 2024-02-15T18:29:03.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:01.700945+0000 mon.smithi038 (mon.0) 215 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:03.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:01.709916+0000 mon.smithi038 (mon.0) 216 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:03.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:01.700945+0000 mon.smithi038 (mon.0) 215 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:03.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:01.709916+0000 mon.smithi038 (mon.0) 216 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:04.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:02.235399+0000 mgr.smithi038.khesbl (mgr.14162) 90 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:29:04.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:02.235399+0000 mgr.smithi038.khesbl (mgr.14162) 90 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:29:05.936 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:04.235754+0000 mgr.smithi038.khesbl (mgr.14162) 91 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:29:05.936 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:05.566563+0000 mon.smithi038 (mon.0) 217 : audit [INF] from='osd.0 [v2:172.21.15.38:6802/1826650694,v1:172.21.15.38:6803/1826650694]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-15T18:29:05.937 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:29:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.0[48414]: debug 2024-02-15T18:29:05.562+0000 7f2eb1ec6080 -1 osd.0 0 log_to_monitors {default=true} 2024-02-15T18:29:06.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:04.235754+0000 mgr.smithi038.khesbl (mgr.14162) 91 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:29:06.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:05.566563+0000 mon.smithi038 (mon.0) 217 : audit [INF] from='osd.0 [v2:172.21.15.38:6802/1826650694,v1:172.21.15.38:6803/1826650694]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-15T18:29:06.972 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:29:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.0[48414]: debug 2024-02-15T18:29:06.721+0000 7f2ea8437700 -1 osd.0 0 waiting for initial osdmap 2024-02-15T18:29:06.972 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:29:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.0[48414]: debug 2024-02-15T18:29:06.739+0000 7f2ea2baf700 -1 osd.0 8 set_numa_affinity unable to identify public interface 'ens1f0' numa node: (0) Success 2024-02-15T18:29:06.972 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:05.722649+0000 mon.smithi038 (mon.0) 218 : audit [INF] from='osd.0 [v2:172.21.15.38:6802/1826650694,v1:172.21.15.38:6803/1826650694]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-15T18:29:06.972 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:05.722700+0000 mon.smithi038 (mon.0) 219 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-15T18:29:06.973 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:05.723009+0000 mon.smithi038 (mon.0) 220 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:29:06.973 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:05.723341+0000 mon.smithi038 (mon.0) 221 : audit [INF] from='osd.0 [v2:172.21.15.38:6802/1826650694,v1:172.21.15.38:6803/1826650694]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-02-15T18:29:06.973 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:06.719228+0000 mon.smithi038 (mon.0) 222 : audit [INF] from='osd.0 [v2:172.21.15.38:6802/1826650694,v1:172.21.15.38:6803/1826650694]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-02-15T18:29:06.973 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:06.719284+0000 mon.smithi038 (mon.0) 223 : cluster [DBG] osdmap e8: 1 total, 0 up, 1 in 2024-02-15T18:29:06.973 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:06.719405+0000 mon.smithi038 (mon.0) 224 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:29:06.973 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:06.722463+0000 mon.smithi038 (mon.0) 225 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:29:07.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:05.722649+0000 mon.smithi038 (mon.0) 218 : audit [INF] from='osd.0 [v2:172.21.15.38:6802/1826650694,v1:172.21.15.38:6803/1826650694]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-15T18:29:07.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:05.722700+0000 mon.smithi038 (mon.0) 219 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-15T18:29:07.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:05.723009+0000 mon.smithi038 (mon.0) 220 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:29:07.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:05.723341+0000 mon.smithi038 (mon.0) 221 : audit [INF] from='osd.0 [v2:172.21.15.38:6802/1826650694,v1:172.21.15.38:6803/1826650694]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-02-15T18:29:07.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:06.719228+0000 mon.smithi038 (mon.0) 222 : audit [INF] from='osd.0 [v2:172.21.15.38:6802/1826650694,v1:172.21.15.38:6803/1826650694]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-02-15T18:29:07.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:06.719284+0000 mon.smithi038 (mon.0) 223 : cluster [DBG] osdmap e8: 1 total, 0 up, 1 in 2024-02-15T18:29:07.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:06.719405+0000 mon.smithi038 (mon.0) 224 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:29:07.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:06.722463+0000 mon.smithi038 (mon.0) 225 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:29:08.001 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:06.236101+0000 mgr.smithi038.khesbl (mgr.14162) 92 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:29:08.001 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:07.722585+0000 mon.smithi038 (mon.0) 226 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:29:08.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:06.236101+0000 mgr.smithi038.khesbl (mgr.14162) 92 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:29:08.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:07.722585+0000 mon.smithi038 (mon.0) 226 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:29:08.727 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-15T18:29:08.727 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-15T18:29:08.727 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-02-15T18:29:08.727 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-02-15T18:29:08.727 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0172698 s, 607 MB/s 2024-02-15T18:29:08.727 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-02-15T18:29:09.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:06.517879+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:29:09.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:06.518104+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:29:09.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:07.743863+0000 mon.smithi038 (mon.0) 227 : cluster [INF] osd.0 [v2:172.21.15.38:6802/1826650694,v1:172.21.15.38:6803/1826650694] boot 2024-02-15T18:29:09.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:07.743910+0000 mon.smithi038 (mon.0) 228 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-15T18:29:09.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:07.746251+0000 mon.smithi038 (mon.0) 229 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:29:09.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:06.517879+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:29:09.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:06.518104+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:29:09.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:07.743863+0000 mon.smithi038 (mon.0) 227 : cluster [INF] osd.0 [v2:172.21.15.38:6802/1826650694,v1:172.21.15.38:6803/1826650694] boot 2024-02-15T18:29:09.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:07.743910+0000 mon.smithi038 (mon.0) 228 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-15T18:29:09.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:07.746251+0000 mon.smithi038 (mon.0) 229 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:29:09.368 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi038:vg_nvme/lv_3 2024-02-15T18:29:09.905 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:08.236335+0000 mgr.smithi038.khesbl (mgr.14162) 93 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:29:09.905 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:08.758859+0000 mon.smithi038 (mon.0) 230 : cluster [DBG] osdmap e10: 1 total, 1 up, 1 in 2024-02-15T18:29:09.905 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:09.667074+0000 mon.smithi038 (mon.0) 231 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:09.905 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:09.668010+0000 mon.smithi038 (mon.0) 232 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi038", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-15T18:29:09.905 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:09.668826+0000 mon.smithi038 (mon.0) 233 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:10.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:08.236335+0000 mgr.smithi038.khesbl (mgr.14162) 93 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:29:10.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:08.758859+0000 mon.smithi038 (mon.0) 230 : cluster [DBG] osdmap e10: 1 total, 1 up, 1 in 2024-02-15T18:29:10.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:09.667074+0000 mon.smithi038 (mon.0) 231 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:10.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:09.668010+0000 mon.smithi038 (mon.0) 232 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi038", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-15T18:29:10.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:09.668826+0000 mon.smithi038 (mon.0) 233 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:11.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:29:09.667695+0000 mgr.smithi038.khesbl (mgr.14162) 94 : cephadm [INF] Reconfiguring crash.smithi038 (monmap changed)... 2024-02-15T18:29:11.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:29:09.669601+0000 mgr.smithi038.khesbl (mgr.14162) 95 : cephadm [INF] Reconfiguring daemon crash.smithi038 on smithi038 2024-02-15T18:29:11.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:29:09.667695+0000 mgr.smithi038.khesbl (mgr.14162) 94 : cephadm [INF] Reconfiguring crash.smithi038 (monmap changed)... 2024-02-15T18:29:11.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:29:09.669601+0000 mgr.smithi038.khesbl (mgr.14162) 95 : cephadm [INF] Reconfiguring daemon crash.smithi038 on smithi038 2024-02-15T18:29:12.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:10.236593+0000 mgr.smithi038.khesbl (mgr.14162) 96 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:12.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:10.236593+0000 mgr.smithi038.khesbl (mgr.14162) 96 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:13.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.948+0000 7f17e9306700 1 -- 172.21.15.38:0/1519932302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17e40f6ff0 msgr2=0x7f17e40f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:13.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.948+0000 7f17e9306700 1 --2- 172.21.15.38:0/1519932302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17e40f6ff0 0x7f17e40f7a20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f17cc004660 tx=0x7f17cc009b30).stop 2024-02-15T18:29:13.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.948+0000 7f17e9306700 1 -- 172.21.15.38:0/1519932302 shutdown_connections 2024-02-15T18:29:13.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.948+0000 7f17e9306700 1 --2- 172.21.15.38:0/1519932302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17e40f6ff0 0x7f17e40f7a20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:13.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.948+0000 7f17e9306700 1 --2- 172.21.15.38:0/1519932302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17e40f66e0 0x7f17e40f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:13.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.948+0000 7f17e9306700 1 -- 172.21.15.38:0/1519932302 >> 172.21.15.38:0/1519932302 conn(0x7f17e40f1f00 msgr2=0x7f17e40f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:13.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.948+0000 7f17e9306700 1 -- 172.21.15.38:0/1519932302 shutdown_connections 2024-02-15T18:29:13.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.948+0000 7f17e9306700 1 -- 172.21.15.38:0/1519932302 wait complete. 2024-02-15T18:29:13.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.949+0000 7f17e9306700 1 Processor -- start 2024-02-15T18:29:13.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.949+0000 7f17e9306700 1 -- start start 2024-02-15T18:29:13.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.949+0000 7f17e9306700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17e40f66e0 0x7f17e4107050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:13.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.949+0000 7f17e9306700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17e40f6ff0 0x7f17e4107540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:13.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.949+0000 7f17e9306700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e4107af0 con 0x7f17e40f66e0 2024-02-15T18:29:13.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.949+0000 7f17e9306700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e4107c30 con 0x7f17e40f6ff0 2024-02-15T18:29:13.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.949+0000 7f17e2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17e40f66e0 0x7f17e4107050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:13.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.949+0000 7f17e2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17e40f66e0 0x7f17e4107050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52116/0 (socket says 172.21.15.38:52116) 2024-02-15T18:29:13.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.949+0000 7f17e2ffd700 1 -- 172.21.15.38:0/1707579688 learned_addr learned my addr 172.21.15.38:0/1707579688 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:29:13.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.949+0000 7f17e27fc700 1 --2- 172.21.15.38:0/1707579688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17e40f6ff0 0x7f17e4107540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:13.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.950+0000 7f17e2ffd700 1 -- 172.21.15.38:0/1707579688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17e40f6ff0 msgr2=0x7f17e4107540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:13.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.950+0000 7f17e2ffd700 1 --2- 172.21.15.38:0/1707579688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17e40f6ff0 0x7f17e4107540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:13.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.950+0000 7f17e2ffd700 1 -- 172.21.15.38:0/1707579688 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc005040 con 0x7f17e40f66e0 2024-02-15T18:29:13.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.950+0000 7f17e2ffd700 1 --2- 172.21.15.38:0/1707579688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17e40f66e0 0x7f17e4107050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f17d400c790 tx=0x7f17d4007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:13.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.950+0000 7f17db7fe700 1 -- 172.21.15.38:0/1707579688 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17d400e680 con 0x7f17e40f66e0 2024-02-15T18:29:13.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.950+0000 7f17db7fe700 1 -- 172.21.15.38:0/1707579688 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17d400ecb0 con 0x7f17e40f66e0 2024-02-15T18:29:13.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.950+0000 7f17e9306700 1 -- 172.21.15.38:0/1707579688 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e41085d0 con 0x7f17e40f66e0 2024-02-15T18:29:13.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.950+0000 7f17db7fe700 1 -- 172.21.15.38:0/1707579688 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17d4013e10 con 0x7f17e40f66e0 2024-02-15T18:29:13.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.950+0000 7f17e9306700 1 -- 172.21.15.38:0/1707579688 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e4108b90 con 0x7f17e40f66e0 2024-02-15T18:29:13.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.951+0000 7f17e9306700 1 -- 172.21.15.38:0/1707579688 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e4045ad0 con 0x7f17e40f66e0 2024-02-15T18:29:13.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.951+0000 7f17db7fe700 1 -- 172.21.15.38:0/1707579688 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f17d401a060 con 0x7f17e40f66e0 2024-02-15T18:29:13.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.951+0000 7f17db7fe700 1 --2- 172.21.15.38:0/1707579688 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f17d005c2b0 0x7f17d005e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:13.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.951+0000 7f17db7fe700 1 -- 172.21.15.38:0/1707579688 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1882+0+0 (secure 0 0 0) 0x7f17d4075390 con 0x7f17e40f66e0 2024-02-15T18:29:13.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.952+0000 7f17e27fc700 1 --2- 172.21.15.38:0/1707579688 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f17d005c2b0 0x7f17d005e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:13.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.952+0000 7f17e27fc700 1 --2- 172.21.15.38:0/1707579688 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f17d005c2b0 0x7f17d005e770 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f17cc009d60 tx=0x7f17cc009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:13.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:12.956+0000 7f17db7fe700 1 -- 172.21.15.38:0/1707579688 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f17d4048e00 con 0x7f17e40f66e0 2024-02-15T18:29:13.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:13.141+0000 7f17e9306700 1 -- 172.21.15.38:0/1707579688 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f17e40ff0f0 con 0x7f17d005c2b0 2024-02-15T18:29:13.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:12.218032+0000 mon.smithi038 (mon.0) 234 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:13.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:29:12.218667+0000 mgr.smithi038.khesbl (mgr.14162) 97 : cephadm [INF] Reconfiguring grafana.smithi038 (dependencies changed)... 2024-02-15T18:29:13.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:29:12.222875+0000 mgr.smithi038.khesbl (mgr.14162) 98 : cephadm [INF] Reconfiguring daemon grafana.smithi038 on smithi038 2024-02-15T18:29:13.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:12.236883+0000 mgr.smithi038.khesbl (mgr.14162) 99 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:13.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:13.143074+0000 mon.smithi038 (mon.0) 235 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:29:13.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:13.144350+0000 mon.smithi038 (mon.0) 236 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:29:13.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:13.144878+0000 mon.smithi038 (mon.0) 237 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:13.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:12.218032+0000 mon.smithi038 (mon.0) 234 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:13.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:29:12.218667+0000 mgr.smithi038.khesbl (mgr.14162) 97 : cephadm [INF] Reconfiguring grafana.smithi038 (dependencies changed)... 2024-02-15T18:29:13.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:29:12.222875+0000 mgr.smithi038.khesbl (mgr.14162) 98 : cephadm [INF] Reconfiguring daemon grafana.smithi038 on smithi038 2024-02-15T18:29:13.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:12.236883+0000 mgr.smithi038.khesbl (mgr.14162) 99 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:13.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:13.143074+0000 mon.smithi038 (mon.0) 235 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:29:13.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:13.144350+0000 mon.smithi038 (mon.0) 236 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:29:13.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:13.144878+0000 mon.smithi038 (mon.0) 237 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:14.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:13.142287+0000 mgr.smithi038.khesbl (mgr.14162) 100 : audit [DBG] from='client.14240 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:29:14.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:29:13.144136+0000 mgr.smithi038.khesbl (mgr.14162) 101 : cephadm [INF] Applying service osd.None on host smithi038... 2024-02-15T18:29:14.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:13.142287+0000 mgr.smithi038.khesbl (mgr.14162) 100 : audit [DBG] from='client.14240 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:29:14.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:29:13.144136+0000 mgr.smithi038.khesbl (mgr.14162) 101 : cephadm [INF] Applying service osd.None on host smithi038... 2024-02-15T18:29:15.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:14.237154+0000 mgr.smithi038.khesbl (mgr.14162) 102 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:15.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:14.237154+0000 mgr.smithi038.khesbl (mgr.14162) 102 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:17.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:16.237415+0000 mgr.smithi038.khesbl (mgr.14162) 103 : cluster [DBG] pgmap v77: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:17.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:16.237415+0000 mgr.smithi038.khesbl (mgr.14162) 103 : cluster [DBG] pgmap v77: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:19.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:17.731606+0000 mon.smithi038 (mon.0) 238 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:19.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:29:17.732296+0000 mgr.smithi038.khesbl (mgr.14162) 104 : cephadm [INF] Reconfiguring prometheus.smithi038 (dependencies changed)... 2024-02-15T18:29:19.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:29:17.734503+0000 mgr.smithi038.khesbl (mgr.14162) 105 : cephadm [INF] Reconfiguring daemon prometheus.smithi038 on smithi038 2024-02-15T18:29:19.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:17.731606+0000 mon.smithi038 (mon.0) 238 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:19.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:29:17.732296+0000 mgr.smithi038.khesbl (mgr.14162) 104 : cephadm [INF] Reconfiguring prometheus.smithi038 (dependencies changed)... 2024-02-15T18:29:19.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:29:17.734503+0000 mgr.smithi038.khesbl (mgr.14162) 105 : cephadm [INF] Reconfiguring daemon prometheus.smithi038 on smithi038 2024-02-15T18:29:20.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:18.237633+0000 mgr.smithi038.khesbl (mgr.14162) 106 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:20.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:18.775408+0000 mon.smithi038 (mon.0) 239 : audit [INF] from='client.? 172.21.15.38:0/3393003832' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "740eed48-d34e-4e71-a859-035f01b1fdd1"}]: dispatch 2024-02-15T18:29:20.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:18.779011+0000 mon.smithi038 (mon.0) 240 : audit [INF] from='client.? 172.21.15.38:0/3393003832' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "740eed48-d34e-4e71-a859-035f01b1fdd1"}]': finished 2024-02-15T18:29:20.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:18.779058+0000 mon.smithi038 (mon.0) 241 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-15T18:29:20.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:18.779150+0000 mon.smithi038 (mon.0) 242 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:29:20.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:19.210086+0000 mon.smithi038 (mon.0) 243 : audit [DBG] from='client.? 172.21.15.38:0/1243208640' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:29:20.360 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:18.237633+0000 mgr.smithi038.khesbl (mgr.14162) 106 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:20.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:18.775408+0000 mon.smithi038 (mon.0) 239 : audit [INF] from='client.? 172.21.15.38:0/3393003832' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "740eed48-d34e-4e71-a859-035f01b1fdd1"}]: dispatch 2024-02-15T18:29:20.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:18.779011+0000 mon.smithi038 (mon.0) 240 : audit [INF] from='client.? 172.21.15.38:0/3393003832' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "740eed48-d34e-4e71-a859-035f01b1fdd1"}]': finished 2024-02-15T18:29:20.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:18.779058+0000 mon.smithi038 (mon.0) 241 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-15T18:29:20.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:18.779150+0000 mon.smithi038 (mon.0) 242 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:29:20.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:19.210086+0000 mon.smithi038 (mon.0) 243 : audit [DBG] from='client.? 172.21.15.38:0/1243208640' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:29:21.265 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:20 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:20.237922+0000 mgr.smithi038.khesbl (mgr.14162) 107 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:21.360 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:20 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:20.237922+0000 mgr.smithi038.khesbl (mgr.14162) 107 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:23.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:22.238275+0000 mgr.smithi038.khesbl (mgr.14162) 108 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:23.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:22.238275+0000 mgr.smithi038.khesbl (mgr.14162) 108 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:25.602 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:24.238593+0000 mgr.smithi038.khesbl (mgr.14162) 109 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:25.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:24.238593+0000 mgr.smithi038.khesbl (mgr.14162) 109 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:27.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:26.239008+0000 mgr.smithi038.khesbl (mgr.14162) 110 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:27.656 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:26.239008+0000 mgr.smithi038.khesbl (mgr.14162) 110 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:29.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:28.239331+0000 mgr.smithi038.khesbl (mgr.14162) 111 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:29.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:28.239331+0000 mgr.smithi038.khesbl (mgr.14162) 111 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:31.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:30.239620+0000 mgr.smithi038.khesbl (mgr.14162) 112 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:31.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:30.239943+0000 mon.smithi038 (mon.0) 244 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:31.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:29:30.240527+0000 mgr.smithi038.khesbl (mgr.14162) 113 : cephadm [INF] Reconfiguring crash.smithi121 (monmap changed)... 2024-02-15T18:29:31.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:30.240767+0000 mon.smithi038 (mon.0) 245 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi121", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-15T18:29:31.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:30.241443+0000 mon.smithi038 (mon.0) 246 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:31.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:29:30.241952+0000 mgr.smithi038.khesbl (mgr.14162) 114 : cephadm [INF] Reconfiguring daemon crash.smithi121 on smithi121 2024-02-15T18:29:31.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:30.239620+0000 mgr.smithi038.khesbl (mgr.14162) 112 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:31.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:30.239943+0000 mon.smithi038 (mon.0) 244 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:31.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:29:30.240527+0000 mgr.smithi038.khesbl (mgr.14162) 113 : cephadm [INF] Reconfiguring crash.smithi121 (monmap changed)... 2024-02-15T18:29:31.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:30.240767+0000 mon.smithi038 (mon.0) 245 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi121", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-15T18:29:31.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:30.241443+0000 mon.smithi038 (mon.0) 246 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:31.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:29:30.241952+0000 mgr.smithi038.khesbl (mgr.14162) 114 : cephadm [INF] Reconfiguring daemon crash.smithi121 on smithi121 2024-02-15T18:29:33.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:31.737847+0000 mon.smithi038 (mon.0) 247 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:33.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:29:31.738342+0000 mgr.smithi038.khesbl (mgr.14162) 115 : cephadm [INF] Reconfiguring mgr.smithi121.ckiryi (monmap changed)... 2024-02-15T18:29:33.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:31.738602+0000 mon.smithi038 (mon.0) 248 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi121.ckiryi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:29:33.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:31.739352+0000 mon.smithi038 (mon.0) 249 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:29:33.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:31.740000+0000 mon.smithi038 (mon.0) 250 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:33.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:29:31.740576+0000 mgr.smithi038.khesbl (mgr.14162) 116 : cephadm [INF] Reconfiguring daemon mgr.smithi121.ckiryi on smithi121 2024-02-15T18:29:33.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:31.793576+0000 mon.smithi038 (mon.0) 251 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-15T18:29:33.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:31.794199+0000 mon.smithi038 (mon.0) 252 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:33.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:29:31.794722+0000 mgr.smithi038.khesbl (mgr.14162) 117 : cephadm [INF] Deploying daemon osd.1 on smithi038 2024-02-15T18:29:33.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:31.737847+0000 mon.smithi038 (mon.0) 247 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:33.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:29:31.738342+0000 mgr.smithi038.khesbl (mgr.14162) 115 : cephadm [INF] Reconfiguring mgr.smithi121.ckiryi (monmap changed)... 2024-02-15T18:29:33.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:31.738602+0000 mon.smithi038 (mon.0) 248 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi121.ckiryi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:29:33.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:31.739352+0000 mon.smithi038 (mon.0) 249 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:29:33.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:31.740000+0000 mon.smithi038 (mon.0) 250 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:33.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:29:31.740576+0000 mgr.smithi038.khesbl (mgr.14162) 116 : cephadm [INF] Reconfiguring daemon mgr.smithi121.ckiryi on smithi121 2024-02-15T18:29:33.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:31.793576+0000 mon.smithi038 (mon.0) 251 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-15T18:29:33.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:31.794199+0000 mon.smithi038 (mon.0) 252 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:33.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:29:31.794722+0000 mgr.smithi038.khesbl (mgr.14162) 117 : cephadm [INF] Deploying daemon osd.1 on smithi038 2024-02-15T18:29:34.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:32.240050+0000 mgr.smithi038.khesbl (mgr.14162) 118 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:34.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:33.246381+0000 mon.smithi038 (mon.0) 253 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:34.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:33.247269+0000 mon.smithi038 (mon.0) 254 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-15T18:29:34.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:33.248032+0000 mon.smithi038 (mon.0) 255 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-15T18:29:34.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:33.248765+0000 mon.smithi038 (mon.0) 256 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:34.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:32.240050+0000 mgr.smithi038.khesbl (mgr.14162) 118 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:34.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:33.246381+0000 mon.smithi038 (mon.0) 253 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:34.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:33.247269+0000 mon.smithi038 (mon.0) 254 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-15T18:29:34.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:33.248032+0000 mon.smithi038 (mon.0) 255 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-15T18:29:34.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:33.248765+0000 mon.smithi038 (mon.0) 256 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:34.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:29:33.246980+0000 mgr.smithi038.khesbl (mgr.14162) 119 : cephadm [INF] Reconfiguring mon.smithi121 (monmap changed)... 2024-02-15T18:29:34.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:29:33.249371+0000 mgr.smithi038.khesbl (mgr.14162) 120 : cephadm [INF] Reconfiguring daemon mon.smithi121 on smithi121 2024-02-15T18:29:35.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:29:33.246980+0000 mgr.smithi038.khesbl (mgr.14162) 119 : cephadm [INF] Reconfiguring mon.smithi121 (monmap changed)... 2024-02-15T18:29:35.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:29:33.249371+0000 mgr.smithi038.khesbl (mgr.14162) 120 : cephadm [INF] Reconfiguring daemon mon.smithi121 on smithi121 2024-02-15T18:29:35.710 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 1 on host 'smithi038' 2024-02-15T18:29:35.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:35.714+0000 7f17db7fe700 1 -- 172.21.15.38:0/1707579688 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f17e40ff0f0 con 0x7f17d005c2b0 2024-02-15T18:29:35.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:35.717+0000 7f17e9306700 1 -- 172.21.15.38:0/1707579688 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f17d005c2b0 msgr2=0x7f17d005e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:35.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:35.717+0000 7f17e9306700 1 --2- 172.21.15.38:0/1707579688 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f17d005c2b0 0x7f17d005e770 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f17cc009d60 tx=0x7f17cc009b30).stop 2024-02-15T18:29:35.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:35.718+0000 7f17e9306700 1 -- 172.21.15.38:0/1707579688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17e40f66e0 msgr2=0x7f17e4107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:35.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:35.718+0000 7f17e9306700 1 --2- 172.21.15.38:0/1707579688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17e40f66e0 0x7f17e4107050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f17d400c790 tx=0x7f17d4007d70).stop 2024-02-15T18:29:35.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:35.718+0000 7f17e9306700 1 -- 172.21.15.38:0/1707579688 shutdown_connections 2024-02-15T18:29:35.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:35.718+0000 7f17e9306700 1 --2- 172.21.15.38:0/1707579688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f17e40f66e0 0x7f17e4107050 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:35.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:35.718+0000 7f17e9306700 1 --2- 172.21.15.38:0/1707579688 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f17d005c2b0 0x7f17d005e770 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:35.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:35.718+0000 7f17e9306700 1 --2- 172.21.15.38:0/1707579688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17e40f6ff0 0x7f17e4107540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:35.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:35.718+0000 7f17e9306700 1 -- 172.21.15.38:0/1707579688 >> 172.21.15.38:0/1707579688 conn(0x7f17e40f1f00 msgr2=0x7f17e40fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:35.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:35.718+0000 7f17e9306700 1 -- 172.21.15.38:0/1707579688 shutdown_connections 2024-02-15T18:29:35.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:35.718+0000 7f17e9306700 1 -- 172.21.15.38:0/1707579688 wait complete. 2024-02-15T18:29:36.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:34.240396+0000 mgr.smithi038.khesbl (mgr.14162) 121 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:36.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:34.765483+0000 mon.smithi038 (mon.0) 257 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:36.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:34.767819+0000 mon.smithi038 (mon.0) 258 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-15T18:29:36.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:34.768132+0000 mgr.smithi038.khesbl (mgr.14162) 122 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-15T18:29:36.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:35.711222+0000 mon.smithi038 (mon.0) 259 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:36.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:35.714973+0000 mon.smithi038 (mon.0) 260 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:36.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:35.733534+0000 mon.smithi038 (mon.0) 261 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.38:9093"}]: dispatch 2024-02-15T18:29:36.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:35.737658+0000 mon.smithi038 (mon.0) 262 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:36.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:35.740597+0000 mon.smithi038 (mon.0) 263 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-15T18:29:36.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:35.747565+0000 mon.smithi038 (mon.0) 264 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.38:3000"}]: dispatch 2024-02-15T18:29:36.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:35.751882+0000 mon.smithi038 (mon.0) 265 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:36.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:35.755269+0000 mon.smithi038 (mon.0) 266 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:29:36.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:35.759516+0000 mon.smithi038 (mon.0) 267 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.38:9095"}]: dispatch 2024-02-15T18:29:36.028 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:35.763307+0000 mon.smithi038 (mon.0) 268 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:36.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:34.240396+0000 mgr.smithi038.khesbl (mgr.14162) 121 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:36.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:34.765483+0000 mon.smithi038 (mon.0) 257 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:36.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:34.767819+0000 mon.smithi038 (mon.0) 258 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-15T18:29:36.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:34.768132+0000 mgr.smithi038.khesbl (mgr.14162) 122 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-15T18:29:36.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:35.711222+0000 mon.smithi038 (mon.0) 259 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:36.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:35.714973+0000 mon.smithi038 (mon.0) 260 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:36.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:35.733534+0000 mon.smithi038 (mon.0) 261 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.38:9093"}]: dispatch 2024-02-15T18:29:36.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:35.737658+0000 mon.smithi038 (mon.0) 262 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:36.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:35.740597+0000 mon.smithi038 (mon.0) 263 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-15T18:29:36.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:35.747565+0000 mon.smithi038 (mon.0) 264 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.38:3000"}]: dispatch 2024-02-15T18:29:36.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:35.751882+0000 mon.smithi038 (mon.0) 265 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:36.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:35.755269+0000 mon.smithi038 (mon.0) 266 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:29:36.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:35.759516+0000 mon.smithi038 (mon.0) 267 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.38:9095"}]: dispatch 2024-02-15T18:29:36.111 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:35.763307+0000 mon.smithi038 (mon.0) 268 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:36.495 DEBUG:teuthology.orchestra.run.smithi038:osd.1> sudo journalctl -f -n 0 -u ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.1.service 2024-02-15T18:29:36.498 INFO:tasks.cephadm:Deploying osd.2 on smithi038 with /dev/vg_nvme/lv_2... 2024-02-15T18:29:36.498 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-15T18:29:36.533 INFO:journalctl@ceph.osd.1.smithi038.stdout:-- Logs begin at Thu 2024-02-15 18:12:54 UTC. -- 2024-02-15T18:29:37.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:35.733834+0000 mgr.smithi038.khesbl (mgr.14162) 123 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.38:9093"}]: dispatch 2024-02-15T18:29:37.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:35.740949+0000 mgr.smithi038.khesbl (mgr.14162) 124 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-15T18:29:37.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:35.747888+0000 mgr.smithi038.khesbl (mgr.14162) 125 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.38:3000"}]: dispatch 2024-02-15T18:29:37.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:35.755734+0000 mgr.smithi038.khesbl (mgr.14162) 126 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:29:37.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:35.759782+0000 mgr.smithi038.khesbl (mgr.14162) 127 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.38:9095"}]: dispatch 2024-02-15T18:29:37.027 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:35.765555+0000 mon.smithi038 (mon.0) 269 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:29:37.071 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:36 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:35.733834+0000 mgr.smithi038.khesbl (mgr.14162) 123 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.38:9093"}]: dispatch 2024-02-15T18:29:37.071 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:36 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:35.740949+0000 mgr.smithi038.khesbl (mgr.14162) 124 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-15T18:29:37.071 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:36 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:35.747888+0000 mgr.smithi038.khesbl (mgr.14162) 125 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.38:3000"}]: dispatch 2024-02-15T18:29:37.072 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:36 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:35.755734+0000 mgr.smithi038.khesbl (mgr.14162) 126 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:29:37.072 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:36 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:35.759782+0000 mgr.smithi038.khesbl (mgr.14162) 127 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.38:9095"}]: dispatch 2024-02-15T18:29:37.072 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:36 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:35.765555+0000 mon.smithi038 (mon.0) 269 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:29:38.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:36.240745+0000 mgr.smithi038.khesbl (mgr.14162) 128 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:38.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:37.539782+0000 mon.smithi038 (mon.0) 270 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:38.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:36.240745+0000 mgr.smithi038.khesbl (mgr.14162) 128 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:38.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:37.539782+0000 mon.smithi038 (mon.0) 270 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:39.487 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-15T18:29:39.487 INFO:teuthology.orchestra.run.smithi038.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-15T18:29:39.487 INFO:teuthology.orchestra.run.smithi038.stderr: stderr: 10+0 records in 2024-02-15T18:29:39.487 INFO:teuthology.orchestra.run.smithi038.stderr:10+0 records out 2024-02-15T18:29:39.487 INFO:teuthology.orchestra.run.smithi038.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00918205 s, 1.1 GB/s 2024-02-15T18:29:39.487 INFO:teuthology.orchestra.run.smithi038.stderr:--> Zapping successful for: 2024-02-15T18:29:39.772 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:29:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.1[52470]: debug 2024-02-15T18:29:39.616+0000 7fb675935080 -1 osd.1 0 log_to_monitors {default=true} 2024-02-15T18:29:40.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:38.241085+0000 mgr.smithi038.khesbl (mgr.14162) 129 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:40.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:39.620588+0000 mon.smithi038 (mon.0) 271 : audit [INF] from='osd.1 [v2:172.21.15.38:6810/3796697403,v1:172.21.15.38:6811/3796697403]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-15T18:29:40.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:38.241085+0000 mgr.smithi038.khesbl (mgr.14162) 129 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:40.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:39.620588+0000 mon.smithi038 (mon.0) 271 : audit [INF] from='osd.1 [v2:172.21.15.38:6810/3796697403,v1:172.21.15.38:6811/3796697403]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-15T18:29:40.382 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi038:vg_nvme/lv_2 2024-02-15T18:29:41.519 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:29:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.1[52470]: debug 2024-02-15T18:29:41.265+0000 7fb66bea6700 -1 osd.1 0 waiting for initial osdmap 2024-02-15T18:29:41.519 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:29:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.1[52470]: debug 2024-02-15T18:29:41.271+0000 7fb666e1f700 -1 osd.1 13 set_numa_affinity unable to identify public interface 'ens1f0' numa node: (0) Success 2024-02-15T18:29:41.519 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:40.241462+0000 mgr.smithi038.khesbl (mgr.14162) 130 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:41.519 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:40.255286+0000 mon.smithi038 (mon.0) 272 : audit [INF] from='osd.1 [v2:172.21.15.38:6810/3796697403,v1:172.21.15.38:6811/3796697403]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-15T18:29:41.519 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:40.255364+0000 mon.smithi038 (mon.0) 273 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-15T18:29:41.519 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:40.255483+0000 mon.smithi038 (mon.0) 274 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:29:41.519 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:40.255809+0000 mon.smithi038 (mon.0) 275 : audit [INF] from='osd.1 [v2:172.21.15.38:6810/3796697403,v1:172.21.15.38:6811/3796697403]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-02-15T18:29:41.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:40.350781+0000 mon.smithi038 (mon.0) 276 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:29:41.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:40.353334+0000 mon.smithi038 (mon.0) 277 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:29:41.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:40.241462+0000 mgr.smithi038.khesbl (mgr.14162) 130 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:41.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:40.255286+0000 mon.smithi038 (mon.0) 272 : audit [INF] from='osd.1 [v2:172.21.15.38:6810/3796697403,v1:172.21.15.38:6811/3796697403]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-15T18:29:41.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:40.255364+0000 mon.smithi038 (mon.0) 273 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-15T18:29:41.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:40.255483+0000 mon.smithi038 (mon.0) 274 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:29:41.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:40.255809+0000 mon.smithi038 (mon.0) 275 : audit [INF] from='osd.1 [v2:172.21.15.38:6810/3796697403,v1:172.21.15.38:6811/3796697403]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-02-15T18:29:41.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:40.350781+0000 mon.smithi038 (mon.0) 276 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:29:41.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:40.353334+0000 mon.smithi038 (mon.0) 277 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:29:42.344 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:41.261034+0000 mon.smithi038 (mon.0) 278 : audit [INF] from='osd.1 [v2:172.21.15.38:6810/3796697403,v1:172.21.15.38:6811/3796697403]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-02-15T18:29:42.344 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:41.261108+0000 mon.smithi038 (mon.0) 279 : cluster [DBG] osdmap e13: 2 total, 1 up, 2 in 2024-02-15T18:29:42.344 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:41.263942+0000 mon.smithi038 (mon.0) 280 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:29:42.345 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:41.267994+0000 mon.smithi038 (mon.0) 281 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:29:42.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:41.261034+0000 mon.smithi038 (mon.0) 278 : audit [INF] from='osd.1 [v2:172.21.15.38:6810/3796697403,v1:172.21.15.38:6811/3796697403]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-02-15T18:29:42.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:41.261108+0000 mon.smithi038 (mon.0) 279 : cluster [DBG] osdmap e13: 2 total, 1 up, 2 in 2024-02-15T18:29:42.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:41.263942+0000 mon.smithi038 (mon.0) 280 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:29:42.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:41.267994+0000 mon.smithi038 (mon.0) 281 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:29:43.595 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:40.582098+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:29:43.596 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:40.582224+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:29:43.596 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:42.241797+0000 mgr.smithi038.khesbl (mgr.14162) 131 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:43.596 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:42.266605+0000 mon.smithi038 (mon.0) 282 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:29:43.596 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:42.278044+0000 mon.smithi038 (mon.0) 283 : cluster [INF] osd.1 [v2:172.21.15.38:6810/3796697403,v1:172.21.15.38:6811/3796697403] boot 2024-02-15T18:29:43.596 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:42.278085+0000 mon.smithi038 (mon.0) 284 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-15T18:29:43.596 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:42.278630+0000 mon.smithi038 (mon.0) 285 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:29:43.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:40.582098+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:29:43.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:40.582224+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:29:43.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:42.241797+0000 mgr.smithi038.khesbl (mgr.14162) 131 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:29:43.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:42.266605+0000 mon.smithi038 (mon.0) 282 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:29:43.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:42.278044+0000 mon.smithi038 (mon.0) 283 : cluster [INF] osd.1 [v2:172.21.15.38:6810/3796697403,v1:172.21.15.38:6811/3796697403] boot 2024-02-15T18:29:43.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:42.278085+0000 mon.smithi038 (mon.0) 284 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-15T18:29:43.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:42.278630+0000 mon.smithi038 (mon.0) 285 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:29:44.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.095+0000 7fd244713700 1 -- 172.21.15.38:0/671010451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd23c0f55c0 msgr2=0x7fd23c0f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:44.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.095+0000 7fd244713700 1 --2- 172.21.15.38:0/671010451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd23c0f55c0 0x7fd23c0f59e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fd238004660 tx=0x7fd238009b30).stop 2024-02-15T18:29:44.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.095+0000 7fd244713700 1 -- 172.21.15.38:0/671010451 shutdown_connections 2024-02-15T18:29:44.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.095+0000 7fd244713700 1 --2- 172.21.15.38:0/671010451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd23c0f6860 0x7fd23c0f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:29:44.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.095+0000 7fd244713700 1 --2- 172.21.15.38:0/671010451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd23c0f55c0 0x7fd23c0f59e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:44.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.095+0000 7fd244713700 1 -- 172.21.15.38:0/671010451 >> 172.21.15.38:0/671010451 conn(0x7fd23c0f0d40 msgr2=0x7fd23c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:44.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.095+0000 7fd244713700 1 -- 172.21.15.38:0/671010451 shutdown_connections 2024-02-15T18:29:44.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.095+0000 7fd244713700 1 -- 172.21.15.38:0/671010451 wait complete. 2024-02-15T18:29:44.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.096+0000 7fd244713700 1 Processor -- start 2024-02-15T18:29:44.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.096+0000 7fd244713700 1 -- start start 2024-02-15T18:29:44.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.096+0000 7fd244713700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd23c0f55c0 0x7fd23c0f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:44.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.096+0000 7fd244713700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd23c0f6860 0x7fd23c0f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:44.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.096+0000 7fd244713700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c0f8190 con 0x7fd23c0f55c0 2024-02-15T18:29:44.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.096+0000 7fd244713700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c0f82d0 con 0x7fd23c0f6860 2024-02-15T18:29:44.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.096+0000 7fd2424af700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd23c0f55c0 0x7fd23c0f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:44.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.096+0000 7fd2424af700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd23c0f55c0 0x7fd23c0f9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57444/0 (socket says 172.21.15.38:57444) 2024-02-15T18:29:44.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.096+0000 7fd2424af700 1 -- 172.21.15.38:0/528416156 learned_addr learned my addr 172.21.15.38:0/528416156 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:29:44.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.096+0000 7fd2424af700 1 -- 172.21.15.38:0/528416156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd23c0f6860 msgr2=0x7fd23c0f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:44.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.096+0000 7fd2424af700 1 --2- 172.21.15.38:0/528416156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd23c0f6860 0x7fd23c0f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:29:44.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.096+0000 7fd2424af700 1 -- 172.21.15.38:0/528416156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd238005040 con 0x7fd23c0f55c0 2024-02-15T18:29:44.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.096+0000 7fd2424af700 1 --2- 172.21.15.38:0/528416156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd23c0f55c0 0x7fd23c0f9720 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fd238009f50 tx=0x7fd238009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:44.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.097+0000 7fd232ffd700 1 -- 172.21.15.38:0/528416156 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd238018070 con 0x7fd23c0f55c0 2024-02-15T18:29:44.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.097+0000 7fd232ffd700 1 -- 172.21.15.38:0/528416156 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd238007d70 con 0x7fd23c0f55c0 2024-02-15T18:29:44.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.097+0000 7fd244713700 1 -- 172.21.15.38:0/528416156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd23c0f8550 con 0x7fd23c0f55c0 2024-02-15T18:29:44.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.097+0000 7fd232ffd700 1 -- 172.21.15.38:0/528416156 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd23801c620 con 0x7fd23c0f55c0 2024-02-15T18:29:44.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.097+0000 7fd244713700 1 -- 172.21.15.38:0/528416156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd23c10f0e0 con 0x7fd23c0f55c0 2024-02-15T18:29:44.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.098+0000 7fd244713700 1 -- 172.21.15.38:0/528416156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd23c045ad0 con 0x7fd23c0f55c0 2024-02-15T18:29:44.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.098+0000 7fd232ffd700 1 -- 172.21.15.38:0/528416156 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fd23801c8e0 con 0x7fd23c0f55c0 2024-02-15T18:29:44.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.098+0000 7fd232ffd700 1 --2- 172.21.15.38:0/528416156 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd22805c2b0 0x7fd22805e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:44.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.098+0000 7fd232ffd700 1 -- 172.21.15.38:0/528416156 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2314+0+0 (secure 0 0 0) 0x7fd238077150 con 0x7fd23c0f55c0 2024-02-15T18:29:44.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.099+0000 7fd241cae700 1 --2- 172.21.15.38:0/528416156 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd22805c2b0 0x7fd22805e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:44.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.099+0000 7fd241cae700 1 --2- 172.21.15.38:0/528416156 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd22805c2b0 0x7fd22805e770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fd22c009610 tx=0x7fd22c006b50).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:44.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.103+0000 7fd232ffd700 1 -- 172.21.15.38:0/528416156 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fd23804aa10 con 0x7fd23c0f55c0 2024-02-15T18:29:44.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:29:44.287+0000 7fd244713700 1 -- 172.21.15.38:0/528416156 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd23c0faca0 con 0x7fd22805c2b0 2024-02-15T18:29:44.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:43.291406+0000 mon.smithi038 (mon.0) 286 : cluster [DBG] osdmap e15: 2 total, 2 up, 2 in 2024-02-15T18:29:44.739 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:43.291406+0000 mon.smithi038 (mon.0) 286 : cluster [DBG] osdmap e15: 2 total, 2 up, 2 in 2024-02-15T18:29:45.594 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:44.242066+0000 mgr.smithi038.khesbl (mgr.14162) 132 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:29:45.595 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:44.289733+0000 mgr.smithi038.khesbl (mgr.14162) 133 : audit [DBG] from='client.14258 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:29:45.595 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:44.292266+0000 mon.smithi038 (mon.0) 287 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:29:45.595 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:29:44.293289+0000 mgr.smithi038.khesbl (mgr.14162) 134 : cephadm [INF] Applying service osd.None on host smithi038... 2024-02-15T18:29:45.595 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:44.294653+0000 mon.smithi038 (mon.0) 288 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:29:45.595 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:44.295241+0000 mon.smithi038 (mon.0) 289 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:45.595 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:44.810247+0000 mon.smithi038 (mon.0) 290 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:45.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:44.242066+0000 mgr.smithi038.khesbl (mgr.14162) 132 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:29:45.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:44.289733+0000 mgr.smithi038.khesbl (mgr.14162) 133 : audit [DBG] from='client.14258 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:29:45.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:44.292266+0000 mon.smithi038 (mon.0) 287 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:29:45.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:29:44.293289+0000 mgr.smithi038.khesbl (mgr.14162) 134 : cephadm [INF] Applying service osd.None on host smithi038... 2024-02-15T18:29:45.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:44.294653+0000 mon.smithi038 (mon.0) 288 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:29:45.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:44.295241+0000 mon.smithi038 (mon.0) 289 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:45.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:44.810247+0000 mon.smithi038 (mon.0) 290 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:47.435 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:46.242353+0000 mgr.smithi038.khesbl (mgr.14162) 135 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:29:47.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:46.242353+0000 mgr.smithi038.khesbl (mgr.14162) 135 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:29:48.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:47.305481+0000 mon.smithi038 (mon.0) 291 : audit [INF] from='client.? 172.21.15.38:0/2885501549' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a397970-64c0-47d7-94c8-77fcf9fe579d"}]: dispatch 2024-02-15T18:29:48.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:47.316446+0000 mon.smithi038 (mon.0) 292 : audit [INF] from='client.? 172.21.15.38:0/2885501549' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a397970-64c0-47d7-94c8-77fcf9fe579d"}]': finished 2024-02-15T18:29:48.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:47.316484+0000 mon.smithi038 (mon.0) 293 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-15T18:29:48.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:47.316576+0000 mon.smithi038 (mon.0) 294 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:29:48.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:47.743099+0000 mon.smithi038 (mon.0) 295 : audit [DBG] from='client.? 172.21.15.38:0/3539865531' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:29:48.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:47.305481+0000 mon.smithi038 (mon.0) 291 : audit [INF] from='client.? 172.21.15.38:0/2885501549' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a397970-64c0-47d7-94c8-77fcf9fe579d"}]: dispatch 2024-02-15T18:29:48.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:47.316446+0000 mon.smithi038 (mon.0) 292 : audit [INF] from='client.? 172.21.15.38:0/2885501549' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a397970-64c0-47d7-94c8-77fcf9fe579d"}]': finished 2024-02-15T18:29:48.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:47.316484+0000 mon.smithi038 (mon.0) 293 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-15T18:29:48.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:47.316576+0000 mon.smithi038 (mon.0) 294 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:29:48.861 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:47.743099+0000 mon.smithi038 (mon.0) 295 : audit [DBG] from='client.? 172.21.15.38:0/3539865531' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:29:50.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:48.242754+0000 mgr.smithi038.khesbl (mgr.14162) 136 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:29:50.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:48.242754+0000 mgr.smithi038.khesbl (mgr.14162) 136 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:29:51.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:50.243072+0000 mgr.smithi038.khesbl (mgr.14162) 137 : cluster [DBG] pgmap v100: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:29:52.002 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:50.243072+0000 mgr.smithi038.khesbl (mgr.14162) 137 : cluster [DBG] pgmap v100: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:29:53.701 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:52.243551+0000 mgr.smithi038.khesbl (mgr.14162) 138 : cluster [DBG] pgmap v101: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:29:54.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:52.243551+0000 mgr.smithi038.khesbl (mgr.14162) 138 : cluster [DBG] pgmap v101: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:29:55.914 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:54.243824+0000 mgr.smithi038.khesbl (mgr.14162) 139 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:29:56.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:54.243824+0000 mgr.smithi038.khesbl (mgr.14162) 139 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:29:57.123 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:29:55.968456+0000 mgr.smithi038.khesbl (mgr.14162) 140 : cephadm [INF] Detected new or changed devices on smithi038 2024-02-15T18:29:57.123 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:55.973450+0000 mon.smithi038 (mon.0) 296 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:57.123 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:56.244054+0000 mgr.smithi038.khesbl (mgr.14162) 141 : cluster [DBG] pgmap v103: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:29:57.123 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:56.308085+0000 mon.smithi038 (mon.0) 297 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:57.123 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:56.310562+0000 mon.smithi038 (mon.0) 298 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:29:57.360 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:29:55.968456+0000 mgr.smithi038.khesbl (mgr.14162) 140 : cephadm [INF] Detected new or changed devices on smithi038 2024-02-15T18:29:57.360 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:55.973450+0000 mon.smithi038 (mon.0) 296 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:57.360 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:56.244054+0000 mgr.smithi038.khesbl (mgr.14162) 141 : cluster [DBG] pgmap v103: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:29:57.360 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:56.308085+0000 mon.smithi038 (mon.0) 297 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:57.360 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:56.310562+0000 mon.smithi038 (mon.0) 298 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:29:58.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:57.151091+0000 mon.smithi038 (mon.0) 299 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-15T18:29:58.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:57.151733+0000 mon.smithi038 (mon.0) 300 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:58.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:29:57.152230+0000 mgr.smithi038.khesbl (mgr.14162) 142 : cephadm [INF] Deploying daemon osd.2 on smithi038 2024-02-15T18:29:58.236 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:57.151091+0000 mon.smithi038 (mon.0) 299 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-15T18:29:58.236 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:57.151733+0000 mon.smithi038 (mon.0) 300 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:58.236 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:29:57.152230+0000 mgr.smithi038.khesbl (mgr.14162) 142 : cephadm [INF] Deploying daemon osd.2 on smithi038 2024-02-15T18:29:59.331 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:29:58.238107+0000 mon.smithi038 (mon.0) 301 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:59.332 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:29:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:29:58.244272+0000 mgr.smithi038.khesbl (mgr.14162) 143 : cluster [DBG] pgmap v104: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:29:59.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:29:58.238107+0000 mon.smithi038 (mon.0) 301 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:29:59.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:29:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:29:58.244272+0000 mgr.smithi038.khesbl (mgr.14162) 143 : cluster [DBG] pgmap v104: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:30:00.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:00.000101+0000 mon.smithi038 (mon.0) 302 : cluster [INF] overall HEALTH_OK 2024-02-15T18:30:00.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:00.000101+0000 mon.smithi038 (mon.0) 302 : cluster [INF] overall HEALTH_OK 2024-02-15T18:30:01.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:00.244563+0000 mgr.smithi038.khesbl (mgr.14162) 144 : cluster [DBG] pgmap v105: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:30:01.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:00.244563+0000 mgr.smithi038.khesbl (mgr.14162) 144 : cluster [DBG] pgmap v105: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:30:03.244 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 2 on host 'smithi038' 2024-02-15T18:30:03.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:30:03.250+0000 7fd232ffd700 1 -- 172.21.15.38:0/528416156 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd23c0faca0 con 0x7fd22805c2b0 2024-02-15T18:30:03.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:30:03.252+0000 7fd244713700 1 -- 172.21.15.38:0/528416156 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd22805c2b0 msgr2=0x7fd22805e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:03.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:30:03.252+0000 7fd244713700 1 --2- 172.21.15.38:0/528416156 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd22805c2b0 0x7fd22805e770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fd22c009610 tx=0x7fd22c006b50).stop 2024-02-15T18:30:03.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:30:03.252+0000 7fd244713700 1 -- 172.21.15.38:0/528416156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd23c0f55c0 msgr2=0x7fd23c0f9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:03.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:30:03.252+0000 7fd244713700 1 --2- 172.21.15.38:0/528416156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd23c0f55c0 0x7fd23c0f9720 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fd238009f50 tx=0x7fd238009b30).stop 2024-02-15T18:30:03.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:30:03.253+0000 7fd244713700 1 -- 172.21.15.38:0/528416156 shutdown_connections 2024-02-15T18:30:03.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:30:03.253+0000 7fd244713700 1 --2- 172.21.15.38:0/528416156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd23c0f55c0 0x7fd23c0f9720 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:03.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:30:03.253+0000 7fd244713700 1 --2- 172.21.15.38:0/528416156 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd22805c2b0 0x7fd22805e770 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:03.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:30:03.253+0000 7fd244713700 1 --2- 172.21.15.38:0/528416156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd23c0f6860 0x7fd23c0f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:30:03.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:30:03.253+0000 7fd244713700 1 -- 172.21.15.38:0/528416156 >> 172.21.15.38:0/528416156 conn(0x7fd23c0f0d40 msgr2=0x7fd23c05fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:03.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:30:03.253+0000 7fd244713700 1 -- 172.21.15.38:0/528416156 shutdown_connections 2024-02-15T18:30:03.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:30:03.253+0000 7fd244713700 1 -- 172.21.15.38:0/528416156 wait complete. 2024-02-15T18:30:03.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:02.244929+0000 mgr.smithi038.khesbl (mgr.14162) 145 : cluster [DBG] pgmap v106: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:30:03.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:03.243346+0000 mon.smithi038 (mon.0) 303 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:03.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:03.249941+0000 mon.smithi038 (mon.0) 304 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:03.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:02.244929+0000 mgr.smithi038.khesbl (mgr.14162) 145 : cluster [DBG] pgmap v106: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:30:03.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:03.243346+0000 mon.smithi038 (mon.0) 303 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:03.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:03.249941+0000 mon.smithi038 (mon.0) 304 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:04.352 DEBUG:teuthology.orchestra.run.smithi038:osd.2> sudo journalctl -f -n 0 -u ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.2.service 2024-02-15T18:30:04.355 INFO:tasks.cephadm:Deploying osd.3 on smithi121 with /dev/vg_nvme/lv_4... 2024-02-15T18:30:04.355 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-15T18:30:04.389 INFO:journalctl@ceph.osd.2.smithi038.stdout:-- Logs begin at Thu 2024-02-15 18:12:54 UTC. -- 2024-02-15T18:30:05.967 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:04.245601+0000 mgr.smithi038.khesbl (mgr.14162) 146 : cluster [DBG] pgmap v107: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:30:05.967 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:04.788532+0000 mon.smithi038 (mon.0) 305 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:06.178 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:04.245601+0000 mgr.smithi038.khesbl (mgr.14162) 146 : cluster [DBG] pgmap v107: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:30:06.178 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:04.788532+0000 mon.smithi038 (mon.0) 305 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:06.466 INFO:teuthology.orchestra.run.smithi121.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-15T18:30:06.466 INFO:teuthology.orchestra.run.smithi121.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-15T18:30:06.466 INFO:teuthology.orchestra.run.smithi121.stderr: stderr: 10+0 records in 2024-02-15T18:30:06.467 INFO:teuthology.orchestra.run.smithi121.stderr:10+0 records out 2024-02-15T18:30:06.467 INFO:teuthology.orchestra.run.smithi121.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0135724 s, 773 MB/s 2024-02-15T18:30:06.467 INFO:teuthology.orchestra.run.smithi121.stderr:--> Zapping successful for: 2024-02-15T18:30:07.025 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:06.787718+0000 mon.smithi038 (mon.0) 306 : audit [INF] from='osd.2 [v2:172.21.15.38:6818/420320553,v1:172.21.15.38:6819/420320553]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-15T18:30:07.043 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi121:vg_nvme/lv_4 2024-02-15T18:30:07.276 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:06.787718+0000 mon.smithi038 (mon.0) 306 : audit [INF] from='osd.2 [v2:172.21.15.38:6818/420320553,v1:172.21.15.38:6819/420320553]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-15T18:30:07.276 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:30:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.2[56774]: debug 2024-02-15T18:30:06.784+0000 7fb03eafd080 -1 osd.2 0 log_to_monitors {default=true} 2024-02-15T18:30:07.926 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:30:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.2[56774]: debug 2024-02-15T18:30:07.819+0000 7fb03506e700 -1 osd.2 0 waiting for initial osdmap 2024-02-15T18:30:07.926 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:30:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.2[56774]: debug 2024-02-15T18:30:07.824+0000 7fb02dfe3700 -1 osd.2 18 set_numa_affinity unable to identify public interface 'ens1f0' numa node: (0) Success 2024-02-15T18:30:07.926 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:06.245878+0000 mgr.smithi038.khesbl (mgr.14162) 147 : cluster [DBG] pgmap v108: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:30:07.926 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:06.809456+0000 mon.smithi038 (mon.0) 307 : audit [INF] from='osd.2 [v2:172.21.15.38:6818/420320553,v1:172.21.15.38:6819/420320553]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-15T18:30:07.926 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:06.809537+0000 mon.smithi038 (mon.0) 308 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-15T18:30:07.927 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:06.809653+0000 mon.smithi038 (mon.0) 309 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:30:07.927 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:06.810099+0000 mon.smithi038 (mon.0) 310 : audit [INF] from='osd.2 [v2:172.21.15.38:6818/420320553,v1:172.21.15.38:6819/420320553]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-02-15T18:30:08.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:06.245878+0000 mgr.smithi038.khesbl (mgr.14162) 147 : cluster [DBG] pgmap v108: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:30:08.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:06.809456+0000 mon.smithi038 (mon.0) 307 : audit [INF] from='osd.2 [v2:172.21.15.38:6818/420320553,v1:172.21.15.38:6819/420320553]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-15T18:30:08.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:06.809537+0000 mon.smithi038 (mon.0) 308 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-15T18:30:08.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:06.809653+0000 mon.smithi038 (mon.0) 309 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:30:08.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:06.810099+0000 mon.smithi038 (mon.0) 310 : audit [INF] from='osd.2 [v2:172.21.15.38:6818/420320553,v1:172.21.15.38:6819/420320553]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-02-15T18:30:09.121 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.896+0000 7f40f0b53700 1 -- 172.21.15.121:0/1720855405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40ec0f7980 msgr2=0x7f40ec0f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:09.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.896+0000 7f40f0b53700 1 --2- 172.21.15.121:0/1720855405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40ec0f7980 0x7f40ec0f7de0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f40dc004660 tx=0x7f40dc009b30).stop 2024-02-15T18:30:09.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.896+0000 7f40f0b53700 1 -- 172.21.15.121:0/1720855405 shutdown_connections 2024-02-15T18:30:09.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.896+0000 7f40f0b53700 1 --2- 172.21.15.121:0/1720855405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40ec0f7980 0x7f40ec0f7de0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:09.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.896+0000 7f40f0b53700 1 --2- 172.21.15.121:0/1720855405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40ec0f66e0 0x7f40ec0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:09.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.896+0000 7f40f0b53700 1 -- 172.21.15.121:0/1720855405 >> 172.21.15.121:0/1720855405 conn(0x7f40ec0f1e40 msgr2=0x7f40ec0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:09.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.896+0000 7f40f0b53700 1 -- 172.21.15.121:0/1720855405 shutdown_connections 2024-02-15T18:30:09.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.896+0000 7f40f0b53700 1 -- 172.21.15.121:0/1720855405 wait complete. 2024-02-15T18:30:09.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.896+0000 7f40f0b53700 1 Processor -- start 2024-02-15T18:30:09.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.896+0000 7f40f0b53700 1 -- start start 2024-02-15T18:30:09.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.896+0000 7f40f0b53700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40ec0f66e0 0x7f40ec06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:09.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.896+0000 7f40f0b53700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40ec0f7980 0x7f40ec069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:09.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.896+0000 7f40f0b53700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec06a380 con 0x7f40ec0f66e0 2024-02-15T18:30:09.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.896+0000 7f40f0b53700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec06a4c0 con 0x7f40ec0f7980 2024-02-15T18:30:09.123 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.896+0000 7f40e9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40ec0f7980 0x7f40ec069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:09.123 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.897+0000 7f40e9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40ec0f7980 0x7f40ec069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36056/0 (socket says 172.21.15.121:36056) 2024-02-15T18:30:09.123 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.897+0000 7f40e9d9b700 1 -- 172.21.15.121:0/3176715499 learned_addr learned my addr 172.21.15.121:0/3176715499 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-02-15T18:30:09.123 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.897+0000 7f40ea59c700 1 --2- 172.21.15.121:0/3176715499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40ec0f66e0 0x7f40ec06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:09.123 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.897+0000 7f40e9d9b700 1 -- 172.21.15.121:0/3176715499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40ec0f66e0 msgr2=0x7f40ec06b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:09.123 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.897+0000 7f40e9d9b700 1 --2- 172.21.15.121:0/3176715499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40ec0f66e0 0x7f40ec06b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:09.123 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.897+0000 7f40e9d9b700 1 -- 172.21.15.121:0/3176715499 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40dc005040 con 0x7f40ec0f7980 2024-02-15T18:30:09.123 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.897+0000 7f40e9d9b700 1 --2- 172.21.15.121:0/3176715499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40ec0f7980 0x7f40ec069e20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f40dc009830 tx=0x7f40dc009b30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:09.123 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.897+0000 7f40daffd700 1 -- 172.21.15.121:0/3176715499 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40dc018070 con 0x7f40ec0f7980 2024-02-15T18:30:09.123 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.897+0000 7f40daffd700 1 -- 172.21.15.121:0/3176715499 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40dc007d70 con 0x7f40ec0f7980 2024-02-15T18:30:09.123 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.897+0000 7f40f0b53700 1 -- 172.21.15.121:0/3176715499 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ec06a740 con 0x7f40ec0f7980 2024-02-15T18:30:09.123 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.897+0000 7f40daffd700 1 -- 172.21.15.121:0/3176715499 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40dc01c620 con 0x7f40ec0f7980 2024-02-15T18:30:09.123 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.897+0000 7f40f0b53700 1 -- 172.21.15.121:0/3176715499 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ec06aca0 con 0x7f40ec0f7980 2024-02-15T18:30:09.123 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.898+0000 7f40f0b53700 1 -- 172.21.15.121:0/3176715499 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40ec045ad0 con 0x7f40ec0f7980 2024-02-15T18:30:09.124 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.898+0000 7f40daffd700 1 -- 172.21.15.121:0/3176715499 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f40dc01d980 con 0x7f40ec0f7980 2024-02-15T18:30:09.124 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.898+0000 7f40daffd700 1 --2- 172.21.15.121:0/3176715499 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f40d405c2b0 0x7f40d405e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:09.124 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.899+0000 7f40daffd700 1 -- 172.21.15.121:0/3176715499 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2746+0+0 (secure 0 0 0) 0x7f40dc077290 con 0x7f40ec0f7980 2024-02-15T18:30:09.124 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.899+0000 7f40ea59c700 1 --2- 172.21.15.121:0/3176715499 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f40d405c2b0 0x7f40d405e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:09.124 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.899+0000 7f40ea59c700 1 --2- 172.21.15.121:0/3176715499 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f40d405c2b0 0x7f40d405e770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f40ec06b730 tx=0x7f40e0009040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:09.124 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:08.903+0000 7f40daffd700 1 -- 172.21.15.121:0/3176715499 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f40dc04a9c0 con 0x7f40ec0f7980 2024-02-15T18:30:09.124 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:09.082+0000 7f40f0b53700 1 -- 172.21.15.121:0/3176715499 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f40ec0ff120 con 0x7f40d405c2b0 2024-02-15T18:30:09.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:07.815188+0000 mon.smithi038 (mon.0) 311 : audit [INF] from='osd.2 [v2:172.21.15.38:6818/420320553,v1:172.21.15.38:6819/420320553]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-02-15T18:30:09.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:07.815273+0000 mon.smithi038 (mon.0) 312 : cluster [DBG] osdmap e18: 3 total, 2 up, 3 in 2024-02-15T18:30:09.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:07.815717+0000 mon.smithi038 (mon.0) 313 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:30:09.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:07.819857+0000 mon.smithi038 (mon.0) 314 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:30:09.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:08.172948+0000 mon.smithi038 (mon.0) 315 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:09.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:08.176079+0000 mon.smithi038 (mon.0) 316 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:09.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:08.178331+0000 mon.smithi038 (mon.0) 317 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:30:09.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:07.815188+0000 mon.smithi038 (mon.0) 311 : audit [INF] from='osd.2 [v2:172.21.15.38:6818/420320553,v1:172.21.15.38:6819/420320553]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-02-15T18:30:09.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:07.815273+0000 mon.smithi038 (mon.0) 312 : cluster [DBG] osdmap e18: 3 total, 2 up, 3 in 2024-02-15T18:30:09.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:07.815717+0000 mon.smithi038 (mon.0) 313 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:30:09.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:07.819857+0000 mon.smithi038 (mon.0) 314 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:30:09.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:08.172948+0000 mon.smithi038 (mon.0) 315 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:09.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:08.176079+0000 mon.smithi038 (mon.0) 316 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:09.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:08.178331+0000 mon.smithi038 (mon.0) 317 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:30:10.360 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:07.799040+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:30:10.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:07.799233+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:30:10.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:08.246159+0000 mgr.smithi038.khesbl (mgr.14162) 148 : cluster [DBG] pgmap v111: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:30:10.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:08.818636+0000 mon.smithi038 (mon.0) 318 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:30:10.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:08.821337+0000 mon.smithi038 (mon.0) 319 : cluster [INF] osd.2 [v2:172.21.15.38:6818/420320553,v1:172.21.15.38:6819/420320553] boot 2024-02-15T18:30:10.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:08.821371+0000 mon.smithi038 (mon.0) 320 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-15T18:30:10.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:08.822086+0000 mon.smithi038 (mon.0) 321 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:30:10.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:08.822448+0000 mon.smithi038 (mon.0) 322 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]: dispatch 2024-02-15T18:30:10.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:09.083910+0000 mgr.smithi038.khesbl (mgr.14162) 149 : audit [DBG] from='client.24135 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:10.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:09.824425+0000 mon.smithi038 (mon.0) 323 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]': finished 2024-02-15T18:30:10.362 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:09.824480+0000 mon.smithi038 (mon.0) 324 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-15T18:30:10.362 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:09.825353+0000 mon.smithi038 (mon.0) 325 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-15T18:30:10.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:07.799040+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:30:10.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:07.799233+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:30:10.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:08.246159+0000 mgr.smithi038.khesbl (mgr.14162) 148 : cluster [DBG] pgmap v111: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:30:10.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:08.818636+0000 mon.smithi038 (mon.0) 318 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:30:10.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:08.821337+0000 mon.smithi038 (mon.0) 319 : cluster [INF] osd.2 [v2:172.21.15.38:6818/420320553,v1:172.21.15.38:6819/420320553] boot 2024-02-15T18:30:10.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:08.821371+0000 mon.smithi038 (mon.0) 320 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-15T18:30:10.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:08.822086+0000 mon.smithi038 (mon.0) 321 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:30:10.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:08.822448+0000 mon.smithi038 (mon.0) 322 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]: dispatch 2024-02-15T18:30:10.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:09.083910+0000 mgr.smithi038.khesbl (mgr.14162) 149 : audit [DBG] from='client.24135 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:10.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:09.824425+0000 mon.smithi038 (mon.0) 323 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]': finished 2024-02-15T18:30:10.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:09.824480+0000 mon.smithi038 (mon.0) 324 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-15T18:30:10.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:09.825353+0000 mon.smithi038 (mon.0) 325 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-15T18:30:11.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:10.246510+0000 mgr.smithi038.khesbl (mgr.14162) 150 : cluster [DBG] pgmap v114: 1 pgs: 1 unknown; 0 B data, 14 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:11.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:10.599949+0000 mon.smithi038 (mon.0) 326 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:11.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:10.830638+0000 mon.smithi038 (mon.0) 327 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-15T18:30:11.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:10.830720+0000 mon.smithi038 (mon.0) 328 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-02-15T18:30:11.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:10.832962+0000 mon.smithi038 (mon.0) 329 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:30:11.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:30:10.834134+0000 mgr.smithi038.khesbl (mgr.14162) 151 : cephadm [INF] Applying service osd.None on host smithi121... 2024-02-15T18:30:11.861 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:10.834319+0000 mon.smithi038 (mon.0) 330 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:30:11.861 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:10.834878+0000 mon.smithi038 (mon.0) 331 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:30:12.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:10.246510+0000 mgr.smithi038.khesbl (mgr.14162) 150 : cluster [DBG] pgmap v114: 1 pgs: 1 unknown; 0 B data, 14 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:12.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:10.599949+0000 mon.smithi038 (mon.0) 326 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:12.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:10.830638+0000 mon.smithi038 (mon.0) 327 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-15T18:30:12.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:10.830720+0000 mon.smithi038 (mon.0) 328 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-02-15T18:30:12.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:10.832962+0000 mon.smithi038 (mon.0) 329 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:30:12.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:30:10.834134+0000 mgr.smithi038.khesbl (mgr.14162) 151 : cephadm [INF] Applying service osd.None on host smithi121... 2024-02-15T18:30:12.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:10.834319+0000 mon.smithi038 (mon.0) 330 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:30:12.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:10.834878+0000 mon.smithi038 (mon.0) 331 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:30:13.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:11.832016+0000 mon.smithi038 (mon.0) 332 : cluster [DBG] osdmap e22: 3 total, 3 up, 3 in 2024-02-15T18:30:13.259 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:11.832016+0000 mon.smithi038 (mon.0) 332 : cluster [DBG] osdmap e22: 3 total, 3 up, 3 in 2024-02-15T18:30:14.196 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:12.246795+0000 mgr.smithi038.khesbl (mgr.14162) 152 : cluster [DBG] pgmap v117: 1 pgs: 1 unknown; 0 B data, 14 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:14.196 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:12.841599+0000 mon.smithi038 (mon.0) 333 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:14.196 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:13.260284+0000 mon.smithi121 (mon.1) 2 : audit [INF] from='client.? 172.21.15.121:0/159443056' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a590f02-56f3-43e7-894f-741f95d2facd"}]: dispatch 2024-02-15T18:30:14.196 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:13.260736+0000 mon.smithi038 (mon.0) 334 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a590f02-56f3-43e7-894f-741f95d2facd"}]: dispatch 2024-02-15T18:30:14.197 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:13.266766+0000 mon.smithi038 (mon.0) 335 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a590f02-56f3-43e7-894f-741f95d2facd"}]': finished 2024-02-15T18:30:14.197 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:13.266837+0000 mon.smithi038 (mon.0) 336 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-15T18:30:14.197 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:13.266973+0000 mon.smithi038 (mon.0) 337 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:30:14.197 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:13.678328+0000 mon.smithi121 (mon.1) 3 : audit [DBG] from='client.? 172.21.15.121:0/2154131374' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:30:14.360 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:12.246795+0000 mgr.smithi038.khesbl (mgr.14162) 152 : cluster [DBG] pgmap v117: 1 pgs: 1 unknown; 0 B data, 14 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:14.360 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:12.841599+0000 mon.smithi038 (mon.0) 333 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:14.360 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:13.260284+0000 mon.smithi121 (mon.1) 2 : audit [INF] from='client.? 172.21.15.121:0/159443056' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a590f02-56f3-43e7-894f-741f95d2facd"}]: dispatch 2024-02-15T18:30:14.360 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:13.260736+0000 mon.smithi038 (mon.0) 334 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a590f02-56f3-43e7-894f-741f95d2facd"}]: dispatch 2024-02-15T18:30:14.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:13.266766+0000 mon.smithi038 (mon.0) 335 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a590f02-56f3-43e7-894f-741f95d2facd"}]': finished 2024-02-15T18:30:14.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:13.266837+0000 mon.smithi038 (mon.0) 336 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-15T18:30:14.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:13.266973+0000 mon.smithi038 (mon.0) 337 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:30:14.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:13.678328+0000 mon.smithi121 (mon.1) 3 : audit [DBG] from='client.? 172.21.15.121:0/2154131374' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:30:16.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:14.247165+0000 mgr.smithi038.khesbl (mgr.14162) 153 : cluster [DBG] pgmap v119: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:16.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:14.247165+0000 mgr.smithi038.khesbl (mgr.14162) 153 : cluster [DBG] pgmap v119: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:18.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:16.247506+0000 mgr.smithi038.khesbl (mgr.14162) 154 : cluster [DBG] pgmap v120: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:18.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:16.713526+0000 mon.smithi038 (mon.0) 338 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:18.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:16.717741+0000 mon.smithi038 (mon.0) 339 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:18.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:16.247506+0000 mgr.smithi038.khesbl (mgr.14162) 154 : cluster [DBG] pgmap v120: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:18.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:16.713526+0000 mon.smithi038 (mon.0) 338 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:18.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:16.717741+0000 mon.smithi038 (mon.0) 339 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:20.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:18.247957+0000 mgr.smithi038.khesbl (mgr.14162) 155 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:20.084 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:18.247957+0000 mgr.smithi038.khesbl (mgr.14162) 155 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:21.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:20 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:20.385596+0000 mon.smithi038 (mon.0) 340 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-15T18:30:21.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:20 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:20.386450+0000 mon.smithi038 (mon.0) 341 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:30:21.044 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:20 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:20.385596+0000 mon.smithi038 (mon.0) 340 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-15T18:30:21.044 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:20 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:20.386450+0000 mon.smithi038 (mon.0) 341 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:30:22.024 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:20.248318+0000 mgr.smithi038.khesbl (mgr.14162) 156 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:22.080 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:30:20.387153+0000 mgr.smithi038.khesbl (mgr.14162) 157 : cephadm [INF] Deploying daemon osd.3 on smithi121 2024-02-15T18:30:22.080 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:20.248318+0000 mgr.smithi038.khesbl (mgr.14162) 156 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:22.080 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:30:20.387153+0000 mgr.smithi038.khesbl (mgr.14162) 157 : cephadm [INF] Deploying daemon osd.3 on smithi121 2024-02-15T18:30:24.017 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:22.248675+0000 mgr.smithi038.khesbl (mgr.14162) 158 : cluster [DBG] pgmap v123: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:24.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:22.248675+0000 mgr.smithi038.khesbl (mgr.14162) 158 : cluster [DBG] pgmap v123: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:24.187 INFO:teuthology.orchestra.run.smithi121.stdout:Created osd(s) 3 on host 'smithi121' 2024-02-15T18:30:24.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:24.191+0000 7f40daffd700 1 -- 172.21.15.121:0/3176715499 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f40ec0ff120 con 0x7f40d405c2b0 2024-02-15T18:30:24.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:24.194+0000 7f40f0b53700 1 -- 172.21.15.121:0/3176715499 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f40d405c2b0 msgr2=0x7f40d405e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:24.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:24.194+0000 7f40f0b53700 1 --2- 172.21.15.121:0/3176715499 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f40d405c2b0 0x7f40d405e770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f40ec06b730 tx=0x7f40e0009040).stop 2024-02-15T18:30:24.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:24.194+0000 7f40f0b53700 1 -- 172.21.15.121:0/3176715499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40ec0f7980 msgr2=0x7f40ec069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:24.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:24.194+0000 7f40f0b53700 1 --2- 172.21.15.121:0/3176715499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40ec0f7980 0x7f40ec069e20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f40dc009830 tx=0x7f40dc009b30).stop 2024-02-15T18:30:24.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:24.194+0000 7f40f0b53700 1 -- 172.21.15.121:0/3176715499 shutdown_connections 2024-02-15T18:30:24.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:24.194+0000 7f40f0b53700 1 --2- 172.21.15.121:0/3176715499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f40ec0f66e0 0x7f40ec06b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:24.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:24.194+0000 7f40f0b53700 1 --2- 172.21.15.121:0/3176715499 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f40d405c2b0 0x7f40d405e770 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:24.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:24.194+0000 7f40f0b53700 1 --2- 172.21.15.121:0/3176715499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40ec0f7980 0x7f40ec069e20 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:24.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:24.194+0000 7f40f0b53700 1 -- 172.21.15.121:0/3176715499 >> 172.21.15.121:0/3176715499 conn(0x7f40ec0f1e40 msgr2=0x7f40ec0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:24.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:24.194+0000 7f40f0b53700 1 -- 172.21.15.121:0/3176715499 shutdown_connections 2024-02-15T18:30:24.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:24.194+0000 7f40f0b53700 1 -- 172.21.15.121:0/3176715499 wait complete. 2024-02-15T18:30:24.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:24 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:24.178870+0000 mon.smithi038 (mon.0) 342 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:30:24.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:24 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:24.182528+0000 mon.smithi038 (mon.0) 343 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:24.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:24 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:24.192015+0000 mon.smithi038 (mon.0) 344 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:24.935 DEBUG:teuthology.orchestra.run.smithi121:osd.3> sudo journalctl -f -n 0 -u ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.3.service 2024-02-15T18:30:24.937 INFO:tasks.cephadm:Deploying osd.4 on smithi121 with /dev/vg_nvme/lv_3... 2024-02-15T18:30:24.937 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-15T18:30:24.968 INFO:journalctl@ceph.osd.3.smithi121.stdout:-- Logs begin at Thu 2024-02-15 18:14:30 UTC. -- 2024-02-15T18:30:25.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:24.178870+0000 mon.smithi038 (mon.0) 342 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:30:25.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:24.182528+0000 mon.smithi038 (mon.0) 343 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:25.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:24.192015+0000 mon.smithi038 (mon.0) 344 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:26.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:24.249052+0000 mgr.smithi038.khesbl (mgr.14162) 159 : cluster [DBG] pgmap v124: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:26.095 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:24.249052+0000 mgr.smithi038.khesbl (mgr.14162) 159 : cluster [DBG] pgmap v124: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:27.575 INFO:teuthology.orchestra.run.smithi121.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-15T18:30:27.575 INFO:teuthology.orchestra.run.smithi121.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-15T18:30:27.575 INFO:teuthology.orchestra.run.smithi121.stderr: stderr: 10+0 records in 2024-02-15T18:30:27.575 INFO:teuthology.orchestra.run.smithi121.stderr:10+0 records out 2024-02-15T18:30:27.575 INFO:teuthology.orchestra.run.smithi121.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.012166 s, 862 MB/s 2024-02-15T18:30:27.575 INFO:teuthology.orchestra.run.smithi121.stderr:--> Zapping successful for: 2024-02-15T18:30:28.018 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:26.249532+0000 mgr.smithi038.khesbl (mgr.14162) 160 : cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:28.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:26.249532+0000 mgr.smithi038.khesbl (mgr.14162) 160 : cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:28.360 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:30:28 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.3[45185]: debug 2024-02-15T18:30:28.024+0000 7fc22b64d080 -1 osd.3 0 log_to_monitors {default=true} 2024-02-15T18:30:28.708 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi121:vg_nvme/lv_3 2024-02-15T18:30:28.981 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:28 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:28.028434+0000 mon.smithi121 (mon.1) 4 : audit [INF] from='osd.3 [v2:172.21.15.121:6800/3071118694,v1:172.21.15.121:6801/3071118694]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-15T18:30:28.981 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:28 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:28.028752+0000 mon.smithi038 (mon.0) 345 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-15T18:30:28.981 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:28 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:28.347842+0000 mon.smithi038 (mon.0) 346 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:29.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:28.028434+0000 mon.smithi121 (mon.1) 4 : audit [INF] from='osd.3 [v2:172.21.15.121:6800/3071118694,v1:172.21.15.121:6801/3071118694]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-15T18:30:29.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:28.028752+0000 mon.smithi038 (mon.0) 345 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-15T18:30:29.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:28.347842+0000 mon.smithi038 (mon.0) 346 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:30.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:28.249880+0000 mgr.smithi038.khesbl (mgr.14162) 161 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:30.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:28.770980+0000 mon.smithi038 (mon.0) 347 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-15T18:30:30.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:28.771062+0000 mon.smithi038 (mon.0) 348 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-15T18:30:30.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:28.771198+0000 mon.smithi038 (mon.0) 349 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:30:30.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:28.771582+0000 mon.smithi121 (mon.1) 5 : audit [INF] from='osd.3 [v2:172.21.15.121:6800/3071118694,v1:172.21.15.121:6801/3071118694]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:30:30.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:28.771879+0000 mon.smithi038 (mon.0) 350 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:30:30.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:28.843607+0000 mon.smithi038 (mon.0) 351 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:30.043 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:30:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.3[45185]: debug 2024-02-15T18:30:29.775+0000 7fc221bbe700 -1 osd.3 0 waiting for initial osdmap 2024-02-15T18:30:30.043 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:30:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.3[45185]: debug 2024-02-15T18:30:29.783+0000 7fc21c336700 -1 osd.3 25 set_numa_affinity unable to identify public interface 'enp3s0f1' numa node: (0) Success 2024-02-15T18:30:30.043 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:28.249880+0000 mgr.smithi038.khesbl (mgr.14162) 161 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:30.043 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:28.770980+0000 mon.smithi038 (mon.0) 347 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-15T18:30:30.043 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:28.771062+0000 mon.smithi038 (mon.0) 348 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-15T18:30:30.043 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:28.771198+0000 mon.smithi038 (mon.0) 349 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:30:30.044 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:28.771582+0000 mon.smithi121 (mon.1) 5 : audit [INF] from='osd.3 [v2:172.21.15.121:6800/3071118694,v1:172.21.15.121:6801/3071118694]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:30:30.044 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:28.771879+0000 mon.smithi038 (mon.0) 350 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:30:30.044 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:28.843607+0000 mon.smithi038 (mon.0) 351 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:30.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:30 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:29.773665+0000 mon.smithi038 (mon.0) 352 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-02-15T18:30:30.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:30 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:29.773759+0000 mon.smithi038 (mon.0) 353 : cluster [DBG] osdmap e25: 4 total, 3 up, 4 in 2024-02-15T18:30:30.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:30 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:29.775836+0000 mon.smithi038 (mon.0) 354 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:30:30.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:30 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:29.777912+0000 mon.smithi038 (mon.0) 355 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:30:30.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:30 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:30.777315+0000 mon.smithi038 (mon.0) 356 : cluster [INF] osd.3 [v2:172.21.15.121:6800/3071118694,v1:172.21.15.121:6801/3071118694] boot 2024-02-15T18:30:30.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:30 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:30.777371+0000 mon.smithi038 (mon.0) 357 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-02-15T18:30:30.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:30 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:30.777608+0000 mon.smithi038 (mon.0) 358 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:30:31.264 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:29.773665+0000 mon.smithi038 (mon.0) 352 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-02-15T18:30:31.265 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:29.773759+0000 mon.smithi038 (mon.0) 353 : cluster [DBG] osdmap e25: 4 total, 3 up, 4 in 2024-02-15T18:30:31.265 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:29.775836+0000 mon.smithi038 (mon.0) 354 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:30:31.265 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:29.777912+0000 mon.smithi038 (mon.0) 355 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:30:31.265 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:30.777315+0000 mon.smithi038 (mon.0) 356 : cluster [INF] osd.3 [v2:172.21.15.121:6800/3071118694,v1:172.21.15.121:6801/3071118694] boot 2024-02-15T18:30:31.265 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:30.777371+0000 mon.smithi038 (mon.0) 357 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-02-15T18:30:31.265 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:30.777608+0000 mon.smithi038 (mon.0) 358 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:30:32.090 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.047+0000 7f9cfa9e1700 1 -- 172.21.15.121:0/2102673085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cf40f66e0 msgr2=0x7f9cf40f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:32.090 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.047+0000 7f9cfa9e1700 1 --2- 172.21.15.121:0/2102673085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cf40f66e0 0x7f9cf40f6b00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f9ce4004660 tx=0x7f9ce4009b30).stop 2024-02-15T18:30:32.090 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.047+0000 7f9cfa9e1700 1 -- 172.21.15.121:0/2102673085 shutdown_connections 2024-02-15T18:30:32.090 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.047+0000 7f9cfa9e1700 1 --2- 172.21.15.121:0/2102673085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cf40f7980 0x7f9cf40f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:30:32.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.047+0000 7f9cfa9e1700 1 --2- 172.21.15.121:0/2102673085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cf40f66e0 0x7f9cf40f6b00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:32.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.047+0000 7f9cfa9e1700 1 -- 172.21.15.121:0/2102673085 >> 172.21.15.121:0/2102673085 conn(0x7f9cf40f1e40 msgr2=0x7f9cf40f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:32.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.047+0000 7f9cfa9e1700 1 -- 172.21.15.121:0/2102673085 shutdown_connections 2024-02-15T18:30:32.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.047+0000 7f9cfa9e1700 1 -- 172.21.15.121:0/2102673085 wait complete. 2024-02-15T18:30:32.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.048+0000 7f9cfa9e1700 1 Processor -- start 2024-02-15T18:30:32.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.048+0000 7f9cfa9e1700 1 -- start start 2024-02-15T18:30:32.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.048+0000 7f9cfa9e1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cf40f66e0 0x7f9cf41070a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:32.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.048+0000 7f9cfa9e1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cf40f7980 0x7f9cf4107590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:32.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.048+0000 7f9cfa9e1700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf4107bd0 con 0x7f9cf40f7980 2024-02-15T18:30:32.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.048+0000 7f9cfa9e1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf4108480 con 0x7f9cf40f66e0 2024-02-15T18:30:32.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.048+0000 7f9cf3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cf40f66e0 0x7f9cf41070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:32.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.048+0000 7f9cf3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cf40f66e0 0x7f9cf41070a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44882/0 (socket says 172.21.15.121:44882) 2024-02-15T18:30:32.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.048+0000 7f9cf3fff700 1 -- 172.21.15.121:0/3281719354 learned_addr learned my addr 172.21.15.121:0/3281719354 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-02-15T18:30:32.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.048+0000 7f9cf37fe700 1 --2- 172.21.15.121:0/3281719354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cf40f7980 0x7f9cf4107590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:32.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.048+0000 7f9cf3fff700 1 -- 172.21.15.121:0/3281719354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cf40f7980 msgr2=0x7f9cf4107590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:32.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.048+0000 7f9cf3fff700 1 --2- 172.21.15.121:0/3281719354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cf40f7980 0x7f9cf4107590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:32.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.048+0000 7f9cf3fff700 1 -- 172.21.15.121:0/3281719354 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce4005040 con 0x7f9cf40f66e0 2024-02-15T18:30:32.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.049+0000 7f9cf3fff700 1 --2- 172.21.15.121:0/3281719354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cf40f66e0 0x7f9cf41070a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f9ce4009f50 tx=0x7f9ce4009b30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:32.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.049+0000 7f9cf0ff9700 1 -- 172.21.15.121:0/3281719354 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ce4018070 con 0x7f9cf40f66e0 2024-02-15T18:30:32.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.049+0000 7f9cfa9e1700 1 -- 172.21.15.121:0/3281719354 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf4108700 con 0x7f9cf40f66e0 2024-02-15T18:30:32.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.049+0000 7f9cf0ff9700 1 -- 172.21.15.121:0/3281719354 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ce4007d70 con 0x7f9cf40f66e0 2024-02-15T18:30:32.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.049+0000 7f9cf0ff9700 1 -- 172.21.15.121:0/3281719354 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ce401c620 con 0x7f9cf40f66e0 2024-02-15T18:30:32.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.049+0000 7f9cfa9e1700 1 -- 172.21.15.121:0/3281719354 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf4108c60 con 0x7f9cf40f66e0 2024-02-15T18:30:32.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.050+0000 7f9cfa9e1700 1 -- 172.21.15.121:0/3281719354 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf40f5ad0 con 0x7f9cf40f66e0 2024-02-15T18:30:32.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.050+0000 7f9cf0ff9700 1 -- 172.21.15.121:0/3281719354 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f9ce401c8e0 con 0x7f9cf40f66e0 2024-02-15T18:30:32.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.050+0000 7f9cf0ff9700 1 --2- 172.21.15.121:0/3281719354 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9cdc05c2b0 0x7f9cdc05e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:32.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.050+0000 7f9cf0ff9700 1 -- 172.21.15.121:0/3281719354 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3688+0+0 (secure 0 0 0) 0x7f9ce4077730 con 0x7f9cf40f66e0 2024-02-15T18:30:32.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.050+0000 7f9cf37fe700 1 --2- 172.21.15.121:0/3281719354 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9cdc05c2b0 0x7f9cdc05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:32.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.051+0000 7f9cf37fe700 1 --2- 172.21.15.121:0/3281719354 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9cdc05c2b0 0x7f9cdc05e770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f9ce800b790 tx=0x7f9ce8008040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:32.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.054+0000 7f9cf0ff9700 1 -- 172.21.15.121:0/3281719354 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9ce404aa90 con 0x7f9cf40f66e0 2024-02-15T18:30:32.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:29.057259+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:30:32.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:29.057358+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:30:32.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:30.250217+0000 mgr.smithi038.khesbl (mgr.14162) 162 : cluster [DBG] pgmap v129: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:32.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:31.778730+0000 mon.smithi038 (mon.0) 359 : cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2024-02-15T18:30:32.232 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:32.238+0000 7f9cfa9e1700 1 -- 172.21.15.121:0/3281719354 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9cf40ff030 con 0x7f9cdc05c2b0 2024-02-15T18:30:32.276 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:29.057259+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:30:32.276 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:29.057358+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:30:32.276 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:30.250217+0000 mgr.smithi038.khesbl (mgr.14162) 162 : cluster [DBG] pgmap v129: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:30:32.276 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:31.778730+0000 mon.smithi038 (mon.0) 359 : cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2024-02-15T18:30:33.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:32.241584+0000 mon.smithi038 (mon.0) 360 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:30:33.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:32.243260+0000 mon.smithi038 (mon.0) 361 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:30:33.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:32.244093+0000 mon.smithi038 (mon.0) 362 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:30:33.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:32.780773+0000 mon.smithi038 (mon.0) 363 : cluster [DBG] osdmap e28: 4 total, 4 up, 4 in 2024-02-15T18:30:33.276 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:32.241584+0000 mon.smithi038 (mon.0) 360 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:30:33.276 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:32.243260+0000 mon.smithi038 (mon.0) 361 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:30:33.276 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:32.244093+0000 mon.smithi038 (mon.0) 362 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:30:33.276 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:32.780773+0000 mon.smithi038 (mon.0) 363 : cluster [DBG] osdmap e28: 4 total, 4 up, 4 in 2024-02-15T18:30:34.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:32.240679+0000 mgr.smithi038.khesbl (mgr.14162) 163 : audit [DBG] from='client.24153 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:34.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:30:32.242992+0000 mgr.smithi038.khesbl (mgr.14162) 164 : cephadm [INF] Applying service osd.None on host smithi121... 2024-02-15T18:30:34.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:32.250542+0000 mgr.smithi038.khesbl (mgr.14162) 165 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:34.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:32.240679+0000 mgr.smithi038.khesbl (mgr.14162) 163 : audit [DBG] from='client.24153 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:34.276 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:30:32.242992+0000 mgr.smithi038.khesbl (mgr.14162) 164 : cephadm [INF] Applying service osd.None on host smithi121... 2024-02-15T18:30:34.276 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:32.250542+0000 mgr.smithi038.khesbl (mgr.14162) 165 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:35.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:34.250855+0000 mgr.smithi038.khesbl (mgr.14162) 166 : cluster [DBG] pgmap v134: 1 pgs: 1 activating; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:35.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:30:34.405820+0000 mgr.smithi038.khesbl (mgr.14162) 167 : cephadm [INF] Detected new or changed devices on smithi121 2024-02-15T18:30:35.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:34.410143+0000 mon.smithi038 (mon.0) 364 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:35.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:34.415280+0000 mon.smithi038 (mon.0) 365 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:35.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:34.418180+0000 mon.smithi038 (mon.0) 366 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:30:35.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:35.224786+0000 mon.smithi121 (mon.1) 6 : audit [INF] from='client.? 172.21.15.121:0/2964031630' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e13d4f8-2233-4674-9b52-a3fb9dde34ed"}]: dispatch 2024-02-15T18:30:35.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:35.225390+0000 mon.smithi038 (mon.0) 367 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e13d4f8-2233-4674-9b52-a3fb9dde34ed"}]: dispatch 2024-02-15T18:30:35.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:35.229453+0000 mon.smithi038 (mon.0) 368 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e13d4f8-2233-4674-9b52-a3fb9dde34ed"}]': finished 2024-02-15T18:30:35.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:35.229502+0000 mon.smithi038 (mon.0) 369 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-02-15T18:30:35.777 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:35.229588+0000 mon.smithi038 (mon.0) 370 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:30:35.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:34.250855+0000 mgr.smithi038.khesbl (mgr.14162) 166 : cluster [DBG] pgmap v134: 1 pgs: 1 activating; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:35.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:30:34.405820+0000 mgr.smithi038.khesbl (mgr.14162) 167 : cephadm [INF] Detected new or changed devices on smithi121 2024-02-15T18:30:35.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:34.410143+0000 mon.smithi038 (mon.0) 364 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:35.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:34.415280+0000 mon.smithi038 (mon.0) 365 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:35.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:34.418180+0000 mon.smithi038 (mon.0) 366 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:30:35.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:35.224786+0000 mon.smithi121 (mon.1) 6 : audit [INF] from='client.? 172.21.15.121:0/2964031630' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e13d4f8-2233-4674-9b52-a3fb9dde34ed"}]: dispatch 2024-02-15T18:30:35.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:35.225390+0000 mon.smithi038 (mon.0) 367 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e13d4f8-2233-4674-9b52-a3fb9dde34ed"}]: dispatch 2024-02-15T18:30:35.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:35.229453+0000 mon.smithi038 (mon.0) 368 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e13d4f8-2233-4674-9b52-a3fb9dde34ed"}]': finished 2024-02-15T18:30:35.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:35.229502+0000 mon.smithi038 (mon.0) 369 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-02-15T18:30:35.861 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:35.229588+0000 mon.smithi038 (mon.0) 370 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:30:36.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:35.642633+0000 mon.smithi121 (mon.1) 7 : audit [DBG] from='client.? 172.21.15.121:0/1731632700' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:30:36.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:36 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:35.642633+0000 mon.smithi121 (mon.1) 7 : audit [DBG] from='client.? 172.21.15.121:0/1731632700' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:30:37.676 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:36.251293+0000 mgr.smithi038.khesbl (mgr.14162) 168 : cluster [DBG] pgmap v136: 1 pgs: 1 activating; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:37.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:36.251293+0000 mgr.smithi038.khesbl (mgr.14162) 168 : cluster [DBG] pgmap v136: 1 pgs: 1 activating; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:39.545 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:38.251654+0000 mgr.smithi038.khesbl (mgr.14162) 169 : cluster [DBG] pgmap v137: 1 pgs: 1 activating; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:39.545 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:39.171212+0000 mon.smithi038 (mon.0) 371 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:39.545 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:39.175870+0000 mon.smithi038 (mon.0) 372 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:39.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:38.251654+0000 mgr.smithi038.khesbl (mgr.14162) 169 : cluster [DBG] pgmap v137: 1 pgs: 1 activating; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:39.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:39.171212+0000 mon.smithi038 (mon.0) 371 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:39.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:39.175870+0000 mon.smithi038 (mon.0) 372 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:40.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:40.355052+0000 mon.smithi038 (mon.0) 373 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:30:40.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:40.355338+0000 mon.smithi038 (mon.0) 374 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:30:40.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:40.355052+0000 mon.smithi038 (mon.0) 373 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:30:40.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:40.355338+0000 mon.smithi038 (mon.0) 374 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:30:41.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:40.251986+0000 mgr.smithi038.khesbl (mgr.14162) 170 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:41.609 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:40.251986+0000 mgr.smithi038.khesbl (mgr.14162) 170 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:42.609 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:42.376341+0000 mon.smithi038 (mon.0) 375 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-15T18:30:42.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:42.377256+0000 mon.smithi038 (mon.0) 376 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:30:42.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:42.376341+0000 mon.smithi038 (mon.0) 375 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-15T18:30:42.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:42.377256+0000 mon.smithi038 (mon.0) 376 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:30:43.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:42.252370+0000 mgr.smithi038.khesbl (mgr.14162) 171 : cluster [DBG] pgmap v139: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:43.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:30:42.377958+0000 mgr.smithi038.khesbl (mgr.14162) 172 : cephadm [INF] Deploying daemon osd.4 on smithi121 2024-02-15T18:30:43.785 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:42.252370+0000 mgr.smithi038.khesbl (mgr.14162) 171 : cluster [DBG] pgmap v139: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:43.785 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:30:42.377958+0000 mgr.smithi038.khesbl (mgr.14162) 172 : cephadm [INF] Deploying daemon osd.4 on smithi121 2024-02-15T18:30:45.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:44.252754+0000 mgr.smithi038.khesbl (mgr.14162) 173 : cluster [DBG] pgmap v140: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:45.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:44.252754+0000 mgr.smithi038.khesbl (mgr.14162) 173 : cluster [DBG] pgmap v140: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:46.135 INFO:teuthology.orchestra.run.smithi121.stdout:Created osd(s) 4 on host 'smithi121' 2024-02-15T18:30:46.135 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:46.139+0000 7f9cf0ff9700 1 -- 172.21.15.121:0/3281719354 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9cf40ff030 con 0x7f9cdc05c2b0 2024-02-15T18:30:46.137 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:46.143+0000 7f9cfa9e1700 1 -- 172.21.15.121:0/3281719354 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9cdc05c2b0 msgr2=0x7f9cdc05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:46.137 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:46.143+0000 7f9cfa9e1700 1 --2- 172.21.15.121:0/3281719354 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9cdc05c2b0 0x7f9cdc05e770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f9ce800b790 tx=0x7f9ce8008040).stop 2024-02-15T18:30:46.137 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:46.143+0000 7f9cfa9e1700 1 -- 172.21.15.121:0/3281719354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cf40f66e0 msgr2=0x7f9cf41070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:46.137 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:46.143+0000 7f9cfa9e1700 1 --2- 172.21.15.121:0/3281719354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cf40f66e0 0x7f9cf41070a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f9ce4009f50 tx=0x7f9ce4009b30).stop 2024-02-15T18:30:46.137 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:46.143+0000 7f9cfa9e1700 1 -- 172.21.15.121:0/3281719354 shutdown_connections 2024-02-15T18:30:46.137 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:46.143+0000 7f9cfa9e1700 1 --2- 172.21.15.121:0/3281719354 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9cdc05c2b0 0x7f9cdc05e770 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:46.137 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:46.143+0000 7f9cfa9e1700 1 --2- 172.21.15.121:0/3281719354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cf40f66e0 0x7f9cf41070a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:46.137 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:46.143+0000 7f9cfa9e1700 1 --2- 172.21.15.121:0/3281719354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cf40f7980 0x7f9cf4107590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:46.138 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:46.143+0000 7f9cfa9e1700 1 -- 172.21.15.121:0/3281719354 >> 172.21.15.121:0/3281719354 conn(0x7f9cf40f1e40 msgr2=0x7f9cf40fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:46.138 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:46.143+0000 7f9cfa9e1700 1 -- 172.21.15.121:0/3281719354 shutdown_connections 2024-02-15T18:30:46.138 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:46.143+0000 7f9cfa9e1700 1 -- 172.21.15.121:0/3281719354 wait complete. 2024-02-15T18:30:46.763 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:46.124681+0000 mon.smithi038 (mon.0) 377 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:30:46.763 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:46.129882+0000 mon.smithi038 (mon.0) 378 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:46.763 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:46.140195+0000 mon.smithi038 (mon.0) 379 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:46.805 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:46 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:46.124681+0000 mon.smithi038 (mon.0) 377 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:30:46.806 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:46 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:46.129882+0000 mon.smithi038 (mon.0) 378 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:46.806 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:46 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:46.140195+0000 mon.smithi038 (mon.0) 379 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:46.832 DEBUG:teuthology.orchestra.run.smithi121:osd.4> sudo journalctl -f -n 0 -u ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.4.service 2024-02-15T18:30:46.834 INFO:tasks.cephadm:Deploying osd.5 on smithi121 with /dev/vg_nvme/lv_2... 2024-02-15T18:30:46.834 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-15T18:30:46.865 INFO:journalctl@ceph.osd.4.smithi121.stdout:-- Logs begin at Thu 2024-02-15 18:14:30 UTC. -- 2024-02-15T18:30:47.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:46.253029+0000 mgr.smithi038.khesbl (mgr.14162) 174 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:47.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:46.253029+0000 mgr.smithi038.khesbl (mgr.14162) 174 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:49.525 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:48.253296+0000 mgr.smithi038.khesbl (mgr.14162) 175 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:49.583 INFO:teuthology.orchestra.run.smithi121.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-15T18:30:49.583 INFO:teuthology.orchestra.run.smithi121.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-15T18:30:49.583 INFO:teuthology.orchestra.run.smithi121.stderr: stderr: 10+0 records in 2024-02-15T18:30:49.583 INFO:teuthology.orchestra.run.smithi121.stderr:10+0 records out 2024-02-15T18:30:49.583 INFO:teuthology.orchestra.run.smithi121.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0121234 s, 865 MB/s 2024-02-15T18:30:49.583 INFO:teuthology.orchestra.run.smithi121.stderr:--> Zapping successful for: 2024-02-15T18:30:49.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:48.253296+0000 mgr.smithi038.khesbl (mgr.14162) 175 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:50.359 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:30:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.4[48784]: debug 2024-02-15T18:30:50.002+0000 7f792a2ff080 -1 osd.4 0 log_to_monitors {default=true} 2024-02-15T18:30:50.703 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:50.006805+0000 mon.smithi121 (mon.1) 8 : audit [INF] from='osd.4 [v2:172.21.15.121:6808/3016245330,v1:172.21.15.121:6809/3016245330]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-15T18:30:50.704 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:50.007171+0000 mon.smithi038 (mon.0) 380 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-15T18:30:50.704 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:50.325671+0000 mon.smithi038 (mon.0) 381 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:50.724 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi121:vg_nvme/lv_2 2024-02-15T18:30:50.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:50.006805+0000 mon.smithi121 (mon.1) 8 : audit [INF] from='osd.4 [v2:172.21.15.121:6808/3016245330,v1:172.21.15.121:6809/3016245330]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-15T18:30:50.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:50.007171+0000 mon.smithi038 (mon.0) 380 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-15T18:30:50.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:50.325671+0000 mon.smithi038 (mon.0) 381 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:51.525 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:50.253585+0000 mgr.smithi038.khesbl (mgr.14162) 176 : cluster [DBG] pgmap v143: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:51.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:50.466726+0000 mon.smithi038 (mon.0) 382 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-15T18:30:51.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:50.466826+0000 mon.smithi038 (mon.0) 383 : cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2024-02-15T18:30:51.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:50.466952+0000 mon.smithi038 (mon.0) 384 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:30:51.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:50.467294+0000 mon.smithi121 (mon.1) 9 : audit [INF] from='osd.4 [v2:172.21.15.121:6808/3016245330,v1:172.21.15.121:6809/3016245330]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:30:51.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:50.467580+0000 mon.smithi038 (mon.0) 385 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:30:51.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:50.787624+0000 mon.smithi038 (mon.0) 386 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:51.610 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:30:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.4[48784]: debug 2024-02-15T18:30:51.474+0000 7f7920870700 -1 osd.4 0 waiting for initial osdmap 2024-02-15T18:30:51.610 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:30:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.4[48784]: debug 2024-02-15T18:30:51.480+0000 7f791cfec700 -1 osd.4 31 set_numa_affinity unable to identify public interface 'enp3s0f1' numa node: (0) Success 2024-02-15T18:30:51.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:50.253585+0000 mgr.smithi038.khesbl (mgr.14162) 176 : cluster [DBG] pgmap v143: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:51.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:50.466726+0000 mon.smithi038 (mon.0) 382 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-15T18:30:51.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:50.466826+0000 mon.smithi038 (mon.0) 383 : cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2024-02-15T18:30:51.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:50.466952+0000 mon.smithi038 (mon.0) 384 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:30:51.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:50.467294+0000 mon.smithi121 (mon.1) 9 : audit [INF] from='osd.4 [v2:172.21.15.121:6808/3016245330,v1:172.21.15.121:6809/3016245330]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:30:51.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:50.467580+0000 mon.smithi038 (mon.0) 385 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:30:51.611 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:50.787624+0000 mon.smithi038 (mon.0) 386 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:52.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:51.471386+0000 mon.smithi038 (mon.0) 387 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-02-15T18:30:52.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:51.471460+0000 mon.smithi038 (mon.0) 388 : cluster [DBG] osdmap e31: 5 total, 4 up, 5 in 2024-02-15T18:30:52.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:51.471843+0000 mon.smithi038 (mon.0) 389 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:30:52.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:51.475632+0000 mon.smithi038 (mon.0) 390 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:30:52.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:52.471896+0000 mon.smithi038 (mon.0) 391 : cluster [INF] osd.4 [v2:172.21.15.121:6808/3016245330,v1:172.21.15.121:6809/3016245330] boot 2024-02-15T18:30:52.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:52.471952+0000 mon.smithi038 (mon.0) 392 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-02-15T18:30:52.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:52.472086+0000 mon.smithi038 (mon.0) 393 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:30:52.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:52 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:51.471386+0000 mon.smithi038 (mon.0) 387 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-02-15T18:30:52.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:52 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:51.471460+0000 mon.smithi038 (mon.0) 388 : cluster [DBG] osdmap e31: 5 total, 4 up, 5 in 2024-02-15T18:30:52.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:52 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:51.471843+0000 mon.smithi038 (mon.0) 389 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:30:52.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:52 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:51.475632+0000 mon.smithi038 (mon.0) 390 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:30:52.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:52 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:52.471896+0000 mon.smithi038 (mon.0) 391 : cluster [INF] osd.4 [v2:172.21.15.121:6808/3016245330,v1:172.21.15.121:6809/3016245330] boot 2024-02-15T18:30:52.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:52 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:52.471952+0000 mon.smithi038 (mon.0) 392 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-02-15T18:30:52.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:52 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:52.472086+0000 mon.smithi038 (mon.0) 393 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:30:53.609 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:51.020733+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:30:53.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:51.020863+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:30:53.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:52.253931+0000 mgr.smithi038.khesbl (mgr.14162) 177 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:53.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:51.020733+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:30:53.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:51.020863+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:30:53.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:52.253931+0000 mgr.smithi038.khesbl (mgr.14162) 177 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:30:53.960 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.916+0000 7ff5b6cf3700 1 -- 172.21.15.121:0/1513781418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5b00f7980 msgr2=0x7ff5b00f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:53.960 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.916+0000 7ff5b6cf3700 1 --2- 172.21.15.121:0/1513781418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5b00f7980 0x7ff5b00f7de0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7ff59c004660 tx=0x7ff59c009b30).stop 2024-02-15T18:30:53.960 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.916+0000 7ff5b6cf3700 1 -- 172.21.15.121:0/1513781418 shutdown_connections 2024-02-15T18:30:53.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.916+0000 7ff5b6cf3700 1 --2- 172.21.15.121:0/1513781418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5b00f7980 0x7ff5b00f7de0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:53.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.916+0000 7ff5b6cf3700 1 --2- 172.21.15.121:0/1513781418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5b00f66e0 0x7ff5b00f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:30:53.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.916+0000 7ff5b6cf3700 1 -- 172.21.15.121:0/1513781418 >> 172.21.15.121:0/1513781418 conn(0x7ff5b00f1e40 msgr2=0x7ff5b00f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:53.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.916+0000 7ff5b6cf3700 1 -- 172.21.15.121:0/1513781418 shutdown_connections 2024-02-15T18:30:53.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.916+0000 7ff5b6cf3700 1 -- 172.21.15.121:0/1513781418 wait complete. 2024-02-15T18:30:53.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.917+0000 7ff5b6cf3700 1 Processor -- start 2024-02-15T18:30:53.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.917+0000 7ff5b6cf3700 1 -- start start 2024-02-15T18:30:53.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.917+0000 7ff5b6cf3700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5b00f66e0 0x7ff5b006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:53.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.917+0000 7ff5b6cf3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5b00f7980 0x7ff5b0069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:53.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.917+0000 7ff5b6cf3700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b006a410 con 0x7ff5b00f66e0 2024-02-15T18:30:53.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.917+0000 7ff5b6cf3700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b006a550 con 0x7ff5b00f7980 2024-02-15T18:30:53.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.917+0000 7ff5affff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5b00f7980 0x7ff5b0069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:53.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.917+0000 7ff5affff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5b00f7980 0x7ff5b0069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41352/0 (socket says 172.21.15.121:41352) 2024-02-15T18:30:53.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.917+0000 7ff5affff700 1 -- 172.21.15.121:0/3395104943 learned_addr learned my addr 172.21.15.121:0/3395104943 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-02-15T18:30:53.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.917+0000 7ff5b4a8f700 1 --2- 172.21.15.121:0/3395104943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5b00f66e0 0x7ff5b006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:53.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.917+0000 7ff5affff700 1 -- 172.21.15.121:0/3395104943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5b00f66e0 msgr2=0x7ff5b006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:53.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.917+0000 7ff5affff700 1 --2- 172.21.15.121:0/3395104943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5b00f66e0 0x7ff5b006b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:53.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.917+0000 7ff5affff700 1 -- 172.21.15.121:0/3395104943 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c005040 con 0x7ff5b00f7980 2024-02-15T18:30:53.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.917+0000 7ff5b4a8f700 1 --2- 172.21.15.121:0/3395104943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5b00f66e0 0x7ff5b006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:30:53.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.917+0000 7ff5affff700 1 --2- 172.21.15.121:0/3395104943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5b00f7980 0x7ff5b0069e20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7ff59c009830 tx=0x7ff59c009b30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:53.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.918+0000 7ff5ad7fa700 1 -- 172.21.15.121:0/3395104943 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff59c018070 con 0x7ff5b00f7980 2024-02-15T18:30:53.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.918+0000 7ff5b6cf3700 1 -- 172.21.15.121:0/3395104943 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b006a7d0 con 0x7ff5b00f7980 2024-02-15T18:30:53.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.918+0000 7ff5ad7fa700 1 -- 172.21.15.121:0/3395104943 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff59c007d70 con 0x7ff5b00f7980 2024-02-15T18:30:53.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.918+0000 7ff5ad7fa700 1 -- 172.21.15.121:0/3395104943 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff59c01c620 con 0x7ff5b00f7980 2024-02-15T18:30:53.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.918+0000 7ff5b6cf3700 1 -- 172.21.15.121:0/3395104943 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b010f0d0 con 0x7ff5b00f7980 2024-02-15T18:30:53.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.919+0000 7ff5b6cf3700 1 -- 172.21.15.121:0/3395104943 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b0045ad0 con 0x7ff5b00f7980 2024-02-15T18:30:53.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.919+0000 7ff5ad7fa700 1 -- 172.21.15.121:0/3395104943 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7ff59c01d980 con 0x7ff5b00f7980 2024-02-15T18:30:53.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.919+0000 7ff5ad7fa700 1 --2- 172.21.15.121:0/3395104943 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff5a005c2b0 0x7ff5a005e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:53.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.919+0000 7ff5ad7fa700 1 -- 172.21.15.121:0/3395104943 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4120+0+0 (secure 0 0 0) 0x7ff59c0778c0 con 0x7ff5b00f7980 2024-02-15T18:30:53.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.920+0000 7ff5b4a8f700 1 --2- 172.21.15.121:0/3395104943 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff5a005c2b0 0x7ff5a005e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:53.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.920+0000 7ff5b4a8f700 1 --2- 172.21.15.121:0/3395104943 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff5a005c2b0 0x7ff5a005e770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7ff5b006b730 tx=0x7ff5a4008040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:53.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:53.923+0000 7ff5ad7fa700 1 -- 172.21.15.121:0/3395104943 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ff59c04aa10 con 0x7ff5b00f7980 2024-02-15T18:30:54.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:30:54.104+0000 7ff5b6cf3700 1 -- 172.21.15.121:0/3395104943 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5b00ff160 con 0x7ff5a005c2b0 2024-02-15T18:30:54.702 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:53.483708+0000 mon.smithi038 (mon.0) 394 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-02-15T18:30:54.702 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:54.107753+0000 mon.smithi038 (mon.0) 395 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:30:54.702 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:54.109513+0000 mon.smithi038 (mon.0) 396 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:30:54.702 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:54.110294+0000 mon.smithi038 (mon.0) 397 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:30:54.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:53.483708+0000 mon.smithi038 (mon.0) 394 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-02-15T18:30:54.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:54.107753+0000 mon.smithi038 (mon.0) 395 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:30:54.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:54.109513+0000 mon.smithi038 (mon.0) 396 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:30:54.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:54.110294+0000 mon.smithi038 (mon.0) 397 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:30:55.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:54.106808+0000 mgr.smithi038.khesbl (mgr.14162) 178 : audit [DBG] from='client.24171 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:55.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:30:54.109236+0000 mgr.smithi038.khesbl (mgr.14162) 179 : cephadm [INF] Applying service osd.None on host smithi121... 2024-02-15T18:30:55.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:54.254401+0000 mgr.smithi038.khesbl (mgr.14162) 180 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:30:55.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:54.992087+0000 mon.smithi038 (mon.0) 398 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:55.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:54.106808+0000 mgr.smithi038.khesbl (mgr.14162) 178 : audit [DBG] from='client.24171 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:55.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:30:54.109236+0000 mgr.smithi038.khesbl (mgr.14162) 179 : cephadm [INF] Applying service osd.None on host smithi121... 2024-02-15T18:30:55.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:54.254401+0000 mgr.smithi038.khesbl (mgr.14162) 180 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:30:55.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:54.992087+0000 mon.smithi038 (mon.0) 398 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:56.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:30:54.987178+0000 mgr.smithi038.khesbl (mgr.14162) 181 : cephadm [INF] Detected new or changed devices on smithi121 2024-02-15T18:30:56.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:56.279102+0000 mon.smithi038 (mon.0) 399 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:56.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:56.282222+0000 mon.smithi038 (mon.0) 400 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:30:56.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:30:54.987178+0000 mgr.smithi038.khesbl (mgr.14162) 181 : cephadm [INF] Detected new or changed devices on smithi121 2024-02-15T18:30:56.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:56.279102+0000 mon.smithi038 (mon.0) 399 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:30:56.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:56.282222+0000 mon.smithi038 (mon.0) 400 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:30:57.769 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:56.254949+0000 mgr.smithi038.khesbl (mgr.14162) 182 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:30:57.769 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:56.974223+0000 mon.smithi121 (mon.1) 10 : audit [INF] from='client.? 172.21.15.121:0/1255311306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cd02c52-b9f4-494e-b67e-7019e4c224af"}]: dispatch 2024-02-15T18:30:57.769 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:56.974647+0000 mon.smithi038 (mon.0) 401 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cd02c52-b9f4-494e-b67e-7019e4c224af"}]: dispatch 2024-02-15T18:30:57.769 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:56.979722+0000 mon.smithi038 (mon.0) 402 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6cd02c52-b9f4-494e-b67e-7019e4c224af"}]': finished 2024-02-15T18:30:57.770 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:56.979784+0000 mon.smithi038 (mon.0) 403 : cluster [DBG] osdmap e34: 6 total, 5 up, 6 in 2024-02-15T18:30:57.770 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:56.979877+0000 mon.smithi038 (mon.0) 404 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:30:57.770 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:30:57.384342+0000 mon.smithi121 (mon.1) 11 : audit [DBG] from='client.? 172.21.15.121:0/860782558' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:30:57.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:56.254949+0000 mgr.smithi038.khesbl (mgr.14162) 182 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:30:57.894 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:56.974223+0000 mon.smithi121 (mon.1) 10 : audit [INF] from='client.? 172.21.15.121:0/1255311306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cd02c52-b9f4-494e-b67e-7019e4c224af"}]: dispatch 2024-02-15T18:30:57.894 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:56.974647+0000 mon.smithi038 (mon.0) 401 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cd02c52-b9f4-494e-b67e-7019e4c224af"}]: dispatch 2024-02-15T18:30:57.894 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:56.979722+0000 mon.smithi038 (mon.0) 402 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6cd02c52-b9f4-494e-b67e-7019e4c224af"}]': finished 2024-02-15T18:30:57.894 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:56.979784+0000 mon.smithi038 (mon.0) 403 : cluster [DBG] osdmap e34: 6 total, 5 up, 6 in 2024-02-15T18:30:57.895 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:56.979877+0000 mon.smithi038 (mon.0) 404 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:30:57.895 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:30:57.384342+0000 mon.smithi121 (mon.1) 11 : audit [DBG] from='client.? 172.21.15.121:0/860782558' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:30:59.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:30:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:30:58.255312+0000 mgr.smithi038.khesbl (mgr.14162) 183 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:30:59.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:30:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:30:58.255312+0000 mgr.smithi038.khesbl (mgr.14162) 183 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:31:01.609 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:00.255638+0000 mgr.smithi038.khesbl (mgr.14162) 184 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:31:01.609 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:00.928931+0000 mon.smithi038 (mon.0) 405 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:31:01.610 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:01.186515+0000 mon.smithi038 (mon.0) 406 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:31:01.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:00.255638+0000 mgr.smithi038.khesbl (mgr.14162) 184 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:31:01.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:00.928931+0000 mon.smithi038 (mon.0) 405 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:31:01.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:01.186515+0000 mon.smithi038 (mon.0) 406 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:31:03.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:02.256137+0000 mgr.smithi038.khesbl (mgr.14162) 185 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:31:03.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:02.256137+0000 mgr.smithi038.khesbl (mgr.14162) 185 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:31:05.541 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:04.256627+0000 mgr.smithi038.khesbl (mgr.14162) 186 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:31:05.541 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:04.343139+0000 mon.smithi038 (mon.0) 407 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-15T18:31:05.541 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:04.344098+0000 mon.smithi038 (mon.0) 408 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:31:05.541 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:31:04.344856+0000 mgr.smithi038.khesbl (mgr.14162) 187 : cephadm [INF] Deploying daemon osd.5 on smithi121 2024-02-15T18:31:05.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:04.256627+0000 mgr.smithi038.khesbl (mgr.14162) 186 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:31:05.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:04.343139+0000 mon.smithi038 (mon.0) 407 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-15T18:31:05.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:04.344098+0000 mon.smithi038 (mon.0) 408 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:31:05.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:31:04.344856+0000 mgr.smithi038.khesbl (mgr.14162) 187 : cephadm [INF] Deploying daemon osd.5 on smithi121 2024-02-15T18:31:07.609 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:06.257039+0000 mgr.smithi038.khesbl (mgr.14162) 188 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:31:07.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:06.257039+0000 mgr.smithi038.khesbl (mgr.14162) 188 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:31:08.159 INFO:teuthology.orchestra.run.smithi121.stdout:Created osd(s) 5 on host 'smithi121' 2024-02-15T18:31:08.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:08.165+0000 7ff5ad7fa700 1 -- 172.21.15.121:0/3395104943 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff5b00ff160 con 0x7ff5a005c2b0 2024-02-15T18:31:08.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:08.167+0000 7ff5b6cf3700 1 -- 172.21.15.121:0/3395104943 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff5a005c2b0 msgr2=0x7ff5a005e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:08.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:08.167+0000 7ff5b6cf3700 1 --2- 172.21.15.121:0/3395104943 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff5a005c2b0 0x7ff5a005e770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7ff5b006b730 tx=0x7ff5a4008040).stop 2024-02-15T18:31:08.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:08.167+0000 7ff5b6cf3700 1 -- 172.21.15.121:0/3395104943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5b00f7980 msgr2=0x7ff5b0069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:08.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:08.167+0000 7ff5b6cf3700 1 --2- 172.21.15.121:0/3395104943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5b00f7980 0x7ff5b0069e20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7ff59c009830 tx=0x7ff59c009b30).stop 2024-02-15T18:31:08.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:08.168+0000 7ff5b6cf3700 1 -- 172.21.15.121:0/3395104943 shutdown_connections 2024-02-15T18:31:08.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:08.168+0000 7ff5b6cf3700 1 --2- 172.21.15.121:0/3395104943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5b00f66e0 0x7ff5b006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:08.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:08.168+0000 7ff5b6cf3700 1 --2- 172.21.15.121:0/3395104943 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff5a005c2b0 0x7ff5a005e770 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:08.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:08.168+0000 7ff5b6cf3700 1 --2- 172.21.15.121:0/3395104943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5b00f7980 0x7ff5b0069e20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:08.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:08.168+0000 7ff5b6cf3700 1 -- 172.21.15.121:0/3395104943 >> 172.21.15.121:0/3395104943 conn(0x7ff5b00f1e40 msgr2=0x7ff5b00fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:08.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:08.168+0000 7ff5b6cf3700 1 -- 172.21.15.121:0/3395104943 shutdown_connections 2024-02-15T18:31:08.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:08.168+0000 7ff5b6cf3700 1 -- 172.21.15.121:0/3395104943 wait complete. 2024-02-15T18:31:08.478 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:08.150427+0000 mon.smithi038 (mon.0) 409 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:31:08.478 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:08.155071+0000 mon.smithi038 (mon.0) 410 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:31:08.478 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:08.164926+0000 mon.smithi038 (mon.0) 411 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:31:08.720 DEBUG:teuthology.orchestra.run.smithi121:osd.5> sudo journalctl -f -n 0 -u ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.5.service 2024-02-15T18:31:08.722 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-02-15T18:31:08.723 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph osd stat -f json 2024-02-15T18:31:08.752 INFO:journalctl@ceph.osd.5.smithi121.stdout:-- Logs begin at Thu 2024-02-15 18:14:30 UTC. -- 2024-02-15T18:31:08.752 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:08.150427+0000 mon.smithi038 (mon.0) 409 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:31:08.752 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:08.155071+0000 mon.smithi038 (mon.0) 410 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:31:08.752 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:08.164926+0000 mon.smithi038 (mon.0) 411 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:31:09.556 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:08.257343+0000 mgr.smithi038.khesbl (mgr.14162) 189 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:31:09.609 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:08.257343+0000 mgr.smithi038.khesbl (mgr.14162) 189 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:31:11.983 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:10.257630+0000 mgr.smithi038.khesbl (mgr.14162) 190 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:31:11.983 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:10.690906+0000 mon.smithi038 (mon.0) 412 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:31:12.001 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:10.257630+0000 mgr.smithi038.khesbl (mgr.14162) 190 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:31:12.001 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:10.690906+0000 mon.smithi038 (mon.0) 412 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:31:12.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.097+0000 7f5e587b8700 1 -- 172.21.15.38:0/2448248266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e500f7980 msgr2=0x7f5e500f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:12.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.097+0000 7f5e587b8700 1 --2- 172.21.15.38:0/2448248266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e500f7980 0x7f5e500f7de0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f5e44004660 tx=0x7f5e44009b30).stop 2024-02-15T18:31:12.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.097+0000 7f5e587b8700 1 -- 172.21.15.38:0/2448248266 shutdown_connections 2024-02-15T18:31:12.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.097+0000 7f5e587b8700 1 --2- 172.21.15.38:0/2448248266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e500f7980 0x7f5e500f7de0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:12.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.097+0000 7f5e587b8700 1 --2- 172.21.15.38:0/2448248266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e500f66e0 0x7f5e500f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:12.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.097+0000 7f5e587b8700 1 -- 172.21.15.38:0/2448248266 >> 172.21.15.38:0/2448248266 conn(0x7f5e500f1e40 msgr2=0x7f5e500f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:12.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.097+0000 7f5e587b8700 1 -- 172.21.15.38:0/2448248266 shutdown_connections 2024-02-15T18:31:12.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.097+0000 7f5e587b8700 1 -- 172.21.15.38:0/2448248266 wait complete. 2024-02-15T18:31:12.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.098+0000 7f5e587b8700 1 Processor -- start 2024-02-15T18:31:12.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.098+0000 7f5e587b8700 1 -- start start 2024-02-15T18:31:12.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.098+0000 7f5e587b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e500f66e0 0x7f5e50109220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:12.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.098+0000 7f5e587b8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e500f7980 0x7f5e50109710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:12.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.098+0000 7f5e587b8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e50109cc0 con 0x7f5e500f7980 2024-02-15T18:31:12.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.098+0000 7f5e587b8700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e50109e00 con 0x7f5e500f66e0 2024-02-15T18:31:12.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.098+0000 7f5e55d53700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e500f7980 0x7f5e50109710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:12.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.098+0000 7f5e55d53700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e500f7980 0x7f5e50109710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46148/0 (socket says 172.21.15.38:46148) 2024-02-15T18:31:12.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.098+0000 7f5e55d53700 1 -- 172.21.15.38:0/3387981680 learned_addr learned my addr 172.21.15.38:0/3387981680 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:31:12.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.098+0000 7f5e56554700 1 --2- 172.21.15.38:0/3387981680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e500f66e0 0x7f5e50109220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:12.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.099+0000 7f5e56554700 1 -- 172.21.15.38:0/3387981680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e500f7980 msgr2=0x7f5e50109710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:12.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.099+0000 7f5e56554700 1 --2- 172.21.15.38:0/3387981680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e500f7980 0x7f5e50109710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:12.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.099+0000 7f5e56554700 1 -- 172.21.15.38:0/3387981680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e44005040 con 0x7f5e500f66e0 2024-02-15T18:31:12.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.099+0000 7f5e55d53700 1 --2- 172.21.15.38:0/3387981680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e500f7980 0x7f5e50109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:31:12.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.099+0000 7f5e56554700 1 --2- 172.21.15.38:0/3387981680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e500f66e0 0x7f5e50109220 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f5e500f77e0 tx=0x7f5e4c00b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:12.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.099+0000 7f5e42ffd700 1 -- 172.21.15.38:0/3387981680 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e4c0092e0 con 0x7f5e500f66e0 2024-02-15T18:31:12.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.099+0000 7f5e42ffd700 1 -- 172.21.15.38:0/3387981680 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e4c00f040 con 0x7f5e500f66e0 2024-02-15T18:31:12.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.099+0000 7f5e587b8700 1 -- 172.21.15.38:0/3387981680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5010a7a0 con 0x7f5e500f66e0 2024-02-15T18:31:12.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.099+0000 7f5e42ffd700 1 -- 172.21.15.38:0/3387981680 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e4c014880 con 0x7f5e500f66e0 2024-02-15T18:31:12.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.099+0000 7f5e587b8700 1 -- 172.21.15.38:0/3387981680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5010ad60 con 0x7f5e500f66e0 2024-02-15T18:31:12.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.100+0000 7f5e587b8700 1 -- 172.21.15.38:0/3387981680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e50045ad0 con 0x7f5e500f66e0 2024-02-15T18:31:12.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.100+0000 7f5e42ffd700 1 -- 172.21.15.38:0/3387981680 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f5e4c01b030 con 0x7f5e500f66e0 2024-02-15T18:31:12.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.100+0000 7f5e42ffd700 1 --2- 172.21.15.38:0/3387981680 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f5e3c05c2b0 0x7f5e3c05e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:12.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.101+0000 7f5e42ffd700 1 -- 172.21.15.38:0/3387981680 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4235+0+0 (secure 0 0 0) 0x7f5e4c075990 con 0x7f5e500f66e0 2024-02-15T18:31:12.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.101+0000 7f5e55d53700 1 --2- 172.21.15.38:0/3387981680 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f5e3c05c2b0 0x7f5e3c05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:12.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.101+0000 7f5e55d53700 1 --2- 172.21.15.38:0/3387981680 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f5e3c05c2b0 0x7f5e3c05e770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f5e44009d60 tx=0x7f5e44016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:12.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.105+0000 7f5e42ffd700 1 -- 172.21.15.38:0/3387981680 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f5e4c048a10 con 0x7f5e500f66e0 2024-02-15T18:31:12.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.263+0000 7f5e587b8700 1 -- 172.21.15.38:0/3387981680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f5e500ff130 con 0x7f5e500f66e0 2024-02-15T18:31:12.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.263+0000 7f5e42ffd700 1 -- 172.21.15.38:0/3387981680 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v34) v1 ==== 74+0+130 (secure 0 0 0) 0x7f5e4c019020 con 0x7f5e500f66e0 2024-02-15T18:31:12.257 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:31:12.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.265+0000 7f5e587b8700 1 -- 172.21.15.38:0/3387981680 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f5e3c05c2b0 msgr2=0x7f5e3c05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:12.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.265+0000 7f5e587b8700 1 --2- 172.21.15.38:0/3387981680 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f5e3c05c2b0 0x7f5e3c05e770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f5e44009d60 tx=0x7f5e44016040).stop 2024-02-15T18:31:12.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.265+0000 7f5e587b8700 1 -- 172.21.15.38:0/3387981680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e500f66e0 msgr2=0x7f5e50109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:12.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.265+0000 7f5e587b8700 1 --2- 172.21.15.38:0/3387981680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e500f66e0 0x7f5e50109220 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f5e500f77e0 tx=0x7f5e4c00b040).stop 2024-02-15T18:31:12.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.265+0000 7f5e587b8700 1 -- 172.21.15.38:0/3387981680 shutdown_connections 2024-02-15T18:31:12.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.265+0000 7f5e587b8700 1 --2- 172.21.15.38:0/3387981680 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f5e3c05c2b0 0x7f5e3c05e770 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:12.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.265+0000 7f5e587b8700 1 --2- 172.21.15.38:0/3387981680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e500f66e0 0x7f5e50109220 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:12.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.265+0000 7f5e587b8700 1 --2- 172.21.15.38:0/3387981680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e500f7980 0x7f5e50109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:12.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.265+0000 7f5e587b8700 1 -- 172.21.15.38:0/3387981680 >> 172.21.15.38:0/3387981680 conn(0x7f5e500f1e40 msgr2=0x7f5e5006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:12.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.265+0000 7f5e587b8700 1 -- 172.21.15.38:0/3387981680 shutdown_connections 2024-02-15T18:31:12.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:12.265+0000 7f5e587b8700 1 -- 172.21.15.38:0/3387981680 wait complete. 2024-02-15T18:31:12.359 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:31:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.5[52426]: debug 2024-02-15T18:31:12.008+0000 7f97fbb15080 -1 osd.5 0 log_to_monitors {default=true} 2024-02-15T18:31:12.972 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:12.011726+0000 mon.smithi121 (mon.1) 12 : audit [INF] from='osd.5 [v2:172.21.15.121:6816/4062288290,v1:172.21.15.121:6817/4062288290]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-15T18:31:12.972 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:12.012078+0000 mon.smithi038 (mon.0) 413 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-15T18:31:12.972 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:12.265060+0000 mon.smithi121 (mon.1) 13 : audit [DBG] from='client.? 172.21.15.38:0/3387981680' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-15T18:31:13.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:12.011726+0000 mon.smithi121 (mon.1) 12 : audit [INF] from='osd.5 [v2:172.21.15.121:6816/4062288290,v1:172.21.15.121:6817/4062288290]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-15T18:31:13.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:12.012078+0000 mon.smithi038 (mon.0) 413 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-15T18:31:13.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:12.265060+0000 mon.smithi121 (mon.1) 13 : audit [DBG] from='client.? 172.21.15.38:0/3387981680' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-15T18:31:13.329 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":34,"num_osds":6,"num_up_osds":5,"osd_up_since":1708021852,"num_in_osds":6,"osd_in_since":1708021856,"num_remapped_pgs":0} 2024-02-15T18:31:14.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:12.257915+0000 mgr.smithi038.khesbl (mgr.14162) 191 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:31:14.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:12.706748+0000 mon.smithi038 (mon.0) 414 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-15T18:31:14.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:12.706796+0000 mon.smithi038 (mon.0) 415 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-15T18:31:14.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:12.706899+0000 mon.smithi038 (mon.0) 416 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:31:14.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:12.707287+0000 mon.smithi121 (mon.1) 14 : audit [INF] from='osd.5 [v2:172.21.15.121:6816/4062288290,v1:172.21.15.121:6817/4062288290]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:31:14.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:12.707574+0000 mon.smithi038 (mon.0) 417 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:31:14.047 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:31:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.5[52426]: debug 2024-02-15T18:31:13.709+0000 7f97f2086700 -1 osd.5 0 waiting for initial osdmap 2024-02-15T18:31:14.047 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:31:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.5[52426]: debug 2024-02-15T18:31:13.716+0000 7f97ec7fe700 -1 osd.5 36 set_numa_affinity unable to identify public interface 'enp3s0f1' numa node: (0) Success 2024-02-15T18:31:14.047 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:12.257915+0000 mgr.smithi038.khesbl (mgr.14162) 191 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:31:14.047 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:12.706748+0000 mon.smithi038 (mon.0) 414 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-15T18:31:14.047 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:12.706796+0000 mon.smithi038 (mon.0) 415 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-15T18:31:14.047 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:12.706899+0000 mon.smithi038 (mon.0) 416 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:31:14.047 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:12.707287+0000 mon.smithi121 (mon.1) 14 : audit [INF] from='osd.5 [v2:172.21.15.121:6816/4062288290,v1:172.21.15.121:6817/4062288290]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:31:14.048 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:12.707574+0000 mon.smithi038 (mon.0) 417 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:31:14.331 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph osd stat -f json 2024-02-15T18:31:15.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:13.707802+0000 mon.smithi038 (mon.0) 418 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-02-15T18:31:15.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:13.707900+0000 mon.smithi038 (mon.0) 419 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-02-15T18:31:15.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:13.710387+0000 mon.smithi038 (mon.0) 420 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:31:15.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:13.710833+0000 mon.smithi038 (mon.0) 421 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:31:15.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:14.285703+0000 mon.smithi038 (mon.0) 422 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:31:15.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:13.707802+0000 mon.smithi038 (mon.0) 418 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-02-15T18:31:15.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:13.707900+0000 mon.smithi038 (mon.0) 419 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-02-15T18:31:15.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:13.710387+0000 mon.smithi038 (mon.0) 420 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:31:15.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:13.710833+0000 mon.smithi038 (mon.0) 421 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:31:15.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:14.285703+0000 mon.smithi038 (mon.0) 422 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:31:16.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:13.005125+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:31:16.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:13.005229+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:31:16.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:14.258249+0000 mgr.smithi038.khesbl (mgr.14162) 192 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:31:16.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:31:14.276940+0000 mgr.smithi038.khesbl (mgr.14162) 193 : cephadm [INF] Detected new or changed devices on smithi121 2024-02-15T18:31:16.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:14.711204+0000 mon.smithi038 (mon.0) 423 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:31:16.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:14.721823+0000 mon.smithi038 (mon.0) 424 : cluster [INF] osd.5 [v2:172.21.15.121:6816/4062288290,v1:172.21.15.121:6817/4062288290] boot 2024-02-15T18:31:16.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:14.721873+0000 mon.smithi038 (mon.0) 425 : cluster [DBG] osdmap e37: 6 total, 6 up, 6 in 2024-02-15T18:31:16.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:14.722127+0000 mon.smithi038 (mon.0) 426 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:31:16.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:13.005125+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:31:16.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:13.005229+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:31:16.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:14.258249+0000 mgr.smithi038.khesbl (mgr.14162) 192 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:31:16.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:31:14.276940+0000 mgr.smithi038.khesbl (mgr.14162) 193 : cephadm [INF] Detected new or changed devices on smithi121 2024-02-15T18:31:16.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:14.711204+0000 mon.smithi038 (mon.0) 423 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:31:16.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:14.721823+0000 mon.smithi038 (mon.0) 424 : cluster [INF] osd.5 [v2:172.21.15.121:6816/4062288290,v1:172.21.15.121:6817/4062288290] boot 2024-02-15T18:31:16.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:14.721873+0000 mon.smithi038 (mon.0) 425 : cluster [DBG] osdmap e37: 6 total, 6 up, 6 in 2024-02-15T18:31:16.110 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:14.722127+0000 mon.smithi038 (mon.0) 426 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:31:16.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.822+0000 7fcc40154700 1 -- 172.21.15.38:0/558580384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc380f7980 msgr2=0x7fcc380f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.822+0000 7fcc40154700 1 --2- 172.21.15.38:0/558580384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc380f7980 0x7fcc380f7de0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fcc2c004660 tx=0x7fcc2c009b30).stop 2024-02-15T18:31:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.822+0000 7fcc40154700 1 -- 172.21.15.38:0/558580384 shutdown_connections 2024-02-15T18:31:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.822+0000 7fcc40154700 1 --2- 172.21.15.38:0/558580384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc380f7980 0x7fcc380f7de0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.822+0000 7fcc40154700 1 --2- 172.21.15.38:0/558580384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc380f66e0 0x7fcc380f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.822+0000 7fcc40154700 1 -- 172.21.15.38:0/558580384 >> 172.21.15.38:0/558580384 conn(0x7fcc380f1e40 msgr2=0x7fcc380f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.822+0000 7fcc40154700 1 -- 172.21.15.38:0/558580384 shutdown_connections 2024-02-15T18:31:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.822+0000 7fcc40154700 1 -- 172.21.15.38:0/558580384 wait complete. 2024-02-15T18:31:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.823+0000 7fcc40154700 1 Processor -- start 2024-02-15T18:31:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.823+0000 7fcc40154700 1 -- start start 2024-02-15T18:31:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.823+0000 7fcc40154700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc380f66e0 0x7fcc38107090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.823+0000 7fcc40154700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc380f7980 0x7fcc38107580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.823+0000 7fcc40154700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc38107b30 con 0x7fcc380f66e0 2024-02-15T18:31:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.823+0000 7fcc40154700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc38107c70 con 0x7fcc380f7980 2024-02-15T18:31:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.823+0000 7fcc3def0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc380f66e0 0x7fcc38107090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.823+0000 7fcc3def0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc380f66e0 0x7fcc38107090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46030/0 (socket says 172.21.15.38:46030) 2024-02-15T18:31:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.823+0000 7fcc3def0700 1 -- 172.21.15.38:0/2849577632 learned_addr learned my addr 172.21.15.38:0/2849577632 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:31:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.823+0000 7fcc3d6ef700 1 --2- 172.21.15.38:0/2849577632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc380f7980 0x7fcc38107580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.823+0000 7fcc3def0700 1 -- 172.21.15.38:0/2849577632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc380f7980 msgr2=0x7fcc38107580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.823+0000 7fcc3def0700 1 --2- 172.21.15.38:0/2849577632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc380f7980 0x7fcc38107580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.823+0000 7fcc3def0700 1 -- 172.21.15.38:0/2849577632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2c005040 con 0x7fcc380f66e0 2024-02-15T18:31:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.824+0000 7fcc3def0700 1 --2- 172.21.15.38:0/2849577632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc380f66e0 0x7fcc38107090 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fcc34002700 tx=0x7fcc34009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.824+0000 7fcc2a7fc700 1 -- 172.21.15.38:0/2849577632 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc34009320 con 0x7fcc380f66e0 2024-02-15T18:31:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.824+0000 7fcc40154700 1 -- 172.21.15.38:0/2849577632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc38108610 con 0x7fcc380f66e0 2024-02-15T18:31:16.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.824+0000 7fcc2a7fc700 1 -- 172.21.15.38:0/2849577632 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc3400f040 con 0x7fcc380f66e0 2024-02-15T18:31:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.824+0000 7fcc2a7fc700 1 -- 172.21.15.38:0/2849577632 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc34014940 con 0x7fcc380f66e0 2024-02-15T18:31:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.824+0000 7fcc40154700 1 -- 172.21.15.38:0/2849577632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc38108bd0 con 0x7fcc380f66e0 2024-02-15T18:31:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.825+0000 7fcc40154700 1 -- 172.21.15.38:0/2849577632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc38045ad0 con 0x7fcc380f66e0 2024-02-15T18:31:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.825+0000 7fcc2a7fc700 1 -- 172.21.15.38:0/2849577632 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fcc340077b0 con 0x7fcc380f66e0 2024-02-15T18:31:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.825+0000 7fcc2a7fc700 1 --2- 172.21.15.38:0/2849577632 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fcc2405c2b0 0x7fcc2405e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.825+0000 7fcc2a7fc700 1 -- 172.21.15.38:0/2849577632 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7fcc34075de0 con 0x7fcc380f66e0 2024-02-15T18:31:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.825+0000 7fcc3d6ef700 1 --2- 172.21.15.38:0/2849577632 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fcc2405c2b0 0x7fcc2405e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.826+0000 7fcc3d6ef700 1 --2- 172.21.15.38:0/2849577632 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fcc2405c2b0 0x7fcc2405e770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fcc2c004e30 tx=0x7fcc2c009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:16.830+0000 7fcc2a7fc700 1 -- 172.21.15.38:0/2849577632 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fcc34048de0 con 0x7fcc380f66e0 2024-02-15T18:31:16.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:17.002+0000 7fcc40154700 1 -- 172.21.15.38:0/2849577632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fcc380ff130 con 0x7fcc380f66e0 2024-02-15T18:31:16.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:17.002+0000 7fcc2a7fc700 1 -- 172.21.15.38:0/2849577632 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v38) v1 ==== 74+0+130 (secure 0 0 0) 0x7fcc3404c400 con 0x7fcc380f66e0 2024-02-15T18:31:16.996 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:31:16.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:17.004+0000 7fcc40154700 1 -- 172.21.15.38:0/2849577632 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fcc2405c2b0 msgr2=0x7fcc2405e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:16.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:17.004+0000 7fcc40154700 1 --2- 172.21.15.38:0/2849577632 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fcc2405c2b0 0x7fcc2405e770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fcc2c004e30 tx=0x7fcc2c009b30).stop 2024-02-15T18:31:16.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:17.004+0000 7fcc40154700 1 -- 172.21.15.38:0/2849577632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc380f66e0 msgr2=0x7fcc38107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:16.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:17.004+0000 7fcc40154700 1 --2- 172.21.15.38:0/2849577632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc380f66e0 0x7fcc38107090 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fcc34002700 tx=0x7fcc34009d80).stop 2024-02-15T18:31:16.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:17.004+0000 7fcc40154700 1 -- 172.21.15.38:0/2849577632 shutdown_connections 2024-02-15T18:31:16.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:17.004+0000 7fcc40154700 1 --2- 172.21.15.38:0/2849577632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc380f66e0 0x7fcc38107090 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:16.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:17.004+0000 7fcc40154700 1 --2- 172.21.15.38:0/2849577632 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fcc2405c2b0 0x7fcc2405e770 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:16.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:17.004+0000 7fcc40154700 1 --2- 172.21.15.38:0/2849577632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc380f7980 0x7fcc38107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:16.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:17.004+0000 7fcc40154700 1 -- 172.21.15.38:0/2849577632 >> 172.21.15.38:0/2849577632 conn(0x7fcc380f1e40 msgr2=0x7fcc380fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:16.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:17.004+0000 7fcc40154700 1 -- 172.21.15.38:0/2849577632 shutdown_connections 2024-02-15T18:31:16.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:17.004+0000 7fcc40154700 1 -- 172.21.15.38:0/2849577632 wait complete. 2024-02-15T18:31:17.018 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:15.736634+0000 mon.smithi038 (mon.0) 427 : cluster [DBG] osdmap e38: 6 total, 6 up, 6 in 2024-02-15T18:31:17.018 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:15.759522+0000 mon.smithi038 (mon.0) 428 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:31:17.018 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:15.765313+0000 mon.smithi038 (mon.0) 429 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:31:17.018 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:15.767926+0000 mon.smithi038 (mon.0) 430 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:31:17.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:15.736634+0000 mon.smithi038 (mon.0) 427 : cluster [DBG] osdmap e38: 6 total, 6 up, 6 in 2024-02-15T18:31:17.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:15.759522+0000 mon.smithi038 (mon.0) 428 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:31:17.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:15.765313+0000 mon.smithi038 (mon.0) 429 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:31:17.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:15.767926+0000 mon.smithi038 (mon.0) 430 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:31:18.016 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:16.258713+0000 mgr.smithi038.khesbl (mgr.14162) 194 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:18.016 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:17.004023+0000 mon.smithi038 (mon.0) 431 : audit [DBG] from='client.? 172.21.15.38:0/2849577632' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-15T18:31:18.017 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":38,"num_osds":6,"num_up_osds":6,"osd_up_since":1708021874,"num_in_osds":6,"osd_in_since":1708021856,"num_remapped_pgs":0} 2024-02-15T18:31:18.018 INFO:tasks.cephadm:Setting up client nodes... 2024-02-15T18:31:18.018 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-15T18:31:18.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:16.258713+0000 mgr.smithi038.khesbl (mgr.14162) 194 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:18.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:17.004023+0000 mon.smithi038 (mon.0) 431 : audit [DBG] from='client.? 172.21.15.38:0/2849577632' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-15T18:31:20.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:18.259322+0000 mgr.smithi038.khesbl (mgr.14162) 195 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:20.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:18.259322+0000 mgr.smithi038.khesbl (mgr.14162) 195 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:21.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.457+0000 7fb9831c7700 1 -- 172.21.15.38:0/1110801922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb97c0f66e0 msgr2=0x7fb97c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:21.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.457+0000 7fb9831c7700 1 --2- 172.21.15.38:0/1110801922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb97c0f66e0 0x7fb97c0f6b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fb970004660 tx=0x7fb970009b30).stop 2024-02-15T18:31:21.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.457+0000 7fb9831c7700 1 -- 172.21.15.38:0/1110801922 shutdown_connections 2024-02-15T18:31:21.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.457+0000 7fb9831c7700 1 --2- 172.21.15.38:0/1110801922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb97c0f7980 0x7fb97c0f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:21.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.457+0000 7fb9831c7700 1 --2- 172.21.15.38:0/1110801922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb97c0f66e0 0x7fb97c0f6b00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:21.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.457+0000 7fb9831c7700 1 -- 172.21.15.38:0/1110801922 >> 172.21.15.38:0/1110801922 conn(0x7fb97c0f1e40 msgr2=0x7fb97c0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:21.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.457+0000 7fb9831c7700 1 -- 172.21.15.38:0/1110801922 shutdown_connections 2024-02-15T18:31:21.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.457+0000 7fb9831c7700 1 -- 172.21.15.38:0/1110801922 wait complete. 2024-02-15T18:31:21.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.457+0000 7fb9831c7700 1 Processor -- start 2024-02-15T18:31:21.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.457+0000 7fb9831c7700 1 -- start start 2024-02-15T18:31:21.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.457+0000 7fb9831c7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb97c0f66e0 0x7fb97c1070b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:21.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.457+0000 7fb9831c7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb97c0f7980 0x7fb97c1075a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:21.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.458+0000 7fb9831c7700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c107b50 con 0x7fb97c0f66e0 2024-02-15T18:31:21.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.458+0000 7fb9831c7700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c107c90 con 0x7fb97c0f7980 2024-02-15T18:31:21.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.458+0000 7fb980f63700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb97c0f66e0 0x7fb97c1070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:21.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.458+0000 7fb980f63700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb97c0f66e0 0x7fb97c1070b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46048/0 (socket says 172.21.15.38:46048) 2024-02-15T18:31:21.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.458+0000 7fb980f63700 1 -- 172.21.15.38:0/1361591042 learned_addr learned my addr 172.21.15.38:0/1361591042 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:31:21.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.458+0000 7fb97bfff700 1 --2- 172.21.15.38:0/1361591042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb97c0f7980 0x7fb97c1075a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:21.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.458+0000 7fb980f63700 1 -- 172.21.15.38:0/1361591042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb97c0f7980 msgr2=0x7fb97c1075a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:21.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.458+0000 7fb980f63700 1 --2- 172.21.15.38:0/1361591042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb97c0f7980 0x7fb97c1075a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:21.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.458+0000 7fb980f63700 1 -- 172.21.15.38:0/1361591042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb970005040 con 0x7fb97c0f66e0 2024-02-15T18:31:21.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.458+0000 7fb97bfff700 1 --2- 172.21.15.38:0/1361591042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb97c0f7980 0x7fb97c1075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:31:21.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.458+0000 7fb980f63700 1 --2- 172.21.15.38:0/1361591042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb97c0f66e0 0x7fb97c1070b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fb970009f50 tx=0x7fb970009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:21.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.458+0000 7fb9797fa700 1 -- 172.21.15.38:0/1361591042 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb970018070 con 0x7fb97c0f66e0 2024-02-15T18:31:21.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.458+0000 7fb9797fa700 1 -- 172.21.15.38:0/1361591042 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb970007d70 con 0x7fb97c0f66e0 2024-02-15T18:31:21.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.458+0000 7fb9831c7700 1 -- 172.21.15.38:0/1361591042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97c1085d0 con 0x7fb97c0f66e0 2024-02-15T18:31:21.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.458+0000 7fb9797fa700 1 -- 172.21.15.38:0/1361591042 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb97001c620 con 0x7fb97c0f66e0 2024-02-15T18:31:21.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.458+0000 7fb9831c7700 1 -- 172.21.15.38:0/1361591042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97c108b30 con 0x7fb97c0f66e0 2024-02-15T18:31:21.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.459+0000 7fb9831c7700 1 -- 172.21.15.38:0/1361591042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97c045ad0 con 0x7fb97c0f66e0 2024-02-15T18:31:21.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.459+0000 7fb9797fa700 1 -- 172.21.15.38:0/1361591042 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fb97001c8e0 con 0x7fb97c0f66e0 2024-02-15T18:31:21.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.460+0000 7fb9797fa700 1 --2- 172.21.15.38:0/1361591042 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb96c05c2b0 0x7fb96c05e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:21.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.460+0000 7fb9797fa700 1 -- 172.21.15.38:0/1361591042 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7fb970077a10 con 0x7fb97c0f66e0 2024-02-15T18:31:21.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.460+0000 7fb97bfff700 1 --2- 172.21.15.38:0/1361591042 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb96c05c2b0 0x7fb96c05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:21.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.460+0000 7fb97bfff700 1 --2- 172.21.15.38:0/1361591042 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb96c05c2b0 0x7fb96c05e770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fb9680041d0 tx=0x7fb968009040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:21.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.464+0000 7fb9797fa700 1 -- 172.21.15.38:0/1361591042 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb97004aa10 con 0x7fb97c0f66e0 2024-02-15T18:31:21.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.682+0000 7fb9831c7700 1 -- 172.21.15.38:0/1361591042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fb97c0ff0f0 con 0x7fb97c0f66e0 2024-02-15T18:31:21.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.686+0000 7fb9797fa700 1 -- 172.21.15.38:0/1361591042 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7fb97004e030 con 0x7fb97c0f66e0 2024-02-15T18:31:21.680 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-02-15T18:31:21.680 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQB5WM5lom3GKBAAYb5BhqfT9rJHADQzPKD3DA== 2024-02-15T18:31:21.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.688+0000 7fb9831c7700 1 -- 172.21.15.38:0/1361591042 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb96c05c2b0 msgr2=0x7fb96c05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:21.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.688+0000 7fb9831c7700 1 --2- 172.21.15.38:0/1361591042 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb96c05c2b0 0x7fb96c05e770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fb9680041d0 tx=0x7fb968009040).stop 2024-02-15T18:31:21.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.688+0000 7fb9831c7700 1 -- 172.21.15.38:0/1361591042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb97c0f66e0 msgr2=0x7fb97c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:21.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.688+0000 7fb9831c7700 1 --2- 172.21.15.38:0/1361591042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb97c0f66e0 0x7fb97c1070b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fb970009f50 tx=0x7fb970009b30).stop 2024-02-15T18:31:21.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.688+0000 7fb9831c7700 1 -- 172.21.15.38:0/1361591042 shutdown_connections 2024-02-15T18:31:21.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.688+0000 7fb9831c7700 1 --2- 172.21.15.38:0/1361591042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb97c0f66e0 0x7fb97c1070b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:21.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.688+0000 7fb9831c7700 1 --2- 172.21.15.38:0/1361591042 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb96c05c2b0 0x7fb96c05e770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:21.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.688+0000 7fb9831c7700 1 --2- 172.21.15.38:0/1361591042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb97c0f7980 0x7fb97c1075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:21.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.688+0000 7fb9831c7700 1 -- 172.21.15.38:0/1361591042 >> 172.21.15.38:0/1361591042 conn(0x7fb97c0f1e40 msgr2=0x7fb97c0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:21.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.688+0000 7fb9831c7700 1 -- 172.21.15.38:0/1361591042 shutdown_connections 2024-02-15T18:31:21.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:21.688+0000 7fb9831c7700 1 -- 172.21.15.38:0/1361591042 wait complete. 2024-02-15T18:31:22.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:20.259791+0000 mgr.smithi038.khesbl (mgr.14162) 196 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:22.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:21.683933+0000 mon.smithi038 (mon.0) 432 : audit [INF] from='client.? 172.21.15.38:0/1361591042' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-15T18:31:22.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:21.687613+0000 mon.smithi038 (mon.0) 433 : audit [INF] from='client.? 172.21.15.38:0/1361591042' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-15T18:31:22.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:20.259791+0000 mgr.smithi038.khesbl (mgr.14162) 196 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:22.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:21.683933+0000 mon.smithi038 (mon.0) 432 : audit [INF] from='client.? 172.21.15.38:0/1361591042' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-15T18:31:22.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:21.687613+0000 mon.smithi038 (mon.0) 433 : audit [INF] from='client.? 172.21.15.38:0/1361591042' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-15T18:31:23.311 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-02-15T18:31:23.312 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-15T18:31:23.312 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-15T18:31:23.360 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-15T18:31:24.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:22.260107+0000 mgr.smithi038.khesbl (mgr.14162) 197 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:24.077 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:22.260107+0000 mgr.smithi038.khesbl (mgr.14162) 197 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:25.356 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.362+0000 7f4f9251a700 1 -- 172.21.15.121:0/1479029940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f8c0f7980 msgr2=0x7f4f8c0f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:25.356 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.362+0000 7f4f9251a700 1 --2- 172.21.15.121:0/1479029940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f8c0f7980 0x7f4f8c0f7de0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f4f7c004660 tx=0x7f4f7c009b30).stop 2024-02-15T18:31:25.356 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.362+0000 7f4f9251a700 1 -- 172.21.15.121:0/1479029940 shutdown_connections 2024-02-15T18:31:25.356 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.362+0000 7f4f9251a700 1 --2- 172.21.15.121:0/1479029940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f8c0f7980 0x7f4f8c0f7de0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:25.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.362+0000 7f4f9251a700 1 --2- 172.21.15.121:0/1479029940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f8c0f66e0 0x7f4f8c0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:31:25.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.362+0000 7f4f9251a700 1 -- 172.21.15.121:0/1479029940 >> 172.21.15.121:0/1479029940 conn(0x7f4f8c0f1e40 msgr2=0x7f4f8c0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:25.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.362+0000 7f4f9251a700 1 -- 172.21.15.121:0/1479029940 shutdown_connections 2024-02-15T18:31:25.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.362+0000 7f4f9251a700 1 -- 172.21.15.121:0/1479029940 wait complete. 2024-02-15T18:31:25.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.363+0000 7f4f9251a700 1 Processor -- start 2024-02-15T18:31:25.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.363+0000 7f4f9251a700 1 -- start start 2024-02-15T18:31:25.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.363+0000 7f4f9251a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f8c0f66e0 0x7f4f8c107080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:25.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.363+0000 7f4f9251a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f8c0f7980 0x7f4f8c107570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:25.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.363+0000 7f4f9251a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c107bb0 con 0x7f4f8c0f66e0 2024-02-15T18:31:25.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.363+0000 7f4f9251a700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c108460 con 0x7f4f8c0f7980 2024-02-15T18:31:25.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.363+0000 7f4f8b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f8c0f7980 0x7f4f8c107570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:25.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.363+0000 7f4f8b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f8c0f7980 0x7f4f8c107570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54930/0 (socket says 172.21.15.121:54930) 2024-02-15T18:31:25.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.363+0000 7f4f8b7fe700 1 -- 172.21.15.121:0/2325691927 learned_addr learned my addr 172.21.15.121:0/2325691927 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-02-15T18:31:25.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.364+0000 7f4f8bfff700 1 --2- 172.21.15.121:0/2325691927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f8c0f66e0 0x7f4f8c107080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:25.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.364+0000 7f4f8b7fe700 1 -- 172.21.15.121:0/2325691927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f8c0f66e0 msgr2=0x7f4f8c107080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:25.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.364+0000 7f4f8b7fe700 1 --2- 172.21.15.121:0/2325691927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f8c0f66e0 0x7f4f8c107080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:25.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.364+0000 7f4f8b7fe700 1 -- 172.21.15.121:0/2325691927 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7c005040 con 0x7f4f8c0f7980 2024-02-15T18:31:25.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.364+0000 7f4f8bfff700 1 --2- 172.21.15.121:0/2325691927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f8c0f66e0 0x7f4f8c107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:31:25.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.364+0000 7f4f8b7fe700 1 --2- 172.21.15.121:0/2325691927 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f8c0f7980 0x7f4f8c107570 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f4f7c009830 tx=0x7f4f7c009b30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:25.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.364+0000 7f4f88ff9700 1 -- 172.21.15.121:0/2325691927 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f7c018070 con 0x7f4f8c0f7980 2024-02-15T18:31:25.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.364+0000 7f4f9251a700 1 -- 172.21.15.121:0/2325691927 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8c1086e0 con 0x7f4f8c0f7980 2024-02-15T18:31:25.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.364+0000 7f4f88ff9700 1 -- 172.21.15.121:0/2325691927 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f7c007d70 con 0x7f4f8c0f7980 2024-02-15T18:31:25.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.364+0000 7f4f88ff9700 1 -- 172.21.15.121:0/2325691927 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f7c01c620 con 0x7f4f8c0f7980 2024-02-15T18:31:25.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.364+0000 7f4f9251a700 1 -- 172.21.15.121:0/2325691927 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8c108c40 con 0x7f4f8c0f7980 2024-02-15T18:31:25.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.365+0000 7f4f9251a700 1 -- 172.21.15.121:0/2325691927 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f8c0f5ad0 con 0x7f4f8c0f7980 2024-02-15T18:31:25.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.365+0000 7f4f88ff9700 1 -- 172.21.15.121:0/2325691927 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f4f7c01d980 con 0x7f4f8c0f7980 2024-02-15T18:31:25.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.366+0000 7f4f88ff9700 1 --2- 172.21.15.121:0/2325691927 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f4f7805c2b0 0x7f4f7805e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:25.360 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:24.260440+0000 mgr.smithi038.khesbl (mgr.14162) 198 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:25.360 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:24.301258+0000 mon.smithi038 (mon.0) 434 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:31:25.361 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:24.304243+0000 mon.smithi038 (mon.0) 435 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:31:25.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.366+0000 7f4f88ff9700 1 -- 172.21.15.121:0/2325691927 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f4f7c077a70 con 0x7f4f8c0f7980 2024-02-15T18:31:25.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.366+0000 7f4f8bfff700 1 --2- 172.21.15.121:0/2325691927 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f4f7805c2b0 0x7f4f7805e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:25.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.366+0000 7f4f8bfff700 1 --2- 172.21.15.121:0/2325691927 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f4f7805c2b0 0x7f4f7805e770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f4f8c0f77e0 tx=0x7f4f74008040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:25.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.369+0000 7f4f88ff9700 1 -- 172.21.15.121:0/2325691927 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f4f7c04aa10 con 0x7f4f8c0f7980 2024-02-15T18:31:25.568 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.575+0000 7f4f9251a700 1 -- 172.21.15.121:0/2325691927 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f4f8c0ff050 con 0x7f4f8c0f7980 2024-02-15T18:31:25.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.580+0000 7f4f88ff9700 1 -- 172.21.15.121:0/2325691927 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f4f7c077e60 con 0x7f4f8c0f7980 2024-02-15T18:31:25.573 INFO:teuthology.orchestra.run.smithi121.stdout:[client.1] 2024-02-15T18:31:25.574 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQB9WM5lgURvIhAAfjIGybxpFHg83+BZdWaU+Q== 2024-02-15T18:31:25.575 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.582+0000 7f4f9251a700 1 -- 172.21.15.121:0/2325691927 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f4f7805c2b0 msgr2=0x7f4f7805e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:25.575 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.582+0000 7f4f9251a700 1 --2- 172.21.15.121:0/2325691927 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f4f7805c2b0 0x7f4f7805e770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f4f8c0f77e0 tx=0x7f4f74008040).stop 2024-02-15T18:31:25.575 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.582+0000 7f4f9251a700 1 -- 172.21.15.121:0/2325691927 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f8c0f7980 msgr2=0x7f4f8c107570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:25.575 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.582+0000 7f4f9251a700 1 --2- 172.21.15.121:0/2325691927 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f8c0f7980 0x7f4f8c107570 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f4f7c009830 tx=0x7f4f7c009b30).stop 2024-02-15T18:31:25.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.582+0000 7f4f9251a700 1 -- 172.21.15.121:0/2325691927 shutdown_connections 2024-02-15T18:31:25.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.582+0000 7f4f9251a700 1 --2- 172.21.15.121:0/2325691927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f8c0f66e0 0x7f4f8c107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:25.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.582+0000 7f4f9251a700 1 --2- 172.21.15.121:0/2325691927 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f4f7805c2b0 0x7f4f7805e770 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:25.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.582+0000 7f4f9251a700 1 --2- 172.21.15.121:0/2325691927 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f8c0f7980 0x7f4f8c107570 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:25.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.582+0000 7f4f9251a700 1 -- 172.21.15.121:0/2325691927 >> 172.21.15.121:0/2325691927 conn(0x7f4f8c0f1e40 msgr2=0x7f4f8c0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:25.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.582+0000 7f4f9251a700 1 -- 172.21.15.121:0/2325691927 shutdown_connections 2024-02-15T18:31:25.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:31:25.582+0000 7f4f9251a700 1 -- 172.21.15.121:0/2325691927 wait complete. 2024-02-15T18:31:25.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:24.260440+0000 mgr.smithi038.khesbl (mgr.14162) 198 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:25.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:24.301258+0000 mon.smithi038 (mon.0) 434 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:31:25.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:24.304243+0000 mon.smithi038 (mon.0) 435 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:31:26.280 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-02-15T18:31:26.280 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-15T18:31:26.280 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-15T18:31:26.321 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-15T18:31:26.565 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/config 2024-02-15T18:31:26.609 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:25.576885+0000 mon.smithi121 (mon.1) 15 : audit [INF] from='client.? 172.21.15.121:0/2325691927' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-15T18:31:26.609 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:25.577544+0000 mon.smithi038 (mon.0) 436 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-15T18:31:26.609 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:25.581435+0000 mon.smithi038 (mon.0) 437 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-15T18:31:26.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:25.576885+0000 mon.smithi121 (mon.1) 15 : audit [INF] from='client.? 172.21.15.121:0/2325691927' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-15T18:31:26.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:25.577544+0000 mon.smithi038 (mon.0) 436 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-15T18:31:26.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:25.581435+0000 mon.smithi038 (mon.0) 437 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-15T18:31:27.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:26.261068+0000 mgr.smithi038.khesbl (mgr.14162) 199 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:27.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:26.261068+0000 mgr.smithi038.khesbl (mgr.14162) 199 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:28.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.397+0000 7f6c97fff700 1 -- 172.21.15.38:0/1929484677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c980f67b0 msgr2=0x7f6c980f6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:28.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.397+0000 7f6c97fff700 1 --2- 172.21.15.38:0/1929484677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c980f67b0 0x7f6c980f6c10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f6c80004660 tx=0x7f6c80009b30).stop 2024-02-15T18:31:28.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.397+0000 7f6c97fff700 1 -- 172.21.15.38:0/1929484677 shutdown_connections 2024-02-15T18:31:28.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.397+0000 7f6c97fff700 1 --2- 172.21.15.38:0/1929484677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c980f67b0 0x7f6c980f6c10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:28.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.397+0000 7f6c97fff700 1 --2- 172.21.15.38:0/1929484677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c980f5510 0x7f6c980f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:31:28.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.397+0000 7f6c97fff700 1 -- 172.21.15.38:0/1929484677 >> 172.21.15.38:0/1929484677 conn(0x7f6c980f0c70 msgr2=0x7f6c980f3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:28.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.397+0000 7f6c97fff700 1 -- 172.21.15.38:0/1929484677 shutdown_connections 2024-02-15T18:31:28.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.397+0000 7f6c97fff700 1 -- 172.21.15.38:0/1929484677 wait complete. 2024-02-15T18:31:28.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.397+0000 7f6c97fff700 1 Processor -- start 2024-02-15T18:31:28.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.398+0000 7f6c97fff700 1 -- start start 2024-02-15T18:31:28.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.398+0000 7f6c97fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c980f5510 0x7f6c98109cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:28.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.398+0000 7f6c97fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c980f67b0 0x7f6c9810c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:28.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.398+0000 7f6c97fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c980609b0 con 0x7f6c980f5510 2024-02-15T18:31:28.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.398+0000 7f6c97fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c98060af0 con 0x7f6c980f67b0 2024-02-15T18:31:28.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.398+0000 7f6c96ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c980f5510 0x7f6c98109cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:28.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.398+0000 7f6c96ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c980f5510 0x7f6c98109cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49844/0 (socket says 172.21.15.38:49844) 2024-02-15T18:31:28.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.398+0000 7f6c96ffd700 1 -- 172.21.15.38:0/1505138618 learned_addr learned my addr 172.21.15.38:0/1505138618 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:31:28.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.398+0000 7f6c967fc700 1 --2- 172.21.15.38:0/1505138618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c980f67b0 0x7f6c9810c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:28.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.398+0000 7f6c96ffd700 1 -- 172.21.15.38:0/1505138618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c980f67b0 msgr2=0x7f6c9810c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:28.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.398+0000 7f6c96ffd700 1 --2- 172.21.15.38:0/1505138618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c980f67b0 0x7f6c9810c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:28.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.398+0000 7f6c96ffd700 1 -- 172.21.15.38:0/1505138618 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c80005040 con 0x7f6c980f5510 2024-02-15T18:31:28.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.398+0000 7f6c96ffd700 1 --2- 172.21.15.38:0/1505138618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c980f5510 0x7f6c98109cf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f6c880093f0 tx=0x7f6c88006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:28.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.399+0000 7f6c8f7fe700 1 -- 172.21.15.38:0/1505138618 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c88013640 con 0x7f6c980f5510 2024-02-15T18:31:28.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.399+0000 7f6c97fff700 1 -- 172.21.15.38:0/1505138618 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c9810c800 con 0x7f6c980f5510 2024-02-15T18:31:28.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.399+0000 7f6c8f7fe700 1 -- 172.21.15.38:0/1505138618 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c88009e40 con 0x7f6c980f5510 2024-02-15T18:31:28.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.399+0000 7f6c8f7fe700 1 -- 172.21.15.38:0/1505138618 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c8800dc90 con 0x7f6c980f5510 2024-02-15T18:31:28.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.399+0000 7f6c97fff700 1 -- 172.21.15.38:0/1505138618 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c9810cdc0 con 0x7f6c980f5510 2024-02-15T18:31:28.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.399+0000 7f6c97fff700 1 -- 172.21.15.38:0/1505138618 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c98045ad0 con 0x7f6c980f5510 2024-02-15T18:31:28.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.400+0000 7f6c8f7fe700 1 -- 172.21.15.38:0/1505138618 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f6c8801a060 con 0x7f6c980f5510 2024-02-15T18:31:28.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.401+0000 7f6c8f7fe700 1 --2- 172.21.15.38:0/1505138618 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6c8405c2b0 0x7f6c8405e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:28.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.401+0000 7f6c8f7fe700 1 -- 172.21.15.38:0/1505138618 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f6c88075fb0 con 0x7f6c980f5510 2024-02-15T18:31:28.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.401+0000 7f6c967fc700 1 --2- 172.21.15.38:0/1505138618 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6c8405c2b0 0x7f6c8405e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:28.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.401+0000 7f6c967fc700 1 --2- 172.21.15.38:0/1505138618 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6c8405c2b0 0x7f6c8405e770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f6c80004e30 tx=0x7f6c80009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:28.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.404+0000 7f6c8f7fe700 1 -- 172.21.15.38:0/1505138618 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f6c880490a0 con 0x7f6c980f5510 2024-02-15T18:31:28.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.562+0000 7f6c97fff700 1 -- 172.21.15.38:0/1505138618 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c98027a00 con 0x7f6c980f5510 2024-02-15T18:31:28.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.562+0000 7f6c8f7fe700 1 -- 172.21.15.38:0/1505138618 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+10923 (secure 0 0 0) 0x7f6c8804c6c0 con 0x7f6c980f5510 2024-02-15T18:31:28.556 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:31:28.556 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":38,"fsid":"7e05753e-cc2f-11ee-95ba-87774f69a715","created":"2024-02-15T18:25:33.849561+0000","modified":"2024-02-15T18:31:15.719708+0000","last_up_change":"2024-02-15T18:31:14.711602+0000","last_in_change":"2024-02-15T18:30:56.975140+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-15T18:30:09.268924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"eee9713b-ff79-4169-8c04-b1def80d2045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1826650694},{"type":"v1","addr":"172.21.15.38:6803","nonce":1826650694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1826650694},{"type":"v1","addr":"172.21.15.38:6805","nonce":1826650694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1826650694},{"type":"v1","addr":"172.21.15.38:6809","nonce":1826650694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1826650694},{"type":"v1","addr":"172.21.15.38:6807","nonce":1826650694}]},"public_addr":"172.21.15.38:6803/1826650694","cluster_addr":"172.21.15.38:6805/1826650694","heartbeat_back_addr":"172.21.15.38:6809/1826650694","heartbeat_front_addr":"172.21.15.38:6807/1826650694","state":["exists","up"]},{"osd":1,"uuid":"740eed48-d34e-4e71-a859-035f01b1fdd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3796697403},{"type":"v1","addr":"172.21.15.38:6811","nonce":3796697403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3796697403},{"type":"v1","addr":"172.21.15.38:6813","nonce":3796697403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3796697403},{"type":"v1","addr":"172.21.15.38:6817","nonce":3796697403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3796697403},{"type":"v1","addr":"172.21.15.38:6815","nonce":3796697403}]},"public_addr":"172.21.15.38:6811/3796697403","cluster_addr":"172.21.15.38:6813/3796697403","heartbeat_back_addr":"172.21.15.38:6817/3796697403","heartbeat_front_addr":"172.21.15.38:6815/3796697403","state":["exists","up"]},{"osd":2,"uuid":"8a397970-64c0-47d7-94c8-77fcf9fe579d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":420320553},{"type":"v1","addr":"172.21.15.38:6819","nonce":420320553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":420320553},{"type":"v1","addr":"172.21.15.38:6821","nonce":420320553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":420320553},{"type":"v1","addr":"172.21.15.38:6825","nonce":420320553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":420320553},{"type":"v1","addr":"172.21.15.38:6823","nonce":420320553}]},"public_addr":"172.21.15.38:6819/420320553","cluster_addr":"172.21.15.38:6821/420320553","heartbeat_back_addr":"172.21.15.38:6825/420320553","heartbeat_front_addr":"172.21.15.38:6823/420320553","state":["exists","up"]},{"osd":3,"uuid":"0a590f02-56f3-43e7-894f-741f95d2facd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3071118694},{"type":"v1","addr":"172.21.15.121:6801","nonce":3071118694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3071118694},{"type":"v1","addr":"172.21.15.121:6803","nonce":3071118694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3071118694},{"type":"v1","addr":"172.21.15.121:6807","nonce":3071118694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3071118694},{"type":"v1","addr":"172.21.15.121:6805","nonce":3071118694}]},"public_addr":"172.21.15.121:6801/3071118694","cluster_addr":"172.21.15.121:6803/3071118694","heartbeat_back_addr":"172.21.15.121:6807/3071118694","heartbeat_front_addr":"172.21.15.121:6805/3071118694","state":["exists","up"]},{"osd":4,"uuid":"5e13d4f8-2233-4674-9b52-a3fb9dde34ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3016245330},{"type":"v1","addr":"172.21.15.121:6809","nonce":3016245330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3016245330},{"type":"v1","addr":"172.21.15.121:6811","nonce":3016245330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3016245330},{"type":"v1","addr":"172.21.15.121:6815","nonce":3016245330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3016245330},{"type":"v1","addr":"172.21.15.121:6813","nonce":3016245330}]},"public_addr":"172.21.15.121:6809/3016245330","cluster_addr":"172.21.15.121:6811/3016245330","heartbeat_back_addr":"172.21.15.121:6815/3016245330","heartbeat_front_addr":"172.21.15.121:6813/3016245330","state":["exists","up"]},{"osd":5,"uuid":"6cd02c52-b9f4-494e-b67e-7019e4c224af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4062288290},{"type":"v1","addr":"172.21.15.121:6817","nonce":4062288290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4062288290},{"type":"v1","addr":"172.21.15.121:6819","nonce":4062288290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4062288290},{"type":"v1","addr":"172.21.15.121:6823","nonce":4062288290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4062288290},{"type":"v1","addr":"172.21.15.121:6821","nonce":4062288290}]},"public_addr":"172.21.15.121:6817/4062288290","cluster_addr":"172.21.15.121:6819/4062288290","heartbeat_back_addr":"172.21.15.121:6823/4062288290","heartbeat_front_addr":"172.21.15.121:6821/4062288290","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:29:06.518109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:29:40.582228+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:30:07.799238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:30:29.057360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:30:51.020867+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:31:13.005231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/3518675152":"2024-02-16T18:26:40.033295+0000","172.21.15.38:0/1774333839":"2024-02-16T18:26:40.033295+0000","172.21.15.38:6800/2095530547":"2024-02-16T18:26:40.033295+0000","172.21.15.38:0/2413778685":"2024-02-16T18:26:23.102293+0000","172.21.15.38:6801/3009942445":"2024-02-16T18:26:23.102293+0000","172.21.15.38:0/788545352":"2024-02-16T18:26:23.102293+0000","172.21.15.38:6800/3009942445":"2024-02-16T18:26:23.102293+0000","172.21.15.38:0/3436541874":"2024-02-16T18:26:02.772407+0000","172.21.15.38:0/4035950047":"2024-02-16T18:26:02.772407+0000","172.21.15.38:6801/2095530547":"2024-02-16T18:26:40.033295+0000","172.21.15.38:6800/4189944318":"2024-02-16T18:26:02.772407+0000","172.21.15.38:6801/4189944318":"2024-02-16T18:26:02.772407+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-15T18:31:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.564+0000 7f6c97fff700 1 -- 172.21.15.38:0/1505138618 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6c8405c2b0 msgr2=0x7f6c8405e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.564+0000 7f6c97fff700 1 --2- 172.21.15.38:0/1505138618 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6c8405c2b0 0x7f6c8405e770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f6c80004e30 tx=0x7f6c80009b30).stop 2024-02-15T18:31:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.564+0000 7f6c97fff700 1 -- 172.21.15.38:0/1505138618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c980f5510 msgr2=0x7f6c98109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.564+0000 7f6c97fff700 1 --2- 172.21.15.38:0/1505138618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c980f5510 0x7f6c98109cf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f6c880093f0 tx=0x7f6c88006ec0).stop 2024-02-15T18:31:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.564+0000 7f6c97fff700 1 -- 172.21.15.38:0/1505138618 shutdown_connections 2024-02-15T18:31:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.564+0000 7f6c97fff700 1 --2- 172.21.15.38:0/1505138618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c980f5510 0x7f6c98109cf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.564+0000 7f6c97fff700 1 --2- 172.21.15.38:0/1505138618 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6c8405c2b0 0x7f6c8405e770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.564+0000 7f6c97fff700 1 --2- 172.21.15.38:0/1505138618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c980f67b0 0x7f6c9810c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.564+0000 7f6c97fff700 1 -- 172.21.15.38:0/1505138618 >> 172.21.15.38:0/1505138618 conn(0x7f6c980f0c70 msgr2=0x7f6c980fa200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:28.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.564+0000 7f6c97fff700 1 -- 172.21.15.38:0/1505138618 shutdown_connections 2024-02-15T18:31:28.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:28.564+0000 7f6c97fff700 1 -- 172.21.15.38:0/1505138618 wait complete. 2024-02-15T18:31:28.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:28 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:28.563817+0000 mon.smithi038 (mon.0) 438 : audit [DBG] from='client.? 172.21.15.38:0/1505138618' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-15T18:31:28.992 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:28.563817+0000 mon.smithi038 (mon.0) 438 : audit [DBG] from='client.? 172.21.15.38:0/1505138618' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-15T18:31:29.442 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-15T18:30:09.268924+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}] 2024-02-15T18:31:29.442 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph osd pool get device_health_metrics pg_num 2024-02-15T18:31:29.687 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/config 2024-02-15T18:31:29.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:28.261433+0000 mgr.smithi038.khesbl (mgr.14162) 200 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:29.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:28.261433+0000 mgr.smithi038.khesbl (mgr.14162) 200 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.631+0000 7fbdad953700 1 -- 172.21.15.38:0/2059321850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbda80f7a80 msgr2=0x7fbda80f7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.631+0000 7fbdad953700 1 --2- 172.21.15.38:0/2059321850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbda80f7a80 0x7fbda80f7ee0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fbd90004660 tx=0x7fbd90009b30).stop 2024-02-15T18:31:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.631+0000 7fbdad953700 1 -- 172.21.15.38:0/2059321850 shutdown_connections 2024-02-15T18:31:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.631+0000 7fbdad953700 1 --2- 172.21.15.38:0/2059321850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbda80f7a80 0x7fbda80f7ee0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.631+0000 7fbdad953700 1 --2- 172.21.15.38:0/2059321850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbda80f67e0 0x7fbda80f6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.631+0000 7fbdad953700 1 -- 172.21.15.38:0/2059321850 >> 172.21.15.38:0/2059321850 conn(0x7fbda80f1f20 msgr2=0x7fbda80f4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.631+0000 7fbdad953700 1 -- 172.21.15.38:0/2059321850 shutdown_connections 2024-02-15T18:31:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.631+0000 7fbdad953700 1 -- 172.21.15.38:0/2059321850 wait complete. 2024-02-15T18:31:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.632+0000 7fbdad953700 1 Processor -- start 2024-02-15T18:31:31.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.632+0000 7fbdad953700 1 -- start start 2024-02-15T18:31:31.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.632+0000 7fbdad953700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbda80f67e0 0x7fbda806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:31.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.632+0000 7fbdad953700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbda80f7a80 0x7fbda8069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:31.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.632+0000 7fbdad953700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda806a380 con 0x7fbda80f7a80 2024-02-15T18:31:31.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.632+0000 7fbdad953700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda806a4c0 con 0x7fbda80f67e0 2024-02-15T18:31:31.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.632+0000 7fbda67fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbda80f7a80 0x7fbda8069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:31.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.632+0000 7fbda67fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbda80f7a80 0x7fbda8069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49860/0 (socket says 172.21.15.38:49860) 2024-02-15T18:31:31.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.632+0000 7fbda67fc700 1 -- 172.21.15.38:0/942509012 learned_addr learned my addr 172.21.15.38:0/942509012 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:31:31.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.632+0000 7fbda6ffd700 1 --2- 172.21.15.38:0/942509012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbda80f67e0 0x7fbda806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:31.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.632+0000 7fbda67fc700 1 -- 172.21.15.38:0/942509012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbda80f67e0 msgr2=0x7fbda806b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:31.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.632+0000 7fbda67fc700 1 --2- 172.21.15.38:0/942509012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbda80f67e0 0x7fbda806b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:31.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.632+0000 7fbda67fc700 1 -- 172.21.15.38:0/942509012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd90005040 con 0x7fbda80f7a80 2024-02-15T18:31:31.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.633+0000 7fbda67fc700 1 --2- 172.21.15.38:0/942509012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbda80f7a80 0x7fbda8069e20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fbd90009830 tx=0x7fbd90009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:31.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.633+0000 7fbd9f7fe700 1 -- 172.21.15.38:0/942509012 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd90018070 con 0x7fbda80f7a80 2024-02-15T18:31:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.633+0000 7fbd9f7fe700 1 -- 172.21.15.38:0/942509012 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd90007ba0 con 0x7fbda80f7a80 2024-02-15T18:31:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.633+0000 7fbdad953700 1 -- 172.21.15.38:0/942509012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda806a740 con 0x7fbda80f7a80 2024-02-15T18:31:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.633+0000 7fbd9f7fe700 1 -- 172.21.15.38:0/942509012 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd9001c3c0 con 0x7fbda80f7a80 2024-02-15T18:31:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.633+0000 7fbdad953700 1 -- 172.21.15.38:0/942509012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda806aca0 con 0x7fbda80f7a80 2024-02-15T18:31:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.634+0000 7fbdad953700 1 -- 172.21.15.38:0/942509012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda8045ad0 con 0x7fbda80f7a80 2024-02-15T18:31:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.634+0000 7fbd9f7fe700 1 -- 172.21.15.38:0/942509012 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fbd90007d10 con 0x7fbda80f7a80 2024-02-15T18:31:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.635+0000 7fbd9f7fe700 1 --2- 172.21.15.38:0/942509012 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fbd9405c2a0 0x7fbd9405e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.635+0000 7fbd9f7fe700 1 -- 172.21.15.38:0/942509012 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7fbd90077720 con 0x7fbda80f7a80 2024-02-15T18:31:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.635+0000 7fbda6ffd700 1 --2- 172.21.15.38:0/942509012 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fbd9405c2a0 0x7fbd9405e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.635+0000 7fbda6ffd700 1 --2- 172.21.15.38:0/942509012 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fbd9405c2a0 0x7fbd9405e760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fbda806b730 tx=0x7fbd9800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:31.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.639+0000 7fbd9f7fe700 1 -- 172.21.15.38:0/942509012 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fbd9004a740 con 0x7fbda80f7a80 2024-02-15T18:31:31.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:30.261871+0000 mgr.smithi038.khesbl (mgr.14162) 201 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:31.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.796+0000 7fbdad953700 1 -- 172.21.15.38:0/942509012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fbda8027a00 con 0x7fbda80f7a80 2024-02-15T18:31:31.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.797+0000 7fbd9f7fe700 1 -- 172.21.15.38:0/942509012 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v38) v1 ==== 110+0+10 (secure 0 0 0) 0x7fbd9004dd60 con 0x7fbda80f7a80 2024-02-15T18:31:31.790 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-02-15T18:31:31.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.798+0000 7fbdad953700 1 -- 172.21.15.38:0/942509012 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fbd9405c2a0 msgr2=0x7fbd9405e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:31.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.798+0000 7fbdad953700 1 --2- 172.21.15.38:0/942509012 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fbd9405c2a0 0x7fbd9405e760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fbda806b730 tx=0x7fbd9800b040).stop 2024-02-15T18:31:31.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.798+0000 7fbdad953700 1 -- 172.21.15.38:0/942509012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbda80f7a80 msgr2=0x7fbda8069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:31.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.798+0000 7fbdad953700 1 --2- 172.21.15.38:0/942509012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbda80f7a80 0x7fbda8069e20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fbd90009830 tx=0x7fbd90009b30).stop 2024-02-15T18:31:31.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.798+0000 7fbdad953700 1 -- 172.21.15.38:0/942509012 shutdown_connections 2024-02-15T18:31:31.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.798+0000 7fbdad953700 1 --2- 172.21.15.38:0/942509012 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fbd9405c2a0 0x7fbd9405e760 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:31.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.798+0000 7fbdad953700 1 --2- 172.21.15.38:0/942509012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbda80f67e0 0x7fbda806b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:31.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.798+0000 7fbdad953700 1 --2- 172.21.15.38:0/942509012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbda80f7a80 0x7fbda8069e20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:31.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.799+0000 7fbdad953700 1 -- 172.21.15.38:0/942509012 >> 172.21.15.38:0/942509012 conn(0x7fbda80f1f20 msgr2=0x7fbda80fb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:31.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.799+0000 7fbdad953700 1 -- 172.21.15.38:0/942509012 shutdown_connections 2024-02-15T18:31:31.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:31.799+0000 7fbdad953700 1 -- 172.21.15.38:0/942509012 wait complete. 2024-02-15T18:31:31.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:30.261871+0000 mgr.smithi038.khesbl (mgr.14162) 201 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:32.622 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-15T18:31:32.622 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-15T18:31:32.623 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph mgr dump --format=json 2024-02-15T18:31:32.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:31.797678+0000 mon.smithi038 (mon.0) 439 : audit [DBG] from='client.? 172.21.15.38:0/942509012' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-15T18:31:32.865 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/config 2024-02-15T18:31:33.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:31.797678+0000 mon.smithi038 (mon.0) 439 : audit [DBG] from='client.? 172.21.15.38:0/942509012' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-15T18:31:33.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:32.262269+0000 mgr.smithi038.khesbl (mgr.14162) 202 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:34.012 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:32.262269+0000 mgr.smithi038.khesbl (mgr.14162) 202 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:34.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.609+0000 7f1213360700 1 -- 172.21.15.38:0/3219124413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120c0f56c0 msgr2=0x7f120c0f5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.609+0000 7f1213360700 1 --2- 172.21.15.38:0/3219124413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120c0f56c0 0x7f120c0f5ae0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f1208004660 tx=0x7f1208009b30).stop 2024-02-15T18:31:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.609+0000 7f1213360700 1 -- 172.21.15.38:0/3219124413 shutdown_connections 2024-02-15T18:31:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.609+0000 7f1213360700 1 --2- 172.21.15.38:0/3219124413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f120c0f6960 0x7f120c0f6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.609+0000 7f1213360700 1 --2- 172.21.15.38:0/3219124413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120c0f56c0 0x7f120c0f5ae0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.609+0000 7f1213360700 1 -- 172.21.15.38:0/3219124413 >> 172.21.15.38:0/3219124413 conn(0x7f120c0f0e20 msgr2=0x7f120c0f3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.610+0000 7f1213360700 1 -- 172.21.15.38:0/3219124413 shutdown_connections 2024-02-15T18:31:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.610+0000 7f1213360700 1 -- 172.21.15.38:0/3219124413 wait complete. 2024-02-15T18:31:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.610+0000 7f1213360700 1 Processor -- start 2024-02-15T18:31:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.610+0000 7f1213360700 1 -- start start 2024-02-15T18:31:34.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.610+0000 7f1213360700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120c0f56c0 0x7f120c107180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:34.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.610+0000 7f1213360700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f120c0f6960 0x7f120c107670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:34.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.610+0000 7f1213360700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c107c20 con 0x7f120c0f56c0 2024-02-15T18:31:34.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.610+0000 7f1213360700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c107d60 con 0x7f120c0f6960 2024-02-15T18:31:34.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.611+0000 7f12110fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120c0f56c0 0x7f120c107180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:34.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.611+0000 7f12110fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120c0f56c0 0x7f120c107180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48246/0 (socket says 172.21.15.38:48246) 2024-02-15T18:31:34.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.611+0000 7f12110fc700 1 -- 172.21.15.38:0/2632642375 learned_addr learned my addr 172.21.15.38:0/2632642375 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:31:34.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.611+0000 7f12108fb700 1 --2- 172.21.15.38:0/2632642375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f120c0f6960 0x7f120c107670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.611+0000 7f12110fc700 1 -- 172.21.15.38:0/2632642375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f120c0f6960 msgr2=0x7f120c107670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.611+0000 7f12110fc700 1 --2- 172.21.15.38:0/2632642375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f120c0f6960 0x7f120c107670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.611+0000 7f12110fc700 1 -- 172.21.15.38:0/2632642375 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1208005040 con 0x7f120c0f56c0 2024-02-15T18:31:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.611+0000 7f12108fb700 1 --2- 172.21.15.38:0/2632642375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f120c0f6960 0x7f120c107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:31:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.611+0000 7f12110fc700 1 --2- 172.21.15.38:0/2632642375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120c0f56c0 0x7f120c107180 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f1208002040 tx=0x7f120800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.611+0000 7f11fdffb700 1 -- 172.21.15.38:0/2632642375 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1208019070 con 0x7f120c0f56c0 2024-02-15T18:31:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.611+0000 7f1213360700 1 -- 172.21.15.38:0/2632642375 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120c1086a0 con 0x7f120c0f56c0 2024-02-15T18:31:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.611+0000 7f11fdffb700 1 -- 172.21.15.38:0/2632642375 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1208007e90 con 0x7f120c0f56c0 2024-02-15T18:31:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.611+0000 7f11fdffb700 1 -- 172.21.15.38:0/2632642375 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1208014d90 con 0x7f120c0f56c0 2024-02-15T18:31:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.612+0000 7f1213360700 1 -- 172.21.15.38:0/2632642375 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120c108c00 con 0x7f120c0f56c0 2024-02-15T18:31:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.613+0000 7f1213360700 1 -- 172.21.15.38:0/2632642375 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120c045ad0 con 0x7f120c0f56c0 2024-02-15T18:31:34.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.615+0000 7f11fdffb700 1 -- 172.21.15.38:0/2632642375 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f1208025070 con 0x7f120c0f56c0 2024-02-15T18:31:34.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.615+0000 7f11fdffb700 1 --2- 172.21.15.38:0/2632642375 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f11f805c2a0 0x7f11f805e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:34.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.615+0000 7f11fdffb700 1 -- 172.21.15.38:0/2632642375 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f1208076e50 con 0x7f120c0f56c0 2024-02-15T18:31:34.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.615+0000 7f12108fb700 1 --2- 172.21.15.38:0/2632642375 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f11f805c2a0 0x7f11f805e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:34.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.615+0000 7f12108fb700 1 --2- 172.21.15.38:0/2632642375 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f11f805c2a0 0x7f11f805e760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f1200009da0 tx=0x7f120000b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:34.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.618+0000 7f11fdffb700 1 -- 172.21.15.38:0/2632642375 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f1208049f40 con 0x7f120c0f56c0 2024-02-15T18:31:34.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.815+0000 7f1213360700 1 -- 172.21.15.38:0/2632642375 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f120c109300 con 0x7f120c0f56c0 2024-02-15T18:31:34.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.818+0000 7f11fdffb700 1 -- 172.21.15.38:0/2632642375 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v18) v1 ==== 74+0+143500 (secure 0 0 0) 0x7f1208023030 con 0x7f120c0f56c0 2024-02-15T18:31:34.813 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:31:34.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.820+0000 7f1213360700 1 -- 172.21.15.38:0/2632642375 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f11f805c2a0 msgr2=0x7f11f805e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:34.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.820+0000 7f1213360700 1 --2- 172.21.15.38:0/2632642375 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f11f805c2a0 0x7f11f805e760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f1200009da0 tx=0x7f120000b040).stop 2024-02-15T18:31:34.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.820+0000 7f1213360700 1 -- 172.21.15.38:0/2632642375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120c0f56c0 msgr2=0x7f120c107180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:34.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.820+0000 7f1213360700 1 --2- 172.21.15.38:0/2632642375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120c0f56c0 0x7f120c107180 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f1208002040 tx=0x7f120800b040).stop 2024-02-15T18:31:34.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.821+0000 7f1213360700 1 -- 172.21.15.38:0/2632642375 shutdown_connections 2024-02-15T18:31:34.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.821+0000 7f1213360700 1 --2- 172.21.15.38:0/2632642375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f120c0f56c0 0x7f120c107180 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:34.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.821+0000 7f1213360700 1 --2- 172.21.15.38:0/2632642375 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f11f805c2a0 0x7f11f805e760 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:34.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.821+0000 7f1213360700 1 --2- 172.21.15.38:0/2632642375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f120c0f6960 0x7f120c107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:34.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.821+0000 7f1213360700 1 -- 172.21.15.38:0/2632642375 >> 172.21.15.38:0/2632642375 conn(0x7f120c0f0e20 msgr2=0x7f120c0603a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:34.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.821+0000 7f1213360700 1 -- 172.21.15.38:0/2632642375 shutdown_connections 2024-02-15T18:31:34.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:34.821+0000 7f1213360700 1 -- 172.21.15.38:0/2632642375 wait complete. 2024-02-15T18:31:35.620 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":18,"active_gid":14162,"active_name":"smithi038.khesbl","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":2900497610},{"type":"v1","addr":"172.21.15.38:6801","nonce":2900497610}]},"active_addr":"172.21.15.38:6801/2900497610","active_change":"2024-02-15T18:26:40.033686+0000","active_mgr_features":4540138297136906239,"available":true,"standbys":[{"gid":14208,"name":"smithi121.ckiryi","mgr_features":4540138297136906239,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"haproxy","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Registry to which we should normalize unqualified image names","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"haproxy","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Registry to which we should normalize unqualified image names","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://smithi038.front.sepia.ceph.com:8443/","prometheus":"http://smithi038.front.sepia.ceph.com:9283/"},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":4,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":2424130559}]},{"addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":1003542778}]}]}} 2024-02-15T18:31:35.624 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-15T18:31:35.624 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-15T18:31:35.624 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-15T18:31:35.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:34.262781+0000 mgr.smithi038.khesbl (mgr.14162) 203 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:35.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:34.816595+0000 mon.smithi038 (mon.0) 440 : audit [DBG] from='client.? 172.21.15.38:0/2632642375' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-15T18:31:35.866 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/config 2024-02-15T18:31:36.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:34.262781+0000 mgr.smithi038.khesbl (mgr.14162) 203 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:36.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:34.816595+0000 mon.smithi038 (mon.0) 440 : audit [DBG] from='client.? 172.21.15.38:0/2632642375' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-15T18:31:37.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.863+0000 7ff9aa910700 1 -- 172.21.15.38:0/1694626536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9a40f67c0 msgr2=0x7ff9a40f6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:37.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.863+0000 7ff9aa910700 1 --2- 172.21.15.38:0/1694626536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9a40f67c0 0x7ff9a40f6be0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7ff994004660 tx=0x7ff994009b30).stop 2024-02-15T18:31:37.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.863+0000 7ff9aa910700 1 -- 172.21.15.38:0/1694626536 shutdown_connections 2024-02-15T18:31:37.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.863+0000 7ff9aa910700 1 --2- 172.21.15.38:0/1694626536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9a40f7a60 0x7ff9a40f7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:37.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.863+0000 7ff9aa910700 1 --2- 172.21.15.38:0/1694626536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9a40f67c0 0x7ff9a40f6be0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:37.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.863+0000 7ff9aa910700 1 -- 172.21.15.38:0/1694626536 >> 172.21.15.38:0/1694626536 conn(0x7ff9a40f1f20 msgr2=0x7ff9a40f4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:37.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.863+0000 7ff9aa910700 1 -- 172.21.15.38:0/1694626536 shutdown_connections 2024-02-15T18:31:37.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.863+0000 7ff9aa910700 1 -- 172.21.15.38:0/1694626536 wait complete. 2024-02-15T18:31:37.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.863+0000 7ff9aa910700 1 Processor -- start 2024-02-15T18:31:37.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.863+0000 7ff9aa910700 1 -- start start 2024-02-15T18:31:37.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.864+0000 7ff9aa910700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9a40f67c0 0x7ff9a406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:37.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.864+0000 7ff9aa910700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9a40f7a60 0x7ff9a4069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:37.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.864+0000 7ff9aa910700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a406a380 con 0x7ff9a40f7a60 2024-02-15T18:31:37.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.864+0000 7ff9aa910700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a406a4c0 con 0x7ff9a40f67c0 2024-02-15T18:31:37.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.864+0000 7ff9a37fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9a40f7a60 0x7ff9a4069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:37.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.864+0000 7ff9a37fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9a40f7a60 0x7ff9a4069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48256/0 (socket says 172.21.15.38:48256) 2024-02-15T18:31:37.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.864+0000 7ff9a37fe700 1 -- 172.21.15.38:0/2530766222 learned_addr learned my addr 172.21.15.38:0/2530766222 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:31:37.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.864+0000 7ff9a3fff700 1 --2- 172.21.15.38:0/2530766222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9a40f67c0 0x7ff9a406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:37.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.864+0000 7ff9a37fe700 1 -- 172.21.15.38:0/2530766222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9a40f67c0 msgr2=0x7ff9a406b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:37.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.864+0000 7ff9a37fe700 1 --2- 172.21.15.38:0/2530766222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9a40f67c0 0x7ff9a406b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:37.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.864+0000 7ff9a37fe700 1 -- 172.21.15.38:0/2530766222 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff994005040 con 0x7ff9a40f7a60 2024-02-15T18:31:37.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.864+0000 7ff9a37fe700 1 --2- 172.21.15.38:0/2530766222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9a40f7a60 0x7ff9a4069e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7ff99800c530 tx=0x7ff998007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:37.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.864+0000 7ff9a0ff9700 1 -- 172.21.15.38:0/2530766222 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff99800f650 con 0x7ff9a40f7a60 2024-02-15T18:31:37.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.864+0000 7ff9a0ff9700 1 -- 172.21.15.38:0/2530766222 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff99800fc80 con 0x7ff9a40f7a60 2024-02-15T18:31:37.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.864+0000 7ff9aa910700 1 -- 172.21.15.38:0/2530766222 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a406a7a0 con 0x7ff9a40f7a60 2024-02-15T18:31:37.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.864+0000 7ff9a0ff9700 1 -- 172.21.15.38:0/2530766222 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff998012d80 con 0x7ff9a40f7a60 2024-02-15T18:31:37.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.864+0000 7ff9aa910700 1 -- 172.21.15.38:0/2530766222 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a410f1d0 con 0x7ff9a40f7a60 2024-02-15T18:31:37.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.865+0000 7ff9aa910700 1 -- 172.21.15.38:0/2530766222 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a4045ad0 con 0x7ff9a40f7a60 2024-02-15T18:31:37.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.866+0000 7ff9a0ff9700 1 -- 172.21.15.38:0/2530766222 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7ff998010070 con 0x7ff9a40f7a60 2024-02-15T18:31:37.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.866+0000 7ff9a0ff9700 1 --2- 172.21.15.38:0/2530766222 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff98c05c2a0 0x7ff98c05e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:37.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.866+0000 7ff9a0ff9700 1 -- 172.21.15.38:0/2530766222 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7ff998075930 con 0x7ff9a40f7a60 2024-02-15T18:31:37.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.866+0000 7ff9a3fff700 1 --2- 172.21.15.38:0/2530766222 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff98c05c2a0 0x7ff98c05e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:37.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.867+0000 7ff9a3fff700 1 --2- 172.21.15.38:0/2530766222 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff98c05c2a0 0x7ff98c05e760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7ff9a406b730 tx=0x7ff994009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:37.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:37.871+0000 7ff9a0ff9700 1 -- 172.21.15.38:0/2530766222 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ff9980452b0 con 0x7ff9a40f7a60 2024-02-15T18:31:38.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:38.029+0000 7ff9aa910700 1 -- 172.21.15.38:0/2530766222 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9a4027a00 con 0x7ff9a40f7a60 2024-02-15T18:31:38.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:38.030+0000 7ff9a0ff9700 1 -- 172.21.15.38:0/2530766222 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+10923 (secure 0 0 0) 0x7ff998048930 con 0x7ff9a40f7a60 2024-02-15T18:31:38.022 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:31:38.022 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":38,"fsid":"7e05753e-cc2f-11ee-95ba-87774f69a715","created":"2024-02-15T18:25:33.849561+0000","modified":"2024-02-15T18:31:15.719708+0000","last_up_change":"2024-02-15T18:31:14.711602+0000","last_in_change":"2024-02-15T18:30:56.975140+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-15T18:30:09.268924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"eee9713b-ff79-4169-8c04-b1def80d2045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1826650694},{"type":"v1","addr":"172.21.15.38:6803","nonce":1826650694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1826650694},{"type":"v1","addr":"172.21.15.38:6805","nonce":1826650694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1826650694},{"type":"v1","addr":"172.21.15.38:6809","nonce":1826650694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1826650694},{"type":"v1","addr":"172.21.15.38:6807","nonce":1826650694}]},"public_addr":"172.21.15.38:6803/1826650694","cluster_addr":"172.21.15.38:6805/1826650694","heartbeat_back_addr":"172.21.15.38:6809/1826650694","heartbeat_front_addr":"172.21.15.38:6807/1826650694","state":["exists","up"]},{"osd":1,"uuid":"740eed48-d34e-4e71-a859-035f01b1fdd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3796697403},{"type":"v1","addr":"172.21.15.38:6811","nonce":3796697403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3796697403},{"type":"v1","addr":"172.21.15.38:6813","nonce":3796697403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3796697403},{"type":"v1","addr":"172.21.15.38:6817","nonce":3796697403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3796697403},{"type":"v1","addr":"172.21.15.38:6815","nonce":3796697403}]},"public_addr":"172.21.15.38:6811/3796697403","cluster_addr":"172.21.15.38:6813/3796697403","heartbeat_back_addr":"172.21.15.38:6817/3796697403","heartbeat_front_addr":"172.21.15.38:6815/3796697403","state":["exists","up"]},{"osd":2,"uuid":"8a397970-64c0-47d7-94c8-77fcf9fe579d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":420320553},{"type":"v1","addr":"172.21.15.38:6819","nonce":420320553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":420320553},{"type":"v1","addr":"172.21.15.38:6821","nonce":420320553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":420320553},{"type":"v1","addr":"172.21.15.38:6825","nonce":420320553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":420320553},{"type":"v1","addr":"172.21.15.38:6823","nonce":420320553}]},"public_addr":"172.21.15.38:6819/420320553","cluster_addr":"172.21.15.38:6821/420320553","heartbeat_back_addr":"172.21.15.38:6825/420320553","heartbeat_front_addr":"172.21.15.38:6823/420320553","state":["exists","up"]},{"osd":3,"uuid":"0a590f02-56f3-43e7-894f-741f95d2facd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3071118694},{"type":"v1","addr":"172.21.15.121:6801","nonce":3071118694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3071118694},{"type":"v1","addr":"172.21.15.121:6803","nonce":3071118694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3071118694},{"type":"v1","addr":"172.21.15.121:6807","nonce":3071118694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3071118694},{"type":"v1","addr":"172.21.15.121:6805","nonce":3071118694}]},"public_addr":"172.21.15.121:6801/3071118694","cluster_addr":"172.21.15.121:6803/3071118694","heartbeat_back_addr":"172.21.15.121:6807/3071118694","heartbeat_front_addr":"172.21.15.121:6805/3071118694","state":["exists","up"]},{"osd":4,"uuid":"5e13d4f8-2233-4674-9b52-a3fb9dde34ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3016245330},{"type":"v1","addr":"172.21.15.121:6809","nonce":3016245330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3016245330},{"type":"v1","addr":"172.21.15.121:6811","nonce":3016245330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3016245330},{"type":"v1","addr":"172.21.15.121:6815","nonce":3016245330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3016245330},{"type":"v1","addr":"172.21.15.121:6813","nonce":3016245330}]},"public_addr":"172.21.15.121:6809/3016245330","cluster_addr":"172.21.15.121:6811/3016245330","heartbeat_back_addr":"172.21.15.121:6815/3016245330","heartbeat_front_addr":"172.21.15.121:6813/3016245330","state":["exists","up"]},{"osd":5,"uuid":"6cd02c52-b9f4-494e-b67e-7019e4c224af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4062288290},{"type":"v1","addr":"172.21.15.121:6817","nonce":4062288290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4062288290},{"type":"v1","addr":"172.21.15.121:6819","nonce":4062288290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4062288290},{"type":"v1","addr":"172.21.15.121:6823","nonce":4062288290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4062288290},{"type":"v1","addr":"172.21.15.121:6821","nonce":4062288290}]},"public_addr":"172.21.15.121:6817/4062288290","cluster_addr":"172.21.15.121:6819/4062288290","heartbeat_back_addr":"172.21.15.121:6823/4062288290","heartbeat_front_addr":"172.21.15.121:6821/4062288290","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:29:06.518109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:29:40.582228+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:30:07.799238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:30:29.057360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:30:51.020867+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:31:13.005231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/3518675152":"2024-02-16T18:26:40.033295+0000","172.21.15.38:0/1774333839":"2024-02-16T18:26:40.033295+0000","172.21.15.38:6800/2095530547":"2024-02-16T18:26:40.033295+0000","172.21.15.38:0/2413778685":"2024-02-16T18:26:23.102293+0000","172.21.15.38:6801/3009942445":"2024-02-16T18:26:23.102293+0000","172.21.15.38:0/788545352":"2024-02-16T18:26:23.102293+0000","172.21.15.38:6800/3009942445":"2024-02-16T18:26:23.102293+0000","172.21.15.38:0/3436541874":"2024-02-16T18:26:02.772407+0000","172.21.15.38:0/4035950047":"2024-02-16T18:26:02.772407+0000","172.21.15.38:6801/2095530547":"2024-02-16T18:26:40.033295+0000","172.21.15.38:6800/4189944318":"2024-02-16T18:26:02.772407+0000","172.21.15.38:6801/4189944318":"2024-02-16T18:26:02.772407+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-15T18:31:38.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:38.031+0000 7ff9aa910700 1 -- 172.21.15.38:0/2530766222 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff98c05c2a0 msgr2=0x7ff98c05e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:38.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:38.031+0000 7ff9aa910700 1 --2- 172.21.15.38:0/2530766222 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff98c05c2a0 0x7ff98c05e760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7ff9a406b730 tx=0x7ff994009b30).stop 2024-02-15T18:31:38.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:38.031+0000 7ff9aa910700 1 -- 172.21.15.38:0/2530766222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9a40f7a60 msgr2=0x7ff9a4069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:38.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:38.031+0000 7ff9aa910700 1 --2- 172.21.15.38:0/2530766222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9a40f7a60 0x7ff9a4069e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7ff99800c530 tx=0x7ff998007e20).stop 2024-02-15T18:31:38.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:38.031+0000 7ff9aa910700 1 -- 172.21.15.38:0/2530766222 shutdown_connections 2024-02-15T18:31:38.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:38.031+0000 7ff9aa910700 1 --2- 172.21.15.38:0/2530766222 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff98c05c2a0 0x7ff98c05e760 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:38.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:38.031+0000 7ff9aa910700 1 --2- 172.21.15.38:0/2530766222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9a40f67c0 0x7ff9a406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:38.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:38.031+0000 7ff9aa910700 1 --2- 172.21.15.38:0/2530766222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9a40f7a60 0x7ff9a4069e20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:38.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:38.031+0000 7ff9aa910700 1 -- 172.21.15.38:0/2530766222 >> 172.21.15.38:0/2530766222 conn(0x7ff9a40f1f20 msgr2=0x7ff9a40fb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:38.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:38.031+0000 7ff9aa910700 1 -- 172.21.15.38:0/2530766222 shutdown_connections 2024-02-15T18:31:38.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:38.032+0000 7ff9aa910700 1 -- 172.21.15.38:0/2530766222 wait complete. 2024-02-15T18:31:38.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:36.263074+0000 mgr.smithi038.khesbl (mgr.14162) 204 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:38.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:36.263074+0000 mgr.smithi038.khesbl (mgr.14162) 204 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:38.742 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:38 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:38.030037+0000 mon.smithi038 (mon.0) 441 : audit [DBG] from='client.? 172.21.15.38:0/2530766222' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-15T18:31:38.766 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-15T18:31:38.766 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-15T18:31:39.005 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/config 2024-02-15T18:31:39.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:38 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:38.030037+0000 mon.smithi038 (mon.0) 441 : audit [DBG] from='client.? 172.21.15.38:0/2530766222' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-15T18:31:39.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:38.263333+0000 mgr.smithi038.khesbl (mgr.14162) 205 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:40.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:38.263333+0000 mgr.smithi038.khesbl (mgr.14162) 205 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.848+0000 7f8df96e0700 1 -- 172.21.15.38:0/1691450606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8df40f7a70 msgr2=0x7f8df40f7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.848+0000 7f8df96e0700 1 --2- 172.21.15.38:0/1691450606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8df40f7a70 0x7f8df40f7ed0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f8ddc004660 tx=0x7f8ddc009b30).stop 2024-02-15T18:31:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.848+0000 7f8df96e0700 1 -- 172.21.15.38:0/1691450606 shutdown_connections 2024-02-15T18:31:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.848+0000 7f8df96e0700 1 --2- 172.21.15.38:0/1691450606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8df40f7a70 0x7f8df40f7ed0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.848+0000 7f8df96e0700 1 --2- 172.21.15.38:0/1691450606 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8df40f67d0 0x7f8df40f6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.848+0000 7f8df96e0700 1 -- 172.21.15.38:0/1691450606 >> 172.21.15.38:0/1691450606 conn(0x7f8df40f1f50 msgr2=0x7f8df40f4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.848+0000 7f8df96e0700 1 -- 172.21.15.38:0/1691450606 shutdown_connections 2024-02-15T18:31:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.848+0000 7f8df96e0700 1 -- 172.21.15.38:0/1691450606 wait complete. 2024-02-15T18:31:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.848+0000 7f8df96e0700 1 Processor -- start 2024-02-15T18:31:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.849+0000 7f8df96e0700 1 -- start start 2024-02-15T18:31:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.849+0000 7f8df96e0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8df40f67d0 0x7f8df4107180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.849+0000 7f8df96e0700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8df40f7a70 0x7f8df4107670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.849+0000 7f8df96e0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df4107c20 con 0x7f8df40f67d0 2024-02-15T18:31:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.849+0000 7f8df96e0700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df4107d60 con 0x7f8df40f7a70 2024-02-15T18:31:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.849+0000 7f8df2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8df40f67d0 0x7f8df4107180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.849+0000 7f8df2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8df40f67d0 0x7f8df4107180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48278/0 (socket says 172.21.15.38:48278) 2024-02-15T18:31:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.849+0000 7f8df2ffd700 1 -- 172.21.15.38:0/537058027 learned_addr learned my addr 172.21.15.38:0/537058027 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:31:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.849+0000 7f8df2ffd700 1 -- 172.21.15.38:0/537058027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8df40f7a70 msgr2=0x7f8df4107670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.849+0000 7f8df2ffd700 1 --2- 172.21.15.38:0/537058027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8df40f7a70 0x7f8df4107670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:31:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.849+0000 7f8df2ffd700 1 -- 172.21.15.38:0/537058027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc005040 con 0x7f8df40f67d0 2024-02-15T18:31:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.849+0000 7f8df2ffd700 1 --2- 172.21.15.38:0/537058027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8df40f67d0 0x7f8df4107180 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f8de4002700 tx=0x7f8de4009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.849+0000 7f8deb7fe700 1 -- 172.21.15.38:0/537058027 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8de4009320 con 0x7f8df40f67d0 2024-02-15T18:31:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.849+0000 7f8deb7fe700 1 -- 172.21.15.38:0/537058027 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8de400f040 con 0x7f8df40f67d0 2024-02-15T18:31:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.849+0000 7f8df96e0700 1 -- 172.21.15.38:0/537058027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df4108700 con 0x7f8df40f67d0 2024-02-15T18:31:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.849+0000 7f8deb7fe700 1 -- 172.21.15.38:0/537058027 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8de4014940 con 0x7f8df40f67d0 2024-02-15T18:31:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.849+0000 7f8df96e0700 1 -- 172.21.15.38:0/537058027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df4108cc0 con 0x7f8df40f67d0 2024-02-15T18:31:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.850+0000 7f8df96e0700 1 -- 172.21.15.38:0/537058027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df4045ad0 con 0x7f8df40f67d0 2024-02-15T18:31:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.851+0000 7f8deb7fe700 1 -- 172.21.15.38:0/537058027 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f8de40077b0 con 0x7f8df40f67d0 2024-02-15T18:31:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.851+0000 7f8deb7fe700 1 --2- 172.21.15.38:0/537058027 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f8de005c2a0 0x7f8de005e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.851+0000 7f8deb7fe700 1 -- 172.21.15.38:0/537058027 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f8de4075de0 con 0x7f8df40f67d0 2024-02-15T18:31:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.851+0000 7f8df27fc700 1 --2- 172.21.15.38:0/537058027 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f8de005c2a0 0x7f8de005e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.851+0000 7f8df27fc700 1 --2- 172.21.15.38:0/537058027 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f8de005c2a0 0x7f8de005e760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f8ddc009f50 tx=0x7f8ddc009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:40.855+0000 7f8deb7fe700 1 -- 172.21.15.38:0/537058027 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f8de4048de0 con 0x7f8df40f67d0 2024-02-15T18:31:41.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:41.012+0000 7f8df96e0700 1 -- 172.21.15.38:0/537058027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8df4027a00 con 0x7f8df40f67d0 2024-02-15T18:31:41.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:41.013+0000 7f8deb7fe700 1 -- 172.21.15.38:0/537058027 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+10923 (secure 0 0 0) 0x7f8de404c400 con 0x7f8df40f67d0 2024-02-15T18:31:41.006 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:31:41.006 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":38,"fsid":"7e05753e-cc2f-11ee-95ba-87774f69a715","created":"2024-02-15T18:25:33.849561+0000","modified":"2024-02-15T18:31:15.719708+0000","last_up_change":"2024-02-15T18:31:14.711602+0000","last_in_change":"2024-02-15T18:30:56.975140+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-15T18:30:09.268924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"eee9713b-ff79-4169-8c04-b1def80d2045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1826650694},{"type":"v1","addr":"172.21.15.38:6803","nonce":1826650694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1826650694},{"type":"v1","addr":"172.21.15.38:6805","nonce":1826650694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1826650694},{"type":"v1","addr":"172.21.15.38:6809","nonce":1826650694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1826650694},{"type":"v1","addr":"172.21.15.38:6807","nonce":1826650694}]},"public_addr":"172.21.15.38:6803/1826650694","cluster_addr":"172.21.15.38:6805/1826650694","heartbeat_back_addr":"172.21.15.38:6809/1826650694","heartbeat_front_addr":"172.21.15.38:6807/1826650694","state":["exists","up"]},{"osd":1,"uuid":"740eed48-d34e-4e71-a859-035f01b1fdd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3796697403},{"type":"v1","addr":"172.21.15.38:6811","nonce":3796697403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3796697403},{"type":"v1","addr":"172.21.15.38:6813","nonce":3796697403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3796697403},{"type":"v1","addr":"172.21.15.38:6817","nonce":3796697403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3796697403},{"type":"v1","addr":"172.21.15.38:6815","nonce":3796697403}]},"public_addr":"172.21.15.38:6811/3796697403","cluster_addr":"172.21.15.38:6813/3796697403","heartbeat_back_addr":"172.21.15.38:6817/3796697403","heartbeat_front_addr":"172.21.15.38:6815/3796697403","state":["exists","up"]},{"osd":2,"uuid":"8a397970-64c0-47d7-94c8-77fcf9fe579d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":420320553},{"type":"v1","addr":"172.21.15.38:6819","nonce":420320553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":420320553},{"type":"v1","addr":"172.21.15.38:6821","nonce":420320553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":420320553},{"type":"v1","addr":"172.21.15.38:6825","nonce":420320553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":420320553},{"type":"v1","addr":"172.21.15.38:6823","nonce":420320553}]},"public_addr":"172.21.15.38:6819/420320553","cluster_addr":"172.21.15.38:6821/420320553","heartbeat_back_addr":"172.21.15.38:6825/420320553","heartbeat_front_addr":"172.21.15.38:6823/420320553","state":["exists","up"]},{"osd":3,"uuid":"0a590f02-56f3-43e7-894f-741f95d2facd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3071118694},{"type":"v1","addr":"172.21.15.121:6801","nonce":3071118694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3071118694},{"type":"v1","addr":"172.21.15.121:6803","nonce":3071118694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3071118694},{"type":"v1","addr":"172.21.15.121:6807","nonce":3071118694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3071118694},{"type":"v1","addr":"172.21.15.121:6805","nonce":3071118694}]},"public_addr":"172.21.15.121:6801/3071118694","cluster_addr":"172.21.15.121:6803/3071118694","heartbeat_back_addr":"172.21.15.121:6807/3071118694","heartbeat_front_addr":"172.21.15.121:6805/3071118694","state":["exists","up"]},{"osd":4,"uuid":"5e13d4f8-2233-4674-9b52-a3fb9dde34ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3016245330},{"type":"v1","addr":"172.21.15.121:6809","nonce":3016245330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3016245330},{"type":"v1","addr":"172.21.15.121:6811","nonce":3016245330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3016245330},{"type":"v1","addr":"172.21.15.121:6815","nonce":3016245330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3016245330},{"type":"v1","addr":"172.21.15.121:6813","nonce":3016245330}]},"public_addr":"172.21.15.121:6809/3016245330","cluster_addr":"172.21.15.121:6811/3016245330","heartbeat_back_addr":"172.21.15.121:6815/3016245330","heartbeat_front_addr":"172.21.15.121:6813/3016245330","state":["exists","up"]},{"osd":5,"uuid":"6cd02c52-b9f4-494e-b67e-7019e4c224af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4062288290},{"type":"v1","addr":"172.21.15.121:6817","nonce":4062288290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4062288290},{"type":"v1","addr":"172.21.15.121:6819","nonce":4062288290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4062288290},{"type":"v1","addr":"172.21.15.121:6823","nonce":4062288290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4062288290},{"type":"v1","addr":"172.21.15.121:6821","nonce":4062288290}]},"public_addr":"172.21.15.121:6817/4062288290","cluster_addr":"172.21.15.121:6819/4062288290","heartbeat_back_addr":"172.21.15.121:6823/4062288290","heartbeat_front_addr":"172.21.15.121:6821/4062288290","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:29:06.518109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:29:40.582228+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:30:07.799238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:30:29.057360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:30:51.020867+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:31:13.005231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/3518675152":"2024-02-16T18:26:40.033295+0000","172.21.15.38:0/1774333839":"2024-02-16T18:26:40.033295+0000","172.21.15.38:6800/2095530547":"2024-02-16T18:26:40.033295+0000","172.21.15.38:0/2413778685":"2024-02-16T18:26:23.102293+0000","172.21.15.38:6801/3009942445":"2024-02-16T18:26:23.102293+0000","172.21.15.38:0/788545352":"2024-02-16T18:26:23.102293+0000","172.21.15.38:6800/3009942445":"2024-02-16T18:26:23.102293+0000","172.21.15.38:0/3436541874":"2024-02-16T18:26:02.772407+0000","172.21.15.38:0/4035950047":"2024-02-16T18:26:02.772407+0000","172.21.15.38:6801/2095530547":"2024-02-16T18:26:40.033295+0000","172.21.15.38:6800/4189944318":"2024-02-16T18:26:02.772407+0000","172.21.15.38:6801/4189944318":"2024-02-16T18:26:02.772407+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-15T18:31:41.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:41.015+0000 7f8df96e0700 1 -- 172.21.15.38:0/537058027 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f8de005c2a0 msgr2=0x7f8de005e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:41.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:41.015+0000 7f8df96e0700 1 --2- 172.21.15.38:0/537058027 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f8de005c2a0 0x7f8de005e760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f8ddc009f50 tx=0x7f8ddc009b30).stop 2024-02-15T18:31:41.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:41.015+0000 7f8df96e0700 1 -- 172.21.15.38:0/537058027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8df40f67d0 msgr2=0x7f8df4107180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:41.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:41.015+0000 7f8df96e0700 1 --2- 172.21.15.38:0/537058027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8df40f67d0 0x7f8df4107180 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f8de4002700 tx=0x7f8de4009d80).stop 2024-02-15T18:31:41.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:41.015+0000 7f8df96e0700 1 -- 172.21.15.38:0/537058027 shutdown_connections 2024-02-15T18:31:41.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:41.015+0000 7f8df96e0700 1 --2- 172.21.15.38:0/537058027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8df40f67d0 0x7f8df4107180 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:41.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:41.015+0000 7f8df96e0700 1 --2- 172.21.15.38:0/537058027 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f8de005c2a0 0x7f8de005e760 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:41.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:41.015+0000 7f8df96e0700 1 --2- 172.21.15.38:0/537058027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8df40f7a70 0x7f8df4107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:31:41.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:41.015+0000 7f8df96e0700 1 -- 172.21.15.38:0/537058027 >> 172.21.15.38:0/537058027 conn(0x7f8df40f1f50 msgr2=0x7f8df40fb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:41.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:41.015+0000 7f8df96e0700 1 -- 172.21.15.38:0/537058027 shutdown_connections 2024-02-15T18:31:41.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:41.015+0000 7f8df96e0700 1 -- 172.21.15.38:0/537058027 wait complete. 2024-02-15T18:31:41.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:40.357401+0000 mon.smithi038 (mon.0) 442 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:31:41.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:40.357605+0000 mon.smithi038 (mon.0) 443 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:31:41.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:40.357401+0000 mon.smithi038 (mon.0) 442 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:31:41.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:40.357605+0000 mon.smithi038 (mon.0) 443 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:31:41.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:40.263771+0000 mgr.smithi038.khesbl (mgr.14162) 206 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:41.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:31:41.013672+0000 mon.smithi038 (mon.0) 444 : audit [DBG] from='client.? 172.21.15.38:0/537058027' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-15T18:31:41.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:40.263771+0000 mgr.smithi038.khesbl (mgr.14162) 206 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:41.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:31:41.013672+0000 mon.smithi038 (mon.0) 444 : audit [DBG] from='client.? 172.21.15.38:0/537058027' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-15T18:31:41.956 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-15T18:31:42.196 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/config 2024-02-15T18:31:43.896 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:42.264153+0000 mgr.smithi038.khesbl (mgr.14162) 207 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:44.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:42.264153+0000 mgr.smithi038.khesbl (mgr.14162) 207 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:44.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.145+0000 7f81df851700 1 -- 172.21.15.38:0/4136212883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81d80f67c0 msgr2=0x7f81d80f6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:44.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.145+0000 7f81df851700 1 --2- 172.21.15.38:0/4136212883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81d80f67c0 0x7f81d80f6be0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f81d4004660 tx=0x7f81d4009b30).stop 2024-02-15T18:31:44.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.145+0000 7f81df851700 1 -- 172.21.15.38:0/4136212883 shutdown_connections 2024-02-15T18:31:44.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.145+0000 7f81df851700 1 --2- 172.21.15.38:0/4136212883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81d80f7a60 0x7f81d80f7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:44.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.145+0000 7f81df851700 1 --2- 172.21.15.38:0/4136212883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81d80f67c0 0x7f81d80f6be0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:44.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.145+0000 7f81df851700 1 -- 172.21.15.38:0/4136212883 >> 172.21.15.38:0/4136212883 conn(0x7f81d80f1f20 msgr2=0x7f81d80f4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:44.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.145+0000 7f81df851700 1 -- 172.21.15.38:0/4136212883 shutdown_connections 2024-02-15T18:31:44.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.145+0000 7f81df851700 1 -- 172.21.15.38:0/4136212883 wait complete. 2024-02-15T18:31:44.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.146+0000 7f81df851700 1 Processor -- start 2024-02-15T18:31:44.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.146+0000 7f81df851700 1 -- start start 2024-02-15T18:31:44.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.146+0000 7f81df851700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81d80f67c0 0x7f81d8107120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:44.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.146+0000 7f81df851700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81d80f7a60 0x7f81d8107610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:44.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.146+0000 7f81df851700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d8107bc0 con 0x7f81d80f67c0 2024-02-15T18:31:44.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.146+0000 7f81df851700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d8107d00 con 0x7f81d80f7a60 2024-02-15T18:31:44.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.146+0000 7f81dd5ed700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81d80f67c0 0x7f81d8107120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:44.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.146+0000 7f81dd5ed700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81d80f67c0 0x7f81d8107120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44360/0 (socket says 172.21.15.38:44360) 2024-02-15T18:31:44.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.146+0000 7f81dd5ed700 1 -- 172.21.15.38:0/1846076398 learned_addr learned my addr 172.21.15.38:0/1846076398 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:31:44.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.146+0000 7f81dcdec700 1 --2- 172.21.15.38:0/1846076398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81d80f7a60 0x7f81d8107610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:44.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.146+0000 7f81dd5ed700 1 -- 172.21.15.38:0/1846076398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81d80f7a60 msgr2=0x7f81d8107610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:44.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.146+0000 7f81dd5ed700 1 --2- 172.21.15.38:0/1846076398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81d80f7a60 0x7f81d8107610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:44.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.146+0000 7f81dd5ed700 1 -- 172.21.15.38:0/1846076398 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d4005040 con 0x7f81d80f67c0 2024-02-15T18:31:44.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.146+0000 7f81dcdec700 1 --2- 172.21.15.38:0/1846076398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81d80f7a60 0x7f81d8107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:31:44.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.146+0000 7f81dd5ed700 1 --2- 172.21.15.38:0/1846076398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81d80f67c0 0x7f81d8107120 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f81d4002040 tx=0x7f81d400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:44.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.146+0000 7f81c9ffb700 1 -- 172.21.15.38:0/1846076398 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81d4019070 con 0x7f81d80f67c0 2024-02-15T18:31:44.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.146+0000 7f81c9ffb700 1 -- 172.21.15.38:0/1846076398 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81d4007e90 con 0x7f81d80f67c0 2024-02-15T18:31:44.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.146+0000 7f81df851700 1 -- 172.21.15.38:0/1846076398 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d8108640 con 0x7f81d80f67c0 2024-02-15T18:31:44.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.146+0000 7f81c9ffb700 1 -- 172.21.15.38:0/1846076398 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81d4014d90 con 0x7f81d80f67c0 2024-02-15T18:31:44.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.147+0000 7f81df851700 1 -- 172.21.15.38:0/1846076398 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d8108ba0 con 0x7f81d80f67c0 2024-02-15T18:31:44.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.148+0000 7f81df851700 1 -- 172.21.15.38:0/1846076398 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f81d805dd30 con 0x7f81d80f67c0 2024-02-15T18:31:44.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.148+0000 7f81c9ffb700 1 -- 172.21.15.38:0/1846076398 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f81d4025070 con 0x7f81d80f67c0 2024-02-15T18:31:44.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.148+0000 7f81c9ffb700 1 --2- 172.21.15.38:0/1846076398 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f81c405c2a0 0x7f81c405e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:44.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.148+0000 7f81c9ffb700 1 -- 172.21.15.38:0/1846076398 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f81d4076a90 con 0x7f81d80f67c0 2024-02-15T18:31:44.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.149+0000 7f81dcdec700 1 --2- 172.21.15.38:0/1846076398 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f81c405c2a0 0x7f81c405e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:44.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.149+0000 7f81c9ffb700 1 --2- 172.21.15.38:0/1846076398 >> [v2:172.21.15.38:6802/1826650694,v1:172.21.15.38:6803/1826650694] conn(0x7f81c4062030 0x7f81c4064490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:44.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.149+0000 7f81c9ffb700 1 -- 172.21.15.38:0/1846076398 --> [v2:172.21.15.38:6802/1826650694,v1:172.21.15.38:6803/1826650694] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f81c4064b10 con 0x7f81c4062030 2024-02-15T18:31:44.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.149+0000 7f81c9ffb700 1 -- 172.21.15.38:0/1846076398 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81d407b220 con 0x7f81d80f67c0 2024-02-15T18:31:44.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.149+0000 7f81dddee700 1 --2- 172.21.15.38:0/1846076398 >> [v2:172.21.15.38:6802/1826650694,v1:172.21.15.38:6803/1826650694] conn(0x7f81c4062030 0x7f81c4064490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:44.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.149+0000 7f81dcdec700 1 --2- 172.21.15.38:0/1846076398 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f81c405c2a0 0x7f81c405e760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f81cc009da0 tx=0x7f81cc00b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:44.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.149+0000 7f81dddee700 1 --2- 172.21.15.38:0/1846076398 >> [v2:172.21.15.38:6802/1826650694,v1:172.21.15.38:6803/1826650694] conn(0x7f81c4062030 0x7f81c4064490 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:44.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.150+0000 7f81c9ffb700 1 -- 172.21.15.38:0/1846076398 <== osd.0 v2:172.21.15.38:6802/1826650694 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f81c4064b10 con 0x7f81c4062030 2024-02-15T18:31:44.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.164+0000 7f81df851700 1 -- 172.21.15.38:0/1846076398 --> [v2:172.21.15.38:6802/1826650694,v1:172.21.15.38:6803/1826650694] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f81d8109110 con 0x7f81c4062030 2024-02-15T18:31:44.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.164+0000 7f81c9ffb700 1 -- 172.21.15.38:0/1846076398 <== osd.0 v2:172.21.15.38:6802/1826650694 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f81d8109110 con 0x7f81c4062030 2024-02-15T18:31:44.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.165+0000 7f81df851700 1 -- 172.21.15.38:0/1846076398 >> [v2:172.21.15.38:6802/1826650694,v1:172.21.15.38:6803/1826650694] conn(0x7f81c4062030 msgr2=0x7f81c4064490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:44.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.165+0000 7f81df851700 1 --2- 172.21.15.38:0/1846076398 >> [v2:172.21.15.38:6802/1826650694,v1:172.21.15.38:6803/1826650694] conn(0x7f81c4062030 0x7f81c4064490 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:44.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.165+0000 7f81df851700 1 -- 172.21.15.38:0/1846076398 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f81c405c2a0 msgr2=0x7f81c405e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:44.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.165+0000 7f81df851700 1 --2- 172.21.15.38:0/1846076398 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f81c405c2a0 0x7f81c405e760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f81cc009da0 tx=0x7f81cc00b040).stop 2024-02-15T18:31:44.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.165+0000 7f81df851700 1 -- 172.21.15.38:0/1846076398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81d80f67c0 msgr2=0x7f81d8107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:44.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.165+0000 7f81df851700 1 --2- 172.21.15.38:0/1846076398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81d80f67c0 0x7f81d8107120 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f81d4002040 tx=0x7f81d400b040).stop 2024-02-15T18:31:44.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.165+0000 7f81df851700 1 -- 172.21.15.38:0/1846076398 shutdown_connections 2024-02-15T18:31:44.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.165+0000 7f81df851700 1 --2- 172.21.15.38:0/1846076398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81d80f67c0 0x7f81d8107120 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:44.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.165+0000 7f81df851700 1 --2- 172.21.15.38:0/1846076398 >> [v2:172.21.15.38:6802/1826650694,v1:172.21.15.38:6803/1826650694] conn(0x7f81c4062030 0x7f81c4064490 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:44.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.165+0000 7f81df851700 1 --2- 172.21.15.38:0/1846076398 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f81c405c2a0 0x7f81c405e760 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:44.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.165+0000 7f81df851700 1 --2- 172.21.15.38:0/1846076398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81d80f7a60 0x7f81d8107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:44.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.165+0000 7f81df851700 1 -- 172.21.15.38:0/1846076398 >> 172.21.15.38:0/1846076398 conn(0x7f81d80f1f20 msgr2=0x7f81d80fb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:44.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.165+0000 7f81df851700 1 -- 172.21.15.38:0/1846076398 shutdown_connections 2024-02-15T18:31:44.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:44.165+0000 7f81df851700 1 -- 172.21.15.38:0/1846076398 wait complete. 2024-02-15T18:31:44.846 INFO:teuthology.orchestra.run.smithi038.stdout:38654705699 2024-02-15T18:31:44.846 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-15T18:31:45.087 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/config 2024-02-15T18:31:46.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:44.264611+0000 mgr.smithi038.khesbl (mgr.14162) 208 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:46.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:44.264611+0000 mgr.smithi038.khesbl (mgr.14162) 208 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:47.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.102+0000 7f4431c63700 1 -- 172.21.15.38:0/3927233309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c0f67b0 msgr2=0x7f442c0f6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:47.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.102+0000 7f4431c63700 1 --2- 172.21.15.38:0/3927233309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c0f67b0 0x7f442c0f6c10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f4418004660 tx=0x7f4418009b30).stop 2024-02-15T18:31:47.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.102+0000 7f4431c63700 1 -- 172.21.15.38:0/3927233309 shutdown_connections 2024-02-15T18:31:47.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.102+0000 7f4431c63700 1 --2- 172.21.15.38:0/3927233309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c0f67b0 0x7f442c0f6c10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:47.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.102+0000 7f4431c63700 1 --2- 172.21.15.38:0/3927233309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f442c0f5510 0x7f442c0f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:47.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.102+0000 7f4431c63700 1 -- 172.21.15.38:0/3927233309 >> 172.21.15.38:0/3927233309 conn(0x7f442c0f0c70 msgr2=0x7f442c0f3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:47.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.102+0000 7f4431c63700 1 -- 172.21.15.38:0/3927233309 shutdown_connections 2024-02-15T18:31:47.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.102+0000 7f4431c63700 1 -- 172.21.15.38:0/3927233309 wait complete. 2024-02-15T18:31:47.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.102+0000 7f4431c63700 1 Processor -- start 2024-02-15T18:31:47.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.102+0000 7f4431c63700 1 -- start start 2024-02-15T18:31:47.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.103+0000 7f4431c63700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c0f5510 0x7f442c109c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:47.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.103+0000 7f4431c63700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f442c0f67b0 0x7f442c10c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:47.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.103+0000 7f4431c63700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442c060980 con 0x7f442c0f5510 2024-02-15T18:31:47.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.103+0000 7f4431c63700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442c060ac0 con 0x7f442c0f67b0 2024-02-15T18:31:47.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.103+0000 7f4430c61700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c0f5510 0x7f442c109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:47.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.103+0000 7f4430c61700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c0f5510 0x7f442c109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44384/0 (socket says 172.21.15.38:44384) 2024-02-15T18:31:47.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.103+0000 7f4430c61700 1 -- 172.21.15.38:0/3205708247 learned_addr learned my addr 172.21.15.38:0/3205708247 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:31:47.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.103+0000 7f4430c61700 1 -- 172.21.15.38:0/3205708247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f442c0f67b0 msgr2=0x7f442c10c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:47.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.103+0000 7f4430c61700 1 --2- 172.21.15.38:0/3205708247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f442c0f67b0 0x7f442c10c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:31:47.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.103+0000 7f4430c61700 1 -- 172.21.15.38:0/3205708247 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4418005040 con 0x7f442c0f5510 2024-02-15T18:31:47.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.103+0000 7f4430c61700 1 --2- 172.21.15.38:0/3205708247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c0f5510 0x7f442c109c10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f4420002700 tx=0x7f4420009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:47.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.103+0000 7f44297fa700 1 -- 172.21.15.38:0/3205708247 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4420009320 con 0x7f442c0f5510 2024-02-15T18:31:47.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.103+0000 7f44297fa700 1 -- 172.21.15.38:0/3205708247 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f442000f040 con 0x7f442c0f5510 2024-02-15T18:31:47.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.103+0000 7f4431c63700 1 -- 172.21.15.38:0/3205708247 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f442c10c720 con 0x7f442c0f5510 2024-02-15T18:31:47.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.103+0000 7f44297fa700 1 -- 172.21.15.38:0/3205708247 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4420014940 con 0x7f442c0f5510 2024-02-15T18:31:47.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.103+0000 7f4431c63700 1 -- 172.21.15.38:0/3205708247 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442c10ccb0 con 0x7f442c0f5510 2024-02-15T18:31:47.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.104+0000 7f4431c63700 1 -- 172.21.15.38:0/3205708247 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f442c05db80 con 0x7f442c0f5510 2024-02-15T18:31:47.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.105+0000 7f44297fa700 1 -- 172.21.15.38:0/3205708247 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f44200077b0 con 0x7f442c0f5510 2024-02-15T18:31:47.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.105+0000 7f44297fa700 1 --2- 172.21.15.38:0/3205708247 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f441c05c2b0 0x7f441c05e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:47.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.105+0000 7f44297fa700 1 -- 172.21.15.38:0/3205708247 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f4420075de0 con 0x7f442c0f5510 2024-02-15T18:31:47.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.105+0000 7f442bfff700 1 --2- 172.21.15.38:0/3205708247 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f441c05c2b0 0x7f441c05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:47.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.105+0000 7f44297fa700 1 --2- 172.21.15.38:0/3205708247 >> [v2:172.21.15.38:6810/3796697403,v1:172.21.15.38:6811/3796697403] conn(0x7f441c062040 0x7f441c0644a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:47.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.105+0000 7f44297fa700 1 -- 172.21.15.38:0/3205708247 --> [v2:172.21.15.38:6810/3796697403,v1:172.21.15.38:6811/3796697403] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f441c064b20 con 0x7f441c062040 2024-02-15T18:31:47.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.105+0000 7f44297fa700 1 -- 172.21.15.38:0/3205708247 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4420078450 con 0x7f442c0f5510 2024-02-15T18:31:47.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.105+0000 7f4431462700 1 --2- 172.21.15.38:0/3205708247 >> [v2:172.21.15.38:6810/3796697403,v1:172.21.15.38:6811/3796697403] conn(0x7f441c062040 0x7f441c0644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:47.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.106+0000 7f442bfff700 1 --2- 172.21.15.38:0/3205708247 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f441c05c2b0 0x7f441c05e770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f4418009f50 tx=0x7f4418009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:47.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.106+0000 7f4431462700 1 --2- 172.21.15.38:0/3205708247 >> [v2:172.21.15.38:6810/3796697403,v1:172.21.15.38:6811/3796697403] conn(0x7f441c062040 0x7f441c0644a0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:47.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.106+0000 7f44297fa700 1 -- 172.21.15.38:0/3205708247 <== osd.1 v2:172.21.15.38:6810/3796697403 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f441c064b20 con 0x7f441c062040 2024-02-15T18:31:47.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.121+0000 7f4431c63700 1 -- 172.21.15.38:0/3205708247 --> [v2:172.21.15.38:6810/3796697403,v1:172.21.15.38:6811/3796697403] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f442c0581d0 con 0x7f441c062040 2024-02-15T18:31:47.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.121+0000 7f44297fa700 1 -- 172.21.15.38:0/3205708247 <== osd.1 v2:172.21.15.38:6810/3796697403 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f442c0581d0 con 0x7f441c062040 2024-02-15T18:31:47.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.121+0000 7f4431c63700 1 -- 172.21.15.38:0/3205708247 >> [v2:172.21.15.38:6810/3796697403,v1:172.21.15.38:6811/3796697403] conn(0x7f441c062040 msgr2=0x7f441c0644a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:47.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.121+0000 7f4431c63700 1 --2- 172.21.15.38:0/3205708247 >> [v2:172.21.15.38:6810/3796697403,v1:172.21.15.38:6811/3796697403] conn(0x7f441c062040 0x7f441c0644a0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:47.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.122+0000 7f4431c63700 1 -- 172.21.15.38:0/3205708247 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f441c05c2b0 msgr2=0x7f441c05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:47.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.122+0000 7f4431c63700 1 --2- 172.21.15.38:0/3205708247 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f441c05c2b0 0x7f441c05e770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f4418009f50 tx=0x7f4418009b30).stop 2024-02-15T18:31:47.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.122+0000 7f4431c63700 1 -- 172.21.15.38:0/3205708247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c0f5510 msgr2=0x7f442c109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:47.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.122+0000 7f4431c63700 1 --2- 172.21.15.38:0/3205708247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c0f5510 0x7f442c109c10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f4420002700 tx=0x7f4420009d80).stop 2024-02-15T18:31:47.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.122+0000 7f4431c63700 1 -- 172.21.15.38:0/3205708247 shutdown_connections 2024-02-15T18:31:47.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.122+0000 7f4431c63700 1 --2- 172.21.15.38:0/3205708247 >> [v2:172.21.15.38:6810/3796697403,v1:172.21.15.38:6811/3796697403] conn(0x7f441c062040 0x7f441c0644a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:47.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.122+0000 7f4431c63700 1 --2- 172.21.15.38:0/3205708247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f442c0f5510 0x7f442c109c10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:47.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.122+0000 7f4431c63700 1 --2- 172.21.15.38:0/3205708247 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f441c05c2b0 0x7f441c05e770 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:47.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.122+0000 7f4431c63700 1 --2- 172.21.15.38:0/3205708247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f442c0f67b0 0x7f442c10c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:31:47.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.122+0000 7f4431c63700 1 -- 172.21.15.38:0/3205708247 >> 172.21.15.38:0/3205708247 conn(0x7f442c0f0c70 msgr2=0x7f442c0fa200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:47.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.122+0000 7f4431c63700 1 -- 172.21.15.38:0/3205708247 shutdown_connections 2024-02-15T18:31:47.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:47.122+0000 7f4431c63700 1 -- 172.21.15.38:0/3205708247 wait complete. 2024-02-15T18:31:47.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:46.265123+0000 mgr.smithi038.khesbl (mgr.14162) 209 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:47.885 INFO:teuthology.orchestra.run.smithi038.stdout:60129542171 2024-02-15T18:31:47.885 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-15T18:31:48.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:46.265123+0000 mgr.smithi038.khesbl (mgr.14162) 209 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:48.126 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/config 2024-02-15T18:31:49.745 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:48.265572+0000 mgr.smithi038.khesbl (mgr.14162) 210 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:50.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.023+0000 7f513b487700 1 -- 172.21.15.38:0/767915986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51340f6960 msgr2=0x7f51340f6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:50.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.023+0000 7f513b487700 1 --2- 172.21.15.38:0/767915986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51340f6960 0x7f51340f6dc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f5124004660 tx=0x7f5124009b30).stop 2024-02-15T18:31:50.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.023+0000 7f513b487700 1 -- 172.21.15.38:0/767915986 shutdown_connections 2024-02-15T18:31:50.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.023+0000 7f513b487700 1 --2- 172.21.15.38:0/767915986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51340f6960 0x7f51340f6dc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:50.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.023+0000 7f513b487700 1 --2- 172.21.15.38:0/767915986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51340f56c0 0x7f51340f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:50.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.023+0000 7f513b487700 1 -- 172.21.15.38:0/767915986 >> 172.21.15.38:0/767915986 conn(0x7f51340f0e20 msgr2=0x7f51340f3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:50.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.023+0000 7f513b487700 1 -- 172.21.15.38:0/767915986 shutdown_connections 2024-02-15T18:31:50.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.023+0000 7f513b487700 1 -- 172.21.15.38:0/767915986 wait complete. 2024-02-15T18:31:50.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.024+0000 7f513b487700 1 Processor -- start 2024-02-15T18:31:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.024+0000 7f513b487700 1 -- start start 2024-02-15T18:31:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.024+0000 7f513b487700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51340f56c0 0x7f5134109ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.024+0000 7f513b487700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51340f6960 0x7f513410c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.024+0000 7f513b487700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5134060940 con 0x7f51340f56c0 2024-02-15T18:31:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.024+0000 7f513b487700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5134060a80 con 0x7f51340f6960 2024-02-15T18:31:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.024+0000 7f5139223700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51340f56c0 0x7f5134109ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.024+0000 7f5139223700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51340f56c0 0x7f5134109ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44394/0 (socket says 172.21.15.38:44394) 2024-02-15T18:31:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.024+0000 7f5139223700 1 -- 172.21.15.38:0/3621015615 learned_addr learned my addr 172.21.15.38:0/3621015615 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:31:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.024+0000 7f5138a22700 1 --2- 172.21.15.38:0/3621015615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51340f6960 0x7f513410c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.025+0000 7f5139223700 1 -- 172.21.15.38:0/3621015615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51340f6960 msgr2=0x7f513410c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.025+0000 7f5139223700 1 --2- 172.21.15.38:0/3621015615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51340f6960 0x7f513410c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.025+0000 7f5139223700 1 -- 172.21.15.38:0/3621015615 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5124005040 con 0x7f51340f56c0 2024-02-15T18:31:50.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.025+0000 7f5138a22700 1 --2- 172.21.15.38:0/3621015615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51340f6960 0x7f513410c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:31:50.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.025+0000 7f5139223700 1 --2- 172.21.15.38:0/3621015615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51340f56c0 0x7f5134109ea0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f513000c530 tx=0x7f5130007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:50.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.025+0000 7f5129ffb700 1 -- 172.21.15.38:0/3621015615 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f513000f650 con 0x7f51340f56c0 2024-02-15T18:31:50.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.025+0000 7f513b487700 1 -- 172.21.15.38:0/3621015615 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f513410c950 con 0x7f51340f56c0 2024-02-15T18:31:50.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.025+0000 7f5129ffb700 1 -- 172.21.15.38:0/3621015615 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f513000fc80 con 0x7f51340f56c0 2024-02-15T18:31:50.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.025+0000 7f5129ffb700 1 -- 172.21.15.38:0/3621015615 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5130012e20 con 0x7f51340f56c0 2024-02-15T18:31:50.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.025+0000 7f513b487700 1 -- 172.21.15.38:0/3621015615 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513410cf10 con 0x7f51340f56c0 2024-02-15T18:31:50.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.026+0000 7f513b487700 1 -- 172.21.15.38:0/3621015615 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f513405dd30 con 0x7f51340f56c0 2024-02-15T18:31:50.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.026+0000 7f5129ffb700 1 -- 172.21.15.38:0/3621015615 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f5130019030 con 0x7f51340f56c0 2024-02-15T18:31:50.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.027+0000 7f5129ffb700 1 --2- 172.21.15.38:0/3621015615 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f512005c2a0 0x7f512005e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:50.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.027+0000 7f5129ffb700 1 -- 172.21.15.38:0/3621015615 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f51300758a0 con 0x7f51340f56c0 2024-02-15T18:31:50.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.027+0000 7f5138a22700 1 --2- 172.21.15.38:0/3621015615 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f512005c2a0 0x7f512005e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:50.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.027+0000 7f5129ffb700 1 --2- 172.21.15.38:0/3621015615 >> [v2:172.21.15.38:6818/420320553,v1:172.21.15.38:6819/420320553] conn(0x7f5120062080 0x7f51200644e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:50.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.027+0000 7f5129ffb700 1 -- 172.21.15.38:0/3621015615 --> [v2:172.21.15.38:6818/420320553,v1:172.21.15.38:6819/420320553] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5120064b60 con 0x7f5120062080 2024-02-15T18:31:50.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.027+0000 7f5129ffb700 1 -- 172.21.15.38:0/3621015615 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f513007a220 con 0x7f51340f56c0 2024-02-15T18:31:50.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.027+0000 7f5139a24700 1 --2- 172.21.15.38:0/3621015615 >> [v2:172.21.15.38:6818/420320553,v1:172.21.15.38:6819/420320553] conn(0x7f5120062080 0x7f51200644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:50.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.027+0000 7f5138a22700 1 --2- 172.21.15.38:0/3621015615 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f512005c2a0 0x7f512005e760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f5124005010 tx=0x7f5124009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:50.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.027+0000 7f5139a24700 1 --2- 172.21.15.38:0/3621015615 >> [v2:172.21.15.38:6818/420320553,v1:172.21.15.38:6819/420320553] conn(0x7f5120062080 0x7f51200644e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:50.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.028+0000 7f5129ffb700 1 -- 172.21.15.38:0/3621015615 <== osd.2 v2:172.21.15.38:6818/420320553 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f5120064b60 con 0x7f5120062080 2024-02-15T18:31:50.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.043+0000 7f513b487700 1 -- 172.21.15.38:0/3621015615 --> [v2:172.21.15.38:6818/420320553,v1:172.21.15.38:6819/420320553] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f51340581d0 con 0x7f5120062080 2024-02-15T18:31:50.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.043+0000 7f5129ffb700 1 -- 172.21.15.38:0/3621015615 <== osd.2 v2:172.21.15.38:6818/420320553 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f51340581d0 con 0x7f5120062080 2024-02-15T18:31:50.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.043+0000 7f513b487700 1 -- 172.21.15.38:0/3621015615 >> [v2:172.21.15.38:6818/420320553,v1:172.21.15.38:6819/420320553] conn(0x7f5120062080 msgr2=0x7f51200644e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:50.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.043+0000 7f513b487700 1 --2- 172.21.15.38:0/3621015615 >> [v2:172.21.15.38:6818/420320553,v1:172.21.15.38:6819/420320553] conn(0x7f5120062080 0x7f51200644e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:50.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.043+0000 7f513b487700 1 -- 172.21.15.38:0/3621015615 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f512005c2a0 msgr2=0x7f512005e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:50.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.043+0000 7f513b487700 1 --2- 172.21.15.38:0/3621015615 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f512005c2a0 0x7f512005e760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f5124005010 tx=0x7f5124009b30).stop 2024-02-15T18:31:50.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.043+0000 7f513b487700 1 -- 172.21.15.38:0/3621015615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51340f56c0 msgr2=0x7f5134109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:50.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.043+0000 7f513b487700 1 --2- 172.21.15.38:0/3621015615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51340f56c0 0x7f5134109ea0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f513000c530 tx=0x7f5130007e10).stop 2024-02-15T18:31:50.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.043+0000 7f513b487700 1 -- 172.21.15.38:0/3621015615 shutdown_connections 2024-02-15T18:31:50.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.043+0000 7f513b487700 1 --2- 172.21.15.38:0/3621015615 >> [v2:172.21.15.38:6818/420320553,v1:172.21.15.38:6819/420320553] conn(0x7f5120062080 0x7f51200644e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:50.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.043+0000 7f513b487700 1 --2- 172.21.15.38:0/3621015615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51340f56c0 0x7f5134109ea0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:50.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.043+0000 7f513b487700 1 --2- 172.21.15.38:0/3621015615 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f512005c2a0 0x7f512005e760 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:50.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.043+0000 7f513b487700 1 --2- 172.21.15.38:0/3621015615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51340f6960 0x7f513410c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:50.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.043+0000 7f513b487700 1 -- 172.21.15.38:0/3621015615 >> 172.21.15.38:0/3621015615 conn(0x7f51340f0e20 msgr2=0x7f51340fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:50.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.043+0000 7f513b487700 1 -- 172.21.15.38:0/3621015615 shutdown_connections 2024-02-15T18:31:50.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:50.043+0000 7f513b487700 1 -- 172.21.15.38:0/3621015615 wait complete. 2024-02-15T18:31:50.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:48.265572+0000 mgr.smithi038.khesbl (mgr.14162) 210 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:50.797 INFO:teuthology.orchestra.run.smithi038.stdout:81604378648 2024-02-15T18:31:50.797 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-15T18:31:51.032 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/config 2024-02-15T18:31:52.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:50.266023+0000 mgr.smithi038.khesbl (mgr.14162) 211 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:52.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:50.266023+0000 mgr.smithi038.khesbl (mgr.14162) 211 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:53.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.293+0000 7f1c68eb7700 1 -- 172.21.15.38:0/2664426185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c640f67b0 msgr2=0x7f1c640f6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:53.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.293+0000 7f1c68eb7700 1 --2- 172.21.15.38:0/2664426185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c640f67b0 0x7f1c640f6c10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f1c58004660 tx=0x7f1c58009b30).stop 2024-02-15T18:31:53.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.293+0000 7f1c68eb7700 1 -- 172.21.15.38:0/2664426185 shutdown_connections 2024-02-15T18:31:53.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.293+0000 7f1c68eb7700 1 --2- 172.21.15.38:0/2664426185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c640f67b0 0x7f1c640f6c10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:53.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.293+0000 7f1c68eb7700 1 --2- 172.21.15.38:0/2664426185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c640f5510 0x7f1c640f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:53.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.293+0000 7f1c68eb7700 1 -- 172.21.15.38:0/2664426185 >> 172.21.15.38:0/2664426185 conn(0x7f1c640f0c70 msgr2=0x7f1c640f3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:53.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.293+0000 7f1c68eb7700 1 -- 172.21.15.38:0/2664426185 shutdown_connections 2024-02-15T18:31:53.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.293+0000 7f1c68eb7700 1 -- 172.21.15.38:0/2664426185 wait complete. 2024-02-15T18:31:53.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.294+0000 7f1c68eb7700 1 Processor -- start 2024-02-15T18:31:53.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.294+0000 7f1c68eb7700 1 -- start start 2024-02-15T18:31:53.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.294+0000 7f1c68eb7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c640f5510 0x7f1c64105ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:53.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.294+0000 7f1c68eb7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c640f67b0 0x7f1c641063b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:53.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.294+0000 7f1c68eb7700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c641069b0 con 0x7f1c640f67b0 2024-02-15T18:31:53.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.294+0000 7f1c68eb7700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c641071c0 con 0x7f1c640f5510 2024-02-15T18:31:53.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.294+0000 7f1c62ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c640f67b0 0x7f1c641063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:53.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.294+0000 7f1c62ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c640f67b0 0x7f1c641063b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44418/0 (socket says 172.21.15.38:44418) 2024-02-15T18:31:53.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.294+0000 7f1c62ffd700 1 -- 172.21.15.38:0/3877412376 learned_addr learned my addr 172.21.15.38:0/3877412376 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:31:53.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.294+0000 7f1c637fe700 1 --2- 172.21.15.38:0/3877412376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c640f5510 0x7f1c64105ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:53.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.294+0000 7f1c62ffd700 1 -- 172.21.15.38:0/3877412376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c640f5510 msgr2=0x7f1c64105ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:53.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.294+0000 7f1c62ffd700 1 --2- 172.21.15.38:0/3877412376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c640f5510 0x7f1c64105ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:53.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.295+0000 7f1c62ffd700 1 -- 172.21.15.38:0/3877412376 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c58005040 con 0x7f1c640f67b0 2024-02-15T18:31:53.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.295+0000 7f1c62ffd700 1 --2- 172.21.15.38:0/3877412376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c640f67b0 0x7f1c641063b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f1c58009830 tx=0x7f1c58009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:53.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.295+0000 7f1c4bfff700 1 -- 172.21.15.38:0/3877412376 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c58018070 con 0x7f1c640f67b0 2024-02-15T18:31:53.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.295+0000 7f1c4bfff700 1 -- 172.21.15.38:0/3877412376 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c58007d70 con 0x7f1c640f67b0 2024-02-15T18:31:53.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.295+0000 7f1c68eb7700 1 -- 172.21.15.38:0/3877412376 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c641073e0 con 0x7f1c640f67b0 2024-02-15T18:31:53.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.295+0000 7f1c4bfff700 1 -- 172.21.15.38:0/3877412376 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c5801c620 con 0x7f1c640f67b0 2024-02-15T18:31:53.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.295+0000 7f1c68eb7700 1 -- 172.21.15.38:0/3877412376 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c64107940 con 0x7f1c640f67b0 2024-02-15T18:31:53.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.296+0000 7f1c68eb7700 1 -- 172.21.15.38:0/3877412376 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1c6405db80 con 0x7f1c640f67b0 2024-02-15T18:31:53.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.296+0000 7f1c4bfff700 1 -- 172.21.15.38:0/3877412376 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f1c5801d980 con 0x7f1c640f67b0 2024-02-15T18:31:53.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.297+0000 7f1c4bfff700 1 --2- 172.21.15.38:0/3877412376 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1c4c05c300 0x7f1c4c05e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:53.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.297+0000 7f1c4bfff700 1 -- 172.21.15.38:0/3877412376 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f1c58077a90 con 0x7f1c640f67b0 2024-02-15T18:31:53.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.297+0000 7f1c637fe700 1 --2- 172.21.15.38:0/3877412376 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1c4c05c300 0x7f1c4c05e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:53.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.297+0000 7f1c4bfff700 1 --2- 172.21.15.38:0/3877412376 >> [v2:172.21.15.121:6800/3071118694,v1:172.21.15.121:6801/3071118694] conn(0x7f1c4c062090 0x7f1c4c0644f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:53.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.297+0000 7f1c4bfff700 1 -- 172.21.15.38:0/3877412376 --> [v2:172.21.15.121:6800/3071118694,v1:172.21.15.121:6801/3071118694] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c4c064b70 con 0x7f1c4c062090 2024-02-15T18:31:53.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.297+0000 7f1c4bfff700 1 -- 172.21.15.38:0/3877412376 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c5807c1a0 con 0x7f1c640f67b0 2024-02-15T18:31:53.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.297+0000 7f1c63fff700 1 --2- 172.21.15.38:0/3877412376 >> [v2:172.21.15.121:6800/3071118694,v1:172.21.15.121:6801/3071118694] conn(0x7f1c4c062090 0x7f1c4c0644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:53.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.297+0000 7f1c637fe700 1 --2- 172.21.15.38:0/3877412376 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1c4c05c300 0x7f1c4c05e7c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f1c64105c30 tx=0x7f1c54009040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:53.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.297+0000 7f1c63fff700 1 --2- 172.21.15.38:0/3877412376 >> [v2:172.21.15.121:6800/3071118694,v1:172.21.15.121:6801/3071118694] conn(0x7f1c4c062090 0x7f1c4c0644f0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:53.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.298+0000 7f1c4bfff700 1 -- 172.21.15.38:0/3877412376 <== osd.3 v2:172.21.15.121:6800/3071118694 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f1c4c064b70 con 0x7f1c4c062090 2024-02-15T18:31:53.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.313+0000 7f1c68eb7700 1 -- 172.21.15.38:0/3877412376 --> [v2:172.21.15.121:6800/3071118694,v1:172.21.15.121:6801/3071118694] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1c64107eb0 con 0x7f1c4c062090 2024-02-15T18:31:53.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.313+0000 7f1c4bfff700 1 -- 172.21.15.38:0/3877412376 <== osd.3 v2:172.21.15.121:6800/3071118694 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1c64107eb0 con 0x7f1c4c062090 2024-02-15T18:31:53.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.314+0000 7f1c68eb7700 1 -- 172.21.15.38:0/3877412376 >> [v2:172.21.15.121:6800/3071118694,v1:172.21.15.121:6801/3071118694] conn(0x7f1c4c062090 msgr2=0x7f1c4c0644f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:53.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.314+0000 7f1c68eb7700 1 --2- 172.21.15.38:0/3877412376 >> [v2:172.21.15.121:6800/3071118694,v1:172.21.15.121:6801/3071118694] conn(0x7f1c4c062090 0x7f1c4c0644f0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:53.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.314+0000 7f1c68eb7700 1 -- 172.21.15.38:0/3877412376 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1c4c05c300 msgr2=0x7f1c4c05e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:53.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.314+0000 7f1c68eb7700 1 --2- 172.21.15.38:0/3877412376 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1c4c05c300 0x7f1c4c05e7c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f1c64105c30 tx=0x7f1c54009040).stop 2024-02-15T18:31:53.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.314+0000 7f1c68eb7700 1 -- 172.21.15.38:0/3877412376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c640f67b0 msgr2=0x7f1c641063b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:53.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.314+0000 7f1c68eb7700 1 --2- 172.21.15.38:0/3877412376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c640f67b0 0x7f1c641063b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f1c58009830 tx=0x7f1c58009b30).stop 2024-02-15T18:31:53.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.314+0000 7f1c68eb7700 1 -- 172.21.15.38:0/3877412376 shutdown_connections 2024-02-15T18:31:53.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.314+0000 7f1c68eb7700 1 --2- 172.21.15.38:0/3877412376 >> [v2:172.21.15.121:6800/3071118694,v1:172.21.15.121:6801/3071118694] conn(0x7f1c4c062090 0x7f1c4c0644f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:53.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.314+0000 7f1c68eb7700 1 --2- 172.21.15.38:0/3877412376 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1c4c05c300 0x7f1c4c05e7c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:53.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.314+0000 7f1c68eb7700 1 --2- 172.21.15.38:0/3877412376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c640f5510 0x7f1c64105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:53.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.314+0000 7f1c68eb7700 1 --2- 172.21.15.38:0/3877412376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1c640f67b0 0x7f1c641063b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:53.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.314+0000 7f1c68eb7700 1 -- 172.21.15.38:0/3877412376 >> 172.21.15.38:0/3877412376 conn(0x7f1c640f0c70 msgr2=0x7f1c640fa200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:53.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.314+0000 7f1c68eb7700 1 -- 172.21.15.38:0/3877412376 shutdown_connections 2024-02-15T18:31:53.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:53.314+0000 7f1c68eb7700 1 -- 172.21.15.38:0/3877412376 wait complete. 2024-02-15T18:31:54.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:52.266551+0000 mgr.smithi038.khesbl (mgr.14162) 212 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:54.062 INFO:teuthology.orchestra.run.smithi038.stdout:111669149715 2024-02-15T18:31:54.062 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-15T18:31:54.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:52.266551+0000 mgr.smithi038.khesbl (mgr.14162) 212 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:54.302 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/config 2024-02-15T18:31:55.970 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:54.266893+0000 mgr.smithi038.khesbl (mgr.14162) 213 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:56.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:54.266893+0000 mgr.smithi038.khesbl (mgr.14162) 213 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:56.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.227+0000 7fef94e53700 1 -- 172.21.15.38:0/3830791765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef900607d0 msgr2=0x7fef900f9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:56.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.227+0000 7fef94e53700 1 --2- 172.21.15.38:0/3830791765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef900607d0 0x7fef900f9440 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fef880099e0 tx=0x7fef88004d10).stop 2024-02-15T18:31:56.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.227+0000 7fef94e53700 1 -- 172.21.15.38:0/3830791765 shutdown_connections 2024-02-15T18:31:56.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.227+0000 7fef94e53700 1 --2- 172.21.15.38:0/3830791765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef900f9930 0x7fef900fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:56.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.227+0000 7fef94e53700 1 --2- 172.21.15.38:0/3830791765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef900607d0 0x7fef900f9440 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:56.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.227+0000 7fef94e53700 1 -- 172.21.15.38:0/3830791765 >> 172.21.15.38:0/3830791765 conn(0x7fef900f0e20 msgr2=0x7fef900f3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:56.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.227+0000 7fef94e53700 1 -- 172.21.15.38:0/3830791765 shutdown_connections 2024-02-15T18:31:56.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.227+0000 7fef94e53700 1 -- 172.21.15.38:0/3830791765 wait complete. 2024-02-15T18:31:56.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.228+0000 7fef94e53700 1 Processor -- start 2024-02-15T18:31:56.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.228+0000 7fef94e53700 1 -- start start 2024-02-15T18:31:56.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.228+0000 7fef94e53700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef900607d0 0x7fef901072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:56.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.228+0000 7fef94e53700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef900f9930 0x7fef90107790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:56.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.228+0000 7fef94e53700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef90107d40 con 0x7fef900607d0 2024-02-15T18:31:56.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.228+0000 7fef94e53700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef90107e80 con 0x7fef900f9930 2024-02-15T18:31:56.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.228+0000 7fef8e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef900607d0 0x7fef901072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:56.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.228+0000 7fef8e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef900607d0 0x7fef901072a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47368/0 (socket says 172.21.15.38:47368) 2024-02-15T18:31:56.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.228+0000 7fef8e59c700 1 -- 172.21.15.38:0/1787745181 learned_addr learned my addr 172.21.15.38:0/1787745181 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:31:56.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.228+0000 7fef8dd9b700 1 --2- 172.21.15.38:0/1787745181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef900f9930 0x7fef90107790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:56.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.228+0000 7fef8e59c700 1 -- 172.21.15.38:0/1787745181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef900f9930 msgr2=0x7fef90107790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:56.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.228+0000 7fef8e59c700 1 --2- 172.21.15.38:0/1787745181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef900f9930 0x7fef90107790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:56.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.228+0000 7fef8e59c700 1 -- 172.21.15.38:0/1787745181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef88009640 con 0x7fef900607d0 2024-02-15T18:31:56.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.228+0000 7fef8dd9b700 1 --2- 172.21.15.38:0/1787745181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef900f9930 0x7fef90107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:31:56.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.229+0000 7fef8e59c700 1 --2- 172.21.15.38:0/1787745181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef900607d0 0x7fef901072a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fef880078d0 tx=0x7fef88004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:56.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.229+0000 7fef86ffd700 1 -- 172.21.15.38:0/1787745181 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef8800e070 con 0x7fef900607d0 2024-02-15T18:31:56.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.229+0000 7fef94e53700 1 -- 172.21.15.38:0/1787745181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef901087c0 con 0x7fef900607d0 2024-02-15T18:31:56.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.229+0000 7fef86ffd700 1 -- 172.21.15.38:0/1787745181 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef8801d8f0 con 0x7fef900607d0 2024-02-15T18:31:56.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.229+0000 7fef86ffd700 1 -- 172.21.15.38:0/1787745181 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef88018070 con 0x7fef900607d0 2024-02-15T18:31:56.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.229+0000 7fef94e53700 1 -- 172.21.15.38:0/1787745181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef90108d20 con 0x7fef900607d0 2024-02-15T18:31:56.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.230+0000 7fef94e53700 1 -- 172.21.15.38:0/1787745181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fef9005dd30 con 0x7fef900607d0 2024-02-15T18:31:56.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.230+0000 7fef86ffd700 1 -- 172.21.15.38:0/1787745181 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fef88024060 con 0x7fef900607d0 2024-02-15T18:31:56.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.230+0000 7fef86ffd700 1 --2- 172.21.15.38:0/1787745181 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fef7c05c2f0 0x7fef7c05e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:56.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.231+0000 7fef86ffd700 1 -- 172.21.15.38:0/1787745181 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7fef88077430 con 0x7fef900607d0 2024-02-15T18:31:56.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.231+0000 7fef8dd9b700 1 --2- 172.21.15.38:0/1787745181 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fef7c05c2f0 0x7fef7c05e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:56.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.231+0000 7fef86ffd700 1 --2- 172.21.15.38:0/1787745181 >> [v2:172.21.15.121:6808/3016245330,v1:172.21.15.121:6809/3016245330] conn(0x7fef7c062080 0x7fef7c0644e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:56.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.231+0000 7fef86ffd700 1 -- 172.21.15.38:0/1787745181 --> [v2:172.21.15.121:6808/3016245330,v1:172.21.15.121:6809/3016245330] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef7c064b60 con 0x7fef7c062080 2024-02-15T18:31:56.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.231+0000 7fef86ffd700 1 -- 172.21.15.38:0/1787745181 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef88079910 con 0x7fef900607d0 2024-02-15T18:31:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.231+0000 7fef8dd9b700 1 --2- 172.21.15.38:0/1787745181 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fef7c05c2f0 0x7fef7c05e7b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fef900605a0 tx=0x7fef7800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.231+0000 7fef8ed9d700 1 --2- 172.21.15.38:0/1787745181 >> [v2:172.21.15.121:6808/3016245330,v1:172.21.15.121:6809/3016245330] conn(0x7fef7c062080 0x7fef7c0644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.232+0000 7fef8ed9d700 1 --2- 172.21.15.38:0/1787745181 >> [v2:172.21.15.121:6808/3016245330,v1:172.21.15.121:6809/3016245330] conn(0x7fef7c062080 0x7fef7c0644e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.232+0000 7fef86ffd700 1 -- 172.21.15.38:0/1787745181 <== osd.4 v2:172.21.15.121:6808/3016245330 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fef7c064b60 con 0x7fef7c062080 2024-02-15T18:31:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.247+0000 7fef94e53700 1 -- 172.21.15.38:0/1787745181 --> [v2:172.21.15.121:6808/3016245330,v1:172.21.15.121:6809/3016245330] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fef90109180 con 0x7fef7c062080 2024-02-15T18:31:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.247+0000 7fef86ffd700 1 -- 172.21.15.38:0/1787745181 <== osd.4 v2:172.21.15.121:6808/3016245330 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fef90109180 con 0x7fef7c062080 2024-02-15T18:31:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.248+0000 7fef94e53700 1 -- 172.21.15.38:0/1787745181 >> [v2:172.21.15.121:6808/3016245330,v1:172.21.15.121:6809/3016245330] conn(0x7fef7c062080 msgr2=0x7fef7c0644e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:56.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.248+0000 7fef94e53700 1 --2- 172.21.15.38:0/1787745181 >> [v2:172.21.15.121:6808/3016245330,v1:172.21.15.121:6809/3016245330] conn(0x7fef7c062080 0x7fef7c0644e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:56.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.248+0000 7fef94e53700 1 -- 172.21.15.38:0/1787745181 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fef7c05c2f0 msgr2=0x7fef7c05e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:56.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.248+0000 7fef94e53700 1 --2- 172.21.15.38:0/1787745181 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fef7c05c2f0 0x7fef7c05e7b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fef900605a0 tx=0x7fef7800b040).stop 2024-02-15T18:31:56.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.248+0000 7fef94e53700 1 -- 172.21.15.38:0/1787745181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef900607d0 msgr2=0x7fef901072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:56.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.248+0000 7fef94e53700 1 --2- 172.21.15.38:0/1787745181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef900607d0 0x7fef901072a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fef880078d0 tx=0x7fef88004d10).stop 2024-02-15T18:31:56.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.248+0000 7fef94e53700 1 -- 172.21.15.38:0/1787745181 shutdown_connections 2024-02-15T18:31:56.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.248+0000 7fef94e53700 1 --2- 172.21.15.38:0/1787745181 >> [v2:172.21.15.121:6808/3016245330,v1:172.21.15.121:6809/3016245330] conn(0x7fef7c062080 0x7fef7c0644e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:56.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.248+0000 7fef94e53700 1 --2- 172.21.15.38:0/1787745181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef900607d0 0x7fef901072a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:56.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.248+0000 7fef94e53700 1 --2- 172.21.15.38:0/1787745181 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fef7c05c2f0 0x7fef7c05e7b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:56.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.248+0000 7fef94e53700 1 --2- 172.21.15.38:0/1787745181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef900f9930 0x7fef90107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:56.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.248+0000 7fef94e53700 1 -- 172.21.15.38:0/1787745181 >> 172.21.15.38:0/1787745181 conn(0x7fef900f0e20 msgr2=0x7fef900f3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:56.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.248+0000 7fef94e53700 1 -- 172.21.15.38:0/1787745181 shutdown_connections 2024-02-15T18:31:56.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:56.248+0000 7fef94e53700 1 -- 172.21.15.38:0/1787745181 wait complete. 2024-02-15T18:31:57.110 INFO:teuthology.orchestra.run.smithi038.stdout:137438953486 2024-02-15T18:31:57.111 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-15T18:31:57.353 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/config 2024-02-15T18:31:57.904 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:56.267370+0000 mgr.smithi038.khesbl (mgr.14162) 214 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:58.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:56.267370+0000 mgr.smithi038.khesbl (mgr.14162) 214 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:59.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.269+0000 7f0c04ce8700 1 -- 172.21.15.38:0/383574160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c000f5fa0 msgr2=0x7f0c000f69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:59.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.269+0000 7f0c04ce8700 1 --2- 172.21.15.38:0/383574160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c000f5fa0 0x7f0c000f69d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f0be8004660 tx=0x7f0be8009b30).stop 2024-02-15T18:31:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.269+0000 7f0c04ce8700 1 -- 172.21.15.38:0/383574160 shutdown_connections 2024-02-15T18:31:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.269+0000 7f0c04ce8700 1 --2- 172.21.15.38:0/383574160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c000f5fa0 0x7f0c000f69d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.269+0000 7f0c04ce8700 1 --2- 172.21.15.38:0/383574160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c000f5690 0x7f0c000f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:31:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.269+0000 7f0c04ce8700 1 -- 172.21.15.38:0/383574160 >> 172.21.15.38:0/383574160 conn(0x7f0c000f0eb0 msgr2=0x7f0c000f32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.269+0000 7f0c04ce8700 1 -- 172.21.15.38:0/383574160 shutdown_connections 2024-02-15T18:31:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.269+0000 7f0c04ce8700 1 -- 172.21.15.38:0/383574160 wait complete. 2024-02-15T18:31:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.270+0000 7f0c04ce8700 1 Processor -- start 2024-02-15T18:31:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.270+0000 7f0c04ce8700 1 -- start start 2024-02-15T18:31:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.270+0000 7f0c04ce8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c000f5690 0x7f0c0010c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.270+0000 7f0c04ce8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c000f5fa0 0x7f0c0010e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.270+0000 7f0c04ce8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c00060810 con 0x7f0c000f5690 2024-02-15T18:31:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.270+0000 7f0c04ce8700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c00060950 con 0x7f0c000f5fa0 2024-02-15T18:31:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.270+0000 7f0bfe59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c000f5690 0x7f0c0010c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.270+0000 7f0bfe59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c000f5690 0x7f0c0010c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47372/0 (socket says 172.21.15.38:47372) 2024-02-15T18:31:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.270+0000 7f0bfe59c700 1 -- 172.21.15.38:0/4143827491 learned_addr learned my addr 172.21.15.38:0/4143827491 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:31:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.270+0000 7f0bfe59c700 1 -- 172.21.15.38:0/4143827491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c000f5fa0 msgr2=0x7f0c0010e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:59.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.270+0000 7f0bfdd9b700 1 --2- 172.21.15.38:0/4143827491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c000f5fa0 0x7f0c0010e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:59.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.271+0000 7f0bfe59c700 1 --2- 172.21.15.38:0/4143827491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c000f5fa0 0x7f0c0010e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.271+0000 7f0bfe59c700 1 -- 172.21.15.38:0/4143827491 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be8005040 con 0x7f0c000f5690 2024-02-15T18:31:59.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.271+0000 7f0bfdd9b700 1 --2- 172.21.15.38:0/4143827491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c000f5fa0 0x7f0c0010e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:31:59.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.271+0000 7f0bfe59c700 1 --2- 172.21.15.38:0/4143827491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c000f5690 0x7f0c0010c1e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f0bf000b410 tx=0x7f0bf0006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:59.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.271+0000 7f0bf6ffd700 1 -- 172.21.15.38:0/4143827491 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bf00125f0 con 0x7f0c000f5690 2024-02-15T18:31:59.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.271+0000 7f0bf6ffd700 1 -- 172.21.15.38:0/4143827491 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0bf0012c20 con 0x7f0c000f5690 2024-02-15T18:31:59.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.271+0000 7f0c04ce8700 1 -- 172.21.15.38:0/4143827491 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0010ebd0 con 0x7f0c000f5690 2024-02-15T18:31:59.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.271+0000 7f0bf6ffd700 1 -- 172.21.15.38:0/4143827491 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bf0002b30 con 0x7f0c000f5690 2024-02-15T18:31:59.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.271+0000 7f0c04ce8700 1 -- 172.21.15.38:0/4143827491 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0010f030 con 0x7f0c000f5690 2024-02-15T18:31:59.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.272+0000 7f0c04ce8700 1 -- 172.21.15.38:0/4143827491 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0c0005dc70 con 0x7f0c000f5690 2024-02-15T18:31:59.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.272+0000 7f0bf6ffd700 1 -- 172.21.15.38:0/4143827491 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f0bf000f030 con 0x7f0c000f5690 2024-02-15T18:31:59.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.272+0000 7f0bf6ffd700 1 --2- 172.21.15.38:0/4143827491 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f0bec05c300 0x7f0bec05e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:59.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.273+0000 7f0bf6ffd700 1 -- 172.21.15.38:0/4143827491 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f0bf0076010 con 0x7f0c000f5690 2024-02-15T18:31:59.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.273+0000 7f0bfdd9b700 1 --2- 172.21.15.38:0/4143827491 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f0bec05c300 0x7f0bec05e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:59.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.273+0000 7f0bf6ffd700 1 --2- 172.21.15.38:0/4143827491 >> [v2:172.21.15.121:6816/4062288290,v1:172.21.15.121:6817/4062288290] conn(0x7f0bec062090 0x7f0bec0644f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:59.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.273+0000 7f0bf6ffd700 1 -- 172.21.15.38:0/4143827491 --> [v2:172.21.15.121:6816/4062288290,v1:172.21.15.121:6817/4062288290] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0bec064b70 con 0x7f0bec062090 2024-02-15T18:31:59.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.273+0000 7f0bf6ffd700 1 -- 172.21.15.38:0/4143827491 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bf0078680 con 0x7f0c000f5690 2024-02-15T18:31:59.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.273+0000 7f0bfdd9b700 1 --2- 172.21.15.38:0/4143827491 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f0bec05c300 0x7f0bec05e7c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f0be8005010 tx=0x7f0be8009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:59.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.273+0000 7f0bfed9d700 1 --2- 172.21.15.38:0/4143827491 >> [v2:172.21.15.121:6816/4062288290,v1:172.21.15.121:6817/4062288290] conn(0x7f0bec062090 0x7f0bec0644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:59.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.274+0000 7f0bfed9d700 1 --2- 172.21.15.38:0/4143827491 >> [v2:172.21.15.121:6816/4062288290,v1:172.21.15.121:6817/4062288290] conn(0x7f0bec062090 0x7f0bec0644f0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:59.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.274+0000 7f0bf6ffd700 1 -- 172.21.15.38:0/4143827491 <== osd.5 v2:172.21.15.121:6816/4062288290 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f0bec064b70 con 0x7f0bec062090 2024-02-15T18:31:59.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.289+0000 7f0c04ce8700 1 -- 172.21.15.38:0/4143827491 --> [v2:172.21.15.121:6816/4062288290,v1:172.21.15.121:6817/4062288290] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0c000581d0 con 0x7f0bec062090 2024-02-15T18:31:59.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.290+0000 7f0bf6ffd700 1 -- 172.21.15.38:0/4143827491 <== osd.5 v2:172.21.15.121:6816/4062288290 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0c000581d0 con 0x7f0bec062090 2024-02-15T18:31:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.290+0000 7f0c04ce8700 1 -- 172.21.15.38:0/4143827491 >> [v2:172.21.15.121:6816/4062288290,v1:172.21.15.121:6817/4062288290] conn(0x7f0bec062090 msgr2=0x7f0bec0644f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.290+0000 7f0c04ce8700 1 --2- 172.21.15.38:0/4143827491 >> [v2:172.21.15.121:6816/4062288290,v1:172.21.15.121:6817/4062288290] conn(0x7f0bec062090 0x7f0bec0644f0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.290+0000 7f0c04ce8700 1 -- 172.21.15.38:0/4143827491 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f0bec05c300 msgr2=0x7f0bec05e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.290+0000 7f0c04ce8700 1 --2- 172.21.15.38:0/4143827491 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f0bec05c300 0x7f0bec05e7c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f0be8005010 tx=0x7f0be8009b30).stop 2024-02-15T18:31:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.290+0000 7f0c04ce8700 1 -- 172.21.15.38:0/4143827491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c000f5690 msgr2=0x7f0c0010c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.290+0000 7f0c04ce8700 1 --2- 172.21.15.38:0/4143827491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c000f5690 0x7f0c0010c1e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f0bf000b410 tx=0x7f0bf0006ef0).stop 2024-02-15T18:31:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.290+0000 7f0c04ce8700 1 -- 172.21.15.38:0/4143827491 shutdown_connections 2024-02-15T18:31:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.290+0000 7f0c04ce8700 1 --2- 172.21.15.38:0/4143827491 >> [v2:172.21.15.121:6816/4062288290,v1:172.21.15.121:6817/4062288290] conn(0x7f0bec062090 0x7f0bec0644f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.290+0000 7f0c04ce8700 1 --2- 172.21.15.38:0/4143827491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c000f5690 0x7f0c0010c1e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.290+0000 7f0c04ce8700 1 --2- 172.21.15.38:0/4143827491 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f0bec05c300 0x7f0bec05e7c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.290+0000 7f0c04ce8700 1 --2- 172.21.15.38:0/4143827491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c000f5fa0 0x7f0c0010e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.290+0000 7f0c04ce8700 1 -- 172.21.15.38:0/4143827491 >> 172.21.15.38:0/4143827491 conn(0x7f0c000f0eb0 msgr2=0x7f0c000fa420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.290+0000 7f0c04ce8700 1 -- 172.21.15.38:0/4143827491 shutdown_connections 2024-02-15T18:31:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:31:59.290+0000 7f0c04ce8700 1 -- 172.21.15.38:0/4143827491 wait complete. 2024-02-15T18:31:59.934 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:31:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:31:58.267801+0000 mgr.smithi038.khesbl (mgr.14162) 215 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:31:59.957 INFO:teuthology.orchestra.run.smithi038.stdout:158913789963 2024-02-15T18:31:59.957 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-15T18:32:00.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:31:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:31:58.267801+0000 mgr.smithi038.khesbl (mgr.14162) 215 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:00.197 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/config 2024-02-15T18:32:01.867 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:00.268112+0000 mgr.smithi038.khesbl (mgr.14162) 216 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:02.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:00.268112+0000 mgr.smithi038.khesbl (mgr.14162) 216 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:02.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.142+0000 7f2496737700 1 -- 172.21.15.38:0/2448621388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24900f67c0 msgr2=0x7f24900f6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:02.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.142+0000 7f2496737700 1 --2- 172.21.15.38:0/2448621388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24900f67c0 0x7f24900f6be0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f2480004660 tx=0x7f2480009b30).stop 2024-02-15T18:32:02.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.142+0000 7f2496737700 1 -- 172.21.15.38:0/2448621388 shutdown_connections 2024-02-15T18:32:02.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.142+0000 7f2496737700 1 --2- 172.21.15.38:0/2448621388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24900f7a60 0x7f24900f7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:02.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.142+0000 7f2496737700 1 --2- 172.21.15.38:0/2448621388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24900f67c0 0x7f24900f6be0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:02.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.142+0000 7f2496737700 1 -- 172.21.15.38:0/2448621388 >> 172.21.15.38:0/2448621388 conn(0x7f24900f1f20 msgr2=0x7f24900f4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:02.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.142+0000 7f2496737700 1 -- 172.21.15.38:0/2448621388 shutdown_connections 2024-02-15T18:32:02.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.142+0000 7f2496737700 1 -- 172.21.15.38:0/2448621388 wait complete. 2024-02-15T18:32:02.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.142+0000 7f2496737700 1 Processor -- start 2024-02-15T18:32:02.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.142+0000 7f2496737700 1 -- start start 2024-02-15T18:32:02.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.143+0000 7f2496737700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24900f67c0 0x7f24901071a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:02.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.143+0000 7f2496737700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24900f7a60 0x7f2490107690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:02.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.143+0000 7f2496737700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2490107c40 con 0x7f24900f67c0 2024-02-15T18:32:02.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.143+0000 7f2496737700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2490107d80 con 0x7f24900f7a60 2024-02-15T18:32:02.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.143+0000 7f248ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24900f67c0 0x7f24901071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:02.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.143+0000 7f248ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24900f67c0 0x7f24901071a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47386/0 (socket says 172.21.15.38:47386) 2024-02-15T18:32:02.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.143+0000 7f248ffff700 1 -- 172.21.15.38:0/2796407731 learned_addr learned my addr 172.21.15.38:0/2796407731 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:32:02.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.143+0000 7f248f7fe700 1 --2- 172.21.15.38:0/2796407731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24900f7a60 0x7f2490107690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:02.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.143+0000 7f248ffff700 1 -- 172.21.15.38:0/2796407731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24900f7a60 msgr2=0x7f2490107690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:02.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.143+0000 7f248ffff700 1 --2- 172.21.15.38:0/2796407731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24900f7a60 0x7f2490107690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:02.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.143+0000 7f248ffff700 1 -- 172.21.15.38:0/2796407731 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2480005040 con 0x7f24900f67c0 2024-02-15T18:32:02.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.143+0000 7f248ffff700 1 --2- 172.21.15.38:0/2796407731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24900f67c0 0x7f24901071a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f2480004e30 tx=0x7f2480009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:02.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.143+0000 7f248cff9700 1 -- 172.21.15.38:0/2796407731 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2480018070 con 0x7f24900f67c0 2024-02-15T18:32:02.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.143+0000 7f248cff9700 1 -- 172.21.15.38:0/2796407731 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2480007ba0 con 0x7f24900f67c0 2024-02-15T18:32:02.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.143+0000 7f2496737700 1 -- 172.21.15.38:0/2796407731 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24901086c0 con 0x7f24900f67c0 2024-02-15T18:32:02.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.144+0000 7f248cff9700 1 -- 172.21.15.38:0/2796407731 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f248001c3c0 con 0x7f24900f67c0 2024-02-15T18:32:02.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.144+0000 7f2496737700 1 -- 172.21.15.38:0/2796407731 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2490108c20 con 0x7f24900f67c0 2024-02-15T18:32:02.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.144+0000 7f2496737700 1 -- 172.21.15.38:0/2796407731 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2490045ad0 con 0x7f24900f67c0 2024-02-15T18:32:02.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.145+0000 7f248cff9700 1 -- 172.21.15.38:0/2796407731 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f2480007d10 con 0x7f24900f67c0 2024-02-15T18:32:02.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.145+0000 7f248cff9700 1 --2- 172.21.15.38:0/2796407731 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f247c05c2a0 0x7f247c05e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:02.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.145+0000 7f248cff9700 1 -- 172.21.15.38:0/2796407731 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f24800776b0 con 0x7f24900f67c0 2024-02-15T18:32:02.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.146+0000 7f248f7fe700 1 --2- 172.21.15.38:0/2796407731 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f247c05c2a0 0x7f247c05e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:02.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.146+0000 7f248f7fe700 1 --2- 172.21.15.38:0/2796407731 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f247c05c2a0 0x7f247c05e760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f2478004c40 tx=0x7f247800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:02.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.150+0000 7f248cff9700 1 -- 172.21.15.38:0/2796407731 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f248004a730 con 0x7f24900f67c0 2024-02-15T18:32:02.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.307+0000 7f2496737700 1 -- 172.21.15.38:0/2796407731 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f24900ff210 con 0x7f24900f67c0 2024-02-15T18:32:02.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.307+0000 7f248cff9700 1 -- 172.21.15.38:0/2796407731 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f248004dd50 con 0x7f24900f67c0 2024-02-15T18:32:02.301 INFO:teuthology.orchestra.run.smithi038.stdout:38654705702 2024-02-15T18:32:02.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.309+0000 7f2496737700 1 -- 172.21.15.38:0/2796407731 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f247c05c2a0 msgr2=0x7f247c05e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:02.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.309+0000 7f2496737700 1 --2- 172.21.15.38:0/2796407731 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f247c05c2a0 0x7f247c05e760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f2478004c40 tx=0x7f247800b040).stop 2024-02-15T18:32:02.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.309+0000 7f2496737700 1 -- 172.21.15.38:0/2796407731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24900f67c0 msgr2=0x7f24901071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:02.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.309+0000 7f2496737700 1 --2- 172.21.15.38:0/2796407731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24900f67c0 0x7f24901071a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f2480004e30 tx=0x7f2480009b30).stop 2024-02-15T18:32:02.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.309+0000 7f2496737700 1 -- 172.21.15.38:0/2796407731 shutdown_connections 2024-02-15T18:32:02.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.309+0000 7f2496737700 1 --2- 172.21.15.38:0/2796407731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f24900f67c0 0x7f24901071a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:02.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.309+0000 7f2496737700 1 --2- 172.21.15.38:0/2796407731 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f247c05c2a0 0x7f247c05e760 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:02.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.309+0000 7f2496737700 1 --2- 172.21.15.38:0/2796407731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24900f7a60 0x7f2490107690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:02.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.309+0000 7f2496737700 1 -- 172.21.15.38:0/2796407731 >> 172.21.15.38:0/2796407731 conn(0x7f24900f1f20 msgr2=0x7f24900fb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:02.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.310+0000 7f2496737700 1 -- 172.21.15.38:0/2796407731 shutdown_connections 2024-02-15T18:32:02.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:02.310+0000 7f2496737700 1 -- 172.21.15.38:0/2796407731 wait complete. 2024-02-15T18:32:02.947 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:02.309320+0000 mon.smithi038 (mon.0) 445 : audit [DBG] from='client.? 172.21.15.38:0/2796407731' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-15T18:32:02.968 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705699 got 38654705702 for osd.0 2024-02-15T18:32:02.969 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-15T18:32:03.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:02.309320+0000 mon.smithi038 (mon.0) 445 : audit [DBG] from='client.? 172.21.15.38:0/2796407731' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-15T18:32:03.204 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/config 2024-02-15T18:32:03.759 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:02.268398+0000 mgr.smithi038.khesbl (mgr.14162) 217 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:04.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:02.268398+0000 mgr.smithi038.khesbl (mgr.14162) 217 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:05.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.227+0000 7fe96aa32700 1 -- 172.21.15.38:0/147082583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9640f6960 msgr2=0x7fe9640f6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:05.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.227+0000 7fe96aa32700 1 --2- 172.21.15.38:0/147082583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9640f6960 0x7fe9640f6dc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fe94c004660 tx=0x7fe94c009b30).stop 2024-02-15T18:32:05.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.227+0000 7fe96aa32700 1 -- 172.21.15.38:0/147082583 shutdown_connections 2024-02-15T18:32:05.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.227+0000 7fe96aa32700 1 --2- 172.21.15.38:0/147082583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9640f6960 0x7fe9640f6dc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:05.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.227+0000 7fe96aa32700 1 --2- 172.21.15.38:0/147082583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9640f56c0 0x7fe9640f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:05.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.227+0000 7fe96aa32700 1 -- 172.21.15.38:0/147082583 >> 172.21.15.38:0/147082583 conn(0x7fe9640f0e20 msgr2=0x7fe9640f3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:05.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.227+0000 7fe96aa32700 1 -- 172.21.15.38:0/147082583 shutdown_connections 2024-02-15T18:32:05.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.227+0000 7fe96aa32700 1 -- 172.21.15.38:0/147082583 wait complete. 2024-02-15T18:32:05.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.228+0000 7fe96aa32700 1 Processor -- start 2024-02-15T18:32:05.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.228+0000 7fe96aa32700 1 -- start start 2024-02-15T18:32:05.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.228+0000 7fe96aa32700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9640f56c0 0x7fe9640f9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:05.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.228+0000 7fe96aa32700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9640f6960 0x7fe9640f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:05.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.228+0000 7fe96aa32700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9640f8200 con 0x7fe9640f6960 2024-02-15T18:32:05.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.228+0000 7fe96aa32700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9640f8340 con 0x7fe9640f56c0 2024-02-15T18:32:05.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.229+0000 7fe9637fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9640f6960 0x7fe9640f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:05.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.229+0000 7fe9637fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9640f6960 0x7fe9640f7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60794/0 (socket says 172.21.15.38:60794) 2024-02-15T18:32:05.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.229+0000 7fe9637fe700 1 -- 172.21.15.38:0/489101134 learned_addr learned my addr 172.21.15.38:0/489101134 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:32:05.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.229+0000 7fe963fff700 1 --2- 172.21.15.38:0/489101134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9640f56c0 0x7fe9640f9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:05.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.229+0000 7fe9637fe700 1 -- 172.21.15.38:0/489101134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9640f56c0 msgr2=0x7fe9640f9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:05.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.229+0000 7fe9637fe700 1 --2- 172.21.15.38:0/489101134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9640f56c0 0x7fe9640f9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:05.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.229+0000 7fe9637fe700 1 -- 172.21.15.38:0/489101134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94c005040 con 0x7fe9640f6960 2024-02-15T18:32:05.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.229+0000 7fe9637fe700 1 --2- 172.21.15.38:0/489101134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9640f6960 0x7fe9640f7ca0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fe94c009830 tx=0x7fe94c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:05.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.229+0000 7fe960ff9700 1 -- 172.21.15.38:0/489101134 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe94c018070 con 0x7fe9640f6960 2024-02-15T18:32:05.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.229+0000 7fe960ff9700 1 -- 172.21.15.38:0/489101134 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe94c007c70 con 0x7fe9640f6960 2024-02-15T18:32:05.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.229+0000 7fe96aa32700 1 -- 172.21.15.38:0/489101134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9640f85c0 con 0x7fe9640f6960 2024-02-15T18:32:05.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.229+0000 7fe960ff9700 1 -- 172.21.15.38:0/489101134 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe94c01c620 con 0x7fe9640f6960 2024-02-15T18:32:05.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.229+0000 7fe96aa32700 1 -- 172.21.15.38:0/489101134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9640f8b20 con 0x7fe9640f6960 2024-02-15T18:32:05.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.230+0000 7fe96aa32700 1 -- 172.21.15.38:0/489101134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe964045ad0 con 0x7fe9640f6960 2024-02-15T18:32:05.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.231+0000 7fe960ff9700 1 -- 172.21.15.38:0/489101134 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fe94c01c8e0 con 0x7fe9640f6960 2024-02-15T18:32:05.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.231+0000 7fe960ff9700 1 --2- 172.21.15.38:0/489101134 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fe95005c2a0 0x7fe95005e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:05.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.231+0000 7fe960ff9700 1 -- 172.21.15.38:0/489101134 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7fe94c0779d0 con 0x7fe9640f6960 2024-02-15T18:32:05.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.231+0000 7fe963fff700 1 --2- 172.21.15.38:0/489101134 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fe95005c2a0 0x7fe95005e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:05.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.232+0000 7fe963fff700 1 --2- 172.21.15.38:0/489101134 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fe95005c2a0 0x7fe95005e760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fe9640f95b0 tx=0x7fe954009040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:05.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.235+0000 7fe960ff9700 1 -- 172.21.15.38:0/489101134 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe94c04a970 con 0x7fe9640f6960 2024-02-15T18:32:05.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.395+0000 7fe96aa32700 1 -- 172.21.15.38:0/489101134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe96410f820 con 0x7fe9640f6960 2024-02-15T18:32:05.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.395+0000 7fe960ff9700 1 -- 172.21.15.38:0/489101134 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe94c04df90 con 0x7fe9640f6960 2024-02-15T18:32:05.389 INFO:teuthology.orchestra.run.smithi038.stdout:60129542175 2024-02-15T18:32:05.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.397+0000 7fe96aa32700 1 -- 172.21.15.38:0/489101134 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fe95005c2a0 msgr2=0x7fe95005e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:05.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.397+0000 7fe96aa32700 1 --2- 172.21.15.38:0/489101134 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fe95005c2a0 0x7fe95005e760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fe9640f95b0 tx=0x7fe954009040).stop 2024-02-15T18:32:05.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.397+0000 7fe96aa32700 1 -- 172.21.15.38:0/489101134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9640f6960 msgr2=0x7fe9640f7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:05.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.397+0000 7fe96aa32700 1 --2- 172.21.15.38:0/489101134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9640f6960 0x7fe9640f7ca0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fe94c009830 tx=0x7fe94c009b30).stop 2024-02-15T18:32:05.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.397+0000 7fe96aa32700 1 -- 172.21.15.38:0/489101134 shutdown_connections 2024-02-15T18:32:05.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.397+0000 7fe96aa32700 1 --2- 172.21.15.38:0/489101134 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fe95005c2a0 0x7fe95005e760 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:05.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.397+0000 7fe96aa32700 1 --2- 172.21.15.38:0/489101134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9640f56c0 0x7fe9640f9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:05.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.397+0000 7fe96aa32700 1 --2- 172.21.15.38:0/489101134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9640f6960 0x7fe9640f7ca0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:05.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.397+0000 7fe96aa32700 1 -- 172.21.15.38:0/489101134 >> 172.21.15.38:0/489101134 conn(0x7fe9640f0e20 msgr2=0x7fe9640603a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:05.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.397+0000 7fe96aa32700 1 -- 172.21.15.38:0/489101134 shutdown_connections 2024-02-15T18:32:05.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:05.397+0000 7fe96aa32700 1 -- 172.21.15.38:0/489101134 wait complete. 2024-02-15T18:32:06.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:04.268751+0000 mgr.smithi038.khesbl (mgr.14162) 218 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:06.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:05.396884+0000 mon.smithi038 (mon.0) 446 : audit [DBG] from='client.? 172.21.15.38:0/489101134' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-15T18:32:06.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:04.268751+0000 mgr.smithi038.khesbl (mgr.14162) 218 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:06.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:05.396884+0000 mon.smithi038 (mon.0) 446 : audit [DBG] from='client.? 172.21.15.38:0/489101134' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-15T18:32:06.160 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542171 got 60129542175 for osd.1 2024-02-15T18:32:06.160 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-15T18:32:06.400 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/config 2024-02-15T18:32:08.019 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:06.269066+0000 mgr.smithi038.khesbl (mgr.14162) 219 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:08.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:06.269066+0000 mgr.smithi038.khesbl (mgr.14162) 219 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:08.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.283+0000 7f3837576700 1 -- 172.21.15.38:0/3467116776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38300f5280 msgr2=0x7f38300f7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:08.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.283+0000 7f3837576700 1 --2- 172.21.15.38:0/3467116776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38300f5280 0x7f38300f7a00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f3820004660 tx=0x7f3820009b30).stop 2024-02-15T18:32:08.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.283+0000 7f3837576700 1 -- 172.21.15.38:0/3467116776 shutdown_connections 2024-02-15T18:32:08.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.283+0000 7f3837576700 1 --2- 172.21.15.38:0/3467116776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38300f5280 0x7f38300f7a00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:08.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.283+0000 7f3837576700 1 --2- 172.21.15.38:0/3467116776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38300f48e0 0x7f38300f4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:32:08.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.283+0000 7f3837576700 1 -- 172.21.15.38:0/3467116776 >> 172.21.15.38:0/3467116776 conn(0x7f38300f0400 msgr2=0x7f38300f2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:08.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.283+0000 7f3837576700 1 -- 172.21.15.38:0/3467116776 shutdown_connections 2024-02-15T18:32:08.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.283+0000 7f3837576700 1 -- 172.21.15.38:0/3467116776 wait complete. 2024-02-15T18:32:08.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.283+0000 7f3837576700 1 Processor -- start 2024-02-15T18:32:08.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.283+0000 7f3837576700 1 -- start start 2024-02-15T18:32:08.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.284+0000 7f3837576700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38300f48e0 0x7f3830107030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:08.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.284+0000 7f3837576700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38300f5280 0x7f3830107520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:08.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.284+0000 7f3837576700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3830107b60 con 0x7f38300f5280 2024-02-15T18:32:08.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.284+0000 7f3837576700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3830108410 con 0x7f38300f48e0 2024-02-15T18:32:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.284+0000 7f3835d73700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38300f5280 0x7f3830107520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.284+0000 7f3835d73700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38300f5280 0x7f3830107520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60824/0 (socket says 172.21.15.38:60824) 2024-02-15T18:32:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.284+0000 7f3835d73700 1 -- 172.21.15.38:0/1277459768 learned_addr learned my addr 172.21.15.38:0/1277459768 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:32:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.284+0000 7f3836574700 1 --2- 172.21.15.38:0/1277459768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38300f48e0 0x7f3830107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.284+0000 7f3835d73700 1 -- 172.21.15.38:0/1277459768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38300f48e0 msgr2=0x7f3830107030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.284+0000 7f3835d73700 1 --2- 172.21.15.38:0/1277459768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38300f48e0 0x7f3830107030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.284+0000 7f3835d73700 1 -- 172.21.15.38:0/1277459768 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3820005040 con 0x7f38300f5280 2024-02-15T18:32:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.284+0000 7f3835d73700 1 --2- 172.21.15.38:0/1277459768 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38300f5280 0x7f3830107520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f3820009f50 tx=0x7f3820009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.284+0000 7f3826ffd700 1 -- 172.21.15.38:0/1277459768 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3820018070 con 0x7f38300f5280 2024-02-15T18:32:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.284+0000 7f3826ffd700 1 -- 172.21.15.38:0/1277459768 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3820007d70 con 0x7f38300f5280 2024-02-15T18:32:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.284+0000 7f3837576700 1 -- 172.21.15.38:0/1277459768 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3830108690 con 0x7f38300f5280 2024-02-15T18:32:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.284+0000 7f3826ffd700 1 -- 172.21.15.38:0/1277459768 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f382001c620 con 0x7f38300f5280 2024-02-15T18:32:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.284+0000 7f3837576700 1 -- 172.21.15.38:0/1277459768 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3830108bf0 con 0x7f38300f5280 2024-02-15T18:32:08.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.285+0000 7f3837576700 1 -- 172.21.15.38:0/1277459768 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3830059100 con 0x7f38300f5280 2024-02-15T18:32:08.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.286+0000 7f3826ffd700 1 -- 172.21.15.38:0/1277459768 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f382001d420 con 0x7f38300f5280 2024-02-15T18:32:08.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.286+0000 7f3826ffd700 1 --2- 172.21.15.38:0/1277459768 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f381c05c2b0 0x7f381c05e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:08.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.286+0000 7f3826ffd700 1 -- 172.21.15.38:0/1277459768 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f3820077970 con 0x7f38300f5280 2024-02-15T18:32:08.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.286+0000 7f3836574700 1 --2- 172.21.15.38:0/1277459768 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f381c05c2b0 0x7f381c05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:08.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.286+0000 7f3836574700 1 --2- 172.21.15.38:0/1277459768 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f381c05c2b0 0x7f381c05e770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f383005f8e0 tx=0x7f382c008040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:08.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.290+0000 7f3826ffd700 1 -- 172.21.15.38:0/1277459768 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f382004a9f0 con 0x7f38300f5280 2024-02-15T18:32:08.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.448+0000 7f3837576700 1 -- 172.21.15.38:0/1277459768 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f38300fefb0 con 0x7f38300f5280 2024-02-15T18:32:08.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.448+0000 7f3826ffd700 1 -- 172.21.15.38:0/1277459768 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f382004e010 con 0x7f38300f5280 2024-02-15T18:32:08.442 INFO:teuthology.orchestra.run.smithi038.stdout:81604378651 2024-02-15T18:32:08.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.450+0000 7f3837576700 1 -- 172.21.15.38:0/1277459768 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f381c05c2b0 msgr2=0x7f381c05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:08.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.450+0000 7f3837576700 1 --2- 172.21.15.38:0/1277459768 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f381c05c2b0 0x7f381c05e770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f383005f8e0 tx=0x7f382c008040).stop 2024-02-15T18:32:08.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.450+0000 7f3837576700 1 -- 172.21.15.38:0/1277459768 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38300f5280 msgr2=0x7f3830107520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:08.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.450+0000 7f3837576700 1 --2- 172.21.15.38:0/1277459768 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38300f5280 0x7f3830107520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f3820009f50 tx=0x7f3820009b30).stop 2024-02-15T18:32:08.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.450+0000 7f3837576700 1 -- 172.21.15.38:0/1277459768 shutdown_connections 2024-02-15T18:32:08.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.450+0000 7f3837576700 1 --2- 172.21.15.38:0/1277459768 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f381c05c2b0 0x7f381c05e770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:08.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.450+0000 7f3837576700 1 --2- 172.21.15.38:0/1277459768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38300f48e0 0x7f3830107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:08.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.450+0000 7f3837576700 1 --2- 172.21.15.38:0/1277459768 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38300f5280 0x7f3830107520 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:08.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.450+0000 7f3837576700 1 -- 172.21.15.38:0/1277459768 >> 172.21.15.38:0/1277459768 conn(0x7f38300f0400 msgr2=0x7f38300fb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:08.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.450+0000 7f3837576700 1 -- 172.21.15.38:0/1277459768 shutdown_connections 2024-02-15T18:32:08.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:08.450+0000 7f3837576700 1 -- 172.21.15.38:0/1277459768 wait complete. 2024-02-15T18:32:09.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:08.449950+0000 mon.smithi038 (mon.0) 447 : audit [DBG] from='client.? 172.21.15.38:0/1277459768' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-15T18:32:09.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:08.449950+0000 mon.smithi038 (mon.0) 447 : audit [DBG] from='client.? 172.21.15.38:0/1277459768' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-15T18:32:09.164 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378648 got 81604378651 for osd.2 2024-02-15T18:32:09.164 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-15T18:32:09.403 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/config 2024-02-15T18:32:09.940 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:08.269368+0000 mgr.smithi038.khesbl (mgr.14162) 220 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:10.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:08.269368+0000 mgr.smithi038.khesbl (mgr.14162) 220 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:11.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.255+0000 7fc612a5c700 1 -- 172.21.15.38:0/1677803673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc60c0f7a80 msgr2=0x7fc60c0f7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:11.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.255+0000 7fc612a5c700 1 --2- 172.21.15.38:0/1677803673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc60c0f7a80 0x7fc60c0f7ee0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fc5f4004660 tx=0x7fc5f4009b30).stop 2024-02-15T18:32:11.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.255+0000 7fc612a5c700 1 -- 172.21.15.38:0/1677803673 shutdown_connections 2024-02-15T18:32:11.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.255+0000 7fc612a5c700 1 --2- 172.21.15.38:0/1677803673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc60c0f7a80 0x7fc60c0f7ee0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:11.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.255+0000 7fc612a5c700 1 --2- 172.21.15.38:0/1677803673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc60c0f67e0 0x7fc60c0f6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:11.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.255+0000 7fc612a5c700 1 -- 172.21.15.38:0/1677803673 >> 172.21.15.38:0/1677803673 conn(0x7fc60c0f1f40 msgr2=0x7fc60c0f4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:11.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.255+0000 7fc612a5c700 1 -- 172.21.15.38:0/1677803673 shutdown_connections 2024-02-15T18:32:11.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.255+0000 7fc612a5c700 1 -- 172.21.15.38:0/1677803673 wait complete. 2024-02-15T18:32:11.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.256+0000 7fc612a5c700 1 Processor -- start 2024-02-15T18:32:11.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.256+0000 7fc612a5c700 1 -- start start 2024-02-15T18:32:11.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.256+0000 7fc612a5c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc60c0f67e0 0x7fc60c107190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:11.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.256+0000 7fc612a5c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc60c0f7a80 0x7fc60c107680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:11.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.256+0000 7fc612a5c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c107c30 con 0x7fc60c0f7a80 2024-02-15T18:32:11.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.256+0000 7fc612a5c700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c107d70 con 0x7fc60c0f67e0 2024-02-15T18:32:11.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.256+0000 7fc60b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc60c0f7a80 0x7fc60c107680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:11.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.256+0000 7fc60b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc60c0f7a80 0x7fc60c107680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60840/0 (socket says 172.21.15.38:60840) 2024-02-15T18:32:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.256+0000 7fc60b7fe700 1 -- 172.21.15.38:0/539948904 learned_addr learned my addr 172.21.15.38:0/539948904 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:32:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.256+0000 7fc60bfff700 1 --2- 172.21.15.38:0/539948904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc60c0f67e0 0x7fc60c107190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.256+0000 7fc60b7fe700 1 -- 172.21.15.38:0/539948904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc60c0f67e0 msgr2=0x7fc60c107190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.256+0000 7fc60b7fe700 1 --2- 172.21.15.38:0/539948904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc60c0f67e0 0x7fc60c107190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.256+0000 7fc60b7fe700 1 -- 172.21.15.38:0/539948904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f4005040 con 0x7fc60c0f7a80 2024-02-15T18:32:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.257+0000 7fc60b7fe700 1 --2- 172.21.15.38:0/539948904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc60c0f7a80 0x7fc60c107680 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fc5f4009830 tx=0x7fc5f4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.257+0000 7fc608ff9700 1 -- 172.21.15.38:0/539948904 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5f4018070 con 0x7fc60c0f7a80 2024-02-15T18:32:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.257+0000 7fc608ff9700 1 -- 172.21.15.38:0/539948904 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5f4007c70 con 0x7fc60c0f7a80 2024-02-15T18:32:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.257+0000 7fc612a5c700 1 -- 172.21.15.38:0/539948904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60c1086b0 con 0x7fc60c0f7a80 2024-02-15T18:32:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.257+0000 7fc608ff9700 1 -- 172.21.15.38:0/539948904 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5f401c620 con 0x7fc60c0f7a80 2024-02-15T18:32:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.257+0000 7fc612a5c700 1 -- 172.21.15.38:0/539948904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60c108c10 con 0x7fc60c0f7a80 2024-02-15T18:32:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.258+0000 7fc612a5c700 1 -- 172.21.15.38:0/539948904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc60c045ad0 con 0x7fc60c0f7a80 2024-02-15T18:32:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.258+0000 7fc608ff9700 1 -- 172.21.15.38:0/539948904 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fc5f401c8e0 con 0x7fc60c0f7a80 2024-02-15T18:32:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.258+0000 7fc608ff9700 1 --2- 172.21.15.38:0/539948904 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fc5f805c2a0 0x7fc5f805e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.259+0000 7fc608ff9700 1 -- 172.21.15.38:0/539948904 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7fc5f4077a90 con 0x7fc60c0f7a80 2024-02-15T18:32:11.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.259+0000 7fc60bfff700 1 --2- 172.21.15.38:0/539948904 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fc5f805c2a0 0x7fc5f805e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:11.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.259+0000 7fc60bfff700 1 --2- 172.21.15.38:0/539948904 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fc5f805c2a0 0x7fc5f805e760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fc60c0f78e0 tx=0x7fc5fc009040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:11.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.263+0000 7fc608ff9700 1 -- 172.21.15.38:0/539948904 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc5f404a970 con 0x7fc60c0f7a80 2024-02-15T18:32:11.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.424+0000 7fc612a5c700 1 -- 172.21.15.38:0/539948904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc60c027a00 con 0x7fc60c0f7a80 2024-02-15T18:32:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.425+0000 7fc608ff9700 1 -- 172.21.15.38:0/539948904 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc5f404df90 con 0x7fc60c0f7a80 2024-02-15T18:32:11.418 INFO:teuthology.orchestra.run.smithi038.stdout:111669149718 2024-02-15T18:32:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.426+0000 7fc612a5c700 1 -- 172.21.15.38:0/539948904 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fc5f805c2a0 msgr2=0x7fc5f805e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.426+0000 7fc612a5c700 1 --2- 172.21.15.38:0/539948904 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fc5f805c2a0 0x7fc5f805e760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fc60c0f78e0 tx=0x7fc5fc009040).stop 2024-02-15T18:32:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.426+0000 7fc612a5c700 1 -- 172.21.15.38:0/539948904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc60c0f7a80 msgr2=0x7fc60c107680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.426+0000 7fc612a5c700 1 --2- 172.21.15.38:0/539948904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc60c0f7a80 0x7fc60c107680 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fc5f4009830 tx=0x7fc5f4009b30).stop 2024-02-15T18:32:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.426+0000 7fc612a5c700 1 -- 172.21.15.38:0/539948904 shutdown_connections 2024-02-15T18:32:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.426+0000 7fc612a5c700 1 --2- 172.21.15.38:0/539948904 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fc5f805c2a0 0x7fc5f805e760 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.426+0000 7fc612a5c700 1 --2- 172.21.15.38:0/539948904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc60c0f67e0 0x7fc60c107190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.426+0000 7fc612a5c700 1 --2- 172.21.15.38:0/539948904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc60c0f7a80 0x7fc60c107680 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.427+0000 7fc612a5c700 1 -- 172.21.15.38:0/539948904 >> 172.21.15.38:0/539948904 conn(0x7fc60c0f1f40 msgr2=0x7fc60c0fb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.427+0000 7fc612a5c700 1 -- 172.21.15.38:0/539948904 shutdown_connections 2024-02-15T18:32:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:11.427+0000 7fc612a5c700 1 -- 172.21.15.38:0/539948904 wait complete. 2024-02-15T18:32:12.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:10.269743+0000 mgr.smithi038.khesbl (mgr.14162) 221 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:12.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:11.426423+0000 mon.smithi038 (mon.0) 448 : audit [DBG] from='client.? 172.21.15.38:0/539948904' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-15T18:32:12.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:10.269743+0000 mgr.smithi038.khesbl (mgr.14162) 221 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:12.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:11.426423+0000 mon.smithi038 (mon.0) 448 : audit [DBG] from='client.? 172.21.15.38:0/539948904' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-15T18:32:12.214 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149715 got 111669149718 for osd.3 2024-02-15T18:32:12.215 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-15T18:32:12.454 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/config 2024-02-15T18:32:13.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:12.270046+0000 mgr.smithi038.khesbl (mgr.14162) 222 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:14.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:12.270046+0000 mgr.smithi038.khesbl (mgr.14162) 222 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:14.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.346+0000 7fdb9330b700 1 -- 172.21.15.38:0/2944979915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb8c0f67c0 msgr2=0x7fdb8c0f6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:14.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.346+0000 7fdb9330b700 1 --2- 172.21.15.38:0/2944979915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb8c0f67c0 0x7fdb8c0f6be0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fdb7c004660 tx=0x7fdb7c009b30).stop 2024-02-15T18:32:14.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.346+0000 7fdb9330b700 1 -- 172.21.15.38:0/2944979915 shutdown_connections 2024-02-15T18:32:14.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.346+0000 7fdb9330b700 1 --2- 172.21.15.38:0/2944979915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c0f7a60 0x7fdb8c0f7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:14.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.346+0000 7fdb9330b700 1 --2- 172.21.15.38:0/2944979915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb8c0f67c0 0x7fdb8c0f6be0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:14.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.346+0000 7fdb9330b700 1 -- 172.21.15.38:0/2944979915 >> 172.21.15.38:0/2944979915 conn(0x7fdb8c0f1f20 msgr2=0x7fdb8c0f4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:14.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.346+0000 7fdb9330b700 1 -- 172.21.15.38:0/2944979915 shutdown_connections 2024-02-15T18:32:14.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.346+0000 7fdb9330b700 1 -- 172.21.15.38:0/2944979915 wait complete. 2024-02-15T18:32:14.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.346+0000 7fdb9330b700 1 Processor -- start 2024-02-15T18:32:14.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.347+0000 7fdb9330b700 1 -- start start 2024-02-15T18:32:14.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.347+0000 7fdb9330b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb8c0f67c0 0x7fdb8c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:14.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.347+0000 7fdb9330b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c0f7a60 0x7fdb8c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:14.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.347+0000 7fdb9330b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c06a380 con 0x7fdb8c0f67c0 2024-02-15T18:32:14.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.347+0000 7fdb9330b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c06a4c0 con 0x7fdb8c0f7a60 2024-02-15T18:32:14.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.347+0000 7fdb910a7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb8c0f67c0 0x7fdb8c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:14.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.347+0000 7fdb910a7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb8c0f67c0 0x7fdb8c06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54824/0 (socket says 172.21.15.38:54824) 2024-02-15T18:32:14.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.347+0000 7fdb910a7700 1 -- 172.21.15.38:0/756497306 learned_addr learned my addr 172.21.15.38:0/756497306 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:32:14.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.347+0000 7fdb910a7700 1 -- 172.21.15.38:0/756497306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c0f7a60 msgr2=0x7fdb8c069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:14.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.347+0000 7fdb910a7700 1 --2- 172.21.15.38:0/756497306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c0f7a60 0x7fdb8c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:32:14.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.347+0000 7fdb910a7700 1 -- 172.21.15.38:0/756497306 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c005040 con 0x7fdb8c0f67c0 2024-02-15T18:32:14.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.347+0000 7fdb910a7700 1 --2- 172.21.15.38:0/756497306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb8c0f67c0 0x7fdb8c06b9a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fdb7c004e30 tx=0x7fdb7c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:14.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.347+0000 7fdb81ffb700 1 -- 172.21.15.38:0/756497306 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb7c018070 con 0x7fdb8c0f67c0 2024-02-15T18:32:14.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.347+0000 7fdb81ffb700 1 -- 172.21.15.38:0/756497306 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb7c007ba0 con 0x7fdb8c0f67c0 2024-02-15T18:32:14.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.347+0000 7fdb9330b700 1 -- 172.21.15.38:0/756497306 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8c06a740 con 0x7fdb8c0f67c0 2024-02-15T18:32:14.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.347+0000 7fdb81ffb700 1 -- 172.21.15.38:0/756497306 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb7c01c620 con 0x7fdb8c0f67c0 2024-02-15T18:32:14.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.348+0000 7fdb9330b700 1 -- 172.21.15.38:0/756497306 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8c06aca0 con 0x7fdb8c0f67c0 2024-02-15T18:32:14.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.348+0000 7fdb9330b700 1 -- 172.21.15.38:0/756497306 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb8c045ad0 con 0x7fdb8c0f67c0 2024-02-15T18:32:14.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.349+0000 7fdb81ffb700 1 -- 172.21.15.38:0/756497306 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fdb7c007d10 con 0x7fdb8c0f67c0 2024-02-15T18:32:14.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.349+0000 7fdb81ffb700 1 --2- 172.21.15.38:0/756497306 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fdb7805c2a0 0x7fdb7805e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:14.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.349+0000 7fdb81ffb700 1 -- 172.21.15.38:0/756497306 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7fdb7c0777b0 con 0x7fdb8c0f67c0 2024-02-15T18:32:14.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.349+0000 7fdb908a6700 1 --2- 172.21.15.38:0/756497306 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fdb7805c2a0 0x7fdb7805e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:14.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.350+0000 7fdb908a6700 1 --2- 172.21.15.38:0/756497306 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fdb7805c2a0 0x7fdb7805e760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fdb88004bd0 tx=0x7fdb88009b10).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:14.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.353+0000 7fdb81ffb700 1 -- 172.21.15.38:0/756497306 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fdb7c04a830 con 0x7fdb8c0f67c0 2024-02-15T18:32:14.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.512+0000 7fdb9330b700 1 -- 172.21.15.38:0/756497306 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fdb8c027a00 con 0x7fdb8c0f67c0 2024-02-15T18:32:14.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.513+0000 7fdb81ffb700 1 -- 172.21.15.38:0/756497306 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdb7c04de50 con 0x7fdb8c0f67c0 2024-02-15T18:32:14.506 INFO:teuthology.orchestra.run.smithi038.stdout:137438953489 2024-02-15T18:32:14.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.514+0000 7fdb9330b700 1 -- 172.21.15.38:0/756497306 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fdb7805c2a0 msgr2=0x7fdb7805e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:14.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.514+0000 7fdb9330b700 1 --2- 172.21.15.38:0/756497306 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fdb7805c2a0 0x7fdb7805e760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fdb88004bd0 tx=0x7fdb88009b10).stop 2024-02-15T18:32:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.514+0000 7fdb9330b700 1 -- 172.21.15.38:0/756497306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb8c0f67c0 msgr2=0x7fdb8c06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.514+0000 7fdb9330b700 1 --2- 172.21.15.38:0/756497306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb8c0f67c0 0x7fdb8c06b9a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fdb7c004e30 tx=0x7fdb7c009b30).stop 2024-02-15T18:32:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.514+0000 7fdb9330b700 1 -- 172.21.15.38:0/756497306 shutdown_connections 2024-02-15T18:32:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.514+0000 7fdb9330b700 1 --2- 172.21.15.38:0/756497306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb8c0f67c0 0x7fdb8c06b9a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.514+0000 7fdb9330b700 1 --2- 172.21.15.38:0/756497306 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fdb7805c2a0 0x7fdb7805e760 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.514+0000 7fdb9330b700 1 --2- 172.21.15.38:0/756497306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb8c0f7a60 0x7fdb8c069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:32:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.514+0000 7fdb9330b700 1 -- 172.21.15.38:0/756497306 >> 172.21.15.38:0/756497306 conn(0x7fdb8c0f1f20 msgr2=0x7fdb8c0fb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.514+0000 7fdb9330b700 1 -- 172.21.15.38:0/756497306 shutdown_connections 2024-02-15T18:32:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:14.514+0000 7fdb9330b700 1 -- 172.21.15.38:0/756497306 wait complete. 2024-02-15T18:32:15.246 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953486 got 137438953489 for osd.4 2024-02-15T18:32:15.246 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-15T18:32:15.486 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/config 2024-02-15T18:32:15.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:14.270357+0000 mgr.smithi038.khesbl (mgr.14162) 223 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:15.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:14.514578+0000 mon.smithi038 (mon.0) 449 : audit [DBG] from='client.? 172.21.15.38:0/756497306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-15T18:32:15.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:14.270357+0000 mgr.smithi038.khesbl (mgr.14162) 223 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:15.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:14.514578+0000 mon.smithi038 (mon.0) 449 : audit [DBG] from='client.? 172.21.15.38:0/756497306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-15T18:32:17.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.519+0000 7f2c5c2a6700 1 -- 172.21.15.38:0/3520071629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c540f67d0 msgr2=0x7f2c540f6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:17.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.519+0000 7f2c5c2a6700 1 --2- 172.21.15.38:0/3520071629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c540f67d0 0x7f2c540f6bf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f2c44004660 tx=0x7f2c44009b30).stop 2024-02-15T18:32:17.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.520+0000 7f2c5c2a6700 1 -- 172.21.15.38:0/3520071629 shutdown_connections 2024-02-15T18:32:17.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.520+0000 7f2c5c2a6700 1 --2- 172.21.15.38:0/3520071629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c540f7a70 0x7f2c540f7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:17.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.520+0000 7f2c5c2a6700 1 --2- 172.21.15.38:0/3520071629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c540f67d0 0x7f2c540f6bf0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:17.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.520+0000 7f2c5c2a6700 1 -- 172.21.15.38:0/3520071629 >> 172.21.15.38:0/3520071629 conn(0x7f2c540f1f50 msgr2=0x7f2c540f4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:17.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.520+0000 7f2c5c2a6700 1 -- 172.21.15.38:0/3520071629 shutdown_connections 2024-02-15T18:32:17.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.520+0000 7f2c5c2a6700 1 -- 172.21.15.38:0/3520071629 wait complete. 2024-02-15T18:32:17.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.520+0000 7f2c5c2a6700 1 Processor -- start 2024-02-15T18:32:17.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.520+0000 7f2c5c2a6700 1 -- start start 2024-02-15T18:32:17.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.520+0000 7f2c5c2a6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c540f67d0 0x7f2c5406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:17.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.520+0000 7f2c5c2a6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c540f7a70 0x7f2c54069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:17.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.520+0000 7f2c5c2a6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5406a380 con 0x7f2c540f67d0 2024-02-15T18:32:17.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.520+0000 7f2c5c2a6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5406a4c0 con 0x7f2c540f7a70 2024-02-15T18:32:17.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.520+0000 7f2c5a042700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c540f67d0 0x7f2c5406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:17.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.520+0000 7f2c5a042700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c540f67d0 0x7f2c5406b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54852/0 (socket says 172.21.15.38:54852) 2024-02-15T18:32:17.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.520+0000 7f2c5a042700 1 -- 172.21.15.38:0/2915679738 learned_addr learned my addr 172.21.15.38:0/2915679738 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:32:17.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.520+0000 7f2c59841700 1 --2- 172.21.15.38:0/2915679738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c540f7a70 0x7f2c54069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:17.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.521+0000 7f2c5a042700 1 -- 172.21.15.38:0/2915679738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c540f7a70 msgr2=0x7f2c54069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:17.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.521+0000 7f2c5a042700 1 --2- 172.21.15.38:0/2915679738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c540f7a70 0x7f2c54069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:17.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.521+0000 7f2c5a042700 1 -- 172.21.15.38:0/2915679738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c44005040 con 0x7f2c540f67d0 2024-02-15T18:32:17.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.521+0000 7f2c5a042700 1 --2- 172.21.15.38:0/2915679738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c540f67d0 0x7f2c5406b9a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f2c44009f50 tx=0x7f2c44009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:17.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.521+0000 7f2c4affd700 1 -- 172.21.15.38:0/2915679738 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c44018070 con 0x7f2c540f67d0 2024-02-15T18:32:17.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.521+0000 7f2c4affd700 1 -- 172.21.15.38:0/2915679738 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c44007c70 con 0x7f2c540f67d0 2024-02-15T18:32:17.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.521+0000 7f2c5c2a6700 1 -- 172.21.15.38:0/2915679738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5406a740 con 0x7f2c540f67d0 2024-02-15T18:32:17.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.521+0000 7f2c4affd700 1 -- 172.21.15.38:0/2915679738 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c4401c620 con 0x7f2c540f67d0 2024-02-15T18:32:17.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.521+0000 7f2c5c2a6700 1 -- 172.21.15.38:0/2915679738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5406aca0 con 0x7f2c540f67d0 2024-02-15T18:32:17.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.522+0000 7f2c5c2a6700 1 -- 172.21.15.38:0/2915679738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c54045ad0 con 0x7f2c540f67d0 2024-02-15T18:32:17.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.522+0000 7f2c4affd700 1 -- 172.21.15.38:0/2915679738 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f2c4401d420 con 0x7f2c540f67d0 2024-02-15T18:32:17.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.523+0000 7f2c4affd700 1 --2- 172.21.15.38:0/2915679738 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f2c4005c2a0 0x7f2c4005e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:17.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.523+0000 7f2c4affd700 1 -- 172.21.15.38:0/2915679738 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f2c44077870 con 0x7f2c540f67d0 2024-02-15T18:32:17.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.523+0000 7f2c59841700 1 --2- 172.21.15.38:0/2915679738 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f2c4005c2a0 0x7f2c4005e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:17.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.523+0000 7f2c59841700 1 --2- 172.21.15.38:0/2915679738 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f2c4005c2a0 0x7f2c4005e760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f2c500041d0 tx=0x7f2c50009040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:17.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.527+0000 7f2c4affd700 1 -- 172.21.15.38:0/2915679738 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f2c4404a8f0 con 0x7f2c540f67d0 2024-02-15T18:32:17.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:16.270783+0000 mgr.smithi038.khesbl (mgr.14162) 224 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:17.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:16.270783+0000 mgr.smithi038.khesbl (mgr.14162) 224 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:17.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.687+0000 7f2c5c2a6700 1 -- 172.21.15.38:0/2915679738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f2c54027a00 con 0x7f2c540f67d0 2024-02-15T18:32:17.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.687+0000 7f2c4affd700 1 -- 172.21.15.38:0/2915679738 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2c4404df10 con 0x7f2c540f67d0 2024-02-15T18:32:17.681 INFO:teuthology.orchestra.run.smithi038.stdout:158913789967 2024-02-15T18:32:17.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.689+0000 7f2c5c2a6700 1 -- 172.21.15.38:0/2915679738 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f2c4005c2a0 msgr2=0x7f2c4005e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:17.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.689+0000 7f2c5c2a6700 1 --2- 172.21.15.38:0/2915679738 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f2c4005c2a0 0x7f2c4005e760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f2c500041d0 tx=0x7f2c50009040).stop 2024-02-15T18:32:17.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.689+0000 7f2c5c2a6700 1 -- 172.21.15.38:0/2915679738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c540f67d0 msgr2=0x7f2c5406b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:17.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.689+0000 7f2c5c2a6700 1 --2- 172.21.15.38:0/2915679738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c540f67d0 0x7f2c5406b9a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f2c44009f50 tx=0x7f2c44009b30).stop 2024-02-15T18:32:17.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.689+0000 7f2c5c2a6700 1 -- 172.21.15.38:0/2915679738 shutdown_connections 2024-02-15T18:32:17.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.689+0000 7f2c5c2a6700 1 --2- 172.21.15.38:0/2915679738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c540f67d0 0x7f2c5406b9a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:17.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.689+0000 7f2c5c2a6700 1 --2- 172.21.15.38:0/2915679738 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f2c4005c2a0 0x7f2c4005e760 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:17.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.689+0000 7f2c5c2a6700 1 --2- 172.21.15.38:0/2915679738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c540f7a70 0x7f2c54069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:17.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.689+0000 7f2c5c2a6700 1 -- 172.21.15.38:0/2915679738 >> 172.21.15.38:0/2915679738 conn(0x7f2c540f1f50 msgr2=0x7f2c540fb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:17.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.689+0000 7f2c5c2a6700 1 -- 172.21.15.38:0/2915679738 shutdown_connections 2024-02-15T18:32:17.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:17.689+0000 7f2c5c2a6700 1 -- 172.21.15.38:0/2915679738 wait complete. 2024-02-15T18:32:18.315 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789963 got 158913789967 for osd.5 2024-02-15T18:32:18.315 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-15T18:32:18.316 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-15T18:32:18.553 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/config 2024-02-15T18:32:18.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:17.689256+0000 mon.smithi038 (mon.0) 450 : audit [DBG] from='client.? 172.21.15.38:0/2915679738' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-15T18:32:18.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:17.689256+0000 mon.smithi038 (mon.0) 450 : audit [DBG] from='client.? 172.21.15.38:0/2915679738' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-15T18:32:19.594 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:18.271139+0000 mgr.smithi038.khesbl (mgr.14162) 225 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:19.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:18.271139+0000 mgr.smithi038.khesbl (mgr.14162) 225 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:20.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.444+0000 7f02e332a700 1 -- 172.21.15.38:0/2904391083 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02dc0f9770 msgr2=0x7f02dc0fbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:20.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.444+0000 7f02e332a700 1 --2- 172.21.15.38:0/2904391083 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02dc0f9770 0x7f02dc0fbbc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f02cc004660 tx=0x7f02cc009b30).stop 2024-02-15T18:32:20.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.445+0000 7f02e332a700 1 -- 172.21.15.38:0/2904391083 shutdown_connections 2024-02-15T18:32:20.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.445+0000 7f02e332a700 1 --2- 172.21.15.38:0/2904391083 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02dc0f9770 0x7f02dc0fbbc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:20.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.445+0000 7f02e332a700 1 --2- 172.21.15.38:0/2904391083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02dc060870 0x7f02dc0f91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:20.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.445+0000 7f02e332a700 1 -- 172.21.15.38:0/2904391083 >> 172.21.15.38:0/2904391083 conn(0x7f02dc0f0c70 msgr2=0x7f02dc0f3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:20.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.445+0000 7f02e332a700 1 -- 172.21.15.38:0/2904391083 shutdown_connections 2024-02-15T18:32:20.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.445+0000 7f02e332a700 1 -- 172.21.15.38:0/2904391083 wait complete. 2024-02-15T18:32:20.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.445+0000 7f02e332a700 1 Processor -- start 2024-02-15T18:32:20.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.445+0000 7f02e332a700 1 -- start start 2024-02-15T18:32:20.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.445+0000 7f02e332a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02dc060870 0x7f02dc106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:20.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.445+0000 7f02e332a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02dc0f9770 0x7f02dc107490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:20.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.445+0000 7f02e332a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc107a40 con 0x7f02dc060870 2024-02-15T18:32:20.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.445+0000 7f02e332a700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc107b80 con 0x7f02dc0f9770 2024-02-15T18:32:20.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.446+0000 7f02e2328700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02dc060870 0x7f02dc106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:20.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.446+0000 7f02e2328700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02dc060870 0x7f02dc106fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54874/0 (socket says 172.21.15.38:54874) 2024-02-15T18:32:20.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.446+0000 7f02e2328700 1 -- 172.21.15.38:0/1423652220 learned_addr learned my addr 172.21.15.38:0/1423652220 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:32:20.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.446+0000 7f02e1b27700 1 --2- 172.21.15.38:0/1423652220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02dc0f9770 0x7f02dc107490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:20.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.446+0000 7f02e2328700 1 -- 172.21.15.38:0/1423652220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02dc0f9770 msgr2=0x7f02dc107490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:20.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.446+0000 7f02e2328700 1 --2- 172.21.15.38:0/1423652220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02dc0f9770 0x7f02dc107490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:20.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.446+0000 7f02e2328700 1 -- 172.21.15.38:0/1423652220 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02cc005040 con 0x7f02dc060870 2024-02-15T18:32:20.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.446+0000 7f02e2328700 1 --2- 172.21.15.38:0/1423652220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02dc060870 0x7f02dc106fa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f02d800c790 tx=0x7f02d8007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:20.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.446+0000 7f02d2ffd700 1 -- 172.21.15.38:0/1423652220 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02d800e680 con 0x7f02dc060870 2024-02-15T18:32:20.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.446+0000 7f02d2ffd700 1 -- 172.21.15.38:0/1423652220 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02d800ecb0 con 0x7f02dc060870 2024-02-15T18:32:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.446+0000 7f02e332a700 1 -- 172.21.15.38:0/1423652220 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02dc108520 con 0x7f02dc060870 2024-02-15T18:32:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.446+0000 7f02d2ffd700 1 -- 172.21.15.38:0/1423652220 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02d8013e10 con 0x7f02dc060870 2024-02-15T18:32:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.446+0000 7f02e332a700 1 -- 172.21.15.38:0/1423652220 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02dc108ae0 con 0x7f02dc060870 2024-02-15T18:32:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.447+0000 7f02e332a700 1 -- 172.21.15.38:0/1423652220 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02dc045ad0 con 0x7f02dc060870 2024-02-15T18:32:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.447+0000 7f02d2ffd700 1 -- 172.21.15.38:0/1423652220 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f02d801a060 con 0x7f02dc060870 2024-02-15T18:32:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.448+0000 7f02d2ffd700 1 --2- 172.21.15.38:0/1423652220 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f02c805c2b0 0x7f02c805e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.448+0000 7f02d2ffd700 1 -- 172.21.15.38:0/1423652220 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f02d8075e10 con 0x7f02dc060870 2024-02-15T18:32:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.448+0000 7f02e1b27700 1 --2- 172.21.15.38:0/1423652220 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f02c805c2b0 0x7f02c805e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.448+0000 7f02e1b27700 1 --2- 172.21.15.38:0/1423652220 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f02c805c2b0 0x7f02c805e770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f02cc004e30 tx=0x7f02cc009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.452+0000 7f02d2ffd700 1 -- 172.21.15.38:0/1423652220 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f02d8048e10 con 0x7f02dc060870 2024-02-15T18:32:20.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.605+0000 7f02e332a700 1 -- 172.21.15.38:0/1423652220 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f02dc0fefc0 con 0x7f02c805c2b0 2024-02-15T18:32:20.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.607+0000 7f02d2ffd700 1 -- 172.21.15.38:0/1423652220 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+29068 (secure 0 0 0) 0x7f02dc0fefc0 con 0x7f02c805c2b0 2024-02-15T18:32:20.600 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:32:20.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.608+0000 7f02e332a700 1 -- 172.21.15.38:0/1423652220 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f02c805c2b0 msgr2=0x7f02c805e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:20.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.608+0000 7f02e332a700 1 --2- 172.21.15.38:0/1423652220 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f02c805c2b0 0x7f02c805e770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f02cc004e30 tx=0x7f02cc009b30).stop 2024-02-15T18:32:20.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.608+0000 7f02e332a700 1 -- 172.21.15.38:0/1423652220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02dc060870 msgr2=0x7f02dc106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:20.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.608+0000 7f02e332a700 1 --2- 172.21.15.38:0/1423652220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02dc060870 0x7f02dc106fa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f02d800c790 tx=0x7f02d8007d70).stop 2024-02-15T18:32:20.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.608+0000 7f02e332a700 1 -- 172.21.15.38:0/1423652220 shutdown_connections 2024-02-15T18:32:20.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.608+0000 7f02e332a700 1 --2- 172.21.15.38:0/1423652220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02dc060870 0x7f02dc106fa0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:20.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.608+0000 7f02e332a700 1 --2- 172.21.15.38:0/1423652220 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f02c805c2b0 0x7f02c805e770 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:20.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.608+0000 7f02e332a700 1 --2- 172.21.15.38:0/1423652220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02dc0f9770 0x7f02dc107490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:20.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.608+0000 7f02e332a700 1 -- 172.21.15.38:0/1423652220 >> 172.21.15.38:0/1423652220 conn(0x7f02dc0f0c70 msgr2=0x7f02dc0f3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:20.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.609+0000 7f02e332a700 1 -- 172.21.15.38:0/1423652220 shutdown_connections 2024-02-15T18:32:20.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:20.609+0000 7f02e332a700 1 -- 172.21.15.38:0/1423652220 wait complete. 2024-02-15T18:32:20.603 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-02-15T18:32:21.338 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-02-15T18:32:20.271372+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":31224,"kb_used_data":1992,"kb_used_omap":0,"kb_used_meta":29184,"kb_avail":562439688,"statfs":{"total":575970213888,"available":575938240512,"internally_reserved":0,"allocated":2039808,"data_stored":768834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29884416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":"36","reported_epoch":"37","state":"active+clean","last_fresh":"2024-02-15T18:31:14.829043+0000","last_change":"2024-02-15T18:30:52.837897+0000","last_active":"2024-02-15T18:31:14.829043+0000","last_peered":"2024-02-15T18:31:14.829043+0000","last_clean":"2024-02-15T18:31:14.829043+0000","last_became_active":"2024-02-15T18:30:52.837507+0000","last_became_peered":"2024-02-15T18:30:52.837507+0000","last_unstale":"2024-02-15T18:31:14.829043+0000","last_undegraded":"2024-02-15T18:31:14.829043+0000","last_fullsized":"2024-02-15T18:31:14.829043+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-15T18:30:09.818898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-15T18:30:09.818898+0000","last_clean_scrub_stamp":"2024-02-15T18:30:09.818898+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":37,"seq":158913789967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5076,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":93740076,"statfs":{"total":95995035648,"available":95989837824,"internally_reserved":0,"allocated":339968,"data_stored":128139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":4,"up_from":32,"seq":137438953491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5140,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":93740012,"statfs":{"total":95995035648,"available":95989772288,"internally_reserved":0,"allocated":339968,"data_stored":128139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 15 18:31:54 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.40899999999999997}]},{"osd":1,"last update":"Thu Feb 15 18:31:54 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Feb 15 18:31:54 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.38},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.41899999999999998}]},{"osd":3,"last update":"Thu Feb 15 18:31:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.29799999999999999}]},{"osd":5,"last update":"Thu Feb 15 18:32:16 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.32700000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149720,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5204,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":93739948,"statfs":{"total":95995035648,"available":95989706752,"internally_reserved":0,"allocated":339968,"data_stored":128139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 15 18:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.57899999999999996}]},{"osd":1,"last update":"Thu Feb 15 18:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.54500000000000004}]},{"osd":2,"last update":"Thu Feb 15 18:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.628}]},{"osd":4,"last update":"Thu Feb 15 18:31:56 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.41399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]}]},{"osd":2,"up_from":19,"seq":81604378654,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5204,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":93739948,"statfs":{"total":95995035648,"available":95989706752,"internally_reserved":0,"allocated":339968,"data_stored":128139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 15 18:32:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.33200000000000002,"15min":0.33000000000000002},"min":{"1min":0.21099999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.35199999999999998,"15min":0.35399999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.51900000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Feb 15 18:32:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.26600000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.375,"15min":0.379},"min":{"1min":0.23599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54800000000000004,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Feb 15 18:31:33 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.77600000000000002}]},{"osd":4,"last update":"Thu Feb 15 18:31:56 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.74299999999999999}]},{"osd":5,"last update":"Thu Feb 15 18:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.73199999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705706,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5332,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":93739820,"statfs":{"total":95995035648,"available":95989575680,"internally_reserved":0,"allocated":339968,"data_stored":128139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 15 18:31:45 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.32200000000000001,"15min":0.318},"min":{"1min":0.24199999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.32400000000000001,"15min":0.31900000000000001},"min":{"1min":0.248,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.26900000000000002}]},{"osd":2,"last update":"Thu Feb 15 18:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.35399999999999998,"15min":0.35199999999999998},"min":{"1min":0.255,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.255},{"interface":"front","average":{"1min":0.372,"5min":0.375,"15min":0.375},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.315}]},{"osd":3,"last update":"Thu Feb 15 18:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Feb 15 18:31:53 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.57899999999999996}]},{"osd":5,"last update":"Thu Feb 15 18:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.54600000000000004}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5268,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":93739884,"statfs":{"total":95995035648,"available":95989641216,"internally_reserved":0,"allocated":339968,"data_stored":128139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 15 18:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.34599999999999997,"15min":0.34200000000000003},"min":{"1min":0.27900000000000003,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.372},{"interface":"front","average":{"1min":0.376,"5min":0.34899999999999998,"15min":0.34499999999999997},"min":{"1min":0.29199999999999998,"5min":0.223,"15min":0.223},"max":{"1min":0.52800000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Feb 15 18:32:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.371,"15min":0.372},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.36699999999999999,"15min":0.36799999999999999},"min":{"1min":0.246,"5min":0.219,"15min":0.219},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.35399999999999998}]},{"osd":3,"last update":"Thu Feb 15 18:31:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.69599999999999995}]},{"osd":4,"last update":"Thu Feb 15 18:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.621},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.66800000000000004}]},{"osd":5,"last update":"Thu Feb 15 18:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.56299999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-15T18:32:21.340 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-15T18:32:21.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:20.271530+0000 mgr.smithi038.khesbl (mgr.14162) 226 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:21.525 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:20.607470+0000 mgr.smithi038.khesbl (mgr.14162) 227 : audit [DBG] from='client.14400 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-15T18:32:21.576 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/config 2024-02-15T18:32:21.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:20.271530+0000 mgr.smithi038.khesbl (mgr.14162) 226 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:21.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:20.607470+0000 mgr.smithi038.khesbl (mgr.14162) 227 : audit [DBG] from='client.14400 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-15T18:32:23.582 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:22.271839+0000 mgr.smithi038.khesbl (mgr.14162) 228 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:23.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:22.271839+0000 mgr.smithi038.khesbl (mgr.14162) 228 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:23.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.820+0000 7f03c6a61700 1 -- 172.21.15.38:0/149681406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03c00f7a80 msgr2=0x7f03c00f7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:23.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.820+0000 7f03c6a61700 1 --2- 172.21.15.38:0/149681406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03c00f7a80 0x7f03c00f7ee0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f03a8004660 tx=0x7f03a8009b30).stop 2024-02-15T18:32:23.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.821+0000 7f03c6a61700 1 -- 172.21.15.38:0/149681406 shutdown_connections 2024-02-15T18:32:23.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.821+0000 7f03c6a61700 1 --2- 172.21.15.38:0/149681406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03c00f7a80 0x7f03c00f7ee0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:23.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.821+0000 7f03c6a61700 1 --2- 172.21.15.38:0/149681406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03c00f67e0 0x7f03c00f6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:23.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.821+0000 7f03c6a61700 1 -- 172.21.15.38:0/149681406 >> 172.21.15.38:0/149681406 conn(0x7f03c00f1f20 msgr2=0x7f03c00f4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:23.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.821+0000 7f03c6a61700 1 -- 172.21.15.38:0/149681406 shutdown_connections 2024-02-15T18:32:23.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.821+0000 7f03c6a61700 1 -- 172.21.15.38:0/149681406 wait complete. 2024-02-15T18:32:23.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.821+0000 7f03c6a61700 1 Processor -- start 2024-02-15T18:32:23.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.822+0000 7f03c6a61700 1 -- start start 2024-02-15T18:32:23.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.822+0000 7f03c6a61700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03c00f67e0 0x7f03c0107120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:23.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.822+0000 7f03c6a61700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03c00f7a80 0x7f03c0107610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:23.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.822+0000 7f03c6a61700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c0107bc0 con 0x7f03c00f7a80 2024-02-15T18:32:23.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.822+0000 7f03c6a61700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c0107d00 con 0x7f03c00f67e0 2024-02-15T18:32:23.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.822+0000 7f03bf7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03c00f7a80 0x7f03c0107610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:23.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.822+0000 7f03bf7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03c00f7a80 0x7f03c0107610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38994/0 (socket says 172.21.15.38:38994) 2024-02-15T18:32:23.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.822+0000 7f03bf7fe700 1 -- 172.21.15.38:0/1017056091 learned_addr learned my addr 172.21.15.38:0/1017056091 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:32:23.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.822+0000 7f03bffff700 1 --2- 172.21.15.38:0/1017056091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03c00f67e0 0x7f03c0107120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:23.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.822+0000 7f03bf7fe700 1 -- 172.21.15.38:0/1017056091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03c00f67e0 msgr2=0x7f03c0107120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:23.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.822+0000 7f03bf7fe700 1 --2- 172.21.15.38:0/1017056091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03c00f67e0 0x7f03c0107120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:23.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.822+0000 7f03bf7fe700 1 -- 172.21.15.38:0/1017056091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a8005040 con 0x7f03c00f7a80 2024-02-15T18:32:23.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.822+0000 7f03bf7fe700 1 --2- 172.21.15.38:0/1017056091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03c00f7a80 0x7f03c0107610 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f03a8009830 tx=0x7f03a8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:23.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.822+0000 7f03bcff9700 1 -- 172.21.15.38:0/1017056091 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03a8018070 con 0x7f03c00f7a80 2024-02-15T18:32:23.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.823+0000 7f03c6a61700 1 -- 172.21.15.38:0/1017056091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c0108640 con 0x7f03c00f7a80 2024-02-15T18:32:23.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.823+0000 7f03bcff9700 1 -- 172.21.15.38:0/1017056091 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03a8007ba0 con 0x7f03c00f7a80 2024-02-15T18:32:23.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.823+0000 7f03bcff9700 1 -- 172.21.15.38:0/1017056091 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03a801c3c0 con 0x7f03c00f7a80 2024-02-15T18:32:23.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.823+0000 7f03c6a61700 1 -- 172.21.15.38:0/1017056091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c0108ba0 con 0x7f03c00f7a80 2024-02-15T18:32:23.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.824+0000 7f03c6a61700 1 -- 172.21.15.38:0/1017056091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c0045ad0 con 0x7f03c00f7a80 2024-02-15T18:32:23.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.824+0000 7f03bcff9700 1 -- 172.21.15.38:0/1017056091 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f03a8007d10 con 0x7f03c00f7a80 2024-02-15T18:32:23.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.824+0000 7f03bcff9700 1 --2- 172.21.15.38:0/1017056091 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f03ac05c2a0 0x7f03ac05e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:23.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.824+0000 7f03bcff9700 1 -- 172.21.15.38:0/1017056091 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f03a8077800 con 0x7f03c00f7a80 2024-02-15T18:32:23.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.824+0000 7f03bffff700 1 --2- 172.21.15.38:0/1017056091 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f03ac05c2a0 0x7f03ac05e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:23.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.825+0000 7f03bffff700 1 --2- 172.21.15.38:0/1017056091 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f03ac05c2a0 0x7f03ac05e760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f03c00f78e0 tx=0x7f03b000b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:23.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.829+0000 7f03bcff9700 1 -- 172.21.15.38:0/1017056091 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f03a804a7a0 con 0x7f03c00f7a80 2024-02-15T18:32:23.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.980+0000 7f03c6a61700 1 -- 172.21.15.38:0/1017056091 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f03c0108dc0 con 0x7f03ac05c2a0 2024-02-15T18:32:23.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.983+0000 7f03bcff9700 1 -- 172.21.15.38:0/1017056091 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+31754 (secure 0 0 0) 0x7f03c0108dc0 con 0x7f03ac05c2a0 2024-02-15T18:32:23.975 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:32:23.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.983+0000 7f03c6a61700 1 -- 172.21.15.38:0/1017056091 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f03ac05c2a0 msgr2=0x7f03ac05e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:23.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.983+0000 7f03c6a61700 1 --2- 172.21.15.38:0/1017056091 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f03ac05c2a0 0x7f03ac05e760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f03c00f78e0 tx=0x7f03b000b040).stop 2024-02-15T18:32:23.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.984+0000 7f03c6a61700 1 -- 172.21.15.38:0/1017056091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03c00f7a80 msgr2=0x7f03c0107610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:23.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.984+0000 7f03c6a61700 1 --2- 172.21.15.38:0/1017056091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03c00f7a80 0x7f03c0107610 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f03a8009830 tx=0x7f03a8009b30).stop 2024-02-15T18:32:23.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.984+0000 7f03c6a61700 1 -- 172.21.15.38:0/1017056091 shutdown_connections 2024-02-15T18:32:23.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.984+0000 7f03c6a61700 1 --2- 172.21.15.38:0/1017056091 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f03ac05c2a0 0x7f03ac05e760 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:23.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.984+0000 7f03c6a61700 1 --2- 172.21.15.38:0/1017056091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03c00f67e0 0x7f03c0107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:23.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.984+0000 7f03c6a61700 1 --2- 172.21.15.38:0/1017056091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03c00f7a80 0x7f03c0107610 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:23.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.984+0000 7f03c6a61700 1 -- 172.21.15.38:0/1017056091 >> 172.21.15.38:0/1017056091 conn(0x7f03c00f1f20 msgr2=0x7f03c00fb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:23.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.984+0000 7f03c6a61700 1 -- 172.21.15.38:0/1017056091 shutdown_connections 2024-02-15T18:32:23.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:23.984+0000 7f03c6a61700 1 -- 172.21.15.38:0/1017056091 wait complete. 2024-02-15T18:32:23.978 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-02-15T18:32:24.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:24 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:24.306979+0000 mon.smithi038 (mon.0) 451 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:24.683 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:24.306979+0000 mon.smithi038 (mon.0) 451 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:24.704 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-02-15T18:32:22.271696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":31224,"kb_used_data":1992,"kb_used_omap":0,"kb_used_meta":29184,"kb_avail":562439688,"statfs":{"total":575970213888,"available":575938240512,"internally_reserved":0,"allocated":2039808,"data_stored":768834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29884416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":"36","reported_epoch":"37","state":"active+clean","last_fresh":"2024-02-15T18:31:14.829043+0000","last_change":"2024-02-15T18:30:52.837897+0000","last_active":"2024-02-15T18:31:14.829043+0000","last_peered":"2024-02-15T18:31:14.829043+0000","last_clean":"2024-02-15T18:31:14.829043+0000","last_became_active":"2024-02-15T18:30:52.837507+0000","last_became_peered":"2024-02-15T18:30:52.837507+0000","last_unstale":"2024-02-15T18:31:14.829043+0000","last_undegraded":"2024-02-15T18:31:14.829043+0000","last_fullsized":"2024-02-15T18:31:14.829043+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-15T18:30:09.818898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-15T18:30:09.818898+0000","last_clean_scrub_stamp":"2024-02-15T18:30:09.818898+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":37,"seq":158913789968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5076,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":93740076,"statfs":{"total":95995035648,"available":95989837824,"internally_reserved":0,"allocated":339968,"data_stored":128139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 15 18:32:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.435}]},{"osd":1,"last update":"Thu Feb 15 18:32:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.47799999999999998}]},{"osd":2,"last update":"Thu Feb 15 18:32:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.55000000000000004}]},{"osd":3,"last update":"Thu Feb 15 18:32:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.255}]},{"osd":4,"last update":"Thu Feb 15 18:32:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.34599999999999997}]}]},{"osd":4,"up_from":32,"seq":137438953491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5140,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":93740012,"statfs":{"total":95995035648,"available":95989772288,"internally_reserved":0,"allocated":339968,"data_stored":128139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 15 18:31:54 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.40899999999999997}]},{"osd":1,"last update":"Thu Feb 15 18:31:54 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Feb 15 18:31:54 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.38},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.41899999999999998}]},{"osd":3,"last update":"Thu Feb 15 18:31:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.29799999999999999}]},{"osd":5,"last update":"Thu Feb 15 18:32:16 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.32700000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149721,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5204,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":93739948,"statfs":{"total":95995035648,"available":95989706752,"internally_reserved":0,"allocated":339968,"data_stored":128139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 15 18:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.41399999999999998}]},{"osd":1,"last update":"Thu Feb 15 18:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.32400000000000001}]},{"osd":2,"last update":"Thu Feb 15 18:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.372},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.39700000000000002}]},{"osd":4,"last update":"Thu Feb 15 18:31:56 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.313},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.46700000000000003}]},{"osd":5,"last update":"Thu Feb 15 18:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.42899999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378654,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5204,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":93739948,"statfs":{"total":95995035648,"available":95989706752,"internally_reserved":0,"allocated":339968,"data_stored":128139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 15 18:32:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.33200000000000002,"15min":0.33000000000000002},"min":{"1min":0.21099999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.35199999999999998,"15min":0.35399999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.51900000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.47099999999999997}]},{"osd":1,"last update":"Thu Feb 15 18:32:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.26600000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.375,"15min":0.379},"min":{"1min":0.23599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54800000000000004,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Feb 15 18:31:33 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.77600000000000002}]},{"osd":4,"last update":"Thu Feb 15 18:31:56 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.74299999999999999}]},{"osd":5,"last update":"Thu Feb 15 18:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.73199999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705706,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5332,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":93739820,"statfs":{"total":95995035648,"available":95989575680,"internally_reserved":0,"allocated":339968,"data_stored":128139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 15 18:31:45 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.32200000000000001,"15min":0.318},"min":{"1min":0.24199999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.32400000000000001,"15min":0.31900000000000001},"min":{"1min":0.248,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.26900000000000002}]},{"osd":2,"last update":"Thu Feb 15 18:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.35399999999999998,"15min":0.35199999999999998},"min":{"1min":0.255,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.255},{"interface":"front","average":{"1min":0.372,"5min":0.375,"15min":0.375},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.315}]},{"osd":3,"last update":"Thu Feb 15 18:31:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Feb 15 18:31:53 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.57899999999999996}]},{"osd":5,"last update":"Thu Feb 15 18:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.54600000000000004}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5268,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":93739884,"statfs":{"total":95995035648,"available":95989641216,"internally_reserved":0,"allocated":339968,"data_stored":128139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 15 18:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.34599999999999997,"15min":0.34200000000000003},"min":{"1min":0.27900000000000003,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.372},{"interface":"front","average":{"1min":0.376,"5min":0.34899999999999998,"15min":0.34499999999999997},"min":{"1min":0.29199999999999998,"5min":0.223,"15min":0.223},"max":{"1min":0.52800000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Feb 15 18:32:11 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.371,"15min":0.372},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.36699999999999999,"15min":0.36799999999999999},"min":{"1min":0.246,"5min":0.219,"15min":0.219},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.35399999999999998}]},{"osd":3,"last update":"Thu Feb 15 18:31:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.69599999999999995}]},{"osd":4,"last update":"Thu Feb 15 18:31:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.621},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.66800000000000004}]},{"osd":5,"last update":"Thu Feb 15 18:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.56299999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-15T18:32:24.705 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-15T18:32:24.705 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-15T18:32:24.705 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-15T18:32:24.706 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- ceph health --format=json 2024-02-15T18:32:24.943 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/mon.smithi038/config 2024-02-15T18:32:26.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:23.982594+0000 mgr.smithi038.khesbl (mgr.14162) 229 : audit [DBG] from='client.14404 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-15T18:32:26.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:24.272162+0000 mgr.smithi038.khesbl (mgr.14162) 230 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:26.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:24.791613+0000 mon.smithi038 (mon.0) 452 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:32:26.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:23.982594+0000 mgr.smithi038.khesbl (mgr.14162) 229 : audit [DBG] from='client.14404 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-15T18:32:26.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:24.272162+0000 mgr.smithi038.khesbl (mgr.14162) 230 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:26.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:24.791613+0000 mon.smithi038 (mon.0) 452 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:32:26.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.810+0000 7f4786608700 1 -- 172.21.15.38:0/725609489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47800f6960 msgr2=0x7f47800f6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:26.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.810+0000 7f4786608700 1 --2- 172.21.15.38:0/725609489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47800f6960 0x7f47800f6dc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f4770004660 tx=0x7f4770009b30).stop 2024-02-15T18:32:26.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.810+0000 7f4786608700 1 -- 172.21.15.38:0/725609489 shutdown_connections 2024-02-15T18:32:26.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.810+0000 7f4786608700 1 --2- 172.21.15.38:0/725609489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47800f6960 0x7f47800f6dc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:26.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.810+0000 7f4786608700 1 --2- 172.21.15.38:0/725609489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47800f56c0 0x7f47800f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:26.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.810+0000 7f4786608700 1 -- 172.21.15.38:0/725609489 >> 172.21.15.38:0/725609489 conn(0x7f47800f0e20 msgr2=0x7f47800f3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:26.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.810+0000 7f4786608700 1 -- 172.21.15.38:0/725609489 shutdown_connections 2024-02-15T18:32:26.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.810+0000 7f4786608700 1 -- 172.21.15.38:0/725609489 wait complete. 2024-02-15T18:32:26.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.810+0000 7f4786608700 1 Processor -- start 2024-02-15T18:32:26.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.810+0000 7f4786608700 1 -- start start 2024-02-15T18:32:26.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.811+0000 7f4786608700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47800f56c0 0x7f47801060a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.811+0000 7f4786608700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47800f6960 0x7f4780106590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.811+0000 7f4786608700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4780106b40 con 0x7f47800f56c0 2024-02-15T18:32:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.811+0000 7f4786608700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4780106c80 con 0x7f47800f6960 2024-02-15T18:32:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.811+0000 7f477ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47800f56c0 0x7f47801060a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.811+0000 7f477ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47800f56c0 0x7f47801060a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39026/0 (socket says 172.21.15.38:39026) 2024-02-15T18:32:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.811+0000 7f477ffff700 1 -- 172.21.15.38:0/311694174 learned_addr learned my addr 172.21.15.38:0/311694174 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:32:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.811+0000 7f477f7fe700 1 --2- 172.21.15.38:0/311694174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47800f6960 0x7f4780106590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.811+0000 7f477ffff700 1 -- 172.21.15.38:0/311694174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47800f6960 msgr2=0x7f4780106590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.811+0000 7f477ffff700 1 --2- 172.21.15.38:0/311694174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47800f6960 0x7f4780106590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.811+0000 7f477ffff700 1 -- 172.21.15.38:0/311694174 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4770005040 con 0x7f47800f56c0 2024-02-15T18:32:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.811+0000 7f477ffff700 1 --2- 172.21.15.38:0/311694174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47800f56c0 0x7f47801060a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f477400c530 tx=0x7f4774007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:26.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.811+0000 7f477cff9700 1 -- 172.21.15.38:0/311694174 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f477400f650 con 0x7f47800f56c0 2024-02-15T18:32:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.811+0000 7f4786608700 1 -- 172.21.15.38:0/311694174 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4780107620 con 0x7f47800f56c0 2024-02-15T18:32:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.811+0000 7f477cff9700 1 -- 172.21.15.38:0/311694174 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f477400fc80 con 0x7f47800f56c0 2024-02-15T18:32:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.811+0000 7f477cff9700 1 -- 172.21.15.38:0/311694174 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4774012d60 con 0x7f47800f56c0 2024-02-15T18:32:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.811+0000 7f4786608700 1 -- 172.21.15.38:0/311694174 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4780107be0 con 0x7f47800f56c0 2024-02-15T18:32:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.812+0000 7f4786608700 1 -- 172.21.15.38:0/311694174 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4780045ad0 con 0x7f47800f56c0 2024-02-15T18:32:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.813+0000 7f477cff9700 1 -- 172.21.15.38:0/311694174 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f4774019030 con 0x7f47800f56c0 2024-02-15T18:32:26.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.813+0000 7f477cff9700 1 --2- 172.21.15.38:0/311694174 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f476805c2a0 0x7f476805e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:26.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.813+0000 7f477cff9700 1 -- 172.21.15.38:0/311694174 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f4774075740 con 0x7f47800f56c0 2024-02-15T18:32:26.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.813+0000 7f477f7fe700 1 --2- 172.21.15.38:0/311694174 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f476805c2a0 0x7f476805e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:26.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.813+0000 7f477f7fe700 1 --2- 172.21.15.38:0/311694174 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f476805c2a0 0x7f476805e760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f4770009f50 tx=0x7f4770009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:26.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:26.817+0000 7f477cff9700 1 -- 172.21.15.38:0/311694174 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f4774048790 con 0x7f47800f56c0 2024-02-15T18:32:27.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:27.035+0000 7f4786608700 1 -- 172.21.15.38:0/311694174 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f47800fe110 con 0x7f47800f56c0 2024-02-15T18:32:27.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:27.035+0000 7f477cff9700 1 -- 172.21.15.38:0/311694174 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f4774017070 con 0x7f47800f56c0 2024-02-15T18:32:27.028 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:32:27.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:27.037+0000 7f4786608700 1 -- 172.21.15.38:0/311694174 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f476805c2a0 msgr2=0x7f476805e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:27.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:27.037+0000 7f4786608700 1 --2- 172.21.15.38:0/311694174 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f476805c2a0 0x7f476805e760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f4770009f50 tx=0x7f4770009b30).stop 2024-02-15T18:32:27.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:27.037+0000 7f4786608700 1 -- 172.21.15.38:0/311694174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47800f56c0 msgr2=0x7f47801060a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:27.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:27.037+0000 7f4786608700 1 --2- 172.21.15.38:0/311694174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47800f56c0 0x7f47801060a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f477400c530 tx=0x7f4774007e10).stop 2024-02-15T18:32:27.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:27.037+0000 7f4786608700 1 -- 172.21.15.38:0/311694174 shutdown_connections 2024-02-15T18:32:27.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:27.037+0000 7f4786608700 1 --2- 172.21.15.38:0/311694174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47800f56c0 0x7f47801060a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:27.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:27.037+0000 7f4786608700 1 --2- 172.21.15.38:0/311694174 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f476805c2a0 0x7f476805e760 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:27.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:27.037+0000 7f4786608700 1 --2- 172.21.15.38:0/311694174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47800f6960 0x7f4780106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:27.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:27.037+0000 7f4786608700 1 -- 172.21.15.38:0/311694174 >> 172.21.15.38:0/311694174 conn(0x7f47800f0e20 msgr2=0x7f47800fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:27.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:27.037+0000 7f4786608700 1 -- 172.21.15.38:0/311694174 shutdown_connections 2024-02-15T18:32:27.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:27.037+0000 7f4786608700 1 -- 172.21.15.38:0/311694174 wait complete. 2024-02-15T18:32:27.772 INFO:teuthology.orchestra.run.smithi038.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-15T18:32:27.772 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-15T18:32:27.773 INFO:tasks.cephadm:Setup complete, yielding 2024-02-15T18:32:27.773 INFO:teuthology.run_tasks:Running task print... 2024-02-15T18:32:27.782 INFO:teuthology.task.print:**** done starting v16.2.4 2024-02-15T18:32:27.782 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-15T18:32:27.795 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-02-15T18:32:27.796 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- bash -c 'ceph orch status' 2024-02-15T18:32:28.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:26.272558+0000 mgr.smithi038.khesbl (mgr.14162) 231 : cluster [DBG] pgmap v200: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:28.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:27.036300+0000 mon.smithi038 (mon.0) 453 : audit [DBG] from='client.? 172.21.15.38:0/311694174' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-15T18:32:28.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:26.272558+0000 mgr.smithi038.khesbl (mgr.14162) 231 : cluster [DBG] pgmap v200: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:28.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:27.036300+0000 mon.smithi038 (mon.0) 453 : audit [DBG] from='client.? 172.21.15.38:0/311694174' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-15T18:32:29.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.841+0000 7f6cce735700 1 -- 172.21.15.38:0/1293603678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cc80602b0 msgr2=0x7f6cc80f9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:29.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.841+0000 7f6cce735700 1 --2- 172.21.15.38:0/1293603678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cc80602b0 0x7f6cc80f9300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f6cb8004660 tx=0x7f6cb8009b30).stop 2024-02-15T18:32:29.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.841+0000 7f6cce735700 1 -- 172.21.15.38:0/1293603678 shutdown_connections 2024-02-15T18:32:29.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.841+0000 7f6cce735700 1 --2- 172.21.15.38:0/1293603678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cc80f9880 0x7f6cc80fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.841+0000 7f6cce735700 1 --2- 172.21.15.38:0/1293603678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cc80602b0 0x7f6cc80f9300 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.841+0000 7f6cce735700 1 -- 172.21.15.38:0/1293603678 >> 172.21.15.38:0/1293603678 conn(0x7f6cc80f0d40 msgr2=0x7f6cc80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.841+0000 7f6cce735700 1 -- 172.21.15.38:0/1293603678 shutdown_connections 2024-02-15T18:32:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.841+0000 7f6cce735700 1 -- 172.21.15.38:0/1293603678 wait complete. 2024-02-15T18:32:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.841+0000 7f6cce735700 1 Processor -- start 2024-02-15T18:32:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.841+0000 7f6cce735700 1 -- start start 2024-02-15T18:32:29.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.842+0000 7f6cce735700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cc80602b0 0x7f6cc8107040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:29.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.842+0000 7f6cce735700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cc80f9880 0x7f6cc8107530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:29.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.842+0000 7f6cce735700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc8107ae0 con 0x7f6cc80602b0 2024-02-15T18:32:29.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.842+0000 7f6cce735700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc8107c20 con 0x7f6cc80f9880 2024-02-15T18:32:29.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.842+0000 7f6cc7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cc80602b0 0x7f6cc8107040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:29.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.842+0000 7f6cc7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cc80602b0 0x7f6cc8107040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39048/0 (socket says 172.21.15.38:39048) 2024-02-15T18:32:29.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.842+0000 7f6cc7fff700 1 -- 172.21.15.38:0/2854855115 learned_addr learned my addr 172.21.15.38:0/2854855115 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:32:29.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.842+0000 7f6cc77fe700 1 --2- 172.21.15.38:0/2854855115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cc80f9880 0x7f6cc8107530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:29.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.842+0000 7f6cc7fff700 1 -- 172.21.15.38:0/2854855115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cc80f9880 msgr2=0x7f6cc8107530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:29.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.842+0000 7f6cc7fff700 1 --2- 172.21.15.38:0/2854855115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cc80f9880 0x7f6cc8107530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:29.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.842+0000 7f6cc7fff700 1 -- 172.21.15.38:0/2854855115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb8005040 con 0x7f6cc80602b0 2024-02-15T18:32:29.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.842+0000 7f6cc7fff700 1 --2- 172.21.15.38:0/2854855115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cc80602b0 0x7f6cc8107040 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f6cb8009830 tx=0x7f6cb8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:29.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.842+0000 7f6cc4ff9700 1 -- 172.21.15.38:0/2854855115 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cb8018070 con 0x7f6cc80602b0 2024-02-15T18:32:29.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.842+0000 7f6cc4ff9700 1 -- 172.21.15.38:0/2854855115 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cb8007ba0 con 0x7f6cc80602b0 2024-02-15T18:32:29.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.842+0000 7f6cce735700 1 -- 172.21.15.38:0/2854855115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc8108560 con 0x7f6cc80602b0 2024-02-15T18:32:29.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.842+0000 7f6cc4ff9700 1 -- 172.21.15.38:0/2854855115 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cb801c3c0 con 0x7f6cc80602b0 2024-02-15T18:32:29.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.842+0000 7f6cce735700 1 -- 172.21.15.38:0/2854855115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc8108ac0 con 0x7f6cc80602b0 2024-02-15T18:32:29.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.844+0000 7f6cce735700 1 -- 172.21.15.38:0/2854855115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc8045ad0 con 0x7f6cc80602b0 2024-02-15T18:32:29.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.844+0000 7f6cc4ff9700 1 -- 172.21.15.38:0/2854855115 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f6cb8007d10 con 0x7f6cc80602b0 2024-02-15T18:32:29.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.844+0000 7f6cc4ff9700 1 --2- 172.21.15.38:0/2854855115 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6cb005c2b0 0x7f6cb005e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:29.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.844+0000 7f6cc4ff9700 1 -- 172.21.15.38:0/2854855115 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f6cb8077760 con 0x7f6cc80602b0 2024-02-15T18:32:29.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.844+0000 7f6cc77fe700 1 --2- 172.21.15.38:0/2854855115 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6cb005c2b0 0x7f6cb005e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:29.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.844+0000 7f6cc77fe700 1 --2- 172.21.15.38:0/2854855115 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6cb005c2b0 0x7f6cb005e770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f6cc8060080 tx=0x7f6cbc00b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:29.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:29.848+0000 7f6cc4ff9700 1 -- 172.21.15.38:0/2854855115 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f6cb804a780 con 0x7f6cc80602b0 2024-02-15T18:32:30.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:30.041+0000 7f6cce735700 1 -- 172.21.15.38:0/2854855115 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cc80ff0c0 con 0x7f6cb005c2b0 2024-02-15T18:32:30.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:30.042+0000 7f6cc4ff9700 1 -- 172.21.15.38:0/2854855115 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f6cc80ff0c0 con 0x7f6cb005c2b0 2024-02-15T18:32:30.035 INFO:teuthology.orchestra.run.smithi038.stdout:Backend: cephadm 2024-02-15T18:32:30.035 INFO:teuthology.orchestra.run.smithi038.stdout:Available: Yes 2024-02-15T18:32:30.035 INFO:teuthology.orchestra.run.smithi038.stdout:Paused: No 2024-02-15T18:32:30.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:30.044+0000 7f6cce735700 1 -- 172.21.15.38:0/2854855115 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6cb005c2b0 msgr2=0x7f6cb005e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:30.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:30.044+0000 7f6cce735700 1 --2- 172.21.15.38:0/2854855115 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6cb005c2b0 0x7f6cb005e770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f6cc8060080 tx=0x7f6cbc00b040).stop 2024-02-15T18:32:30.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:30.044+0000 7f6cce735700 1 -- 172.21.15.38:0/2854855115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cc80602b0 msgr2=0x7f6cc8107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:30.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:30.044+0000 7f6cce735700 1 --2- 172.21.15.38:0/2854855115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cc80602b0 0x7f6cc8107040 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f6cb8009830 tx=0x7f6cb8009b30).stop 2024-02-15T18:32:30.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:30.044+0000 7f6cce735700 1 -- 172.21.15.38:0/2854855115 shutdown_connections 2024-02-15T18:32:30.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:30.044+0000 7f6cce735700 1 --2- 172.21.15.38:0/2854855115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6cc80602b0 0x7f6cc8107040 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:30.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:30.044+0000 7f6cce735700 1 --2- 172.21.15.38:0/2854855115 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6cb005c2b0 0x7f6cb005e770 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:30.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:30.044+0000 7f6cce735700 1 --2- 172.21.15.38:0/2854855115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cc80f9880 0x7f6cc8107530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:30.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:30.044+0000 7f6cce735700 1 -- 172.21.15.38:0/2854855115 >> 172.21.15.38:0/2854855115 conn(0x7f6cc80f0d40 msgr2=0x7f6cc80f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:30.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:30.044+0000 7f6cce735700 1 -- 172.21.15.38:0/2854855115 shutdown_connections 2024-02-15T18:32:30.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:30.044+0000 7f6cce735700 1 -- 172.21.15.38:0/2854855115 wait complete. 2024-02-15T18:32:30.056 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:28.272903+0000 mgr.smithi038.khesbl (mgr.14162) 232 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:30.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:28.272903+0000 mgr.smithi038.khesbl (mgr.14162) 232 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:30.659 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-15T18:32:32.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:30.042672+0000 mgr.smithi038.khesbl (mgr.14162) 233 : audit [DBG] from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:32.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:30.273255+0000 mgr.smithi038.khesbl (mgr.14162) 234 : cluster [DBG] pgmap v202: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:32.166 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:30.042672+0000 mgr.smithi038.khesbl (mgr.14162) 233 : audit [DBG] from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:32.166 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:30.273255+0000 mgr.smithi038.khesbl (mgr.14162) 234 : cluster [DBG] pgmap v202: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:32.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.686+0000 7fb6e3ad4700 1 -- 172.21.15.38:0/3087463990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6dc0f7980 msgr2=0x7fb6dc0f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:32.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.686+0000 7fb6e3ad4700 1 --2- 172.21.15.38:0/3087463990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6dc0f7980 0x7fb6dc0f7de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fb6cc004660 tx=0x7fb6cc009b30).stop 2024-02-15T18:32:32.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.686+0000 7fb6e3ad4700 1 -- 172.21.15.38:0/3087463990 shutdown_connections 2024-02-15T18:32:32.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.686+0000 7fb6e3ad4700 1 --2- 172.21.15.38:0/3087463990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6dc0f7980 0x7fb6dc0f7de0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:32.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.686+0000 7fb6e3ad4700 1 --2- 172.21.15.38:0/3087463990 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6dc0f66e0 0x7fb6dc0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:32.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.686+0000 7fb6e3ad4700 1 -- 172.21.15.38:0/3087463990 >> 172.21.15.38:0/3087463990 conn(0x7fb6dc0f1e40 msgr2=0x7fb6dc0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:32.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.686+0000 7fb6e3ad4700 1 -- 172.21.15.38:0/3087463990 shutdown_connections 2024-02-15T18:32:32.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.687+0000 7fb6e3ad4700 1 -- 172.21.15.38:0/3087463990 wait complete. 2024-02-15T18:32:32.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.687+0000 7fb6e3ad4700 1 Processor -- start 2024-02-15T18:32:32.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.687+0000 7fb6e3ad4700 1 -- start start 2024-02-15T18:32:32.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.687+0000 7fb6e3ad4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6dc0f66e0 0x7fb6dc107090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:32.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.687+0000 7fb6e3ad4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6dc0f7980 0x7fb6dc107580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:32.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.687+0000 7fb6e3ad4700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc107b30 con 0x7fb6dc0f7980 2024-02-15T18:32:32.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.687+0000 7fb6e3ad4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc107c70 con 0x7fb6dc0f66e0 2024-02-15T18:32:32.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.687+0000 7fb6e106f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6dc0f7980 0x7fb6dc107580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:32.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.687+0000 7fb6e106f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6dc0f7980 0x7fb6dc107580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39058/0 (socket says 172.21.15.38:39058) 2024-02-15T18:32:32.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.687+0000 7fb6e106f700 1 -- 172.21.15.38:0/753849086 learned_addr learned my addr 172.21.15.38:0/753849086 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:32:32.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.687+0000 7fb6e1870700 1 --2- 172.21.15.38:0/753849086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6dc0f66e0 0x7fb6dc107090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:32.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.688+0000 7fb6e106f700 1 -- 172.21.15.38:0/753849086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6dc0f66e0 msgr2=0x7fb6dc107090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:32.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.688+0000 7fb6e106f700 1 --2- 172.21.15.38:0/753849086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6dc0f66e0 0x7fb6dc107090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:32.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.688+0000 7fb6e106f700 1 -- 172.21.15.38:0/753849086 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6cc005040 con 0x7fb6dc0f7980 2024-02-15T18:32:32.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.688+0000 7fb6e106f700 1 --2- 172.21.15.38:0/753849086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6dc0f7980 0x7fb6dc107580 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fb6cc009830 tx=0x7fb6cc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:32.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.688+0000 7fb6d27fc700 1 -- 172.21.15.38:0/753849086 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6cc018070 con 0x7fb6dc0f7980 2024-02-15T18:32:32.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.688+0000 7fb6d27fc700 1 -- 172.21.15.38:0/753849086 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6cc007ba0 con 0x7fb6dc0f7980 2024-02-15T18:32:32.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.688+0000 7fb6e3ad4700 1 -- 172.21.15.38:0/753849086 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6dc1085b0 con 0x7fb6dc0f7980 2024-02-15T18:32:32.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.688+0000 7fb6d27fc700 1 -- 172.21.15.38:0/753849086 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6cc01c3c0 con 0x7fb6dc0f7980 2024-02-15T18:32:32.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.688+0000 7fb6e3ad4700 1 -- 172.21.15.38:0/753849086 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6dc108b10 con 0x7fb6dc0f7980 2024-02-15T18:32:32.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.689+0000 7fb6e3ad4700 1 -- 172.21.15.38:0/753849086 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6dc045ad0 con 0x7fb6dc0f7980 2024-02-15T18:32:32.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.689+0000 7fb6d27fc700 1 -- 172.21.15.38:0/753849086 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fb6cc007d10 con 0x7fb6dc0f7980 2024-02-15T18:32:32.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.690+0000 7fb6d27fc700 1 --2- 172.21.15.38:0/753849086 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb6c805c2b0 0x7fb6c805e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:32.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.690+0000 7fb6d27fc700 1 -- 172.21.15.38:0/753849086 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7fb6cc077720 con 0x7fb6dc0f7980 2024-02-15T18:32:32.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.690+0000 7fb6e1870700 1 --2- 172.21.15.38:0/753849086 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb6c805c2b0 0x7fb6c805e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:32.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.690+0000 7fb6e1870700 1 --2- 172.21.15.38:0/753849086 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb6c805c2b0 0x7fb6c805e770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fb6dc0f77e0 tx=0x7fb6d800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:32.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.694+0000 7fb6d27fc700 1 -- 172.21.15.38:0/753849086 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb6cc04a740 con 0x7fb6dc0f7980 2024-02-15T18:32:32.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.883+0000 7fb6e3ad4700 1 -- 172.21.15.38:0/753849086 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6dc0ff130 con 0x7fb6c805c2b0 2024-02-15T18:32:32.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.890+0000 7fb6d27fc700 1 -- 172.21.15.38:0/753849086 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2160 (secure 0 0 0) 0x7fb6dc0ff130 con 0x7fb6c805c2b0 2024-02-15T18:32:32.882 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE VERSION IMAGE ID CONTAINER ID 2024-02-15T18:32:32.882 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (3m) 68s ago 5m 0.20.0 0881eb8f169f c2034c318e7c 2024-02-15T18:32:32.882 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (5m) 68s ago 5m 16.2.4 8d91d370c2b8 0fd3f945d009 2024-02-15T18:32:32.882 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi121 smithi121 running (4m) 82s ago 4m 16.2.4 8d91d370c2b8 affc02db354e 2024-02-15T18:32:32.882 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (3m) 68s ago 5m 6.7.4 557c83e11646 2dcccfbfbd2a 2024-02-15T18:32:32.882 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.khesbl smithi038 *:9283 running (6m) 68s ago 6m 16.2.4 8d91d370c2b8 b0d099c255f4 2024-02-15T18:32:32.883 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi121.ckiryi smithi121 *:8443,9283 running (4m) 82s ago 4m 16.2.4 8d91d370c2b8 628756a555db 2024-02-15T18:32:32.883 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (6m) 68s ago 6m 16.2.4 8d91d370c2b8 01b30eaebcfe 2024-02-15T18:32:32.883 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi121 smithi121 running (4m) 82s ago 4m 16.2.4 8d91d370c2b8 8f588ecacb71 2024-02-15T18:32:32.883 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (4m) 68s ago 4m 0.18.1 e5a616e4b9cf 037f60e38158 2024-02-15T18:32:32.883 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi121 smithi121 *:9100 running (3m) 82s ago 3m 0.18.1 e5a616e4b9cf a867bff85e18 2024-02-15T18:32:32.883 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (3m) 68s ago 3m 16.2.4 8d91d370c2b8 51512003a2ef 2024-02-15T18:32:32.883 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (2m) 68s ago 2m 16.2.4 8d91d370c2b8 16dbdf20b170 2024-02-15T18:32:32.883 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (2m) 68s ago 2m 16.2.4 8d91d370c2b8 6d6ee813e5be 2024-02-15T18:32:32.883 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi121 running (2m) 82s ago 2m 16.2.4 8d91d370c2b8 cba2d9ed4730 2024-02-15T18:32:32.883 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi121 running (107s) 82s ago 106s 16.2.4 8d91d370c2b8 ac17ad7dc628 2024-02-15T18:32:32.883 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi121 running (85s) 82s ago 84s 16.2.4 8d91d370c2b8 cdc8c669c4db 2024-02-15T18:32:32.883 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (3m) 68s ago 4m 2.18.1 de242295e225 574a5d75991e 2024-02-15T18:32:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.892+0000 7fb6e3ad4700 1 -- 172.21.15.38:0/753849086 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb6c805c2b0 msgr2=0x7fb6c805e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.892+0000 7fb6e3ad4700 1 --2- 172.21.15.38:0/753849086 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb6c805c2b0 0x7fb6c805e770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fb6dc0f77e0 tx=0x7fb6d800b040).stop 2024-02-15T18:32:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.892+0000 7fb6e3ad4700 1 -- 172.21.15.38:0/753849086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6dc0f7980 msgr2=0x7fb6dc107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.892+0000 7fb6e3ad4700 1 --2- 172.21.15.38:0/753849086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6dc0f7980 0x7fb6dc107580 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7fb6cc009830 tx=0x7fb6cc009b30).stop 2024-02-15T18:32:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.892+0000 7fb6e3ad4700 1 -- 172.21.15.38:0/753849086 shutdown_connections 2024-02-15T18:32:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.892+0000 7fb6e3ad4700 1 --2- 172.21.15.38:0/753849086 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb6c805c2b0 0x7fb6c805e770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.892+0000 7fb6e3ad4700 1 --2- 172.21.15.38:0/753849086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6dc0f66e0 0x7fb6dc107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.892+0000 7fb6e3ad4700 1 --2- 172.21.15.38:0/753849086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6dc0f7980 0x7fb6dc107580 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.892+0000 7fb6e3ad4700 1 -- 172.21.15.38:0/753849086 >> 172.21.15.38:0/753849086 conn(0x7fb6dc0f1e40 msgr2=0x7fb6dc0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.892+0000 7fb6e3ad4700 1 -- 172.21.15.38:0/753849086 shutdown_connections 2024-02-15T18:32:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:32.892+0000 7fb6e3ad4700 1 -- 172.21.15.38:0/753849086 wait complete. 2024-02-15T18:32:33.656 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-15T18:32:34.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:32.273668+0000 mgr.smithi038.khesbl (mgr.14162) 235 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:34.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:32.884835+0000 mgr.smithi038.khesbl (mgr.14162) 236 : audit [DBG] from='client.14416 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:34.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:32.273668+0000 mgr.smithi038.khesbl (mgr.14162) 235 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:34.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:32.884835+0000 mgr.smithi038.khesbl (mgr.14162) 236 : audit [DBG] from='client.14416 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:35.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.740+0000 7fca8d86a700 1 -- 172.21.15.38:0/624755434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca880f5570 msgr2=0x7fca880f5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:35.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.740+0000 7fca8d86a700 1 --2- 172.21.15.38:0/624755434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca880f5570 0x7fca880f5990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fca78004660 tx=0x7fca78009b30).stop 2024-02-15T18:32:35.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.740+0000 7fca8d86a700 1 -- 172.21.15.38:0/624755434 shutdown_connections 2024-02-15T18:32:35.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.740+0000 7fca8d86a700 1 --2- 172.21.15.38:0/624755434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca880f6810 0x7fca880f6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:35.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.740+0000 7fca8d86a700 1 --2- 172.21.15.38:0/624755434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca880f5570 0x7fca880f5990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:35.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.740+0000 7fca8d86a700 1 -- 172.21.15.38:0/624755434 >> 172.21.15.38:0/624755434 conn(0x7fca880f0cd0 msgr2=0x7fca880f30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:35.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.740+0000 7fca8d86a700 1 -- 172.21.15.38:0/624755434 shutdown_connections 2024-02-15T18:32:35.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.740+0000 7fca8d86a700 1 -- 172.21.15.38:0/624755434 wait complete. 2024-02-15T18:32:35.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.741+0000 7fca8d86a700 1 Processor -- start 2024-02-15T18:32:35.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.741+0000 7fca8d86a700 1 -- start start 2024-02-15T18:32:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.741+0000 7fca8d86a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca880f5570 0x7fca88106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.741+0000 7fca8d86a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca880f6810 0x7fca881074a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.741+0000 7fca8d86a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca88107a50 con 0x7fca880f6810 2024-02-15T18:32:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.741+0000 7fca8d86a700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca88107b90 con 0x7fca880f5570 2024-02-15T18:32:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.741+0000 7fca867fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca880f6810 0x7fca881074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.741+0000 7fca867fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca880f6810 0x7fca881074a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35126/0 (socket says 172.21.15.38:35126) 2024-02-15T18:32:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.741+0000 7fca867fc700 1 -- 172.21.15.38:0/2529303296 learned_addr learned my addr 172.21.15.38:0/2529303296 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:32:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.741+0000 7fca86ffd700 1 --2- 172.21.15.38:0/2529303296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca880f5570 0x7fca88106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.741+0000 7fca867fc700 1 -- 172.21.15.38:0/2529303296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca880f5570 msgr2=0x7fca88106fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.741+0000 7fca867fc700 1 --2- 172.21.15.38:0/2529303296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca880f5570 0x7fca88106fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.741+0000 7fca867fc700 1 -- 172.21.15.38:0/2529303296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca78005040 con 0x7fca880f6810 2024-02-15T18:32:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.741+0000 7fca86ffd700 1 --2- 172.21.15.38:0/2529303296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca880f5570 0x7fca88106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:32:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.741+0000 7fca867fc700 1 --2- 172.21.15.38:0/2529303296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca880f6810 0x7fca881074a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fca7000c530 tx=0x7fca70007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:35.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.742+0000 7fca7f7fe700 1 -- 172.21.15.38:0/2529303296 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca7000f650 con 0x7fca880f6810 2024-02-15T18:32:35.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.742+0000 7fca8d86a700 1 -- 172.21.15.38:0/2529303296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca88108530 con 0x7fca880f6810 2024-02-15T18:32:35.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.742+0000 7fca7f7fe700 1 -- 172.21.15.38:0/2529303296 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca7000fc80 con 0x7fca880f6810 2024-02-15T18:32:35.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.742+0000 7fca7f7fe700 1 -- 172.21.15.38:0/2529303296 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca70012d80 con 0x7fca880f6810 2024-02-15T18:32:35.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.742+0000 7fca8d86a700 1 -- 172.21.15.38:0/2529303296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca88108af0 con 0x7fca880f6810 2024-02-15T18:32:35.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.743+0000 7fca8d86a700 1 -- 172.21.15.38:0/2529303296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca88045ad0 con 0x7fca880f6810 2024-02-15T18:32:35.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.743+0000 7fca7f7fe700 1 -- 172.21.15.38:0/2529303296 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fca70010070 con 0x7fca880f6810 2024-02-15T18:32:35.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.743+0000 7fca7f7fe700 1 --2- 172.21.15.38:0/2529303296 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fca7405c300 0x7fca7405e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:35.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.743+0000 7fca7f7fe700 1 -- 172.21.15.38:0/2529303296 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7fca70075930 con 0x7fca880f6810 2024-02-15T18:32:35.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.744+0000 7fca86ffd700 1 --2- 172.21.15.38:0/2529303296 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fca7405c300 0x7fca7405e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:35.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.744+0000 7fca86ffd700 1 --2- 172.21.15.38:0/2529303296 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fca7405c300 0x7fca7405e7c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fca78004e30 tx=0x7fca78009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:35.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.748+0000 7fca7f7fe700 1 -- 172.21.15.38:0/2529303296 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fca700452b0 con 0x7fca880f6810 2024-02-15T18:32:35.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.933+0000 7fca8d86a700 1 -- 172.21.15.38:0/2529303296 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fca880fac50 con 0x7fca7405c300 2024-02-15T18:32:35.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.937+0000 7fca7f7fe700 1 -- 172.21.15.38:0/2529303296 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1116 (secure 0 0 0) 0x7fca880fac50 con 0x7fca7405c300 2024-02-15T18:32:35.929 INFO:teuthology.orchestra.run.smithi038.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-15T18:32:35.929 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager 1/1 71s ago 6m count:1 2024-02-15T18:32:35.929 INFO:teuthology.orchestra.run.smithi038.stdout:crash 2/2 85s ago 6m * 2024-02-15T18:32:35.929 INFO:teuthology.orchestra.run.smithi038.stdout:grafana 1/1 71s ago 6m count:1 2024-02-15T18:32:35.929 INFO:teuthology.orchestra.run.smithi038.stdout:mgr 2/2 85s ago 6m count:2 2024-02-15T18:32:35.929 INFO:teuthology.orchestra.run.smithi038.stdout:mon 2/2 85s ago 5m smithi038:172.21.15.38=smithi038;smithi121:172.21.15.121=smithi121;count:2 2024-02-15T18:32:35.929 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter 2/2 85s ago 6m * 2024-02-15T18:32:35.929 INFO:teuthology.orchestra.run.smithi038.stdout:osd.unmanaged 6/6 85s ago - 2024-02-15T18:32:35.929 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus 1/1 71s ago 6m count:1 2024-02-15T18:32:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.938+0000 7fca8d86a700 1 -- 172.21.15.38:0/2529303296 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fca7405c300 msgr2=0x7fca7405e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.938+0000 7fca8d86a700 1 --2- 172.21.15.38:0/2529303296 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fca7405c300 0x7fca7405e7c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fca78004e30 tx=0x7fca78009b30).stop 2024-02-15T18:32:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.939+0000 7fca8d86a700 1 -- 172.21.15.38:0/2529303296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca880f6810 msgr2=0x7fca881074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.939+0000 7fca8d86a700 1 --2- 172.21.15.38:0/2529303296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca880f6810 0x7fca881074a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fca7000c530 tx=0x7fca70007e20).stop 2024-02-15T18:32:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.939+0000 7fca8d86a700 1 -- 172.21.15.38:0/2529303296 shutdown_connections 2024-02-15T18:32:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.939+0000 7fca8d86a700 1 --2- 172.21.15.38:0/2529303296 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fca7405c300 0x7fca7405e7c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.939+0000 7fca8d86a700 1 --2- 172.21.15.38:0/2529303296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca880f5570 0x7fca88106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.939+0000 7fca8d86a700 1 --2- 172.21.15.38:0/2529303296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca880f6810 0x7fca881074a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.939+0000 7fca8d86a700 1 -- 172.21.15.38:0/2529303296 >> 172.21.15.38:0/2529303296 conn(0x7fca880f0cd0 msgr2=0x7fca8805fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.939+0000 7fca8d86a700 1 -- 172.21.15.38:0/2529303296 shutdown_connections 2024-02-15T18:32:35.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:35.939+0000 7fca8d86a700 1 -- 172.21.15.38:0/2529303296 wait complete. 2024-02-15T18:32:36.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:34.273953+0000 mgr.smithi038.khesbl (mgr.14162) 237 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:36.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:34.273953+0000 mgr.smithi038.khesbl (mgr.14162) 237 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:36.640 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-15T18:32:38.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:35.934415+0000 mgr.smithi038.khesbl (mgr.14162) 238 : audit [DBG] from='client.14420 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:38.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:36.274352+0000 mgr.smithi038.khesbl (mgr.14162) 239 : cluster [DBG] pgmap v205: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:38.158 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:35.934415+0000 mgr.smithi038.khesbl (mgr.14162) 238 : audit [DBG] from='client.14420 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:38.158 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:36.274352+0000 mgr.smithi038.khesbl (mgr.14162) 239 : cluster [DBG] pgmap v205: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:38.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.778+0000 7f3145c99700 1 -- 172.21.15.38:0/2096922259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31400f66e0 msgr2=0x7f31400f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:38.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.778+0000 7f3145c99700 1 --2- 172.21.15.38:0/2096922259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31400f66e0 0x7f31400f6b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f3134004660 tx=0x7f3134009b30).stop 2024-02-15T18:32:38.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.778+0000 7f3145c99700 1 -- 172.21.15.38:0/2096922259 shutdown_connections 2024-02-15T18:32:38.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.778+0000 7f3145c99700 1 --2- 172.21.15.38:0/2096922259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31400f7980 0x7f31400f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:38.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.778+0000 7f3145c99700 1 --2- 172.21.15.38:0/2096922259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31400f66e0 0x7f31400f6b00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:38.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.778+0000 7f3145c99700 1 -- 172.21.15.38:0/2096922259 >> 172.21.15.38:0/2096922259 conn(0x7f31400f1e40 msgr2=0x7f31400f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:38.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.778+0000 7f3145c99700 1 -- 172.21.15.38:0/2096922259 shutdown_connections 2024-02-15T18:32:38.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.778+0000 7f3145c99700 1 -- 172.21.15.38:0/2096922259 wait complete. 2024-02-15T18:32:38.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.778+0000 7f3145c99700 1 Processor -- start 2024-02-15T18:32:38.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.779+0000 7f3145c99700 1 -- start start 2024-02-15T18:32:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.779+0000 7f3145c99700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31400f66e0 0x7f314006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.779+0000 7f3145c99700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31400f7980 0x7f3140069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.779+0000 7f3145c99700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314006a380 con 0x7f31400f66e0 2024-02-15T18:32:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.779+0000 7f3145c99700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314006a4c0 con 0x7f31400f7980 2024-02-15T18:32:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.779+0000 7f313f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31400f66e0 0x7f314006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.779+0000 7f313f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31400f66e0 0x7f314006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35146/0 (socket says 172.21.15.38:35146) 2024-02-15T18:32:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.779+0000 7f313f7fe700 1 -- 172.21.15.38:0/1279900213 learned_addr learned my addr 172.21.15.38:0/1279900213 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:32:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.779+0000 7f313f7fe700 1 -- 172.21.15.38:0/1279900213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31400f7980 msgr2=0x7f3140069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.779+0000 7f313effd700 1 --2- 172.21.15.38:0/1279900213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31400f7980 0x7f3140069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.779+0000 7f313f7fe700 1 --2- 172.21.15.38:0/1279900213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31400f7980 0x7f3140069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.779+0000 7f313f7fe700 1 -- 172.21.15.38:0/1279900213 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3134005040 con 0x7f31400f66e0 2024-02-15T18:32:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.779+0000 7f313effd700 1 --2- 172.21.15.38:0/1279900213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31400f7980 0x7f3140069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:32:38.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.780+0000 7f313f7fe700 1 --2- 172.21.15.38:0/1279900213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31400f66e0 0x7f314006b9a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f3134009f50 tx=0x7f3134009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:38.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.780+0000 7f3127fff700 1 -- 172.21.15.38:0/1279900213 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3134018070 con 0x7f31400f66e0 2024-02-15T18:32:38.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.780+0000 7f3127fff700 1 -- 172.21.15.38:0/1279900213 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3134007d70 con 0x7f31400f66e0 2024-02-15T18:32:38.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.780+0000 7f3145c99700 1 -- 172.21.15.38:0/1279900213 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f314006a740 con 0x7f31400f66e0 2024-02-15T18:32:38.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.780+0000 7f3127fff700 1 -- 172.21.15.38:0/1279900213 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f313401c620 con 0x7f31400f66e0 2024-02-15T18:32:38.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.780+0000 7f3145c99700 1 -- 172.21.15.38:0/1279900213 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f314006aca0 con 0x7f31400f66e0 2024-02-15T18:32:38.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.781+0000 7f3145c99700 1 -- 172.21.15.38:0/1279900213 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3140045ad0 con 0x7f31400f66e0 2024-02-15T18:32:38.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.781+0000 7f3127fff700 1 -- 172.21.15.38:0/1279900213 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f313401c8e0 con 0x7f31400f66e0 2024-02-15T18:32:38.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.782+0000 7f3127fff700 1 --2- 172.21.15.38:0/1279900213 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f312805c2b0 0x7f312805e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:38.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.782+0000 7f3127fff700 1 -- 172.21.15.38:0/1279900213 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f3134077a10 con 0x7f31400f66e0 2024-02-15T18:32:38.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.782+0000 7f313effd700 1 --2- 172.21.15.38:0/1279900213 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f312805c2b0 0x7f312805e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:38.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.782+0000 7f313effd700 1 --2- 172.21.15.38:0/1279900213 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f312805c2b0 0x7f312805e770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f3130004ac0 tx=0x7f3130009040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:38.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.786+0000 7f3127fff700 1 -- 172.21.15.38:0/1279900213 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f313404aa10 con 0x7f31400f66e0 2024-02-15T18:32:38.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.969+0000 7f3145c99700 1 -- 172.21.15.38:0/1279900213 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f31400ff160 con 0x7f312805c2b0 2024-02-15T18:32:38.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.970+0000 7f3127fff700 1 -- 172.21.15.38:0/1279900213 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+126 (secure 0 0 0) 0x7f3130004700 con 0x7f312805c2b0 2024-02-15T18:32:38.962 INFO:teuthology.orchestra.run.smithi038.stdout:HOST ADDR LABELS STATUS 2024-02-15T18:32:38.962 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038 172.21.15.38 2024-02-15T18:32:38.962 INFO:teuthology.orchestra.run.smithi038.stdout:smithi121 smithi121 2024-02-15T18:32:38.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.972+0000 7f3145c99700 1 -- 172.21.15.38:0/1279900213 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f312805c2b0 msgr2=0x7f312805e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:38.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.972+0000 7f3145c99700 1 --2- 172.21.15.38:0/1279900213 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f312805c2b0 0x7f312805e770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f3130004ac0 tx=0x7f3130009040).stop 2024-02-15T18:32:38.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.972+0000 7f3145c99700 1 -- 172.21.15.38:0/1279900213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31400f66e0 msgr2=0x7f314006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:38.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.972+0000 7f3145c99700 1 --2- 172.21.15.38:0/1279900213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31400f66e0 0x7f314006b9a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f3134009f50 tx=0x7f3134009b30).stop 2024-02-15T18:32:38.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.972+0000 7f3145c99700 1 -- 172.21.15.38:0/1279900213 shutdown_connections 2024-02-15T18:32:38.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.972+0000 7f3145c99700 1 --2- 172.21.15.38:0/1279900213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31400f66e0 0x7f314006b9a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:38.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.972+0000 7f3145c99700 1 --2- 172.21.15.38:0/1279900213 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f312805c2b0 0x7f312805e770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:38.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.972+0000 7f3145c99700 1 --2- 172.21.15.38:0/1279900213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31400f7980 0x7f3140069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:38.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.972+0000 7f3145c99700 1 -- 172.21.15.38:0/1279900213 >> 172.21.15.38:0/1279900213 conn(0x7f31400f1e40 msgr2=0x7f31400fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:38.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.972+0000 7f3145c99700 1 -- 172.21.15.38:0/1279900213 shutdown_connections 2024-02-15T18:32:38.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:38.972+0000 7f3145c99700 1 -- 172.21.15.38:0/1279900213 wait complete. 2024-02-15T18:32:39.681 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-15T18:32:40.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:38.274814+0000 mgr.smithi038.khesbl (mgr.14162) 240 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:40.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:38.274814+0000 mgr.smithi038.khesbl (mgr.14162) 240 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:41.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:38.970686+0000 mgr.smithi038.khesbl (mgr.14162) 241 : audit [DBG] from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:41.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:40.358471+0000 mon.smithi038 (mon.0) 454 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:32:41.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:40.359233+0000 mon.smithi038 (mon.0) 455 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:32:41.248 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:38.970686+0000 mgr.smithi038.khesbl (mgr.14162) 241 : audit [DBG] from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:41.248 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:40.358471+0000 mon.smithi038 (mon.0) 454 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:32:41.248 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:40.359233+0000 mon.smithi038 (mon.0) 455 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:32:41.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.851+0000 7f26ead6d700 1 -- 172.21.15.38:0/142686301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26e40f55c0 msgr2=0x7f26e40f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:41.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.851+0000 7f26ead6d700 1 --2- 172.21.15.38:0/142686301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26e40f55c0 0x7f26e40f59e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f26d8004660 tx=0x7f26d8009b30).stop 2024-02-15T18:32:41.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.851+0000 7f26ead6d700 1 -- 172.21.15.38:0/142686301 shutdown_connections 2024-02-15T18:32:41.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.851+0000 7f26ead6d700 1 --2- 172.21.15.38:0/142686301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26e40f6860 0x7f26e40f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:41.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.851+0000 7f26ead6d700 1 --2- 172.21.15.38:0/142686301 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26e40f55c0 0x7f26e40f59e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:41.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.851+0000 7f26ead6d700 1 -- 172.21.15.38:0/142686301 >> 172.21.15.38:0/142686301 conn(0x7f26e40f0d40 msgr2=0x7f26e40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:41.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.851+0000 7f26ead6d700 1 -- 172.21.15.38:0/142686301 shutdown_connections 2024-02-15T18:32:41.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.851+0000 7f26ead6d700 1 -- 172.21.15.38:0/142686301 wait complete. 2024-02-15T18:32:41.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.852+0000 7f26ead6d700 1 Processor -- start 2024-02-15T18:32:41.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.852+0000 7f26ead6d700 1 -- start start 2024-02-15T18:32:41.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.852+0000 7f26ead6d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26e40f55c0 0x7f26e4107080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:41.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.852+0000 7f26ead6d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26e40f6860 0x7f26e4107570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:41.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.852+0000 7f26ead6d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e4107b20 con 0x7f26e40f55c0 2024-02-15T18:32:41.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.852+0000 7f26ead6d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e4107c60 con 0x7f26e40f6860 2024-02-15T18:32:41.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.853+0000 7f26e8b09700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26e40f55c0 0x7f26e4107080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:41.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.853+0000 7f26e8b09700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26e40f55c0 0x7f26e4107080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35176/0 (socket says 172.21.15.38:35176) 2024-02-15T18:32:41.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.853+0000 7f26e8b09700 1 -- 172.21.15.38:0/1994393512 learned_addr learned my addr 172.21.15.38:0/1994393512 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:32:41.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.853+0000 7f26e3fff700 1 --2- 172.21.15.38:0/1994393512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26e40f6860 0x7f26e4107570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:41.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.853+0000 7f26e8b09700 1 -- 172.21.15.38:0/1994393512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26e40f6860 msgr2=0x7f26e4107570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:41.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.853+0000 7f26e8b09700 1 --2- 172.21.15.38:0/1994393512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26e40f6860 0x7f26e4107570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:41.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.853+0000 7f26e8b09700 1 -- 172.21.15.38:0/1994393512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d8005040 con 0x7f26e40f55c0 2024-02-15T18:32:41.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.853+0000 7f26e8b09700 1 --2- 172.21.15.38:0/1994393512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26e40f55c0 0x7f26e4107080 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f26d8004e30 tx=0x7f26d8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:41.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.853+0000 7f26e17fa700 1 -- 172.21.15.38:0/1994393512 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26d8018070 con 0x7f26e40f55c0 2024-02-15T18:32:41.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.853+0000 7f26ead6d700 1 -- 172.21.15.38:0/1994393512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e41085a0 con 0x7f26e40f55c0 2024-02-15T18:32:41.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.853+0000 7f26e17fa700 1 -- 172.21.15.38:0/1994393512 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26d8007ba0 con 0x7f26e40f55c0 2024-02-15T18:32:41.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.853+0000 7f26e17fa700 1 -- 172.21.15.38:0/1994393512 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26d801c3c0 con 0x7f26e40f55c0 2024-02-15T18:32:41.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.853+0000 7f26ead6d700 1 -- 172.21.15.38:0/1994393512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e4108b00 con 0x7f26e40f55c0 2024-02-15T18:32:41.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.854+0000 7f26ead6d700 1 -- 172.21.15.38:0/1994393512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e4045ad0 con 0x7f26e40f55c0 2024-02-15T18:32:41.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.855+0000 7f26e17fa700 1 -- 172.21.15.38:0/1994393512 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f26d801d420 con 0x7f26e40f55c0 2024-02-15T18:32:41.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.855+0000 7f26e17fa700 1 --2- 172.21.15.38:0/1994393512 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f26d405c2b0 0x7f26d405e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:41.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.855+0000 7f26e17fa700 1 -- 172.21.15.38:0/1994393512 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f26d8077740 con 0x7f26e40f55c0 2024-02-15T18:32:41.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.855+0000 7f26e3fff700 1 --2- 172.21.15.38:0/1994393512 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f26d405c2b0 0x7f26d405e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:41.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.856+0000 7f26e3fff700 1 --2- 172.21.15.38:0/1994393512 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f26d405c2b0 0x7f26d405e770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f26d0004c40 tx=0x7f26d000b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:41.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:41.859+0000 7f26e17fa700 1 -- 172.21.15.38:0/1994393512 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f26d804a740 con 0x7f26e40f55c0 2024-02-15T18:32:42.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:42.043+0000 7f26ead6d700 1 -- 172.21.15.38:0/1994393512 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f26e40faca0 con 0x7f26d405c2b0 2024-02-15T18:32:42.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:42.044+0000 7f26e17fa700 1 -- 172.21.15.38:0/1994393512 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+279 (secure 0 0 0) 0x7f26e40faca0 con 0x7f26d405c2b0 2024-02-15T18:32:42.036 INFO:teuthology.orchestra.run.smithi038.stdout:Hostname Path Type Serial Size Health Ident Fault Available 2024-02-15T18:32:42.036 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038 /dev/nvme0n1 ssd BTLE736103BB4P0KGN 4000G Unknown N/A N/A No 2024-02-15T18:32:42.036 INFO:teuthology.orchestra.run.smithi038.stdout:smithi121 /dev/nvme0n1 ssd PHFT620400ZF400BGN 400G Unknown N/A N/A No 2024-02-15T18:32:42.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:42.046+0000 7f26ead6d700 1 -- 172.21.15.38:0/1994393512 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f26d405c2b0 msgr2=0x7f26d405e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:42.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:42.046+0000 7f26ead6d700 1 --2- 172.21.15.38:0/1994393512 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f26d405c2b0 0x7f26d405e770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f26d0004c40 tx=0x7f26d000b040).stop 2024-02-15T18:32:42.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:42.046+0000 7f26ead6d700 1 -- 172.21.15.38:0/1994393512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26e40f55c0 msgr2=0x7f26e4107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:42.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:42.046+0000 7f26ead6d700 1 --2- 172.21.15.38:0/1994393512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26e40f55c0 0x7f26e4107080 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f26d8004e30 tx=0x7f26d8009b30).stop 2024-02-15T18:32:42.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:42.046+0000 7f26ead6d700 1 -- 172.21.15.38:0/1994393512 shutdown_connections 2024-02-15T18:32:42.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:42.046+0000 7f26ead6d700 1 --2- 172.21.15.38:0/1994393512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26e40f55c0 0x7f26e4107080 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:42.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:42.046+0000 7f26ead6d700 1 --2- 172.21.15.38:0/1994393512 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f26d405c2b0 0x7f26d405e770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:42.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:42.046+0000 7f26ead6d700 1 --2- 172.21.15.38:0/1994393512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26e40f6860 0x7f26e4107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:42.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:42.046+0000 7f26ead6d700 1 -- 172.21.15.38:0/1994393512 >> 172.21.15.38:0/1994393512 conn(0x7f26e40f0d40 msgr2=0x7f26e405fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:42.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:42.046+0000 7f26ead6d700 1 -- 172.21.15.38:0/1994393512 shutdown_connections 2024-02-15T18:32:42.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:42.046+0000 7f26ead6d700 1 -- 172.21.15.38:0/1994393512 wait complete. 2024-02-15T18:32:42.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:40.275144+0000 mgr.smithi038.khesbl (mgr.14162) 242 : cluster [DBG] pgmap v207: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:42.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:40.275144+0000 mgr.smithi038.khesbl (mgr.14162) 242 : cluster [DBG] pgmap v207: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:42.855 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-15T18:32:42.867 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-02-15T18:32:42.867 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-02-15T18:32:44.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:42.044395+0000 mgr.smithi038.khesbl (mgr.14162) 243 : audit [DBG] from='client.14428 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:44.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:42.275518+0000 mgr.smithi038.khesbl (mgr.14162) 244 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:44.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:42.044395+0000 mgr.smithi038.khesbl (mgr.14162) 243 : audit [DBG] from='client.14428 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:44.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:42.275518+0000 mgr.smithi038.khesbl (mgr.14162) 244 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:44.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.880+0000 7f23f53c5700 1 -- 172.21.15.38:0/2645051738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23f00f77d0 msgr2=0x7f23f00f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:44.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.880+0000 7f23f53c5700 1 --2- 172.21.15.38:0/2645051738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23f00f77d0 0x7f23f00f7c30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f23e4004660 tx=0x7f23e4009b30).stop 2024-02-15T18:32:44.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.880+0000 7f23f53c5700 1 -- 172.21.15.38:0/2645051738 shutdown_connections 2024-02-15T18:32:44.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.880+0000 7f23f53c5700 1 --2- 172.21.15.38:0/2645051738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23f00f77d0 0x7f23f00f7c30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:44.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.880+0000 7f23f53c5700 1 --2- 172.21.15.38:0/2645051738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23f00f6530 0x7f23f00f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.881+0000 7f23f53c5700 1 -- 172.21.15.38:0/2645051738 >> 172.21.15.38:0/2645051738 conn(0x7f23f00f1cd0 msgr2=0x7f23f00f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.881+0000 7f23f53c5700 1 -- 172.21.15.38:0/2645051738 shutdown_connections 2024-02-15T18:32:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.881+0000 7f23f53c5700 1 -- 172.21.15.38:0/2645051738 wait complete. 2024-02-15T18:32:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.881+0000 7f23f53c5700 1 Processor -- start 2024-02-15T18:32:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.881+0000 7f23f53c5700 1 -- start start 2024-02-15T18:32:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.881+0000 7f23f53c5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23f00f6530 0x7f23f0106e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.881+0000 7f23f53c5700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23f00f77d0 0x7f23f0107330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.881+0000 7f23f53c5700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f0107930 con 0x7f23f00f6530 2024-02-15T18:32:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.881+0000 7f23f53c5700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f0108140 con 0x7f23f00f77d0 2024-02-15T18:32:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.882+0000 7f23effff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23f00f6530 0x7f23f0106e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.882+0000 7f23effff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23f00f6530 0x7f23f0106e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38798/0 (socket says 172.21.15.38:38798) 2024-02-15T18:32:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.882+0000 7f23effff700 1 -- 172.21.15.38:0/3019527826 learned_addr learned my addr 172.21.15.38:0/3019527826 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:32:44.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.882+0000 7f23ef7fe700 1 --2- 172.21.15.38:0/3019527826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23f00f77d0 0x7f23f0107330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:44.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.882+0000 7f23effff700 1 -- 172.21.15.38:0/3019527826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23f00f77d0 msgr2=0x7f23f0107330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:44.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.882+0000 7f23effff700 1 --2- 172.21.15.38:0/3019527826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23f00f77d0 0x7f23f0107330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:44.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.882+0000 7f23effff700 1 -- 172.21.15.38:0/3019527826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e4005040 con 0x7f23f00f6530 2024-02-15T18:32:44.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.882+0000 7f23ef7fe700 1 --2- 172.21.15.38:0/3019527826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23f00f77d0 0x7f23f0107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:32:44.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.882+0000 7f23effff700 1 --2- 172.21.15.38:0/3019527826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23f00f6530 0x7f23f0106e40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f23e0002700 tx=0x7f23e0009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:44.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.882+0000 7f23ecff9700 1 -- 172.21.15.38:0/3019527826 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23e0009320 con 0x7f23f00f6530 2024-02-15T18:32:44.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.882+0000 7f23ecff9700 1 -- 172.21.15.38:0/3019527826 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23e000f040 con 0x7f23f00f6530 2024-02-15T18:32:44.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.882+0000 7f23f53c5700 1 -- 172.21.15.38:0/3019527826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f01083c0 con 0x7f23f00f6530 2024-02-15T18:32:44.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.882+0000 7f23ecff9700 1 -- 172.21.15.38:0/3019527826 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23e0014940 con 0x7f23f00f6530 2024-02-15T18:32:44.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.882+0000 7f23f53c5700 1 -- 172.21.15.38:0/3019527826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f0108980 con 0x7f23f00f6530 2024-02-15T18:32:44.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.883+0000 7f23f53c5700 1 -- 172.21.15.38:0/3019527826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23f0059100 con 0x7f23f00f6530 2024-02-15T18:32:44.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.883+0000 7f23ecff9700 1 -- 172.21.15.38:0/3019527826 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f23e0014aa0 con 0x7f23f00f6530 2024-02-15T18:32:44.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.884+0000 7f23ecff9700 1 --2- 172.21.15.38:0/3019527826 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f23d805c300 0x7f23d805e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:44.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.884+0000 7f23ecff9700 1 -- 172.21.15.38:0/3019527826 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f23e0075d50 con 0x7f23f00f6530 2024-02-15T18:32:44.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.884+0000 7f23ef7fe700 1 --2- 172.21.15.38:0/3019527826 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f23d805c300 0x7f23d805e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:44.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.884+0000 7f23ef7fe700 1 --2- 172.21.15.38:0/3019527826 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f23d805c300 0x7f23d805e7c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f23e4005010 tx=0x7f23e4009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:44.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:44.888+0000 7f23ecff9700 1 -- 172.21.15.38:0/3019527826 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f23e0048dd0 con 0x7f23f00f6530 2024-02-15T18:32:45.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:45.074+0000 7f23f53c5700 1 -- 172.21.15.38:0/3019527826 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f23f00fef90 con 0x7f23d805c300 2024-02-15T18:32:46.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:44.275964+0000 mgr.smithi038.khesbl (mgr.14162) 245 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:46.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:45.075916+0000 mon.smithi038 (mon.0) 456 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-02-15T18:32:46.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:44.275964+0000 mgr.smithi038.khesbl (mgr.14162) 245 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:46.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:45.075916+0000 mon.smithi038 (mon.0) 456 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-02-15T18:32:46.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:46.897+0000 7f23ecff9700 1 -- 172.21.15.38:0/3019527826 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f23f00fef90 con 0x7f23d805c300 2024-02-15T18:32:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:46.899+0000 7f23f53c5700 1 -- 172.21.15.38:0/3019527826 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f23d805c300 msgr2=0x7f23d805e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:46.899+0000 7f23f53c5700 1 --2- 172.21.15.38:0/3019527826 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f23d805c300 0x7f23d805e7c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f23e4005010 tx=0x7f23e4009b30).stop 2024-02-15T18:32:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:46.899+0000 7f23f53c5700 1 -- 172.21.15.38:0/3019527826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23f00f6530 msgr2=0x7f23f0106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:46.899+0000 7f23f53c5700 1 --2- 172.21.15.38:0/3019527826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23f00f6530 0x7f23f0106e40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f23e0002700 tx=0x7f23e0009d80).stop 2024-02-15T18:32:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:46.899+0000 7f23f53c5700 1 -- 172.21.15.38:0/3019527826 shutdown_connections 2024-02-15T18:32:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:46.899+0000 7f23f53c5700 1 --2- 172.21.15.38:0/3019527826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23f00f6530 0x7f23f0106e40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:46.899+0000 7f23f53c5700 1 --2- 172.21.15.38:0/3019527826 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f23d805c300 0x7f23d805e7c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:46.899+0000 7f23f53c5700 1 --2- 172.21.15.38:0/3019527826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23f00f77d0 0x7f23f0107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:46.899+0000 7f23f53c5700 1 -- 172.21.15.38:0/3019527826 >> 172.21.15.38:0/3019527826 conn(0x7f23f00f1cd0 msgr2=0x7f23f00fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:46.900+0000 7f23f53c5700 1 -- 172.21.15.38:0/3019527826 shutdown_connections 2024-02-15T18:32:46.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:46.900+0000 7f23f53c5700 1 -- 172.21.15.38:0/3019527826 wait complete. 2024-02-15T18:32:47.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:46 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:45.075408+0000 mgr.smithi038.khesbl (mgr.14162) 246 : audit [DBG] from='client.14432 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:47.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:46 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:45.852041+0000 mon.smithi038 (mon.0) 457 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-02-15T18:32:47.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:46 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:45.852090+0000 mon.smithi038 (mon.0) 458 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-02-15T18:32:47.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:46 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:45.853186+0000 mon.smithi038 (mon.0) 459 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-02-15T18:32:47.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:45.075408+0000 mgr.smithi038.khesbl (mgr.14162) 246 : audit [DBG] from='client.14432 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:47.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:45.852041+0000 mon.smithi038 (mon.0) 457 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-02-15T18:32:47.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:45.852090+0000 mon.smithi038 (mon.0) 458 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-02-15T18:32:47.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:45.853186+0000 mon.smithi038 (mon.0) 459 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-02-15T18:32:47.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:32:46.871+0000 7f5ad13b1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-15T18:32:47.651 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-15T18:32:48.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:46.276283+0000 mgr.smithi038.khesbl (mgr.14162) 247 : cluster [DBG] pgmap v211: 33 pgs: 32 unknown, 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:48.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:46.864874+0000 mon.smithi038 (mon.0) 460 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-02-15T18:32:48.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:46.865091+0000 mon.smithi038 (mon.0) 461 : cluster [DBG] osdmap e40: 6 total, 6 up, 6 in 2024-02-15T18:32:48.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:46.870740+0000 mon.smithi038 (mon.0) 462 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-02-15T18:32:48.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:46.871630+0000 mon.smithi038 (mon.0) 463 : cluster [ERR] Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-15T18:32:48.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:46.871686+0000 mon.smithi038 (mon.0) 464 : cluster [WRN] Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-15T18:32:48.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:46.884095+0000 mon.smithi038 (mon.0) 465 : cluster [DBG] osdmap e41: 6 total, 6 up, 6 in 2024-02-15T18:32:48.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:46.884329+0000 mon.smithi038 (mon.0) 466 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-02-15T18:32:48.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:46.884420+0000 mon.smithi038 (mon.0) 467 : cluster [DBG] fsmap cephfs:0 2024-02-15T18:32:48.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:32:46.889619+0000 mgr.smithi038.khesbl (mgr.14162) 248 : cephadm [INF] Saving service mds.cephfs spec with placement count:4 2024-02-15T18:32:48.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:46.897052+0000 mon.smithi038 (mon.0) 468 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:32:48.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:46.900425+0000 mon.smithi038 (mon.0) 469 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:48.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:46.907709+0000 mon.smithi038 (mon.0) 470 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:32:48.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:46.914211+0000 mon.smithi038 (mon.0) 471 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:32:48.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:46.915145+0000 mon.smithi038 (mon.0) 472 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi121.mvumlh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:32:48.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:46.918550+0000 mon.smithi038 (mon.0) 473 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi121.mvumlh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-15T18:32:48.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:46.919200+0000 mon.smithi038 (mon.0) 474 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:48.088 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:46.276283+0000 mgr.smithi038.khesbl (mgr.14162) 247 : cluster [DBG] pgmap v211: 33 pgs: 32 unknown, 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:48.089 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:46.864874+0000 mon.smithi038 (mon.0) 460 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-02-15T18:32:48.089 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:46.865091+0000 mon.smithi038 (mon.0) 461 : cluster [DBG] osdmap e40: 6 total, 6 up, 6 in 2024-02-15T18:32:48.089 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:46.870740+0000 mon.smithi038 (mon.0) 462 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-02-15T18:32:48.089 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:46.871630+0000 mon.smithi038 (mon.0) 463 : cluster [ERR] Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-15T18:32:48.089 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:46.871686+0000 mon.smithi038 (mon.0) 464 : cluster [WRN] Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-15T18:32:48.089 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:46.884095+0000 mon.smithi038 (mon.0) 465 : cluster [DBG] osdmap e41: 6 total, 6 up, 6 in 2024-02-15T18:32:48.089 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:46.884329+0000 mon.smithi038 (mon.0) 466 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-02-15T18:32:48.089 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:46.884420+0000 mon.smithi038 (mon.0) 467 : cluster [DBG] fsmap cephfs:0 2024-02-15T18:32:48.089 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:32:46.889619+0000 mgr.smithi038.khesbl (mgr.14162) 248 : cephadm [INF] Saving service mds.cephfs spec with placement count:4 2024-02-15T18:32:48.089 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:46.897052+0000 mon.smithi038 (mon.0) 468 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:32:48.089 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:46.900425+0000 mon.smithi038 (mon.0) 469 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:48.089 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:46.907709+0000 mon.smithi038 (mon.0) 470 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:32:48.089 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:46.914211+0000 mon.smithi038 (mon.0) 471 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:32:48.090 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:46.915145+0000 mon.smithi038 (mon.0) 472 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi121.mvumlh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:32:48.090 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:46.918550+0000 mon.smithi038 (mon.0) 473 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi121.mvumlh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-15T18:32:48.090 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:46.919200+0000 mon.smithi038 (mon.0) 474 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:49.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:32:46.919814+0000 mgr.smithi038.khesbl (mgr.14162) 249 : cephadm [INF] Deploying daemon mds.cephfs.smithi121.mvumlh on smithi121 2024-02-15T18:32:49.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:47.886929+0000 mon.smithi038 (mon.0) 475 : cluster [DBG] osdmap e42: 6 total, 6 up, 6 in 2024-02-15T18:32:49.060 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:32:46.919814+0000 mgr.smithi038.khesbl (mgr.14162) 249 : cephadm [INF] Deploying daemon mds.cephfs.smithi121.mvumlh on smithi121 2024-02-15T18:32:49.060 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:47.886929+0000 mon.smithi038 (mon.0) 475 : cluster [DBG] osdmap e42: 6 total, 6 up, 6 in 2024-02-15T18:32:49.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.603+0000 7f3203285700 1 -- 172.21.15.38:0/2089371054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31fc0f9690 msgr2=0x7f31fc0fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:49.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.603+0000 7f3203285700 1 --2- 172.21.15.38:0/2089371054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31fc0f9690 0x7f31fc0fbae0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f31ec004660 tx=0x7f31ec009b30).stop 2024-02-15T18:32:49.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.603+0000 7f3203285700 1 -- 172.21.15.38:0/2089371054 shutdown_connections 2024-02-15T18:32:49.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.603+0000 7f3203285700 1 --2- 172.21.15.38:0/2089371054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31fc0f9690 0x7f31fc0fbae0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:49.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.603+0000 7f3203285700 1 --2- 172.21.15.38:0/2089371054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31fc060310 0x7f31fc0f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:49.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.603+0000 7f3203285700 1 -- 172.21.15.38:0/2089371054 >> 172.21.15.38:0/2089371054 conn(0x7f31fc0f0b90 msgr2=0x7f31fc0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:49.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.604+0000 7f3203285700 1 -- 172.21.15.38:0/2089371054 shutdown_connections 2024-02-15T18:32:49.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.604+0000 7f3203285700 1 -- 172.21.15.38:0/2089371054 wait complete. 2024-02-15T18:32:49.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.604+0000 7f3203285700 1 Processor -- start 2024-02-15T18:32:49.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.604+0000 7f3203285700 1 -- start start 2024-02-15T18:32:49.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.604+0000 7f3203285700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31fc060310 0x7f31fc0f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:49.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.604+0000 7f3203285700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31fc0f9690 0x7f31fc0f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:49.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.604+0000 7f3203285700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc0f5570 con 0x7f31fc060310 2024-02-15T18:32:49.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.604+0000 7f3203285700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31fc0f56b0 con 0x7f31fc0f9690 2024-02-15T18:32:49.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.604+0000 7f3202283700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31fc060310 0x7f31fc0f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:49.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.604+0000 7f3202283700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31fc060310 0x7f31fc0f6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38822/0 (socket says 172.21.15.38:38822) 2024-02-15T18:32:49.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.604+0000 7f3202283700 1 -- 172.21.15.38:0/2525076134 learned_addr learned my addr 172.21.15.38:0/2525076134 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:32:49.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.605+0000 7f3201a82700 1 --2- 172.21.15.38:0/2525076134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31fc0f9690 0x7f31fc0f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:49.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.605+0000 7f3202283700 1 -- 172.21.15.38:0/2525076134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31fc0f9690 msgr2=0x7f31fc0f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:49.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.605+0000 7f3202283700 1 --2- 172.21.15.38:0/2525076134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31fc0f9690 0x7f31fc0f5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:49.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.605+0000 7f3202283700 1 -- 172.21.15.38:0/2525076134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ec005040 con 0x7f31fc060310 2024-02-15T18:32:49.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.605+0000 7f3202283700 1 --2- 172.21.15.38:0/2525076134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31fc060310 0x7f31fc0f6b90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f31f80026d0 tx=0x7f31f8009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:49.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.605+0000 7f31f2ffd700 1 -- 172.21.15.38:0/2525076134 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31f80079c0 con 0x7f31fc060310 2024-02-15T18:32:49.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.605+0000 7f3203285700 1 -- 172.21.15.38:0/2525076134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31fc0f5990 con 0x7f31fc060310 2024-02-15T18:32:49.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.605+0000 7f3203285700 1 -- 172.21.15.38:0/2525076134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31fc10ee90 con 0x7f31fc060310 2024-02-15T18:32:49.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.605+0000 7f31f2ffd700 1 -- 172.21.15.38:0/2525076134 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f31f800f040 con 0x7f31fc060310 2024-02-15T18:32:49.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.605+0000 7f31f2ffd700 1 -- 172.21.15.38:0/2525076134 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31f801d930 con 0x7f31fc060310 2024-02-15T18:32:49.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.606+0000 7f3203285700 1 -- 172.21.15.38:0/2525076134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e4004fa0 con 0x7f31fc060310 2024-02-15T18:32:49.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.607+0000 7f31f2ffd700 1 -- 172.21.15.38:0/2525076134 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f31f8009420 con 0x7f31fc060310 2024-02-15T18:32:49.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.607+0000 7f31f2ffd700 1 --2- 172.21.15.38:0/2525076134 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f31e805c300 0x7f31e805e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:49.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.607+0000 7f31f2ffd700 1 -- 172.21.15.38:0/2525076134 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f31f8075e70 con 0x7f31fc060310 2024-02-15T18:32:49.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.610+0000 7f3201a82700 1 --2- 172.21.15.38:0/2525076134 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f31e805c300 0x7f31e805e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:49.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.610+0000 7f3201a82700 1 --2- 172.21.15.38:0/2525076134 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f31e805c300 0x7f31e805e7c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f31ec004630 tx=0x7f31ec009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:49.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.612+0000 7f31f2ffd700 1 -- 172.21.15.38:0/2525076134 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f31f8048b30 con 0x7f31fc060310 2024-02-15T18:32:49.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.818+0000 7f3203285700 1 -- 172.21.15.38:0/2525076134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f31e4005e80 con 0x7f31fc060310 2024-02-15T18:32:49.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.818+0000 7f31f2ffd700 1 -- 172.21.15.38:0/2525076134 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1003 (secure 0 0 0) 0x7f31f804c150 con 0x7f31fc060310 2024-02-15T18:32:49.812 INFO:teuthology.orchestra.run.smithi038.stdout:e2 2024-02-15T18:32:49.812 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:32:49.812 INFO:teuthology.orchestra.run.smithi038.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:32:49.812 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-02-15T18:32:49.812 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:32:49.812 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:32:49.812 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-02-15T18:32:49.812 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 2 2024-02-15T18:32:49.813 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 2024-02-15T18:32:49.813 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:32:49.813 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-02-15T18:32:46.871523+0000 2024-02-15T18:32:49.813 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-02-15T18:32:49.813 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-02-15T18:32:49.813 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-02-15T18:32:49.813 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-02-15T18:32:49.813 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-02-15T18:32:49.813 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-02-15T18:32:49.813 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-02-15T18:32:49.813 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-02-15T18:32:49.813 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:32:49.813 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 1 2024-02-15T18:32:49.813 INFO:teuthology.orchestra.run.smithi038.stdout:in 2024-02-15T18:32:49.813 INFO:teuthology.orchestra.run.smithi038.stdout:up {} 2024-02-15T18:32:49.814 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-02-15T18:32:49.814 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-02-15T18:32:49.814 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-02-15T18:32:49.814 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-02-15T18:32:49.814 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-02-15T18:32:49.814 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-02-15T18:32:49.814 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-02-15T18:32:49.814 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 0 2024-02-15T18:32:49.814 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:32:49.814 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:32:49.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.820+0000 7f3203285700 1 -- 172.21.15.38:0/2525076134 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f31e805c300 msgr2=0x7f31e805e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:49.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.820+0000 7f3203285700 1 --2- 172.21.15.38:0/2525076134 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f31e805c300 0x7f31e805e7c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f31ec004630 tx=0x7f31ec009b30).stop 2024-02-15T18:32:49.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.820+0000 7f3203285700 1 -- 172.21.15.38:0/2525076134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31fc060310 msgr2=0x7f31fc0f6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:49.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.820+0000 7f3203285700 1 --2- 172.21.15.38:0/2525076134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31fc060310 0x7f31fc0f6b90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f31f80026d0 tx=0x7f31f8009de0).stop 2024-02-15T18:32:49.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.820+0000 7f3203285700 1 -- 172.21.15.38:0/2525076134 shutdown_connections 2024-02-15T18:32:49.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.820+0000 7f3203285700 1 --2- 172.21.15.38:0/2525076134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31fc060310 0x7f31fc0f6b90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:49.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.820+0000 7f3203285700 1 --2- 172.21.15.38:0/2525076134 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f31e805c300 0x7f31e805e7c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:49.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.820+0000 7f3203285700 1 --2- 172.21.15.38:0/2525076134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31fc0f9690 0x7f31fc0f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:49.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.820+0000 7f3203285700 1 -- 172.21.15.38:0/2525076134 >> 172.21.15.38:0/2525076134 conn(0x7f31fc0f0b90 msgr2=0x7f31fc0f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:49.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.820+0000 7f3203285700 1 -- 172.21.15.38:0/2525076134 shutdown_connections 2024-02-15T18:32:49.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:49.820+0000 7f3203285700 1 -- 172.21.15.38:0/2525076134 wait complete. 2024-02-15T18:32:49.816 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 2 2024-02-15T18:32:50.170 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:48.276706+0000 mgr.smithi038.khesbl (mgr.14162) 250 : cluster [DBG] pgmap v215: 65 pgs: 3 creating+peering, 4 active+clean, 58 unknown; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:50.170 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:48.892163+0000 mon.smithi038 (mon.0) 476 : cluster [DBG] osdmap e43: 6 total, 6 up, 6 in 2024-02-15T18:32:50.170 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:49.420853+0000 mon.smithi038 (mon.0) 477 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:32:50.170 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:49.421758+0000 mon.smithi038 (mon.0) 478 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.hxoggl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:32:50.170 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:49.424953+0000 mon.smithi038 (mon.0) 479 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.hxoggl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-15T18:32:50.170 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:49.425855+0000 mon.smithi038 (mon.0) 480 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:50.170 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:49.820146+0000 mon.smithi038 (mon.0) 481 : audit [DBG] from='client.? 172.21.15.38:0/2525076134' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:32:50.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:48.276706+0000 mgr.smithi038.khesbl (mgr.14162) 250 : cluster [DBG] pgmap v215: 65 pgs: 3 creating+peering, 4 active+clean, 58 unknown; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:50.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:48.892163+0000 mon.smithi038 (mon.0) 476 : cluster [DBG] osdmap e43: 6 total, 6 up, 6 in 2024-02-15T18:32:50.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:49.420853+0000 mon.smithi038 (mon.0) 477 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:32:50.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:49.421758+0000 mon.smithi038 (mon.0) 478 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.hxoggl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:32:50.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:49.424953+0000 mon.smithi038 (mon.0) 479 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.hxoggl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-15T18:32:50.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:49.425855+0000 mon.smithi038 (mon.0) 480 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:50.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:49.820146+0000 mon.smithi038 (mon.0) 481 : audit [DBG] from='client.? 172.21.15.38:0/2525076134' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:32:50.490 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-15T18:32:50.501 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-02-15T18:32:50.501 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-02-15T18:32:51.263 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:32:49.426744+0000 mgr.smithi038.khesbl (mgr.14162) 251 : cephadm [INF] Deploying daemon mds.cephfs.smithi038.hxoggl on smithi038 2024-02-15T18:32:51.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:32:49.426744+0000 mgr.smithi038.khesbl (mgr.14162) 251 : cephadm [INF] Deploying daemon mds.cephfs.smithi038.hxoggl on smithi038 2024-02-15T18:32:52.147 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:50.277191+0000 mgr.smithi038.khesbl (mgr.14162) 252 : cluster [DBG] pgmap v217: 65 pgs: 3 creating+peering, 34 active+clean, 28 unknown; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:52.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:50.277191+0000 mgr.smithi038.khesbl (mgr.14162) 252 : cluster [DBG] pgmap v217: 65 pgs: 3 creating+peering, 34 active+clean, 28 unknown; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:53.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.872+0000 7f53f159c700 1 -- 172.21.15.38:0/927941695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53ec060270 msgr2=0x7f53ec0f9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:53.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.872+0000 7f53f159c700 1 --2- 172.21.15.38:0/927941695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53ec060270 0x7f53ec0f9340 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f53e0004660 tx=0x7f53e0009b30).stop 2024-02-15T18:32:53.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.872+0000 7f53f159c700 1 -- 172.21.15.38:0/927941695 shutdown_connections 2024-02-15T18:32:53.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.872+0000 7f53f159c700 1 --2- 172.21.15.38:0/927941695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53ec0f9830 0x7f53ec0fbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:53.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.872+0000 7f53f159c700 1 --2- 172.21.15.38:0/927941695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53ec060270 0x7f53ec0f9340 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:53.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.872+0000 7f53f159c700 1 -- 172.21.15.38:0/927941695 >> 172.21.15.38:0/927941695 conn(0x7f53ec0f0d40 msgr2=0x7f53ec0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:53.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.872+0000 7f53f159c700 1 -- 172.21.15.38:0/927941695 shutdown_connections 2024-02-15T18:32:53.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.872+0000 7f53f159c700 1 -- 172.21.15.38:0/927941695 wait complete. 2024-02-15T18:32:53.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.873+0000 7f53f159c700 1 Processor -- start 2024-02-15T18:32:53.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.873+0000 7f53f159c700 1 -- start start 2024-02-15T18:32:53.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.873+0000 7f53f159c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53ec060270 0x7f53ec0f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:53.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.873+0000 7f53f159c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53ec0f9830 0x7f53ec0f5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:53.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.873+0000 7f53f159c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec0f57a0 con 0x7f53ec060270 2024-02-15T18:32:53.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.873+0000 7f53f159c700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec0f58e0 con 0x7f53ec0f9830 2024-02-15T18:32:53.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.873+0000 7f53eaffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53ec060270 0x7f53ec0f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:53.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.873+0000 7f53eaffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53ec060270 0x7f53ec0f6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47738/0 (socket says 172.21.15.38:47738) 2024-02-15T18:32:53.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.873+0000 7f53eaffd700 1 -- 172.21.15.38:0/299451559 learned_addr learned my addr 172.21.15.38:0/299451559 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:32:53.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.873+0000 7f53ea7fc700 1 --2- 172.21.15.38:0/299451559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53ec0f9830 0x7f53ec0f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:53.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.873+0000 7f53eaffd700 1 -- 172.21.15.38:0/299451559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53ec0f9830 msgr2=0x7f53ec0f5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:53.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.873+0000 7f53eaffd700 1 --2- 172.21.15.38:0/299451559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53ec0f9830 0x7f53ec0f5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:53.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.873+0000 7f53eaffd700 1 -- 172.21.15.38:0/299451559 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e0005040 con 0x7f53ec060270 2024-02-15T18:32:53.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.874+0000 7f53eaffd700 1 --2- 172.21.15.38:0/299451559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53ec060270 0x7f53ec0f6dc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f53e0000c00 tx=0x7f53e000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:53.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.874+0000 7f53d37fe700 1 -- 172.21.15.38:0/299451559 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53e0019070 con 0x7f53ec060270 2024-02-15T18:32:53.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.874+0000 7f53f159c700 1 -- 172.21.15.38:0/299451559 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ec0f5b60 con 0x7f53ec060270 2024-02-15T18:32:53.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.874+0000 7f53d37fe700 1 -- 172.21.15.38:0/299451559 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53e0007e90 con 0x7f53ec060270 2024-02-15T18:32:53.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.874+0000 7f53d37fe700 1 -- 172.21.15.38:0/299451559 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53e0014de0 con 0x7f53ec060270 2024-02-15T18:32:53.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.874+0000 7f53f159c700 1 -- 172.21.15.38:0/299451559 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec0f60c0 con 0x7f53ec060270 2024-02-15T18:32:53.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.875+0000 7f53f159c700 1 -- 172.21.15.38:0/299451559 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ec045ad0 con 0x7f53ec060270 2024-02-15T18:32:53.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.875+0000 7f53d37fe700 1 -- 172.21.15.38:0/299451559 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f53e0025070 con 0x7f53ec060270 2024-02-15T18:32:53.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.875+0000 7f53d37fe700 1 --2- 172.21.15.38:0/299451559 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f53d405c250 0x7f53d405e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:53.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.876+0000 7f53d37fe700 1 -- 172.21.15.38:0/299451559 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f53e0076ea0 con 0x7f53ec060270 2024-02-15T18:32:53.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.876+0000 7f53ea7fc700 1 --2- 172.21.15.38:0/299451559 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f53d405c250 0x7f53d405e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:53.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.876+0000 7f53ea7fc700 1 --2- 172.21.15.38:0/299451559 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f53d405c250 0x7f53d405e710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f53dc009fd0 tx=0x7f53dc00b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:53.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:53.880+0000 7f53d37fe700 1 -- 172.21.15.38:0/299451559 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f53e0049b60 con 0x7f53ec060270 2024-02-15T18:32:54.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:54.090+0000 7f53f159c700 1 -- 172.21.15.38:0/299451559 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f53ec0586e0 con 0x7f53ec060270 2024-02-15T18:32:54.221 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:52.277579+0000 mgr.smithi038.khesbl (mgr.14162) 253 : cluster [DBG] pgmap v218: 65 pgs: 3 creating+peering, 62 active+clean; 0 B data, 32 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:54.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:52.277579+0000 mgr.smithi038.khesbl (mgr.14162) 253 : cluster [DBG] pgmap v218: 65 pgs: 3 creating+peering, 62 active+clean; 0 B data, 32 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:54.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:54.916+0000 7f53d37fe700 1 -- 172.21.15.38:0/299451559 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v5) v1 ==== 105+0+0 (secure 0 0 0) 0x7f53e004d180 con 0x7f53ec060270 2024-02-15T18:32:54.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:54.918+0000 7f53f159c700 1 -- 172.21.15.38:0/299451559 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f53d405c250 msgr2=0x7f53d405e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:54.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:54.918+0000 7f53f159c700 1 --2- 172.21.15.38:0/299451559 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f53d405c250 0x7f53d405e710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f53dc009fd0 tx=0x7f53dc00b040).stop 2024-02-15T18:32:54.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:54.918+0000 7f53f159c700 1 -- 172.21.15.38:0/299451559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53ec060270 msgr2=0x7f53ec0f6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:54.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:54.918+0000 7f53f159c700 1 --2- 172.21.15.38:0/299451559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53ec060270 0x7f53ec0f6dc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f53e0000c00 tx=0x7f53e000b040).stop 2024-02-15T18:32:54.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:54.918+0000 7f53f159c700 1 -- 172.21.15.38:0/299451559 shutdown_connections 2024-02-15T18:32:54.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:54.918+0000 7f53f159c700 1 --2- 172.21.15.38:0/299451559 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53ec060270 0x7f53ec0f6dc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:54.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:54.918+0000 7f53f159c700 1 --2- 172.21.15.38:0/299451559 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f53d405c250 0x7f53d405e710 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:54.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:54.918+0000 7f53f159c700 1 --2- 172.21.15.38:0/299451559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53ec0f9830 0x7f53ec0f5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:54.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:54.918+0000 7f53f159c700 1 -- 172.21.15.38:0/299451559 >> 172.21.15.38:0/299451559 conn(0x7f53ec0f0d40 msgr2=0x7f53ec0f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:54.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:54.918+0000 7f53f159c700 1 -- 172.21.15.38:0/299451559 shutdown_connections 2024-02-15T18:32:54.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:54.918+0000 7f53f159c700 1 -- 172.21.15.38:0/299451559 wait complete. 2024-02-15T18:32:55.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:53.908072+0000 mon.smithi038 (mon.0) 482 : cluster [DBG] mds.? [v2:172.21.15.121:6824/1422707571,v1:172.21.15.121:6825/1422707571] up:boot 2024-02-15T18:32:55.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:53.908133+0000 mon.smithi038 (mon.0) 483 : cluster [INF] daemon mds.cephfs.smithi121.mvumlh assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-02-15T18:32:55.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:53.908198+0000 mon.smithi038 (mon.0) 484 : cluster [INF] Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-15T18:32:55.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:53.908221+0000 mon.smithi038 (mon.0) 485 : cluster [INF] Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-15T18:32:55.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:53.908242+0000 mon.smithi038 (mon.0) 486 : cluster [INF] Cluster is now healthy 2024-02-15T18:32:55.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:53.908517+0000 mon.smithi038 (mon.0) 487 : cluster [DBG] fsmap cephfs:0 1 up:standby 2024-02-15T18:32:55.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:53.908606+0000 mon.smithi038 (mon.0) 488 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.mvumlh"}]: dispatch 2024-02-15T18:32:55.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:53.913200+0000 mon.smithi038 (mon.0) 489 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi121.mvumlh=up:creating} 2024-02-15T18:32:55.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:53.925697+0000 mon.smithi038 (mon.0) 490 : cluster [INF] daemon mds.cephfs.smithi121.mvumlh is now active in filesystem cephfs as rank 0 2024-02-15T18:32:55.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:54.092412+0000 mon.smithi038 (mon.0) 491 : audit [INF] from='client.? 172.21.15.38:0/299451559' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-02-15T18:32:55.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:54.277943+0000 mgr.smithi038.khesbl (mgr.14162) 254 : cluster [DBG] pgmap v219: 65 pgs: 65 active+clean; 0 B data, 32 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:55.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:54.555540+0000 mon.smithi038 (mon.0) 492 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:32:55.276 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:54.556291+0000 mon.smithi038 (mon.0) 493 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi121.ovuzki", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:32:55.276 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:54.560448+0000 mon.smithi038 (mon.0) 494 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi121.ovuzki", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-15T18:32:55.276 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:54.561018+0000 mon.smithi038 (mon.0) 495 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:55.276 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:32:54.561571+0000 mgr.smithi038.khesbl (mgr.14162) 255 : cephadm [INF] Deploying daemon mds.cephfs.smithi121.ovuzki on smithi121 2024-02-15T18:32:55.289 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:53.908072+0000 mon.smithi038 (mon.0) 482 : cluster [DBG] mds.? [v2:172.21.15.121:6824/1422707571,v1:172.21.15.121:6825/1422707571] up:boot 2024-02-15T18:32:55.289 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:53.908133+0000 mon.smithi038 (mon.0) 483 : cluster [INF] daemon mds.cephfs.smithi121.mvumlh assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-02-15T18:32:55.290 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:53.908198+0000 mon.smithi038 (mon.0) 484 : cluster [INF] Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-15T18:32:55.290 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:53.908221+0000 mon.smithi038 (mon.0) 485 : cluster [INF] Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-15T18:32:55.290 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:53.908242+0000 mon.smithi038 (mon.0) 486 : cluster [INF] Cluster is now healthy 2024-02-15T18:32:55.290 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:53.908517+0000 mon.smithi038 (mon.0) 487 : cluster [DBG] fsmap cephfs:0 1 up:standby 2024-02-15T18:32:55.290 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:53.908606+0000 mon.smithi038 (mon.0) 488 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.mvumlh"}]: dispatch 2024-02-15T18:32:55.290 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:53.913200+0000 mon.smithi038 (mon.0) 489 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi121.mvumlh=up:creating} 2024-02-15T18:32:55.290 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:53.925697+0000 mon.smithi038 (mon.0) 490 : cluster [INF] daemon mds.cephfs.smithi121.mvumlh is now active in filesystem cephfs as rank 0 2024-02-15T18:32:55.290 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:54.092412+0000 mon.smithi038 (mon.0) 491 : audit [INF] from='client.? 172.21.15.38:0/299451559' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-02-15T18:32:55.290 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:54.277943+0000 mgr.smithi038.khesbl (mgr.14162) 254 : cluster [DBG] pgmap v219: 65 pgs: 65 active+clean; 0 B data, 32 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:55.290 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:54.555540+0000 mon.smithi038 (mon.0) 492 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:32:55.290 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:54.556291+0000 mon.smithi038 (mon.0) 493 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi121.ovuzki", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:32:55.290 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:54.560448+0000 mon.smithi038 (mon.0) 494 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi121.ovuzki", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-15T18:32:55.291 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:54.561018+0000 mon.smithi038 (mon.0) 495 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:55.291 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:32:54.561571+0000 mgr.smithi038.khesbl (mgr.14162) 255 : cephadm [INF] Deploying daemon mds.cephfs.smithi121.ovuzki on smithi121 2024-02-15T18:32:55.610 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-15T18:32:55.622 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-02-15T18:32:55.622 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-02-15T18:32:56.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:54.911687+0000 mon.smithi038 (mon.0) 496 : cluster [WRN] Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-15T18:32:56.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:54.917585+0000 mon.smithi038 (mon.0) 497 : cluster [DBG] mds.? [v2:172.21.15.121:6824/1422707571,v1:172.21.15.121:6825/1422707571] up:active 2024-02-15T18:32:56.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:54.917671+0000 mon.smithi038 (mon.0) 498 : audit [INF] from='client.? 172.21.15.38:0/299451559' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-02-15T18:32:56.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:54.917758+0000 mon.smithi038 (mon.0) 499 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi121.mvumlh=up:active} 2024-02-15T18:32:56.154 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:54.911687+0000 mon.smithi038 (mon.0) 496 : cluster [WRN] Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-15T18:32:56.154 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:54.917585+0000 mon.smithi038 (mon.0) 497 : cluster [DBG] mds.? [v2:172.21.15.121:6824/1422707571,v1:172.21.15.121:6825/1422707571] up:active 2024-02-15T18:32:56.154 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:54.917671+0000 mon.smithi038 (mon.0) 498 : audit [INF] from='client.? 172.21.15.38:0/299451559' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-02-15T18:32:56.154 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:54.917758+0000 mon.smithi038 (mon.0) 499 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi121.mvumlh=up:active} 2024-02-15T18:32:57.225 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:56.278523+0000 mgr.smithi038.khesbl (mgr.14162) 256 : cluster [DBG] pgmap v220: 65 pgs: 65 active+clean; 1.5 KiB data, 32 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:57.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:56.278523+0000 mgr.smithi038.khesbl (mgr.14162) 256 : cluster [DBG] pgmap v220: 65 pgs: 65 active+clean; 1.5 KiB data, 32 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:32:57.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.777+0000 7f6897c4f700 1 -- 172.21.15.38:0/2653676217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68900f55c0 msgr2=0x7f68900f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:57.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.777+0000 7f6897c4f700 1 --2- 172.21.15.38:0/2653676217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68900f55c0 0x7f68900f59e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f688c004660 tx=0x7f688c009b30).stop 2024-02-15T18:32:57.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.777+0000 7f6897c4f700 1 -- 172.21.15.38:0/2653676217 shutdown_connections 2024-02-15T18:32:57.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.777+0000 7f6897c4f700 1 --2- 172.21.15.38:0/2653676217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68900f5ed0 0x7f68900f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:32:57.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.777+0000 7f6897c4f700 1 --2- 172.21.15.38:0/2653676217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68900f55c0 0x7f68900f59e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:57.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.777+0000 7f6897c4f700 1 -- 172.21.15.38:0/2653676217 >> 172.21.15.38:0/2653676217 conn(0x7f68900f0de0 msgr2=0x7f68900f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:57.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.777+0000 7f6897c4f700 1 -- 172.21.15.38:0/2653676217 shutdown_connections 2024-02-15T18:32:57.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.777+0000 7f6897c4f700 1 -- 172.21.15.38:0/2653676217 wait complete. 2024-02-15T18:32:57.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.778+0000 7f6897c4f700 1 Processor -- start 2024-02-15T18:32:57.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.778+0000 7f6897c4f700 1 -- start start 2024-02-15T18:32:57.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.778+0000 7f6897c4f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68900f55c0 0x7f6890105f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:57.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.778+0000 7f6897c4f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68900f5ed0 0x7f6890106430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:57.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.778+0000 7f6897c4f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6890106a70 con 0x7f68900f55c0 2024-02-15T18:32:57.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.778+0000 7f6897c4f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6890107320 con 0x7f68900f5ed0 2024-02-15T18:32:57.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.778+0000 7f68959eb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68900f55c0 0x7f6890105f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:57.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.778+0000 7f68959eb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68900f55c0 0x7f6890105f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47780/0 (socket says 172.21.15.38:47780) 2024-02-15T18:32:57.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.778+0000 7f68959eb700 1 -- 172.21.15.38:0/792652119 learned_addr learned my addr 172.21.15.38:0/792652119 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:32:57.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.778+0000 7f68951ea700 1 --2- 172.21.15.38:0/792652119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68900f5ed0 0x7f6890106430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:57.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.778+0000 7f68959eb700 1 -- 172.21.15.38:0/792652119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68900f5ed0 msgr2=0x7f6890106430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:57.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.778+0000 7f68959eb700 1 --2- 172.21.15.38:0/792652119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68900f5ed0 0x7f6890106430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:57.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.778+0000 7f68959eb700 1 -- 172.21.15.38:0/792652119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688c005040 con 0x7f68900f55c0 2024-02-15T18:32:57.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.779+0000 7f68959eb700 1 --2- 172.21.15.38:0/792652119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68900f55c0 0x7f6890105f40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f688c009830 tx=0x7f688c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:57.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.779+0000 7f68867fc700 1 -- 172.21.15.38:0/792652119 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f688c01a070 con 0x7f68900f55c0 2024-02-15T18:32:57.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.779+0000 7f68867fc700 1 -- 172.21.15.38:0/792652119 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f688c01fad0 con 0x7f68900f55c0 2024-02-15T18:32:57.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.779+0000 7f6897c4f700 1 -- 172.21.15.38:0/792652119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68901075a0 con 0x7f68900f55c0 2024-02-15T18:32:57.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.779+0000 7f68867fc700 1 -- 172.21.15.38:0/792652119 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f688c01e920 con 0x7f68900f55c0 2024-02-15T18:32:57.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.779+0000 7f6897c4f700 1 -- 172.21.15.38:0/792652119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6890107b00 con 0x7f68900f55c0 2024-02-15T18:32:57.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.780+0000 7f6897c4f700 1 -- 172.21.15.38:0/792652119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68900f4a50 con 0x7f68900f55c0 2024-02-15T18:32:57.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.780+0000 7f68867fc700 1 -- 172.21.15.38:0/792652119 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f688c01f600 con 0x7f68900f55c0 2024-02-15T18:32:57.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.781+0000 7f68867fc700 1 --2- 172.21.15.38:0/792652119 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f687c05c2b0 0x7f687c05e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:57.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.781+0000 7f68867fc700 1 -- 172.21.15.38:0/792652119 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f688c07a3b0 con 0x7f68900f55c0 2024-02-15T18:32:57.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.781+0000 7f68951ea700 1 --2- 172.21.15.38:0/792652119 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f687c05c2b0 0x7f687c05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:57.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.781+0000 7f68951ea700 1 --2- 172.21.15.38:0/792652119 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f687c05c2b0 0x7f687c05e770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f6880006fd0 tx=0x7f6880008040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:57.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.785+0000 7f68867fc700 1 -- 172.21.15.38:0/792652119 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f688c04d070 con 0x7f68900f55c0 2024-02-15T18:32:57.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:57.988+0000 7f6897c4f700 1 -- 172.21.15.38:0/792652119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6890108140 con 0x7f68900f55c0 2024-02-15T18:32:58.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:57.084273+0000 mon.smithi038 (mon.0) 500 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:32:58.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:57.085251+0000 mon.smithi038 (mon.0) 501 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.qgkzgn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:32:58.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:57.090478+0000 mon.smithi038 (mon.0) 502 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.qgkzgn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-15T18:32:58.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:57.091410+0000 mon.smithi038 (mon.0) 503 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:58.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:32:57.092195+0000 mgr.smithi038.khesbl (mgr.14162) 257 : cephadm [INF] Deploying daemon mds.cephfs.smithi038.qgkzgn on smithi038 2024-02-15T18:32:58.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:57.923972+0000 mon.smithi038 (mon.0) 504 : cluster [DBG] mds.? [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845] up:boot 2024-02-15T18:32:58.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:57.924039+0000 mon.smithi038 (mon.0) 505 : cluster [INF] daemon mds.cephfs.smithi121.ovuzki assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-15T18:32:58.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:57.924130+0000 mon.smithi038 (mon.0) 506 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-02-15T18:32:58.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:57.924147+0000 mon.smithi038 (mon.0) 507 : cluster [INF] Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-15T18:32:58.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:57.927068+0000 mon.smithi038 (mon.0) 508 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi121.mvumlh=up:active} 1 up:standby 2024-02-15T18:32:58.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:57.927195+0000 mon.smithi038 (mon.0) 509 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.ovuzki"}]: dispatch 2024-02-15T18:32:58.360 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:57.929510+0000 mon.smithi038 (mon.0) 510 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:creating} 2024-02-15T18:32:58.360 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:57.942169+0000 mon.smithi038 (mon.0) 511 : cluster [INF] daemon mds.cephfs.smithi121.ovuzki is now active in filesystem cephfs as rank 1 2024-02-15T18:32:58.360 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:57.989723+0000 mon.smithi038 (mon.0) 512 : audit [INF] from='client.? 172.21.15.38:0/792652119' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-02-15T18:32:58.362 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:57.084273+0000 mon.smithi038 (mon.0) 500 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:32:58.362 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:57.085251+0000 mon.smithi038 (mon.0) 501 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.qgkzgn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:32:58.362 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:57.090478+0000 mon.smithi038 (mon.0) 502 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.qgkzgn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-15T18:32:58.363 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:57.091410+0000 mon.smithi038 (mon.0) 503 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:58.363 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:32:57.092195+0000 mgr.smithi038.khesbl (mgr.14162) 257 : cephadm [INF] Deploying daemon mds.cephfs.smithi038.qgkzgn on smithi038 2024-02-15T18:32:58.363 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:57.923972+0000 mon.smithi038 (mon.0) 504 : cluster [DBG] mds.? [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845] up:boot 2024-02-15T18:32:58.363 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:57.924039+0000 mon.smithi038 (mon.0) 505 : cluster [INF] daemon mds.cephfs.smithi121.ovuzki assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-15T18:32:58.363 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:57.924130+0000 mon.smithi038 (mon.0) 506 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-02-15T18:32:58.363 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:57.924147+0000 mon.smithi038 (mon.0) 507 : cluster [INF] Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-15T18:32:58.363 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:57.927068+0000 mon.smithi038 (mon.0) 508 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi121.mvumlh=up:active} 1 up:standby 2024-02-15T18:32:58.363 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:57.927195+0000 mon.smithi038 (mon.0) 509 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.ovuzki"}]: dispatch 2024-02-15T18:32:58.363 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:57.929510+0000 mon.smithi038 (mon.0) 510 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:creating} 2024-02-15T18:32:58.363 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:57.942169+0000 mon.smithi038 (mon.0) 511 : cluster [INF] daemon mds.cephfs.smithi121.ovuzki is now active in filesystem cephfs as rank 1 2024-02-15T18:32:58.363 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:57.989723+0000 mon.smithi038 (mon.0) 512 : audit [INF] from='client.? 172.21.15.38:0/792652119' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-02-15T18:32:58.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:58.934+0000 7f68867fc700 1 -- 172.21.15.38:0/792652119 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f688c050690 con 0x7f68900f55c0 2024-02-15T18:32:58.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:58.937+0000 7f6897c4f700 1 -- 172.21.15.38:0/792652119 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f687c05c2b0 msgr2=0x7f687c05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:58.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:58.937+0000 7f6897c4f700 1 --2- 172.21.15.38:0/792652119 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f687c05c2b0 0x7f687c05e770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f6880006fd0 tx=0x7f6880008040).stop 2024-02-15T18:32:58.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:58.937+0000 7f6897c4f700 1 -- 172.21.15.38:0/792652119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68900f55c0 msgr2=0x7f6890105f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:58.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:58.937+0000 7f6897c4f700 1 --2- 172.21.15.38:0/792652119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68900f55c0 0x7f6890105f40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f688c009830 tx=0x7f688c009b30).stop 2024-02-15T18:32:58.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:58.937+0000 7f6897c4f700 1 -- 172.21.15.38:0/792652119 shutdown_connections 2024-02-15T18:32:58.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:58.937+0000 7f6897c4f700 1 --2- 172.21.15.38:0/792652119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68900f55c0 0x7f6890105f40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:58.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:58.937+0000 7f6897c4f700 1 --2- 172.21.15.38:0/792652119 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f687c05c2b0 0x7f687c05e770 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:58.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:58.937+0000 7f6897c4f700 1 --2- 172.21.15.38:0/792652119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68900f5ed0 0x7f6890106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:58.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:58.937+0000 7f6897c4f700 1 -- 172.21.15.38:0/792652119 >> 172.21.15.38:0/792652119 conn(0x7f68900f0de0 msgr2=0x7f68900fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:58.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:58.937+0000 7f6897c4f700 1 -- 172.21.15.38:0/792652119 shutdown_connections 2024-02-15T18:32:58.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:32:58.937+0000 7f6897c4f700 1 -- 172.21.15.38:0/792652119 wait complete. 2024-02-15T18:32:59.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:58.278984+0000 mgr.smithi038.khesbl (mgr.14162) 258 : cluster [DBG] pgmap v221: 65 pgs: 65 active+clean; 1.5 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 1 op/s 2024-02-15T18:32:59.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:58.928061+0000 mon.smithi038 (mon.0) 513 : cluster [INF] Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-02-15T18:32:59.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:58.928091+0000 mon.smithi038 (mon.0) 514 : cluster [INF] Cluster is now healthy 2024-02-15T18:32:59.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:58.936041+0000 mon.smithi038 (mon.0) 515 : cluster [DBG] mds.? [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845] up:active 2024-02-15T18:32:59.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:58.936118+0000 mon.smithi038 (mon.0) 516 : audit [INF] from='client.? 172.21.15.38:0/792652119' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-02-15T18:32:59.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:58.936193+0000 mon.smithi038 (mon.0) 517 : cluster [DBG] mds.? [v2:172.21.15.38:6826/3822880235,v1:172.21.15.38:6827/3822880235] up:boot 2024-02-15T18:32:59.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:32:58.936256+0000 mon.smithi038 (mon.0) 518 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:active} 1 up:standby 2024-02-15T18:32:59.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:32:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:32:58.938425+0000 mon.smithi038 (mon.0) 519 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.hxoggl"}]: dispatch 2024-02-15T18:32:59.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:58.278984+0000 mgr.smithi038.khesbl (mgr.14162) 258 : cluster [DBG] pgmap v221: 65 pgs: 65 active+clean; 1.5 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 1 op/s 2024-02-15T18:32:59.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:58.928061+0000 mon.smithi038 (mon.0) 513 : cluster [INF] Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-02-15T18:32:59.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:58.928091+0000 mon.smithi038 (mon.0) 514 : cluster [INF] Cluster is now healthy 2024-02-15T18:32:59.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:58.936041+0000 mon.smithi038 (mon.0) 515 : cluster [DBG] mds.? [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845] up:active 2024-02-15T18:32:59.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:58.936118+0000 mon.smithi038 (mon.0) 516 : audit [INF] from='client.? 172.21.15.38:0/792652119' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-02-15T18:32:59.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:58.936193+0000 mon.smithi038 (mon.0) 517 : cluster [DBG] mds.? [v2:172.21.15.38:6826/3822880235,v1:172.21.15.38:6827/3822880235] up:boot 2024-02-15T18:32:59.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:32:58.936256+0000 mon.smithi038 (mon.0) 518 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:active} 1 up:standby 2024-02-15T18:32:59.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:32:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:32:58.938425+0000 mon.smithi038 (mon.0) 519 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.hxoggl"}]: dispatch 2024-02-15T18:33:00.038 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-15T18:33:00.049 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-02-15T18:33:00.049 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data false' 2024-02-15T18:33:01.996 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:00.279391+0000 mgr.smithi038.khesbl (mgr.14162) 259 : cluster [DBG] pgmap v222: 65 pgs: 65 active+clean; 3.4 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 3.5 KiB/s wr, 6 op/s 2024-02-15T18:33:01.996 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:00.880811+0000 mon.smithi038 (mon.0) 520 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:33:01.996 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:00.881453+0000 mon.smithi038 (mon.0) 521 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:33:01.996 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:01.455228+0000 mon.smithi038 (mon.0) 522 : cluster [DBG] mds.? [v2:172.21.15.38:6828/703030231,v1:172.21.15.38:6829/703030231] up:boot 2024-02-15T18:33:01.996 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:01.455282+0000 mon.smithi038 (mon.0) 523 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:active} 2 up:standby 2024-02-15T18:33:01.996 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:01.455757+0000 mon.smithi038 (mon.0) 524 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.qgkzgn"}]: dispatch 2024-02-15T18:33:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.273+0000 7f7db4e79700 1 -- 172.21.15.38:0/950193331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db00f49a0 msgr2=0x7f7db00f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.273+0000 7f7db4e79700 1 --2- 172.21.15.38:0/950193331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db00f49a0 0x7f7db00f4dc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f7da0004660 tx=0x7f7da0009b30).stop 2024-02-15T18:33:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.273+0000 7f7db4e79700 1 -- 172.21.15.38:0/950193331 shutdown_connections 2024-02-15T18:33:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.273+0000 7f7db4e79700 1 --2- 172.21.15.38:0/950193331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db00f5340 0x7f7db00f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.273+0000 7f7db4e79700 1 --2- 172.21.15.38:0/950193331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db00f49a0 0x7f7db00f4dc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.273+0000 7f7db4e79700 1 -- 172.21.15.38:0/950193331 >> 172.21.15.38:0/950193331 conn(0x7f7db00f04c0 msgr2=0x7f7db00f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.274+0000 7f7db4e79700 1 -- 172.21.15.38:0/950193331 shutdown_connections 2024-02-15T18:33:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.274+0000 7f7db4e79700 1 -- 172.21.15.38:0/950193331 wait complete. 2024-02-15T18:33:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.274+0000 7f7db4e79700 1 Processor -- start 2024-02-15T18:33:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.274+0000 7f7db4e79700 1 -- start start 2024-02-15T18:33:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.274+0000 7f7db4e79700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db00f49a0 0x7f7db01070d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.274+0000 7f7db4e79700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db00f5340 0x7f7db01075c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.274+0000 7f7db4e79700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db0107b70 con 0x7f7db00f49a0 2024-02-15T18:33:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.274+0000 7f7db4e79700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db0107cb0 con 0x7f7db00f5340 2024-02-15T18:33:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.274+0000 7f7dae59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db00f49a0 0x7f7db01070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.274+0000 7f7dadd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db00f5340 0x7f7db01075c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.274+0000 7f7dae59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db00f49a0 0x7f7db01070d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47822/0 (socket says 172.21.15.38:47822) 2024-02-15T18:33:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.274+0000 7f7dae59c700 1 -- 172.21.15.38:0/943501981 learned_addr learned my addr 172.21.15.38:0/943501981 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:33:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.275+0000 7f7dae59c700 1 -- 172.21.15.38:0/943501981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db00f5340 msgr2=0x7f7db01075c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.275+0000 7f7dae59c700 1 --2- 172.21.15.38:0/943501981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db00f5340 0x7f7db01075c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.275+0000 7f7dae59c700 1 -- 172.21.15.38:0/943501981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da0005040 con 0x7f7db00f49a0 2024-02-15T18:33:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.275+0000 7f7dadd9b700 1 --2- 172.21.15.38:0/943501981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db00f5340 0x7f7db01075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:33:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.275+0000 7f7dae59c700 1 --2- 172.21.15.38:0/943501981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db00f49a0 0x7f7db01070d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f7da0000c00 tx=0x7f7da000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.275+0000 7f7d9effd700 1 -- 172.21.15.38:0/943501981 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7da0019070 con 0x7f7db00f49a0 2024-02-15T18:33:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.275+0000 7f7db4e79700 1 -- 172.21.15.38:0/943501981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db01085f0 con 0x7f7db00f49a0 2024-02-15T18:33:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.275+0000 7f7d9effd700 1 -- 172.21.15.38:0/943501981 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7da0007e60 con 0x7f7db00f49a0 2024-02-15T18:33:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.275+0000 7f7d9effd700 1 -- 172.21.15.38:0/943501981 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7da0014cc0 con 0x7f7db00f49a0 2024-02-15T18:33:02.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.275+0000 7f7db4e79700 1 -- 172.21.15.38:0/943501981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db0108b50 con 0x7f7db00f49a0 2024-02-15T18:33:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.276+0000 7f7db4e79700 1 -- 172.21.15.38:0/943501981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db0045ad0 con 0x7f7db00f49a0 2024-02-15T18:33:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.276+0000 7f7d9effd700 1 -- 172.21.15.38:0/943501981 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f7da0025070 con 0x7f7db00f49a0 2024-02-15T18:33:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.277+0000 7f7d9effd700 1 --2- 172.21.15.38:0/943501981 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f7d9805c2a0 0x7f7d9805e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.277+0000 7f7d9effd700 1 -- 172.21.15.38:0/943501981 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f7da0076ee0 con 0x7f7db00f49a0 2024-02-15T18:33:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.277+0000 7f7dadd9b700 1 --2- 172.21.15.38:0/943501981 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f7d9805c2a0 0x7f7d9805e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.277+0000 7f7dadd9b700 1 --2- 172.21.15.38:0/943501981 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f7d9805c2a0 0x7f7d9805e760 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f7da4009fd0 tx=0x7f7da400b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:02.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.281+0000 7f7d9effd700 1 -- 172.21.15.38:0/943501981 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f7da0049ba0 con 0x7f7db00f49a0 2024-02-15T18:33:02.298 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:00.279391+0000 mgr.smithi038.khesbl (mgr.14162) 259 : cluster [DBG] pgmap v222: 65 pgs: 65 active+clean; 3.4 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 3.5 KiB/s wr, 6 op/s 2024-02-15T18:33:02.298 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:00.880811+0000 mon.smithi038 (mon.0) 520 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:33:02.298 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:00.881453+0000 mon.smithi038 (mon.0) 521 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:33:02.298 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:01.455228+0000 mon.smithi038 (mon.0) 522 : cluster [DBG] mds.? [v2:172.21.15.38:6828/703030231,v1:172.21.15.38:6829/703030231] up:boot 2024-02-15T18:33:02.298 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:01.455282+0000 mon.smithi038 (mon.0) 523 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:active} 2 up:standby 2024-02-15T18:33:02.298 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:01.455757+0000 mon.smithi038 (mon.0) 524 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.qgkzgn"}]: dispatch 2024-02-15T18:33:02.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.479+0000 7f7db4e79700 1 -- 172.21.15.38:0/943501981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f7db0027a00 con 0x7f7db00f49a0 2024-02-15T18:33:02.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.888+0000 7f7d9effd700 1 -- 172.21.15.38:0/943501981 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v10) v1 ==== 133+0+0 (secure 0 0 0) 0x7f7da004d1c0 con 0x7f7db00f49a0 2024-02-15T18:33:02.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.890+0000 7f7db4e79700 1 -- 172.21.15.38:0/943501981 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f7d9805c2a0 msgr2=0x7f7d9805e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:02.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.890+0000 7f7db4e79700 1 --2- 172.21.15.38:0/943501981 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f7d9805c2a0 0x7f7d9805e760 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f7da4009fd0 tx=0x7f7da400b040).stop 2024-02-15T18:33:02.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.890+0000 7f7db4e79700 1 -- 172.21.15.38:0/943501981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db00f49a0 msgr2=0x7f7db01070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:02.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.890+0000 7f7db4e79700 1 --2- 172.21.15.38:0/943501981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db00f49a0 0x7f7db01070d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f7da0000c00 tx=0x7f7da000b040).stop 2024-02-15T18:33:02.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.890+0000 7f7db4e79700 1 -- 172.21.15.38:0/943501981 shutdown_connections 2024-02-15T18:33:02.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.890+0000 7f7db4e79700 1 --2- 172.21.15.38:0/943501981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db00f49a0 0x7f7db01070d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:02.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.890+0000 7f7db4e79700 1 --2- 172.21.15.38:0/943501981 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f7d9805c2a0 0x7f7d9805e760 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:02.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.890+0000 7f7db4e79700 1 --2- 172.21.15.38:0/943501981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db00f5340 0x7f7db01075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:02.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.890+0000 7f7db4e79700 1 -- 172.21.15.38:0/943501981 >> 172.21.15.38:0/943501981 conn(0x7f7db00f04c0 msgr2=0x7f7db00fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:02.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.890+0000 7f7db4e79700 1 -- 172.21.15.38:0/943501981 shutdown_connections 2024-02-15T18:33:02.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:02.890+0000 7f7db4e79700 1 -- 172.21.15.38:0/943501981 wait complete. 2024-02-15T18:33:02.884 INFO:teuthology.orchestra.run.smithi038.stderr:inline data disabled 2024-02-15T18:33:03.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:02.481725+0000 mon.smithi038 (mon.0) 525 : audit [INF] from='client.? 172.21.15.38:0/943501981' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-02-15T18:33:03.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:02.481725+0000 mon.smithi038 (mon.0) 525 : audit [INF] from='client.? 172.21.15.38:0/943501981' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-02-15T18:33:03.722 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-15T18:33:03.732 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-02-15T18:33:03.733 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-15T18:33:04.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:02.279934+0000 mgr.smithi038.khesbl (mgr.14162) 260 : cluster [DBG] pgmap v223: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 3.5 KiB/s wr, 8 op/s 2024-02-15T18:33:04.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:02.889733+0000 mon.smithi038 (mon.0) 526 : cluster [DBG] mds.? [v2:172.21.15.38:6826/3822880235,v1:172.21.15.38:6827/3822880235] up:standby 2024-02-15T18:33:04.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:02.889780+0000 mon.smithi038 (mon.0) 527 : audit [INF] from='client.? 172.21.15.38:0/943501981' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-02-15T18:33:04.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:02.889838+0000 mon.smithi038 (mon.0) 528 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:active} 2 up:standby 2024-02-15T18:33:04.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:03.039842+0000 mon.smithi038 (mon.0) 529 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:33:04.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:02.279934+0000 mgr.smithi038.khesbl (mgr.14162) 260 : cluster [DBG] pgmap v223: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 3.5 KiB/s wr, 8 op/s 2024-02-15T18:33:04.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:02.889733+0000 mon.smithi038 (mon.0) 526 : cluster [DBG] mds.? [v2:172.21.15.38:6826/3822880235,v1:172.21.15.38:6827/3822880235] up:standby 2024-02-15T18:33:04.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:02.889780+0000 mon.smithi038 (mon.0) 527 : audit [INF] from='client.? 172.21.15.38:0/943501981' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-02-15T18:33:04.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:02.889838+0000 mon.smithi038 (mon.0) 528 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:active} 2 up:standby 2024-02-15T18:33:04.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:03.039842+0000 mon.smithi038 (mon.0) 529 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:33:06.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:04.280396+0000 mgr.smithi038.khesbl (mgr.14162) 261 : cluster [DBG] pgmap v224: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s wr, 6 op/s 2024-02-15T18:33:06.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:05.326608+0000 mon.smithi038 (mon.0) 530 : cluster [DBG] mds.? [v2:172.21.15.38:6828/703030231,v1:172.21.15.38:6829/703030231] up:standby 2024-02-15T18:33:06.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:05.326674+0000 mon.smithi038 (mon.0) 531 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:active} 2 up:standby 2024-02-15T18:33:06.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:04.280396+0000 mgr.smithi038.khesbl (mgr.14162) 261 : cluster [DBG] pgmap v224: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s wr, 6 op/s 2024-02-15T18:33:06.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:05.326608+0000 mon.smithi038 (mon.0) 530 : cluster [DBG] mds.? [v2:172.21.15.38:6828/703030231,v1:172.21.15.38:6829/703030231] up:standby 2024-02-15T18:33:06.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:05.326674+0000 mon.smithi038 (mon.0) 531 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:active} 2 up:standby 2024-02-15T18:33:06.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.745+0000 7f4156c2e700 1 -- 172.21.15.38:0/1036421801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41500f5430 msgr2=0x7f41500f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:06.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.745+0000 7f4156c2e700 1 --2- 172.21.15.38:0/1036421801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41500f5430 0x7f41500f5850 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f414c004660 tx=0x7f414c009b30).stop 2024-02-15T18:33:06.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.745+0000 7f4156c2e700 1 -- 172.21.15.38:0/1036421801 shutdown_connections 2024-02-15T18:33:06.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.745+0000 7f4156c2e700 1 --2- 172.21.15.38:0/1036421801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41500f66d0 0x7f41500f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:06.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.745+0000 7f4156c2e700 1 --2- 172.21.15.38:0/1036421801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41500f5430 0x7f41500f5850 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:06.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.745+0000 7f4156c2e700 1 -- 172.21.15.38:0/1036421801 >> 172.21.15.38:0/1036421801 conn(0x7f41500f0b90 msgr2=0x7f41500f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:06.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.745+0000 7f4156c2e700 1 -- 172.21.15.38:0/1036421801 shutdown_connections 2024-02-15T18:33:06.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.745+0000 7f4156c2e700 1 -- 172.21.15.38:0/1036421801 wait complete. 2024-02-15T18:33:06.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.745+0000 7f4156c2e700 1 Processor -- start 2024-02-15T18:33:06.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.745+0000 7f4156c2e700 1 -- start start 2024-02-15T18:33:06.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.746+0000 7f4156c2e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41500f5430 0x7f4150105e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:06.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.746+0000 7f4156c2e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41500f66d0 0x7f4150106300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:06.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.746+0000 7f4156c2e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4150106900 con 0x7f41500f5430 2024-02-15T18:33:06.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.746+0000 7f4156c2e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4150107110 con 0x7f41500f66d0 2024-02-15T18:33:06.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.746+0000 7f4155c2c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41500f5430 0x7f4150105e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:06.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.746+0000 7f4155c2c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41500f5430 0x7f4150105e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54208/0 (socket says 172.21.15.38:54208) 2024-02-15T18:33:06.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.746+0000 7f4155c2c700 1 -- 172.21.15.38:0/2879624570 learned_addr learned my addr 172.21.15.38:0/2879624570 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:33:06.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.746+0000 7f415542b700 1 --2- 172.21.15.38:0/2879624570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41500f66d0 0x7f4150106300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:06.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.746+0000 7f4155c2c700 1 -- 172.21.15.38:0/2879624570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41500f66d0 msgr2=0x7f4150106300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:06.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.746+0000 7f4155c2c700 1 --2- 172.21.15.38:0/2879624570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41500f66d0 0x7f4150106300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:06.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.746+0000 7f4155c2c700 1 -- 172.21.15.38:0/2879624570 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414c005040 con 0x7f41500f5430 2024-02-15T18:33:06.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.746+0000 7f4155c2c700 1 --2- 172.21.15.38:0/2879624570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41500f5430 0x7f4150105e10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f414c004e30 tx=0x7f414c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:06.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.746+0000 7f41467fc700 1 -- 172.21.15.38:0/2879624570 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f414c018070 con 0x7f41500f5430 2024-02-15T18:33:06.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.746+0000 7f41467fc700 1 -- 172.21.15.38:0/2879624570 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f414c007ba0 con 0x7f41500f5430 2024-02-15T18:33:06.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.746+0000 7f41467fc700 1 -- 172.21.15.38:0/2879624570 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f414c01c3c0 con 0x7f41500f5430 2024-02-15T18:33:06.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.747+0000 7f4156c2e700 1 -- 172.21.15.38:0/2879624570 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4150107330 con 0x7f41500f5430 2024-02-15T18:33:06.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.747+0000 7f4156c2e700 1 -- 172.21.15.38:0/2879624570 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4150107840 con 0x7f41500f5430 2024-02-15T18:33:06.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.747+0000 7f4156c2e700 1 -- 172.21.15.38:0/2879624570 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4150059100 con 0x7f41500f5430 2024-02-15T18:33:06.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.748+0000 7f41467fc700 1 -- 172.21.15.38:0/2879624570 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f414c007d10 con 0x7f41500f5430 2024-02-15T18:33:06.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.749+0000 7f41467fc700 1 --2- 172.21.15.38:0/2879624570 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f413c05c300 0x7f413c05e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:06.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.749+0000 7f41467fc700 1 -- 172.21.15.38:0/2879624570 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f414c077eb0 con 0x7f41500f5430 2024-02-15T18:33:06.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.749+0000 7f415542b700 1 --2- 172.21.15.38:0/2879624570 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f413c05c300 0x7f413c05e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:06.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.749+0000 7f415542b700 1 --2- 172.21.15.38:0/2879624570 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f413c05c300 0x7f413c05e7c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f4140004c40 tx=0x7f414000b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:06.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.752+0000 7f41467fc700 1 -- 172.21.15.38:0/2879624570 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f414c04ac60 con 0x7f41500f5430 2024-02-15T18:33:06.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.949+0000 7f4156c2e700 1 -- 172.21.15.38:0/2879624570 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4150027a00 con 0x7f41500f5430 2024-02-15T18:33:06.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.949+0000 7f41467fc700 1 -- 172.21.15.38:0/2879624570 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 11 v11) v1 ==== 76+0+1626 (secure 0 0 0) 0x7f414c04e280 con 0x7f41500f5430 2024-02-15T18:33:06.942 INFO:teuthology.orchestra.run.smithi038.stdout:e11 2024-02-15T18:33:06.942 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:33:06.942 INFO:teuthology.orchestra.run.smithi038.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:33:06.942 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-02-15T18:33:06.942 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:33:06.943 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:33:06.943 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-02-15T18:33:06.943 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 10 2024-02-15T18:33:06.943 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 2024-02-15T18:33:06.943 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:33:06.943 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-02-15T18:33:02.882026+0000 2024-02-15T18:33:06.943 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-02-15T18:33:06.943 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-02-15T18:33:06.943 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-02-15T18:33:06.943 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-02-15T18:33:06.943 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-02-15T18:33:06.944 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-02-15T18:33:06.944 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-02-15T18:33:06.944 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-02-15T18:33:06.944 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:33:06.944 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-02-15T18:33:06.944 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-02-15T18:33:06.944 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=24261,1=24275} 2024-02-15T18:33:06.944 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-02-15T18:33:06.944 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-02-15T18:33:06.944 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-02-15T18:33:06.944 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-02-15T18:33:06.944 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-02-15T18:33:06.944 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-02-15T18:33:06.945 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-02-15T18:33:06.945 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-02-15T18:33:06.945 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.mvumlh{0:24261} state up:active seq 3 join_fscid=1 addr [v2:172.21.15.121:6824/1422707571,v1:172.21.15.121:6825/1422707571]] 2024-02-15T18:33:06.945 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.ovuzki{1:24275} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845]] 2024-02-15T18:33:06.945 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:33:06.945 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:33:06.945 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-02-15T18:33:06.945 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:33:06.945 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.hxoggl{-1:14446} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.38:6826/3822880235,v1:172.21.15.38:6827/3822880235]] 2024-02-15T18:33:06.945 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.qgkzgn{-1:14456} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.38:6828/703030231,v1:172.21.15.38:6829/703030231]] 2024-02-15T18:33:06.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.951+0000 7f4156c2e700 1 -- 172.21.15.38:0/2879624570 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f413c05c300 msgr2=0x7f413c05e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:06.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.951+0000 7f4156c2e700 1 --2- 172.21.15.38:0/2879624570 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f413c05c300 0x7f413c05e7c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f4140004c40 tx=0x7f414000b040).stop 2024-02-15T18:33:06.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.951+0000 7f4156c2e700 1 -- 172.21.15.38:0/2879624570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41500f5430 msgr2=0x7f4150105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:06.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.951+0000 7f4156c2e700 1 --2- 172.21.15.38:0/2879624570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41500f5430 0x7f4150105e10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f414c004e30 tx=0x7f414c009b30).stop 2024-02-15T18:33:06.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.951+0000 7f4156c2e700 1 -- 172.21.15.38:0/2879624570 shutdown_connections 2024-02-15T18:33:06.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.951+0000 7f4156c2e700 1 --2- 172.21.15.38:0/2879624570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41500f5430 0x7f4150105e10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:06.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.951+0000 7f4156c2e700 1 --2- 172.21.15.38:0/2879624570 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f413c05c300 0x7f413c05e7c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:06.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.951+0000 7f4156c2e700 1 --2- 172.21.15.38:0/2879624570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41500f66d0 0x7f4150106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:06.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.951+0000 7f4156c2e700 1 -- 172.21.15.38:0/2879624570 >> 172.21.15.38:0/2879624570 conn(0x7f41500f0b90 msgr2=0x7f41500fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:06.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.951+0000 7f4156c2e700 1 -- 172.21.15.38:0/2879624570 shutdown_connections 2024-02-15T18:33:06.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:06.951+0000 7f4156c2e700 1 -- 172.21.15.38:0/2879624570 wait complete. 2024-02-15T18:33:06.947 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 11 2024-02-15T18:33:08.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:06.280893+0000 mgr.smithi038.khesbl (mgr.14162) 262 : cluster [DBG] pgmap v225: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s wr, 5 op/s 2024-02-15T18:33:08.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:06.951030+0000 mon.smithi038 (mon.0) 532 : audit [DBG] from='client.? 172.21.15.38:0/2879624570' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:33:08.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:06.280893+0000 mgr.smithi038.khesbl (mgr.14162) 262 : cluster [DBG] pgmap v225: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s wr, 5 op/s 2024-02-15T18:33:08.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:06.951030+0000 mon.smithi038 (mon.0) 532 : audit [DBG] from='client.? 172.21.15.38:0/2879624570' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:33:08.849 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-02-15T18:33:09.122 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:08.281409+0000 mgr.smithi038.khesbl (mgr.14162) 263 : cluster [DBG] pgmap v226: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s wr, 4 op/s 2024-02-15T18:33:09.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:08.281409+0000 mgr.smithi038.khesbl (mgr.14162) 263 : cluster [DBG] pgmap v226: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s wr, 4 op/s 2024-02-15T18:33:11.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.411+0000 7f357bb77700 1 -- 172.21.15.38:0/3417490326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35740f6860 msgr2=0x7f35740f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:11.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.411+0000 7f357bb77700 1 --2- 172.21.15.38:0/3417490326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35740f6860 0x7f35740f6cc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f3564004660 tx=0x7f3564009b30).stop 2024-02-15T18:33:11.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.411+0000 7f357bb77700 1 -- 172.21.15.38:0/3417490326 shutdown_connections 2024-02-15T18:33:11.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.411+0000 7f357bb77700 1 --2- 172.21.15.38:0/3417490326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35740f6860 0x7f35740f6cc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.411+0000 7f357bb77700 1 --2- 172.21.15.38:0/3417490326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35740f55c0 0x7f35740f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.411+0000 7f357bb77700 1 -- 172.21.15.38:0/3417490326 >> 172.21.15.38:0/3417490326 conn(0x7f35740f0d40 msgr2=0x7f35740f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.411+0000 7f357bb77700 1 -- 172.21.15.38:0/3417490326 shutdown_connections 2024-02-15T18:33:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.411+0000 7f357bb77700 1 -- 172.21.15.38:0/3417490326 wait complete. 2024-02-15T18:33:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.412+0000 7f357bb77700 1 Processor -- start 2024-02-15T18:33:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.412+0000 7f357bb77700 1 -- start start 2024-02-15T18:33:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.412+0000 7f357bb77700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35740f55c0 0x7f3574109d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.412+0000 7f357bb77700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35740f6860 0x7f357410c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.412+0000 7f357bb77700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35740603c0 con 0x7f35740f55c0 2024-02-15T18:33:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.412+0000 7f357bb77700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3574060500 con 0x7f35740f6860 2024-02-15T18:33:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.412+0000 7f3579913700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35740f55c0 0x7f3574109d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.412+0000 7f3579913700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35740f55c0 0x7f3574109d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54230/0 (socket says 172.21.15.38:54230) 2024-02-15T18:33:11.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.412+0000 7f3579913700 1 -- 172.21.15.38:0/3866112913 learned_addr learned my addr 172.21.15.38:0/3866112913 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:33:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.412+0000 7f3579112700 1 --2- 172.21.15.38:0/3866112913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35740f6860 0x7f357410c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.413+0000 7f3579913700 1 -- 172.21.15.38:0/3866112913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35740f6860 msgr2=0x7f357410c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.413+0000 7f3579913700 1 --2- 172.21.15.38:0/3866112913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35740f6860 0x7f357410c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.413+0000 7f3579913700 1 -- 172.21.15.38:0/3866112913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3564005040 con 0x7f35740f55c0 2024-02-15T18:33:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.413+0000 7f3579913700 1 --2- 172.21.15.38:0/3866112913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35740f55c0 0x7f3574109d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f3570002700 tx=0x7f3570009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.413+0000 7f356a7fc700 1 -- 172.21.15.38:0/3866112913 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35700079c0 con 0x7f35740f55c0 2024-02-15T18:33:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.413+0000 7f357bb77700 1 -- 172.21.15.38:0/3866112913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f357410c840 con 0x7f35740f55c0 2024-02-15T18:33:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.413+0000 7f356a7fc700 1 -- 172.21.15.38:0/3866112913 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f357000f040 con 0x7f35740f55c0 2024-02-15T18:33:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.413+0000 7f356a7fc700 1 -- 172.21.15.38:0/3866112913 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3570014970 con 0x7f35740f55c0 2024-02-15T18:33:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.413+0000 7f357bb77700 1 -- 172.21.15.38:0/3866112913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357410cdd0 con 0x7f35740f55c0 2024-02-15T18:33:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.414+0000 7f356a7fc700 1 -- 172.21.15.38:0/3866112913 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f357001b060 con 0x7f35740f55c0 2024-02-15T18:33:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.414+0000 7f357bb77700 1 -- 172.21.15.38:0/3866112913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35740f49b0 con 0x7f35740f55c0 2024-02-15T18:33:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.415+0000 7f356a7fc700 1 --2- 172.21.15.38:0/3866112913 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f356005c2b0 0x7f356005e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.415+0000 7f356a7fc700 1 -- 172.21.15.38:0/3866112913 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f3570075d10 con 0x7f35740f55c0 2024-02-15T18:33:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.415+0000 7f3579112700 1 --2- 172.21.15.38:0/3866112913 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f356005c2b0 0x7f356005e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.415+0000 7f3579112700 1 --2- 172.21.15.38:0/3866112913 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f356005c2b0 0x7f356005e770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f3564004630 tx=0x7f3564009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:11.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.422+0000 7f356a7fc700 1 -- 172.21.15.38:0/3866112913 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f35700489d0 con 0x7f35740f55c0 2024-02-15T18:33:11.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:10.281942+0000 mgr.smithi038.khesbl (mgr.14162) 264 : cluster [DBG] pgmap v227: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s wr, 4 op/s 2024-02-15T18:33:11.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:10.281942+0000 mgr.smithi038.khesbl (mgr.14162) 264 : cluster [DBG] pgmap v227: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s wr, 4 op/s 2024-02-15T18:33:11.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.634+0000 7f357bb77700 1 -- 172.21.15.38:0/3866112913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35740fdfd0 con 0x7f35740f55c0 2024-02-15T18:33:11.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.635+0000 7f356a7fc700 1 -- 172.21.15.38:0/3866112913 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2942 (secure 0 0 0) 0x7f357004bff0 con 0x7f35740f55c0 2024-02-15T18:33:11.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.637+0000 7f357bb77700 1 -- 172.21.15.38:0/3866112913 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f356005c2b0 msgr2=0x7f356005e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:11.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.637+0000 7f357bb77700 1 --2- 172.21.15.38:0/3866112913 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f356005c2b0 0x7f356005e770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f3564004630 tx=0x7f3564009b30).stop 2024-02-15T18:33:11.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.637+0000 7f357bb77700 1 -- 172.21.15.38:0/3866112913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35740f55c0 msgr2=0x7f3574109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:11.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.637+0000 7f357bb77700 1 --2- 172.21.15.38:0/3866112913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35740f55c0 0x7f3574109d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f3570002700 tx=0x7f3570009d80).stop 2024-02-15T18:33:11.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.637+0000 7f357bb77700 1 -- 172.21.15.38:0/3866112913 shutdown_connections 2024-02-15T18:33:11.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.637+0000 7f357bb77700 1 --2- 172.21.15.38:0/3866112913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35740f55c0 0x7f3574109d90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:11.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.637+0000 7f357bb77700 1 --2- 172.21.15.38:0/3866112913 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f356005c2b0 0x7f356005e770 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:11.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.637+0000 7f357bb77700 1 --2- 172.21.15.38:0/3866112913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35740f6860 0x7f357410c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:11.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.637+0000 7f357bb77700 1 -- 172.21.15.38:0/3866112913 >> 172.21.15.38:0/3866112913 conn(0x7f35740f0d40 msgr2=0x7f35740fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:11.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.637+0000 7f357bb77700 1 -- 172.21.15.38:0/3866112913 shutdown_connections 2024-02-15T18:33:11.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:11.637+0000 7f357bb77700 1 -- 172.21.15.38:0/3866112913 wait complete. 2024-02-15T18:33:11.631 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 11 2024-02-15T18:33:11.641 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-02-15T18:33:12.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:11.343098+0000 mon.smithi038 (mon.0) 533 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:33:12.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:11.351030+0000 mon.smithi038 (mon.0) 534 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:33:12.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:11.354183+0000 mon.smithi038 (mon.0) 535 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:33:12.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:11.636739+0000 mon.smithi038 (mon.0) 536 : audit [DBG] from='client.? 172.21.15.38:0/3866112913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-15T18:33:12.758 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:11.343098+0000 mon.smithi038 (mon.0) 533 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:33:12.758 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:11.351030+0000 mon.smithi038 (mon.0) 534 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:33:12.758 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:11.354183+0000 mon.smithi038 (mon.0) 535 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:33:12.758 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:11.636739+0000 mon.smithi038 (mon.0) 536 : audit [DBG] from='client.? 172.21.15.38:0/3866112913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-15T18:33:13.091 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-02-15T18:33:13.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:12.282530+0000 mgr.smithi038.khesbl (mgr.14162) 265 : cluster [DBG] pgmap v228: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 2 op/s 2024-02-15T18:33:13.721 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:12.282530+0000 mgr.smithi038.khesbl (mgr.14162) 265 : cluster [DBG] pgmap v228: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 2 op/s 2024-02-15T18:33:14.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:13.496243+0000 mon.smithi038 (mon.0) 537 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:33:14.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:13.496243+0000 mon.smithi038 (mon.0) 537 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:33:15.750 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:14.283031+0000 mgr.smithi038.khesbl (mgr.14162) 266 : cluster [DBG] pgmap v229: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-02-15T18:33:15.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:14.283031+0000 mgr.smithi038.khesbl (mgr.14162) 266 : cluster [DBG] pgmap v229: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-02-15T18:33:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.876+0000 7ff3c92e6700 1 -- 172.21.15.38:0/2584243215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3c40f77e0 msgr2=0x7ff3c40f7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.876+0000 7ff3c92e6700 1 --2- 172.21.15.38:0/2584243215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3c40f77e0 0x7ff3c40f7c40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7ff3b8004660 tx=0x7ff3b8009b30).stop 2024-02-15T18:33:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.876+0000 7ff3c92e6700 1 -- 172.21.15.38:0/2584243215 shutdown_connections 2024-02-15T18:33:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.876+0000 7ff3c92e6700 1 --2- 172.21.15.38:0/2584243215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3c40f77e0 0x7ff3c40f7c40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.876+0000 7ff3c92e6700 1 --2- 172.21.15.38:0/2584243215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3c40f6540 0x7ff3c40f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.876+0000 7ff3c92e6700 1 -- 172.21.15.38:0/2584243215 >> 172.21.15.38:0/2584243215 conn(0x7ff3c40f1ca0 msgr2=0x7ff3c40f40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.876+0000 7ff3c92e6700 1 -- 172.21.15.38:0/2584243215 shutdown_connections 2024-02-15T18:33:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.877+0000 7ff3c92e6700 1 -- 172.21.15.38:0/2584243215 wait complete. 2024-02-15T18:33:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.877+0000 7ff3c92e6700 1 Processor -- start 2024-02-15T18:33:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.877+0000 7ff3c92e6700 1 -- start start 2024-02-15T18:33:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.877+0000 7ff3c92e6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3c40f6540 0x7ff3c406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.877+0000 7ff3c92e6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3c40f77e0 0x7ff3c4069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.877+0000 7ff3c92e6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c406a380 con 0x7ff3c40f77e0 2024-02-15T18:33:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.877+0000 7ff3c92e6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c406a4c0 con 0x7ff3c40f6540 2024-02-15T18:33:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.878+0000 7ff3c27fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3c40f77e0 0x7ff3c4069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.878+0000 7ff3c2ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3c40f6540 0x7ff3c406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.878+0000 7ff3c27fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3c40f77e0 0x7ff3c4069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51402/0 (socket says 172.21.15.38:51402) 2024-02-15T18:33:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.878+0000 7ff3c27fc700 1 -- 172.21.15.38:0/3841300596 learned_addr learned my addr 172.21.15.38:0/3841300596 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:33:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.878+0000 7ff3c27fc700 1 -- 172.21.15.38:0/3841300596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3c40f6540 msgr2=0x7ff3c406b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.878+0000 7ff3c27fc700 1 --2- 172.21.15.38:0/3841300596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3c40f6540 0x7ff3c406b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.878+0000 7ff3c27fc700 1 -- 172.21.15.38:0/3841300596 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b8005040 con 0x7ff3c40f77e0 2024-02-15T18:33:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.878+0000 7ff3c27fc700 1 --2- 172.21.15.38:0/3841300596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3c40f77e0 0x7ff3c4069e20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7ff3b8009830 tx=0x7ff3b8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.878+0000 7ff3ab7fe700 1 -- 172.21.15.38:0/3841300596 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3b8018070 con 0x7ff3c40f77e0 2024-02-15T18:33:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.878+0000 7ff3ab7fe700 1 -- 172.21.15.38:0/3841300596 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3b8007ba0 con 0x7ff3c40f77e0 2024-02-15T18:33:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.878+0000 7ff3c92e6700 1 -- 172.21.15.38:0/3841300596 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c406a740 con 0x7ff3c40f77e0 2024-02-15T18:33:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.878+0000 7ff3ab7fe700 1 -- 172.21.15.38:0/3841300596 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3b801c3c0 con 0x7ff3c40f77e0 2024-02-15T18:33:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.878+0000 7ff3c92e6700 1 -- 172.21.15.38:0/3841300596 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c406aca0 con 0x7ff3c40f77e0 2024-02-15T18:33:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.879+0000 7ff3c92e6700 1 -- 172.21.15.38:0/3841300596 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c4045ad0 con 0x7ff3c40f77e0 2024-02-15T18:33:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.879+0000 7ff3ab7fe700 1 -- 172.21.15.38:0/3841300596 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7ff3b8007d10 con 0x7ff3c40f77e0 2024-02-15T18:33:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.880+0000 7ff3ab7fe700 1 --2- 172.21.15.38:0/3841300596 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff3ac05c300 0x7ff3ac05e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.880+0000 7ff3ab7fe700 1 -- 172.21.15.38:0/3841300596 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7ff3b8077b40 con 0x7ff3c40f77e0 2024-02-15T18:33:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.880+0000 7ff3c2ffd700 1 --2- 172.21.15.38:0/3841300596 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff3ac05c300 0x7ff3ac05e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.880+0000 7ff3c2ffd700 1 --2- 172.21.15.38:0/3841300596 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff3ac05c300 0x7ff3ac05e7c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7ff3c406b730 tx=0x7ff3b400b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:15.885+0000 7ff3ab7fe700 1 -- 172.21.15.38:0/3841300596 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ff3b804a7a0 con 0x7ff3c40f77e0 2024-02-15T18:33:16.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:16.106+0000 7ff3c92e6700 1 -- 172.21.15.38:0/3841300596 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7ff3c40fef50 con 0x7ff3c40f77e0 2024-02-15T18:33:16.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:16.106+0000 7ff3ab7fe700 1 -- 172.21.15.38:0/3841300596 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v11) v1 ==== 78+0+86 (secure 0 0 0) 0x7ff3b804ddc0 con 0x7ff3c40f77e0 2024-02-15T18:33:16.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:16.108+0000 7ff3c92e6700 1 -- 172.21.15.38:0/3841300596 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff3ac05c300 msgr2=0x7ff3ac05e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:16.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:16.108+0000 7ff3c92e6700 1 --2- 172.21.15.38:0/3841300596 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff3ac05c300 0x7ff3ac05e7c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7ff3c406b730 tx=0x7ff3b400b040).stop 2024-02-15T18:33:16.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:16.108+0000 7ff3c92e6700 1 -- 172.21.15.38:0/3841300596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3c40f77e0 msgr2=0x7ff3c4069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:16.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:16.108+0000 7ff3c92e6700 1 --2- 172.21.15.38:0/3841300596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3c40f77e0 0x7ff3c4069e20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7ff3b8009830 tx=0x7ff3b8009b30).stop 2024-02-15T18:33:16.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:16.108+0000 7ff3c92e6700 1 -- 172.21.15.38:0/3841300596 shutdown_connections 2024-02-15T18:33:16.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:16.108+0000 7ff3c92e6700 1 --2- 172.21.15.38:0/3841300596 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7ff3ac05c300 0x7ff3ac05e7c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:16.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:16.108+0000 7ff3c92e6700 1 --2- 172.21.15.38:0/3841300596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3c40f6540 0x7ff3c406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:16.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:16.108+0000 7ff3c92e6700 1 --2- 172.21.15.38:0/3841300596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3c40f77e0 0x7ff3c4069e20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:16.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:16.108+0000 7ff3c92e6700 1 -- 172.21.15.38:0/3841300596 >> 172.21.15.38:0/3841300596 conn(0x7ff3c40f1ca0 msgr2=0x7ff3c40fb230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:16.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:16.108+0000 7ff3c92e6700 1 -- 172.21.15.38:0/3841300596 shutdown_connections 2024-02-15T18:33:16.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:16.108+0000 7ff3c92e6700 1 -- 172.21.15.38:0/3841300596 wait complete. 2024-02-15T18:33:16.114 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-02-15T18:33:16.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:16.108205+0000 mon.smithi038 (mon.0) 538 : audit [DBG] from='client.? 172.21.15.38:0/3841300596' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-02-15T18:33:16.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:16.108205+0000 mon.smithi038 (mon.0) 538 : audit [DBG] from='client.? 172.21.15.38:0/3841300596' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-02-15T18:33:16.873 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-02-15T18:33:16.887 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-15T18:33:17.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.290+0000 7f9dd8c8d700 1 -- 172.21.15.38:0/4226205925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dd40f66d0 msgr2=0x7f9dd40f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:17.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.290+0000 7f9dd8c8d700 1 --2- 172.21.15.38:0/4226205925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dd40f66d0 0x7f9dd40f6b30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f9dbc004660 tx=0x7f9dbc009b30).stop 2024-02-15T18:33:17.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.290+0000 7f9dd8c8d700 1 -- 172.21.15.38:0/4226205925 shutdown_connections 2024-02-15T18:33:17.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.290+0000 7f9dd8c8d700 1 --2- 172.21.15.38:0/4226205925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dd40f66d0 0x7f9dd40f6b30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:17.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.290+0000 7f9dd8c8d700 1 --2- 172.21.15.38:0/4226205925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dd40f5430 0x7f9dd40f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:17.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.290+0000 7f9dd8c8d700 1 -- 172.21.15.38:0/4226205925 >> 172.21.15.38:0/4226205925 conn(0x7f9dd40f0b90 msgr2=0x7f9dd40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:17.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.291+0000 7f9dd8c8d700 1 -- 172.21.15.38:0/4226205925 shutdown_connections 2024-02-15T18:33:17.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.291+0000 7f9dd8c8d700 1 -- 172.21.15.38:0/4226205925 wait complete. 2024-02-15T18:33:17.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.291+0000 7f9dd8c8d700 1 Processor -- start 2024-02-15T18:33:17.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.291+0000 7f9dd8c8d700 1 -- start start 2024-02-15T18:33:17.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.291+0000 7f9dd8c8d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dd40f5430 0x7f9dd4106c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:17.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.291+0000 7f9dd8c8d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dd40f66d0 0x7f9dd4107170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:17.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.291+0000 7f9dd8c8d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd4107770 con 0x7f9dd40f5430 2024-02-15T18:33:17.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.291+0000 7f9dd8c8d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd41078e0 con 0x7f9dd40f66d0 2024-02-15T18:33:17.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.291+0000 7f9dd259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dd40f5430 0x7f9dd4106c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:17.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.291+0000 7f9dd259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dd40f5430 0x7f9dd4106c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51412/0 (socket says 172.21.15.38:51412) 2024-02-15T18:33:17.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.291+0000 7f9dd259c700 1 -- 172.21.15.38:0/2766181414 learned_addr learned my addr 172.21.15.38:0/2766181414 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:33:17.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.291+0000 7f9dd1d9b700 1 --2- 172.21.15.38:0/2766181414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dd40f66d0 0x7f9dd4107170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:17.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.292+0000 7f9dd259c700 1 -- 172.21.15.38:0/2766181414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dd40f66d0 msgr2=0x7f9dd4107170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:17.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.292+0000 7f9dd259c700 1 --2- 172.21.15.38:0/2766181414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dd40f66d0 0x7f9dd4107170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:17.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.292+0000 7f9dd259c700 1 -- 172.21.15.38:0/2766181414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc005040 con 0x7f9dd40f5430 2024-02-15T18:33:17.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.292+0000 7f9dd259c700 1 --2- 172.21.15.38:0/2766181414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dd40f5430 0x7f9dd4106c80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f9dc4002700 tx=0x7f9dc4009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:17.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.292+0000 7f9dcaffd700 1 -- 172.21.15.38:0/2766181414 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dc40079c0 con 0x7f9dd40f5430 2024-02-15T18:33:17.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.292+0000 7f9dd8c8d700 1 -- 172.21.15.38:0/2766181414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd410c350 con 0x7f9dd40f5430 2024-02-15T18:33:17.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.292+0000 7f9dcaffd700 1 -- 172.21.15.38:0/2766181414 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9dc400f040 con 0x7f9dd40f5430 2024-02-15T18:33:17.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.292+0000 7f9dcaffd700 1 -- 172.21.15.38:0/2766181414 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dc4014970 con 0x7f9dd40f5430 2024-02-15T18:33:17.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.292+0000 7f9dd8c8d700 1 -- 172.21.15.38:0/2766181414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd410c8e0 con 0x7f9dd40f5430 2024-02-15T18:33:17.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.293+0000 7f9dcaffd700 1 -- 172.21.15.38:0/2766181414 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f9dc401b060 con 0x7f9dd40f5430 2024-02-15T18:33:17.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.293+0000 7f9dd8c8d700 1 -- 172.21.15.38:0/2766181414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd4046330 con 0x7f9dd40f5430 2024-02-15T18:33:17.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.294+0000 7f9dcaffd700 1 --2- 172.21.15.38:0/2766181414 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9dc005c370 0x7f9dc005e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:17.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.294+0000 7f9dcaffd700 1 -- 172.21.15.38:0/2766181414 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f9dc4075d10 con 0x7f9dd40f5430 2024-02-15T18:33:17.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.294+0000 7f9dd1d9b700 1 --2- 172.21.15.38:0/2766181414 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9dc005c370 0x7f9dc005e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:17.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.294+0000 7f9dd1d9b700 1 --2- 172.21.15.38:0/2766181414 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9dc005c370 0x7f9dc005e830 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f9dbc004630 tx=0x7f9dbc009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:17.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.298+0000 7f9dcaffd700 1 -- 172.21.15.38:0/2766181414 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9dc40489d0 con 0x7f9dd40f5430 2024-02-15T18:33:17.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.453+0000 7f9dd8c8d700 1 -- 172.21.15.38:0/2766181414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9dd40fdc90 con 0x7f9dd40f5430 2024-02-15T18:33:17.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.453+0000 7f9dcaffd700 1 -- 172.21.15.38:0/2766181414 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+14098 (secure 0 0 0) 0x7f9dc404bff0 con 0x7f9dd40f5430 2024-02-15T18:33:17.445 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:33:17.445 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":43,"fsid":"7e05753e-cc2f-11ee-95ba-87774f69a715","created":"2024-02-15T18:25:33.849561+0000","modified":"2024-02-15T18:32:48.884238+0000","last_up_change":"2024-02-15T18:31:14.711602+0000","last_in_change":"2024-02-15T18:30:56.975140+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-15T18:30:09.268924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-15T18:32:45.509930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-15T18:32:46.291705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eee9713b-ff79-4169-8c04-b1def80d2045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1826650694},{"type":"v1","addr":"172.21.15.38:6803","nonce":1826650694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1826650694},{"type":"v1","addr":"172.21.15.38:6805","nonce":1826650694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1826650694},{"type":"v1","addr":"172.21.15.38:6809","nonce":1826650694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1826650694},{"type":"v1","addr":"172.21.15.38:6807","nonce":1826650694}]},"public_addr":"172.21.15.38:6803/1826650694","cluster_addr":"172.21.15.38:6805/1826650694","heartbeat_back_addr":"172.21.15.38:6809/1826650694","heartbeat_front_addr":"172.21.15.38:6807/1826650694","state":["exists","up"]},{"osd":1,"uuid":"740eed48-d34e-4e71-a859-035f01b1fdd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3796697403},{"type":"v1","addr":"172.21.15.38:6811","nonce":3796697403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3796697403},{"type":"v1","addr":"172.21.15.38:6813","nonce":3796697403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3796697403},{"type":"v1","addr":"172.21.15.38:6817","nonce":3796697403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3796697403},{"type":"v1","addr":"172.21.15.38:6815","nonce":3796697403}]},"public_addr":"172.21.15.38:6811/3796697403","cluster_addr":"172.21.15.38:6813/3796697403","heartbeat_back_addr":"172.21.15.38:6817/3796697403","heartbeat_front_addr":"172.21.15.38:6815/3796697403","state":["exists","up"]},{"osd":2,"uuid":"8a397970-64c0-47d7-94c8-77fcf9fe579d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":420320553},{"type":"v1","addr":"172.21.15.38:6819","nonce":420320553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":420320553},{"type":"v1","addr":"172.21.15.38:6821","nonce":420320553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":420320553},{"type":"v1","addr":"172.21.15.38:6825","nonce":420320553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":420320553},{"type":"v1","addr":"172.21.15.38:6823","nonce":420320553}]},"public_addr":"172.21.15.38:6819/420320553","cluster_addr":"172.21.15.38:6821/420320553","heartbeat_back_addr":"172.21.15.38:6825/420320553","heartbeat_front_addr":"172.21.15.38:6823/420320553","state":["exists","up"]},{"osd":3,"uuid":"0a590f02-56f3-43e7-894f-741f95d2facd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3071118694},{"type":"v1","addr":"172.21.15.121:6801","nonce":3071118694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3071118694},{"type":"v1","addr":"172.21.15.121:6803","nonce":3071118694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3071118694},{"type":"v1","addr":"172.21.15.121:6807","nonce":3071118694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3071118694},{"type":"v1","addr":"172.21.15.121:6805","nonce":3071118694}]},"public_addr":"172.21.15.121:6801/3071118694","cluster_addr":"172.21.15.121:6803/3071118694","heartbeat_back_addr":"172.21.15.121:6807/3071118694","heartbeat_front_addr":"172.21.15.121:6805/3071118694","state":["exists","up"]},{"osd":4,"uuid":"5e13d4f8-2233-4674-9b52-a3fb9dde34ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3016245330},{"type":"v1","addr":"172.21.15.121:6809","nonce":3016245330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3016245330},{"type":"v1","addr":"172.21.15.121:6811","nonce":3016245330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3016245330},{"type":"v1","addr":"172.21.15.121:6815","nonce":3016245330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3016245330},{"type":"v1","addr":"172.21.15.121:6813","nonce":3016245330}]},"public_addr":"172.21.15.121:6809/3016245330","cluster_addr":"172.21.15.121:6811/3016245330","heartbeat_back_addr":"172.21.15.121:6815/3016245330","heartbeat_front_addr":"172.21.15.121:6813/3016245330","state":["exists","up"]},{"osd":5,"uuid":"6cd02c52-b9f4-494e-b67e-7019e4c224af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4062288290},{"type":"v1","addr":"172.21.15.121:6817","nonce":4062288290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4062288290},{"type":"v1","addr":"172.21.15.121:6819","nonce":4062288290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4062288290},{"type":"v1","addr":"172.21.15.121:6823","nonce":4062288290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4062288290},{"type":"v1","addr":"172.21.15.121:6821","nonce":4062288290}]},"public_addr":"172.21.15.121:6817/4062288290","cluster_addr":"172.21.15.121:6819/4062288290","heartbeat_back_addr":"172.21.15.121:6823/4062288290","heartbeat_front_addr":"172.21.15.121:6821/4062288290","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:29:06.518109+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:29:40.582228+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:30:07.799238+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:30:29.057360+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:30:51.020867+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:31:13.005231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:0/3518675152":"2024-02-16T18:26:40.033295+0000","172.21.15.38:0/1774333839":"2024-02-16T18:26:40.033295+0000","172.21.15.38:6800/2095530547":"2024-02-16T18:26:40.033295+0000","172.21.15.38:0/2413778685":"2024-02-16T18:26:23.102293+0000","172.21.15.38:6801/3009942445":"2024-02-16T18:26:23.102293+0000","172.21.15.38:0/788545352":"2024-02-16T18:26:23.102293+0000","172.21.15.38:6800/3009942445":"2024-02-16T18:26:23.102293+0000","172.21.15.38:0/3436541874":"2024-02-16T18:26:02.772407+0000","172.21.15.38:0/4035950047":"2024-02-16T18:26:02.772407+0000","172.21.15.38:6801/2095530547":"2024-02-16T18:26:40.033295+0000","172.21.15.38:6800/4189944318":"2024-02-16T18:26:02.772407+0000","172.21.15.38:6801/4189944318":"2024-02-16T18:26:02.772407+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-15T18:33:17.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.455+0000 7f9dd8c8d700 1 -- 172.21.15.38:0/2766181414 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9dc005c370 msgr2=0x7f9dc005e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:17.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.455+0000 7f9dd8c8d700 1 --2- 172.21.15.38:0/2766181414 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9dc005c370 0x7f9dc005e830 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f9dbc004630 tx=0x7f9dbc009b30).stop 2024-02-15T18:33:17.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.455+0000 7f9dd8c8d700 1 -- 172.21.15.38:0/2766181414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dd40f5430 msgr2=0x7f9dd4106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:17.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.455+0000 7f9dd8c8d700 1 --2- 172.21.15.38:0/2766181414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dd40f5430 0x7f9dd4106c80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f9dc4002700 tx=0x7f9dc4009d80).stop 2024-02-15T18:33:17.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.455+0000 7f9dd8c8d700 1 -- 172.21.15.38:0/2766181414 shutdown_connections 2024-02-15T18:33:17.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.455+0000 7f9dd8c8d700 1 --2- 172.21.15.38:0/2766181414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dd40f5430 0x7f9dd4106c80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:17.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.455+0000 7f9dd8c8d700 1 --2- 172.21.15.38:0/2766181414 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9dc005c370 0x7f9dc005e830 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:17.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.455+0000 7f9dd8c8d700 1 --2- 172.21.15.38:0/2766181414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dd40f66d0 0x7f9dd4107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:17.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.455+0000 7f9dd8c8d700 1 -- 172.21.15.38:0/2766181414 >> 172.21.15.38:0/2766181414 conn(0x7f9dd40f0b90 msgr2=0x7f9dd405fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:17.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.455+0000 7f9dd8c8d700 1 -- 172.21.15.38:0/2766181414 shutdown_connections 2024-02-15T18:33:17.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.455+0000 7f9dd8c8d700 1 -- 172.21.15.38:0/2766181414 wait complete. 2024-02-15T18:33:17.459 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-15T18:30:09.268924+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-15T18:32:45.509930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-15T18:32:46.291705+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-15T18:33:17.459 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-02-15T18:33:17.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.597+0000 7f7868bac700 1 -- 172.21.15.38:0/2198375825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78640f9600 msgr2=0x7f78640fb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:17.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.597+0000 7f7868bac700 1 --2- 172.21.15.38:0/2198375825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78640f9600 0x7f78640fb9f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f7858004660 tx=0x7f7858009b30).stop 2024-02-15T18:33:17.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.597+0000 7f7868bac700 1 -- 172.21.15.38:0/2198375825 shutdown_connections 2024-02-15T18:33:17.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.597+0000 7f7868bac700 1 --2- 172.21.15.38:0/2198375825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f78640f9600 0x7f78640fb9f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:17.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.597+0000 7f7868bac700 1 --2- 172.21.15.38:0/2198375825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f786405fe30 0x7f78640f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:17.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.597+0000 7f7868bac700 1 -- 172.21.15.38:0/2198375825 >> 172.21.15.38:0/2198375825 conn(0x7f78640f0b70 msgr2=0x7f78640f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:17.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.597+0000 7f7868bac700 1 -- 172.21.15.38:0/2198375825 shutdown_connections 2024-02-15T18:33:17.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.597+0000 7f7868bac700 1 -- 172.21.15.38:0/2198375825 wait complete. 2024-02-15T18:33:17.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.597+0000 7f7868bac700 1 Processor -- start 2024-02-15T18:33:17.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.598+0000 7f7868bac700 1 -- start start 2024-02-15T18:33:17.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.598+0000 7f7868bac700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f786405fe30 0x7f7864106c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:17.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.598+0000 7f7868bac700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78640f9600 0x7f7864107180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:17.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.598+0000 7f7868bac700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7864107780 con 0x7f786405fe30 2024-02-15T18:33:17.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.598+0000 7f7868bac700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78641078f0 con 0x7f78640f9600 2024-02-15T18:33:17.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.598+0000 7f786259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f786405fe30 0x7f7864106c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:17.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.598+0000 7f786259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f786405fe30 0x7f7864106c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51434/0 (socket says 172.21.15.38:51434) 2024-02-15T18:33:17.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.598+0000 7f786259c700 1 -- 172.21.15.38:0/2145000654 learned_addr learned my addr 172.21.15.38:0/2145000654 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:33:17.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.598+0000 7f7861d9b700 1 --2- 172.21.15.38:0/2145000654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78640f9600 0x7f7864107180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:17.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.598+0000 7f786259c700 1 -- 172.21.15.38:0/2145000654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78640f9600 msgr2=0x7f7864107180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:17.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.598+0000 7f786259c700 1 --2- 172.21.15.38:0/2145000654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78640f9600 0x7f7864107180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:17.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.598+0000 7f786259c700 1 -- 172.21.15.38:0/2145000654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7858005040 con 0x7f786405fe30 2024-02-15T18:33:17.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.598+0000 7f786259c700 1 --2- 172.21.15.38:0/2145000654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f786405fe30 0x7f7864106c90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f785400e7c0 tx=0x7f785400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:17.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.598+0000 7f7852ffd700 1 -- 172.21.15.38:0/2145000654 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78540092e0 con 0x7f786405fe30 2024-02-15T18:33:17.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.598+0000 7f7868bac700 1 -- 172.21.15.38:0/2145000654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786410c360 con 0x7f786405fe30 2024-02-15T18:33:17.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.598+0000 7f7852ffd700 1 -- 172.21.15.38:0/2145000654 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f785400f040 con 0x7f786405fe30 2024-02-15T18:33:17.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.598+0000 7f7852ffd700 1 -- 172.21.15.38:0/2145000654 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78540147e0 con 0x7f786405fe30 2024-02-15T18:33:17.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.599+0000 7f7868bac700 1 -- 172.21.15.38:0/2145000654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786410c8f0 con 0x7f786405fe30 2024-02-15T18:33:17.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.599+0000 7f7868bac700 1 -- 172.21.15.38:0/2145000654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7864046330 con 0x7f786405fe30 2024-02-15T18:33:17.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.600+0000 7f7852ffd700 1 -- 172.21.15.38:0/2145000654 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f785401b030 con 0x7f786405fe30 2024-02-15T18:33:17.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.600+0000 7f7852ffd700 1 --2- 172.21.15.38:0/2145000654 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f784c05c2b0 0x7f784c05e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:17.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.601+0000 7f7852ffd700 1 -- 172.21.15.38:0/2145000654 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f7854076430 con 0x7f786405fe30 2024-02-15T18:33:17.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.601+0000 7f7861d9b700 1 --2- 172.21.15.38:0/2145000654 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f784c05c2b0 0x7f784c05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:17.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.601+0000 7f7861d9b700 1 --2- 172.21.15.38:0/2145000654 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f784c05c2b0 0x7f784c05e770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f78580043d0 tx=0x7f785800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:17.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.604+0000 7f7852ffd700 1 -- 172.21.15.38:0/2145000654 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f78540490f0 con 0x7f786405fe30 2024-02-15T18:33:17.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.762+0000 7f7868bac700 1 -- 172.21.15.38:0/2145000654 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f78640fed70 con 0x7f786405fe30 2024-02-15T18:33:17.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.763+0000 7f7852ffd700 1 -- 172.21.15.38:0/2145000654 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v43) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7854019020 con 0x7f786405fe30 2024-02-15T18:33:17.754 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-02-15T18:33:17.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.765+0000 7f7868bac700 1 -- 172.21.15.38:0/2145000654 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f784c05c2b0 msgr2=0x7f784c05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:17.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.765+0000 7f7868bac700 1 --2- 172.21.15.38:0/2145000654 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f784c05c2b0 0x7f784c05e770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f78580043d0 tx=0x7f785800b040).stop 2024-02-15T18:33:17.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.765+0000 7f7868bac700 1 -- 172.21.15.38:0/2145000654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f786405fe30 msgr2=0x7f7864106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:17.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.765+0000 7f7868bac700 1 --2- 172.21.15.38:0/2145000654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f786405fe30 0x7f7864106c90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f785400e7c0 tx=0x7f785400b040).stop 2024-02-15T18:33:17.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.765+0000 7f7868bac700 1 -- 172.21.15.38:0/2145000654 shutdown_connections 2024-02-15T18:33:17.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.765+0000 7f7868bac700 1 --2- 172.21.15.38:0/2145000654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f786405fe30 0x7f7864106c90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:17.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.765+0000 7f7868bac700 1 --2- 172.21.15.38:0/2145000654 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f784c05c2b0 0x7f784c05e770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:17.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.765+0000 7f7868bac700 1 --2- 172.21.15.38:0/2145000654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78640f9600 0x7f7864107180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:17.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.765+0000 7f7868bac700 1 -- 172.21.15.38:0/2145000654 >> 172.21.15.38:0/2145000654 conn(0x7f78640f0b70 msgr2=0x7f78640f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:17.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.765+0000 7f7868bac700 1 -- 172.21.15.38:0/2145000654 shutdown_connections 2024-02-15T18:33:17.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.765+0000 7f7868bac700 1 -- 172.21.15.38:0/2145000654 wait complete. 2024-02-15T18:33:17.767 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:16.283434+0000 mgr.smithi038.khesbl (mgr.14162) 267 : cluster [DBG] pgmap v230: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:17.767 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:17.453752+0000 mon.smithi038 (mon.0) 539 : audit [DBG] from='client.? 172.21.15.38:0/2766181414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-15T18:33:17.768 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-15T18:33:17.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:16.283434+0000 mgr.smithi038.khesbl (mgr.14162) 267 : cluster [DBG] pgmap v230: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:17.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:17.453752+0000 mon.smithi038 (mon.0) 539 : audit [DBG] from='client.? 172.21.15.38:0/2766181414' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-15T18:33:17.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.900+0000 7fa4ab975700 1 -- 172.21.15.38:0/403426499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a40f5390 msgr2=0x7fa4a40f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:17.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.900+0000 7fa4ab975700 1 --2- 172.21.15.38:0/403426499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a40f5390 0x7fa4a40f57b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fa4a0004660 tx=0x7fa4a0009b30).stop 2024-02-15T18:33:17.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.900+0000 7fa4ab975700 1 -- 172.21.15.38:0/403426499 shutdown_connections 2024-02-15T18:33:17.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.900+0000 7fa4ab975700 1 --2- 172.21.15.38:0/403426499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4a40f5ca0 0x7fa4a40f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:17.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.900+0000 7fa4ab975700 1 --2- 172.21.15.38:0/403426499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a40f5390 0x7fa4a40f57b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:17.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.900+0000 7fa4ab975700 1 -- 172.21.15.38:0/403426499 >> 172.21.15.38:0/403426499 conn(0x7fa4a40f0b70 msgr2=0x7fa4a40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:17.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.900+0000 7fa4ab975700 1 -- 172.21.15.38:0/403426499 shutdown_connections 2024-02-15T18:33:17.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.900+0000 7fa4ab975700 1 -- 172.21.15.38:0/403426499 wait complete. 2024-02-15T18:33:17.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.901+0000 7fa4ab975700 1 Processor -- start 2024-02-15T18:33:17.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.901+0000 7fa4ab975700 1 -- start start 2024-02-15T18:33:17.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.901+0000 7fa4ab975700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a40f5390 0x7fa4a40f93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:17.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.901+0000 7fa4ab975700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4a40f5ca0 0x7fa4a40f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:17.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.901+0000 7fa4ab975700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40f9960 con 0x7fa4a40f5390 2024-02-15T18:33:17.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.901+0000 7fa4ab975700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40f7f00 con 0x7fa4a40f5ca0 2024-02-15T18:33:17.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.901+0000 7fa4a9711700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a40f5390 0x7fa4a40f93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:17.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.901+0000 7fa4a9711700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a40f5390 0x7fa4a40f93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51450/0 (socket says 172.21.15.38:51450) 2024-02-15T18:33:17.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.901+0000 7fa4a9711700 1 -- 172.21.15.38:0/3107947236 learned_addr learned my addr 172.21.15.38:0/3107947236 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:33:17.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.901+0000 7fa4a9711700 1 -- 172.21.15.38:0/3107947236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4a40f5ca0 msgr2=0x7fa4a40f7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:17.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.901+0000 7fa4a9711700 1 --2- 172.21.15.38:0/3107947236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4a40f5ca0 0x7fa4a40f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:33:17.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.901+0000 7fa4a9711700 1 -- 172.21.15.38:0/3107947236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a0005040 con 0x7fa4a40f5390 2024-02-15T18:33:17.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.901+0000 7fa4a9711700 1 --2- 172.21.15.38:0/3107947236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a40f5390 0x7fa4a40f93b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fa4a0009830 tx=0x7fa4a0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:17.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.901+0000 7fa495ffb700 1 -- 172.21.15.38:0/3107947236 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4a0018070 con 0x7fa4a40f5390 2024-02-15T18:33:17.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.902+0000 7fa495ffb700 1 -- 172.21.15.38:0/3107947236 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4a0007ba0 con 0x7fa4a40f5390 2024-02-15T18:33:17.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.902+0000 7fa4ab975700 1 -- 172.21.15.38:0/3107947236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a40f8120 con 0x7fa4a40f5390 2024-02-15T18:33:17.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.902+0000 7fa495ffb700 1 -- 172.21.15.38:0/3107947236 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4a001c440 con 0x7fa4a40f5390 2024-02-15T18:33:17.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.902+0000 7fa4ab975700 1 -- 172.21.15.38:0/3107947236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a40f8680 con 0x7fa4a40f5390 2024-02-15T18:33:17.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.903+0000 7fa4ab975700 1 -- 172.21.15.38:0/3107947236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a40fdc90 con 0x7fa4a40f5390 2024-02-15T18:33:17.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.903+0000 7fa495ffb700 1 -- 172.21.15.38:0/3107947236 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fa4a001d420 con 0x7fa4a40f5390 2024-02-15T18:33:17.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.903+0000 7fa495ffb700 1 --2- 172.21.15.38:0/3107947236 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fa49005c2b0 0x7fa49005e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:17.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.904+0000 7fa495ffb700 1 -- 172.21.15.38:0/3107947236 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fa4a0077cb0 con 0x7fa4a40f5390 2024-02-15T18:33:17.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.904+0000 7fa4a8f10700 1 --2- 172.21.15.38:0/3107947236 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fa49005c2b0 0x7fa49005e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:17.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.904+0000 7fa4a8f10700 1 --2- 172.21.15.38:0/3107947236 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fa49005c2b0 0x7fa49005e770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fa498009880 tx=0x7fa498009a60).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:17.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:17.907+0000 7fa495ffb700 1 -- 172.21.15.38:0/3107947236 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fa4a004a9f0 con 0x7fa4a40f5390 2024-02-15T18:33:18.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.069+0000 7fa4ab975700 1 -- 172.21.15.38:0/3107947236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fa4a410f1e0 con 0x7fa4a40f5390 2024-02-15T18:33:18.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.069+0000 7fa495ffb700 1 -- 172.21.15.38:0/3107947236 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v43) v1 ==== 107+0+11 (secure 0 0 0) 0x7fa4a004e010 con 0x7fa4a40f5390 2024-02-15T18:33:18.061 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-02-15T18:33:18.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.071+0000 7fa4ab975700 1 -- 172.21.15.38:0/3107947236 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fa49005c2b0 msgr2=0x7fa49005e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:18.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.071+0000 7fa4ab975700 1 --2- 172.21.15.38:0/3107947236 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fa49005c2b0 0x7fa49005e770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fa498009880 tx=0x7fa498009a60).stop 2024-02-15T18:33:18.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.071+0000 7fa4ab975700 1 -- 172.21.15.38:0/3107947236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a40f5390 msgr2=0x7fa4a40f93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:18.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.071+0000 7fa4ab975700 1 --2- 172.21.15.38:0/3107947236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a40f5390 0x7fa4a40f93b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fa4a0009830 tx=0x7fa4a0009b30).stop 2024-02-15T18:33:18.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.071+0000 7fa4ab975700 1 -- 172.21.15.38:0/3107947236 shutdown_connections 2024-02-15T18:33:18.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.071+0000 7fa4ab975700 1 --2- 172.21.15.38:0/3107947236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4a40f5390 0x7fa4a40f93b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:18.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.071+0000 7fa4ab975700 1 --2- 172.21.15.38:0/3107947236 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fa49005c2b0 0x7fa49005e770 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:18.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.071+0000 7fa4ab975700 1 --2- 172.21.15.38:0/3107947236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4a40f5ca0 0x7fa4a40f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:33:18.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.071+0000 7fa4ab975700 1 -- 172.21.15.38:0/3107947236 >> 172.21.15.38:0/3107947236 conn(0x7fa4a40f0b70 msgr2=0x7fa4a405fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:18.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.071+0000 7fa4ab975700 1 -- 172.21.15.38:0/3107947236 shutdown_connections 2024-02-15T18:33:18.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.071+0000 7fa4ab975700 1 -- 172.21.15.38:0/3107947236 wait complete. 2024-02-15T18:33:18.076 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-15T18:33:18.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.208+0000 7f70c263b700 1 -- 172.21.15.38:0/3384243889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70bc05fdd0 msgr2=0x7f70bc0f9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:18.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.208+0000 7f70c263b700 1 --2- 172.21.15.38:0/3384243889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70bc05fdd0 0x7f70bc0f9170 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f70ac004660 tx=0x7f70ac009b30).stop 2024-02-15T18:33:18.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.208+0000 7f70c263b700 1 -- 172.21.15.38:0/3384243889 shutdown_connections 2024-02-15T18:33:18.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.208+0000 7f70c263b700 1 --2- 172.21.15.38:0/3384243889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70bc0f9660 0x7f70bc0fba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:18.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.208+0000 7f70c263b700 1 --2- 172.21.15.38:0/3384243889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70bc05fdd0 0x7f70bc0f9170 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:18.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.208+0000 7f70c263b700 1 -- 172.21.15.38:0/3384243889 >> 172.21.15.38:0/3384243889 conn(0x7f70bc0f0b70 msgr2=0x7f70bc0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:18.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.208+0000 7f70c263b700 1 -- 172.21.15.38:0/3384243889 shutdown_connections 2024-02-15T18:33:18.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.208+0000 7f70c263b700 1 -- 172.21.15.38:0/3384243889 wait complete. 2024-02-15T18:33:18.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.208+0000 7f70c263b700 1 Processor -- start 2024-02-15T18:33:18.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.209+0000 7f70c263b700 1 -- start start 2024-02-15T18:33:18.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.209+0000 7f70c263b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70bc05fdd0 0x7f70bc108eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:18.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.209+0000 7f70c263b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70bc0f9660 0x7f70bc1093a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:18.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.209+0000 7f70c263b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc109950 con 0x7f70bc0f9660 2024-02-15T18:33:18.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.209+0000 7f70c263b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc109a90 con 0x7f70bc05fdd0 2024-02-15T18:33:18.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.209+0000 7f70bb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70bc0f9660 0x7f70bc1093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:18.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.209+0000 7f70bb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70bc0f9660 0x7f70bc1093a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51480/0 (socket says 172.21.15.38:51480) 2024-02-15T18:33:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.209+0000 7f70bb7fe700 1 -- 172.21.15.38:0/1863159611 learned_addr learned my addr 172.21.15.38:0/1863159611 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:33:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.209+0000 7f70bbfff700 1 --2- 172.21.15.38:0/1863159611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70bc05fdd0 0x7f70bc108eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.209+0000 7f70bb7fe700 1 -- 172.21.15.38:0/1863159611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70bc05fdd0 msgr2=0x7f70bc108eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.209+0000 7f70bb7fe700 1 --2- 172.21.15.38:0/1863159611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70bc05fdd0 0x7f70bc108eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.209+0000 7f70bb7fe700 1 -- 172.21.15.38:0/1863159611 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ac005040 con 0x7f70bc0f9660 2024-02-15T18:33:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.209+0000 7f70bb7fe700 1 --2- 172.21.15.38:0/1863159611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70bc0f9660 0x7f70bc1093a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f70b000e7f0 tx=0x7f70b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.209+0000 7f70b8ff9700 1 -- 172.21.15.38:0/1863159611 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70b00092e0 con 0x7f70bc0f9660 2024-02-15T18:33:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.209+0000 7f70c263b700 1 -- 172.21.15.38:0/1863159611 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70bc10e530 con 0x7f70bc0f9660 2024-02-15T18:33:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.209+0000 7f70b8ff9700 1 -- 172.21.15.38:0/1863159611 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70b000f040 con 0x7f70bc0f9660 2024-02-15T18:33:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.209+0000 7f70b8ff9700 1 -- 172.21.15.38:0/1863159611 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70b00147e0 con 0x7f70bc0f9660 2024-02-15T18:33:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.210+0000 7f70c263b700 1 -- 172.21.15.38:0/1863159611 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70bc10eaf0 con 0x7f70bc0f9660 2024-02-15T18:33:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.210+0000 7f70c263b700 1 -- 172.21.15.38:0/1863159611 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70bc0fed70 con 0x7f70bc0f9660 2024-02-15T18:33:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.211+0000 7f70b8ff9700 1 -- 172.21.15.38:0/1863159611 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f70b001b030 con 0x7f70bc0f9660 2024-02-15T18:33:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.211+0000 7f70b8ff9700 1 --2- 172.21.15.38:0/1863159611 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f70a405c2b0 0x7f70a405e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:18.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.211+0000 7f70b8ff9700 1 -- 172.21.15.38:0/1863159611 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f70b0075d00 con 0x7f70bc0f9660 2024-02-15T18:33:18.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.211+0000 7f70bbfff700 1 --2- 172.21.15.38:0/1863159611 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f70a405c2b0 0x7f70a405e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:18.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.212+0000 7f70bbfff700 1 --2- 172.21.15.38:0/1863159611 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f70a405c2b0 0x7f70a405e770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f70ac009720 tx=0x7f70ac016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:18.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.215+0000 7f70b8ff9700 1 -- 172.21.15.38:0/1863159611 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f70b0079030 con 0x7f70bc0f9660 2024-02-15T18:33:18.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.377+0000 7f70c263b700 1 -- 172.21.15.38:0/1863159611 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f70bc10f190 con 0x7f70bc0f9660 2024-02-15T18:33:18.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.377+0000 7f70b8ff9700 1 -- 172.21.15.38:0/1863159611 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v43) v1 ==== 107+0+11 (secure 0 0 0) 0x7f70b0019070 con 0x7f70bc0f9660 2024-02-15T18:33:18.369 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-02-15T18:33:18.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.379+0000 7f70c263b700 1 -- 172.21.15.38:0/1863159611 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f70a405c2b0 msgr2=0x7f70a405e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:18.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.379+0000 7f70c263b700 1 --2- 172.21.15.38:0/1863159611 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f70a405c2b0 0x7f70a405e770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f70ac009720 tx=0x7f70ac016040).stop 2024-02-15T18:33:18.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.379+0000 7f70c263b700 1 -- 172.21.15.38:0/1863159611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70bc0f9660 msgr2=0x7f70bc1093a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:18.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.379+0000 7f70c263b700 1 --2- 172.21.15.38:0/1863159611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70bc0f9660 0x7f70bc1093a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f70b000e7f0 tx=0x7f70b000b040).stop 2024-02-15T18:33:18.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.379+0000 7f70c263b700 1 -- 172.21.15.38:0/1863159611 shutdown_connections 2024-02-15T18:33:18.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.379+0000 7f70c263b700 1 --2- 172.21.15.38:0/1863159611 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f70a405c2b0 0x7f70a405e770 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:18.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.379+0000 7f70c263b700 1 --2- 172.21.15.38:0/1863159611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70bc05fdd0 0x7f70bc108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:18.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.379+0000 7f70c263b700 1 --2- 172.21.15.38:0/1863159611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70bc0f9660 0x7f70bc1093a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:18.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.379+0000 7f70c263b700 1 -- 172.21.15.38:0/1863159611 >> 172.21.15.38:0/1863159611 conn(0x7f70bc0f0b70 msgr2=0x7f70bc0f4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:18.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.380+0000 7f70c263b700 1 -- 172.21.15.38:0/1863159611 shutdown_connections 2024-02-15T18:33:18.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.380+0000 7f70c263b700 1 -- 172.21.15.38:0/1863159611 wait complete. 2024-02-15T18:33:18.386 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-15T18:33:18.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.516+0000 7f28e0c0b700 1 -- 172.21.15.38:0/855720861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28dc05fe00 msgr2=0x7f28dc0fb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:18.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.516+0000 7f28e0c0b700 1 --2- 172.21.15.38:0/855720861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28dc05fe00 0x7f28dc0fb5b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f28cc004660 tx=0x7f28cc009b30).stop 2024-02-15T18:33:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.516+0000 7f28e0c0b700 1 -- 172.21.15.38:0/855720861 shutdown_connections 2024-02-15T18:33:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.516+0000 7f28e0c0b700 1 --2- 172.21.15.38:0/855720861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28dc0fbaa0 0x7f28dc0fde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.516+0000 7f28e0c0b700 1 --2- 172.21.15.38:0/855720861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28dc05fe00 0x7f28dc0fb5b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.516+0000 7f28e0c0b700 1 -- 172.21.15.38:0/855720861 >> 172.21.15.38:0/855720861 conn(0x7f28dc0f0bd0 msgr2=0x7f28dc0f2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.516+0000 7f28e0c0b700 1 -- 172.21.15.38:0/855720861 shutdown_connections 2024-02-15T18:33:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.516+0000 7f28e0c0b700 1 -- 172.21.15.38:0/855720861 wait complete. 2024-02-15T18:33:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.517+0000 7f28e0c0b700 1 Processor -- start 2024-02-15T18:33:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.517+0000 7f28e0c0b700 1 -- start start 2024-02-15T18:33:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.517+0000 7f28e0c0b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28dc05fe00 0x7f28dc108eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.517+0000 7f28e0c0b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28dc0fbaa0 0x7f28dc1093a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.517+0000 7f28e0c0b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc109950 con 0x7f28dc0fbaa0 2024-02-15T18:33:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.517+0000 7f28e0c0b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc109a90 con 0x7f28dc05fe00 2024-02-15T18:33:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.517+0000 7f28d9d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28dc0fbaa0 0x7f28dc1093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.517+0000 7f28d9d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28dc0fbaa0 0x7f28dc1093a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51508/0 (socket says 172.21.15.38:51508) 2024-02-15T18:33:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.517+0000 7f28d9d9b700 1 -- 172.21.15.38:0/3107616441 learned_addr learned my addr 172.21.15.38:0/3107616441 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:33:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.517+0000 7f28da59c700 1 --2- 172.21.15.38:0/3107616441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28dc05fe00 0x7f28dc108eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.517+0000 7f28d9d9b700 1 -- 172.21.15.38:0/3107616441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28dc05fe00 msgr2=0x7f28dc108eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.517+0000 7f28d9d9b700 1 --2- 172.21.15.38:0/3107616441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28dc05fe00 0x7f28dc108eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.517+0000 7f28d9d9b700 1 -- 172.21.15.38:0/3107616441 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc005040 con 0x7f28dc0fbaa0 2024-02-15T18:33:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.517+0000 7f28da59c700 1 --2- 172.21.15.38:0/3107616441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28dc05fe00 0x7f28dc108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:33:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.517+0000 7f28d9d9b700 1 --2- 172.21.15.38:0/3107616441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28dc0fbaa0 0x7f28dc1093a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f28d00026d0 tx=0x7f28d0009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.518+0000 7f28caffd700 1 -- 172.21.15.38:0/3107616441 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28d0009320 con 0x7f28dc0fbaa0 2024-02-15T18:33:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.518+0000 7f28caffd700 1 -- 172.21.15.38:0/3107616441 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28d000f040 con 0x7f28dc0fbaa0 2024-02-15T18:33:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.518+0000 7f28e0c0b700 1 -- 172.21.15.38:0/3107616441 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28dc10e530 con 0x7f28dc0fbaa0 2024-02-15T18:33:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.518+0000 7f28caffd700 1 -- 172.21.15.38:0/3107616441 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28d0014990 con 0x7f28dc0fbaa0 2024-02-15T18:33:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.518+0000 7f28e0c0b700 1 -- 172.21.15.38:0/3107616441 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28dc10eaf0 con 0x7f28dc0fbaa0 2024-02-15T18:33:18.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.519+0000 7f28e0c0b700 1 -- 172.21.15.38:0/3107616441 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28dc027a00 con 0x7f28dc0fbaa0 2024-02-15T18:33:18.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.519+0000 7f28caffd700 1 -- 172.21.15.38:0/3107616441 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f28d001b060 con 0x7f28dc0fbaa0 2024-02-15T18:33:18.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.519+0000 7f28caffd700 1 --2- 172.21.15.38:0/3107616441 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f28c405c370 0x7f28c405e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:18.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.519+0000 7f28caffd700 1 -- 172.21.15.38:0/3107616441 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f28d0075d30 con 0x7f28dc0fbaa0 2024-02-15T18:33:18.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.519+0000 7f28da59c700 1 --2- 172.21.15.38:0/3107616441 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f28c405c370 0x7f28c405e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:18.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.520+0000 7f28da59c700 1 --2- 172.21.15.38:0/3107616441 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f28c405c370 0x7f28c405e830 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f28cc004e30 tx=0x7f28cc009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:18.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.523+0000 7f28caffd700 1 -- 172.21.15.38:0/3107616441 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f28d0048a70 con 0x7f28dc0fbaa0 2024-02-15T18:33:18.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.723+0000 7f28e0c0b700 1 -- 172.21.15.38:0/3107616441 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28dc100fb0 con 0x7f28dc0fbaa0 2024-02-15T18:33:18.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.723+0000 7f28caffd700 1 -- 172.21.15.38:0/3107616441 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2942 (secure 0 0 0) 0x7f28d004c090 con 0x7f28dc0fbaa0 2024-02-15T18:33:18.716 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:33:18.717 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14446,"name":"cephfs.smithi038.hxoggl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.38:6827/3822880235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3822880235},{"type":"v1","addr":"172.21.15.38:6827","nonce":3822880235}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10},{"gid":14456,"name":"cephfs.smithi038.qgkzgn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.38:6829/703030231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":703030231},{"type":"v1","addr":"172.21.15.38:6829","nonce":703030231}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":11}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:33:02.882026+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":24261,"mds_1":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi121.mvumlh","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6825/1422707571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1422707571},{"type":"v1","addr":"172.21.15.121:6825","nonce":1422707571}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_24275":{"gid":24275,"name":"cephfs.smithi121.ovuzki","rank":1,"incarnation":7,"state":"up:active","state_seq":2,"addr":"172.21.15.121:6827/562926845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":562926845},{"type":"v1","addr":"172.21.15.121:6827","nonce":562926845}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:33:18.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.725+0000 7f28e0c0b700 1 -- 172.21.15.38:0/3107616441 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f28c405c370 msgr2=0x7f28c405e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.725+0000 7f28e0c0b700 1 --2- 172.21.15.38:0/3107616441 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f28c405c370 0x7f28c405e830 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f28cc004e30 tx=0x7f28cc009b30).stop 2024-02-15T18:33:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.725+0000 7f28e0c0b700 1 -- 172.21.15.38:0/3107616441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28dc0fbaa0 msgr2=0x7f28dc1093a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.725+0000 7f28e0c0b700 1 --2- 172.21.15.38:0/3107616441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28dc0fbaa0 0x7f28dc1093a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f28d00026d0 tx=0x7f28d0009de0).stop 2024-02-15T18:33:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.725+0000 7f28e0c0b700 1 -- 172.21.15.38:0/3107616441 shutdown_connections 2024-02-15T18:33:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.725+0000 7f28e0c0b700 1 --2- 172.21.15.38:0/3107616441 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f28c405c370 0x7f28c405e830 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.725+0000 7f28e0c0b700 1 --2- 172.21.15.38:0/3107616441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28dc05fe00 0x7f28dc108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.725+0000 7f28e0c0b700 1 --2- 172.21.15.38:0/3107616441 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28dc0fbaa0 0x7f28dc1093a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.725+0000 7f28e0c0b700 1 -- 172.21.15.38:0/3107616441 >> 172.21.15.38:0/3107616441 conn(0x7f28dc0f0bd0 msgr2=0x7f28dc0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.725+0000 7f28e0c0b700 1 -- 172.21.15.38:0/3107616441 shutdown_connections 2024-02-15T18:33:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:18.725+0000 7f28e0c0b700 1 -- 172.21.15.38:0/3107616441 wait complete. 2024-02-15T18:33:18.718 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 11 2024-02-15T18:33:18.728 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:17.763708+0000 mon.smithi038 (mon.0) 540 : audit [DBG] from='client.? 172.21.15.38:0/2145000654' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-15T18:33:18.728 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:18.069838+0000 mon.smithi038 (mon.0) 541 : audit [DBG] from='client.? 172.21.15.38:0/3107947236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-15T18:33:18.728 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:18.377803+0000 mon.smithi038 (mon.0) 542 : audit [DBG] from='client.? 172.21.15.38:0/1863159611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-15T18:33:18.729 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 10, 'max_mds': 2, 'flags': 18} 2024-02-15T18:33:18.729 INFO:teuthology.run_tasks:Running task kclient... 2024-02-15T18:33:18.751 INFO:tasks.kclient:Mounting kernel clients... 2024-02-15T18:33:18.751 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-02-15T18:33:18.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:18.752 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-02-15T18:33:18.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:18.771 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-02-15T18:33:18.840 INFO:teuthology.orchestra.run.smithi038.stderr:Cannot find device "ceph-brx" 2024-02-15T18:33:18.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-15T18:33:18.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:18.842 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-02-15T18:33:18.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:17.763708+0000 mon.smithi038 (mon.0) 540 : audit [DBG] from='client.? 172.21.15.38:0/2145000654' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-15T18:33:18.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:18.069838+0000 mon.smithi038 (mon.0) 541 : audit [DBG] from='client.? 172.21.15.38:0/3107947236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-15T18:33:18.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:18.377803+0000 mon.smithi038 (mon.0) 542 : audit [DBG] from='client.? 172.21.15.38:0/1863159611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-15T18:33:18.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:18.866 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link delete ceph-brx 2024-02-15T18:33:18.944 INFO:teuthology.orchestra.run.smithi121.stderr:Cannot find device "ceph-brx" 2024-02-15T18:33:18.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-15T18:33:18.945 INFO:tasks.kclient:client.0 config is {} 2024-02-15T18:33:18.946 DEBUG:teuthology.orchestra.run.smithi038:> ceph fs ls 2024-02-15T18:33:19.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.070+0000 7fe367e60700 1 -- 172.21.15.38:0/274141239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3600f64e0 msgr2=0x7fe3600f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:19.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.070+0000 7fe367e60700 1 --2- 172.21.15.38:0/274141239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3600f64e0 0x7fe3600f6900 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fe35c004660 tx=0x7fe35c009b30).stop 2024-02-15T18:33:19.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.070+0000 7fe367e60700 1 -- 172.21.15.38:0/274141239 shutdown_connections 2024-02-15T18:33:19.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.070+0000 7fe367e60700 1 --2- 172.21.15.38:0/274141239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3600f7780 0x7fe3600f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:19.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.070+0000 7fe367e60700 1 --2- 172.21.15.38:0/274141239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3600f64e0 0x7fe3600f6900 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:19.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.070+0000 7fe367e60700 1 -- 172.21.15.38:0/274141239 >> 172.21.15.38:0/274141239 conn(0x7fe3600f1c40 msgr2=0x7fe3600f4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:19.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.070+0000 7fe367e60700 1 -- 172.21.15.38:0/274141239 shutdown_connections 2024-02-15T18:33:19.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.070+0000 7fe367e60700 1 -- 172.21.15.38:0/274141239 wait complete. 2024-02-15T18:33:19.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.071+0000 7fe367e60700 1 Processor -- start 2024-02-15T18:33:19.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.071+0000 7fe367e60700 1 -- start start 2024-02-15T18:33:19.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.071+0000 7fe367e60700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3600f64e0 0x7fe360106db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:19.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.072+0000 7fe367e60700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3600f7780 0x7fe3601072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:19.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.072+0000 7fe367e60700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe360107850 con 0x7fe3600f7780 2024-02-15T18:33:19.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.072+0000 7fe367e60700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe360107990 con 0x7fe3600f64e0 2024-02-15T18:33:19.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.072+0000 7fe3653fb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3600f7780 0x7fe3601072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:19.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.072+0000 7fe3653fb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3600f7780 0x7fe3601072a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51540/0 (socket says 172.21.15.38:51540) 2024-02-15T18:33:19.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.072+0000 7fe3653fb700 1 -- 172.21.15.38:0/4197739218 learned_addr learned my addr 172.21.15.38:0/4197739218 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:33:19.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.072+0000 7fe365bfc700 1 --2- 172.21.15.38:0/4197739218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3600f64e0 0x7fe360106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:19.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.072+0000 7fe3653fb700 1 -- 172.21.15.38:0/4197739218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3600f64e0 msgr2=0x7fe360106db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:19.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.072+0000 7fe3653fb700 1 --2- 172.21.15.38:0/4197739218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3600f64e0 0x7fe360106db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:19.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.072+0000 7fe3653fb700 1 -- 172.21.15.38:0/4197739218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c005040 con 0x7fe3600f7780 2024-02-15T18:33:19.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.072+0000 7fe3653fb700 1 --2- 172.21.15.38:0/4197739218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3600f7780 0x7fe3601072a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fe35000c790 tx=0x7fe350007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:19.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.072+0000 7fe3567fc700 1 -- 172.21.15.38:0/4197739218 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe35000e680 con 0x7fe3600f7780 2024-02-15T18:33:19.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.072+0000 7fe3567fc700 1 -- 172.21.15.38:0/4197739218 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe35000ecb0 con 0x7fe3600f7780 2024-02-15T18:33:19.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.072+0000 7fe367e60700 1 -- 172.21.15.38:0/4197739218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36010c430 con 0x7fe3600f7780 2024-02-15T18:33:19.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.072+0000 7fe3567fc700 1 -- 172.21.15.38:0/4197739218 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe350013e10 con 0x7fe3600f7780 2024-02-15T18:33:19.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.072+0000 7fe367e60700 1 -- 172.21.15.38:0/4197739218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36010c9f0 con 0x7fe3600f7780 2024-02-15T18:33:19.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.073+0000 7fe367e60700 1 -- 172.21.15.38:0/4197739218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe360027a00 con 0x7fe3600f7780 2024-02-15T18:33:19.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.074+0000 7fe3567fc700 1 -- 172.21.15.38:0/4197739218 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fe35001a060 con 0x7fe3600f7780 2024-02-15T18:33:19.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.074+0000 7fe3567fc700 1 --2- 172.21.15.38:0/4197739218 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fe34c05c2b0 0x7fe34c05e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:19.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.074+0000 7fe3567fc700 1 -- 172.21.15.38:0/4197739218 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fe350076140 con 0x7fe3600f7780 2024-02-15T18:33:19.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.074+0000 7fe365bfc700 1 --2- 172.21.15.38:0/4197739218 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fe34c05c2b0 0x7fe34c05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:19.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.075+0000 7fe365bfc700 1 --2- 172.21.15.38:0/4197739218 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fe34c05c2b0 0x7fe34c05e770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fe35c009490 tx=0x7fe35c009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:19.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.078+0000 7fe3567fc700 1 -- 172.21.15.38:0/4197739218 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe350048e00 con 0x7fe3600f7780 2024-02-15T18:33:19.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.282+0000 7fe367e60700 1 -- 172.21.15.38:0/4197739218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe3600586e0 con 0x7fe3600f7780 2024-02-15T18:33:19.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.282+0000 7fe3567fc700 1 -- 172.21.15.38:0/4197739218 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v11) v1 ==== 53+0+83 (secure 0 0 0) 0x7fe35004c420 con 0x7fe3600f7780 2024-02-15T18:33:19.274 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-02-15T18:33:19.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.284+0000 7fe367e60700 1 -- 172.21.15.38:0/4197739218 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fe34c05c2b0 msgr2=0x7fe34c05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:19.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.284+0000 7fe367e60700 1 --2- 172.21.15.38:0/4197739218 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fe34c05c2b0 0x7fe34c05e770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fe35c009490 tx=0x7fe35c009b30).stop 2024-02-15T18:33:19.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.284+0000 7fe367e60700 1 -- 172.21.15.38:0/4197739218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3600f7780 msgr2=0x7fe3601072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:19.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.284+0000 7fe367e60700 1 --2- 172.21.15.38:0/4197739218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3600f7780 0x7fe3601072a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fe35000c790 tx=0x7fe350007d70).stop 2024-02-15T18:33:19.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.284+0000 7fe367e60700 1 -- 172.21.15.38:0/4197739218 shutdown_connections 2024-02-15T18:33:19.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.284+0000 7fe367e60700 1 --2- 172.21.15.38:0/4197739218 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fe34c05c2b0 0x7fe34c05e770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:19.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.284+0000 7fe367e60700 1 --2- 172.21.15.38:0/4197739218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3600f64e0 0x7fe360106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:19.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.284+0000 7fe367e60700 1 --2- 172.21.15.38:0/4197739218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3600f7780 0x7fe3601072a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:19.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.284+0000 7fe367e60700 1 -- 172.21.15.38:0/4197739218 >> 172.21.15.38:0/4197739218 conn(0x7fe3600f1c40 msgr2=0x7fe3600fb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:19.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.284+0000 7fe367e60700 1 -- 172.21.15.38:0/4197739218 shutdown_connections 2024-02-15T18:33:19.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:19.284+0000 7fe367e60700 1 -- 172.21.15.38:0/4197739218 wait complete. 2024-02-15T18:33:19.287 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-15T18:33:19.287 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-15T18:33:19.287 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-02-15T18:33:19.287 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-15T18:33:19.287 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-15T18:33:19.287 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-15T18:33:19.287 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-15T18:33:19.287 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-15T18:33:19.287 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-15T18:33:19.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:19.287 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-02-15T18:33:19.306 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-15T18:33:19.306 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-15T18:33:19.306 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-02-15T18:33:19.306 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-02-15T18:33:19.307 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-02-15T18:33:19.307 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-02-15T18:33:19.307 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-15T18:33:19.307 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-02-15T18:33:19.307 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-02-15T18:33:19.307 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-15T18:33:19.307 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-02-15T18:33:19.307 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-02-15T18:33:19.307 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-15T18:33:19.307 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-02-15T18:33:19.307 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-02-15T18:33:19.307 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-02-15T18:33:19.307 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 41984sec preferred_lft 41984sec 2024-02-15T18:33:19.307 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-02-15T18:33:19.307 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-02-15T18:33:19.308 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-15T18:33:19.308 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-02-15T18:33:19.308 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-02-15T18:33:19.308 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-15T18:33:19.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:19.308 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-02-15T18:33:19.308 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-02-15T18:33:19.309 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add name ceph-brx type bridge 2024-02-15T18:33:19.309 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr flush dev ceph-brx 2024-02-15T18:33:19.309 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set ceph-brx up 2024-02-15T18:33:19.309 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-15T18:33:19.309 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-02-15T18:33:19.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:19.480 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-15T18:33:19.511 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-02-15T18:33:19.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:19.512 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-02-15T18:33:19.589 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-02-15T18:33:19.589 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-15T18:33:19.589 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-02-15T18:33:19.589 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-02-15T18:33:19.589 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-15T18:33:19.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:19.590 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-02-15T18:33:19.590 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-02-15T18:33:19.590 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-02-15T18:33:19.590 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-02-15T18:33:19.590 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-02-15T18:33:19.590 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-02-15T18:33:19.670 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:18.283723+0000 mgr.smithi038.khesbl (mgr.14162) 268 : cluster [DBG] pgmap v231: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:19.670 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:18.724178+0000 mon.smithi038 (mon.0) 543 : audit [DBG] from='client.? 172.21.15.38:0/3107616441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-15T18:33:19.670 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:19.123435+0000 mon.smithi038 (mon.0) 544 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:33:19.670 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:19.128678+0000 mon.smithi038 (mon.0) 545 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:33:19.670 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:19.283191+0000 mon.smithi038 (mon.0) 546 : audit [DBG] from='client.? 172.21.15.38:0/4197739218' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-15T18:33:19.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:18.283723+0000 mgr.smithi038.khesbl (mgr.14162) 268 : cluster [DBG] pgmap v231: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:19.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:18.724178+0000 mon.smithi038 (mon.0) 543 : audit [DBG] from='client.? 172.21.15.38:0/3107616441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-15T18:33:19.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:19.123435+0000 mon.smithi038 (mon.0) 544 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:33:19.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:19.128678+0000 mon.smithi038 (mon.0) 545 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:33:19.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:19.283191+0000 mon.smithi038 (mon.0) 546 : audit [DBG] from='client.? 172.21.15.38:0/4197739218' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-15T18:33:19.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:19.920 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-02-15T18:33:19.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:19.980 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list-id 2024-02-15T18:33:20.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:20.038 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-02-15T18:33:20.038 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-02-15T18:33:20.038 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-15T18:33:20.038 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-15T18:33:20.038 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-02-15T18:33:20.135 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-15T18:33:20.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:20.135 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-02-15T18:33:20.135 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-02-15T18:33:20.135 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-15T18:33:20.135 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-15T18:33:20.135 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-15T18:33:20.136 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-15T18:33:20.136 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-15T18:33:20.136 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-02-15T18:33:20.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:20.372 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-02-15T18:33:20.372 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-02-15T18:33:20.372 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-02-15T18:33:20.372 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set dev brx.0 master ceph-brx 2024-02-15T18:33:20.372 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-02-15T18:33:20.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:20.470 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p /home/ubuntu/cephtest/mnt.0 2024-02-15T18:33:20.488 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-02-15T18:33:20.488 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-02-15T18:33:20.589 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-02-15T18:33:20.590 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-02-15T18:33:20.610 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-02-15T18:33:20.611 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-02-15T18:33:20.614 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-15T18:33:20.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:20.614 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-15T18:33:20.646 INFO:tasks.kclient:client.1 config is {} 2024-02-15T18:33:20.646 DEBUG:teuthology.orchestra.run.smithi121:> ceph fs ls 2024-02-15T18:33:20.846 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.852+0000 7fab2284c700 1 -- 172.21.15.121:0/564098518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab1c0f5ce0 msgr2=0x7fab1c0f6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:20.846 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.852+0000 7fab2284c700 1 --2- 172.21.15.121:0/564098518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab1c0f5ce0 0x7fab1c0f6710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fab10004660 tx=0x7fab10009b30).stop 2024-02-15T18:33:20.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.852+0000 7fab2284c700 1 -- 172.21.15.121:0/564098518 shutdown_connections 2024-02-15T18:33:20.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.852+0000 7fab2284c700 1 --2- 172.21.15.121:0/564098518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab1c0f5ce0 0x7fab1c0f6710 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:20.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.852+0000 7fab2284c700 1 --2- 172.21.15.121:0/564098518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab1c0f53d0 0x7fab1c0f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:20.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.852+0000 7fab2284c700 1 -- 172.21.15.121:0/564098518 >> 172.21.15.121:0/564098518 conn(0x7fab1c0f0bd0 msgr2=0x7fab1c0f2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:20.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.852+0000 7fab2284c700 1 -- 172.21.15.121:0/564098518 shutdown_connections 2024-02-15T18:33:20.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.852+0000 7fab2284c700 1 -- 172.21.15.121:0/564098518 wait complete. 2024-02-15T18:33:20.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.852+0000 7fab2284c700 1 Processor -- start 2024-02-15T18:33:20.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.853+0000 7fab2284c700 1 -- start start 2024-02-15T18:33:20.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.853+0000 7fab2284c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab1c0f53d0 0x7fab1c106da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:20.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.853+0000 7fab2284c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab1c0f5ce0 0x7fab1c107290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:20.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.853+0000 7fab2284c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c107890 con 0x7fab1c0f5ce0 2024-02-15T18:33:20.848 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.853+0000 7fab2284c700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c1079d0 con 0x7fab1c0f53d0 2024-02-15T18:33:20.848 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.853+0000 7fab1bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab1c0f53d0 0x7fab1c106da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:20.848 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.853+0000 7fab1bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab1c0f53d0 0x7fab1c106da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:38558/0 (socket says 172.21.15.121:38558) 2024-02-15T18:33:20.848 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.853+0000 7fab1bfff700 1 -- 172.21.15.121:0/234185068 learned_addr learned my addr 172.21.15.121:0/234185068 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-02-15T18:33:20.848 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.853+0000 7fab1b7fe700 1 --2- 172.21.15.121:0/234185068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab1c0f5ce0 0x7fab1c107290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:20.848 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.853+0000 7fab1bfff700 1 -- 172.21.15.121:0/234185068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab1c0f5ce0 msgr2=0x7fab1c107290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:20.848 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.853+0000 7fab1bfff700 1 --2- 172.21.15.121:0/234185068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab1c0f5ce0 0x7fab1c107290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:20.848 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.853+0000 7fab1bfff700 1 -- 172.21.15.121:0/234185068 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab10005040 con 0x7fab1c0f53d0 2024-02-15T18:33:20.848 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.853+0000 7fab1bfff700 1 --2- 172.21.15.121:0/234185068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab1c0f53d0 0x7fab1c106da0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fab0c0026d0 tx=0x7fab0c009de0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:20.848 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.853+0000 7fab18ff9700 1 -- 172.21.15.121:0/234185068 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab0c009320 con 0x7fab1c0f53d0 2024-02-15T18:33:20.848 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.854+0000 7fab18ff9700 1 -- 172.21.15.121:0/234185068 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab0c00f040 con 0x7fab1c0f53d0 2024-02-15T18:33:20.848 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.854+0000 7fab2284c700 1 -- 172.21.15.121:0/234185068 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1c10c3e0 con 0x7fab1c0f53d0 2024-02-15T18:33:20.848 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.854+0000 7fab18ff9700 1 -- 172.21.15.121:0/234185068 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab0c014910 con 0x7fab1c0f53d0 2024-02-15T18:33:20.848 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.854+0000 7fab2284c700 1 -- 172.21.15.121:0/234185068 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c10c9a0 con 0x7fab1c0f53d0 2024-02-15T18:33:20.849 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.855+0000 7fab2284c700 1 -- 172.21.15.121:0/234185068 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1c0fdcd0 con 0x7fab1c0f53d0 2024-02-15T18:33:20.849 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.855+0000 7fab18ff9700 1 -- 172.21.15.121:0/234185068 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fab0c0077b0 con 0x7fab1c0f53d0 2024-02-15T18:33:20.850 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.855+0000 7fab18ff9700 1 --2- 172.21.15.121:0/234185068 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fab0405c2b0 0x7fab0405e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:20.850 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.855+0000 7fab18ff9700 1 -- 172.21.15.121:0/234185068 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fab0c0764c0 con 0x7fab1c0f53d0 2024-02-15T18:33:20.850 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.855+0000 7fab1b7fe700 1 --2- 172.21.15.121:0/234185068 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fab0405c2b0 0x7fab0405e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:20.850 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.856+0000 7fab1b7fe700 1 --2- 172.21.15.121:0/234185068 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fab0405c2b0 0x7fab0405e770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fab10004e30 tx=0x7fab10009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:20.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:20.859+0000 7fab18ff9700 1 -- 172.21.15.121:0/234185068 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fab0c049180 con 0x7fab1c0f53d0 2024-02-15T18:33:21.037 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:21.044+0000 7fab2284c700 1 -- 172.21.15.121:0/234185068 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fab1c10d040 con 0x7fab1c0f53d0 2024-02-15T18:33:21.037 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:21.045+0000 7fab18ff9700 1 -- 172.21.15.121:0/234185068 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v11) v1 ==== 53+0+83 (secure 0 0 0) 0x7fab0c04c7a0 con 0x7fab1c0f53d0 2024-02-15T18:33:21.037 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-02-15T18:33:21.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:21.046+0000 7fab2284c700 1 -- 172.21.15.121:0/234185068 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fab0405c2b0 msgr2=0x7fab0405e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:21.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:21.047+0000 7fab2284c700 1 --2- 172.21.15.121:0/234185068 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fab0405c2b0 0x7fab0405e770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fab10004e30 tx=0x7fab10009b30).stop 2024-02-15T18:33:21.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:21.047+0000 7fab2284c700 1 -- 172.21.15.121:0/234185068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab1c0f53d0 msgr2=0x7fab1c106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:21.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:21.047+0000 7fab2284c700 1 --2- 172.21.15.121:0/234185068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab1c0f53d0 0x7fab1c106da0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fab0c0026d0 tx=0x7fab0c009de0).stop 2024-02-15T18:33:21.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:21.047+0000 7fab2284c700 1 -- 172.21.15.121:0/234185068 shutdown_connections 2024-02-15T18:33:21.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:21.047+0000 7fab2284c700 1 --2- 172.21.15.121:0/234185068 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fab0405c2b0 0x7fab0405e770 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:21.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:21.047+0000 7fab2284c700 1 --2- 172.21.15.121:0/234185068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab1c0f53d0 0x7fab1c106da0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:21.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:21.047+0000 7fab2284c700 1 --2- 172.21.15.121:0/234185068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab1c0f5ce0 0x7fab1c107290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:21.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:21.047+0000 7fab2284c700 1 -- 172.21.15.121:0/234185068 >> 172.21.15.121:0/234185068 conn(0x7fab1c0f0bd0 msgr2=0x7fab1c05f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:21.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:21.047+0000 7fab2284c700 1 -- 172.21.15.121:0/234185068 shutdown_connections 2024-02-15T18:33:21.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-02-15T18:33:21.047+0000 7fab2284c700 1 -- 172.21.15.121:0/234185068 wait complete. 2024-02-15T18:33:21.049 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-15T18:33:21.049 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-15T18:33:21.050 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2024-02-15T18:33:21.050 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-15T18:33:21.050 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-15T18:33:21.050 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-15T18:33:21.050 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-15T18:33:21.050 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-15T18:33:21.050 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-15T18:33:21.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:21.050 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2024-02-15T18:33:21.069 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-15T18:33:21.069 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-15T18:33:21.069 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-02-15T18:33:21.069 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-02-15T18:33:21.069 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-02-15T18:33:21.069 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-02-15T18:33:21.069 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-15T18:33:21.069 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2024-02-15T18:33:21.069 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2024-02-15T18:33:21.069 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-15T18:33:21.069 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2024-02-15T18:33:21.070 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2024-02-15T18:33:21.070 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-15T18:33:21.070 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2024-02-15T18:33:21.070 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-15T18:33:21.070 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2024-02-15T18:33:21.070 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-15T18:33:21.070 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42083sec preferred_lft 42083sec 2024-02-15T18:33:21.070 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2024-02-15T18:33:21.070 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-02-15T18:33:21.070 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-15T18:33:21.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:21.071 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2024-02-15T18:33:21.071 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-02-15T18:33:21.071 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add name ceph-brx type bridge 2024-02-15T18:33:21.071 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip addr flush dev ceph-brx 2024-02-15T18:33:21.071 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set ceph-brx up 2024-02-15T18:33:21.071 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-15T18:33:21.071 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-02-15T18:33:21.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:21.229 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-15T18:33:21.257 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-02-15T18:33:21.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:21.258 DEBUG:teuthology.orchestra.run.smithi121:> route 2024-02-15T18:33:21.340 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2024-02-15T18:33:21.340 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-15T18:33:21.340 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-15T18:33:21.340 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-15T18:33:21.340 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-15T18:33:21.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:21.341 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2024-02-15T18:33:21.341 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-02-15T18:33:21.341 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-15T18:33:21.341 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-15T18:33:21.341 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-15T18:33:21.341 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-02-15T18:33:21.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:21.634 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-02-15T18:33:21.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:21.652 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list-id 2024-02-15T18:33:21.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:21.710 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2024-02-15T18:33:21.710 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-02-15T18:33:21.710 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-15T18:33:21.710 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-02-15T18:33:21.710 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-02-15T18:33:21.788 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:20.284112+0000 mgr.smithi038.khesbl (mgr.14162) 269 : cluster [DBG] pgmap v232: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:21.788 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:21.046302+0000 mon.smithi121 (mon.1) 16 : audit [DBG] from='client.? 172.21.15.121:0/234185068' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-15T18:33:21.803 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-02-15T18:33:21.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:21.803 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2024-02-15T18:33:21.803 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-02-15T18:33:21.804 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-02-15T18:33:21.804 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-15T18:33:21.804 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-02-15T18:33:21.804 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-02-15T18:33:21.804 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-02-15T18:33:21.804 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-02-15T18:33:22.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:20.284112+0000 mgr.smithi038.khesbl (mgr.14162) 269 : cluster [DBG] pgmap v232: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:22.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:21.046302+0000 mon.smithi121 (mon.1) 16 : audit [DBG] from='client.? 172.21.15.121:0/234185068' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-15T18:33:22.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:22.069 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2024-02-15T18:33:22.069 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-02-15T18:33:22.069 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2024-02-15T18:33:22.069 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set dev brx.0 master ceph-brx 2024-02-15T18:33:22.069 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-02-15T18:33:22.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:22.165 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p /home/ubuntu/cephtest/mnt.1 2024-02-15T18:33:22.182 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-02-15T18:33:22.182 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-02-15T18:33:22.278 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-02-15T18:33:22.278 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=1,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-02-15T18:33:22.298 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-02-15T18:33:22.298 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,name=1,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-02-15T18:33:22.302 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-15T18:33:22.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:33:22.302 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-15T18:33:22.329 INFO:teuthology.run_tasks:Running task print... 2024-02-15T18:33:22.344 INFO:teuthology.task.print:**** done client 2024-02-15T18:33:22.344 INFO:teuthology.run_tasks:Running task parallel... 2024-02-15T18:33:22.368 INFO:teuthology.task.parallel:starting parallel... 2024-02-15T18:33:22.368 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-02-15T18:33:22.368 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-15T18:33:22.369 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-02-15T18:33:22.369 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-02-15T18:33:22.369 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-02-15T18:33:22.369 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-02-15T18:33:22.372 INFO:tasks.workunit:Pulling workunits from ref bedfc49c6a823fb0ff75911e476e1f3cb464ad3e 2024-02-15T18:33:22.373 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-15T18:33:22.373 INFO:tasks.workunit:timeout=3h 2024-02-15T18:33:22.373 INFO:tasks.workunit:cleanup=True 2024-02-15T18:33:22.373 DEBUG:teuthology.orchestra.run.smithi038:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-15T18:33:22.397 INFO:teuthology.orchestra.run.smithi038.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-02-15T18:33:22.397 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-02-15T18:33:22.397 INFO:teuthology.orchestra.run.smithi038.stdout:Device: a1h/161d Inode: 1 Links: 2 2024-02-15T18:33:22.397 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-15T18:33:22.397 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:cephfs_t:s0 2024-02-15T18:33:22.397 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-02-15T18:33:22.398 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-02-15 18:32:53.912412829 +0000 2024-02-15T18:33:22.398 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-02-15 18:33:20.649405182 +0000 2024-02-15T18:33:22.398 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-02-15 18:32:53.912412829 +0000 2024-02-15T18:33:22.398 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-02-15T18:33:22.398 DEBUG:teuthology.orchestra.run.smithi038:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-02-15T18:33:22.480 DEBUG:teuthology.orchestra.run.smithi121:> stat -- /home/ubuntu/cephtest/mnt.1 2024-02-15T18:33:22.500 INFO:teuthology.orchestra.run.smithi121.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-02-15T18:33:22.500 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 1 Blocks: 0 IO Block: 65536 directory 2024-02-15T18:33:22.500 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 7dh/125d Inode: 1 Links: 3 2024-02-15T18:33:22.500 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-15T18:33:22.500 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:cephfs_t:s0 2024-02-15T18:33:22.500 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-02-15T18:33:22.500 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-02-15 18:33:22.476370307 +0000 2024-02-15T18:33:22.500 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-02-15 18:33:22.476370307 +0000 2024-02-15T18:33:22.501 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-02-15 18:32:53.912412829 +0000 2024-02-15T18:33:22.501 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-02-15T18:33:22.501 DEBUG:teuthology.orchestra.run.smithi121:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-02-15T18:33:22.572 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ljflores/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bedfc49c6a823fb0ff75911e476e1f3cb464ad3e 2024-02-15T18:33:22.572 DEBUG:teuthology.orchestra.run.smithi121:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://github.com/ljflores/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout bedfc49c6a823fb0ff75911e476e1f3cb464ad3e 2024-02-15T18:33:22.592 INFO:tasks.workunit.client.0.smithi038.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-15T18:33:22.630 INFO:tasks.workunit.client.1.smithi121.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-02-15T18:33:23.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:22.284851+0000 mgr.smithi038.khesbl (mgr.14162) 270 : cluster [DBG] pgmap v233: 65 pgs: 65 active+clean; 3.8 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-15T18:33:23.942 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:22.284851+0000 mgr.smithi038.khesbl (mgr.14162) 270 : cluster [DBG] pgmap v233: 65 pgs: 65 active+clean; 3.8 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-15T18:33:24.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.930+0000 7efcc4a36700 1 -- 172.21.15.38:0/2950100786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcc00f8610 msgr2=0x7efcc00f4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:24.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.930+0000 7efcc4a36700 1 --2- 172.21.15.38:0/2950100786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcc00f8610 0x7efcc00f4820 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7efcb4004660 tx=0x7efcb4009b30).stop 2024-02-15T18:33:24.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.930+0000 7efcc4a36700 1 -- 172.21.15.38:0/2950100786 shutdown_connections 2024-02-15T18:33:24.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.930+0000 7efcc4a36700 1 --2- 172.21.15.38:0/2950100786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcc00f8610 0x7efcc00f4820 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:24.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.930+0000 7efcc4a36700 1 --2- 172.21.15.38:0/2950100786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcc00f7d00 0x7efcc00f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:24.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.930+0000 7efcc4a36700 1 -- 172.21.15.38:0/2950100786 >> 172.21.15.38:0/2950100786 conn(0x7efcc00f0540 msgr2=0x7efcc00f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:24.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.930+0000 7efcc4a36700 1 -- 172.21.15.38:0/2950100786 shutdown_connections 2024-02-15T18:33:24.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.930+0000 7efcc4a36700 1 -- 172.21.15.38:0/2950100786 wait complete. 2024-02-15T18:33:24.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.931+0000 7efcc4a36700 1 Processor -- start 2024-02-15T18:33:24.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.931+0000 7efcc4a36700 1 -- start start 2024-02-15T18:33:24.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.931+0000 7efcc4a36700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcc00f7d00 0x7efcc0107020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:24.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.931+0000 7efcc4a36700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcc00f8610 0x7efcc0107510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.931+0000 7efcc4a36700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc0107ac0 con 0x7efcc00f8610 2024-02-15T18:33:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.931+0000 7efcc4a36700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc0107c00 con 0x7efcc00f7d00 2024-02-15T18:33:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.931+0000 7efcbdd9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcc00f8610 0x7efcc0107510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.931+0000 7efcbdd9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcc00f8610 0x7efcc0107510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33894/0 (socket says 172.21.15.38:33894) 2024-02-15T18:33:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.931+0000 7efcbdd9b700 1 -- 172.21.15.38:0/767290640 learned_addr learned my addr 172.21.15.38:0/767290640 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:33:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.931+0000 7efcbe59c700 1 --2- 172.21.15.38:0/767290640 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcc00f7d00 0x7efcc0107020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.931+0000 7efcbdd9b700 1 -- 172.21.15.38:0/767290640 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcc00f7d00 msgr2=0x7efcc0107020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.931+0000 7efcbdd9b700 1 --2- 172.21.15.38:0/767290640 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcc00f7d00 0x7efcc0107020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.931+0000 7efcbdd9b700 1 -- 172.21.15.38:0/767290640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb4005040 con 0x7efcc00f8610 2024-02-15T18:33:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.931+0000 7efcbe59c700 1 --2- 172.21.15.38:0/767290640 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcc00f7d00 0x7efcc0107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:33:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.931+0000 7efcbdd9b700 1 --2- 172.21.15.38:0/767290640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcc00f8610 0x7efcc0107510 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7efcb4009830 tx=0x7efcb4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.932+0000 7efcaeffd700 1 -- 172.21.15.38:0/767290640 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcb4018070 con 0x7efcc00f8610 2024-02-15T18:33:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.932+0000 7efcaeffd700 1 -- 172.21.15.38:0/767290640 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efcb4007d70 con 0x7efcc00f8610 2024-02-15T18:33:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.932+0000 7efcc4a36700 1 -- 172.21.15.38:0/767290640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc0108540 con 0x7efcc00f8610 2024-02-15T18:33:24.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.932+0000 7efcaeffd700 1 -- 172.21.15.38:0/767290640 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcb401c620 con 0x7efcc00f8610 2024-02-15T18:33:24.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.932+0000 7efcc4a36700 1 -- 172.21.15.38:0/767290640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc0108aa0 con 0x7efcc00f8610 2024-02-15T18:33:24.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.933+0000 7efcc4a36700 1 -- 172.21.15.38:0/767290640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc0045ad0 con 0x7efcc00f8610 2024-02-15T18:33:24.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.933+0000 7efcaeffd700 1 -- 172.21.15.38:0/767290640 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7efcb401d980 con 0x7efcc00f8610 2024-02-15T18:33:24.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.933+0000 7efcaeffd700 1 --2- 172.21.15.38:0/767290640 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7efca805c2b0 0x7efca805e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:24.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.933+0000 7efcaeffd700 1 -- 172.21.15.38:0/767290640 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7efcb4077ce0 con 0x7efcc00f8610 2024-02-15T18:33:24.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.933+0000 7efcbe59c700 1 --2- 172.21.15.38:0/767290640 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7efca805c2b0 0x7efca805e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:24.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.934+0000 7efcbe59c700 1 --2- 172.21.15.38:0/767290640 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7efca805c2b0 0x7efca805e770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7efcb00041d0 tx=0x7efcb0009040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:24.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:24.937+0000 7efcaeffd700 1 -- 172.21.15.38:0/767290640 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7efcb404a9c0 con 0x7efcc00f8610 2024-02-15T18:33:25.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:25.095+0000 7efcc4a36700 1 -- 172.21.15.38:0/767290640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7efcc0027a00 con 0x7efcc00f8610 2024-02-15T18:33:25.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:25.096+0000 7efcaeffd700 1 -- 172.21.15.38:0/767290640 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v16)=0 v16) v1 ==== 155+0+0 (secure 0 0 0) 0x7efcb404dfe0 con 0x7efcc00f8610 2024-02-15T18:33:25.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:25.098+0000 7efcc4a36700 1 -- 172.21.15.38:0/767290640 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7efca805c2b0 msgr2=0x7efca805e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:25.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:25.098+0000 7efcc4a36700 1 --2- 172.21.15.38:0/767290640 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7efca805c2b0 0x7efca805e770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7efcb00041d0 tx=0x7efcb0009040).stop 2024-02-15T18:33:25.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:25.098+0000 7efcc4a36700 1 -- 172.21.15.38:0/767290640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcc00f8610 msgr2=0x7efcc0107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:25.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:25.098+0000 7efcc4a36700 1 --2- 172.21.15.38:0/767290640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcc00f8610 0x7efcc0107510 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7efcb4009830 tx=0x7efcb4009b30).stop 2024-02-15T18:33:25.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:25.098+0000 7efcc4a36700 1 -- 172.21.15.38:0/767290640 shutdown_connections 2024-02-15T18:33:25.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:25.098+0000 7efcc4a36700 1 --2- 172.21.15.38:0/767290640 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7efca805c2b0 0x7efca805e770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:25.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:25.098+0000 7efcc4a36700 1 --2- 172.21.15.38:0/767290640 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcc00f7d00 0x7efcc0107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:25.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:25.098+0000 7efcc4a36700 1 --2- 172.21.15.38:0/767290640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcc00f8610 0x7efcc0107510 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:25.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:25.098+0000 7efcc4a36700 1 -- 172.21.15.38:0/767290640 >> 172.21.15.38:0/767290640 conn(0x7efcc00f0540 msgr2=0x7efcc00fb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:25.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:25.098+0000 7efcc4a36700 1 -- 172.21.15.38:0/767290640 shutdown_connections 2024-02-15T18:33:25.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:25.098+0000 7efcc4a36700 1 -- 172.21.15.38:0/767290640 wait complete. 2024-02-15T18:33:25.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:24.285309+0000 mgr.smithi038.khesbl (mgr.14162) 271 : cluster [DBG] pgmap v234: 65 pgs: 65 active+clean; 4.1 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-02-15T18:33:25.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:24.285309+0000 mgr.smithi038.khesbl (mgr.14162) 271 : cluster [DBG] pgmap v234: 65 pgs: 65 active+clean; 4.1 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-02-15T18:33:25.885 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-02-15T18:33:27.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:26.285772+0000 mgr.smithi038.khesbl (mgr.14162) 272 : cluster [DBG] pgmap v235: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-02-15T18:33:27.957 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:26.285772+0000 mgr.smithi038.khesbl (mgr.14162) 272 : cluster [DBG] pgmap v235: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-02-15T18:33:28.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.251+0000 7fd06116e700 1 -- 172.21.15.38:0/1035060851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd05c060310 msgr2=0x7fd05c0f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:28.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.251+0000 7fd06116e700 1 --2- 172.21.15.38:0/1035060851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd05c060310 0x7fd05c0f9110 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fd04c004660 tx=0x7fd04c009b30).stop 2024-02-15T18:33:28.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.251+0000 7fd06116e700 1 -- 172.21.15.38:0/1035060851 shutdown_connections 2024-02-15T18:33:28.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.251+0000 7fd06116e700 1 --2- 172.21.15.38:0/1035060851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd05c0f9690 0x7fd05c0fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:28.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.251+0000 7fd06116e700 1 --2- 172.21.15.38:0/1035060851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd05c060310 0x7fd05c0f9110 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:28.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.251+0000 7fd06116e700 1 -- 172.21.15.38:0/1035060851 >> 172.21.15.38:0/1035060851 conn(0x7fd05c0f0b90 msgr2=0x7fd05c0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:28.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.251+0000 7fd06116e700 1 -- 172.21.15.38:0/1035060851 shutdown_connections 2024-02-15T18:33:28.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.251+0000 7fd06116e700 1 -- 172.21.15.38:0/1035060851 wait complete. 2024-02-15T18:33:28.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.251+0000 7fd06116e700 1 Processor -- start 2024-02-15T18:33:28.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.252+0000 7fd06116e700 1 -- start start 2024-02-15T18:33:28.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.252+0000 7fd06116e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd05c060310 0x7fd05c106e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:28.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.252+0000 7fd06116e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd05c0f9690 0x7fd05c107300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:28.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.252+0000 7fd06116e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c107900 con 0x7fd05c060310 2024-02-15T18:33:28.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.252+0000 7fd06116e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c108110 con 0x7fd05c0f9690 2024-02-15T18:33:28.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.252+0000 7fd05bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd05c060310 0x7fd05c106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:28.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.252+0000 7fd05bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd05c060310 0x7fd05c106e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33912/0 (socket says 172.21.15.38:33912) 2024-02-15T18:33:28.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.252+0000 7fd05bfff700 1 -- 172.21.15.38:0/548181838 learned_addr learned my addr 172.21.15.38:0/548181838 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:33:28.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.252+0000 7fd05b7fe700 1 --2- 172.21.15.38:0/548181838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd05c0f9690 0x7fd05c107300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:28.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.252+0000 7fd05bfff700 1 -- 172.21.15.38:0/548181838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd05c0f9690 msgr2=0x7fd05c107300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:28.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.252+0000 7fd05bfff700 1 --2- 172.21.15.38:0/548181838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd05c0f9690 0x7fd05c107300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:28.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.252+0000 7fd05bfff700 1 -- 172.21.15.38:0/548181838 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c005040 con 0x7fd05c060310 2024-02-15T18:33:28.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.252+0000 7fd05bfff700 1 --2- 172.21.15.38:0/548181838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd05c060310 0x7fd05c106e10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fd04c000c00 tx=0x7fd04c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:28.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.252+0000 7fd058ff9700 1 -- 172.21.15.38:0/548181838 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd04c019070 con 0x7fd05c060310 2024-02-15T18:33:28.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.253+0000 7fd058ff9700 1 -- 172.21.15.38:0/548181838 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd04c007e90 con 0x7fd05c060310 2024-02-15T18:33:28.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.253+0000 7fd06116e700 1 -- 172.21.15.38:0/548181838 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd05c108330 con 0x7fd05c060310 2024-02-15T18:33:28.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.253+0000 7fd058ff9700 1 -- 172.21.15.38:0/548181838 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd04c014d50 con 0x7fd05c060310 2024-02-15T18:33:28.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.253+0000 7fd06116e700 1 -- 172.21.15.38:0/548181838 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd05c108890 con 0x7fd05c060310 2024-02-15T18:33:28.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.254+0000 7fd06116e700 1 -- 172.21.15.38:0/548181838 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05c059100 con 0x7fd05c060310 2024-02-15T18:33:28.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.254+0000 7fd058ff9700 1 -- 172.21.15.38:0/548181838 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fd04c025070 con 0x7fd05c060310 2024-02-15T18:33:28.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.254+0000 7fd058ff9700 1 --2- 172.21.15.38:0/548181838 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd04405c300 0x7fd04405e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:28.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.254+0000 7fd058ff9700 1 -- 172.21.15.38:0/548181838 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fd04c076da0 con 0x7fd05c060310 2024-02-15T18:33:28.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.254+0000 7fd05b7fe700 1 --2- 172.21.15.38:0/548181838 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd04405c300 0x7fd04405e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:28.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.255+0000 7fd05b7fe700 1 --2- 172.21.15.38:0/548181838 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd04405c300 0x7fd04405e7c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fd050009fd0 tx=0x7fd05000b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:28.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.259+0000 7fd058ff9700 1 -- 172.21.15.38:0/548181838 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fd04c049a60 con 0x7fd05c060310 2024-02-15T18:33:28.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.421+0000 7fd06116e700 1 -- 172.21.15.38:0/548181838 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fd05c045ad0 con 0x7fd05c060310 2024-02-15T18:33:28.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.421+0000 7fd058ff9700 1 -- 172.21.15.38:0/548181838 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v16)=0 v16) v1 ==== 163+0+0 (secure 0 0 0) 0x7fd04c04d080 con 0x7fd05c060310 2024-02-15T18:33:28.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.423+0000 7fd06116e700 1 -- 172.21.15.38:0/548181838 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd04405c300 msgr2=0x7fd04405e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:28.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.423+0000 7fd06116e700 1 --2- 172.21.15.38:0/548181838 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd04405c300 0x7fd04405e7c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fd050009fd0 tx=0x7fd05000b040).stop 2024-02-15T18:33:28.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.423+0000 7fd06116e700 1 -- 172.21.15.38:0/548181838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd05c060310 msgr2=0x7fd05c106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:28.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.423+0000 7fd06116e700 1 --2- 172.21.15.38:0/548181838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd05c060310 0x7fd05c106e10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fd04c000c00 tx=0x7fd04c00b040).stop 2024-02-15T18:33:28.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.423+0000 7fd06116e700 1 -- 172.21.15.38:0/548181838 shutdown_connections 2024-02-15T18:33:28.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.423+0000 7fd06116e700 1 --2- 172.21.15.38:0/548181838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd05c060310 0x7fd05c106e10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:28.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.423+0000 7fd06116e700 1 --2- 172.21.15.38:0/548181838 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd04405c300 0x7fd04405e7c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:28.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.423+0000 7fd06116e700 1 --2- 172.21.15.38:0/548181838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd05c0f9690 0x7fd05c107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:28.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.423+0000 7fd06116e700 1 -- 172.21.15.38:0/548181838 >> 172.21.15.38:0/548181838 conn(0x7fd05c0f0b90 msgr2=0x7fd05c0f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:28.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.423+0000 7fd06116e700 1 -- 172.21.15.38:0/548181838 shutdown_connections 2024-02-15T18:33:28.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:28.423+0000 7fd06116e700 1 -- 172.21.15.38:0/548181838 wait complete. 2024-02-15T18:33:29.387 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'ceph config set global log_to_journald false --force' 2024-02-15T18:33:29.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:28.286174+0000 mgr.smithi038.khesbl (mgr.14162) 273 : cluster [DBG] pgmap v236: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-15T18:33:29.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:28.286174+0000 mgr.smithi038.khesbl (mgr.14162) 273 : cluster [DBG] pgmap v236: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-15T18:33:31.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.684+0000 7f087a04c700 1 -- 172.21.15.38:0/2336555306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08740f66e0 msgr2=0x7f08740f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:31.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.684+0000 7f087a04c700 1 --2- 172.21.15.38:0/2336555306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08740f66e0 0x7f08740f6b00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f0864004660 tx=0x7f0864009b30).stop 2024-02-15T18:33:31.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.684+0000 7f087a04c700 1 -- 172.21.15.38:0/2336555306 shutdown_connections 2024-02-15T18:33:31.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.684+0000 7f087a04c700 1 --2- 172.21.15.38:0/2336555306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08740f7980 0x7f08740f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:31.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.684+0000 7f087a04c700 1 --2- 172.21.15.38:0/2336555306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08740f66e0 0x7f08740f6b00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:31.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.684+0000 7f087a04c700 1 -- 172.21.15.38:0/2336555306 >> 172.21.15.38:0/2336555306 conn(0x7f08740f1e40 msgr2=0x7f08740f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:31.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.684+0000 7f087a04c700 1 -- 172.21.15.38:0/2336555306 shutdown_connections 2024-02-15T18:33:31.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.684+0000 7f087a04c700 1 -- 172.21.15.38:0/2336555306 wait complete. 2024-02-15T18:33:31.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.685+0000 7f087a04c700 1 Processor -- start 2024-02-15T18:33:31.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.685+0000 7f087a04c700 1 -- start start 2024-02-15T18:33:31.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.685+0000 7f087a04c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08740f66e0 0x7f0874107030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:31.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.685+0000 7f087a04c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08740f7980 0x7f0874107520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:31.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.685+0000 7f087a04c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0874107ad0 con 0x7f08740f7980 2024-02-15T18:33:31.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.685+0000 7f087a04c700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0874107c10 con 0x7f08740f66e0 2024-02-15T18:33:31.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.685+0000 7f0872ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08740f7980 0x7f0874107520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:31.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.686+0000 7f08737fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08740f66e0 0x7f0874107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:31.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.686+0000 7f0872ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08740f7980 0x7f0874107520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33924/0 (socket says 172.21.15.38:33924) 2024-02-15T18:33:31.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.686+0000 7f0872ffd700 1 -- 172.21.15.38:0/2707099059 learned_addr learned my addr 172.21.15.38:0/2707099059 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:33:31.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.686+0000 7f08737fe700 1 --2- 172.21.15.38:0/2707099059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08740f66e0 0x7f0874107030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.38:44820/0 (socket says 172.21.15.38:44820) 2024-02-15T18:33:31.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.686+0000 7f0872ffd700 1 -- 172.21.15.38:0/2707099059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08740f66e0 msgr2=0x7f0874107030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:31.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.686+0000 7f0872ffd700 1 --2- 172.21.15.38:0/2707099059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08740f66e0 0x7f0874107030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:31.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.686+0000 7f0872ffd700 1 -- 172.21.15.38:0/2707099059 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0864005040 con 0x7f08740f7980 2024-02-15T18:33:31.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.686+0000 7f0872ffd700 1 --2- 172.21.15.38:0/2707099059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08740f7980 0x7f0874107520 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f086800e7f0 tx=0x7f086800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:31.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.686+0000 7f085bfff700 1 -- 172.21.15.38:0/2707099059 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08680092e0 con 0x7f08740f7980 2024-02-15T18:33:31.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.686+0000 7f085bfff700 1 -- 172.21.15.38:0/2707099059 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f086800f040 con 0x7f08740f7980 2024-02-15T18:33:31.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.686+0000 7f087a04c700 1 -- 172.21.15.38:0/2707099059 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08741085b0 con 0x7f08740f7980 2024-02-15T18:33:31.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.686+0000 7f085bfff700 1 -- 172.21.15.38:0/2707099059 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08680147e0 con 0x7f08740f7980 2024-02-15T18:33:31.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.686+0000 7f087a04c700 1 -- 172.21.15.38:0/2707099059 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0874108b70 con 0x7f08740f7980 2024-02-15T18:33:31.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.687+0000 7f087a04c700 1 -- 172.21.15.38:0/2707099059 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0874045ad0 con 0x7f08740f7980 2024-02-15T18:33:31.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.688+0000 7f085bfff700 1 -- 172.21.15.38:0/2707099059 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f086801b030 con 0x7f08740f7980 2024-02-15T18:33:31.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.688+0000 7f085bfff700 1 --2- 172.21.15.38:0/2707099059 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f085c05c2b0 0x7f085c05e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:31.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.688+0000 7f085bfff700 1 -- 172.21.15.38:0/2707099059 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f0868075cc0 con 0x7f08740f7980 2024-02-15T18:33:31.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.688+0000 7f08737fe700 1 --2- 172.21.15.38:0/2707099059 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f085c05c2b0 0x7f085c05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:31.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.688+0000 7f08737fe700 1 --2- 172.21.15.38:0/2707099059 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f085c05c2b0 0x7f085c05e770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f0864009720 tx=0x7f0864016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:31.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.692+0000 7f085bfff700 1 -- 172.21.15.38:0/2707099059 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f08680489d0 con 0x7f08740f7980 2024-02-15T18:33:31.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:30.286604+0000 mgr.smithi038.khesbl (mgr.14162) 274 : cluster [DBG] pgmap v237: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-15T18:33:31.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.854+0000 7f087a04c700 1 -- 172.21.15.38:0/2707099059 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f0874027a00 con 0x7f08740f7980 2024-02-15T18:33:31.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.860+0000 7f085bfff700 1 -- 172.21.15.38:0/2707099059 <== mon.0 v2:172.21.15.38:3300/0 7 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f0868004510 con 0x7f08740f7980 2024-02-15T18:33:31.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.860+0000 7f085bfff700 1 -- 172.21.15.38:0/2707099059 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v17)=0 v17) v1 ==== 135+0+0 (secure 0 0 0) 0x7f0868019070 con 0x7f08740f7980 2024-02-15T18:33:31.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.862+0000 7f087a04c700 1 -- 172.21.15.38:0/2707099059 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f085c05c2b0 msgr2=0x7f085c05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:31.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.862+0000 7f087a04c700 1 --2- 172.21.15.38:0/2707099059 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f085c05c2b0 0x7f085c05e770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f0864009720 tx=0x7f0864016040).stop 2024-02-15T18:33:31.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.862+0000 7f087a04c700 1 -- 172.21.15.38:0/2707099059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08740f7980 msgr2=0x7f0874107520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:31.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.862+0000 7f087a04c700 1 --2- 172.21.15.38:0/2707099059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08740f7980 0x7f0874107520 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f086800e7f0 tx=0x7f086800b040).stop 2024-02-15T18:33:31.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.862+0000 7f087a04c700 1 -- 172.21.15.38:0/2707099059 shutdown_connections 2024-02-15T18:33:31.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.862+0000 7f087a04c700 1 --2- 172.21.15.38:0/2707099059 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f085c05c2b0 0x7f085c05e770 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:31.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.862+0000 7f087a04c700 1 --2- 172.21.15.38:0/2707099059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08740f66e0 0x7f0874107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:31.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.862+0000 7f087a04c700 1 --2- 172.21.15.38:0/2707099059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08740f7980 0x7f0874107520 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:31.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.862+0000 7f087a04c700 1 -- 172.21.15.38:0/2707099059 >> 172.21.15.38:0/2707099059 conn(0x7f08740f1e40 msgr2=0x7f08740fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:31.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.863+0000 7f087a04c700 1 -- 172.21.15.38:0/2707099059 shutdown_connections 2024-02-15T18:33:31.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:31.863+0000 7f087a04c700 1 -- 172.21.15.38:0/2707099059 wait complete. 2024-02-15T18:33:31.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:30.286604+0000 mgr.smithi038.khesbl (mgr.14162) 274 : cluster [DBG] pgmap v237: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-15T18:33:32.804 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-02-15T18:33:33.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:31.860259+0000 mon.smithi038 (mon.0) 547 : audit [INF] from='client.? 172.21.15.38:0/2707099059' entity='client.admin' 2024-02-15T18:33:33.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:31.863521+0000 mon.smithi038 (mon.0) 548 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:33:33.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:32.247726+0000 mon.smithi038 (mon.0) 549 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:33:33.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:31.860259+0000 mon.smithi038 (mon.0) 547 : audit [INF] from='client.? 172.21.15.38:0/2707099059' entity='client.admin' 2024-02-15T18:33:33.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:31.863521+0000 mon.smithi038 (mon.0) 548 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:33:33.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:32.247726+0000 mon.smithi038 (mon.0) 549 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:33:34.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:32.286982+0000 mgr.smithi038.khesbl (mgr.14162) 275 : cluster [DBG] pgmap v238: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-15T18:33:34.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:32.286982+0000 mgr.smithi038.khesbl (mgr.14162) 275 : cluster [DBG] pgmap v238: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-15T18:33:35.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.217+0000 7f26e0682700 1 -- 172.21.15.38:0/1097121922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26d80f9820 msgr2=0x7f26d80fbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:35.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.217+0000 7f26e0682700 1 --2- 172.21.15.38:0/1097121922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26d80f9820 0x7f26d80fbc70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f26c8004660 tx=0x7f26c8009b30).stop 2024-02-15T18:33:35.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.217+0000 7f26e0682700 1 -- 172.21.15.38:0/1097121922 shutdown_connections 2024-02-15T18:33:35.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.217+0000 7f26e0682700 1 --2- 172.21.15.38:0/1097121922 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26d80f9820 0x7f26d80fbc70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.217+0000 7f26e0682700 1 --2- 172.21.15.38:0/1097121922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26d8060310 0x7f26d80f92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.217+0000 7f26e0682700 1 -- 172.21.15.38:0/1097121922 >> 172.21.15.38:0/1097121922 conn(0x7f26d80f0d40 msgr2=0x7f26d80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:35.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.217+0000 7f26e0682700 1 -- 172.21.15.38:0/1097121922 shutdown_connections 2024-02-15T18:33:35.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.218+0000 7f26e0682700 1 -- 172.21.15.38:0/1097121922 wait complete. 2024-02-15T18:33:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.219+0000 7f26e0682700 1 Processor -- start 2024-02-15T18:33:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.219+0000 7f26e0682700 1 -- start start 2024-02-15T18:33:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.219+0000 7f26e0682700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26d8060310 0x7f26d8109200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.219+0000 7f26e0682700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26d80f9820 0x7f26d81096f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.219+0000 7f26e0682700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d8109ca0 con 0x7f26d80f9820 2024-02-15T18:33:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.219+0000 7f26e0682700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d8109de0 con 0x7f26d8060310 2024-02-15T18:33:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.219+0000 7f26de41e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26d8060310 0x7f26d8109200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.219+0000 7f26ddc1d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26d80f9820 0x7f26d81096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.219+0000 7f26de41e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26d8060310 0x7f26d8109200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.38:33076/0 (socket says 172.21.15.38:33076) 2024-02-15T18:33:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.219+0000 7f26de41e700 1 -- 172.21.15.38:0/1678407947 learned_addr learned my addr 172.21.15.38:0/1678407947 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:33:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.219+0000 7f26ddc1d700 1 -- 172.21.15.38:0/1678407947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26d8060310 msgr2=0x7f26d8109200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.219+0000 7f26ddc1d700 1 --2- 172.21.15.38:0/1678407947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26d8060310 0x7f26d8109200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.219+0000 7f26ddc1d700 1 -- 172.21.15.38:0/1678407947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c8005040 con 0x7f26d80f9820 2024-02-15T18:33:35.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.219+0000 7f26de41e700 1 --2- 172.21.15.38:0/1678407947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26d8060310 0x7f26d8109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:33:35.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.219+0000 7f26ddc1d700 1 --2- 172.21.15.38:0/1678407947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26d80f9820 0x7f26d81096f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f26d80600e0 tx=0x7f26c800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:35.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.220+0000 7f26ceffd700 1 -- 172.21.15.38:0/1678407947 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26c8019070 con 0x7f26d80f9820 2024-02-15T18:33:35.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.220+0000 7f26ceffd700 1 -- 172.21.15.38:0/1678407947 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f26c8007e80 con 0x7f26d80f9820 2024-02-15T18:33:35.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.220+0000 7f26e0682700 1 -- 172.21.15.38:0/1678407947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d810a720 con 0x7f26d80f9820 2024-02-15T18:33:35.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.220+0000 7f26ceffd700 1 -- 172.21.15.38:0/1678407947 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26c8014e30 con 0x7f26d80f9820 2024-02-15T18:33:35.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.220+0000 7f26e0682700 1 -- 172.21.15.38:0/1678407947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d810ac80 con 0x7f26d80f9820 2024-02-15T18:33:35.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.221+0000 7f26e0682700 1 -- 172.21.15.38:0/1678407947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d8045ad0 con 0x7f26d80f9820 2024-02-15T18:33:35.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.221+0000 7f26ceffd700 1 -- 172.21.15.38:0/1678407947 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f26c8025070 con 0x7f26d80f9820 2024-02-15T18:33:35.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.221+0000 7f26ceffd700 1 --2- 172.21.15.38:0/1678407947 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f26c405c2b0 0x7f26c405e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:35.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.222+0000 7f26ceffd700 1 -- 172.21.15.38:0/1678407947 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f26c8076b50 con 0x7f26d80f9820 2024-02-15T18:33:35.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.222+0000 7f26de41e700 1 --2- 172.21.15.38:0/1678407947 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f26c405c2b0 0x7f26c405e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:35.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.222+0000 7f26de41e700 1 --2- 172.21.15.38:0/1678407947 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f26c405c2b0 0x7f26c405e770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f26d4007a30 tx=0x7f26d400b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:35.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.226+0000 7f26ceffd700 1 -- 172.21.15.38:0/1678407947 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f26c8049890 con 0x7f26d80f9820 2024-02-15T18:33:35.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.418+0000 7f26e0682700 1 -- 172.21.15.38:0/1678407947 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1", "target": ["mon-mgr", ""]}) v1 -- 0x7f26d80f5880 con 0x7f26c405c2b0 2024-02-15T18:33:35.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.428+0000 7f26ceffd700 1 -- 172.21.15.38:0/1678407947 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f26d80f5880 con 0x7f26c405c2b0 2024-02-15T18:33:35.419 INFO:teuthology.orchestra.run.smithi038.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:33:35.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.430+0000 7f26e0682700 1 -- 172.21.15.38:0/1678407947 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f26c405c2b0 msgr2=0x7f26c405e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:35.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.430+0000 7f26e0682700 1 --2- 172.21.15.38:0/1678407947 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f26c405c2b0 0x7f26c405e770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f26d4007a30 tx=0x7f26d400b040).stop 2024-02-15T18:33:35.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.430+0000 7f26e0682700 1 -- 172.21.15.38:0/1678407947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26d80f9820 msgr2=0x7f26d81096f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:35.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.430+0000 7f26e0682700 1 --2- 172.21.15.38:0/1678407947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26d80f9820 0x7f26d81096f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f26d80600e0 tx=0x7f26c800b040).stop 2024-02-15T18:33:35.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.430+0000 7f26e0682700 1 -- 172.21.15.38:0/1678407947 shutdown_connections 2024-02-15T18:33:35.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.430+0000 7f26e0682700 1 --2- 172.21.15.38:0/1678407947 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f26c405c2b0 0x7f26c405e770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.430+0000 7f26e0682700 1 --2- 172.21.15.38:0/1678407947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26d8060310 0x7f26d8109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.430+0000 7f26e0682700 1 --2- 172.21.15.38:0/1678407947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26d80f9820 0x7f26d81096f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.430+0000 7f26e0682700 1 -- 172.21.15.38:0/1678407947 >> 172.21.15.38:0/1678407947 conn(0x7f26d80f0d40 msgr2=0x7f26d80f50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:35.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.430+0000 7f26e0682700 1 -- 172.21.15.38:0/1678407947 shutdown_connections 2024-02-15T18:33:35.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:35.430+0000 7f26e0682700 1 -- 172.21.15.38:0/1678407947 wait complete. 2024-02-15T18:33:36.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:34.287379+0000 mgr.smithi038.khesbl (mgr.14162) 276 : cluster [DBG] pgmap v239: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-02-15T18:33:36.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:35.428307+0000 mon.smithi038 (mon.0) 550 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:33:36.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:35.428875+0000 mon.smithi038 (mon.0) 551 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:33:36.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:35.443127+0000 mon.smithi038 (mon.0) 552 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:33:36.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:34.287379+0000 mgr.smithi038.khesbl (mgr.14162) 276 : cluster [DBG] pgmap v239: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-02-15T18:33:36.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:35.428307+0000 mon.smithi038 (mon.0) 550 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:33:36.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:35.428875+0000 mon.smithi038 (mon.0) 551 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:33:36.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:35.443127+0000 mon.smithi038 (mon.0) 552 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:33:36.531 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-15T18:33:36.531 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-02-15T18:33:36.532 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; sleep 30 ; done' 2024-02-15T18:33:37.135 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:35.419702+0000 mgr.smithi038.khesbl (mgr.14162) 277 : audit [DBG] from='client.14520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:37.136 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:33:35.420307+0000 mgr.smithi038.khesbl (mgr.14162) 278 : cephadm [INF] Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:33:37.136 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:33:35.446242+0000 mgr.smithi038.khesbl (mgr.14162) 279 : cephadm [INF] Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:33:37.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:36 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:35.419702+0000 mgr.smithi038.khesbl (mgr.14162) 277 : audit [DBG] from='client.14520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:37.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:36 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:33:35.420307+0000 mgr.smithi038.khesbl (mgr.14162) 278 : cephadm [INF] Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:33:37.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:36 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:33:35.446242+0000 mgr.smithi038.khesbl (mgr.14162) 279 : cephadm [INF] Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:33:38.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:36.287858+0000 mgr.smithi038.khesbl (mgr.14162) 280 : cluster [DBG] pgmap v240: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-15T18:33:38.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:36.287858+0000 mgr.smithi038.khesbl (mgr.14162) 280 : cluster [DBG] pgmap v240: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-15T18:33:38.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.929+0000 7fe7887f2700 1 -- 172.21.15.38:0/917993068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe780060270 msgr2=0x7fe7800f9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:38.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.929+0000 7fe7887f2700 1 --2- 172.21.15.38:0/917993068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe780060270 0x7fe7800f9340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fe77c004660 tx=0x7fe77c009b30).stop 2024-02-15T18:33:38.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.929+0000 7fe7887f2700 1 -- 172.21.15.38:0/917993068 shutdown_connections 2024-02-15T18:33:38.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.929+0000 7fe7887f2700 1 --2- 172.21.15.38:0/917993068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7800f9830 0x7fe7800fbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:38.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.929+0000 7fe7887f2700 1 --2- 172.21.15.38:0/917993068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe780060270 0x7fe7800f9340 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:38.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.929+0000 7fe7887f2700 1 -- 172.21.15.38:0/917993068 >> 172.21.15.38:0/917993068 conn(0x7fe7800f0d40 msgr2=0x7fe7800f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:38.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.929+0000 7fe7887f2700 1 -- 172.21.15.38:0/917993068 shutdown_connections 2024-02-15T18:33:38.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.929+0000 7fe7887f2700 1 -- 172.21.15.38:0/917993068 wait complete. 2024-02-15T18:33:38.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.930+0000 7fe7887f2700 1 Processor -- start 2024-02-15T18:33:38.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.930+0000 7fe7887f2700 1 -- start start 2024-02-15T18:33:38.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.930+0000 7fe7887f2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe780060270 0x7fe780107000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:38.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.930+0000 7fe7887f2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7800f9830 0x7fe7801074f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:38.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.930+0000 7fe7887f2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe780107aa0 con 0x7fe7800f9830 2024-02-15T18:33:38.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.930+0000 7fe7887f2700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe780107be0 con 0x7fe780060270 2024-02-15T18:33:38.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.930+0000 7fe785d8d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7800f9830 0x7fe7801074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:38.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.930+0000 7fe785d8d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7800f9830 0x7fe7801074f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59166/0 (socket says 172.21.15.38:59166) 2024-02-15T18:33:38.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.930+0000 7fe785d8d700 1 -- 172.21.15.38:0/779308556 learned_addr learned my addr 172.21.15.38:0/779308556 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:33:38.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.930+0000 7fe78658e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe780060270 0x7fe780107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:38.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.931+0000 7fe785d8d700 1 -- 172.21.15.38:0/779308556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe780060270 msgr2=0x7fe780107000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:38.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.931+0000 7fe785d8d700 1 --2- 172.21.15.38:0/779308556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe780060270 0x7fe780107000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:38.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.931+0000 7fe785d8d700 1 -- 172.21.15.38:0/779308556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77c005040 con 0x7fe7800f9830 2024-02-15T18:33:38.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.931+0000 7fe78658e700 1 --2- 172.21.15.38:0/779308556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe780060270 0x7fe780107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:33:38.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.931+0000 7fe785d8d700 1 --2- 172.21.15.38:0/779308556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7800f9830 0x7fe7801074f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fe77000e7c0 tx=0x7fe77000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:38.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.931+0000 7fe776ffd700 1 -- 172.21.15.38:0/779308556 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7700092e0 con 0x7fe7800f9830 2024-02-15T18:33:38.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.931+0000 7fe7887f2700 1 -- 172.21.15.38:0/779308556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe780108580 con 0x7fe7800f9830 2024-02-15T18:33:38.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.931+0000 7fe776ffd700 1 -- 172.21.15.38:0/779308556 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fe77000f040 con 0x7fe7800f9830 2024-02-15T18:33:38.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.931+0000 7fe776ffd700 1 -- 172.21.15.38:0/779308556 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe770014920 con 0x7fe7800f9830 2024-02-15T18:33:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.931+0000 7fe7887f2700 1 -- 172.21.15.38:0/779308556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe780108b40 con 0x7fe7800f9830 2024-02-15T18:33:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.932+0000 7fe7887f2700 1 -- 172.21.15.38:0/779308556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7800f49b0 con 0x7fe7800f9830 2024-02-15T18:33:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.933+0000 7fe776ffd700 1 -- 172.21.15.38:0/779308556 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fe770009440 con 0x7fe7800f9830 2024-02-15T18:33:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.934+0000 7fe776ffd700 1 --2- 172.21.15.38:0/779308556 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fe76c05c2b0 0x7fe76c05e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:38.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.934+0000 7fe776ffd700 1 -- 172.21.15.38:0/779308556 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fe770076120 con 0x7fe7800f9830 2024-02-15T18:33:38.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.934+0000 7fe78658e700 1 --2- 172.21.15.38:0/779308556 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fe76c05c2b0 0x7fe76c05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:38.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.934+0000 7fe78658e700 1 --2- 172.21.15.38:0/779308556 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fe76c05c2b0 0x7fe76c05e770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fe77c009880 tx=0x7fe77c016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:38.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:38.939+0000 7fe776ffd700 1 -- 172.21.15.38:0/779308556 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe77004c750 con 0x7fe7800f9830 2024-02-15T18:33:39.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.173+0000 7fe7887f2700 1 -- 172.21.15.38:0/779308556 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7800ff0d0 con 0x7fe76c05c2b0 2024-02-15T18:33:39.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.174+0000 7fe776ffd700 1 -- 172.21.15.38:0/779308556 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+278 (secure 0 0 0) 0x7fe7800ff0d0 con 0x7fe76c05c2b0 2024-02-15T18:33:39.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.176+0000 7fe7887f2700 1 -- 172.21.15.38:0/779308556 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fe76c05c2b0 msgr2=0x7fe76c05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:39.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.176+0000 7fe7887f2700 1 --2- 172.21.15.38:0/779308556 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fe76c05c2b0 0x7fe76c05e770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fe77c009880 tx=0x7fe77c016040).stop 2024-02-15T18:33:39.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.176+0000 7fe7887f2700 1 -- 172.21.15.38:0/779308556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7800f9830 msgr2=0x7fe7801074f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:39.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.176+0000 7fe7887f2700 1 --2- 172.21.15.38:0/779308556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7800f9830 0x7fe7801074f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fe77000e7c0 tx=0x7fe77000b040).stop 2024-02-15T18:33:39.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.177+0000 7fe7887f2700 1 -- 172.21.15.38:0/779308556 shutdown_connections 2024-02-15T18:33:39.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.177+0000 7fe7887f2700 1 --2- 172.21.15.38:0/779308556 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fe76c05c2b0 0x7fe76c05e770 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:39.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.177+0000 7fe7887f2700 1 --2- 172.21.15.38:0/779308556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe780060270 0x7fe780107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:39.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.177+0000 7fe7887f2700 1 --2- 172.21.15.38:0/779308556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7800f9830 0x7fe7801074f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:39.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.177+0000 7fe7887f2700 1 -- 172.21.15.38:0/779308556 >> 172.21.15.38:0/779308556 conn(0x7fe7800f0d40 msgr2=0x7fe7800f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:39.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.177+0000 7fe7887f2700 1 -- 172.21.15.38:0/779308556 shutdown_connections 2024-02-15T18:33:39.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.177+0000 7fe7887f2700 1 -- 172.21.15.38:0/779308556 wait complete. 2024-02-15T18:33:39.180 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-02-15T18:33:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.303+0000 7f92df59e700 1 -- 172.21.15.38:0/3790198084 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92e0060310 msgr2=0x7f92e00f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.303+0000 7f92df59e700 1 --2- 172.21.15.38:0/3790198084 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92e0060310 0x7f92e00f9110 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f92d00045a0 tx=0x7f92d0009b30).stop 2024-02-15T18:33:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.303+0000 7f92df59e700 1 -- 172.21.15.38:0/3790198084 shutdown_connections 2024-02-15T18:33:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.303+0000 7f92df59e700 1 --2- 172.21.15.38:0/3790198084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92e00f9690 0x7f92e00fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.303+0000 7f92df59e700 1 --2- 172.21.15.38:0/3790198084 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92e0060310 0x7f92e00f9110 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.303+0000 7f92df59e700 1 -- 172.21.15.38:0/3790198084 >> 172.21.15.38:0/3790198084 conn(0x7f92e00f0b90 msgr2=0x7f92e00f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.303+0000 7f92df59e700 1 -- 172.21.15.38:0/3790198084 shutdown_connections 2024-02-15T18:33:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.303+0000 7f92df59e700 1 -- 172.21.15.38:0/3790198084 wait complete. 2024-02-15T18:33:39.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.304+0000 7f92df59e700 1 Processor -- start 2024-02-15T18:33:39.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.304+0000 7f92df59e700 1 -- start start 2024-02-15T18:33:39.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.304+0000 7f92df59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92e0060310 0x7f92e0106e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:39.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.304+0000 7f92df59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92e00f9690 0x7f92e0107330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:39.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.304+0000 7f92df59e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e0107930 con 0x7f92e0060310 2024-02-15T18:33:39.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.304+0000 7f92df59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e0108140 con 0x7f92e00f9690 2024-02-15T18:33:39.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.304+0000 7f92de59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92e0060310 0x7f92e0106e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:39.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.304+0000 7f92de59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92e0060310 0x7f92e0106e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59190/0 (socket says 172.21.15.38:59190) 2024-02-15T18:33:39.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.304+0000 7f92de59c700 1 -- 172.21.15.38:0/283809443 learned_addr learned my addr 172.21.15.38:0/283809443 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:33:39.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.304+0000 7f92ddd9b700 1 --2- 172.21.15.38:0/283809443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92e00f9690 0x7f92e0107330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:39.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.305+0000 7f92de59c700 1 -- 172.21.15.38:0/283809443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92e00f9690 msgr2=0x7f92e0107330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:39.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.305+0000 7f92de59c700 1 --2- 172.21.15.38:0/283809443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92e00f9690 0x7f92e0107330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:39.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.305+0000 7f92de59c700 1 -- 172.21.15.38:0/283809443 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d0005040 con 0x7f92e0060310 2024-02-15T18:33:39.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.305+0000 7f92de59c700 1 --2- 172.21.15.38:0/283809443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92e0060310 0x7f92e0106e40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f92d0009d30 tx=0x7f92d000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:39.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.305+0000 7f92ceffd700 1 -- 172.21.15.38:0/283809443 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92d000e070 con 0x7f92e0060310 2024-02-15T18:33:39.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.305+0000 7f92ceffd700 1 -- 172.21.15.38:0/283809443 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f92d0007e80 con 0x7f92e0060310 2024-02-15T18:33:39.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.305+0000 7f92df59e700 1 -- 172.21.15.38:0/283809443 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e0108360 con 0x7f92e0060310 2024-02-15T18:33:39.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.305+0000 7f92ceffd700 1 -- 172.21.15.38:0/283809443 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92d0019070 con 0x7f92e0060310 2024-02-15T18:33:39.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.305+0000 7f92df59e700 1 -- 172.21.15.38:0/283809443 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e0108840 con 0x7f92e0060310 2024-02-15T18:33:39.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.306+0000 7f92df59e700 1 -- 172.21.15.38:0/283809443 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e0059100 con 0x7f92e0060310 2024-02-15T18:33:39.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.307+0000 7f92ceffd700 1 -- 172.21.15.38:0/283809443 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f92d0025070 con 0x7f92e0060310 2024-02-15T18:33:39.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.307+0000 7f92ceffd700 1 --2- 172.21.15.38:0/283809443 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f92c805c300 0x7f92c805e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:39.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.307+0000 7f92ceffd700 1 -- 172.21.15.38:0/283809443 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f92d0076ba0 con 0x7f92e0060310 2024-02-15T18:33:39.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.308+0000 7f92ddd9b700 1 --2- 172.21.15.38:0/283809443 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f92c805c300 0x7f92c805e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:39.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.308+0000 7f92ddd9b700 1 --2- 172.21.15.38:0/283809443 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f92c805c300 0x7f92c805e7c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f92d4009880 tx=0x7f92d400b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:39.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.312+0000 7f92ceffd700 1 -- 172.21.15.38:0/283809443 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f92d00498e0 con 0x7f92e0060310 2024-02-15T18:33:39.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.555+0000 7f92df59e700 1 -- 172.21.15.38:0/283809443 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f92e00fef30 con 0x7f92c805c300 2024-02-15T18:33:39.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.556+0000 7f92ceffd700 1 -- 172.21.15.38:0/283809443 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+278 (secure 0 0 0) 0x7f92e00fef30 con 0x7f92c805c300 2024-02-15T18:33:39.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.558+0000 7f92df59e700 1 -- 172.21.15.38:0/283809443 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f92c805c300 msgr2=0x7f92c805e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:39.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.558+0000 7f92df59e700 1 --2- 172.21.15.38:0/283809443 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f92c805c300 0x7f92c805e7c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f92d4009880 tx=0x7f92d400b040).stop 2024-02-15T18:33:39.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.558+0000 7f92df59e700 1 -- 172.21.15.38:0/283809443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92e0060310 msgr2=0x7f92e0106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:39.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.558+0000 7f92df59e700 1 --2- 172.21.15.38:0/283809443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92e0060310 0x7f92e0106e40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f92d0009d30 tx=0x7f92d000b040).stop 2024-02-15T18:33:39.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.559+0000 7f92df59e700 1 -- 172.21.15.38:0/283809443 shutdown_connections 2024-02-15T18:33:39.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.559+0000 7f92df59e700 1 --2- 172.21.15.38:0/283809443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92e0060310 0x7f92e0106e40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:39.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.559+0000 7f92df59e700 1 --2- 172.21.15.38:0/283809443 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f92c805c300 0x7f92c805e7c0 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0x7f92d4009880 tx=0x7f92d400b040).stop 2024-02-15T18:33:39.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.559+0000 7f92df59e700 1 --2- 172.21.15.38:0/283809443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92e00f9690 0x7f92e0107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:39.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.559+0000 7f92df59e700 1 -- 172.21.15.38:0/283809443 >> 172.21.15.38:0/283809443 conn(0x7f92e00f0b90 msgr2=0x7f92e00f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:39.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.560+0000 7f92df59e700 1 -- 172.21.15.38:0/283809443 shutdown_connections 2024-02-15T18:33:39.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.560+0000 7f92df59e700 1 -- 172.21.15.38:0/283809443 wait complete. 2024-02-15T18:33:39.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.684+0000 7f905d208700 1 -- 172.21.15.38:0/1772036680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90580f6860 msgr2=0x7f90580f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:39.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.684+0000 7f905d208700 1 --2- 172.21.15.38:0/1772036680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90580f6860 0x7f90580f6cc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f9040004660 tx=0x7f9040009b30).stop 2024-02-15T18:33:39.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.684+0000 7f905d208700 1 -- 172.21.15.38:0/1772036680 shutdown_connections 2024-02-15T18:33:39.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.684+0000 7f905d208700 1 --2- 172.21.15.38:0/1772036680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90580f6860 0x7f90580f6cc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:39.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.684+0000 7f905d208700 1 --2- 172.21.15.38:0/1772036680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90580f55c0 0x7f90580f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:39.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.684+0000 7f905d208700 1 -- 172.21.15.38:0/1772036680 >> 172.21.15.38:0/1772036680 conn(0x7f90580f0d40 msgr2=0x7f90580f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:39.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.684+0000 7f905d208700 1 -- 172.21.15.38:0/1772036680 shutdown_connections 2024-02-15T18:33:39.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.684+0000 7f905d208700 1 -- 172.21.15.38:0/1772036680 wait complete. 2024-02-15T18:33:39.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.685+0000 7f905d208700 1 Processor -- start 2024-02-15T18:33:39.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.685+0000 7f905d208700 1 -- start start 2024-02-15T18:33:39.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.685+0000 7f905d208700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90580f55c0 0x7f90580f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:39.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.685+0000 7f905d208700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90580f6860 0x7f90580f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:39.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.685+0000 7f905d208700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90580f8100 con 0x7f90580f6860 2024-02-15T18:33:39.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.685+0000 7f905d208700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90580f8240 con 0x7f90580f55c0 2024-02-15T18:33:39.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.685+0000 7f905659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90580f6860 0x7f90580f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:39.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.685+0000 7f9056d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90580f55c0 0x7f90580f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:39.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.685+0000 7f905659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90580f6860 0x7f90580f7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59212/0 (socket says 172.21.15.38:59212) 2024-02-15T18:33:39.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.685+0000 7f9056d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90580f55c0 0x7f90580f9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.38:33138/0 (socket says 172.21.15.38:33138) 2024-02-15T18:33:39.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.685+0000 7f905659c700 1 -- 172.21.15.38:0/474934165 learned_addr learned my addr 172.21.15.38:0/474934165 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:33:39.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.686+0000 7f9056d9d700 1 -- 172.21.15.38:0/474934165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90580f6860 msgr2=0x7f90580f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:39.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.686+0000 7f9056d9d700 1 --2- 172.21.15.38:0/474934165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90580f6860 0x7f90580f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:39.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.686+0000 7f9056d9d700 1 -- 172.21.15.38:0/474934165 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9048005040 con 0x7f90580f55c0 2024-02-15T18:33:39.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.686+0000 7f9056d9d700 1 --2- 172.21.15.38:0/474934165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90580f55c0 0x7f90580f9720 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f9048004870 tx=0x7f9048009b30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:39.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.686+0000 7f904f7fe700 1 -- 172.21.15.38:0/474934165 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f904800e030 con 0x7f90580f55c0 2024-02-15T18:33:39.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.686+0000 7f904f7fe700 1 -- 172.21.15.38:0/474934165 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f904800f040 con 0x7f90580f55c0 2024-02-15T18:33:39.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.686+0000 7f905d208700 1 -- 172.21.15.38:0/474934165 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9040005040 con 0x7f90580f55c0 2024-02-15T18:33:39.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.686+0000 7f904f7fe700 1 -- 172.21.15.38:0/474934165 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90480148d0 con 0x7f90580f55c0 2024-02-15T18:33:39.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.686+0000 7f905d208700 1 -- 172.21.15.38:0/474934165 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90580f8950 con 0x7f90580f55c0 2024-02-15T18:33:39.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.687+0000 7f905d208700 1 -- 172.21.15.38:0/474934165 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9058045ad0 con 0x7f90580f55c0 2024-02-15T18:33:39.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.687+0000 7f904f7fe700 1 -- 172.21.15.38:0/474934165 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f9048014b40 con 0x7f90580f55c0 2024-02-15T18:33:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.688+0000 7f904f7fe700 1 --2- 172.21.15.38:0/474934165 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f904405c2a0 0x7f904405e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.688+0000 7f904f7fe700 1 -- 172.21.15.38:0/474934165 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f9048076420 con 0x7f90580f55c0 2024-02-15T18:33:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.688+0000 7f905659c700 1 --2- 172.21.15.38:0/474934165 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f904405c2a0 0x7f904405e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:39.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.688+0000 7f905659c700 1 --2- 172.21.15.38:0/474934165 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f904405c2a0 0x7f904405e760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f9040004630 tx=0x7f9040016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:39.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.692+0000 7f904f7fe700 1 -- 172.21.15.38:0/474934165 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9048049160 con 0x7f90580f55c0 2024-02-15T18:33:39.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.930+0000 7f905d208700 1 -- 172.21.15.38:0/474934165 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f90580faca0 con 0x7f904405c2a0 2024-02-15T18:33:39.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.941+0000 7f904f7fe700 1 -- 172.21.15.38:0/474934165 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2684 (secure 0 0 0) 0x7f90580faca0 con 0x7f904405c2a0 2024-02-15T18:33:39.932 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE VERSION IMAGE ID CONTAINER ID 2024-02-15T18:33:39.932 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (4m) 20s ago 6m 0.20.0 0881eb8f169f c2034c318e7c 2024-02-15T18:33:39.933 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (6m) 20s ago 6m 16.2.4 8d91d370c2b8 0fd3f945d009 2024-02-15T18:33:39.933 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi121 smithi121 running (5m) 26s ago 5m 16.2.4 8d91d370c2b8 affc02db354e 2024-02-15T18:33:39.933 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (4m) 20s ago 6m 6.7.4 557c83e11646 2dcccfbfbd2a 2024-02-15T18:33:39.933 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.hxoggl smithi038 running (45s) 20s ago 45s 16.2.4 8d91d370c2b8 be2533754b9c 2024-02-15T18:33:39.933 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.qgkzgn smithi038 running (40s) 20s ago 39s 16.2.4 8d91d370c2b8 dd3ffafba9bd 2024-02-15T18:33:39.933 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.mvumlh smithi121 running (51s) 26s ago 50s 16.2.4 8d91d370c2b8 88b7d876a39b 2024-02-15T18:33:39.933 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.ovuzki smithi121 running (43s) 26s ago 42s 16.2.4 8d91d370c2b8 21cacf0012f9 2024-02-15T18:33:39.933 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.khesbl smithi038 *:9283 running (7m) 20s ago 7m 16.2.4 8d91d370c2b8 b0d099c255f4 2024-02-15T18:33:39.933 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi121.ckiryi smithi121 *:8443,9283 running (5m) 26s ago 5m 16.2.4 8d91d370c2b8 628756a555db 2024-02-15T18:33:39.933 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (8m) 20s ago 8m 16.2.4 8d91d370c2b8 01b30eaebcfe 2024-02-15T18:33:39.933 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi121 smithi121 running (5m) 26s ago 5m 16.2.4 8d91d370c2b8 8f588ecacb71 2024-02-15T18:33:39.933 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (5m) 20s ago 5m 0.18.1 e5a616e4b9cf 037f60e38158 2024-02-15T18:33:39.933 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi121 smithi121 *:9100 running (4m) 26s ago 4m 0.18.1 e5a616e4b9cf a867bff85e18 2024-02-15T18:33:39.933 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (4m) 20s ago 4m 16.2.4 8d91d370c2b8 51512003a2ef 2024-02-15T18:33:39.933 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (4m) 20s ago 4m 16.2.4 8d91d370c2b8 16dbdf20b170 2024-02-15T18:33:39.934 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (3m) 20s ago 3m 16.2.4 8d91d370c2b8 6d6ee813e5be 2024-02-15T18:33:39.934 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi121 running (3m) 26s ago 3m 16.2.4 8d91d370c2b8 cba2d9ed4730 2024-02-15T18:33:39.934 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi121 running (2m) 26s ago 2m 16.2.4 8d91d370c2b8 ac17ad7dc628 2024-02-15T18:33:39.934 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi121 running (2m) 26s ago 2m 16.2.4 8d91d370c2b8 cdc8c669c4db 2024-02-15T18:33:39.934 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (4m) 20s ago 5m 2.18.1 de242295e225 574a5d75991e 2024-02-15T18:33:39.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.943+0000 7f905d208700 1 -- 172.21.15.38:0/474934165 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f904405c2a0 msgr2=0x7f904405e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.943+0000 7f905d208700 1 --2- 172.21.15.38:0/474934165 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f904405c2a0 0x7f904405e760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f9040004630 tx=0x7f9040016040).stop 2024-02-15T18:33:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.943+0000 7f905d208700 1 -- 172.21.15.38:0/474934165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90580f55c0 msgr2=0x7f90580f9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.943+0000 7f905d208700 1 --2- 172.21.15.38:0/474934165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90580f55c0 0x7f90580f9720 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f9048004870 tx=0x7f9048009b30).stop 2024-02-15T18:33:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.943+0000 7f905d208700 1 -- 172.21.15.38:0/474934165 shutdown_connections 2024-02-15T18:33:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.943+0000 7f905d208700 1 --2- 172.21.15.38:0/474934165 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f904405c2a0 0x7f904405e760 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.943+0000 7f905d208700 1 --2- 172.21.15.38:0/474934165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90580f55c0 0x7f90580f9720 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.943+0000 7f905d208700 1 --2- 172.21.15.38:0/474934165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90580f6860 0x7f90580f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.943+0000 7f905d208700 1 -- 172.21.15.38:0/474934165 >> 172.21.15.38:0/474934165 conn(0x7f90580f0d40 msgr2=0x7f905805fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.943+0000 7f905d208700 1 -- 172.21.15.38:0/474934165 shutdown_connections 2024-02-15T18:33:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:39.943+0000 7f905d208700 1 -- 172.21.15.38:0/474934165 wait complete. 2024-02-15T18:33:40.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.066+0000 7fb7316cd700 1 -- 172.21.15.38:0/2624880543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb72c0f66d0 msgr2=0x7fb72c0f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:40.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.066+0000 7fb7316cd700 1 --2- 172.21.15.38:0/2624880543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb72c0f66d0 0x7fb72c0f6b30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fb720004660 tx=0x7fb720009b30).stop 2024-02-15T18:33:40.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.066+0000 7fb7316cd700 1 -- 172.21.15.38:0/2624880543 shutdown_connections 2024-02-15T18:33:40.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.066+0000 7fb7316cd700 1 --2- 172.21.15.38:0/2624880543 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb72c0f66d0 0x7fb72c0f6b30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:40.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.066+0000 7fb7316cd700 1 --2- 172.21.15.38:0/2624880543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb72c0f5430 0x7fb72c0f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:40.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.066+0000 7fb7316cd700 1 -- 172.21.15.38:0/2624880543 >> 172.21.15.38:0/2624880543 conn(0x7fb72c0f0b90 msgr2=0x7fb72c0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:40.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.066+0000 7fb7316cd700 1 -- 172.21.15.38:0/2624880543 shutdown_connections 2024-02-15T18:33:40.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.066+0000 7fb7316cd700 1 -- 172.21.15.38:0/2624880543 wait complete. 2024-02-15T18:33:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.067+0000 7fb7316cd700 1 Processor -- start 2024-02-15T18:33:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.067+0000 7fb7316cd700 1 -- start start 2024-02-15T18:33:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.067+0000 7fb7316cd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb72c0f5430 0x7fb72c109ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.067+0000 7fb7316cd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb72c0f66d0 0x7fb72c10bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.067+0000 7fb7316cd700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0603c0 con 0x7fb72c0f66d0 2024-02-15T18:33:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.067+0000 7fb7316cd700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c060500 con 0x7fb72c0f5430 2024-02-15T18:33:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.067+0000 7fb72b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb72c0f66d0 0x7fb72c10bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.067+0000 7fb72bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb72c0f5430 0x7fb72c109ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.067+0000 7fb72bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb72c0f5430 0x7fb72c109ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.38:33154/0 (socket says 172.21.15.38:33154) 2024-02-15T18:33:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.068+0000 7fb72bfff700 1 -- 172.21.15.38:0/2373678590 learned_addr learned my addr 172.21.15.38:0/2373678590 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:33:40.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.068+0000 7fb72bfff700 1 -- 172.21.15.38:0/2373678590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb72c0f66d0 msgr2=0x7fb72c10bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:40.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.068+0000 7fb72bfff700 1 --2- 172.21.15.38:0/2373678590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb72c0f66d0 0x7fb72c10bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:40.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.068+0000 7fb72bfff700 1 -- 172.21.15.38:0/2373678590 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb720005040 con 0x7fb72c0f5430 2024-02-15T18:33:40.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.068+0000 7fb72b7fe700 1 --2- 172.21.15.38:0/2373678590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb72c0f66d0 0x7fb72c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:33:40.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.068+0000 7fb72bfff700 1 --2- 172.21.15.38:0/2373678590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb72c0f5430 0x7fb72c109ad0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fb71c00e7c0 tx=0x7fb71c00b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:40.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.068+0000 7fb728ff9700 1 -- 172.21.15.38:0/2373678590 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb71c0092e0 con 0x7fb72c0f5430 2024-02-15T18:33:40.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.068+0000 7fb7316cd700 1 -- 172.21.15.38:0/2373678590 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb72c10c580 con 0x7fb72c0f5430 2024-02-15T18:33:40.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.068+0000 7fb728ff9700 1 -- 172.21.15.38:0/2373678590 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fb71c00f040 con 0x7fb72c0f5430 2024-02-15T18:33:40.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.068+0000 7fb728ff9700 1 -- 172.21.15.38:0/2373678590 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb71c014920 con 0x7fb72c0f5430 2024-02-15T18:33:40.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.068+0000 7fb7316cd700 1 -- 172.21.15.38:0/2373678590 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb72c10cb10 con 0x7fb72c0f5430 2024-02-15T18:33:40.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.069+0000 7fb7316cd700 1 -- 172.21.15.38:0/2373678590 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72c059100 con 0x7fb72c0f5430 2024-02-15T18:33:40.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.070+0000 7fb728ff9700 1 -- 172.21.15.38:0/2373678590 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fb71c01b030 con 0x7fb72c0f5430 2024-02-15T18:33:40.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.070+0000 7fb728ff9700 1 --2- 172.21.15.38:0/2373678590 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb71405c300 0x7fb71405e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:40.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.070+0000 7fb728ff9700 1 -- 172.21.15.38:0/2373678590 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fb71c075db0 con 0x7fb72c0f5430 2024-02-15T18:33:40.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.070+0000 7fb72b7fe700 1 --2- 172.21.15.38:0/2373678590 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb71405c300 0x7fb71405e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:40.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.071+0000 7fb72b7fe700 1 --2- 172.21.15.38:0/2373678590 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb71405c300 0x7fb71405e7c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fb720005010 tx=0x7fb720016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:40.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.074+0000 7fb728ff9700 1 -- 172.21.15.38:0/2373678590 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb71c048a70 con 0x7fb72c0f5430 2024-02-15T18:33:40.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:38.288372+0000 mgr.smithi038.khesbl (mgr.14162) 281 : cluster [DBG] pgmap v241: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-15T18:33:40.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.342+0000 7fb7316cd700 1 -- 172.21.15.38:0/2373678590 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb72c045ad0 con 0x7fb72c0f5430 2024-02-15T18:33:40.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.343+0000 7fb728ff9700 1 -- 172.21.15.38:0/2373678590 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+573 (secure 0 0 0) 0x7fb71c019020 con 0x7fb72c0f5430 2024-02-15T18:33:40.334 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:33:40.335 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-02-15T18:33:40.335 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2024-02-15T18:33:40.335 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:33:40.335 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-02-15T18:33:40.335 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2024-02-15T18:33:40.335 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:33:40.335 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-02-15T18:33:40.335 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2024-02-15T18:33:40.335 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:33:40.335 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-02-15T18:33:40.335 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:33:40.335 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:33:40.335 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-02-15T18:33:40.335 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 14 2024-02-15T18:33:40.336 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-02-15T18:33:40.336 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:33:40.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.345+0000 7fb7316cd700 1 -- 172.21.15.38:0/2373678590 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb71405c300 msgr2=0x7fb71405e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:40.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.345+0000 7fb7316cd700 1 --2- 172.21.15.38:0/2373678590 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb71405c300 0x7fb71405e7c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fb720005010 tx=0x7fb720016040).stop 2024-02-15T18:33:40.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.345+0000 7fb7316cd700 1 -- 172.21.15.38:0/2373678590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb72c0f5430 msgr2=0x7fb72c109ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:40.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.345+0000 7fb7316cd700 1 --2- 172.21.15.38:0/2373678590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb72c0f5430 0x7fb72c109ad0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fb71c00e7c0 tx=0x7fb71c00b040).stop 2024-02-15T18:33:40.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.346+0000 7fb7316cd700 1 -- 172.21.15.38:0/2373678590 shutdown_connections 2024-02-15T18:33:40.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.346+0000 7fb7316cd700 1 --2- 172.21.15.38:0/2373678590 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb71405c300 0x7fb71405e7c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:40.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.346+0000 7fb7316cd700 1 --2- 172.21.15.38:0/2373678590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb72c0f5430 0x7fb72c109ad0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:40.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.346+0000 7fb7316cd700 1 --2- 172.21.15.38:0/2373678590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb72c0f66d0 0x7fb72c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:40.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.346+0000 7fb7316cd700 1 -- 172.21.15.38:0/2373678590 >> 172.21.15.38:0/2373678590 conn(0x7fb72c0f0b90 msgr2=0x7fb72c0fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:40.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.346+0000 7fb7316cd700 1 -- 172.21.15.38:0/2373678590 shutdown_connections 2024-02-15T18:33:40.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.346+0000 7fb7316cd700 1 -- 172.21.15.38:0/2373678590 wait complete. 2024-02-15T18:33:40.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:38.288372+0000 mgr.smithi038.khesbl (mgr.14162) 281 : cluster [DBG] pgmap v241: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-15T18:33:40.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.477+0000 7f3ef359e700 1 -- 172.21.15.38:0/3766621563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ef40f51a0 msgr2=0x7f3ef40f7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:40.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.477+0000 7f3ef359e700 1 --2- 172.21.15.38:0/3766621563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ef40f51a0 0x7f3ef40f7920 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f3ee8004660 tx=0x7f3ee8009b30).stop 2024-02-15T18:33:40.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.477+0000 7f3ef359e700 1 -- 172.21.15.38:0/3766621563 shutdown_connections 2024-02-15T18:33:40.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.477+0000 7f3ef359e700 1 --2- 172.21.15.38:0/3766621563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ef40f51a0 0x7f3ef40f7920 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:40.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.477+0000 7f3ef359e700 1 --2- 172.21.15.38:0/3766621563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ef40f4800 0x7f3ef40f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:40.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.477+0000 7f3ef359e700 1 -- 172.21.15.38:0/3766621563 >> 172.21.15.38:0/3766621563 conn(0x7f3ef40f0320 msgr2=0x7f3ef40f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:40.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.477+0000 7f3ef359e700 1 -- 172.21.15.38:0/3766621563 shutdown_connections 2024-02-15T18:33:40.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.477+0000 7f3ef359e700 1 -- 172.21.15.38:0/3766621563 wait complete. 2024-02-15T18:33:40.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.478+0000 7f3ef359e700 1 Processor -- start 2024-02-15T18:33:40.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.478+0000 7f3ef359e700 1 -- start start 2024-02-15T18:33:40.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.478+0000 7f3ef359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ef40f4800 0x7f3ef40f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:40.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.478+0000 7f3ef359e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ef40f51a0 0x7f3ef40f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:40.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.478+0000 7f3ef359e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef40f5d90 con 0x7f3ef40f51a0 2024-02-15T18:33:40.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.478+0000 7f3ef359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef40f5ed0 con 0x7f3ef40f4800 2024-02-15T18:33:40.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.478+0000 7f3ef1d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ef40f51a0 0x7f3ef40f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:40.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.478+0000 7f3ef1d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ef40f51a0 0x7f3ef40f5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59254/0 (socket says 172.21.15.38:59254) 2024-02-15T18:33:40.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.478+0000 7f3ef1d9b700 1 -- 172.21.15.38:0/780169682 learned_addr learned my addr 172.21.15.38:0/780169682 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:33:40.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.478+0000 7f3ef259c700 1 --2- 172.21.15.38:0/780169682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ef40f4800 0x7f3ef40f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:40.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.478+0000 7f3ef1d9b700 1 -- 172.21.15.38:0/780169682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ef40f4800 msgr2=0x7f3ef40f73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:40.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.478+0000 7f3ef1d9b700 1 --2- 172.21.15.38:0/780169682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ef40f4800 0x7f3ef40f73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:40.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.478+0000 7f3ef1d9b700 1 -- 172.21.15.38:0/780169682 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee8005040 con 0x7f3ef40f51a0 2024-02-15T18:33:40.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.478+0000 7f3ef1d9b700 1 --2- 172.21.15.38:0/780169682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ef40f51a0 0x7f3ef40f5830 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f3ee8002040 tx=0x7f3ee800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:40.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.479+0000 7f3ee2ffd700 1 -- 172.21.15.38:0/780169682 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ee8019070 con 0x7f3ef40f51a0 2024-02-15T18:33:40.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.479+0000 7f3ef359e700 1 -- 172.21.15.38:0/780169682 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef40f6150 con 0x7f3ef40f51a0 2024-02-15T18:33:40.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.479+0000 7f3ee2ffd700 1 -- 172.21.15.38:0/780169682 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f3ee8007e80 con 0x7f3ef40f51a0 2024-02-15T18:33:40.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.479+0000 7f3ee2ffd700 1 -- 172.21.15.38:0/780169682 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ee8014db0 con 0x7f3ef40f51a0 2024-02-15T18:33:40.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.479+0000 7f3ef359e700 1 -- 172.21.15.38:0/780169682 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef40f66b0 con 0x7f3ef40f51a0 2024-02-15T18:33:40.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.480+0000 7f3ef359e700 1 -- 172.21.15.38:0/780169682 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef4045ad0 con 0x7f3ef40f51a0 2024-02-15T18:33:40.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.480+0000 7f3ee2ffd700 1 -- 172.21.15.38:0/780169682 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f3ee8025070 con 0x7f3ef40f51a0 2024-02-15T18:33:40.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.481+0000 7f3ee2ffd700 1 --2- 172.21.15.38:0/780169682 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f3edc05c2a0 0x7f3edc05e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:40.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.481+0000 7f3ee2ffd700 1 -- 172.21.15.38:0/780169682 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f3ee8076d80 con 0x7f3ef40f51a0 2024-02-15T18:33:40.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.481+0000 7f3ef259c700 1 --2- 172.21.15.38:0/780169682 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f3edc05c2a0 0x7f3edc05e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:40.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.481+0000 7f3ef259c700 1 --2- 172.21.15.38:0/780169682 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f3edc05c2a0 0x7f3edc05e760 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f3ef40f7140 tx=0x7f3ee400b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:40.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.488+0000 7f3ee2ffd700 1 -- 172.21.15.38:0/780169682 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f3ee80499e0 con 0x7f3ef40f51a0 2024-02-15T18:33:40.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.763+0000 7f3ef359e700 1 -- 172.21.15.38:0/780169682 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3ef4027a00 con 0x7f3ef40f51a0 2024-02-15T18:33:40.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.764+0000 7f3ee2ffd700 1 -- 172.21.15.38:0/780169682 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 11 v11) v1 ==== 76+0+1626 (secure 0 0 0) 0x7f3ee8078020 con 0x7f3ef40f51a0 2024-02-15T18:33:40.755 INFO:teuthology.orchestra.run.smithi038.stdout:e11 2024-02-15T18:33:40.755 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:33:40.756 INFO:teuthology.orchestra.run.smithi038.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:33:40.756 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-02-15T18:33:40.756 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:33:40.756 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:33:40.756 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-02-15T18:33:40.756 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 10 2024-02-15T18:33:40.756 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 2024-02-15T18:33:40.756 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:33:40.756 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-02-15T18:33:02.882026+0000 2024-02-15T18:33:40.756 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-02-15T18:33:40.756 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-02-15T18:33:40.757 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-02-15T18:33:40.757 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-02-15T18:33:40.757 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-02-15T18:33:40.757 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-02-15T18:33:40.757 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-02-15T18:33:40.757 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-02-15T18:33:40.757 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:33:40.757 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-02-15T18:33:40.757 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-02-15T18:33:40.757 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=24261,1=24275} 2024-02-15T18:33:40.758 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-02-15T18:33:40.758 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-02-15T18:33:40.758 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-02-15T18:33:40.758 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-02-15T18:33:40.758 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-02-15T18:33:40.758 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-02-15T18:33:40.758 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-02-15T18:33:40.758 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-02-15T18:33:40.758 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.mvumlh{0:24261} state up:active seq 3 join_fscid=1 addr [v2:172.21.15.121:6824/1422707571,v1:172.21.15.121:6825/1422707571]] 2024-02-15T18:33:40.758 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.ovuzki{1:24275} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845]] 2024-02-15T18:33:40.758 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:33:40.759 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:33:40.759 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-02-15T18:33:40.759 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:33:40.759 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.hxoggl{-1:14446} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.38:6826/3822880235,v1:172.21.15.38:6827/3822880235]] 2024-02-15T18:33:40.759 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.qgkzgn{-1:14456} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.38:6828/703030231,v1:172.21.15.38:6829/703030231]] 2024-02-15T18:33:40.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.766+0000 7f3ef359e700 1 -- 172.21.15.38:0/780169682 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f3edc05c2a0 msgr2=0x7f3edc05e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:40.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.766+0000 7f3ef359e700 1 --2- 172.21.15.38:0/780169682 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f3edc05c2a0 0x7f3edc05e760 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f3ef40f7140 tx=0x7f3ee400b040).stop 2024-02-15T18:33:40.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.766+0000 7f3ef359e700 1 -- 172.21.15.38:0/780169682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ef40f51a0 msgr2=0x7f3ef40f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:40.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.766+0000 7f3ef359e700 1 --2- 172.21.15.38:0/780169682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ef40f51a0 0x7f3ef40f5830 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f3ee8002040 tx=0x7f3ee800b040).stop 2024-02-15T18:33:40.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.766+0000 7f3ef359e700 1 -- 172.21.15.38:0/780169682 shutdown_connections 2024-02-15T18:33:40.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.766+0000 7f3ef359e700 1 --2- 172.21.15.38:0/780169682 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f3edc05c2a0 0x7f3edc05e760 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:40.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.766+0000 7f3ef359e700 1 --2- 172.21.15.38:0/780169682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ef40f4800 0x7f3ef40f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:40.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.766+0000 7f3ef359e700 1 --2- 172.21.15.38:0/780169682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ef40f51a0 0x7f3ef40f5830 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:40.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.766+0000 7f3ef359e700 1 -- 172.21.15.38:0/780169682 >> 172.21.15.38:0/780169682 conn(0x7f3ef40f0320 msgr2=0x7f3ef40fb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:40.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.766+0000 7f3ef359e700 1 -- 172.21.15.38:0/780169682 shutdown_connections 2024-02-15T18:33:40.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.766+0000 7f3ef359e700 1 -- 172.21.15.38:0/780169682 wait complete. 2024-02-15T18:33:40.761 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 11 2024-02-15T18:33:40.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.887+0000 7f76c31e5700 1 -- 172.21.15.38:0/2213630647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc0f49a0 msgr2=0x7f76bc0f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:40.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.887+0000 7f76c31e5700 1 --2- 172.21.15.38:0/2213630647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc0f49a0 0x7f76bc0f4dc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f76b0004660 tx=0x7f76b0009b30).stop 2024-02-15T18:33:40.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.887+0000 7f76c31e5700 1 -- 172.21.15.38:0/2213630647 shutdown_connections 2024-02-15T18:33:40.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.887+0000 7f76c31e5700 1 --2- 172.21.15.38:0/2213630647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76bc0f5340 0x7f76bc0f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:40.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.887+0000 7f76c31e5700 1 --2- 172.21.15.38:0/2213630647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc0f49a0 0x7f76bc0f4dc0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:40.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.887+0000 7f76c31e5700 1 -- 172.21.15.38:0/2213630647 >> 172.21.15.38:0/2213630647 conn(0x7f76bc0f04c0 msgr2=0x7f76bc0f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.887+0000 7f76c31e5700 1 -- 172.21.15.38:0/2213630647 shutdown_connections 2024-02-15T18:33:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.887+0000 7f76c31e5700 1 -- 172.21.15.38:0/2213630647 wait complete. 2024-02-15T18:33:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.888+0000 7f76c31e5700 1 Processor -- start 2024-02-15T18:33:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.888+0000 7f76c31e5700 1 -- start start 2024-02-15T18:33:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.889+0000 7f76c31e5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc0f49a0 0x7f76bc1070f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.889+0000 7f76c31e5700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76bc0f5340 0x7f76bc1075e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.889+0000 7f76c31e5700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc107b90 con 0x7f76bc0f49a0 2024-02-15T18:33:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.889+0000 7f76c31e5700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc107cd0 con 0x7f76bc0f5340 2024-02-15T18:33:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.889+0000 7f76c0f81700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc0f49a0 0x7f76bc1070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.889+0000 7f76c0f81700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc0f49a0 0x7f76bc1070f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59274/0 (socket says 172.21.15.38:59274) 2024-02-15T18:33:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.889+0000 7f76c0f81700 1 -- 172.21.15.38:0/269359707 learned_addr learned my addr 172.21.15.38:0/269359707 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:33:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.889+0000 7f76bbfff700 1 --2- 172.21.15.38:0/269359707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76bc0f5340 0x7f76bc1075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.889+0000 7f76c0f81700 1 -- 172.21.15.38:0/269359707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76bc0f5340 msgr2=0x7f76bc1075e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.889+0000 7f76c0f81700 1 --2- 172.21.15.38:0/269359707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76bc0f5340 0x7f76bc1075e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:40.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.889+0000 7f76c0f81700 1 -- 172.21.15.38:0/269359707 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b0005040 con 0x7f76bc0f49a0 2024-02-15T18:33:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.889+0000 7f76c0f81700 1 --2- 172.21.15.38:0/269359707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc0f49a0 0x7f76bc1070f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f76b00043d0 tx=0x7f76b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.889+0000 7f76b97fa700 1 -- 172.21.15.38:0/269359707 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76b0019070 con 0x7f76bc0f49a0 2024-02-15T18:33:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.890+0000 7f76c31e5700 1 -- 172.21.15.38:0/269359707 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76bc108610 con 0x7f76bc0f49a0 2024-02-15T18:33:40.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.890+0000 7f76b97fa700 1 -- 172.21.15.38:0/269359707 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f76b0007e80 con 0x7f76bc0f49a0 2024-02-15T18:33:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.890+0000 7f76b97fa700 1 -- 172.21.15.38:0/269359707 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76b0014cf0 con 0x7f76bc0f49a0 2024-02-15T18:33:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.890+0000 7f76c31e5700 1 -- 172.21.15.38:0/269359707 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76bc108b70 con 0x7f76bc0f49a0 2024-02-15T18:33:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.891+0000 7f76c31e5700 1 -- 172.21.15.38:0/269359707 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76bc045ad0 con 0x7f76bc0f49a0 2024-02-15T18:33:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.891+0000 7f76b97fa700 1 -- 172.21.15.38:0/269359707 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f76b0025070 con 0x7f76bc0f49a0 2024-02-15T18:33:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.891+0000 7f76b97fa700 1 --2- 172.21.15.38:0/269359707 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f76ac05c2b0 0x7f76ac05e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:40.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.891+0000 7f76b97fa700 1 -- 172.21.15.38:0/269359707 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f76b0076e60 con 0x7f76bc0f49a0 2024-02-15T18:33:40.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.895+0000 7f76bbfff700 1 --2- 172.21.15.38:0/269359707 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f76ac05c2b0 0x7f76ac05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:40.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.895+0000 7f76bbfff700 1 --2- 172.21.15.38:0/269359707 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f76ac05c2b0 0x7f76ac05e770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f76a8009fd0 tx=0x7f76a800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:40.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:40.899+0000 7f76b97fa700 1 -- 172.21.15.38:0/269359707 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f76b0049b20 con 0x7f76bc0f49a0 2024-02-15T18:33:41.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:41.151+0000 7f76c31e5700 1 -- 172.21.15.38:0/269359707 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f76bc0ff130 con 0x7f76ac05c2b0 2024-02-15T18:33:41.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:41.153+0000 7f76b97fa700 1 -- 172.21.15.38:0/269359707 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+278 (secure 0 0 0) 0x7f76bc0ff130 con 0x7f76ac05c2b0 2024-02-15T18:33:41.144 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:33:41.144 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1", 2024-02-15T18:33:41.144 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-02-15T18:33:41.144 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [], 2024-02-15T18:33:41.144 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "", 2024-02-15T18:33:41.144 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 image" 2024-02-15T18:33:41.145 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:33:41.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:41.155+0000 7f76c31e5700 1 -- 172.21.15.38:0/269359707 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f76ac05c2b0 msgr2=0x7f76ac05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:41.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:41.155+0000 7f76c31e5700 1 --2- 172.21.15.38:0/269359707 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f76ac05c2b0 0x7f76ac05e770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f76a8009fd0 tx=0x7f76a800b040).stop 2024-02-15T18:33:41.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:41.155+0000 7f76c31e5700 1 -- 172.21.15.38:0/269359707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc0f49a0 msgr2=0x7f76bc1070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:41.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:41.155+0000 7f76c31e5700 1 --2- 172.21.15.38:0/269359707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc0f49a0 0x7f76bc1070f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f76b00043d0 tx=0x7f76b000b040).stop 2024-02-15T18:33:41.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:41.156+0000 7f76c31e5700 1 -- 172.21.15.38:0/269359707 shutdown_connections 2024-02-15T18:33:41.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:41.156+0000 7f76c31e5700 1 --2- 172.21.15.38:0/269359707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76bc0f49a0 0x7f76bc1070f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:41.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:41.156+0000 7f76c31e5700 1 --2- 172.21.15.38:0/269359707 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f76ac05c2b0 0x7f76ac05e770 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:41.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:41.156+0000 7f76c31e5700 1 --2- 172.21.15.38:0/269359707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76bc0f5340 0x7f76bc1075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:41.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:41.156+0000 7f76c31e5700 1 -- 172.21.15.38:0/269359707 >> 172.21.15.38:0/269359707 conn(0x7f76bc0f04c0 msgr2=0x7f76bc0fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:41.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:41.156+0000 7f76c31e5700 1 -- 172.21.15.38:0/269359707 shutdown_connections 2024-02-15T18:33:41.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:33:41.156+0000 7f76c31e5700 1 -- 172.21.15.38:0/269359707 wait complete. 2024-02-15T18:33:41.262 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:39.174577+0000 mgr.smithi038.khesbl (mgr.14162) 282 : audit [DBG] from='client.14524 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:41.263 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:39.556574+0000 mgr.smithi038.khesbl (mgr.14162) 283 : audit [DBG] from='client.14528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:41.263 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:40.343647+0000 mon.smithi121 (mon.1) 17 : audit [DBG] from='client.? 172.21.15.38:0/2373678590' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:41.263 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:40.369615+0000 mon.smithi038 (mon.0) 553 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:33:41.263 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:40.369853+0000 mon.smithi038 (mon.0) 554 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:33:41.263 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:40.764331+0000 mon.smithi038 (mon.0) 555 : audit [DBG] from='client.? 172.21.15.38:0/780169682' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:33:41.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:39.174577+0000 mgr.smithi038.khesbl (mgr.14162) 282 : audit [DBG] from='client.14524 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:41.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:39.556574+0000 mgr.smithi038.khesbl (mgr.14162) 283 : audit [DBG] from='client.14528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:41.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:40.343647+0000 mon.smithi121 (mon.1) 17 : audit [DBG] from='client.? 172.21.15.38:0/2373678590' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:41.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:40.369615+0000 mon.smithi038 (mon.0) 553 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:33:41.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:40.369853+0000 mon.smithi038 (mon.0) 554 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:33:41.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:40.764331+0000 mon.smithi038 (mon.0) 555 : audit [DBG] from='client.? 172.21.15.38:0/780169682' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:33:42.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:39.931384+0000 mgr.smithi038.khesbl (mgr.14162) 284 : audit [DBG] from='client.24345 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:42.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:40.289126+0000 mgr.smithi038.khesbl (mgr.14162) 285 : cluster [DBG] pgmap v242: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:42.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:33:41.153151+0000 mgr.smithi038.khesbl (mgr.14162) 286 : audit [DBG] from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:42.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:39.931384+0000 mgr.smithi038.khesbl (mgr.14162) 284 : audit [DBG] from='client.24345 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:42.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:40.289126+0000 mgr.smithi038.khesbl (mgr.14162) 285 : cluster [DBG] pgmap v242: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:42.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:33:41.153151+0000 mgr.smithi038.khesbl (mgr.14162) 286 : audit [DBG] from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:43.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:42.289552+0000 mgr.smithi038.khesbl (mgr.14162) 287 : cluster [DBG] pgmap v243: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:43.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:42.289552+0000 mgr.smithi038.khesbl (mgr.14162) 287 : cluster [DBG] pgmap v243: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:45.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:44.290020+0000 mgr.smithi038.khesbl (mgr.14162) 288 : cluster [DBG] pgmap v244: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:45.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:44.290020+0000 mgr.smithi038.khesbl (mgr.14162) 288 : cluster [DBG] pgmap v244: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:47.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:46.290529+0000 mgr.smithi038.khesbl (mgr.14162) 289 : cluster [DBG] pgmap v245: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:47.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:46.290529+0000 mgr.smithi038.khesbl (mgr.14162) 289 : cluster [DBG] pgmap v245: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:49.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:48.290874+0000 mgr.smithi038.khesbl (mgr.14162) 290 : cluster [DBG] pgmap v246: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:49.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:48.290874+0000 mgr.smithi038.khesbl (mgr.14162) 290 : cluster [DBG] pgmap v246: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:51.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:50.291247+0000 mgr.smithi038.khesbl (mgr.14162) 291 : cluster [DBG] pgmap v247: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:51.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:50.291247+0000 mgr.smithi038.khesbl (mgr.14162) 291 : cluster [DBG] pgmap v247: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:53.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:52.291707+0000 mgr.smithi038.khesbl (mgr.14162) 292 : cluster [DBG] pgmap v248: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:53.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:52.291707+0000 mgr.smithi038.khesbl (mgr.14162) 292 : cluster [DBG] pgmap v248: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:55.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:54.292159+0000 mgr.smithi038.khesbl (mgr.14162) 293 : cluster [DBG] pgmap v249: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:55.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:54.292159+0000 mgr.smithi038.khesbl (mgr.14162) 293 : cluster [DBG] pgmap v249: 65 pgs: 65 active+clean; 9.0 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:33:57.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:56.292648+0000 mgr.smithi038.khesbl (mgr.14162) 294 : cluster [DBG] pgmap v250: 65 pgs: 65 active+clean; 11 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-15T18:33:57.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:56.292648+0000 mgr.smithi038.khesbl (mgr.14162) 294 : cluster [DBG] pgmap v250: 65 pgs: 65 active+clean; 11 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-15T18:33:59.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:33:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:33:58.293066+0000 mgr.smithi038.khesbl (mgr.14162) 295 : cluster [DBG] pgmap v251: 65 pgs: 65 active+clean; 11 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-15T18:33:59.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:33:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:33:58.293066+0000 mgr.smithi038.khesbl (mgr.14162) 295 : cluster [DBG] pgmap v251: 65 pgs: 65 active+clean; 11 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-15T18:34:01.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:00.293525+0000 mgr.smithi038.khesbl (mgr.14162) 296 : cluster [DBG] pgmap v252: 65 pgs: 65 active+clean; 11 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-15T18:34:01.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:00.293525+0000 mgr.smithi038.khesbl (mgr.14162) 296 : cluster [DBG] pgmap v252: 65 pgs: 65 active+clean; 11 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-15T18:34:03.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:02.293910+0000 mgr.smithi038.khesbl (mgr.14162) 297 : cluster [DBG] pgmap v253: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-15T18:34:03.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:02.293910+0000 mgr.smithi038.khesbl (mgr.14162) 297 : cluster [DBG] pgmap v253: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-15T18:34:05.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:04.294711+0000 mgr.smithi038.khesbl (mgr.14162) 298 : cluster [DBG] pgmap v254: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-15T18:34:05.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:04.294711+0000 mgr.smithi038.khesbl (mgr.14162) 298 : cluster [DBG] pgmap v254: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-15T18:34:07.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:06.295137+0000 mgr.smithi038.khesbl (mgr.14162) 299 : cluster [DBG] pgmap v255: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-15T18:34:07.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:06.295137+0000 mgr.smithi038.khesbl (mgr.14162) 299 : cluster [DBG] pgmap v255: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-15T18:34:09.404 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:08.295555+0000 mgr.smithi038.khesbl (mgr.14162) 300 : cluster [DBG] pgmap v256: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-15T18:34:09.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:08.295555+0000 mgr.smithi038.khesbl (mgr.14162) 300 : cluster [DBG] pgmap v256: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-15T18:34:11.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.313+0000 7f75181a1700 1 -- 172.21.15.38:0/3123000459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75100f9850 msgr2=0x7f75100fbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:11.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.313+0000 7f75181a1700 1 --2- 172.21.15.38:0/3123000459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75100f9850 0x7f75100fbc80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f7500004660 tx=0x7f7500009b30).stop 2024-02-15T18:34:11.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.313+0000 7f75181a1700 1 -- 172.21.15.38:0/3123000459 shutdown_connections 2024-02-15T18:34:11.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.313+0000 7f75181a1700 1 --2- 172.21.15.38:0/3123000459 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75100f9850 0x7f75100fbc80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:11.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.313+0000 7f75181a1700 1 --2- 172.21.15.38:0/3123000459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7510060250 0x7f75100f9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:11.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.313+0000 7f75181a1700 1 -- 172.21.15.38:0/3123000459 >> 172.21.15.38:0/3123000459 conn(0x7f75100f0d40 msgr2=0x7f75100f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:11.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.313+0000 7f75181a1700 1 -- 172.21.15.38:0/3123000459 shutdown_connections 2024-02-15T18:34:11.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.313+0000 7f75181a1700 1 -- 172.21.15.38:0/3123000459 wait complete. 2024-02-15T18:34:11.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.314+0000 7f75181a1700 1 Processor -- start 2024-02-15T18:34:11.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.314+0000 7f75181a1700 1 -- start start 2024-02-15T18:34:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.314+0000 7f75181a1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7510060250 0x7f7510107000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.314+0000 7f75181a1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75100f9850 0x7f75101074f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.314+0000 7f75181a1700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7510107aa0 con 0x7f7510060250 2024-02-15T18:34:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.314+0000 7f75181a1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7510107be0 con 0x7f75100f9850 2024-02-15T18:34:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.314+0000 7f751573c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75100f9850 0x7f75101074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.314+0000 7f751573c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75100f9850 0x7f75101074f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.38:34050/0 (socket says 172.21.15.38:34050) 2024-02-15T18:34:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.314+0000 7f751573c700 1 -- 172.21.15.38:0/3658993931 learned_addr learned my addr 172.21.15.38:0/3658993931 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:34:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.315+0000 7f751573c700 1 -- 172.21.15.38:0/3658993931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7510060250 msgr2=0x7f7510107000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.315+0000 7f751573c700 1 --2- 172.21.15.38:0/3658993931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7510060250 0x7f7510107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:34:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.315+0000 7f751573c700 1 -- 172.21.15.38:0/3658993931 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7500005040 con 0x7f75100f9850 2024-02-15T18:34:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.315+0000 7f751573c700 1 --2- 172.21.15.38:0/3658993931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75100f9850 0x7f75101074f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f7500009870 tx=0x7f750000b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.315+0000 7f75067fc700 1 -- 172.21.15.38:0/3658993931 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7500019070 con 0x7f75100f9850 2024-02-15T18:34:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.315+0000 7f75067fc700 1 -- 172.21.15.38:0/3658993931 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f7500007e80 con 0x7f75100f9850 2024-02-15T18:34:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.315+0000 7f75181a1700 1 -- 172.21.15.38:0/3658993931 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7510108520 con 0x7f75100f9850 2024-02-15T18:34:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.315+0000 7f75067fc700 1 -- 172.21.15.38:0/3658993931 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f750001d3c0 con 0x7f75100f9850 2024-02-15T18:34:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.315+0000 7f75181a1700 1 -- 172.21.15.38:0/3658993931 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7510108a80 con 0x7f75100f9850 2024-02-15T18:34:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.316+0000 7f75181a1700 1 -- 172.21.15.38:0/3658993931 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7510045ad0 con 0x7f75100f9850 2024-02-15T18:34:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.317+0000 7f75067fc700 1 -- 172.21.15.38:0/3658993931 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f7500025070 con 0x7f75100f9850 2024-02-15T18:34:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.317+0000 7f75067fc700 1 --2- 172.21.15.38:0/3658993931 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f74fc05c2a0 0x7f74fc05e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:11.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.317+0000 7f75067fc700 1 -- 172.21.15.38:0/3658993931 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f7500076f00 con 0x7f75100f9850 2024-02-15T18:34:11.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.317+0000 7f7515f3d700 1 --2- 172.21.15.38:0/3658993931 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f74fc05c2a0 0x7f74fc05e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:11.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.318+0000 7f7515f3d700 1 --2- 172.21.15.38:0/3658993931 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f74fc05c2a0 0x7f74fc05e760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f750c004b70 tx=0x7f750c00b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:11.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.321+0000 7f75067fc700 1 -- 172.21.15.38:0/3658993931 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f7500049bc0 con 0x7f75100f9850 2024-02-15T18:34:11.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:10.048593+0000 mon.smithi038 (mon.0) 556 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:34:11.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:34:10.049198+0000 mgr.smithi038.khesbl (mgr.14162) 301 : cephadm [INF] Upgrade: Target is version 16.2.14-521-g0e714d9a (pacific) 2024-02-15T18:34:11.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:34:10.049287+0000 mgr.smithi038.khesbl (mgr.14162) 302 : cephadm [INF] Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc'] 2024-02-15T18:34:11.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:10.050193+0000 mon.smithi038 (mon.0) 557 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:34:11.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:34:10.051426+0000 mgr.smithi038.khesbl (mgr.14162) 303 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi038.khesbl) 2024-02-15T18:34:11.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:10.295999+0000 mgr.smithi038.khesbl (mgr.14162) 304 : cluster [DBG] pgmap v257: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-15T18:34:11.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:34:10.312997+0000 mgr.smithi038.khesbl (mgr.14162) 305 : cephadm [INF] Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc on smithi121 2024-02-15T18:34:11.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:10.048593+0000 mon.smithi038 (mon.0) 556 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:34:11.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:34:10.049198+0000 mgr.smithi038.khesbl (mgr.14162) 301 : cephadm [INF] Upgrade: Target is version 16.2.14-521-g0e714d9a (pacific) 2024-02-15T18:34:11.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:34:10.049287+0000 mgr.smithi038.khesbl (mgr.14162) 302 : cephadm [INF] Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc'] 2024-02-15T18:34:11.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:10.050193+0000 mon.smithi038 (mon.0) 557 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:34:11.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:34:10.051426+0000 mgr.smithi038.khesbl (mgr.14162) 303 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi038.khesbl) 2024-02-15T18:34:11.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:10.295999+0000 mgr.smithi038.khesbl (mgr.14162) 304 : cluster [DBG] pgmap v257: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-15T18:34:11.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:34:10.312997+0000 mgr.smithi038.khesbl (mgr.14162) 305 : cephadm [INF] Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc on smithi121 2024-02-15T18:34:11.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.532+0000 7f75181a1700 1 -- 172.21.15.38:0/3658993931 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f75100ff0e0 con 0x7f74fc05c2a0 2024-02-15T18:34:11.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.534+0000 7f75067fc700 1 -- 172.21.15.38:0/3658993931 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+372 (secure 0 0 0) 0x7f75100ff0e0 con 0x7f74fc05c2a0 2024-02-15T18:34:11.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.536+0000 7f75181a1700 1 -- 172.21.15.38:0/3658993931 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f74fc05c2a0 msgr2=0x7f74fc05e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:11.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.536+0000 7f75181a1700 1 --2- 172.21.15.38:0/3658993931 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f74fc05c2a0 0x7f74fc05e760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f750c004b70 tx=0x7f750c00b040).stop 2024-02-15T18:34:11.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.536+0000 7f75181a1700 1 -- 172.21.15.38:0/3658993931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75100f9850 msgr2=0x7f75101074f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:11.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.536+0000 7f75181a1700 1 --2- 172.21.15.38:0/3658993931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75100f9850 0x7f75101074f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f7500009870 tx=0x7f750000b040).stop 2024-02-15T18:34:11.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.536+0000 7f75181a1700 1 -- 172.21.15.38:0/3658993931 shutdown_connections 2024-02-15T18:34:11.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.536+0000 7f75181a1700 1 --2- 172.21.15.38:0/3658993931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7510060250 0x7f7510107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:34:11.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.536+0000 7f75181a1700 1 --2- 172.21.15.38:0/3658993931 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f74fc05c2a0 0x7f74fc05e760 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:11.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.536+0000 7f75181a1700 1 --2- 172.21.15.38:0/3658993931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75100f9850 0x7f75101074f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:11.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.536+0000 7f75181a1700 1 -- 172.21.15.38:0/3658993931 >> 172.21.15.38:0/3658993931 conn(0x7f75100f0d40 msgr2=0x7f75100f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:11.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.536+0000 7f75181a1700 1 -- 172.21.15.38:0/3658993931 shutdown_connections 2024-02-15T18:34:11.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.536+0000 7f75181a1700 1 -- 172.21.15.38:0/3658993931 wait complete. 2024-02-15T18:34:11.542 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-02-15T18:34:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.648+0000 7f7471f5f700 1 -- 172.21.15.38:0/146198969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f746c060310 msgr2=0x7f746c0f6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.648+0000 7f7471f5f700 1 --2- 172.21.15.38:0/146198969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f746c060310 0x7f746c0f6f10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f7460004660 tx=0x7f7460009b30).stop 2024-02-15T18:34:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.648+0000 7f7471f5f700 1 -- 172.21.15.38:0/146198969 shutdown_connections 2024-02-15T18:34:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.648+0000 7f7471f5f700 1 --2- 172.21.15.38:0/146198969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f746c0f7490 0x7f746c0f9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:34:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.648+0000 7f7471f5f700 1 --2- 172.21.15.38:0/146198969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f746c060310 0x7f746c0f6f10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:11.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.648+0000 7f7471f5f700 1 -- 172.21.15.38:0/146198969 >> 172.21.15.38:0/146198969 conn(0x7f746c0f0ba0 msgr2=0x7f746c0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:11.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.648+0000 7f7471f5f700 1 -- 172.21.15.38:0/146198969 shutdown_connections 2024-02-15T18:34:11.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.648+0000 7f7471f5f700 1 -- 172.21.15.38:0/146198969 wait complete. 2024-02-15T18:34:11.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.649+0000 7f7471f5f700 1 Processor -- start 2024-02-15T18:34:11.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.649+0000 7f7471f5f700 1 -- start start 2024-02-15T18:34:11.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.649+0000 7f7471f5f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f746c060310 0x7f746c10d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:11.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.649+0000 7f7471f5f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f746c0f7490 0x7f746c10d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:11.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.649+0000 7f7471f5f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c10df30 con 0x7f746c060310 2024-02-15T18:34:11.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.649+0000 7f7471f5f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c10e7e0 con 0x7f746c0f7490 2024-02-15T18:34:11.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.649+0000 7f746bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f746c0f7490 0x7f746c10d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:11.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.649+0000 7f7470f5d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f746c060310 0x7f746c10d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:11.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.649+0000 7f746bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f746c0f7490 0x7f746c10d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.38:34062/0 (socket says 172.21.15.38:34062) 2024-02-15T18:34:11.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.649+0000 7f746bfff700 1 -- 172.21.15.38:0/1969483879 learned_addr learned my addr 172.21.15.38:0/1969483879 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:34:11.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.650+0000 7f7470f5d700 1 -- 172.21.15.38:0/1969483879 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f746c0f7490 msgr2=0x7f746c10d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:11.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.650+0000 7f7470f5d700 1 --2- 172.21.15.38:0/1969483879 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f746c0f7490 0x7f746c10d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:11.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.650+0000 7f7470f5d700 1 -- 172.21.15.38:0/1969483879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7460005040 con 0x7f746c060310 2024-02-15T18:34:11.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.650+0000 7f746bfff700 1 --2- 172.21.15.38:0/1969483879 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f746c0f7490 0x7f746c10d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:34:11.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.650+0000 7f7470f5d700 1 --2- 172.21.15.38:0/1969483879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f746c060310 0x7f746c10d400 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f7460009f50 tx=0x7f7460009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:11.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.650+0000 7f74697fa700 1 -- 172.21.15.38:0/1969483879 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7460018070 con 0x7f746c060310 2024-02-15T18:34:11.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.650+0000 7f74697fa700 1 -- 172.21.15.38:0/1969483879 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f7460007c70 con 0x7f746c060310 2024-02-15T18:34:11.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.650+0000 7f7471f5f700 1 -- 172.21.15.38:0/1969483879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746c10e9a0 con 0x7f746c060310 2024-02-15T18:34:11.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.650+0000 7f74697fa700 1 -- 172.21.15.38:0/1969483879 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7460012620 con 0x7f746c060310 2024-02-15T18:34:11.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.650+0000 7f7471f5f700 1 -- 172.21.15.38:0/1969483879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746c10ee50 con 0x7f746c060310 2024-02-15T18:34:11.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.651+0000 7f7471f5f700 1 -- 172.21.15.38:0/1969483879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f746c059100 con 0x7f746c060310 2024-02-15T18:34:11.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.652+0000 7f74697fa700 1 -- 172.21.15.38:0/1969483879 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f746001da10 con 0x7f746c060310 2024-02-15T18:34:11.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.653+0000 7f74697fa700 1 --2- 172.21.15.38:0/1969483879 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f745c05c300 0x7f745c05e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:11.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.653+0000 7f74697fa700 1 -- 172.21.15.38:0/1969483879 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f74600780c0 con 0x7f746c060310 2024-02-15T18:34:11.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.654+0000 7f746bfff700 1 --2- 172.21.15.38:0/1969483879 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f745c05c300 0x7f745c05e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:11.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.654+0000 7f746bfff700 1 --2- 172.21.15.38:0/1969483879 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f745c05c300 0x7f745c05e7c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f7458009e20 tx=0x7f7458006ef0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:11.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.656+0000 7f74697fa700 1 -- 172.21.15.38:0/1969483879 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f746004ae50 con 0x7f746c060310 2024-02-15T18:34:11.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.850+0000 7f7471f5f700 1 -- 172.21.15.38:0/1969483879 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f746c105560 con 0x7f745c05c300 2024-02-15T18:34:11.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.851+0000 7f74697fa700 1 -- 172.21.15.38:0/1969483879 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+372 (secure 0 0 0) 0x7f746c105560 con 0x7f745c05c300 2024-02-15T18:34:11.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.854+0000 7f7471f5f700 1 -- 172.21.15.38:0/1969483879 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f745c05c300 msgr2=0x7f745c05e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:11.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.854+0000 7f7471f5f700 1 --2- 172.21.15.38:0/1969483879 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f745c05c300 0x7f745c05e7c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f7458009e20 tx=0x7f7458006ef0).stop 2024-02-15T18:34:11.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.854+0000 7f7471f5f700 1 -- 172.21.15.38:0/1969483879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f746c060310 msgr2=0x7f746c10d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:11.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.854+0000 7f7471f5f700 1 --2- 172.21.15.38:0/1969483879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f746c060310 0x7f746c10d400 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f7460009f50 tx=0x7f7460009b30).stop 2024-02-15T18:34:11.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.854+0000 7f7471f5f700 1 -- 172.21.15.38:0/1969483879 shutdown_connections 2024-02-15T18:34:11.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.854+0000 7f7471f5f700 1 --2- 172.21.15.38:0/1969483879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f746c060310 0x7f746c10d400 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:11.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.854+0000 7f7471f5f700 1 --2- 172.21.15.38:0/1969483879 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f745c05c300 0x7f745c05e7c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:11.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.854+0000 7f7471f5f700 1 --2- 172.21.15.38:0/1969483879 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f746c0f7490 0x7f746c10d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:11.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.854+0000 7f7471f5f700 1 -- 172.21.15.38:0/1969483879 >> 172.21.15.38:0/1969483879 conn(0x7f746c0f0ba0 msgr2=0x7f746c0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:11.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.854+0000 7f7471f5f700 1 -- 172.21.15.38:0/1969483879 shutdown_connections 2024-02-15T18:34:11.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.854+0000 7f7471f5f700 1 -- 172.21.15.38:0/1969483879 wait complete. 2024-02-15T18:34:11.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.961+0000 7f1d3359e700 1 -- 172.21.15.38:0/4274731595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d340f96b0 msgr2=0x7f1d340fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:11.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.961+0000 7f1d3359e700 1 --2- 172.21.15.38:0/4274731595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d340f96b0 0x7f1d340fbae0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f1d28004660 tx=0x7f1d28009b30).stop 2024-02-15T18:34:11.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.961+0000 7f1d3359e700 1 -- 172.21.15.38:0/4274731595 shutdown_connections 2024-02-15T18:34:11.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.962+0000 7f1d3359e700 1 --2- 172.21.15.38:0/4274731595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d340f96b0 0x7f1d340fbae0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:11.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.962+0000 7f1d3359e700 1 --2- 172.21.15.38:0/4274731595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d340602f0 0x7f1d340f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:11.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.962+0000 7f1d3359e700 1 -- 172.21.15.38:0/4274731595 >> 172.21.15.38:0/4274731595 conn(0x7f1d340f0b90 msgr2=0x7f1d340f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:11.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.962+0000 7f1d3359e700 1 -- 172.21.15.38:0/4274731595 shutdown_connections 2024-02-15T18:34:11.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.962+0000 7f1d3359e700 1 -- 172.21.15.38:0/4274731595 wait complete. 2024-02-15T18:34:11.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.962+0000 7f1d3359e700 1 Processor -- start 2024-02-15T18:34:11.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.962+0000 7f1d3359e700 1 -- start start 2024-02-15T18:34:11.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.962+0000 7f1d3359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d340602f0 0x7f1d34106e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:11.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.962+0000 7f1d3359e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d340f96b0 0x7f1d34107300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:11.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.963+0000 7f1d3359e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d34107900 con 0x7f1d340f96b0 2024-02-15T18:34:11.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.963+0000 7f1d3359e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d34108110 con 0x7f1d340602f0 2024-02-15T18:34:11.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.963+0000 7f1d31d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d340f96b0 0x7f1d34107300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:11.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.963+0000 7f1d31d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d340f96b0 0x7f1d34107300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33972/0 (socket says 172.21.15.38:33972) 2024-02-15T18:34:11.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.963+0000 7f1d31d9b700 1 -- 172.21.15.38:0/3968605693 learned_addr learned my addr 172.21.15.38:0/3968605693 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:34:11.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.963+0000 7f1d3259c700 1 --2- 172.21.15.38:0/3968605693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d340602f0 0x7f1d34106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:11.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.963+0000 7f1d31d9b700 1 -- 172.21.15.38:0/3968605693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d340602f0 msgr2=0x7f1d34106e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:11.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.963+0000 7f1d31d9b700 1 --2- 172.21.15.38:0/3968605693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d340602f0 0x7f1d34106e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:11.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.963+0000 7f1d31d9b700 1 -- 172.21.15.38:0/3968605693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d28005040 con 0x7f1d340f96b0 2024-02-15T18:34:11.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.963+0000 7f1d31d9b700 1 --2- 172.21.15.38:0/3968605693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d340f96b0 0x7f1d34107300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f1d28002040 tx=0x7f1d2800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:11.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.963+0000 7f1d22ffd700 1 -- 172.21.15.38:0/3968605693 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d28019070 con 0x7f1d340f96b0 2024-02-15T18:34:11.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.963+0000 7f1d3359e700 1 -- 172.21.15.38:0/3968605693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d34108330 con 0x7f1d340f96b0 2024-02-15T18:34:11.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.963+0000 7f1d22ffd700 1 -- 172.21.15.38:0/3968605693 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f1d28007e80 con 0x7f1d340f96b0 2024-02-15T18:34:11.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.963+0000 7f1d22ffd700 1 -- 172.21.15.38:0/3968605693 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d28014db0 con 0x7f1d340f96b0 2024-02-15T18:34:11.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.964+0000 7f1d3359e700 1 -- 172.21.15.38:0/3968605693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d34108890 con 0x7f1d340f96b0 2024-02-15T18:34:11.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.965+0000 7f1d3359e700 1 -- 172.21.15.38:0/3968605693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d34059100 con 0x7f1d340f96b0 2024-02-15T18:34:11.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.965+0000 7f1d22ffd700 1 -- 172.21.15.38:0/3968605693 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f1d28025070 con 0x7f1d340f96b0 2024-02-15T18:34:11.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.966+0000 7f1d22ffd700 1 --2- 172.21.15.38:0/3968605693 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1d1c05c300 0x7f1d1c05e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:11.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.966+0000 7f1d22ffd700 1 -- 172.21.15.38:0/3968605693 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f1d28076cb0 con 0x7f1d340f96b0 2024-02-15T18:34:11.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.966+0000 7f1d3259c700 1 --2- 172.21.15.38:0/3968605693 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1d1c05c300 0x7f1d1c05e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:11.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.966+0000 7f1d3259c700 1 --2- 172.21.15.38:0/3968605693 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1d1c05c300 0x7f1d1c05e7c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f1d34106b80 tx=0x7f1d2400b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:11.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:11.970+0000 7f1d22ffd700 1 -- 172.21.15.38:0/3968605693 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f1d280499f0 con 0x7f1d340f96b0 2024-02-15T18:34:12.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.186+0000 7f1d3359e700 1 -- 172.21.15.38:0/3968605693 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d340fef40 con 0x7f1d1c05c300 2024-02-15T18:34:12.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.200+0000 7f1d22ffd700 1 -- 172.21.15.38:0/3968605693 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2684 (secure 0 0 0) 0x7f1d340fef40 con 0x7f1d1c05c300 2024-02-15T18:34:12.192 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE VERSION IMAGE ID CONTAINER ID 2024-02-15T18:34:12.192 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (5m) 53s ago 7m 0.20.0 0881eb8f169f c2034c318e7c 2024-02-15T18:34:12.192 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (7m) 53s ago 7m 16.2.4 8d91d370c2b8 0fd3f945d009 2024-02-15T18:34:12.192 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi121 smithi121 running (5m) 58s ago 5m 16.2.4 8d91d370c2b8 affc02db354e 2024-02-15T18:34:12.192 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (4m) 53s ago 6m 6.7.4 557c83e11646 2dcccfbfbd2a 2024-02-15T18:34:12.192 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.hxoggl smithi038 running (78s) 53s ago 77s 16.2.4 8d91d370c2b8 be2533754b9c 2024-02-15T18:34:12.192 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.qgkzgn smithi038 running (72s) 53s ago 71s 16.2.4 8d91d370c2b8 dd3ffafba9bd 2024-02-15T18:34:12.192 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.mvumlh smithi121 running (83s) 58s ago 82s 16.2.4 8d91d370c2b8 88b7d876a39b 2024-02-15T18:34:12.192 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.ovuzki smithi121 running (75s) 58s ago 75s 16.2.4 8d91d370c2b8 21cacf0012f9 2024-02-15T18:34:12.192 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.khesbl smithi038 *:9283 running (8m) 53s ago 8m 16.2.4 8d91d370c2b8 b0d099c255f4 2024-02-15T18:34:12.193 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi121.ckiryi smithi121 *:8443,9283 running (5m) 58s ago 5m 16.2.4 8d91d370c2b8 628756a555db 2024-02-15T18:34:12.193 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (8m) 53s ago 8m 16.2.4 8d91d370c2b8 01b30eaebcfe 2024-02-15T18:34:12.193 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi121 smithi121 running (5m) 58s ago 5m 16.2.4 8d91d370c2b8 8f588ecacb71 2024-02-15T18:34:12.193 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (6m) 53s ago 6m 0.18.1 e5a616e4b9cf 037f60e38158 2024-02-15T18:34:12.193 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi121 smithi121 *:9100 running (5m) 58s ago 5m 0.18.1 e5a616e4b9cf a867bff85e18 2024-02-15T18:34:12.193 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (5m) 53s ago 5m 16.2.4 8d91d370c2b8 51512003a2ef 2024-02-15T18:34:12.193 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (4m) 53s ago 4m 16.2.4 8d91d370c2b8 16dbdf20b170 2024-02-15T18:34:12.193 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (4m) 53s ago 4m 16.2.4 8d91d370c2b8 6d6ee813e5be 2024-02-15T18:34:12.193 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi121 running (3m) 58s ago 3m 16.2.4 8d91d370c2b8 cba2d9ed4730 2024-02-15T18:34:12.193 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi121 running (3m) 58s ago 3m 16.2.4 8d91d370c2b8 ac17ad7dc628 2024-02-15T18:34:12.193 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi121 running (3m) 58s ago 3m 16.2.4 8d91d370c2b8 cdc8c669c4db 2024-02-15T18:34:12.193 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (4m) 53s ago 6m 2.18.1 de242295e225 574a5d75991e 2024-02-15T18:34:12.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.203+0000 7f1d3359e700 1 -- 172.21.15.38:0/3968605693 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1d1c05c300 msgr2=0x7f1d1c05e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:12.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.203+0000 7f1d3359e700 1 --2- 172.21.15.38:0/3968605693 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1d1c05c300 0x7f1d1c05e7c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f1d34106b80 tx=0x7f1d2400b040).stop 2024-02-15T18:34:12.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.203+0000 7f1d3359e700 1 -- 172.21.15.38:0/3968605693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d340f96b0 msgr2=0x7f1d34107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:12.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.203+0000 7f1d3359e700 1 --2- 172.21.15.38:0/3968605693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d340f96b0 0x7f1d34107300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f1d28002040 tx=0x7f1d2800b040).stop 2024-02-15T18:34:12.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.203+0000 7f1d3359e700 1 -- 172.21.15.38:0/3968605693 shutdown_connections 2024-02-15T18:34:12.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.203+0000 7f1d3359e700 1 --2- 172.21.15.38:0/3968605693 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1d1c05c300 0x7f1d1c05e7c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:12.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.203+0000 7f1d3359e700 1 --2- 172.21.15.38:0/3968605693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d340602f0 0x7f1d34106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:12.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.203+0000 7f1d3359e700 1 --2- 172.21.15.38:0/3968605693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d340f96b0 0x7f1d34107300 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:12.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.203+0000 7f1d3359e700 1 -- 172.21.15.38:0/3968605693 >> 172.21.15.38:0/3968605693 conn(0x7f1d340f0b90 msgr2=0x7f1d340f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:12.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.203+0000 7f1d3359e700 1 -- 172.21.15.38:0/3968605693 shutdown_connections 2024-02-15T18:34:12.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.203+0000 7f1d3359e700 1 -- 172.21.15.38:0/3968605693 wait complete. 2024-02-15T18:34:12.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.336+0000 7f89ca442700 1 -- 172.21.15.38:0/3892336742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89c40f5430 msgr2=0x7f89c40f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:12.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.336+0000 7f89ca442700 1 --2- 172.21.15.38:0/3892336742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89c40f5430 0x7f89c40f5850 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f89c0004660 tx=0x7f89c0009b30).stop 2024-02-15T18:34:12.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.336+0000 7f89ca442700 1 -- 172.21.15.38:0/3892336742 shutdown_connections 2024-02-15T18:34:12.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.336+0000 7f89ca442700 1 --2- 172.21.15.38:0/3892336742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89c40f66d0 0x7f89c40f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:12.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.337+0000 7f89ca442700 1 --2- 172.21.15.38:0/3892336742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89c40f5430 0x7f89c40f5850 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:12.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.337+0000 7f89ca442700 1 -- 172.21.15.38:0/3892336742 >> 172.21.15.38:0/3892336742 conn(0x7f89c40f0b90 msgr2=0x7f89c40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:12.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.337+0000 7f89ca442700 1 -- 172.21.15.38:0/3892336742 shutdown_connections 2024-02-15T18:34:12.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.337+0000 7f89ca442700 1 -- 172.21.15.38:0/3892336742 wait complete. 2024-02-15T18:34:12.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.337+0000 7f89ca442700 1 Processor -- start 2024-02-15T18:34:12.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.337+0000 7f89ca442700 1 -- start start 2024-02-15T18:34:12.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.338+0000 7f89ca442700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89c40f5430 0x7f89c4106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:12.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.338+0000 7f89ca442700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89c40f66d0 0x7f89c41073b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:12.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.338+0000 7f89ca442700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c4107960 con 0x7f89c40f5430 2024-02-15T18:34:12.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.338+0000 7f89ca442700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c4108160 con 0x7f89c40f66d0 2024-02-15T18:34:12.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.338+0000 7f89c8c3f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89c40f66d0 0x7f89c41073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:12.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.338+0000 7f89c9440700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89c40f5430 0x7f89c4106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:12.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.338+0000 7f89c8c3f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89c40f66d0 0x7f89c41073b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.38:34104/0 (socket says 172.21.15.38:34104) 2024-02-15T18:34:12.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.338+0000 7f89c9440700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89c40f5430 0x7f89c4106ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33988/0 (socket says 172.21.15.38:33988) 2024-02-15T18:34:12.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.338+0000 7f89c8c3f700 1 -- 172.21.15.38:0/542848755 learned_addr learned my addr 172.21.15.38:0/542848755 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:34:12.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.338+0000 7f89c8c3f700 1 -- 172.21.15.38:0/542848755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89c40f5430 msgr2=0x7f89c4106ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:12.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.338+0000 7f89c8c3f700 1 --2- 172.21.15.38:0/542848755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89c40f5430 0x7f89c4106ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:12.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.338+0000 7f89c8c3f700 1 -- 172.21.15.38:0/542848755 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c0005040 con 0x7f89c40f66d0 2024-02-15T18:34:12.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.338+0000 7f89c8c3f700 1 --2- 172.21.15.38:0/542848755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89c40f66d0 0x7f89c41073b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f89b400e7f0 tx=0x7f89b400b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:12.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.339+0000 7f89b9ffb700 1 -- 172.21.15.38:0/542848755 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89b40092e0 con 0x7f89c40f66d0 2024-02-15T18:34:12.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.339+0000 7f89ca442700 1 -- 172.21.15.38:0/542848755 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c4108440 con 0x7f89c40f66d0 2024-02-15T18:34:12.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.339+0000 7f89b9ffb700 1 -- 172.21.15.38:0/542848755 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f89b400f040 con 0x7f89c40f66d0 2024-02-15T18:34:12.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.339+0000 7f89b9ffb700 1 -- 172.21.15.38:0/542848755 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89b4014850 con 0x7f89c40f66d0 2024-02-15T18:34:12.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.339+0000 7f89ca442700 1 -- 172.21.15.38:0/542848755 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c4108a00 con 0x7f89c40f66d0 2024-02-15T18:34:12.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.340+0000 7f89ca442700 1 -- 172.21.15.38:0/542848755 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a4004fa0 con 0x7f89c40f66d0 2024-02-15T18:34:12.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.341+0000 7f89b9ffb700 1 -- 172.21.15.38:0/542848755 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f89b401b030 con 0x7f89c40f66d0 2024-02-15T18:34:12.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.341+0000 7f89b9ffb700 1 --2- 172.21.15.38:0/542848755 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f89b005c300 0x7f89b005e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:12.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.341+0000 7f89b9ffb700 1 -- 172.21.15.38:0/542848755 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f89b4077cc0 con 0x7f89c40f66d0 2024-02-15T18:34:12.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.342+0000 7f89c9440700 1 --2- 172.21.15.38:0/542848755 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f89b005c300 0x7f89b005e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:12.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.342+0000 7f89c9440700 1 --2- 172.21.15.38:0/542848755 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f89b005c300 0x7f89b005e7c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f89c0009720 tx=0x7f89c0016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:12.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.348+0000 7f89b9ffb700 1 -- 172.21.15.38:0/542848755 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f89b407b060 con 0x7f89c40f66d0 2024-02-15T18:34:12.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:11.534209+0000 mgr.smithi038.khesbl (mgr.14162) 306 : audit [DBG] from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:12.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:11.851813+0000 mgr.smithi038.khesbl (mgr.14162) 307 : audit [DBG] from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:12.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:11.534209+0000 mgr.smithi038.khesbl (mgr.14162) 306 : audit [DBG] from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:12.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:11.851813+0000 mgr.smithi038.khesbl (mgr.14162) 307 : audit [DBG] from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:12.614 INFO:tasks.workunit.client.1.smithi121.stderr:Updating files: 85% (10427/12167) Updating files: 86% (10464/12167) Updating files: 87% (10586/12167) Updating files: 88% (10707/12167) Updating files: 89% (10829/12167) Updating files: 90% (10951/12167) Updating files: 91% (11072/12167) Updating files: 92% (11194/12167) Updating files: 93% (11316/12167) Updating files: 94% (11437/12167) Updating files: 95% (11559/12167) Updating files: 96% (11681/12167) Updating files: 97% (11802/12167) Updating files: 98% (11924/12167) Updating files: 99% (12046/12167) Updating files: 100% (12167/12167) Updating files: 100% (12167/12167), done. 2024-02-15T18:34:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.671+0000 7f89ca442700 1 -- 172.21.15.38:0/542848755 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f89a4005e80 con 0x7f89c40f66d0 2024-02-15T18:34:12.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.672+0000 7f89b9ffb700 1 -- 172.21.15.38:0/542848755 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+573 (secure 0 0 0) 0x7f89b4019070 con 0x7f89c40f66d0 2024-02-15T18:34:12.663 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:34:12.663 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-02-15T18:34:12.663 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2024-02-15T18:34:12.663 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:34:12.663 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-02-15T18:34:12.663 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2024-02-15T18:34:12.663 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:34:12.663 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-02-15T18:34:12.663 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2024-02-15T18:34:12.664 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:34:12.664 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-02-15T18:34:12.664 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:34:12.664 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:34:12.664 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-02-15T18:34:12.664 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 14 2024-02-15T18:34:12.664 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-02-15T18:34:12.664 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:34:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.674+0000 7f89ca442700 1 -- 172.21.15.38:0/542848755 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f89b005c300 msgr2=0x7f89b005e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.674+0000 7f89ca442700 1 --2- 172.21.15.38:0/542848755 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f89b005c300 0x7f89b005e7c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f89c0009720 tx=0x7f89c0016040).stop 2024-02-15T18:34:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.674+0000 7f89ca442700 1 -- 172.21.15.38:0/542848755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89c40f66d0 msgr2=0x7f89c41073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.674+0000 7f89ca442700 1 --2- 172.21.15.38:0/542848755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89c40f66d0 0x7f89c41073b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f89b400e7f0 tx=0x7f89b400b040).stop 2024-02-15T18:34:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.674+0000 7f89ca442700 1 -- 172.21.15.38:0/542848755 shutdown_connections 2024-02-15T18:34:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.674+0000 7f89ca442700 1 --2- 172.21.15.38:0/542848755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89c40f5430 0x7f89c4106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.674+0000 7f89ca442700 1 --2- 172.21.15.38:0/542848755 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f89b005c300 0x7f89b005e7c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:12.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.674+0000 7f89ca442700 1 --2- 172.21.15.38:0/542848755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89c40f66d0 0x7f89c41073b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:12.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.675+0000 7f89ca442700 1 -- 172.21.15.38:0/542848755 >> 172.21.15.38:0/542848755 conn(0x7f89c40f0b90 msgr2=0x7f89c405f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:12.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.675+0000 7f89ca442700 1 -- 172.21.15.38:0/542848755 shutdown_connections 2024-02-15T18:34:12.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.675+0000 7f89ca442700 1 -- 172.21.15.38:0/542848755 wait complete. 2024-02-15T18:34:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.788+0000 7fe013067700 1 -- 172.21.15.38:0/3248413148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe00c0f6530 msgr2=0x7fe00c0f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.788+0000 7fe013067700 1 --2- 172.21.15.38:0/3248413148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe00c0f6530 0x7fe00c0f6950 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7fe008004660 tx=0x7fe008009b30).stop 2024-02-15T18:34:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.788+0000 7fe013067700 1 -- 172.21.15.38:0/3248413148 shutdown_connections 2024-02-15T18:34:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.788+0000 7fe013067700 1 --2- 172.21.15.38:0/3248413148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe00c0f77d0 0x7fe00c0f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.788+0000 7fe013067700 1 --2- 172.21.15.38:0/3248413148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe00c0f6530 0x7fe00c0f6950 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.788+0000 7fe013067700 1 -- 172.21.15.38:0/3248413148 >> 172.21.15.38:0/3248413148 conn(0x7fe00c0f1c90 msgr2=0x7fe00c0f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.788+0000 7fe013067700 1 -- 172.21.15.38:0/3248413148 shutdown_connections 2024-02-15T18:34:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.788+0000 7fe013067700 1 -- 172.21.15.38:0/3248413148 wait complete. 2024-02-15T18:34:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.789+0000 7fe013067700 1 Processor -- start 2024-02-15T18:34:12.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.789+0000 7fe013067700 1 -- start start 2024-02-15T18:34:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.789+0000 7fe013067700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe00c0f6530 0x7fe00c106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.789+0000 7fe013067700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe00c0f77d0 0x7fe00c1073b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.789+0000 7fe013067700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c107960 con 0x7fe00c0f6530 2024-02-15T18:34:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.789+0000 7fe013067700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c108160 con 0x7fe00c0f77d0 2024-02-15T18:34:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.789+0000 7fe011864700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe00c0f77d0 0x7fe00c1073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.789+0000 7fe011864700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe00c0f77d0 0x7fe00c1073b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.38:34114/0 (socket says 172.21.15.38:34114) 2024-02-15T18:34:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.789+0000 7fe011864700 1 -- 172.21.15.38:0/2207366028 learned_addr learned my addr 172.21.15.38:0/2207366028 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:34:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.789+0000 7fe012065700 1 --2- 172.21.15.38:0/2207366028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe00c0f6530 0x7fe00c106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.789+0000 7fe011864700 1 -- 172.21.15.38:0/2207366028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe00c0f6530 msgr2=0x7fe00c106ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.789+0000 7fe011864700 1 --2- 172.21.15.38:0/2207366028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe00c0f6530 0x7fe00c106ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.789+0000 7fe011864700 1 -- 172.21.15.38:0/2207366028 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe008005040 con 0x7fe00c0f77d0 2024-02-15T18:34:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.790+0000 7fe011864700 1 --2- 172.21.15.38:0/2207366028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe00c0f77d0 0x7fe00c1073b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fdffc00e7f0 tx=0x7fdffc00b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.790+0000 7fe002ffd700 1 -- 172.21.15.38:0/2207366028 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdffc0092e0 con 0x7fe00c0f77d0 2024-02-15T18:34:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.790+0000 7fe013067700 1 -- 172.21.15.38:0/2207366028 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00c1083e0 con 0x7fe00c0f77d0 2024-02-15T18:34:12.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.790+0000 7fe013067700 1 -- 172.21.15.38:0/2207366028 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00c108950 con 0x7fe00c0f77d0 2024-02-15T18:34:12.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.791+0000 7fe002ffd700 1 -- 172.21.15.38:0/2207366028 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fdffc00f040 con 0x7fe00c0f77d0 2024-02-15T18:34:12.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.791+0000 7fe013067700 1 -- 172.21.15.38:0/2207366028 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe00c059100 con 0x7fe00c0f77d0 2024-02-15T18:34:12.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.791+0000 7fe002ffd700 1 -- 172.21.15.38:0/2207366028 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdffc014850 con 0x7fe00c0f77d0 2024-02-15T18:34:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.793+0000 7fe002ffd700 1 -- 172.21.15.38:0/2207366028 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fdffc01b030 con 0x7fe00c0f77d0 2024-02-15T18:34:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.793+0000 7fe002ffd700 1 --2- 172.21.15.38:0/2207366028 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fdff805c300 0x7fdff805e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:12.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.793+0000 7fe002ffd700 1 -- 172.21.15.38:0/2207366028 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fdffc076140 con 0x7fe00c0f77d0 2024-02-15T18:34:12.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.793+0000 7fe012065700 1 --2- 172.21.15.38:0/2207366028 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fdff805c300 0x7fdff805e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:12.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.794+0000 7fe012065700 1 --2- 172.21.15.38:0/2207366028 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fdff805c300 0x7fdff805e7c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fe008009720 tx=0x7fe008016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:12.799+0000 7fe002ffd700 1 -- 172.21.15.38:0/2207366028 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fdffc048df0 con 0x7fe00c0f77d0 2024-02-15T18:34:13.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.010+0000 7fe013067700 1 -- 172.21.15.38:0/2207366028 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe00c0feee0 con 0x7fe00c0f77d0 2024-02-15T18:34:13.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.010+0000 7fe002ffd700 1 -- 172.21.15.38:0/2207366028 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 11 v11) v1 ==== 76+0+1626 (secure 0 0 0) 0x7fdffc019670 con 0x7fe00c0f77d0 2024-02-15T18:34:13.002 INFO:teuthology.orchestra.run.smithi038.stdout:e11 2024-02-15T18:34:13.002 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:34:13.002 INFO:teuthology.orchestra.run.smithi038.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:34:13.002 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-02-15T18:34:13.002 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:34:13.002 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:34:13.002 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-02-15T18:34:13.002 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 10 2024-02-15T18:34:13.002 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 2024-02-15T18:34:13.002 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:34:13.002 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-02-15T18:33:02.882026+0000 2024-02-15T18:34:13.002 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-02-15T18:34:13.003 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-02-15T18:34:13.003 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-02-15T18:34:13.003 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-02-15T18:34:13.003 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-02-15T18:34:13.003 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-02-15T18:34:13.003 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-02-15T18:34:13.003 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-02-15T18:34:13.003 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:34:13.003 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-02-15T18:34:13.003 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-02-15T18:34:13.003 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=24261,1=24275} 2024-02-15T18:34:13.003 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-02-15T18:34:13.003 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-02-15T18:34:13.003 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-02-15T18:34:13.004 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-02-15T18:34:13.004 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-02-15T18:34:13.004 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-02-15T18:34:13.004 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-02-15T18:34:13.004 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-02-15T18:34:13.004 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.mvumlh{0:24261} state up:active seq 3 join_fscid=1 addr [v2:172.21.15.121:6824/1422707571,v1:172.21.15.121:6825/1422707571]] 2024-02-15T18:34:13.004 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.ovuzki{1:24275} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845]] 2024-02-15T18:34:13.004 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:34:13.004 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:34:13.004 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-02-15T18:34:13.004 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:34:13.004 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.hxoggl{-1:14446} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.38:6826/3822880235,v1:172.21.15.38:6827/3822880235]] 2024-02-15T18:34:13.005 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.qgkzgn{-1:14456} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.38:6828/703030231,v1:172.21.15.38:6829/703030231]] 2024-02-15T18:34:13.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.013+0000 7fe013067700 1 -- 172.21.15.38:0/2207366028 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fdff805c300 msgr2=0x7fdff805e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:13.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.013+0000 7fe013067700 1 --2- 172.21.15.38:0/2207366028 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fdff805c300 0x7fdff805e7c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fe008009720 tx=0x7fe008016040).stop 2024-02-15T18:34:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.013+0000 7fe013067700 1 -- 172.21.15.38:0/2207366028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe00c0f77d0 msgr2=0x7fe00c1073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.013+0000 7fe013067700 1 --2- 172.21.15.38:0/2207366028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe00c0f77d0 0x7fe00c1073b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fdffc00e7f0 tx=0x7fdffc00b040).stop 2024-02-15T18:34:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.013+0000 7fe013067700 1 -- 172.21.15.38:0/2207366028 shutdown_connections 2024-02-15T18:34:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.013+0000 7fe013067700 1 --2- 172.21.15.38:0/2207366028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe00c0f6530 0x7fe00c106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.013+0000 7fe013067700 1 --2- 172.21.15.38:0/2207366028 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fdff805c300 0x7fdff805e7c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.013+0000 7fe013067700 1 --2- 172.21.15.38:0/2207366028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe00c0f77d0 0x7fe00c1073b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.013+0000 7fe013067700 1 -- 172.21.15.38:0/2207366028 >> 172.21.15.38:0/2207366028 conn(0x7fe00c0f1c90 msgr2=0x7fe00c0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.013+0000 7fe013067700 1 -- 172.21.15.38:0/2207366028 shutdown_connections 2024-02-15T18:34:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.013+0000 7fe013067700 1 -- 172.21.15.38:0/2207366028 wait complete. 2024-02-15T18:34:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 11 2024-02-15T18:34:13.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.140+0000 7fd3b533d700 1 -- 172.21.15.38:0/3657018144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3b00f6860 msgr2=0x7fd3b00f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:13.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.140+0000 7fd3b533d700 1 --2- 172.21.15.38:0/3657018144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3b00f6860 0x7fd3b00f6cc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fd3a4004660 tx=0x7fd3a4009b30).stop 2024-02-15T18:34:13.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.140+0000 7fd3b533d700 1 -- 172.21.15.38:0/3657018144 shutdown_connections 2024-02-15T18:34:13.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.140+0000 7fd3b533d700 1 --2- 172.21.15.38:0/3657018144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3b00f6860 0x7fd3b00f6cc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:13.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.140+0000 7fd3b533d700 1 --2- 172.21.15.38:0/3657018144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3b00f55c0 0x7fd3b00f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:13.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.140+0000 7fd3b533d700 1 -- 172.21.15.38:0/3657018144 >> 172.21.15.38:0/3657018144 conn(0x7fd3b00f0d40 msgr2=0x7fd3b00f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:13.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.140+0000 7fd3b533d700 1 -- 172.21.15.38:0/3657018144 shutdown_connections 2024-02-15T18:34:13.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.140+0000 7fd3b533d700 1 -- 172.21.15.38:0/3657018144 wait complete. 2024-02-15T18:34:13.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.141+0000 7fd3b533d700 1 Processor -- start 2024-02-15T18:34:13.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.141+0000 7fd3b533d700 1 -- start start 2024-02-15T18:34:13.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.141+0000 7fd3b533d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3b00f55c0 0x7fd3b0109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:13.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.141+0000 7fd3b533d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3b00f6860 0x7fd3b010c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:13.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.141+0000 7fd3b533d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b00603c0 con 0x7fd3b00f55c0 2024-02-15T18:34:13.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.141+0000 7fd3b533d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b0060500 con 0x7fd3b00f6860 2024-02-15T18:34:13.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.141+0000 7fd3aeffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3b00f55c0 0x7fd3b0109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:13.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.141+0000 7fd3aeffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3b00f55c0 0x7fd3b0109cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34012/0 (socket says 172.21.15.38:34012) 2024-02-15T18:34:13.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.141+0000 7fd3aeffd700 1 -- 172.21.15.38:0/3561695851 learned_addr learned my addr 172.21.15.38:0/3561695851 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:34:13.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.141+0000 7fd3ae7fc700 1 --2- 172.21.15.38:0/3561695851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3b00f6860 0x7fd3b010c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:13.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.142+0000 7fd3ae7fc700 1 -- 172.21.15.38:0/3561695851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3b00f55c0 msgr2=0x7fd3b0109cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:13.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.142+0000 7fd3ae7fc700 1 --2- 172.21.15.38:0/3561695851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3b00f55c0 0x7fd3b0109cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:13.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.142+0000 7fd3ae7fc700 1 -- 172.21.15.38:0/3561695851 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a4005040 con 0x7fd3b00f6860 2024-02-15T18:34:13.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.142+0000 7fd3aeffd700 1 --2- 172.21.15.38:0/3561695851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3b00f55c0 0x7fd3b0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:34:13.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.142+0000 7fd3ae7fc700 1 --2- 172.21.15.38:0/3561695851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3b00f6860 0x7fd3b010c1d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fd3a4004660 tx=0x7fd3a400b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:13.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.142+0000 7fd3977fe700 1 -- 172.21.15.38:0/3561695851 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3a4019070 con 0x7fd3b00f6860 2024-02-15T18:34:13.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.142+0000 7fd3b533d700 1 -- 172.21.15.38:0/3561695851 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b010c720 con 0x7fd3b00f6860 2024-02-15T18:34:13.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.142+0000 7fd3b533d700 1 -- 172.21.15.38:0/3561695851 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b010cc50 con 0x7fd3b00f6860 2024-02-15T18:34:13.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.143+0000 7fd3b533d700 1 -- 172.21.15.38:0/3561695851 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b0045ad0 con 0x7fd3b00f6860 2024-02-15T18:34:13.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.143+0000 7fd3977fe700 1 -- 172.21.15.38:0/3561695851 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fd3a4007e80 con 0x7fd3b00f6860 2024-02-15T18:34:13.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.143+0000 7fd3977fe700 1 -- 172.21.15.38:0/3561695851 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3a4014e30 con 0x7fd3b00f6860 2024-02-15T18:34:13.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.144+0000 7fd3977fe700 1 -- 172.21.15.38:0/3561695851 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fd3a4025070 con 0x7fd3b00f6860 2024-02-15T18:34:13.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.144+0000 7fd3977fe700 1 --2- 172.21.15.38:0/3561695851 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd39805c2b0 0x7fd39805e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:13.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.144+0000 7fd3977fe700 1 -- 172.21.15.38:0/3561695851 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fd3a4077d20 con 0x7fd3b00f6860 2024-02-15T18:34:13.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.144+0000 7fd3aeffd700 1 --2- 172.21.15.38:0/3561695851 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd39805c2b0 0x7fd39805e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:13.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.145+0000 7fd3aeffd700 1 --2- 172.21.15.38:0/3561695851 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd39805c2b0 0x7fd39805e770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fd3a0004fd0 tx=0x7fd3a000b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.150+0000 7fd3977fe700 1 -- 172.21.15.38:0/3561695851 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fd3a404aa00 con 0x7fd3b00f6860 2024-02-15T18:34:13.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:12.187654+0000 mgr.smithi038.khesbl (mgr.14162) 308 : audit [DBG] from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:13.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:12.296479+0000 mgr.smithi038.khesbl (mgr.14162) 309 : cluster [DBG] pgmap v258: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-15T18:34:13.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:12.672300+0000 mon.smithi121 (mon.1) 18 : audit [DBG] from='client.? 172.21.15.38:0/542848755' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:13.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:13.010934+0000 mon.smithi121 (mon.1) 19 : audit [DBG] from='client.? 172.21.15.38:0/2207366028' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:34:13.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.381+0000 7fd3b533d700 1 -- 172.21.15.38:0/3561695851 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3b00fe000 con 0x7fd39805c2b0 2024-02-15T18:34:13.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.382+0000 7fd3977fe700 1 -- 172.21.15.38:0/3561695851 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+372 (secure 0 0 0) 0x7fd3b00fe000 con 0x7fd39805c2b0 2024-02-15T18:34:13.373 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:34:13.374 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:34:13.374 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-02-15T18:34:13.374 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [], 2024-02-15T18:34:13.374 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "0/21 ceph daemons upgraded", 2024-02-15T18:34:13.374 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc image on host smithi121" 2024-02-15T18:34:13.374 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:34:13.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.385+0000 7fd3b533d700 1 -- 172.21.15.38:0/3561695851 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd39805c2b0 msgr2=0x7fd39805e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:13.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.385+0000 7fd3b533d700 1 --2- 172.21.15.38:0/3561695851 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd39805c2b0 0x7fd39805e770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fd3a0004fd0 tx=0x7fd3a000b040).stop 2024-02-15T18:34:13.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.385+0000 7fd3b533d700 1 -- 172.21.15.38:0/3561695851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3b00f6860 msgr2=0x7fd3b010c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:13.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.385+0000 7fd3b533d700 1 --2- 172.21.15.38:0/3561695851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3b00f6860 0x7fd3b010c1d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fd3a4004660 tx=0x7fd3a400b040).stop 2024-02-15T18:34:13.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.385+0000 7fd3b533d700 1 -- 172.21.15.38:0/3561695851 shutdown_connections 2024-02-15T18:34:13.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.385+0000 7fd3b533d700 1 --2- 172.21.15.38:0/3561695851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3b00f55c0 0x7fd3b0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:13.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.385+0000 7fd3b533d700 1 --2- 172.21.15.38:0/3561695851 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd39805c2b0 0x7fd39805e770 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:13.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.385+0000 7fd3b533d700 1 --2- 172.21.15.38:0/3561695851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3b00f6860 0x7fd3b010c1d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:13.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.385+0000 7fd3b533d700 1 -- 172.21.15.38:0/3561695851 >> 172.21.15.38:0/3561695851 conn(0x7fd3b00f0d40 msgr2=0x7fd3b00fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:13.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.385+0000 7fd3b533d700 1 -- 172.21.15.38:0/3561695851 shutdown_connections 2024-02-15T18:34:13.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:13.385+0000 7fd3b533d700 1 -- 172.21.15.38:0/3561695851 wait complete. 2024-02-15T18:34:13.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:12.187654+0000 mgr.smithi038.khesbl (mgr.14162) 308 : audit [DBG] from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:13.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:12.296479+0000 mgr.smithi038.khesbl (mgr.14162) 309 : cluster [DBG] pgmap v258: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-15T18:34:13.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:12.672300+0000 mon.smithi121 (mon.1) 18 : audit [DBG] from='client.? 172.21.15.38:0/542848755' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:13.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:13.010934+0000 mon.smithi121 (mon.1) 19 : audit [DBG] from='client.? 172.21.15.38:0/2207366028' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:34:13.876 INFO:tasks.workunit.client.1.smithi121.stderr:Note: switching to 'bedfc49c6a823fb0ff75911e476e1f3cb464ad3e'. 2024-02-15T18:34:13.877 INFO:tasks.workunit.client.1.smithi121.stderr: 2024-02-15T18:34:13.877 INFO:tasks.workunit.client.1.smithi121.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-15T18:34:13.877 INFO:tasks.workunit.client.1.smithi121.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-15T18:34:13.877 INFO:tasks.workunit.client.1.smithi121.stderr:state without impacting any branches by switching back to a branch. 2024-02-15T18:34:13.877 INFO:tasks.workunit.client.1.smithi121.stderr: 2024-02-15T18:34:13.877 INFO:tasks.workunit.client.1.smithi121.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-15T18:34:13.877 INFO:tasks.workunit.client.1.smithi121.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-15T18:34:13.877 INFO:tasks.workunit.client.1.smithi121.stderr: 2024-02-15T18:34:13.877 INFO:tasks.workunit.client.1.smithi121.stderr: git switch -c 2024-02-15T18:34:13.877 INFO:tasks.workunit.client.1.smithi121.stderr: 2024-02-15T18:34:13.877 INFO:tasks.workunit.client.1.smithi121.stderr:Or undo this operation with: 2024-02-15T18:34:13.878 INFO:tasks.workunit.client.1.smithi121.stderr: 2024-02-15T18:34:13.878 INFO:tasks.workunit.client.1.smithi121.stderr: git switch - 2024-02-15T18:34:13.878 INFO:tasks.workunit.client.1.smithi121.stderr: 2024-02-15T18:34:13.878 INFO:tasks.workunit.client.1.smithi121.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-15T18:34:13.878 INFO:tasks.workunit.client.1.smithi121.stderr: 2024-02-15T18:34:13.878 INFO:tasks.workunit.client.1.smithi121.stderr:HEAD is now at bedfc49c6a8 qa/suites/orch: whitelist warnings that are expected in test environments 2024-02-15T18:34:13.882 DEBUG:teuthology.orchestra.run.smithi121:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-02-15T18:34:13.947 INFO:tasks.workunit.client.1.smithi121.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-15T18:34:13.949 INFO:tasks.workunit.client.1.smithi121.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-02-15T18:34:13.949 INFO:tasks.workunit.client.1.smithi121.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-15T18:34:14.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:13.382769+0000 mgr.smithi038.khesbl (mgr.14162) 310 : audit [DBG] from='client.24381 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:14.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:13.382769+0000 mgr.smithi038.khesbl (mgr.14162) 310 : audit [DBG] from='client.24381 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:14.672 INFO:tasks.workunit.client.1.smithi121.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-15T18:34:14.732 INFO:tasks.workunit.client.1.smithi121.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-15T18:34:14.774 INFO:tasks.workunit.client.1.smithi121.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-02-15T18:34:14.776 INFO:tasks.workunit.client.1.smithi121.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-02-15T18:34:14.776 INFO:tasks.workunit.client.1.smithi121.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-15T18:34:14.817 INFO:tasks.workunit.client.1.smithi121.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-02-15T18:34:14.820 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-02-15T18:34:14.820 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-02-15T18:34:14.883 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-02-15T18:34:14.884 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-02-15T18:34:14.885 DEBUG:teuthology.orchestra.run.smithi121:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bedfc49c6a823fb0ff75911e476e1f3cb464ad3e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-02-15T18:34:14.950 INFO:tasks.workunit.client.1.smithi121.stderr:+ mkdir -p fsstress 2024-02-15T18:34:14.952 INFO:tasks.workunit.client.1.smithi121.stderr:+ pushd fsstress 2024-02-15T18:34:14.952 INFO:tasks.workunit.client.1.smithi121.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-15T18:34:14.953 INFO:tasks.workunit.client.1.smithi121.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-02-15T18:34:15.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:14.296964+0000 mgr.smithi038.khesbl (mgr.14162) 311 : cluster [DBG] pgmap v259: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-15T18:34:15.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:14.296964+0000 mgr.smithi038.khesbl (mgr.14162) 311 : cluster [DBG] pgmap v259: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-02-15T18:34:15.991 INFO:tasks.workunit.client.1.smithi121.stderr:+ tar xzf ltp-full.tgz 2024-02-15T18:34:17.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:16.297398+0000 mgr.smithi038.khesbl (mgr.14162) 312 : cluster [DBG] pgmap v260: 65 pgs: 65 active+clean; 24 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 0 op/s 2024-02-15T18:34:17.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:16.297398+0000 mgr.smithi038.khesbl (mgr.14162) 312 : cluster [DBG] pgmap v260: 65 pgs: 65 active+clean; 24 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 0 op/s 2024-02-15T18:34:18.661 INFO:tasks.workunit.client.0.smithi038.stderr:Note: switching to 'bedfc49c6a823fb0ff75911e476e1f3cb464ad3e'. 2024-02-15T18:34:18.662 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-02-15T18:34:18.662 INFO:tasks.workunit.client.0.smithi038.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-15T18:34:18.662 INFO:tasks.workunit.client.0.smithi038.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-15T18:34:18.662 INFO:tasks.workunit.client.0.smithi038.stderr:state without impacting any branches by switching back to a branch. 2024-02-15T18:34:18.662 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-02-15T18:34:18.662 INFO:tasks.workunit.client.0.smithi038.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-15T18:34:18.662 INFO:tasks.workunit.client.0.smithi038.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-15T18:34:18.662 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-02-15T18:34:18.662 INFO:tasks.workunit.client.0.smithi038.stderr: git switch -c 2024-02-15T18:34:18.662 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-02-15T18:34:18.662 INFO:tasks.workunit.client.0.smithi038.stderr:Or undo this operation with: 2024-02-15T18:34:18.662 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-02-15T18:34:18.662 INFO:tasks.workunit.client.0.smithi038.stderr: git switch - 2024-02-15T18:34:18.662 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-02-15T18:34:18.662 INFO:tasks.workunit.client.0.smithi038.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-15T18:34:18.662 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-02-15T18:34:18.663 INFO:tasks.workunit.client.0.smithi038.stderr:HEAD is now at bedfc49c6a8 qa/suites/orch: whitelist warnings that are expected in test environments 2024-02-15T18:34:18.667 DEBUG:teuthology.orchestra.run.smithi038:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-15T18:34:18.697 INFO:tasks.workunit.client.0.smithi038.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-15T18:34:18.699 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-15T18:34:18.700 INFO:tasks.workunit.client.0.smithi038.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-15T18:34:19.245 INFO:tasks.workunit.client.0.smithi038.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-15T18:34:19.309 INFO:tasks.workunit.client.0.smithi038.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-15T18:34:19.350 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-15T18:34:19.352 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-15T18:34:19.352 INFO:tasks.workunit.client.0.smithi038.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-15T18:34:19.451 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-15T18:34:19.454 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-02-15T18:34:19.454 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-15T18:34:19.514 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-02-15T18:34:19.515 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-02-15T18:34:19.515 DEBUG:teuthology.orchestra.run.smithi038:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bedfc49c6a823fb0ff75911e476e1f3cb464ad3e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-02-15T18:34:19.579 INFO:tasks.workunit.client.0.smithi038.stderr:+ mkdir -p fsstress 2024-02-15T18:34:19.581 INFO:tasks.workunit.client.0.smithi038.stderr:+ pushd fsstress 2024-02-15T18:34:19.581 INFO:tasks.workunit.client.0.smithi038.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-15T18:34:19.581 INFO:tasks.workunit.client.0.smithi038.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-02-15T18:34:19.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:18.297739+0000 mgr.smithi038.khesbl (mgr.14162) 313 : cluster [DBG] pgmap v261: 65 pgs: 65 active+clean; 24 KiB data, 35 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 0 op/s 2024-02-15T18:34:19.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:18.297739+0000 mgr.smithi038.khesbl (mgr.14162) 313 : cluster [DBG] pgmap v261: 65 pgs: 65 active+clean; 24 KiB data, 35 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 0 op/s 2024-02-15T18:34:20.261 INFO:tasks.workunit.client.0.smithi038.stderr:+ tar xzf ltp-full.tgz 2024-02-15T18:34:21.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:20.298140+0000 mgr.smithi038.khesbl (mgr.14162) 314 : cluster [DBG] pgmap v262: 65 pgs: 65 active+clean; 24 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 0 op/s 2024-02-15T18:34:21.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:20.298140+0000 mgr.smithi038.khesbl (mgr.14162) 314 : cluster [DBG] pgmap v262: 65 pgs: 65 active+clean; 24 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 0 op/s 2024-02-15T18:34:23.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:22.298662+0000 mgr.smithi038.khesbl (mgr.14162) 315 : cluster [DBG] pgmap v263: 65 pgs: 65 active+clean; 5.9 MiB data, 52 MiB used, 536 GiB / 536 GiB avail; 526 KiB/s wr, 36 op/s 2024-02-15T18:34:23.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:22.298662+0000 mgr.smithi038.khesbl (mgr.14162) 315 : cluster [DBG] pgmap v263: 65 pgs: 65 active+clean; 5.9 MiB data, 52 MiB used, 536 GiB / 536 GiB avail; 526 KiB/s wr, 36 op/s 2024-02-15T18:34:25.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:24.299050+0000 mgr.smithi038.khesbl (mgr.14162) 316 : cluster [DBG] pgmap v264: 65 pgs: 65 active+clean; 30 MiB data, 149 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s wr, 197 op/s 2024-02-15T18:34:25.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:24.373307+0000 mon.smithi038 (mon.0) 558 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:active} 2 up:standby 2024-02-15T18:34:25.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:24.299050+0000 mgr.smithi038.khesbl (mgr.14162) 316 : cluster [DBG] pgmap v264: 65 pgs: 65 active+clean; 30 MiB data, 149 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s wr, 197 op/s 2024-02-15T18:34:25.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:24.373307+0000 mon.smithi038 (mon.0) 558 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:active} 2 up:standby 2024-02-15T18:34:27.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:26.299670+0000 mgr.smithi038.khesbl (mgr.14162) 317 : cluster [DBG] pgmap v265: 65 pgs: 65 active+clean; 61 MiB data, 251 MiB used, 536 GiB / 536 GiB avail; 5.3 MiB/s wr, 416 op/s 2024-02-15T18:34:27.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:26.299670+0000 mgr.smithi038.khesbl (mgr.14162) 317 : cluster [DBG] pgmap v265: 65 pgs: 65 active+clean; 61 MiB data, 251 MiB used, 536 GiB / 536 GiB avail; 5.3 MiB/s wr, 416 op/s 2024-02-15T18:34:29.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:28.300309+0000 mgr.smithi038.khesbl (mgr.14162) 318 : cluster [DBG] pgmap v266: 65 pgs: 65 active+clean; 63 MiB data, 280 MiB used, 536 GiB / 536 GiB avail; 5.5 MiB/s wr, 446 op/s 2024-02-15T18:34:29.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:28.300309+0000 mgr.smithi038.khesbl (mgr.14162) 318 : cluster [DBG] pgmap v266: 65 pgs: 65 active+clean; 63 MiB data, 280 MiB used, 536 GiB / 536 GiB avail; 5.5 MiB/s wr, 446 op/s 2024-02-15T18:34:31.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:30.300722+0000 mgr.smithi038.khesbl (mgr.14162) 319 : cluster [DBG] pgmap v267: 65 pgs: 65 active+clean; 73 MiB data, 315 MiB used, 536 GiB / 536 GiB avail; 6.4 MiB/s wr, 484 op/s 2024-02-15T18:34:31.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:30.300722+0000 mgr.smithi038.khesbl (mgr.14162) 319 : cluster [DBG] pgmap v267: 65 pgs: 65 active+clean; 73 MiB data, 315 MiB used, 536 GiB / 536 GiB avail; 6.4 MiB/s wr, 484 op/s 2024-02-15T18:34:33.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:32.301122+0000 mgr.smithi038.khesbl (mgr.14162) 320 : cluster [DBG] pgmap v268: 65 pgs: 65 active+clean; 81 MiB data, 338 MiB used, 536 GiB / 536 GiB avail; 7.0 MiB/s wr, 490 op/s 2024-02-15T18:34:33.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:32.301122+0000 mgr.smithi038.khesbl (mgr.14162) 320 : cluster [DBG] pgmap v268: 65 pgs: 65 active+clean; 81 MiB data, 338 MiB used, 536 GiB / 536 GiB avail; 7.0 MiB/s wr, 490 op/s 2024-02-15T18:34:35.752 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:34.301568+0000 mgr.smithi038.khesbl (mgr.14162) 321 : cluster [DBG] pgmap v269: 65 pgs: 65 active+clean; 87 MiB data, 361 MiB used, 536 GiB / 536 GiB avail; 7.1 MiB/s wr, 457 op/s 2024-02-15T18:34:35.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:34.301568+0000 mgr.smithi038.khesbl (mgr.14162) 321 : cluster [DBG] pgmap v269: 65 pgs: 65 active+clean; 87 MiB data, 361 MiB used, 536 GiB / 536 GiB avail; 7.1 MiB/s wr, 457 op/s 2024-02-15T18:34:37.550 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:34:36.263935+0000 mgr.smithi038.khesbl (mgr.14162) 322 : cephadm [INF] Upgrade: Updating mgr.smithi121.ckiryi 2024-02-15T18:34:37.550 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:36.282593+0000 mon.smithi038 (mon.0) 559 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:34:37.550 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:36.290813+0000 mon.smithi038 (mon.0) 560 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi121.ckiryi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:34:37.550 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:36.296337+0000 mon.smithi038 (mon.0) 561 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:34:37.550 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:36.297614+0000 mon.smithi038 (mon.0) 562 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:34:37.550 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:34:36.298644+0000 mgr.smithi038.khesbl (mgr.14162) 323 : cephadm [INF] Deploying daemon mgr.smithi121.ckiryi on smithi121 2024-02-15T18:34:37.550 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:36.313272+0000 mgr.smithi038.khesbl (mgr.14162) 324 : cluster [DBG] pgmap v270: 65 pgs: 65 active+clean; 101 MiB data, 389 MiB used, 536 GiB / 536 GiB avail; 6.2 MiB/s wr, 421 op/s 2024-02-15T18:34:37.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:34:36.263935+0000 mgr.smithi038.khesbl (mgr.14162) 322 : cephadm [INF] Upgrade: Updating mgr.smithi121.ckiryi 2024-02-15T18:34:37.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:36.282593+0000 mon.smithi038 (mon.0) 559 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:34:37.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:36.290813+0000 mon.smithi038 (mon.0) 560 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi121.ckiryi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:34:37.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:36.296337+0000 mon.smithi038 (mon.0) 561 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:34:37.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:36.297614+0000 mon.smithi038 (mon.0) 562 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:34:37.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:34:36.298644+0000 mgr.smithi038.khesbl (mgr.14162) 323 : cephadm [INF] Deploying daemon mgr.smithi121.ckiryi on smithi121 2024-02-15T18:34:37.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:36.313272+0000 mgr.smithi038.khesbl (mgr.14162) 324 : cluster [DBG] pgmap v270: 65 pgs: 65 active+clean; 101 MiB data, 389 MiB used, 536 GiB / 536 GiB avail; 6.2 MiB/s wr, 421 op/s 2024-02-15T18:34:39.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:38.313643+0000 mgr.smithi038.khesbl (mgr.14162) 325 : cluster [DBG] pgmap v271: 65 pgs: 65 active+clean; 112 MiB data, 489 MiB used, 536 GiB / 536 GiB avail; 4.5 MiB/s wr, 403 op/s 2024-02-15T18:34:39.804 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:38.313643+0000 mgr.smithi038.khesbl (mgr.14162) 325 : cluster [DBG] pgmap v271: 65 pgs: 65 active+clean; 112 MiB data, 489 MiB used, 536 GiB / 536 GiB avail; 4.5 MiB/s wr, 403 op/s 2024-02-15T18:34:41.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:40.314246+0000 mgr.smithi038.khesbl (mgr.14162) 326 : cluster [DBG] pgmap v272: 65 pgs: 65 active+clean; 133 MiB data, 591 MiB used, 536 GiB / 536 GiB avail; 6.2 MiB/s wr, 643 op/s 2024-02-15T18:34:41.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:40.371055+0000 mon.smithi038 (mon.0) 563 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:34:41.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:40.371883+0000 mon.smithi038 (mon.0) 564 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:34:41.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:40.769316+0000 mon.smithi038 (mon.0) 565 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:34:41.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:40.769970+0000 mon.smithi038 (mon.0) 566 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:34:41.668 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:40.314246+0000 mgr.smithi038.khesbl (mgr.14162) 326 : cluster [DBG] pgmap v272: 65 pgs: 65 active+clean; 133 MiB data, 591 MiB used, 536 GiB / 536 GiB avail; 6.2 MiB/s wr, 643 op/s 2024-02-15T18:34:41.668 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:40.371055+0000 mon.smithi038 (mon.0) 563 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:34:41.668 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:40.371883+0000 mon.smithi038 (mon.0) 564 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:34:41.668 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:40.769316+0000 mon.smithi038 (mon.0) 565 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:34:41.668 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:40.769970+0000 mon.smithi038 (mon.0) 566 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:34:42.324 INFO:tasks.workunit.client.1.smithi121.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-02-15T18:34:42.324 INFO:tasks.workunit.client.1.smithi121.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-15T18:34:42.324 INFO:tasks.workunit.client.1.smithi121.stderr:+ make 2024-02-15T18:34:42.467 INFO:tasks.workunit.client.1.smithi121.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-02-15T18:34:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.489+0000 7f6dadb81700 1 -- 172.21.15.38:0/2141638128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da80f99e0 msgr2=0x7f6da80f9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.489+0000 7f6dadb81700 1 --2- 172.21.15.38:0/2141638128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da80f99e0 0x7f6da80f9e40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f6d94004660 tx=0x7f6d94009b30).stop 2024-02-15T18:34:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.489+0000 7f6dadb81700 1 -- 172.21.15.38:0/2141638128 shutdown_connections 2024-02-15T18:34:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.489+0000 7f6dadb81700 1 --2- 172.21.15.38:0/2141638128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da80f99e0 0x7f6da80f9e40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.489+0000 7f6dadb81700 1 --2- 172.21.15.38:0/2141638128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6da80f8740 0x7f6da80f8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.489+0000 7f6dadb81700 1 -- 172.21.15.38:0/2141638128 >> 172.21.15.38:0/2141638128 conn(0x7f6da80f40e0 msgr2=0x7f6da80f64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:43.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.490+0000 7f6dadb81700 1 -- 172.21.15.38:0/2141638128 shutdown_connections 2024-02-15T18:34:43.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.490+0000 7f6dadb81700 1 -- 172.21.15.38:0/2141638128 wait complete. 2024-02-15T18:34:43.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.490+0000 7f6dadb81700 1 Processor -- start 2024-02-15T18:34:43.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.490+0000 7f6dadb81700 1 -- start start 2024-02-15T18:34:43.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.491+0000 7f6dadb81700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6da80f8740 0x7f6da8108fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:43.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.491+0000 7f6dadb81700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da80f99e0 0x7f6da81094c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:43.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.491+0000 7f6dadb81700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da8109a70 con 0x7f6da80f99e0 2024-02-15T18:34:43.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.491+0000 7f6dadb81700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da8109bb0 con 0x7f6da80f8740 2024-02-15T18:34:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.491+0000 7f6da7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da80f99e0 0x7f6da81094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.491+0000 7f6dacb7f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6da80f8740 0x7f6da8108fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.491+0000 7f6da7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da80f99e0 0x7f6da81094c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41538/0 (socket says 172.21.15.38:41538) 2024-02-15T18:34:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.491+0000 7f6dacb7f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6da80f8740 0x7f6da8108fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.38:44600/0 (socket says 172.21.15.38:44600) 2024-02-15T18:34:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.491+0000 7f6da7fff700 1 -- 172.21.15.38:0/3290778883 learned_addr learned my addr 172.21.15.38:0/3290778883 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:34:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.491+0000 7f6da7fff700 1 -- 172.21.15.38:0/3290778883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6da80f8740 msgr2=0x7f6da8108fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.491+0000 7f6da7fff700 1 --2- 172.21.15.38:0/3290778883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6da80f8740 0x7f6da8108fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.491+0000 7f6da7fff700 1 -- 172.21.15.38:0/3290778883 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d94005040 con 0x7f6da80f99e0 2024-02-15T18:34:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.491+0000 7f6da7fff700 1 --2- 172.21.15.38:0/3290778883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da80f99e0 0x7f6da81094c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f6d94000c00 tx=0x7f6d9400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.491+0000 7f6da57fa700 1 -- 172.21.15.38:0/3290778883 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d94019070 con 0x7f6da80f99e0 2024-02-15T18:34:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.492+0000 7f6da57fa700 1 -- 172.21.15.38:0/3290778883 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f6d94007e80 con 0x7f6da80f99e0 2024-02-15T18:34:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.492+0000 7f6dadb81700 1 -- 172.21.15.38:0/3290778883 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da810a4f0 con 0x7f6da80f99e0 2024-02-15T18:34:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.492+0000 7f6da57fa700 1 -- 172.21.15.38:0/3290778883 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d94014db0 con 0x7f6da80f99e0 2024-02-15T18:34:43.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.492+0000 7f6dadb81700 1 -- 172.21.15.38:0/3290778883 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da810aa50 con 0x7f6da80f99e0 2024-02-15T18:34:43.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.493+0000 7f6da57fa700 1 -- 172.21.15.38:0/3290778883 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f6d94025070 con 0x7f6da80f99e0 2024-02-15T18:34:43.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.493+0000 7f6dadb81700 1 -- 172.21.15.38:0/3290778883 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da8045ad0 con 0x7f6da80f99e0 2024-02-15T18:34:43.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.493+0000 7f6da57fa700 1 --2- 172.21.15.38:0/3290778883 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6d9805c300 0x7f6d9805e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:43.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.493+0000 7f6da57fa700 1 -- 172.21.15.38:0/3290778883 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f6d94076cd0 con 0x7f6da80f99e0 2024-02-15T18:34:43.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.493+0000 7f6dacb7f700 1 --2- 172.21.15.38:0/3290778883 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6d9805c300 0x7f6d9805e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:43.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.494+0000 7f6dacb7f700 1 --2- 172.21.15.38:0/3290778883 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6d9805c300 0x7f6d9805e7c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f6d9c009fd0 tx=0x7f6d9c00b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:43.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.497+0000 7f6da57fa700 1 -- 172.21.15.38:0/3290778883 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f6d94049a10 con 0x7f6da80f99e0 2024-02-15T18:34:43.503 INFO:tasks.workunit.client.1.smithi121.stderr:++ readlink -f fsstress 2024-02-15T18:34:43.504 INFO:tasks.workunit.client.1.smithi121.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-02-15T18:34:43.504 INFO:tasks.workunit.client.1.smithi121.stderr:+ popd 2024-02-15T18:34:43.504 INFO:tasks.workunit.client.1.smithi121.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-15T18:34:43.504 INFO:tasks.workunit.client.1.smithi121.stderr:+ popd 2024-02-15T18:34:43.505 INFO:tasks.workunit.client.1.smithi121.stdout:~/cephtest/mnt.1/client.1/tmp 2024-02-15T18:34:43.505 INFO:tasks.workunit.client.1.smithi121.stderr:++ mktemp -d -p . 2024-02-15T18:34:43.506 INFO:tasks.workunit.client.1.smithi121.stderr:+ T=./tmp.qtnpNMw7fT 2024-02-15T18:34:43.506 INFO:tasks.workunit.client.1.smithi121.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.qtnpNMw7fT -l 1 -n 1000 -p 10 -v 2024-02-15T18:34:43.507 INFO:tasks.workunit.client.1.smithi121.stdout:seed = 1707720090 2024-02-15T18:34:43.510 INFO:tasks.workunit.client.1.smithi121.stdout:0/0: dread - no filename 2024-02-15T18:34:43.510 INFO:tasks.workunit.client.1.smithi121.stdout:0/1: dwrite - no filename 2024-02-15T18:34:43.510 INFO:tasks.workunit.client.1.smithi121.stdout:0/2: chown . 1 1 2024-02-15T18:34:43.511 INFO:tasks.workunit.client.1.smithi121.stdout:1/0: write - no filename 2024-02-15T18:34:43.511 INFO:tasks.workunit.client.1.smithi121.stdout:1/1: write - no filename 2024-02-15T18:34:43.512 INFO:tasks.workunit.client.1.smithi121.stdout:0/3: creat f0 x:0 0 0 2024-02-15T18:34:43.513 INFO:tasks.workunit.client.1.smithi121.stdout:1/2: mkdir d0 0 2024-02-15T18:34:43.515 INFO:tasks.workunit.client.1.smithi121.stdout:0/4: rename f0 to f1 0 2024-02-15T18:34:43.515 INFO:tasks.workunit.client.1.smithi121.stdout:0/5: truncate f1 756151 0 2024-02-15T18:34:43.515 INFO:tasks.workunit.client.1.smithi121.stdout:3/0: stat - no entries 2024-02-15T18:34:43.516 INFO:tasks.workunit.client.1.smithi121.stdout:2/0: mknod c0 0 2024-02-15T18:34:43.516 INFO:tasks.workunit.client.1.smithi121.stdout:1/3: symlink d0/l1 0 2024-02-15T18:34:43.517 INFO:tasks.workunit.client.1.smithi121.stdout:4/0: truncate - no filename 2024-02-15T18:34:43.518 INFO:tasks.workunit.client.1.smithi121.stdout:3/1: symlink l0 0 2024-02-15T18:34:43.518 INFO:tasks.workunit.client.1.smithi121.stdout:3/2: stat l0 0 2024-02-15T18:34:43.518 INFO:tasks.workunit.client.1.smithi121.stdout:3/3: rmdir - no directory 2024-02-15T18:34:43.518 INFO:tasks.workunit.client.1.smithi121.stdout:3/4: stat l0 0 2024-02-15T18:34:43.518 INFO:tasks.workunit.client.1.smithi121.stdout:2/1: symlink l1 0 2024-02-15T18:34:43.518 INFO:tasks.workunit.client.1.smithi121.stdout:2/2: dread - no filename 2024-02-15T18:34:43.519 INFO:tasks.workunit.client.1.smithi121.stdout:1/4: creat d0/f2 x:0 0 0 2024-02-15T18:34:43.519 INFO:tasks.workunit.client.1.smithi121.stdout:1/5: stat d0 0 2024-02-15T18:34:43.519 INFO:tasks.workunit.client.1.smithi121.stdout:1/6: truncate d0/f2 628523 0 2024-02-15T18:34:43.519 INFO:tasks.workunit.client.1.smithi121.stdout:1/7: chown d0/f2 449446 1 2024-02-15T18:34:43.522 INFO:tasks.workunit.client.1.smithi121.stdout:3/5: mknod c1 0 2024-02-15T18:34:43.522 INFO:tasks.workunit.client.1.smithi121.stdout:1/8: mknod d0/c3 0 2024-02-15T18:34:43.522 INFO:tasks.workunit.client.1.smithi121.stdout:1/9: fdatasync d0/f2 0 2024-02-15T18:34:43.527 INFO:tasks.workunit.client.1.smithi121.stdout:3/6: mkdir d2 0 2024-02-15T18:34:43.527 INFO:tasks.workunit.client.1.smithi121.stdout:3/7: dwrite - no filename 2024-02-15T18:34:43.527 INFO:tasks.workunit.client.1.smithi121.stdout:3/8: mknod d2/c3 0 2024-02-15T18:34:43.527 INFO:tasks.workunit.client.1.smithi121.stdout:3/9: fdatasync - no filename 2024-02-15T18:34:43.527 INFO:tasks.workunit.client.1.smithi121.stdout:3/10: dwrite - no filename 2024-02-15T18:34:43.527 INFO:tasks.workunit.client.1.smithi121.stdout:7/0: mkdir d0 0 2024-02-15T18:34:43.529 INFO:tasks.workunit.client.1.smithi121.stdout:3/11: creat d2/f4 x:0 0 0 2024-02-15T18:34:43.530 INFO:tasks.workunit.client.1.smithi121.stdout:3/12: creat d2/f5 x:0 0 0 2024-02-15T18:34:43.530 INFO:tasks.workunit.client.1.smithi121.stdout:3/13: dread - d2/f4 zero size 2024-02-15T18:34:43.530 INFO:tasks.workunit.client.1.smithi121.stdout:3/14: truncate d2/f4 269736 0 2024-02-15T18:34:43.717 INFO:tasks.workunit.client.1.smithi121.stdout:1/10: write d0/f2 [25152,88717] 0 2024-02-15T18:34:43.717 INFO:tasks.workunit.client.1.smithi121.stdout:1/11: truncate d0/f2 1476990 0 2024-02-15T18:34:43.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:42.314704+0000 mgr.smithi038.khesbl (mgr.14162) 327 : cluster [DBG] pgmap v273: 65 pgs: 65 active+clean; 163 MiB data, 691 MiB used, 536 GiB / 536 GiB avail; 8.0 MiB/s wr, 855 op/s 2024-02-15T18:34:43.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.802+0000 7f6dadb81700 1 -- 172.21.15.38:0/3290778883 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6da810acc0 con 0x7f6d9805c300 2024-02-15T18:34:43.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.804+0000 7f6da57fa700 1 -- 172.21.15.38:0/3290778883 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+248 (secure 0 0 0) 0x7f6da810acc0 con 0x7f6d9805c300 2024-02-15T18:34:43.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.806+0000 7f6dadb81700 1 -- 172.21.15.38:0/3290778883 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6d9805c300 msgr2=0x7f6d9805e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:43.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.806+0000 7f6dadb81700 1 --2- 172.21.15.38:0/3290778883 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6d9805c300 0x7f6d9805e7c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f6d9c009fd0 tx=0x7f6d9c00b040).stop 2024-02-15T18:34:43.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.806+0000 7f6dadb81700 1 -- 172.21.15.38:0/3290778883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da80f99e0 msgr2=0x7f6da81094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:43.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.806+0000 7f6dadb81700 1 --2- 172.21.15.38:0/3290778883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da80f99e0 0x7f6da81094c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f6d94000c00 tx=0x7f6d9400b040).stop 2024-02-15T18:34:43.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.807+0000 7f6dadb81700 1 -- 172.21.15.38:0/3290778883 shutdown_connections 2024-02-15T18:34:43.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.807+0000 7f6dadb81700 1 --2- 172.21.15.38:0/3290778883 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6d9805c300 0x7f6d9805e7c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:43.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.807+0000 7f6dadb81700 1 --2- 172.21.15.38:0/3290778883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6da80f8740 0x7f6da8108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:43.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.807+0000 7f6dadb81700 1 --2- 172.21.15.38:0/3290778883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6da80f99e0 0x7f6da81094c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:43.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.807+0000 7f6dadb81700 1 -- 172.21.15.38:0/3290778883 >> 172.21.15.38:0/3290778883 conn(0x7f6da80f40e0 msgr2=0x7f6da80fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:43.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.807+0000 7f6dadb81700 1 -- 172.21.15.38:0/3290778883 shutdown_connections 2024-02-15T18:34:43.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.807+0000 7f6dadb81700 1 -- 172.21.15.38:0/3290778883 wait complete. 2024-02-15T18:34:43.810 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-02-15T18:34:43.845 INFO:tasks.workunit.client.1.smithi121.stdout:6/0: rename - no filename 2024-02-15T18:34:43.845 INFO:tasks.workunit.client.1.smithi121.stdout:7/1: symlink d0/l1 0 2024-02-15T18:34:43.845 INFO:tasks.workunit.client.1.smithi121.stdout:8/0: dwrite - no filename 2024-02-15T18:34:43.846 INFO:tasks.workunit.client.1.smithi121.stdout:8/1: dread - no filename 2024-02-15T18:34:43.846 INFO:tasks.workunit.client.1.smithi121.stdout:9/0: write - no filename 2024-02-15T18:34:43.847 INFO:tasks.workunit.client.1.smithi121.stdout:1/12: dwrite d0/f2 [0,4194304] 0 2024-02-15T18:34:43.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:42.314704+0000 mgr.smithi038.khesbl (mgr.14162) 327 : cluster [DBG] pgmap v273: 65 pgs: 65 active+clean; 163 MiB data, 691 MiB used, 536 GiB / 536 GiB avail; 8.0 MiB/s wr, 855 op/s 2024-02-15T18:34:43.871 INFO:tasks.workunit.client.1.smithi121.stdout:0/6: truncate f1 542130 0 2024-02-15T18:34:43.876 INFO:tasks.workunit.client.1.smithi121.stdout:3/15: rmdir d2 39 2024-02-15T18:34:43.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.916+0000 7fd7dcf8b700 1 -- 172.21.15.38:0/2790330408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7d8060310 msgr2=0x7fd7d80f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:43.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.916+0000 7fd7dcf8b700 1 --2- 172.21.15.38:0/2790330408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7d8060310 0x7fd7d80f9110 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fd7cc004660 tx=0x7fd7cc009b30).stop 2024-02-15T18:34:43.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.916+0000 7fd7dcf8b700 1 -- 172.21.15.38:0/2790330408 shutdown_connections 2024-02-15T18:34:43.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.916+0000 7fd7dcf8b700 1 --2- 172.21.15.38:0/2790330408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7d80f9690 0x7fd7d80fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:43.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.916+0000 7fd7dcf8b700 1 --2- 172.21.15.38:0/2790330408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7d8060310 0x7fd7d80f9110 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:43.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.916+0000 7fd7dcf8b700 1 -- 172.21.15.38:0/2790330408 >> 172.21.15.38:0/2790330408 conn(0x7fd7d80f0b90 msgr2=0x7fd7d80f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:43.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.916+0000 7fd7dcf8b700 1 -- 172.21.15.38:0/2790330408 shutdown_connections 2024-02-15T18:34:43.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.916+0000 7fd7dcf8b700 1 -- 172.21.15.38:0/2790330408 wait complete. 2024-02-15T18:34:43.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.917+0000 7fd7dcf8b700 1 Processor -- start 2024-02-15T18:34:43.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.917+0000 7fd7dcf8b700 1 -- start start 2024-02-15T18:34:43.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.917+0000 7fd7dcf8b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7d8060310 0x7fd7d80f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:43.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.917+0000 7fd7dcf8b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7d80f9690 0x7fd7d80f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:43.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.917+0000 7fd7dcf8b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d80f5570 con 0x7fd7d80f9690 2024-02-15T18:34:43.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.917+0000 7fd7dcf8b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d80f56b0 con 0x7fd7d8060310 2024-02-15T18:34:43.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.918+0000 7fd7d6ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7d80f9690 0x7fd7d80f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:43.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.918+0000 7fd7d6ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7d80f9690 0x7fd7d80f5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41558/0 (socket says 172.21.15.38:41558) 2024-02-15T18:34:43.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.918+0000 7fd7d6ffd700 1 -- 172.21.15.38:0/883443203 learned_addr learned my addr 172.21.15.38:0/883443203 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:34:43.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.918+0000 7fd7d77fe700 1 --2- 172.21.15.38:0/883443203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7d8060310 0x7fd7d80f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:43.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.918+0000 7fd7d6ffd700 1 -- 172.21.15.38:0/883443203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7d8060310 msgr2=0x7fd7d80f6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:43.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.918+0000 7fd7d6ffd700 1 --2- 172.21.15.38:0/883443203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7d8060310 0x7fd7d80f6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:43.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.918+0000 7fd7d6ffd700 1 -- 172.21.15.38:0/883443203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc005040 con 0x7fd7d80f9690 2024-02-15T18:34:43.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.918+0000 7fd7d77fe700 1 --2- 172.21.15.38:0/883443203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7d8060310 0x7fd7d80f6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:34:43.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.918+0000 7fd7d6ffd700 1 --2- 172.21.15.38:0/883443203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7d80f9690 0x7fd7d80f5010 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fd7c800c790 tx=0x7fd7c8007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:43.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.918+0000 7fd7bffff700 1 -- 172.21.15.38:0/883443203 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7c800e680 con 0x7fd7d80f9690 2024-02-15T18:34:43.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.918+0000 7fd7dcf8b700 1 -- 172.21.15.38:0/883443203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d80f5990 con 0x7fd7d80f9690 2024-02-15T18:34:43.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.918+0000 7fd7bffff700 1 -- 172.21.15.38:0/883443203 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fd7c800ecd0 con 0x7fd7d80f9690 2024-02-15T18:34:43.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.918+0000 7fd7bffff700 1 -- 172.21.15.38:0/883443203 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7c8013e10 con 0x7fd7d80f9690 2024-02-15T18:34:43.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.918+0000 7fd7dcf8b700 1 -- 172.21.15.38:0/883443203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d810edd0 con 0x7fd7d80f9690 2024-02-15T18:34:43.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.919+0000 7fd7dcf8b700 1 -- 172.21.15.38:0/883443203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d8045ad0 con 0x7fd7d80f9690 2024-02-15T18:34:43.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.920+0000 7fd7bffff700 1 -- 172.21.15.38:0/883443203 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fd7c801a060 con 0x7fd7d80f9690 2024-02-15T18:34:43.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.920+0000 7fd7bffff700 1 --2- 172.21.15.38:0/883443203 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd7c005c300 0x7fd7c005e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:43.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.920+0000 7fd7bffff700 1 -- 172.21.15.38:0/883443203 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fd7c8076120 con 0x7fd7d80f9690 2024-02-15T18:34:43.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.920+0000 7fd7d77fe700 1 --2- 172.21.15.38:0/883443203 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd7c005c300 0x7fd7c005e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:43.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.920+0000 7fd7d77fe700 1 --2- 172.21.15.38:0/883443203 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd7c005c300 0x7fd7c005e7c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fd7cc004e30 tx=0x7fd7cc009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:43.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:43.924+0000 7fd7bffff700 1 -- 172.21.15.38:0/883443203 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fd7c8048de0 con 0x7fd7d80f9690 2024-02-15T18:34:43.968 INFO:tasks.workunit.client.1.smithi121.stdout:0/7: write f1 [136517,86918] 0 2024-02-15T18:34:43.968 INFO:tasks.workunit.client.1.smithi121.stdout:0/8: write f1 [413589,87971] 0 2024-02-15T18:34:43.968 INFO:tasks.workunit.client.1.smithi121.stdout:0/9: rmdir - no directory 2024-02-15T18:34:44.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.116+0000 7fd7dcf8b700 1 -- 172.21.15.38:0/883443203 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7d80fef20 con 0x7fd7c005c300 2024-02-15T18:34:44.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.117+0000 7fd7bffff700 1 -- 172.21.15.38:0/883443203 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+248 (secure 0 0 0) 0x7fd7d80fef20 con 0x7fd7c005c300 2024-02-15T18:34:44.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.118+0000 7fd7dcf8b700 1 -- 172.21.15.38:0/883443203 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd7c005c300 msgr2=0x7fd7c005e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:44.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.118+0000 7fd7dcf8b700 1 --2- 172.21.15.38:0/883443203 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd7c005c300 0x7fd7c005e7c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fd7cc004e30 tx=0x7fd7cc009b30).stop 2024-02-15T18:34:44.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.118+0000 7fd7dcf8b700 1 -- 172.21.15.38:0/883443203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7d80f9690 msgr2=0x7fd7d80f5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:44.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.118+0000 7fd7dcf8b700 1 --2- 172.21.15.38:0/883443203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7d80f9690 0x7fd7d80f5010 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fd7c800c790 tx=0x7fd7c8007d40).stop 2024-02-15T18:34:44.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.118+0000 7fd7dcf8b700 1 -- 172.21.15.38:0/883443203 shutdown_connections 2024-02-15T18:34:44.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.118+0000 7fd7dcf8b700 1 --2- 172.21.15.38:0/883443203 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fd7c005c300 0x7fd7c005e7c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:44.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.118+0000 7fd7dcf8b700 1 --2- 172.21.15.38:0/883443203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7d8060310 0x7fd7d80f6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:44.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.118+0000 7fd7dcf8b700 1 --2- 172.21.15.38:0/883443203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7d80f9690 0x7fd7d80f5010 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:44.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.118+0000 7fd7dcf8b700 1 -- 172.21.15.38:0/883443203 >> 172.21.15.38:0/883443203 conn(0x7fd7d80f0b90 msgr2=0x7fd7d80f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:44.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.119+0000 7fd7dcf8b700 1 -- 172.21.15.38:0/883443203 shutdown_connections 2024-02-15T18:34:44.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.119+0000 7fd7dcf8b700 1 -- 172.21.15.38:0/883443203 wait complete. 2024-02-15T18:34:44.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.224+0000 7fb053fff700 1 -- 172.21.15.38:0/4147006639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0540f5430 msgr2=0x7fb0540f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:44.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.224+0000 7fb053fff700 1 --2- 172.21.15.38:0/4147006639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0540f5430 0x7fb0540f5850 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fb044004660 tx=0x7fb044009b30).stop 2024-02-15T18:34:44.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.224+0000 7fb053fff700 1 -- 172.21.15.38:0/4147006639 shutdown_connections 2024-02-15T18:34:44.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.224+0000 7fb053fff700 1 --2- 172.21.15.38:0/4147006639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0540f66d0 0x7fb0540f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:34:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.224+0000 7fb053fff700 1 --2- 172.21.15.38:0/4147006639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0540f5430 0x7fb0540f5850 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.224+0000 7fb053fff700 1 -- 172.21.15.38:0/4147006639 >> 172.21.15.38:0/4147006639 conn(0x7fb0540f0b90 msgr2=0x7fb0540f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.224+0000 7fb053fff700 1 -- 172.21.15.38:0/4147006639 shutdown_connections 2024-02-15T18:34:44.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.224+0000 7fb053fff700 1 -- 172.21.15.38:0/4147006639 wait complete. 2024-02-15T18:34:44.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.225+0000 7fb053fff700 1 Processor -- start 2024-02-15T18:34:44.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.225+0000 7fb053fff700 1 -- start start 2024-02-15T18:34:44.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.225+0000 7fb053fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0540f5430 0x7fb054106ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:44.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.225+0000 7fb053fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0540f66d0 0x7fb054107390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:44.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.225+0000 7fb053fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0541079d0 con 0x7fb0540f5430 2024-02-15T18:34:44.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.225+0000 7fb053fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb054108280 con 0x7fb0540f66d0 2024-02-15T18:34:44.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.225+0000 7fb052ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0540f5430 0x7fb054106ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:44.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.225+0000 7fb052ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0540f5430 0x7fb054106ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41574/0 (socket says 172.21.15.38:41574) 2024-02-15T18:34:44.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.225+0000 7fb052ffd700 1 -- 172.21.15.38:0/1158079907 learned_addr learned my addr 172.21.15.38:0/1158079907 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:34:44.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.225+0000 7fb0527fc700 1 --2- 172.21.15.38:0/1158079907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0540f66d0 0x7fb054107390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:44.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.226+0000 7fb052ffd700 1 -- 172.21.15.38:0/1158079907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0540f66d0 msgr2=0x7fb054107390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:44.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.226+0000 7fb052ffd700 1 --2- 172.21.15.38:0/1158079907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0540f66d0 0x7fb054107390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:44.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.226+0000 7fb052ffd700 1 -- 172.21.15.38:0/1158079907 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb044005040 con 0x7fb0540f5430 2024-02-15T18:34:44.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.226+0000 7fb052ffd700 1 --2- 172.21.15.38:0/1158079907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0540f5430 0x7fb054106ea0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fb044013fd0 tx=0x7fb044009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:44.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.226+0000 7fb03b7fe700 1 -- 172.21.15.38:0/1158079907 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb044018070 con 0x7fb0540f5430 2024-02-15T18:34:44.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.226+0000 7fb053fff700 1 -- 172.21.15.38:0/1158079907 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb054108500 con 0x7fb0540f5430 2024-02-15T18:34:44.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.226+0000 7fb03b7fe700 1 -- 172.21.15.38:0/1158079907 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fb044007c70 con 0x7fb0540f5430 2024-02-15T18:34:44.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.226+0000 7fb03b7fe700 1 -- 172.21.15.38:0/1158079907 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb04401c620 con 0x7fb0540f5430 2024-02-15T18:34:44.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.226+0000 7fb053fff700 1 -- 172.21.15.38:0/1158079907 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0541089e0 con 0x7fb0540f5430 2024-02-15T18:34:44.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.227+0000 7fb053fff700 1 -- 172.21.15.38:0/1158079907 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb054059100 con 0x7fb0540f5430 2024-02-15T18:34:44.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.227+0000 7fb03b7fe700 1 -- 172.21.15.38:0/1158079907 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fb04401d420 con 0x7fb0540f5430 2024-02-15T18:34:44.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.228+0000 7fb03b7fe700 1 --2- 172.21.15.38:0/1158079907 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb03c05c300 0x7fb03c05e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:44.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.228+0000 7fb03b7fe700 1 -- 172.21.15.38:0/1158079907 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fb044077cd0 con 0x7fb0540f5430 2024-02-15T18:34:44.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.228+0000 7fb0527fc700 1 --2- 172.21.15.38:0/1158079907 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb03c05c300 0x7fb03c05e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:44.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.228+0000 7fb0527fc700 1 --2- 172.21.15.38:0/1158079907 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb03c05c300 0x7fb03c05e7c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fb048006fd0 tx=0x7fb048008040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:44.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.232+0000 7fb03b7fe700 1 -- 172.21.15.38:0/1158079907 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb04404a990 con 0x7fb0540f5430 2024-02-15T18:34:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.424+0000 7fb053fff700 1 -- 172.21.15.38:0/1158079907 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0540fab10 con 0x7fb03c05c300 2024-02-15T18:34:44.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.432+0000 7fb03b7fe700 1 -- 172.21.15.38:0/1158079907 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2772 (secure 0 0 0) 0x7fb0540fab10 con 0x7fb03c05c300 2024-02-15T18:34:44.424 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE VERSION IMAGE ID CONTAINER ID 2024-02-15T18:34:44.424 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (5m) 85s ago 7m 0.20.0 0881eb8f169f c2034c318e7c 2024-02-15T18:34:44.424 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (7m) 85s ago 7m 16.2.4 8d91d370c2b8 0fd3f945d009 2024-02-15T18:34:44.424 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi121 smithi121 running (6m) 90s ago 6m 16.2.4 8d91d370c2b8 affc02db354e 2024-02-15T18:34:44.424 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (5m) 85s ago 7m 6.7.4 557c83e11646 2dcccfbfbd2a 2024-02-15T18:34:44.424 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.hxoggl smithi038 running (110s) 85s ago 109s 16.2.4 8d91d370c2b8 be2533754b9c 2024-02-15T18:34:44.424 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.qgkzgn smithi038 running (104s) 85s ago 103s 16.2.4 8d91d370c2b8 dd3ffafba9bd 2024-02-15T18:34:44.425 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.mvumlh smithi121 running (115s) 90s ago 115s 16.2.4 8d91d370c2b8 88b7d876a39b 2024-02-15T18:34:44.425 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.ovuzki smithi121 running (107s) 90s ago 107s 16.2.4 8d91d370c2b8 21cacf0012f9 2024-02-15T18:34:44.425 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.khesbl smithi038 *:9283 running (9m) 85s ago 9m 16.2.4 8d91d370c2b8 b0d099c255f4 2024-02-15T18:34:44.425 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi121.ckiryi smithi121 *:8443,9283 starting - - 2024-02-15T18:34:44.425 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (9m) 85s ago 9m 16.2.4 8d91d370c2b8 01b30eaebcfe 2024-02-15T18:34:44.425 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi121 smithi121 running (6m) 90s ago 6m 16.2.4 8d91d370c2b8 8f588ecacb71 2024-02-15T18:34:44.425 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (7m) 85s ago 7m 0.18.1 e5a616e4b9cf 037f60e38158 2024-02-15T18:34:44.425 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi121 smithi121 *:9100 running (6m) 90s ago 6m 0.18.1 e5a616e4b9cf a867bff85e18 2024-02-15T18:34:44.425 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (5m) 85s ago 5m 16.2.4 8d91d370c2b8 51512003a2ef 2024-02-15T18:34:44.425 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (5m) 85s ago 5m 16.2.4 8d91d370c2b8 16dbdf20b170 2024-02-15T18:34:44.425 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (4m) 85s ago 4m 16.2.4 8d91d370c2b8 6d6ee813e5be 2024-02-15T18:34:44.425 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi121 running (4m) 90s ago 4m 16.2.4 8d91d370c2b8 cba2d9ed4730 2024-02-15T18:34:44.425 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi121 running (3m) 90s ago 3m 16.2.4 8d91d370c2b8 ac17ad7dc628 2024-02-15T18:34:44.425 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi121 running (3m) 90s ago 3m 16.2.4 8d91d370c2b8 cdc8c669c4db 2024-02-15T18:34:44.425 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (5m) 85s ago 6m 2.18.1 de242295e225 574a5d75991e 2024-02-15T18:34:44.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.434+0000 7fb053fff700 1 -- 172.21.15.38:0/1158079907 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb03c05c300 msgr2=0x7fb03c05e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:44.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.434+0000 7fb053fff700 1 --2- 172.21.15.38:0/1158079907 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb03c05c300 0x7fb03c05e7c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fb048006fd0 tx=0x7fb048008040).stop 2024-02-15T18:34:44.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.434+0000 7fb053fff700 1 -- 172.21.15.38:0/1158079907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0540f5430 msgr2=0x7fb054106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:44.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.434+0000 7fb053fff700 1 --2- 172.21.15.38:0/1158079907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0540f5430 0x7fb054106ea0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fb044013fd0 tx=0x7fb044009b30).stop 2024-02-15T18:34:44.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.434+0000 7fb053fff700 1 -- 172.21.15.38:0/1158079907 shutdown_connections 2024-02-15T18:34:44.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.434+0000 7fb053fff700 1 --2- 172.21.15.38:0/1158079907 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0540f5430 0x7fb054106ea0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:44.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.434+0000 7fb053fff700 1 --2- 172.21.15.38:0/1158079907 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb03c05c300 0x7fb03c05e7c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:44.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.434+0000 7fb053fff700 1 --2- 172.21.15.38:0/1158079907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0540f66d0 0x7fb054107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:44.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.434+0000 7fb053fff700 1 -- 172.21.15.38:0/1158079907 >> 172.21.15.38:0/1158079907 conn(0x7fb0540f0b90 msgr2=0x7fb05405fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:44.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.434+0000 7fb053fff700 1 -- 172.21.15.38:0/1158079907 shutdown_connections 2024-02-15T18:34:44.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.434+0000 7fb053fff700 1 -- 172.21.15.38:0/1158079907 wait complete. 2024-02-15T18:34:44.501 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:43.804980+0000 mgr.smithi038.khesbl (mgr.14162) 328 : audit [DBG] from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:44.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.535+0000 7f61fda1e700 1 -- 172.21.15.38:0/2390454130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61f8060270 msgr2=0x7f61f80f9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:44.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.535+0000 7f61fda1e700 1 --2- 172.21.15.38:0/2390454130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61f8060270 0x7f61f80f9340 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f61e0004660 tx=0x7f61e0009b30).stop 2024-02-15T18:34:44.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.535+0000 7f61fda1e700 1 -- 172.21.15.38:0/2390454130 shutdown_connections 2024-02-15T18:34:44.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.535+0000 7f61fda1e700 1 --2- 172.21.15.38:0/2390454130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61f80f9830 0x7f61f80fbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:44.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.535+0000 7f61fda1e700 1 --2- 172.21.15.38:0/2390454130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61f8060270 0x7f61f80f9340 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:44.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.535+0000 7f61fda1e700 1 -- 172.21.15.38:0/2390454130 >> 172.21.15.38:0/2390454130 conn(0x7f61f80f0d40 msgr2=0x7f61f80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:44.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.535+0000 7f61fda1e700 1 -- 172.21.15.38:0/2390454130 shutdown_connections 2024-02-15T18:34:44.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.535+0000 7f61fda1e700 1 -- 172.21.15.38:0/2390454130 wait complete. 2024-02-15T18:34:44.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.536+0000 7f61fda1e700 1 Processor -- start 2024-02-15T18:34:44.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.536+0000 7f61fda1e700 1 -- start start 2024-02-15T18:34:44.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.536+0000 7f61fda1e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61f8060270 0x7f61f8106fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:44.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.536+0000 7f61fda1e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61f80f9830 0x7f61f81074c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:44.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.536+0000 7f61fda1e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f8107a70 con 0x7f61f80f9830 2024-02-15T18:34:44.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.536+0000 7f61fda1e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f8107bb0 con 0x7f61f8060270 2024-02-15T18:34:44.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.536+0000 7f61f67fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61f80f9830 0x7f61f81074c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:44.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.536+0000 7f61f67fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61f80f9830 0x7f61f81074c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41596/0 (socket says 172.21.15.38:41596) 2024-02-15T18:34:44.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.536+0000 7f61f67fc700 1 -- 172.21.15.38:0/1721982501 learned_addr learned my addr 172.21.15.38:0/1721982501 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:34:44.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.536+0000 7f61f6ffd700 1 --2- 172.21.15.38:0/1721982501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61f8060270 0x7f61f8106fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:44.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.536+0000 7f61f67fc700 1 -- 172.21.15.38:0/1721982501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61f8060270 msgr2=0x7f61f8106fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:44.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.536+0000 7f61f67fc700 1 --2- 172.21.15.38:0/1721982501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61f8060270 0x7f61f8106fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:44.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.536+0000 7f61f67fc700 1 -- 172.21.15.38:0/1721982501 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e0005040 con 0x7f61f80f9830 2024-02-15T18:34:44.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.536+0000 7f61f6ffd700 1 --2- 172.21.15.38:0/1721982501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61f8060270 0x7f61f8106fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:34:44.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.536+0000 7f61f67fc700 1 --2- 172.21.15.38:0/1721982501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61f80f9830 0x7f61f81074c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f61e800e7c0 tx=0x7f61e800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:44.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.537+0000 7f61ef7fe700 1 -- 172.21.15.38:0/1721982501 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61e80092e0 con 0x7f61f80f9830 2024-02-15T18:34:44.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.537+0000 7f61fda1e700 1 -- 172.21.15.38:0/1721982501 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f8108550 con 0x7f61f80f9830 2024-02-15T18:34:44.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.537+0000 7f61ef7fe700 1 -- 172.21.15.38:0/1721982501 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f61e800f040 con 0x7f61f80f9830 2024-02-15T18:34:44.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.537+0000 7f61ef7fe700 1 -- 172.21.15.38:0/1721982501 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61e8014920 con 0x7f61f80f9830 2024-02-15T18:34:44.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.537+0000 7f61fda1e700 1 -- 172.21.15.38:0/1721982501 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f8108b10 con 0x7f61f80f9830 2024-02-15T18:34:44.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.538+0000 7f61fda1e700 1 -- 172.21.15.38:0/1721982501 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f8045ad0 con 0x7f61f80f9830 2024-02-15T18:34:44.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.538+0000 7f61ef7fe700 1 -- 172.21.15.38:0/1721982501 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f61e8009440 con 0x7f61f80f9830 2024-02-15T18:34:44.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.538+0000 7f61ef7fe700 1 --2- 172.21.15.38:0/1721982501 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f61e405c2b0 0x7f61e405e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:44.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.539+0000 7f61ef7fe700 1 -- 172.21.15.38:0/1721982501 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f61e8076170 con 0x7f61f80f9830 2024-02-15T18:34:44.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.539+0000 7f61f6ffd700 1 --2- 172.21.15.38:0/1721982501 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f61e405c2b0 0x7f61e405e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:44.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.539+0000 7f61f6ffd700 1 --2- 172.21.15.38:0/1721982501 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f61e405c2b0 0x7f61e405e770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f61e0009880 tx=0x7f61e0016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:44.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.543+0000 7f61ef7fe700 1 -- 172.21.15.38:0/1721982501 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f61e8048eb0 con 0x7f61f80f9830 2024-02-15T18:34:44.752 INFO:tasks.workunit.client.1.smithi121.stdout:6/1: creat f0 x:0 0 0 2024-02-15T18:34:44.753 INFO:tasks.workunit.client.1.smithi121.stdout:7/2: getdents d0 0 2024-02-15T18:34:44.754 INFO:tasks.workunit.client.1.smithi121.stdout:8/2: mkdir d0 0 2024-02-15T18:34:44.754 INFO:tasks.workunit.client.1.smithi121.stdout:8/3: dread - no filename 2024-02-15T18:34:44.754 INFO:tasks.workunit.client.1.smithi121.stdout:8/4: write - no filename 2024-02-15T18:34:44.754 INFO:tasks.workunit.client.1.smithi121.stdout:9/1: creat f0 x:0 0 0 2024-02-15T18:34:44.754 INFO:tasks.workunit.client.1.smithi121.stdout:9/2: dread - f0 zero size 2024-02-15T18:34:44.756 INFO:tasks.workunit.client.1.smithi121.stdout:1/13: symlink d0/l4 0 2024-02-15T18:34:44.757 INFO:tasks.workunit.client.1.smithi121.stdout:3/16: rename d2/f5 to d2/f6 0 2024-02-15T18:34:44.759 INFO:tasks.workunit.client.1.smithi121.stdout:0/10: creat f2 x:0 0 0 2024-02-15T18:34:44.763 INFO:tasks.workunit.client.1.smithi121.stdout:1/14: dread d0/f2 [0,4194304] 0 2024-02-15T18:34:44.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.776+0000 7f61fda1e700 1 -- 172.21.15.38:0/1721982501 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f61f8108d30 con 0x7f61f80f9830 2024-02-15T18:34:44.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.777+0000 7f61ef7fe700 1 -- 172.21.15.38:0/1721982501 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+573 (secure 0 0 0) 0x7f61e804c4d0 con 0x7f61f80f9830 2024-02-15T18:34:44.768 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:34:44.769 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-02-15T18:34:44.769 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2024-02-15T18:34:44.769 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:34:44.769 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-02-15T18:34:44.769 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2024-02-15T18:34:44.769 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:34:44.769 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-02-15T18:34:44.769 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2024-02-15T18:34:44.769 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:34:44.769 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-02-15T18:34:44.769 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:34:44.769 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:34:44.769 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-02-15T18:34:44.769 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 14 2024-02-15T18:34:44.769 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-02-15T18:34:44.769 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:34:44.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.778+0000 7f61fda1e700 1 -- 172.21.15.38:0/1721982501 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f61e405c2b0 msgr2=0x7f61e405e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:44.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.778+0000 7f61fda1e700 1 --2- 172.21.15.38:0/1721982501 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f61e405c2b0 0x7f61e405e770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f61e0009880 tx=0x7f61e0016040).stop 2024-02-15T18:34:44.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.778+0000 7f61fda1e700 1 -- 172.21.15.38:0/1721982501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61f80f9830 msgr2=0x7f61f81074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:44.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.778+0000 7f61fda1e700 1 --2- 172.21.15.38:0/1721982501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61f80f9830 0x7f61f81074c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f61e800e7c0 tx=0x7f61e800b040).stop 2024-02-15T18:34:44.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.778+0000 7f61fda1e700 1 -- 172.21.15.38:0/1721982501 shutdown_connections 2024-02-15T18:34:44.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.778+0000 7f61fda1e700 1 --2- 172.21.15.38:0/1721982501 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f61e405c2b0 0x7f61e405e770 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:44.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.778+0000 7f61fda1e700 1 --2- 172.21.15.38:0/1721982501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61f8060270 0x7f61f8106fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:44.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.778+0000 7f61fda1e700 1 --2- 172.21.15.38:0/1721982501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61f80f9830 0x7f61f81074c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:44.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.778+0000 7f61fda1e700 1 -- 172.21.15.38:0/1721982501 >> 172.21.15.38:0/1721982501 conn(0x7f61f80f0d40 msgr2=0x7f61f80f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:44.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.778+0000 7f61fda1e700 1 -- 172.21.15.38:0/1721982501 shutdown_connections 2024-02-15T18:34:44.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.778+0000 7f61fda1e700 1 -- 172.21.15.38:0/1721982501 wait complete. 2024-02-15T18:34:44.771 INFO:tasks.workunit.client.1.smithi121.stdout:6/2: mknod c1 0 2024-02-15T18:34:44.772 INFO:tasks.workunit.client.1.smithi121.stdout:7/3: mknod d0/c2 0 2024-02-15T18:34:44.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:43.804980+0000 mgr.smithi038.khesbl (mgr.14162) 328 : audit [DBG] from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:44.773 INFO:tasks.workunit.client.1.smithi121.stdout:8/5: creat d0/f1 x:0 0 0 2024-02-15T18:34:44.774 INFO:tasks.workunit.client.1.smithi121.stdout:9/3: rename f0 to f1 0 2024-02-15T18:34:44.776 INFO:tasks.workunit.client.1.smithi121.stdout:3/17: creat d2/f7 x:0 0 0 2024-02-15T18:34:44.778 INFO:tasks.workunit.client.1.smithi121.stdout:0/11: creat f3 x:0 0 0 2024-02-15T18:34:44.781 INFO:tasks.workunit.client.1.smithi121.stdout:1/15: unlink d0/l1 0 2024-02-15T18:34:44.781 INFO:tasks.workunit.client.1.smithi121.stdout:7/4: readlink d0/l1 0 2024-02-15T18:34:44.781 INFO:tasks.workunit.client.1.smithi121.stdout:7/5: dread - no filename 2024-02-15T18:34:44.781 INFO:tasks.workunit.client.1.smithi121.stdout:7/6: read - no filename 2024-02-15T18:34:44.781 INFO:tasks.workunit.client.1.smithi121.stdout:8/6: creat d0/f2 x:0 0 0 2024-02-15T18:34:44.782 INFO:tasks.workunit.client.1.smithi121.stdout:9/4: symlink l2 0 2024-02-15T18:34:44.784 INFO:tasks.workunit.client.1.smithi121.stdout:0/12: creat f4 x:0 0 0 2024-02-15T18:34:44.784 INFO:tasks.workunit.client.1.smithi121.stdout:0/13: chown f2 44953 1 2024-02-15T18:34:44.786 INFO:tasks.workunit.client.1.smithi121.stdout:7/7: creat d0/f3 x:0 0 0 2024-02-15T18:34:44.786 INFO:tasks.workunit.client.1.smithi121.stdout:8/7: mkdir d0/d3 0 2024-02-15T18:34:44.789 INFO:tasks.workunit.client.1.smithi121.stdout:9/5: creat f3 x:0 0 0 2024-02-15T18:34:44.789 INFO:tasks.workunit.client.1.smithi121.stdout:9/6: dread - f1 zero size 2024-02-15T18:34:44.792 INFO:tasks.workunit.client.1.smithi121.stdout:8/8: symlink d0/l4 0 2024-02-15T18:34:44.792 INFO:tasks.workunit.client.1.smithi121.stdout:8/9: write d0/f1 [425802,90209] 0 2024-02-15T18:34:44.793 INFO:tasks.workunit.client.1.smithi121.stdout:9/7: mknod c4 0 2024-02-15T18:34:44.795 INFO:tasks.workunit.client.1.smithi121.stdout:9/8: rename f3 to f5 0 2024-02-15T18:34:44.795 INFO:tasks.workunit.client.1.smithi121.stdout:9/9: dread - f1 zero size 2024-02-15T18:34:44.795 INFO:tasks.workunit.client.1.smithi121.stdout:6/3: dwrite f0 [0,4194304] 0 2024-02-15T18:34:44.796 INFO:tasks.workunit.client.1.smithi121.stdout:9/10: mknod c6 0 2024-02-15T18:34:44.798 INFO:tasks.workunit.client.1.smithi121.stdout:6/4: creat f2 x:0 0 0 2024-02-15T18:34:44.799 INFO:tasks.workunit.client.1.smithi121.stdout:6/5: creat f3 x:0 0 0 2024-02-15T18:34:44.800 INFO:tasks.workunit.client.1.smithi121.stdout:6/6: mknod c4 0 2024-02-15T18:34:44.801 INFO:tasks.workunit.client.1.smithi121.stdout:6/7: creat f5 x:0 0 0 2024-02-15T18:34:44.801 INFO:tasks.workunit.client.1.smithi121.stdout:6/8: symlink l6 0 2024-02-15T18:34:44.801 INFO:tasks.workunit.client.1.smithi121.stdout:6/9: chown f5 73261 1 2024-02-15T18:34:44.802 INFO:tasks.workunit.client.1.smithi121.stdout:6/10: mkdir d7 0 2024-02-15T18:34:44.804 INFO:tasks.workunit.client.1.smithi121.stdout:6/11: rename f3 to d7/f8 0 2024-02-15T18:34:44.805 INFO:tasks.workunit.client.1.smithi121.stdout:6/12: creat d7/f9 x:0 0 0 2024-02-15T18:34:44.805 INFO:tasks.workunit.client.1.smithi121.stdout:6/13: truncate d7/f9 71980 0 2024-02-15T18:34:44.806 INFO:tasks.workunit.client.1.smithi121.stdout:6/14: mknod d7/ca 0 2024-02-15T18:34:44.806 INFO:tasks.workunit.client.1.smithi121.stdout:6/15: chown d7/f8 1 1 2024-02-15T18:34:44.807 INFO:tasks.workunit.client.1.smithi121.stdout:6/16: read - d7/f8 zero size 2024-02-15T18:34:44.808 INFO:tasks.workunit.client.1.smithi121.stdout:6/17: rename c1 to d7/cb 0 2024-02-15T18:34:44.810 INFO:tasks.workunit.client.1.smithi121.stdout:6/18: mkdir d7/dc 0 2024-02-15T18:34:44.811 INFO:tasks.workunit.client.1.smithi121.stdout:6/19: write d7/f8 [908366,117364] 0 2024-02-15T18:34:44.811 INFO:tasks.workunit.client.1.smithi121.stdout:1/16: dwrite d0/f2 [0,4194304] 0 2024-02-15T18:34:44.812 INFO:tasks.workunit.client.1.smithi121.stdout:3/18: unlink d2/f6 0 2024-02-15T18:34:44.813 INFO:tasks.workunit.client.1.smithi121.stdout:3/19: dread d2/f4 [0,4194304] 0 2024-02-15T18:34:44.814 INFO:tasks.workunit.client.1.smithi121.stdout:3/20: mknod d2/c8 0 2024-02-15T18:34:44.818 INFO:tasks.workunit.client.1.smithi121.stdout:0/14: dwrite f2 [0,4194304] 0 2024-02-15T18:34:44.818 INFO:tasks.workunit.client.1.smithi121.stdout:1/17: write d0/f2 [3214031,38437] 0 2024-02-15T18:34:44.820 INFO:tasks.workunit.client.1.smithi121.stdout:9/11: dwrite f1 [0,4194304] 0 2024-02-15T18:34:44.820 INFO:tasks.workunit.client.1.smithi121.stdout:9/12: write f5 [373077,84230] 0 2024-02-15T18:34:44.822 INFO:tasks.workunit.client.1.smithi121.stdout:6/20: dread f0 [0,4194304] 0 2024-02-15T18:34:44.825 INFO:tasks.workunit.client.1.smithi121.stdout:8/10: truncate d0/f1 432031 0 2024-02-15T18:34:44.826 INFO:tasks.workunit.client.1.smithi121.stdout:6/21: dread f0 [0,4194304] 0 2024-02-15T18:34:44.826 INFO:tasks.workunit.client.1.smithi121.stdout:6/22: dread - f2 zero size 2024-02-15T18:34:44.826 INFO:tasks.workunit.client.1.smithi121.stdout:6/23: chown d7/f9 20940 1 2024-02-15T18:34:44.828 INFO:tasks.workunit.client.1.smithi121.stdout:7/8: rmdir d0 39 2024-02-15T18:34:44.828 INFO:tasks.workunit.client.1.smithi121.stdout:7/9: readlink d0/l1 0 2024-02-15T18:34:44.833 INFO:tasks.workunit.client.1.smithi121.stdout:3/21: rmdir d2 39 2024-02-15T18:34:44.833 INFO:tasks.workunit.client.1.smithi121.stdout:3/22: chown l0 7 1 2024-02-15T18:34:44.835 INFO:tasks.workunit.client.1.smithi121.stdout:0/15: mkdir d5 0 2024-02-15T18:34:44.835 INFO:tasks.workunit.client.1.smithi121.stdout:0/16: fsync f3 0 2024-02-15T18:34:44.838 INFO:tasks.workunit.client.1.smithi121.stdout:1/18: rename d0/c3 to d0/c5 0 2024-02-15T18:34:44.838 INFO:tasks.workunit.client.1.smithi121.stdout:1/19: truncate d0/f2 5074434 0 2024-02-15T18:34:44.839 INFO:tasks.workunit.client.1.smithi121.stdout:9/13: mkdir d7 0 2024-02-15T18:34:44.839 INFO:tasks.workunit.client.1.smithi121.stdout:8/11: creat d0/f5 x:0 0 0 2024-02-15T18:34:44.841 INFO:tasks.workunit.client.1.smithi121.stdout:8/12: write d0/f1 [178766,66026] 0 2024-02-15T18:34:44.841 INFO:tasks.workunit.client.1.smithi121.stdout:8/13: chown d0/f5 15777621 1 2024-02-15T18:34:44.841 INFO:tasks.workunit.client.1.smithi121.stdout:6/24: creat d7/dc/fd x:0 0 0 2024-02-15T18:34:44.841 INFO:tasks.workunit.client.1.smithi121.stdout:6/25: truncate d7/f9 324131 0 2024-02-15T18:34:44.842 INFO:tasks.workunit.client.1.smithi121.stdout:0/17: mkdir d5/d6 0 2024-02-15T18:34:44.844 INFO:tasks.workunit.client.1.smithi121.stdout:1/20: mkdir d0/d6 0 2024-02-15T18:34:44.844 INFO:tasks.workunit.client.1.smithi121.stdout:1/21: readlink d0/l4 0 2024-02-15T18:34:44.847 INFO:tasks.workunit.client.1.smithi121.stdout:8/14: link d0/f2 d0/d3/f6 0 2024-02-15T18:34:44.847 INFO:tasks.workunit.client.1.smithi121.stdout:8/15: dread - d0/f2 zero size 2024-02-15T18:34:44.847 INFO:tasks.workunit.client.1.smithi121.stdout:8/16: dread - d0/f2 zero size 2024-02-15T18:34:44.848 INFO:tasks.workunit.client.1.smithi121.stdout:7/10: rename d0/l1 to d0/l4 0 2024-02-15T18:34:44.848 INFO:tasks.workunit.client.1.smithi121.stdout:7/11: fdatasync d0/f3 0 2024-02-15T18:34:44.850 INFO:tasks.workunit.client.1.smithi121.stdout:9/14: dread f1 [0,4194304] 0 2024-02-15T18:34:44.850 INFO:tasks.workunit.client.1.smithi121.stdout:9/15: fsync f5 0 2024-02-15T18:34:44.850 INFO:tasks.workunit.client.1.smithi121.stdout:9/16: readlink l2 0 2024-02-15T18:34:44.856 INFO:tasks.workunit.client.1.smithi121.stdout:0/18: mknod d5/c7 0 2024-02-15T18:34:44.858 INFO:tasks.workunit.client.1.smithi121.stdout:1/22: unlink d0/f2 0 2024-02-15T18:34:44.858 INFO:tasks.workunit.client.1.smithi121.stdout:1/23: dwrite - no filename 2024-02-15T18:34:44.858 INFO:tasks.workunit.client.1.smithi121.stdout:8/17: mkdir d0/d7 0 2024-02-15T18:34:44.859 INFO:tasks.workunit.client.1.smithi121.stdout:0/19: unlink f1 0 2024-02-15T18:34:44.860 INFO:tasks.workunit.client.1.smithi121.stdout:1/24: mkdir d0/d6/d7 0 2024-02-15T18:34:44.861 INFO:tasks.workunit.client.1.smithi121.stdout:0/20: mknod d5/d6/c8 0 2024-02-15T18:34:44.861 INFO:tasks.workunit.client.1.smithi121.stdout:0/21: fdatasync f3 0 2024-02-15T18:34:44.862 INFO:tasks.workunit.client.1.smithi121.stdout:1/25: symlink d0/l8 0 2024-02-15T18:34:44.863 INFO:tasks.workunit.client.1.smithi121.stdout:1/26: mkdir d0/d6/d7/d9 0 2024-02-15T18:34:44.864 INFO:tasks.workunit.client.1.smithi121.stdout:7/12: dwrite d0/f3 [0,4194304] 0 2024-02-15T18:34:44.865 INFO:tasks.workunit.client.1.smithi121.stdout:6/26: dwrite d7/f9 [0,4194304] 0 2024-02-15T18:34:44.869 INFO:tasks.workunit.client.1.smithi121.stdout:6/27: symlink d7/dc/le 0 2024-02-15T18:34:44.873 INFO:tasks.workunit.client.1.smithi121.stdout:7/13: dread d0/f3 [0,4194304] 0 2024-02-15T18:34:44.877 INFO:tasks.workunit.client.1.smithi121.stdout:6/28: dread d7/f8 [0,4194304] 0 2024-02-15T18:34:44.877 INFO:tasks.workunit.client.1.smithi121.stdout:6/29: fsync f0 0 2024-02-15T18:34:44.877 INFO:tasks.workunit.client.1.smithi121.stdout:6/30: chown d7/dc 3 1 2024-02-15T18:34:44.877 INFO:tasks.workunit.client.1.smithi121.stdout:6/31: write f5 [302474,8725] 0 2024-02-15T18:34:44.877 INFO:tasks.workunit.client.1.smithi121.stdout:6/32: write d7/dc/fd [164551,8050] 0 2024-02-15T18:34:44.877 INFO:tasks.workunit.client.1.smithi121.stdout:8/18: dwrite d0/f1 [0,4194304] 0 2024-02-15T18:34:44.883 INFO:tasks.workunit.client.1.smithi121.stdout:8/19: rmdir d0 39 2024-02-15T18:34:44.883 INFO:tasks.workunit.client.1.smithi121.stdout:8/20: readlink d0/l4 0 2024-02-15T18:34:44.884 INFO:tasks.workunit.client.1.smithi121.stdout:8/21: getdents d0/d3 0 2024-02-15T18:34:44.884 INFO:tasks.workunit.client.1.smithi121.stdout:8/22: getdents d0/d7 0 2024-02-15T18:34:44.885 INFO:tasks.workunit.client.1.smithi121.stdout:1/27: rmdir d0 39 2024-02-15T18:34:44.885 INFO:tasks.workunit.client.1.smithi121.stdout:1/28: write - no filename 2024-02-15T18:34:44.889 INFO:tasks.workunit.client.1.smithi121.stdout:7/14: dwrite d0/f3 [0,4194304] 0 2024-02-15T18:34:44.889 INFO:tasks.workunit.client.1.smithi121.stdout:7/15: chown d0/c2 256 1 2024-02-15T18:34:44.890 INFO:tasks.workunit.client.1.smithi121.stdout:6/33: dread f5 [0,4194304] 0 2024-02-15T18:34:44.892 INFO:tasks.workunit.client.1.smithi121.stdout:7/16: write d0/f3 [2016310,80428] 0 2024-02-15T18:34:44.892 INFO:tasks.workunit.client.1.smithi121.stdout:7/17: chown d0 4 1 2024-02-15T18:34:44.892 INFO:tasks.workunit.client.1.smithi121.stdout:7/18: write d0/f3 [5220809,110923] 0 2024-02-15T18:34:44.892 INFO:tasks.workunit.client.1.smithi121.stdout:6/34: symlink d7/dc/lf 0 2024-02-15T18:34:44.893 INFO:tasks.workunit.client.1.smithi121.stdout:6/35: mkdir d7/dc/d10 0 2024-02-15T18:34:44.894 INFO:tasks.workunit.client.1.smithi121.stdout:6/36: write d7/dc/fd [52687,1413] 0 2024-02-15T18:34:44.895 INFO:tasks.workunit.client.1.smithi121.stdout:7/19: mkdir d0/d5 0 2024-02-15T18:34:44.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.902+0000 7f6ef6086700 1 -- 172.21.15.38:0/244833904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ef00f49a0 msgr2=0x7f6ef00f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:44.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.902+0000 7f6ef6086700 1 --2- 172.21.15.38:0/244833904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ef00f49a0 0x7f6ef00f4dc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f6ee0004660 tx=0x7f6ee0009b30).stop 2024-02-15T18:34:44.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.903+0000 7f6ef6086700 1 -- 172.21.15.38:0/244833904 shutdown_connections 2024-02-15T18:34:44.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.903+0000 7f6ef6086700 1 --2- 172.21.15.38:0/244833904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ef00f5340 0x7f6ef00f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:34:44.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.903+0000 7f6ef6086700 1 --2- 172.21.15.38:0/244833904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ef00f49a0 0x7f6ef00f4dc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:44.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.903+0000 7f6ef6086700 1 -- 172.21.15.38:0/244833904 >> 172.21.15.38:0/244833904 conn(0x7f6ef00f04c0 msgr2=0x7f6ef00f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:44.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.903+0000 7f6ef6086700 1 -- 172.21.15.38:0/244833904 shutdown_connections 2024-02-15T18:34:44.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.903+0000 7f6ef6086700 1 -- 172.21.15.38:0/244833904 wait complete. 2024-02-15T18:34:44.897 INFO:tasks.workunit.client.1.smithi121.stdout:7/20: symlink d0/l6 0 2024-02-15T18:34:44.897 INFO:tasks.workunit.client.1.smithi121.stdout:7/21: rename d0/d5 to d0/d5/d7 22 2024-02-15T18:34:44.897 INFO:tasks.workunit.client.1.smithi121.stdout:6/37: dread f5 [0,4194304] 0 2024-02-15T18:34:44.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.903+0000 7f6ef6086700 1 Processor -- start 2024-02-15T18:34:44.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.904+0000 7f6ef6086700 1 -- start start 2024-02-15T18:34:44.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.904+0000 7f6ef6086700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ef00f49a0 0x7f6ef00f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:44.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.904+0000 7f6ef6086700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ef00f5340 0x7f6ef00f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:44.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.904+0000 7f6ef6086700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef00f5fc0 con 0x7f6ef00f5340 2024-02-15T18:34:44.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.904+0000 7f6ef6086700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef00f6100 con 0x7f6ef00f49a0 2024-02-15T18:34:44.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.904+0000 7f6ee6ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ef00f5340 0x7f6ef00f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:44.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.904+0000 7f6eef7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ef00f49a0 0x7f6ef00f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:44.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.904+0000 7f6ee6ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ef00f5340 0x7f6ef00f59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41616/0 (socket says 172.21.15.38:41616) 2024-02-15T18:34:44.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.904+0000 7f6ee6ffd700 1 -- 172.21.15.38:0/2492331976 learned_addr learned my addr 172.21.15.38:0/2492331976 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:34:44.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.905+0000 7f6ee6ffd700 1 -- 172.21.15.38:0/2492331976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ef00f49a0 msgr2=0x7f6ef00f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:44.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.905+0000 7f6ee6ffd700 1 --2- 172.21.15.38:0/2492331976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ef00f49a0 0x7f6ef00f7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:44.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.905+0000 7f6ee6ffd700 1 -- 172.21.15.38:0/2492331976 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee0005040 con 0x7f6ef00f5340 2024-02-15T18:34:44.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.905+0000 7f6ee6ffd700 1 --2- 172.21.15.38:0/2492331976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ef00f5340 0x7f6ef00f59d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f6ed80093f0 tx=0x7f6ed8006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:44.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.905+0000 7f6eecff9700 1 -- 172.21.15.38:0/2492331976 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ed8013640 con 0x7f6ef00f5340 2024-02-15T18:34:44.899 INFO:tasks.workunit.client.1.smithi121.stdout:7/22: rename d0/l4 to d0/d5/l8 0 2024-02-15T18:34:44.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.905+0000 7f6eecff9700 1 -- 172.21.15.38:0/2492331976 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f6ed8013c90 con 0x7f6ef00f5340 2024-02-15T18:34:44.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.905+0000 7f6ef6086700 1 -- 172.21.15.38:0/2492331976 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef00f63e0 con 0x7f6ef00f5340 2024-02-15T18:34:44.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.906+0000 7f6ef6086700 1 -- 172.21.15.38:0/2492331976 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef010f170 con 0x7f6ef00f5340 2024-02-15T18:34:44.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.906+0000 7f6eecff9700 1 -- 172.21.15.38:0/2492331976 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ed8012de0 con 0x7f6ef00f5340 2024-02-15T18:34:44.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.907+0000 7f6ef6086700 1 -- 172.21.15.38:0/2492331976 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed4004fa0 con 0x7f6ef00f5340 2024-02-15T18:34:44.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.907+0000 7f6eecff9700 1 -- 172.21.15.38:0/2492331976 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f6ed801a060 con 0x7f6ef00f5340 2024-02-15T18:34:44.900 INFO:tasks.workunit.client.1.smithi121.stdout:7/23: creat d0/d5/f9 x:0 0 0 2024-02-15T18:34:44.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.908+0000 7f6eecff9700 1 --2- 172.21.15.38:0/2492331976 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6edc05c300 0x7f6edc05e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:44.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.908+0000 7f6eecff9700 1 -- 172.21.15.38:0/2492331976 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f6ed8075cc0 con 0x7f6ef00f5340 2024-02-15T18:34:44.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.908+0000 7f6eef7fe700 1 --2- 172.21.15.38:0/2492331976 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6edc05c300 0x7f6edc05e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:44.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.908+0000 7f6eef7fe700 1 --2- 172.21.15.38:0/2492331976 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6edc05c300 0x7f6edc05e7c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f6ef00f72e0 tx=0x7f6ee0009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:44.901 INFO:tasks.workunit.client.1.smithi121.stdout:7/24: symlink d0/la 0 2024-02-15T18:34:44.902 INFO:tasks.workunit.client.1.smithi121.stdout:7/25: creat d0/fb x:0 0 0 2024-02-15T18:34:44.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:44.912+0000 7f6eecff9700 1 -- 172.21.15.38:0/2492331976 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f6ed8048a00 con 0x7f6ef00f5340 2024-02-15T18:34:44.904 INFO:tasks.workunit.client.1.smithi121.stdout:7/26: write d0/f3 [870649,91249] 0 2024-02-15T18:34:44.905 INFO:tasks.workunit.client.1.smithi121.stdout:7/27: creat d0/fc x:0 0 0 2024-02-15T18:34:44.905 INFO:tasks.workunit.client.1.smithi121.stdout:7/28: dread - d0/fb zero size 2024-02-15T18:34:44.905 INFO:tasks.workunit.client.1.smithi121.stdout:7/29: write d0/fc [360694,53711] 0 2024-02-15T18:34:44.907 INFO:tasks.workunit.client.1.smithi121.stdout:8/23: dwrite d0/f1 [0,4194304] 0 2024-02-15T18:34:44.907 INFO:tasks.workunit.client.1.smithi121.stdout:7/30: write d0/f3 [1438538,96503] 0 2024-02-15T18:34:44.908 INFO:tasks.workunit.client.1.smithi121.stdout:8/24: creat d0/f8 x:0 0 0 2024-02-15T18:34:44.909 INFO:tasks.workunit.client.1.smithi121.stdout:8/25: stat d0/f8 0 2024-02-15T18:34:44.909 INFO:tasks.workunit.client.1.smithi121.stdout:8/26: write d0/d3/f6 [290449,70027] 0 2024-02-15T18:34:44.910 INFO:tasks.workunit.client.1.smithi121.stdout:8/27: mknod d0/d3/c9 0 2024-02-15T18:34:44.910 INFO:tasks.workunit.client.1.smithi121.stdout:8/28: chown d0 1900565 1 2024-02-15T18:34:44.913 INFO:tasks.workunit.client.1.smithi121.stdout:8/29: dread d0/f2 [0,4194304] 0 2024-02-15T18:34:44.913 INFO:tasks.workunit.client.1.smithi121.stdout:8/30: chown d0/d3/f6 0 1 2024-02-15T18:34:44.914 INFO:tasks.workunit.client.1.smithi121.stdout:7/31: fsync d0/d5/f9 0 2024-02-15T18:34:44.914 INFO:tasks.workunit.client.1.smithi121.stdout:7/32: dread - d0/d5/f9 zero size 2024-02-15T18:34:44.914 INFO:tasks.workunit.client.1.smithi121.stdout:7/33: write d0/d5/f9 [652558,54799] 0 2024-02-15T18:34:44.919 INFO:tasks.workunit.client.1.smithi121.stdout:8/31: link d0/f1 d0/d7/fa 0 2024-02-15T18:34:44.919 INFO:tasks.workunit.client.1.smithi121.stdout:8/32: chown d0/d7 1871778 1 2024-02-15T18:34:44.919 INFO:tasks.workunit.client.1.smithi121.stdout:8/33: truncate d0/f5 829148 0 2024-02-15T18:34:44.919 INFO:tasks.workunit.client.1.smithi121.stdout:7/34: creat d0/fd x:0 0 0 2024-02-15T18:34:44.921 INFO:tasks.workunit.client.1.smithi121.stdout:8/34: creat d0/d7/fb x:0 0 0 2024-02-15T18:34:44.922 INFO:tasks.workunit.client.1.smithi121.stdout:8/35: creat d0/fc x:0 0 0 2024-02-15T18:34:44.924 INFO:tasks.workunit.client.1.smithi121.stdout:7/35: mknod d0/ce 0 2024-02-15T18:34:44.924 INFO:tasks.workunit.client.1.smithi121.stdout:7/36: truncate d0/d5/f9 902404 0 2024-02-15T18:34:44.925 INFO:tasks.workunit.client.1.smithi121.stdout:8/36: mknod d0/d7/cd 0 2024-02-15T18:34:44.933 INFO:tasks.workunit.client.1.smithi121.stdout:7/37: dread d0/f3 [0,4194304] 0 2024-02-15T18:34:44.935 INFO:tasks.workunit.client.1.smithi121.stdout:7/38: mknod d0/cf 0 2024-02-15T18:34:44.935 INFO:tasks.workunit.client.1.smithi121.stdout:7/39: chown d0/fd 2762870 1 2024-02-15T18:34:44.942 INFO:tasks.workunit.client.1.smithi121.stdout:8/37: dwrite d0/d7/fa [0,4194304] 0 2024-02-15T18:34:44.943 INFO:tasks.workunit.client.1.smithi121.stdout:8/38: write d0/d3/f6 [1364402,31268] 0 2024-02-15T18:34:44.944 INFO:tasks.workunit.client.1.smithi121.stdout:8/39: mkdir d0/d7/de 0 2024-02-15T18:34:44.944 INFO:tasks.workunit.client.1.smithi121.stdout:8/40: truncate d0/f8 776499 0 2024-02-15T18:34:44.944 INFO:tasks.workunit.client.1.smithi121.stdout:8/41: chown d0/d3/c9 1142807 1 2024-02-15T18:34:44.944 INFO:tasks.workunit.client.1.smithi121.stdout:8/42: fdatasync d0/f5 0 2024-02-15T18:34:44.951 INFO:tasks.workunit.client.1.smithi121.stdout:8/43: truncate d0/f5 479661 0 2024-02-15T18:34:44.953 INFO:tasks.workunit.client.1.smithi121.stdout:8/44: symlink d0/d7/de/lf 0 2024-02-15T18:34:44.953 INFO:tasks.workunit.client.1.smithi121.stdout:8/45: chown d0/fc 2 1 2024-02-15T18:34:44.953 INFO:tasks.workunit.client.1.smithi121.stdout:8/46: write d0/fc [287396,53259] 0 2024-02-15T18:34:44.953 INFO:tasks.workunit.client.1.smithi121.stdout:8/47: truncate d0/d3/f6 1672662 0 2024-02-15T18:34:44.954 INFO:tasks.workunit.client.1.smithi121.stdout:8/48: getdents d0/d7 0 2024-02-15T18:34:44.956 INFO:tasks.workunit.client.1.smithi121.stdout:8/49: rename d0/d7/fb to d0/d7/f10 0 2024-02-15T18:34:44.957 INFO:tasks.workunit.client.1.smithi121.stdout:8/50: stat d0/d7/de/lf 0 2024-02-15T18:34:44.957 INFO:tasks.workunit.client.1.smithi121.stdout:8/51: chown d0/d3/c9 2568 1 2024-02-15T18:34:44.957 INFO:tasks.workunit.client.1.smithi121.stdout:8/52: write d0/f5 [1020116,7473] 0 2024-02-15T18:34:44.957 INFO:tasks.workunit.client.1.smithi121.stdout:8/53: truncate d0/f1 4507265 0 2024-02-15T18:34:44.959 INFO:tasks.workunit.client.1.smithi121.stdout:8/54: rename d0/d3/f6 to d0/d7/de/f11 0 2024-02-15T18:34:44.959 INFO:tasks.workunit.client.1.smithi121.stdout:7/40: dwrite d0/fc [0,4194304] 0 2024-02-15T18:34:44.961 INFO:tasks.workunit.client.1.smithi121.stdout:8/55: creat d0/d7/f12 x:0 0 0 2024-02-15T18:34:44.961 INFO:tasks.workunit.client.1.smithi121.stdout:8/56: write d0/d7/f10 [602850,96310] 0 2024-02-15T18:34:44.965 INFO:tasks.workunit.client.1.smithi121.stdout:8/57: rename d0/fc to d0/f13 0 2024-02-15T18:34:44.966 INFO:tasks.workunit.client.1.smithi121.stdout:7/41: rename d0/f3 to d0/f10 0 2024-02-15T18:34:44.971 INFO:tasks.workunit.client.1.smithi121.stdout:7/42: rmdir d0/d5 39 2024-02-15T18:34:44.971 INFO:tasks.workunit.client.1.smithi121.stdout:7/43: chown d0/d5/l8 423 1 2024-02-15T18:34:44.971 INFO:tasks.workunit.client.1.smithi121.stdout:7/44: truncate d0/fb 166624 0 2024-02-15T18:34:44.973 INFO:tasks.workunit.client.1.smithi121.stdout:7/45: mkdir d0/d5/d11 0 2024-02-15T18:34:44.975 INFO:tasks.workunit.client.1.smithi121.stdout:7/46: creat d0/d5/d11/f12 x:0 0 0 2024-02-15T18:34:44.977 INFO:tasks.workunit.client.1.smithi121.stdout:7/47: mknod d0/d5/d11/c13 0 2024-02-15T18:34:44.982 INFO:tasks.workunit.client.1.smithi121.stdout:8/58: dwrite d0/f2 [0,4194304] 0 2024-02-15T18:34:44.983 INFO:tasks.workunit.client.1.smithi121.stdout:8/59: dread d0/f8 [0,4194304] 0 2024-02-15T18:34:44.986 INFO:tasks.workunit.client.1.smithi121.stdout:8/60: write d0/d7/fa [4941205,47596] 0 2024-02-15T18:34:44.988 INFO:tasks.workunit.client.1.smithi121.stdout:7/48: dread d0/d5/f9 [0,4194304] 0 2024-02-15T18:34:44.988 INFO:tasks.workunit.client.1.smithi121.stdout:8/61: unlink d0/d7/de/lf 0 2024-02-15T18:34:44.992 INFO:tasks.workunit.client.1.smithi121.stdout:7/49: link d0/l6 d0/l14 0 2024-02-15T18:34:44.993 INFO:tasks.workunit.client.1.smithi121.stdout:7/50: write d0/f10 [1433991,83028] 0 2024-02-15T18:34:45.003 INFO:tasks.workunit.client.1.smithi121.stdout:7/51: fdatasync d0/f10 0 2024-02-15T18:34:45.003 INFO:tasks.workunit.client.1.smithi121.stdout:7/52: chown d0/d5/d11/f12 50170082 1 2024-02-15T18:34:45.003 INFO:tasks.workunit.client.1.smithi121.stdout:7/53: creat d0/d5/f15 x:0 0 0 2024-02-15T18:34:45.005 INFO:tasks.workunit.client.1.smithi121.stdout:7/54: unlink d0/d5/d11/f12 0 2024-02-15T18:34:45.005 INFO:tasks.workunit.client.1.smithi121.stdout:7/55: chown d0/f10 4737 1 2024-02-15T18:34:45.005 INFO:tasks.workunit.client.1.smithi121.stdout:7/56: write d0/d5/f9 [1639277,34656] 0 2024-02-15T18:34:45.006 INFO:tasks.workunit.client.1.smithi121.stdout:7/57: unlink d0/l6 0 2024-02-15T18:34:45.007 INFO:tasks.workunit.client.1.smithi121.stdout:8/62: dwrite d0/d7/f10 [0,4194304] 0 2024-02-15T18:34:45.007 INFO:tasks.workunit.client.1.smithi121.stdout:8/63: chown d0/d7/f10 3653866 1 2024-02-15T18:34:45.010 INFO:tasks.workunit.client.1.smithi121.stdout:7/58: write d0/fc [427840,36688] 0 2024-02-15T18:34:45.010 INFO:tasks.workunit.client.1.smithi121.stdout:7/59: readlink d0/la 0 2024-02-15T18:34:45.010 INFO:tasks.workunit.client.1.smithi121.stdout:8/64: unlink d0/d7/de/f11 0 2024-02-15T18:34:45.011 INFO:tasks.workunit.client.1.smithi121.stdout:8/65: symlink d0/l14 0 2024-02-15T18:34:45.013 INFO:tasks.workunit.client.1.smithi121.stdout:7/60: dread d0/f10 [0,4194304] 0 2024-02-15T18:34:45.014 INFO:tasks.workunit.client.1.smithi121.stdout:7/61: write d0/f10 [1866641,128042] 0 2024-02-15T18:34:45.014 INFO:tasks.workunit.client.1.smithi121.stdout:7/62: readlink d0/la 0 2024-02-15T18:34:45.015 INFO:tasks.workunit.client.1.smithi121.stdout:8/66: creat d0/f15 x:0 0 0 2024-02-15T18:34:45.015 INFO:tasks.workunit.client.1.smithi121.stdout:8/67: fsync d0/d7/f12 0 2024-02-15T18:34:45.017 INFO:tasks.workunit.client.1.smithi121.stdout:8/68: mknod d0/d3/c16 0 2024-02-15T18:34:45.018 INFO:tasks.workunit.client.1.smithi121.stdout:7/63: rename d0 to d0/d16 22 2024-02-15T18:34:45.028 INFO:tasks.workunit.client.1.smithi121.stdout:8/69: mknod d0/d3/c17 0 2024-02-15T18:34:45.028 INFO:tasks.workunit.client.1.smithi121.stdout:8/70: readlink d0/l14 0 2024-02-15T18:34:45.029 INFO:tasks.workunit.client.1.smithi121.stdout:7/64: rename d0 to d0/d5/d17 22 2024-02-15T18:34:45.029 INFO:tasks.workunit.client.1.smithi121.stdout:8/71: getdents d0/d7 0 2024-02-15T18:34:45.031 INFO:tasks.workunit.client.1.smithi121.stdout:7/65: creat d0/f18 x:0 0 0 2024-02-15T18:34:45.031 INFO:tasks.workunit.client.1.smithi121.stdout:8/72: creat d0/d7/de/f18 x:0 0 0 2024-02-15T18:34:45.032 INFO:tasks.workunit.client.1.smithi121.stdout:7/66: symlink d0/d5/d11/l19 0 2024-02-15T18:34:45.032 INFO:tasks.workunit.client.1.smithi121.stdout:7/67: dread - d0/d5/f15 zero size 2024-02-15T18:34:45.033 INFO:tasks.workunit.client.1.smithi121.stdout:8/73: symlink d0/l19 0 2024-02-15T18:34:45.033 INFO:tasks.workunit.client.1.smithi121.stdout:7/68: symlink d0/d5/l1a 0 2024-02-15T18:34:45.035 INFO:tasks.workunit.client.1.smithi121.stdout:7/69: readlink d0/l14 0 2024-02-15T18:34:45.041 INFO:tasks.workunit.client.1.smithi121.stdout:8/74: truncate d0/f2 3426106 0 2024-02-15T18:34:45.043 INFO:tasks.workunit.client.1.smithi121.stdout:8/75: creat d0/d7/de/f1a x:0 0 0 2024-02-15T18:34:45.046 INFO:tasks.workunit.client.1.smithi121.stdout:8/76: fdatasync d0/f5 0 2024-02-15T18:34:45.047 INFO:tasks.workunit.client.1.smithi121.stdout:8/77: creat d0/d7/f1b x:0 0 0 2024-02-15T18:34:45.056 INFO:tasks.workunit.client.1.smithi121.stdout:7/70: dread d0/d5/f9 [0,4194304] 0 2024-02-15T18:34:45.062 INFO:tasks.workunit.client.1.smithi121.stdout:8/78: dwrite d0/d7/f1b [0,4194304] 0 2024-02-15T18:34:45.070 INFO:tasks.workunit.client.1.smithi121.stdout:7/71: rename d0/d5/d11/l19 to d0/d5/l1b 0 2024-02-15T18:34:45.070 INFO:tasks.workunit.client.1.smithi121.stdout:7/72: truncate d0/d5/f15 245097 0 2024-02-15T18:34:45.071 INFO:tasks.workunit.client.1.smithi121.stdout:7/73: mkdir d0/d5/d11/d1c 0 2024-02-15T18:34:45.078 INFO:tasks.workunit.client.1.smithi121.stdout:8/79: dwrite d0/f2 [0,4194304] 0 2024-02-15T18:34:45.080 INFO:tasks.workunit.client.1.smithi121.stdout:8/80: rename d0/f1 to d0/d7/de/f1c 0 2024-02-15T18:34:45.082 INFO:tasks.workunit.client.1.smithi121.stdout:7/74: truncate d0/fc 3324346 0 2024-02-15T18:34:45.084 INFO:tasks.workunit.client.1.smithi121.stdout:8/81: mknod d0/c1d 0 2024-02-15T18:34:45.084 INFO:tasks.workunit.client.1.smithi121.stdout:7/75: fdatasync d0/fb 0 2024-02-15T18:34:45.084 INFO:tasks.workunit.client.1.smithi121.stdout:7/76: readlink d0/d5/l1b 0 2024-02-15T18:34:45.085 INFO:tasks.workunit.client.1.smithi121.stdout:7/77: chown d0/fd 16524595 1 2024-02-15T18:34:45.085 INFO:tasks.workunit.client.1.smithi121.stdout:7/78: read d0/d5/f9 [1516048,112827] 0 2024-02-15T18:34:45.088 INFO:tasks.workunit.client.1.smithi121.stdout:7/79: unlink d0/cf 0 2024-02-15T18:34:45.089 INFO:tasks.workunit.client.1.smithi121.stdout:7/80: mknod d0/c1d 0 2024-02-15T18:34:45.090 INFO:tasks.workunit.client.1.smithi121.stdout:7/81: read d0/fc [2864033,2265] 0 2024-02-15T18:34:45.090 INFO:tasks.workunit.client.1.smithi121.stdout:7/82: fsync d0/d5/f9 0 2024-02-15T18:34:45.093 INFO:tasks.workunit.client.1.smithi121.stdout:7/83: dread d0/d5/f9 [0,4194304] 0 2024-02-15T18:34:45.094 INFO:tasks.workunit.client.1.smithi121.stdout:7/84: creat d0/d5/f1e x:0 0 0 2024-02-15T18:34:45.097 INFO:tasks.workunit.client.1.smithi121.stdout:7/85: creat d0/d5/d11/d1c/f1f x:0 0 0 2024-02-15T18:34:45.097 INFO:tasks.workunit.client.1.smithi121.stdout:7/86: chown d0/d5/f1e 8303 1 2024-02-15T18:34:45.115 INFO:tasks.workunit.client.1.smithi121.stdout:8/82: dwrite d0/d7/fa [0,4194304] 0 2024-02-15T18:34:45.119 INFO:tasks.workunit.client.1.smithi121.stdout:8/83: rename d0/d7/f12 to d0/d7/f1e 0 2024-02-15T18:34:45.120 INFO:tasks.workunit.client.1.smithi121.stdout:8/84: symlink d0/l1f 0 2024-02-15T18:34:45.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.133+0000 7f6ef6086700 1 -- 172.21.15.38:0/2492331976 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6ed4005e80 con 0x7f6ef00f5340 2024-02-15T18:34:45.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.134+0000 7f6eecff9700 1 -- 172.21.15.38:0/2492331976 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1643 (secure 0 0 0) 0x7f6ed804c020 con 0x7f6ef00f5340 2024-02-15T18:34:45.126 INFO:teuthology.orchestra.run.smithi038.stdout:e12 2024-02-15T18:34:45.126 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:34:45.126 INFO:teuthology.orchestra.run.smithi038.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:34:45.126 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-02-15T18:34:45.126 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:34:45.126 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:34:45.126 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-02-15T18:34:45.126 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 12 2024-02-15T18:34:45.126 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 2024-02-15T18:34:45.126 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:34:45.126 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-02-15T18:34:24.368128+0000 2024-02-15T18:34:45.126 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-02-15T18:34:45.127 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-02-15T18:34:45.127 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-02-15T18:34:45.127 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-02-15T18:34:45.127 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-02-15T18:34:45.127 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-02-15T18:34:45.127 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-02-15T18:34:45.127 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-02-15T18:34:45.127 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:34:45.127 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-02-15T18:34:45.127 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-02-15T18:34:45.127 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=24261,1=24275} 2024-02-15T18:34:45.127 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-02-15T18:34:45.127 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-02-15T18:34:45.127 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-02-15T18:34:45.128 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-02-15T18:34:45.128 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-02-15T18:34:45.128 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-02-15T18:34:45.128 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-02-15T18:34:45.128 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-02-15T18:34:45.128 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.mvumlh{0:24261} state up:active seq 3 export targets 1 join_fscid=1 addr [v2:172.21.15.121:6824/1422707571,v1:172.21.15.121:6825/1422707571]] 2024-02-15T18:34:45.128 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.ovuzki{1:24275} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845]] 2024-02-15T18:34:45.128 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:34:45.128 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:34:45.128 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-02-15T18:34:45.128 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:34:45.128 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.hxoggl{-1:14446} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.38:6826/3822880235,v1:172.21.15.38:6827/3822880235]] 2024-02-15T18:34:45.128 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.qgkzgn{-1:14456} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.38:6828/703030231,v1:172.21.15.38:6829/703030231]] 2024-02-15T18:34:45.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.136+0000 7f6ef6086700 1 -- 172.21.15.38:0/2492331976 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6edc05c300 msgr2=0x7f6edc05e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:45.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.136+0000 7f6ef6086700 1 --2- 172.21.15.38:0/2492331976 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6edc05c300 0x7f6edc05e7c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f6ef00f72e0 tx=0x7f6ee0009b30).stop 2024-02-15T18:34:45.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.136+0000 7f6ef6086700 1 -- 172.21.15.38:0/2492331976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ef00f5340 msgr2=0x7f6ef00f59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:45.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.136+0000 7f6ef6086700 1 --2- 172.21.15.38:0/2492331976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ef00f5340 0x7f6ef00f59d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f6ed80093f0 tx=0x7f6ed8006ec0).stop 2024-02-15T18:34:45.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.136+0000 7f6ef6086700 1 -- 172.21.15.38:0/2492331976 shutdown_connections 2024-02-15T18:34:45.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.136+0000 7f6ef6086700 1 --2- 172.21.15.38:0/2492331976 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f6edc05c300 0x7f6edc05e7c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:45.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.136+0000 7f6ef6086700 1 --2- 172.21.15.38:0/2492331976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ef00f49a0 0x7f6ef00f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:45.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.136+0000 7f6ef6086700 1 --2- 172.21.15.38:0/2492331976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ef00f5340 0x7f6ef00f59d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:45.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.136+0000 7f6ef6086700 1 -- 172.21.15.38:0/2492331976 >> 172.21.15.38:0/2492331976 conn(0x7f6ef00f04c0 msgr2=0x7f6ef00fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:45.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.136+0000 7f6ef6086700 1 -- 172.21.15.38:0/2492331976 shutdown_connections 2024-02-15T18:34:45.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.136+0000 7f6ef6086700 1 -- 172.21.15.38:0/2492331976 wait complete. 2024-02-15T18:34:45.130 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 12 2024-02-15T18:34:45.140 INFO:tasks.workunit.client.1.smithi121.stdout:8/85: dwrite d0/d7/de/f18 [0,4194304] 0 2024-02-15T18:34:45.141 INFO:tasks.workunit.client.1.smithi121.stdout:8/86: mkdir d0/d3/d20 0 2024-02-15T18:34:45.142 INFO:tasks.workunit.client.1.smithi121.stdout:8/87: mknod d0/d7/c21 0 2024-02-15T18:34:45.144 INFO:tasks.workunit.client.1.smithi121.stdout:8/88: mknod d0/d7/de/c22 0 2024-02-15T18:34:45.144 INFO:tasks.workunit.client.1.smithi121.stdout:8/89: fdatasync d0/d7/f10 0 2024-02-15T18:34:45.144 INFO:tasks.workunit.client.1.smithi121.stdout:8/90: write d0/d7/f10 [192165,18451] 0 2024-02-15T18:34:45.149 INFO:tasks.workunit.client.1.smithi121.stdout:8/91: mknod d0/d3/c23 0 2024-02-15T18:34:45.157 INFO:tasks.workunit.client.1.smithi121.stdout:8/92: dread d0/d7/f1b [0,4194304] 0 2024-02-15T18:34:45.158 INFO:tasks.workunit.client.1.smithi121.stdout:8/93: read d0/d7/fa [352982,84242] 0 2024-02-15T18:34:45.160 INFO:tasks.workunit.client.1.smithi121.stdout:8/94: write d0/d7/fa [820893,2572] 0 2024-02-15T18:34:45.160 INFO:tasks.workunit.client.1.smithi121.stdout:8/95: chown d0/d3/c9 53 1 2024-02-15T18:34:45.160 INFO:tasks.workunit.client.1.smithi121.stdout:8/96: write d0/f15 [914402,94168] 0 2024-02-15T18:34:45.160 INFO:tasks.workunit.client.1.smithi121.stdout:8/97: read d0/d7/de/f18 [3745072,106512] 0 2024-02-15T18:34:45.161 INFO:tasks.workunit.client.1.smithi121.stdout:8/98: symlink d0/l24 0 2024-02-15T18:34:45.162 INFO:tasks.workunit.client.1.smithi121.stdout:8/99: mkdir d0/d7/d25 0 2024-02-15T18:34:45.169 INFO:tasks.workunit.client.1.smithi121.stdout:8/100: rmdir d0 39 2024-02-15T18:34:45.196 INFO:tasks.workunit.client.1.smithi121.stdout:8/101: dwrite d0/d7/f10 [0,4194304] 0 2024-02-15T18:34:45.197 INFO:tasks.workunit.client.1.smithi121.stdout:8/102: creat d0/d3/f26 x:0 0 0 2024-02-15T18:34:45.198 INFO:tasks.workunit.client.1.smithi121.stdout:8/103: mkdir d0/d7/de/d27 0 2024-02-15T18:34:45.217 INFO:tasks.workunit.client.1.smithi121.stdout:8/104: dwrite d0/f15 [0,4194304] 0 2024-02-15T18:34:45.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.250+0000 7f79d9bb8700 1 -- 172.21.15.38:0/1153585167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79d40f66d0 msgr2=0x7f79d40f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:45.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.250+0000 7f79d9bb8700 1 --2- 172.21.15.38:0/1153585167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79d40f66d0 0x7f79d40f6b30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f79c0004660 tx=0x7f79c0009b30).stop 2024-02-15T18:34:45.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.250+0000 7f79d9bb8700 1 -- 172.21.15.38:0/1153585167 shutdown_connections 2024-02-15T18:34:45.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.250+0000 7f79d9bb8700 1 --2- 172.21.15.38:0/1153585167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79d40f66d0 0x7f79d40f6b30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:45.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.250+0000 7f79d9bb8700 1 --2- 172.21.15.38:0/1153585167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d40f5430 0x7f79d40f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:45.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.250+0000 7f79d9bb8700 1 -- 172.21.15.38:0/1153585167 >> 172.21.15.38:0/1153585167 conn(0x7f79d40f0b90 msgr2=0x7f79d40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:45.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.250+0000 7f79d9bb8700 1 -- 172.21.15.38:0/1153585167 shutdown_connections 2024-02-15T18:34:45.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.250+0000 7f79d9bb8700 1 -- 172.21.15.38:0/1153585167 wait complete. 2024-02-15T18:34:45.243 INFO:tasks.workunit.client.1.smithi121.stdout:8/105: dwrite d0/d3/f26 [0,4194304] 0 2024-02-15T18:34:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.251+0000 7f79d9bb8700 1 Processor -- start 2024-02-15T18:34:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.251+0000 7f79d9bb8700 1 -- start start 2024-02-15T18:34:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.251+0000 7f79d9bb8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d40f5430 0x7f79d4109ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.251+0000 7f79d9bb8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79d40f66d0 0x7f79d410bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.251+0000 7f79d9bb8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d40603e0 con 0x7f79d40f66d0 2024-02-15T18:34:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.251+0000 7f79d9bb8700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d4060520 con 0x7f79d40f5430 2024-02-15T18:34:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.251+0000 7f79d3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79d40f66d0 0x7f79d410bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.251+0000 7f79d3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79d40f66d0 0x7f79d410bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41644/0 (socket says 172.21.15.38:41644) 2024-02-15T18:34:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.251+0000 7f79d3fff700 1 -- 172.21.15.38:0/455205764 learned_addr learned my addr 172.21.15.38:0/455205764 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:34:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.252+0000 7f79d8bb6700 1 --2- 172.21.15.38:0/455205764 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d40f5430 0x7f79d4109ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.252+0000 7f79d3fff700 1 -- 172.21.15.38:0/455205764 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d40f5430 msgr2=0x7f79d4109ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.252+0000 7f79d3fff700 1 --2- 172.21.15.38:0/455205764 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d40f5430 0x7f79d4109ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.252+0000 7f79d3fff700 1 -- 172.21.15.38:0/455205764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c0005040 con 0x7f79d40f66d0 2024-02-15T18:34:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.252+0000 7f79d3fff700 1 --2- 172.21.15.38:0/455205764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79d40f66d0 0x7f79d410bfb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f79c0000c00 tx=0x7f79c000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:45.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.252+0000 7f79d17fa700 1 -- 172.21.15.38:0/455205764 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79c0019070 con 0x7f79d40f66d0 2024-02-15T18:34:45.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.252+0000 7f79d17fa700 1 -- 172.21.15.38:0/455205764 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f79c0007e80 con 0x7f79d40f66d0 2024-02-15T18:34:45.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.252+0000 7f79d9bb8700 1 -- 172.21.15.38:0/455205764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d410c500 con 0x7f79d40f66d0 2024-02-15T18:34:45.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.252+0000 7f79d17fa700 1 -- 172.21.15.38:0/455205764 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79c0014db0 con 0x7f79d40f66d0 2024-02-15T18:34:45.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.252+0000 7f79d9bb8700 1 -- 172.21.15.38:0/455205764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d410ca60 con 0x7f79d40f66d0 2024-02-15T18:34:45.246 INFO:tasks.workunit.client.1.smithi121.stdout:8/106: mkdir d0/d7/de/d27/d28 0 2024-02-15T18:34:45.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.253+0000 7f79d9bb8700 1 -- 172.21.15.38:0/455205764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d4045ad0 con 0x7f79d40f66d0 2024-02-15T18:34:45.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.253+0000 7f79d17fa700 1 -- 172.21.15.38:0/455205764 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f79c0025070 con 0x7f79d40f66d0 2024-02-15T18:34:45.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.254+0000 7f79d17fa700 1 --2- 172.21.15.38:0/455205764 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f79c405c300 0x7f79c405e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:45.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.254+0000 7f79d17fa700 1 -- 172.21.15.38:0/455205764 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f79c0076ce0 con 0x7f79d40f66d0 2024-02-15T18:34:45.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.254+0000 7f79d8bb6700 1 --2- 172.21.15.38:0/455205764 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f79c405c300 0x7f79c405e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:45.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.254+0000 7f79d8bb6700 1 --2- 172.21.15.38:0/455205764 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f79c405c300 0x7f79c405e7c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f79d4060060 tx=0x7f79c800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:45.248 INFO:tasks.workunit.client.1.smithi121.stdout:8/107: symlink d0/d7/de/d27/l29 0 2024-02-15T18:34:45.248 INFO:tasks.workunit.client.1.smithi121.stdout:8/108: dread d0/f5 [0,4194304] 0 2024-02-15T18:34:45.249 INFO:tasks.workunit.client.1.smithi121.stdout:8/109: chown d0/d3/d20 46366309 1 2024-02-15T18:34:45.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.258+0000 7f79d17fa700 1 -- 172.21.15.38:0/455205764 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f79c0049a20 con 0x7f79d40f66d0 2024-02-15T18:34:45.263 INFO:tasks.workunit.client.1.smithi121.stdout:8/110: dwrite d0/d7/f1e [0,4194304] 0 2024-02-15T18:34:45.266 INFO:tasks.workunit.client.1.smithi121.stdout:8/111: mkdir d0/d2a 0 2024-02-15T18:34:45.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.452+0000 7f79d9bb8700 1 -- 172.21.15.38:0/455205764 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f79d4057f10 con 0x7f79c405c300 2024-02-15T18:34:45.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.453+0000 7f79d17fa700 1 -- 172.21.15.38:0/455205764 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+248 (secure 0 0 0) 0x7f79d4057f10 con 0x7f79c405c300 2024-02-15T18:34:45.445 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:34:45.445 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:34:45.446 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-02-15T18:34:45.446 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [], 2024-02-15T18:34:45.446 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "", 2024-02-15T18:34:45.446 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading mgr daemons" 2024-02-15T18:34:45.446 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:34:45.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.455+0000 7f79d9bb8700 1 -- 172.21.15.38:0/455205764 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f79c405c300 msgr2=0x7f79c405e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:45.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.455+0000 7f79d9bb8700 1 --2- 172.21.15.38:0/455205764 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f79c405c300 0x7f79c405e7c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f79d4060060 tx=0x7f79c800b040).stop 2024-02-15T18:34:45.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.455+0000 7f79d9bb8700 1 -- 172.21.15.38:0/455205764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79d40f66d0 msgr2=0x7f79d410bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:45.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.455+0000 7f79d9bb8700 1 --2- 172.21.15.38:0/455205764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79d40f66d0 0x7f79d410bfb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f79c0000c00 tx=0x7f79c000b040).stop 2024-02-15T18:34:45.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.455+0000 7f79d9bb8700 1 -- 172.21.15.38:0/455205764 shutdown_connections 2024-02-15T18:34:45.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.455+0000 7f79d9bb8700 1 --2- 172.21.15.38:0/455205764 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f79c405c300 0x7f79c405e7c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:45.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.455+0000 7f79d9bb8700 1 --2- 172.21.15.38:0/455205764 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d40f5430 0x7f79d4109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:45.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.455+0000 7f79d9bb8700 1 --2- 172.21.15.38:0/455205764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79d40f66d0 0x7f79d410bfb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:45.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.455+0000 7f79d9bb8700 1 -- 172.21.15.38:0/455205764 >> 172.21.15.38:0/455205764 conn(0x7f79d40f0b90 msgr2=0x7f79d40fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:45.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.455+0000 7f79d9bb8700 1 -- 172.21.15.38:0/455205764 shutdown_connections 2024-02-15T18:34:45.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:34:45.455+0000 7f79d9bb8700 1 -- 172.21.15.38:0/455205764 wait complete. 2024-02-15T18:34:45.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:44.118372+0000 mgr.smithi038.khesbl (mgr.14162) 329 : audit [DBG] from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:45.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:44.315081+0000 mgr.smithi038.khesbl (mgr.14162) 330 : cluster [DBG] pgmap v274: 65 pgs: 65 active+clean; 168 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 7.8 MiB/s wr, 876 op/s 2024-02-15T18:34:45.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:44.426501+0000 mgr.smithi038.khesbl (mgr.14162) 331 : audit [DBG] from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:45.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:44.778299+0000 mon.smithi038 (mon.0) 567 : audit [DBG] from='client.? 172.21.15.38:0/1721982501' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:45.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:45.135575+0000 mon.smithi038 (mon.0) 568 : audit [DBG] from='client.? 172.21.15.38:0/2492331976' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:34:45.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:44.118372+0000 mgr.smithi038.khesbl (mgr.14162) 329 : audit [DBG] from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:45.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:44.315081+0000 mgr.smithi038.khesbl (mgr.14162) 330 : cluster [DBG] pgmap v274: 65 pgs: 65 active+clean; 168 MiB data, 736 MiB used, 536 GiB / 536 GiB avail; 7.8 MiB/s wr, 876 op/s 2024-02-15T18:34:45.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:44.426501+0000 mgr.smithi038.khesbl (mgr.14162) 331 : audit [DBG] from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:45.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:44.778299+0000 mon.smithi038 (mon.0) 567 : audit [DBG] from='client.? 172.21.15.38:0/1721982501' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:45.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:45.135575+0000 mon.smithi038 (mon.0) 568 : audit [DBG] from='client.? 172.21.15.38:0/2492331976' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:34:46.496 INFO:tasks.workunit.client.0.smithi038.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-02-15T18:34:46.498 INFO:tasks.workunit.client.0.smithi038.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-15T18:34:46.498 INFO:tasks.workunit.client.0.smithi038.stderr:+ make 2024-02-15T18:34:46.614 INFO:tasks.workunit.client.0.smithi038.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-02-15T18:34:46.771 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:46 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:45.455068+0000 mgr.smithi038.khesbl (mgr.14162) 332 : audit [DBG] from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:46.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:45.455068+0000 mgr.smithi038.khesbl (mgr.14162) 332 : audit [DBG] from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:46.870 INFO:tasks.workunit.client.0.smithi038.stderr:++ readlink -f fsstress 2024-02-15T18:34:46.871 INFO:tasks.workunit.client.0.smithi038.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-02-15T18:34:46.871 INFO:tasks.workunit.client.0.smithi038.stderr:+ popd 2024-02-15T18:34:46.872 INFO:tasks.workunit.client.0.smithi038.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-15T18:34:46.872 INFO:tasks.workunit.client.0.smithi038.stderr:+ popd 2024-02-15T18:34:46.873 INFO:tasks.workunit.client.0.smithi038.stdout:~/cephtest/mnt.0/client.0/tmp 2024-02-15T18:34:46.873 INFO:tasks.workunit.client.0.smithi038.stderr:++ mktemp -d -p . 2024-02-15T18:34:46.874 INFO:tasks.workunit.client.0.smithi038.stderr:+ T=./tmp.aNO9Ecf3kJ 2024-02-15T18:34:46.874 INFO:tasks.workunit.client.0.smithi038.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.aNO9Ecf3kJ -l 1 -n 1000 -p 10 -v 2024-02-15T18:34:46.875 INFO:tasks.workunit.client.0.smithi038.stdout:seed = 1707333784 2024-02-15T18:34:46.877 INFO:tasks.workunit.client.0.smithi038.stdout:0/0: stat - no entries 2024-02-15T18:34:46.877 INFO:tasks.workunit.client.0.smithi038.stdout:0/1: stat - no entries 2024-02-15T18:34:46.877 INFO:tasks.workunit.client.0.smithi038.stdout:0/2: rename - no filename 2024-02-15T18:34:46.877 INFO:tasks.workunit.client.0.smithi038.stdout:0/3: chown . 44065511 1 2024-02-15T18:34:46.877 INFO:tasks.workunit.client.0.smithi038.stdout:0/4: unlink - no file 2024-02-15T18:34:46.877 INFO:tasks.workunit.client.0.smithi038.stdout:0/5: chown . 10855700 1 2024-02-15T18:34:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:1/0: truncate - no filename 2024-02-15T18:34:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:1/1: dread - no filename 2024-02-15T18:34:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:1/2: truncate - no filename 2024-02-15T18:34:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:1/3: fdatasync - no filename 2024-02-15T18:34:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:1/4: truncate - no filename 2024-02-15T18:34:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:1/5: dread - no filename 2024-02-15T18:34:46.878 INFO:tasks.workunit.client.0.smithi038.stdout:1/6: stat - no entries 2024-02-15T18:34:46.879 INFO:tasks.workunit.client.0.smithi038.stdout:1/7: readlink - no filename 2024-02-15T18:34:46.879 INFO:tasks.workunit.client.0.smithi038.stdout:0/6: creat f0 x:0 0 0 2024-02-15T18:34:46.879 INFO:tasks.workunit.client.0.smithi038.stdout:0/7: fdatasync f0 0 2024-02-15T18:34:46.881 INFO:tasks.workunit.client.0.smithi038.stdout:1/8: creat f0 x:0 0 0 2024-02-15T18:34:46.882 INFO:tasks.workunit.client.0.smithi038.stdout:3/0: chown . 14464963 1 2024-02-15T18:34:46.882 INFO:tasks.workunit.client.0.smithi038.stdout:3/1: read - no filename 2024-02-15T18:34:46.883 INFO:tasks.workunit.client.0.smithi038.stdout:2/0: mknod c0 0 2024-02-15T18:34:46.884 INFO:tasks.workunit.client.0.smithi038.stdout:1/9: rename f0 to f1 0 2024-02-15T18:34:46.884 INFO:tasks.workunit.client.0.smithi038.stdout:1/10: readlink - no filename 2024-02-15T18:34:46.884 INFO:tasks.workunit.client.0.smithi038.stdout:1/11: dread - f1 zero size 2024-02-15T18:34:46.885 INFO:tasks.workunit.client.0.smithi038.stdout:3/2: symlink l0 0 2024-02-15T18:34:46.886 INFO:tasks.workunit.client.0.smithi038.stdout:2/1: mknod c1 0 2024-02-15T18:34:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:5/0: dread - no filename 2024-02-15T18:34:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:5/1: chown . 60 1 2024-02-15T18:34:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:5/2: link - no file 2024-02-15T18:34:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:5/3: write - no filename 2024-02-15T18:34:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:5/4: chown . 10826 1 2024-02-15T18:34:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:5/5: dread - no filename 2024-02-15T18:34:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:5/6: readlink - no filename 2024-02-15T18:34:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:5/7: write - no filename 2024-02-15T18:34:46.887 INFO:tasks.workunit.client.0.smithi038.stdout:5/8: link - no file 2024-02-15T18:34:46.888 INFO:tasks.workunit.client.0.smithi038.stdout:4/0: symlink l0 0 2024-02-15T18:34:46.888 INFO:tasks.workunit.client.0.smithi038.stdout:3/3: mknod c1 0 2024-02-15T18:34:46.888 INFO:tasks.workunit.client.0.smithi038.stdout:3/4: dwrite - no filename 2024-02-15T18:34:46.889 INFO:tasks.workunit.client.0.smithi038.stdout:2/2: mknod c2 0 2024-02-15T18:34:46.889 INFO:tasks.workunit.client.0.smithi038.stdout:2/3: write - no filename 2024-02-15T18:34:46.894 INFO:tasks.workunit.client.0.smithi038.stdout:1/12: fsync f1 0 2024-02-15T18:34:46.894 INFO:tasks.workunit.client.0.smithi038.stdout:1/13: write f1 [554210,31069] 0 2024-02-15T18:34:46.895 INFO:tasks.workunit.client.0.smithi038.stdout:6/0: rename - no filename 2024-02-15T18:34:46.896 INFO:tasks.workunit.client.0.smithi038.stdout:5/9: mkdir d0 0 2024-02-15T18:34:46.896 INFO:tasks.workunit.client.0.smithi038.stdout:5/10: dwrite - no filename 2024-02-15T18:34:46.896 INFO:tasks.workunit.client.0.smithi038.stdout:5/11: dread - no filename 2024-02-15T18:34:46.897 INFO:tasks.workunit.client.0.smithi038.stdout:4/1: creat f1 x:0 0 0 2024-02-15T18:34:46.897 INFO:tasks.workunit.client.0.smithi038.stdout:4/2: write f1 [959976,127649] 0 2024-02-15T18:34:46.897 INFO:tasks.workunit.client.0.smithi038.stdout:3/5: mkdir d2 0 2024-02-15T18:34:46.898 INFO:tasks.workunit.client.0.smithi038.stdout:2/4: mknod c3 0 2024-02-15T18:34:46.898 INFO:tasks.workunit.client.0.smithi038.stdout:1/14: mkdir d2 0 2024-02-15T18:34:46.899 INFO:tasks.workunit.client.0.smithi038.stdout:1/15: write f1 [757311,17423] 0 2024-02-15T18:34:46.899 INFO:tasks.workunit.client.0.smithi038.stdout:1/16: write f1 [801750,32213] 0 2024-02-15T18:34:46.899 INFO:tasks.workunit.client.0.smithi038.stdout:7/0: truncate - no filename 2024-02-15T18:34:46.899 INFO:tasks.workunit.client.0.smithi038.stdout:6/1: creat f0 x:0 0 0 2024-02-15T18:34:46.899 INFO:tasks.workunit.client.0.smithi038.stdout:6/2: truncate f0 249246 0 2024-02-15T18:34:46.900 INFO:tasks.workunit.client.0.smithi038.stdout:6/3: chown f0 47609 1 2024-02-15T18:34:46.900 INFO:tasks.workunit.client.0.smithi038.stdout:5/12: rmdir d0 0 2024-02-15T18:34:46.900 INFO:tasks.workunit.client.0.smithi038.stdout:5/13: dread - no filename 2024-02-15T18:34:46.901 INFO:tasks.workunit.client.0.smithi038.stdout:5/14: dwrite - no filename 2024-02-15T18:34:46.901 INFO:tasks.workunit.client.0.smithi038.stdout:6/4: dread f0 [0,4194304] 0 2024-02-15T18:34:46.901 INFO:tasks.workunit.client.0.smithi038.stdout:4/3: creat f2 x:0 0 0 2024-02-15T18:34:46.901 INFO:tasks.workunit.client.0.smithi038.stdout:6/5: dread f0 [0,4194304] 0 2024-02-15T18:34:46.901 INFO:tasks.workunit.client.0.smithi038.stdout:3/6: symlink d2/l3 0 2024-02-15T18:34:46.901 INFO:tasks.workunit.client.0.smithi038.stdout:3/7: dwrite - no filename 2024-02-15T18:34:46.902 INFO:tasks.workunit.client.0.smithi038.stdout:6/6: dread f0 [0,4194304] 0 2024-02-15T18:34:46.902 INFO:tasks.workunit.client.0.smithi038.stdout:1/17: creat d2/f3 x:0 0 0 2024-02-15T18:34:46.904 INFO:tasks.workunit.client.0.smithi038.stdout:7/1: mkdir d0 0 2024-02-15T18:34:46.904 INFO:tasks.workunit.client.0.smithi038.stdout:7/2: unlink - no file 2024-02-15T18:34:46.905 INFO:tasks.workunit.client.0.smithi038.stdout:5/15: creat f1 x:0 0 0 2024-02-15T18:34:46.905 INFO:tasks.workunit.client.0.smithi038.stdout:5/16: write f1 [283665,124949] 0 2024-02-15T18:34:46.908 INFO:tasks.workunit.client.0.smithi038.stdout:2/5: getdents . 0 2024-02-15T18:34:46.908 INFO:tasks.workunit.client.0.smithi038.stdout:2/6: dwrite - no filename 2024-02-15T18:34:46.908 INFO:tasks.workunit.client.0.smithi038.stdout:2/7: dread - no filename 2024-02-15T18:34:46.908 INFO:tasks.workunit.client.0.smithi038.stdout:2/8: fdatasync - no filename 2024-02-15T18:34:46.911 INFO:tasks.workunit.client.0.smithi038.stdout:1/18: link f1 d2/f4 0 2024-02-15T18:34:46.911 INFO:tasks.workunit.client.0.smithi038.stdout:1/19: truncate d2/f4 1876596 0 2024-02-15T18:34:46.912 INFO:tasks.workunit.client.0.smithi038.stdout:0/8: dwrite f0 [0,4194304] 0 2024-02-15T18:34:46.912 INFO:tasks.workunit.client.0.smithi038.stdout:0/9: stat f0 0 2024-02-15T18:34:46.916 INFO:tasks.workunit.client.0.smithi038.stdout:8/0: symlink l0 0 2024-02-15T18:34:46.916 INFO:tasks.workunit.client.0.smithi038.stdout:7/3: creat d0/f1 x:0 0 0 2024-02-15T18:34:46.916 INFO:tasks.workunit.client.0.smithi038.stdout:2/9: creat f4 x:0 0 0 2024-02-15T18:34:46.916 INFO:tasks.workunit.client.0.smithi038.stdout:7/4: write d0/f1 [855662,46477] 0 2024-02-15T18:34:46.916 INFO:tasks.workunit.client.0.smithi038.stdout:5/17: mkdir d2 0 2024-02-15T18:34:46.923 INFO:tasks.workunit.client.0.smithi038.stdout:4/4: rename f1 to f3 0 2024-02-15T18:34:46.923 INFO:tasks.workunit.client.0.smithi038.stdout:4/5: stat f3 0 2024-02-15T18:34:46.933 INFO:tasks.workunit.client.0.smithi038.stdout:1/20: symlink d2/l5 0 2024-02-15T18:34:46.933 INFO:tasks.workunit.client.0.smithi038.stdout:1/21: write d2/f3 [906180,80249] 0 2024-02-15T18:34:46.938 INFO:tasks.workunit.client.0.smithi038.stdout:4/6: creat f4 x:0 0 0 2024-02-15T18:34:46.938 INFO:tasks.workunit.client.0.smithi038.stdout:8/1: mkdir d1 0 2024-02-15T18:34:46.942 INFO:tasks.workunit.client.0.smithi038.stdout:1/22: symlink d2/l6 0 2024-02-15T18:34:46.942 INFO:tasks.workunit.client.0.smithi038.stdout:1/23: chown d2/l6 2014 1 2024-02-15T18:34:46.942 INFO:tasks.workunit.client.0.smithi038.stdout:8/2: creat d1/f2 x:0 0 0 2024-02-15T18:34:46.942 INFO:tasks.workunit.client.0.smithi038.stdout:8/3: dread - d1/f2 zero size 2024-02-15T18:34:46.951 INFO:tasks.workunit.client.0.smithi038.stdout:1/24: mknod d2/c7 0 2024-02-15T18:34:46.954 INFO:tasks.workunit.client.0.smithi038.stdout:2/10: mknod c5 0 2024-02-15T18:34:46.954 INFO:tasks.workunit.client.0.smithi038.stdout:2/11: write f4 [677598,11406] 0 2024-02-15T18:34:47.110 INFO:tasks.workunit.client.0.smithi038.stdout:6/7: dwrite f0 [0,4194304] 0 2024-02-15T18:34:47.128 INFO:tasks.workunit.client.0.smithi038.stdout:6/8: creat f1 x:0 0 0 2024-02-15T18:34:47.128 INFO:tasks.workunit.client.0.smithi038.stdout:6/9: read - f1 zero size 2024-02-15T18:34:47.153 INFO:tasks.workunit.client.0.smithi038.stdout:6/10: mkdir d2 0 2024-02-15T18:34:47.153 INFO:tasks.workunit.client.0.smithi038.stdout:6/11: truncate f1 26179 0 2024-02-15T18:34:47.387 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:46.315645+0000 mgr.smithi038.khesbl (mgr.14162) 333 : cluster [DBG] pgmap v275: 65 pgs: 65 active+clean; 231 MiB data, 855 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 14 MiB/s wr, 938 op/s 2024-02-15T18:34:47.456 INFO:tasks.workunit.client.0.smithi038.stdout:6/12: dread f1 [0,4194304] 0 2024-02-15T18:34:47.561 INFO:tasks.workunit.client.0.smithi038.stdout:0/10: dwrite f0 [0,4194304] 0 2024-02-15T18:34:47.561 INFO:tasks.workunit.client.0.smithi038.stdout:0/11: rmdir - no directory 2024-02-15T18:34:47.620 INFO:tasks.workunit.client.0.smithi038.stdout:7/5: read d0/f1 [335070,92913] 0 2024-02-15T18:34:47.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:46.315645+0000 mgr.smithi038.khesbl (mgr.14162) 333 : cluster [DBG] pgmap v275: 65 pgs: 65 active+clean; 231 MiB data, 855 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 14 MiB/s wr, 938 op/s 2024-02-15T18:34:47.790 INFO:tasks.workunit.client.0.smithi038.stdout:0/12: dwrite f0 [0,4194304] 0 2024-02-15T18:34:47.790 INFO:tasks.workunit.client.0.smithi038.stdout:0/13: readlink - no filename 2024-02-15T18:34:47.841 INFO:tasks.workunit.client.0.smithi038.stdout:0/14: dread f0 [0,4194304] 0 2024-02-15T18:34:47.841 INFO:tasks.workunit.client.0.smithi038.stdout:0/15: fsync f0 0 2024-02-15T18:34:47.841 INFO:tasks.workunit.client.0.smithi038.stdout:0/16: fdatasync f0 0 2024-02-15T18:34:47.842 INFO:tasks.workunit.client.0.smithi038.stdout:5/18: write f1 [141302,46260] 0 2024-02-15T18:34:47.849 INFO:tasks.workunit.client.0.smithi038.stdout:0/17: dread f0 [0,4194304] 0 2024-02-15T18:34:47.983 INFO:tasks.workunit.client.0.smithi038.stdout:4/7: write f3 [429341,127244] 0 2024-02-15T18:34:48.139 INFO:tasks.workunit.client.0.smithi038.stdout:2/12: dwrite f4 [0,4194304] 0 2024-02-15T18:34:48.163 INFO:tasks.workunit.client.0.smithi038.stdout:2/13: dread f4 [0,4194304] 0 2024-02-15T18:34:48.168 INFO:tasks.workunit.client.0.smithi038.stdout:2/14: dread f4 [0,4194304] 0 2024-02-15T18:34:48.173 INFO:tasks.workunit.client.0.smithi038.stdout:1/25: rmdir d2 39 2024-02-15T18:34:48.231 INFO:tasks.workunit.client.0.smithi038.stdout:7/6: dwrite d0/f1 [0,4194304] 0 2024-02-15T18:34:48.250 INFO:tasks.workunit.client.0.smithi038.stdout:7/7: dwrite d0/f1 [0,4194304] 0 2024-02-15T18:34:48.512 INFO:tasks.workunit.client.0.smithi038.stdout:6/13: creat d2/f3 x:0 0 0 2024-02-15T18:34:48.512 INFO:tasks.workunit.client.0.smithi038.stdout:6/14: truncate d2/f3 689741 0 2024-02-15T18:34:48.512 INFO:tasks.workunit.client.0.smithi038.stdout:6/15: write f1 [926440,67252] 0 2024-02-15T18:34:48.512 INFO:tasks.workunit.client.0.smithi038.stdout:6/16: fdatasync f0 0 2024-02-15T18:34:48.513 INFO:tasks.workunit.client.0.smithi038.stdout:6/17: fdatasync d2/f3 0 2024-02-15T18:34:48.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:47.602186+0000 mon.smithi038 (mon.0) 569 : cluster [DBG] Standby manager daemon smithi121.ckiryi restarted 2024-02-15T18:34:48.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:47.602336+0000 mon.smithi038 (mon.0) 570 : cluster [DBG] Standby manager daemon smithi121.ckiryi started 2024-02-15T18:34:48.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:47.618993+0000 mon.smithi121 (mon.1) 20 : audit [DBG] from='mgr.? 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi121.ckiryi/crt"}]: dispatch 2024-02-15T18:34:48.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:47.622392+0000 mon.smithi121 (mon.1) 21 : audit [DBG] from='mgr.? 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:34:48.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:47.628051+0000 mon.smithi121 (mon.1) 22 : audit [DBG] from='mgr.? 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi121.ckiryi/key"}]: dispatch 2024-02-15T18:34:48.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:34:47.629093+0000 mon.smithi121 (mon.1) 23 : audit [DBG] from='mgr.? 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:34:48.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:47.602186+0000 mon.smithi038 (mon.0) 569 : cluster [DBG] Standby manager daemon smithi121.ckiryi restarted 2024-02-15T18:34:48.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:47.602336+0000 mon.smithi038 (mon.0) 570 : cluster [DBG] Standby manager daemon smithi121.ckiryi started 2024-02-15T18:34:48.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:47.618993+0000 mon.smithi121 (mon.1) 20 : audit [DBG] from='mgr.? 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi121.ckiryi/crt"}]: dispatch 2024-02-15T18:34:48.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:47.622392+0000 mon.smithi121 (mon.1) 21 : audit [DBG] from='mgr.? 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:34:48.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:47.628051+0000 mon.smithi121 (mon.1) 22 : audit [DBG] from='mgr.? 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi121.ckiryi/key"}]: dispatch 2024-02-15T18:34:48.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:34:47.629093+0000 mon.smithi121 (mon.1) 23 : audit [DBG] from='mgr.? 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:34:49.575 INFO:tasks.workunit.client.0.smithi038.stdout:5/19: mkdir d2/d3 0 2024-02-15T18:34:49.575 INFO:tasks.workunit.client.0.smithi038.stdout:5/20: chown d2 483 1 2024-02-15T18:34:49.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:48.315982+0000 mgr.smithi038.khesbl (mgr.14162) 334 : cluster [DBG] pgmap v276: 65 pgs: 65 active+clean; 250 MiB data, 971 MiB used, 535 GiB / 536 GiB avail; 2.5 MiB/s rd, 14 MiB/s wr, 926 op/s 2024-02-15T18:34:49.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:48.405583+0000 mon.smithi038 (mon.0) 571 : cluster [DBG] mgrmap e19: smithi038.khesbl(active, since 8m), standbys: smithi121.ckiryi 2024-02-15T18:34:49.774 INFO:tasks.workunit.client.0.smithi038.stdout:0/18: symlink l1 0 2024-02-15T18:34:49.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:48.315982+0000 mgr.smithi038.khesbl (mgr.14162) 334 : cluster [DBG] pgmap v276: 65 pgs: 65 active+clean; 250 MiB data, 971 MiB used, 535 GiB / 536 GiB avail; 2.5 MiB/s rd, 14 MiB/s wr, 926 op/s 2024-02-15T18:34:49.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:48.405583+0000 mon.smithi038 (mon.0) 571 : cluster [DBG] mgrmap e19: smithi038.khesbl(active, since 8m), standbys: smithi121.ckiryi 2024-02-15T18:34:50.154 INFO:tasks.workunit.client.0.smithi038.stdout:4/8: creat f5 x:0 0 0 2024-02-15T18:34:50.155 INFO:tasks.workunit.client.0.smithi038.stdout:4/9: truncate f4 101948 0 2024-02-15T18:34:50.155 INFO:tasks.workunit.client.0.smithi038.stdout:4/10: truncate f2 911500 0 2024-02-15T18:34:50.338 INFO:tasks.workunit.client.0.smithi038.stdout:1/26: symlink d2/l8 0 2024-02-15T18:34:50.338 INFO:tasks.workunit.client.0.smithi038.stdout:1/27: write d2/f3 [1234053,82296] 0 2024-02-15T18:34:50.341 INFO:tasks.workunit.client.0.smithi038.stdout:1/28: dread d2/f4 [0,4194304] 0 2024-02-15T18:34:50.341 INFO:tasks.workunit.client.0.smithi038.stdout:1/29: readlink d2/l5 0 2024-02-15T18:34:50.343 INFO:tasks.workunit.client.0.smithi038.stdout:1/30: dread d2/f4 [0,4194304] 0 2024-02-15T18:34:50.422 INFO:tasks.workunit.client.0.smithi038.stdout:7/8: mknod d0/c2 0 2024-02-15T18:34:50.423 INFO:tasks.workunit.client.0.smithi038.stdout:6/18: mkdir d2/d4 0 2024-02-15T18:34:50.424 INFO:tasks.workunit.client.0.smithi038.stdout:6/19: write f1 [898506,125200] 0 2024-02-15T18:34:50.425 INFO:tasks.workunit.client.0.smithi038.stdout:5/21: unlink f1 0 2024-02-15T18:34:50.425 INFO:tasks.workunit.client.0.smithi038.stdout:5/22: dwrite - no filename 2024-02-15T18:34:50.425 INFO:tasks.workunit.client.0.smithi038.stdout:5/23: write - no filename 2024-02-15T18:34:50.425 INFO:tasks.workunit.client.0.smithi038.stdout:0/19: creat f2 x:0 0 0 2024-02-15T18:34:50.426 INFO:tasks.workunit.client.0.smithi038.stdout:4/11: symlink l6 0 2024-02-15T18:34:50.442 INFO:tasks.workunit.client.0.smithi038.stdout:6/20: dwrite f0 [0,4194304] 0 2024-02-15T18:34:50.443 INFO:tasks.workunit.client.0.smithi038.stdout:6/21: write d2/f3 [424075,90040] 0 2024-02-15T18:34:50.450 INFO:tasks.workunit.client.0.smithi038.stdout:6/22: dread f0 [0,4194304] 0 2024-02-15T18:34:50.985 INFO:tasks.workunit.client.0.smithi038.stdout:1/31: symlink d2/l9 0 2024-02-15T18:34:50.985 INFO:tasks.workunit.client.0.smithi038.stdout:7/9: mkdir d0/d3 0 2024-02-15T18:34:50.986 INFO:tasks.workunit.client.0.smithi038.stdout:5/24: mkdir d2/d4 0 2024-02-15T18:34:50.987 INFO:tasks.workunit.client.0.smithi038.stdout:0/20: symlink l3 0 2024-02-15T18:34:50.987 INFO:tasks.workunit.client.0.smithi038.stdout:4/12: link f5 f7 0 2024-02-15T18:34:50.988 INFO:tasks.workunit.client.0.smithi038.stdout:6/23: symlink d2/l5 0 2024-02-15T18:34:50.992 INFO:tasks.workunit.client.0.smithi038.stdout:6/24: dread f0 [0,4194304] 0 2024-02-15T18:34:50.992 INFO:tasks.workunit.client.0.smithi038.stdout:2/15: rename c2 to c6 0 2024-02-15T18:34:50.992 INFO:tasks.workunit.client.0.smithi038.stdout:2/16: readlink - no filename 2024-02-15T18:34:50.994 INFO:tasks.workunit.client.0.smithi038.stdout:0/21: dread f0 [0,4194304] 0 2024-02-15T18:34:50.994 INFO:tasks.workunit.client.0.smithi038.stdout:0/22: fdatasync f0 0 2024-02-15T18:34:50.994 INFO:tasks.workunit.client.0.smithi038.stdout:0/23: chown f0 897249501 1 2024-02-15T18:34:50.994 INFO:tasks.workunit.client.0.smithi038.stdout:7/10: creat d0/d3/f4 x:0 0 0 2024-02-15T18:34:50.994 INFO:tasks.workunit.client.0.smithi038.stdout:7/11: write d0/d3/f4 [589119,113209] 0 2024-02-15T18:34:50.995 INFO:tasks.workunit.client.0.smithi038.stdout:5/25: creat d2/d3/f5 x:0 0 0 2024-02-15T18:34:50.995 INFO:tasks.workunit.client.0.smithi038.stdout:5/26: dread - d2/d3/f5 zero size 2024-02-15T18:34:50.995 INFO:tasks.workunit.client.0.smithi038.stdout:5/27: dread - d2/d3/f5 zero size 2024-02-15T18:34:50.995 INFO:tasks.workunit.client.0.smithi038.stdout:5/28: read - d2/d3/f5 zero size 2024-02-15T18:34:50.995 INFO:tasks.workunit.client.0.smithi038.stdout:4/13: symlink l8 0 2024-02-15T18:34:50.997 INFO:tasks.workunit.client.0.smithi038.stdout:6/25: rename d2/l5 to d2/d4/l6 0 2024-02-15T18:34:50.998 INFO:tasks.workunit.client.0.smithi038.stdout:7/12: symlink d0/d3/l5 0 2024-02-15T18:34:50.998 INFO:tasks.workunit.client.0.smithi038.stdout:7/13: chown d0/c2 1489941430 1 2024-02-15T18:34:50.998 INFO:tasks.workunit.client.0.smithi038.stdout:2/17: dread f4 [0,4194304] 0 2024-02-15T18:34:50.999 INFO:tasks.workunit.client.0.smithi038.stdout:2/18: write f4 [1519286,51362] 0 2024-02-15T18:34:51.000 INFO:tasks.workunit.client.0.smithi038.stdout:5/29: creat d2/f6 x:0 0 0 2024-02-15T18:34:51.000 INFO:tasks.workunit.client.0.smithi038.stdout:4/14: creat f9 x:0 0 0 2024-02-15T18:34:51.000 INFO:tasks.workunit.client.0.smithi038.stdout:5/30: write d2/d3/f5 [483277,27703] 0 2024-02-15T18:34:51.001 INFO:tasks.workunit.client.0.smithi038.stdout:5/31: write d2/d3/f5 [1060947,41050] 0 2024-02-15T18:34:51.002 INFO:tasks.workunit.client.0.smithi038.stdout:7/14: rename d0/d3/l5 to d0/d3/l6 0 2024-02-15T18:34:51.004 INFO:tasks.workunit.client.0.smithi038.stdout:7/15: write d0/f1 [2551515,71130] 0 2024-02-15T18:34:51.007 INFO:tasks.workunit.client.0.smithi038.stdout:2/19: creat f7 x:0 0 0 2024-02-15T18:34:51.011 INFO:tasks.workunit.client.0.smithi038.stdout:0/24: dwrite f2 [0,4194304] 0 2024-02-15T18:34:51.011 INFO:tasks.workunit.client.0.smithi038.stdout:0/25: fsync f0 0 2024-02-15T18:34:51.012 INFO:tasks.workunit.client.0.smithi038.stdout:4/15: link f3 fa 0 2024-02-15T18:34:51.022 INFO:tasks.workunit.client.0.smithi038.stdout:7/16: creat d0/d3/f7 x:0 0 0 2024-02-15T18:34:51.023 INFO:tasks.workunit.client.0.smithi038.stdout:7/17: readlink d0/d3/l6 0 2024-02-15T18:34:51.023 INFO:tasks.workunit.client.0.smithi038.stdout:7/18: write d0/d3/f7 [261365,81200] 0 2024-02-15T18:34:51.032 INFO:tasks.workunit.client.0.smithi038.stdout:5/32: dwrite d2/d3/f5 [0,4194304] 0 2024-02-15T18:34:51.034 INFO:tasks.workunit.client.0.smithi038.stdout:4/16: dwrite fa [0,4194304] 0 2024-02-15T18:34:51.036 INFO:tasks.workunit.client.0.smithi038.stdout:0/26: mkdir d4 0 2024-02-15T18:34:51.038 INFO:tasks.workunit.client.0.smithi038.stdout:0/27: write f0 [767195,5004] 0 2024-02-15T18:34:51.038 INFO:tasks.workunit.client.0.smithi038.stdout:0/28: readlink l1 0 2024-02-15T18:34:51.045 INFO:tasks.workunit.client.0.smithi038.stdout:7/19: dwrite d0/d3/f7 [0,4194304] 0 2024-02-15T18:34:51.138 INFO:tasks.workunit.client.0.smithi038.stdout:4/17: mknod cb 0 2024-02-15T18:34:51.139 INFO:tasks.workunit.client.0.smithi038.stdout:0/29: rename f2 to d4/f5 0 2024-02-15T18:34:51.147 INFO:tasks.workunit.client.0.smithi038.stdout:2/20: rename c3 to c8 0 2024-02-15T18:34:51.212 INFO:tasks.workunit.client.0.smithi038.stdout:4/18: creat fc x:0 0 0 2024-02-15T18:34:51.213 INFO:tasks.workunit.client.0.smithi038.stdout:4/19: write f2 [825688,66613] 0 2024-02-15T18:34:51.213 INFO:tasks.workunit.client.0.smithi038.stdout:4/20: fdatasync f7 0 2024-02-15T18:34:51.216 INFO:tasks.workunit.client.0.smithi038.stdout:1/32: truncate d2/f3 310575 0 2024-02-15T18:34:51.216 INFO:tasks.workunit.client.0.smithi038.stdout:0/30: symlink d4/l6 0 2024-02-15T18:34:51.218 INFO:tasks.workunit.client.0.smithi038.stdout:5/33: fsync d2/d3/f5 0 2024-02-15T18:34:51.219 INFO:tasks.workunit.client.0.smithi038.stdout:6/26: rename d2/d4/l6 to d2/d4/l7 0 2024-02-15T18:34:51.219 INFO:tasks.workunit.client.0.smithi038.stdout:6/27: truncate d2/f3 980561 0 2024-02-15T18:34:51.219 INFO:tasks.workunit.client.0.smithi038.stdout:4/21: dread f2 [0,4194304] 0 2024-02-15T18:34:51.220 INFO:tasks.workunit.client.0.smithi038.stdout:6/28: write f0 [1598996,22800] 0 2024-02-15T18:34:51.230 INFO:tasks.workunit.client.0.smithi038.stdout:1/33: mknod d2/ca 0 2024-02-15T18:34:51.231 INFO:tasks.workunit.client.0.smithi038.stdout:0/31: creat d4/f7 x:0 0 0 2024-02-15T18:34:51.233 INFO:tasks.workunit.client.0.smithi038.stdout:2/21: symlink l9 0 2024-02-15T18:34:51.235 INFO:tasks.workunit.client.0.smithi038.stdout:5/34: rename d2/d3/f5 to d2/d3/f7 0 2024-02-15T18:34:51.236 INFO:tasks.workunit.client.0.smithi038.stdout:5/35: unlink d2/f6 0 2024-02-15T18:34:51.237 INFO:tasks.workunit.client.0.smithi038.stdout:3/8: sync 2024-02-15T18:34:51.237 INFO:tasks.workunit.client.0.smithi038.stdout:3/9: dread - no filename 2024-02-15T18:34:51.237 INFO:tasks.workunit.client.0.smithi038.stdout:3/10: dread - no filename 2024-02-15T18:34:51.237 INFO:tasks.workunit.client.0.smithi038.stdout:3/11: fsync - no filename 2024-02-15T18:34:51.237 INFO:tasks.workunit.client.0.smithi038.stdout:3/12: dread - no filename 2024-02-15T18:34:51.237 INFO:tasks.workunit.client.0.smithi038.stdout:8/4: sync 2024-02-15T18:34:51.237 INFO:tasks.workunit.client.0.smithi038.stdout:3/13: dwrite - no filename 2024-02-15T18:34:51.237 INFO:tasks.workunit.client.0.smithi038.stdout:9/0: sync 2024-02-15T18:34:51.237 INFO:tasks.workunit.client.0.smithi038.stdout:5/36: mknod d2/d3/c8 0 2024-02-15T18:34:51.239 INFO:tasks.workunit.client.0.smithi038.stdout:3/14: mknod d2/c4 0 2024-02-15T18:34:51.241 INFO:tasks.workunit.client.0.smithi038.stdout:9/1: creat f0 x:0 0 0 2024-02-15T18:34:51.241 INFO:tasks.workunit.client.0.smithi038.stdout:9/2: write f0 [115701,69579] 0 2024-02-15T18:34:51.241 INFO:tasks.workunit.client.0.smithi038.stdout:9/3: write f0 [1129163,89898] 0 2024-02-15T18:34:51.241 INFO:tasks.workunit.client.0.smithi038.stdout:9/4: readlink - no filename 2024-02-15T18:34:51.242 INFO:tasks.workunit.client.0.smithi038.stdout:9/5: mkdir d1 0 2024-02-15T18:34:51.242 INFO:tasks.workunit.client.0.smithi038.stdout:9/6: rename d1 to d1/d2 22 2024-02-15T18:34:51.246 INFO:tasks.workunit.client.0.smithi038.stdout:0/32: dread f0 [0,4194304] 0 2024-02-15T18:34:51.247 INFO:tasks.workunit.client.0.smithi038.stdout:0/33: symlink d4/l8 0 2024-02-15T18:34:51.247 INFO:tasks.workunit.client.0.smithi038.stdout:0/34: truncate d4/f7 465054 0 2024-02-15T18:34:51.247 INFO:tasks.workunit.client.0.smithi038.stdout:0/35: write d4/f7 [845335,14002] 0 2024-02-15T18:34:51.249 INFO:tasks.workunit.client.0.smithi038.stdout:0/36: symlink d4/l9 0 2024-02-15T18:34:51.249 INFO:tasks.workunit.client.0.smithi038.stdout:6/29: dwrite f1 [0,4194304] 0 2024-02-15T18:34:51.250 INFO:tasks.workunit.client.0.smithi038.stdout:0/37: creat d4/fa x:0 0 0 2024-02-15T18:34:51.250 INFO:tasks.workunit.client.0.smithi038.stdout:0/38: dread - d4/fa zero size 2024-02-15T18:34:51.250 INFO:tasks.workunit.client.0.smithi038.stdout:0/39: chown l3 50 1 2024-02-15T18:34:51.252 INFO:tasks.workunit.client.0.smithi038.stdout:6/30: creat d2/f8 x:0 0 0 2024-02-15T18:34:51.253 INFO:tasks.workunit.client.0.smithi038.stdout:4/22: dwrite f2 [0,4194304] 0 2024-02-15T18:34:51.255 INFO:tasks.workunit.client.0.smithi038.stdout:2/22: dwrite f7 [0,4194304] 0 2024-02-15T18:34:51.256 INFO:tasks.workunit.client.0.smithi038.stdout:0/40: symlink d4/lb 0 2024-02-15T18:34:51.256 INFO:tasks.workunit.client.0.smithi038.stdout:7/20: truncate d0/f1 499878 0 2024-02-15T18:34:51.257 INFO:tasks.workunit.client.0.smithi038.stdout:2/23: write f4 [4013122,7976] 0 2024-02-15T18:34:51.257 INFO:tasks.workunit.client.0.smithi038.stdout:6/31: rename f0 to d2/f9 0 2024-02-15T18:34:51.258 INFO:tasks.workunit.client.0.smithi038.stdout:1/34: dwrite d2/f3 [0,4194304] 0 2024-02-15T18:34:51.258 INFO:tasks.workunit.client.0.smithi038.stdout:5/37: dread d2/d3/f7 [0,4194304] 0 2024-02-15T18:34:51.259 INFO:tasks.workunit.client.0.smithi038.stdout:6/32: write f1 [952756,107457] 0 2024-02-15T18:34:51.265 INFO:tasks.workunit.client.0.smithi038.stdout:3/15: getdents d2 0 2024-02-15T18:34:51.265 INFO:tasks.workunit.client.0.smithi038.stdout:3/16: getdents d2 0 2024-02-15T18:34:51.265 INFO:tasks.workunit.client.0.smithi038.stdout:3/17: dwrite - no filename 2024-02-15T18:34:51.265 INFO:tasks.workunit.client.0.smithi038.stdout:3/18: dread - no filename 2024-02-15T18:34:51.265 INFO:tasks.workunit.client.0.smithi038.stdout:3/19: dwrite - no filename 2024-02-15T18:34:51.265 INFO:tasks.workunit.client.0.smithi038.stdout:3/20: dwrite - no filename 2024-02-15T18:34:51.265 INFO:tasks.workunit.client.0.smithi038.stdout:3/21: fdatasync - no filename 2024-02-15T18:34:51.265 INFO:tasks.workunit.client.0.smithi038.stdout:3/22: chown l0 1809753249 1 2024-02-15T18:34:51.265 INFO:tasks.workunit.client.0.smithi038.stdout:4/23: rename l6 to ld 0 2024-02-15T18:34:51.265 INFO:tasks.workunit.client.0.smithi038.stdout:4/24: dread - fc zero size 2024-02-15T18:34:51.266 INFO:tasks.workunit.client.0.smithi038.stdout:4/25: write f4 [940317,23591] 0 2024-02-15T18:34:51.266 INFO:tasks.workunit.client.0.smithi038.stdout:4/26: fdatasync f2 0 2024-02-15T18:34:51.267 INFO:tasks.workunit.client.0.smithi038.stdout:8/5: dwrite d1/f2 [0,4194304] 0 2024-02-15T18:34:51.268 INFO:tasks.workunit.client.0.smithi038.stdout:9/7: truncate f0 558277 0 2024-02-15T18:34:51.271 INFO:tasks.workunit.client.0.smithi038.stdout:8/6: read d1/f2 [64239,97359] 0 2024-02-15T18:34:51.273 INFO:tasks.workunit.client.0.smithi038.stdout:0/41: symlink d4/lc 0 2024-02-15T18:34:51.274 INFO:tasks.workunit.client.0.smithi038.stdout:0/42: write d4/f7 [97368,80428] 0 2024-02-15T18:34:51.277 INFO:tasks.workunit.client.0.smithi038.stdout:1/35: getdents d2 0 2024-02-15T18:34:51.277 INFO:tasks.workunit.client.0.smithi038.stdout:1/36: fdatasync f1 0 2024-02-15T18:34:51.277 INFO:tasks.workunit.client.0.smithi038.stdout:1/37: chown d2 466 1 2024-02-15T18:34:51.279 INFO:tasks.workunit.client.0.smithi038.stdout:6/33: rmdir d2/d4 39 2024-02-15T18:34:51.279 INFO:tasks.workunit.client.0.smithi038.stdout:6/34: readlink d2/d4/l7 0 2024-02-15T18:34:51.279 INFO:tasks.workunit.client.0.smithi038.stdout:4/27: symlink le 0 2024-02-15T18:34:51.285 INFO:tasks.workunit.client.0.smithi038.stdout:9/8: creat d1/f3 x:0 0 0 2024-02-15T18:34:51.285 INFO:tasks.workunit.client.0.smithi038.stdout:9/9: chown d1/f3 1038793 1 2024-02-15T18:34:51.289 INFO:tasks.workunit.client.0.smithi038.stdout:7/21: truncate d0/d3/f4 409891 0 2024-02-15T18:34:51.290 INFO:tasks.workunit.client.0.smithi038.stdout:0/43: symlink d4/ld 0 2024-02-15T18:34:51.291 INFO:tasks.workunit.client.0.smithi038.stdout:1/38: symlink d2/lb 0 2024-02-15T18:34:51.296 INFO:tasks.workunit.client.0.smithi038.stdout:6/35: dwrite f1 [0,4194304] 0 2024-02-15T18:34:51.297 INFO:tasks.workunit.client.0.smithi038.stdout:7/22: mkdir d0/d3/d8 0 2024-02-15T18:34:51.297 INFO:tasks.workunit.client.0.smithi038.stdout:2/24: link c1 ca 0 2024-02-15T18:34:51.298 INFO:tasks.workunit.client.0.smithi038.stdout:0/44: mknod d4/ce 0 2024-02-15T18:34:51.298 INFO:tasks.workunit.client.0.smithi038.stdout:0/45: chown d4/ce 0 1 2024-02-15T18:34:51.298 INFO:tasks.workunit.client.0.smithi038.stdout:0/46: write d4/f5 [5159873,93853] 0 2024-02-15T18:34:51.298 INFO:tasks.workunit.client.0.smithi038.stdout:0/47: chown l3 758 1 2024-02-15T18:34:51.298 INFO:tasks.workunit.client.0.smithi038.stdout:0/48: chown d4/fa 2 1 2024-02-15T18:34:51.298 INFO:tasks.workunit.client.0.smithi038.stdout:9/10: rmdir d1 39 2024-02-15T18:34:51.299 INFO:tasks.workunit.client.0.smithi038.stdout:6/36: dread d2/f3 [0,4194304] 0 2024-02-15T18:34:51.299 INFO:tasks.workunit.client.0.smithi038.stdout:6/37: stat d2/f8 0 2024-02-15T18:34:51.299 INFO:tasks.workunit.client.0.smithi038.stdout:6/38: dread d2/f3 [0,4194304] 0 2024-02-15T18:34:51.300 INFO:tasks.workunit.client.0.smithi038.stdout:7/23: creat d0/f9 x:0 0 0 2024-02-15T18:34:51.301 INFO:tasks.workunit.client.0.smithi038.stdout:0/49: creat d4/ff x:0 0 0 2024-02-15T18:34:51.301 INFO:tasks.workunit.client.0.smithi038.stdout:0/50: fdatasync f0 0 2024-02-15T18:34:51.301 INFO:tasks.workunit.client.0.smithi038.stdout:9/11: dread f0 [0,4194304] 0 2024-02-15T18:34:51.302 INFO:tasks.workunit.client.0.smithi038.stdout:0/51: write d4/f7 [533797,16983] 0 2024-02-15T18:34:51.303 INFO:tasks.workunit.client.0.smithi038.stdout:9/12: dread f0 [0,4194304] 0 2024-02-15T18:34:51.303 INFO:tasks.workunit.client.0.smithi038.stdout:0/52: write d4/f7 [800246,5294] 0 2024-02-15T18:34:51.304 INFO:tasks.workunit.client.0.smithi038.stdout:2/25: rename f4 to fb 0 2024-02-15T18:34:51.305 INFO:tasks.workunit.client.0.smithi038.stdout:6/39: rename d2/f8 to d2/d4/fa 0 2024-02-15T18:34:51.305 INFO:tasks.workunit.client.0.smithi038.stdout:6/40: read - d2/d4/fa zero size 2024-02-15T18:34:51.306 INFO:tasks.workunit.client.0.smithi038.stdout:9/13: symlink d1/l4 0 2024-02-15T18:34:51.306 INFO:tasks.workunit.client.0.smithi038.stdout:2/26: creat fc x:0 0 0 2024-02-15T18:34:51.307 INFO:tasks.workunit.client.0.smithi038.stdout:6/41: mknod d2/cb 0 2024-02-15T18:34:51.308 INFO:tasks.workunit.client.0.smithi038.stdout:9/14: creat d1/f5 x:0 0 0 2024-02-15T18:34:51.308 INFO:tasks.workunit.client.0.smithi038.stdout:0/53: dread f0 [0,4194304] 0 2024-02-15T18:34:51.310 INFO:tasks.workunit.client.0.smithi038.stdout:0/54: mknod d4/c10 0 2024-02-15T18:34:51.310 INFO:tasks.workunit.client.0.smithi038.stdout:0/55: write d4/fa [685797,101737] 0 2024-02-15T18:34:51.310 INFO:tasks.workunit.client.0.smithi038.stdout:0/56: write d4/f5 [4365073,35430] 0 2024-02-15T18:34:51.316 INFO:tasks.workunit.client.0.smithi038.stdout:7/24: write d0/f1 [148426,14338] 0 2024-02-15T18:34:51.316 INFO:tasks.workunit.client.0.smithi038.stdout:8/7: dwrite d1/f2 [4194304,4194304] 0 2024-02-15T18:34:51.316 INFO:tasks.workunit.client.0.smithi038.stdout:1/39: dwrite f1 [0,4194304] 0 2024-02-15T18:34:51.322 INFO:tasks.workunit.client.0.smithi038.stdout:6/42: rmdir d2 39 2024-02-15T18:34:51.324 INFO:tasks.workunit.client.0.smithi038.stdout:7/25: rename d0/d3/f4 to d0/fa 0 2024-02-15T18:34:51.327 INFO:tasks.workunit.client.0.smithi038.stdout:6/43: creat d2/d4/fc x:0 0 0 2024-02-15T18:34:51.335 INFO:tasks.workunit.client.0.smithi038.stdout:2/27: dwrite fb [0,4194304] 0 2024-02-15T18:34:51.343 INFO:tasks.workunit.client.0.smithi038.stdout:2/28: dread f7 [0,4194304] 0 2024-02-15T18:34:51.346 INFO:tasks.workunit.client.0.smithi038.stdout:1/40: dread d2/f3 [0,4194304] 0 2024-02-15T18:34:51.353 INFO:tasks.workunit.client.0.smithi038.stdout:0/57: dwrite d4/f5 [0,4194304] 0 2024-02-15T18:34:51.353 INFO:tasks.workunit.client.0.smithi038.stdout:9/15: dwrite d1/f5 [0,4194304] 0 2024-02-15T18:34:51.353 INFO:tasks.workunit.client.0.smithi038.stdout:6/44: dwrite f1 [0,4194304] 0 2024-02-15T18:34:51.353 INFO:tasks.workunit.client.0.smithi038.stdout:8/8: dwrite d1/f2 [8388608,4194304] 0 2024-02-15T18:34:51.354 INFO:tasks.workunit.client.0.smithi038.stdout:2/29: truncate fb 932146 0 2024-02-15T18:34:51.357 INFO:tasks.workunit.client.0.smithi038.stdout:0/58: creat d4/f11 x:0 0 0 2024-02-15T18:34:51.357 INFO:tasks.workunit.client.0.smithi038.stdout:0/59: chown d4/fa 3 1 2024-02-15T18:34:51.357 INFO:tasks.workunit.client.0.smithi038.stdout:0/60: fdatasync d4/f11 0 2024-02-15T18:34:51.358 INFO:tasks.workunit.client.0.smithi038.stdout:9/16: symlink d1/l6 0 2024-02-15T18:34:51.358 INFO:tasks.workunit.client.0.smithi038.stdout:9/17: write d1/f3 [886825,18501] 0 2024-02-15T18:34:51.358 INFO:tasks.workunit.client.0.smithi038.stdout:8/9: write d1/f2 [10063440,99712] 0 2024-02-15T18:34:51.361 INFO:tasks.workunit.client.0.smithi038.stdout:6/45: mknod d2/cd 0 2024-02-15T18:34:51.363 INFO:tasks.workunit.client.0.smithi038.stdout:0/61: creat d4/f12 x:0 0 0 2024-02-15T18:34:51.364 INFO:tasks.workunit.client.0.smithi038.stdout:9/18: mknod d1/c7 0 2024-02-15T18:34:51.365 INFO:tasks.workunit.client.0.smithi038.stdout:2/30: mkdir dd 0 2024-02-15T18:34:51.366 INFO:tasks.workunit.client.0.smithi038.stdout:9/19: symlink d1/l8 0 2024-02-15T18:34:51.366 INFO:tasks.workunit.client.0.smithi038.stdout:9/20: creat d1/f9 x:0 0 0 2024-02-15T18:34:51.367 INFO:tasks.workunit.client.0.smithi038.stdout:9/21: mkdir d1/da 0 2024-02-15T18:34:51.376 INFO:tasks.workunit.client.0.smithi038.stdout:1/41: dwrite f1 [4194304,4194304] 0 2024-02-15T18:34:51.382 INFO:tasks.workunit.client.0.smithi038.stdout:8/10: dread d1/f2 [8388608,4194304] 0 2024-02-15T18:34:51.382 INFO:tasks.workunit.client.0.smithi038.stdout:2/31: dwrite f7 [0,4194304] 0 2024-02-15T18:34:51.383 INFO:tasks.workunit.client.0.smithi038.stdout:8/11: mknod d1/c3 0 2024-02-15T18:34:51.383 INFO:tasks.workunit.client.0.smithi038.stdout:8/12: readlink l0 0 2024-02-15T18:34:51.384 INFO:tasks.workunit.client.0.smithi038.stdout:9/22: dwrite d1/f3 [0,4194304] 0 2024-02-15T18:34:51.384 INFO:tasks.workunit.client.0.smithi038.stdout:2/32: write f7 [3664311,122423] 0 2024-02-15T18:34:51.384 INFO:tasks.workunit.client.0.smithi038.stdout:2/33: fdatasync fc 0 2024-02-15T18:34:51.384 INFO:tasks.workunit.client.0.smithi038.stdout:0/62: getdents d4 0 2024-02-15T18:34:51.386 INFO:tasks.workunit.client.0.smithi038.stdout:1/42: write f1 [1714740,104191] 0 2024-02-15T18:34:51.388 INFO:tasks.workunit.client.0.smithi038.stdout:8/13: getdents d1 0 2024-02-15T18:34:51.389 INFO:tasks.workunit.client.0.smithi038.stdout:9/23: creat d1/fb x:0 0 0 2024-02-15T18:34:51.390 INFO:tasks.workunit.client.0.smithi038.stdout:0/63: mkdir d4/d13 0 2024-02-15T18:34:51.393 INFO:tasks.workunit.client.0.smithi038.stdout:0/64: truncate d4/f5 3673323 0 2024-02-15T18:34:51.395 INFO:tasks.workunit.client.0.smithi038.stdout:0/65: link d4/lc d4/d13/l14 0 2024-02-15T18:34:51.395 INFO:tasks.workunit.client.0.smithi038.stdout:0/66: fdatasync f0 0 2024-02-15T18:34:51.398 INFO:tasks.workunit.client.0.smithi038.stdout:8/14: dread d1/f2 [0,4194304] 0 2024-02-15T18:34:51.398 INFO:tasks.workunit.client.0.smithi038.stdout:0/67: rename d4/f7 to d4/f15 0 2024-02-15T18:34:51.405 INFO:tasks.workunit.client.0.smithi038.stdout:2/34: dwrite fb [0,4194304] 0 2024-02-15T18:34:51.406 INFO:tasks.workunit.client.0.smithi038.stdout:2/35: creat dd/fe x:0 0 0 2024-02-15T18:34:51.407 INFO:tasks.workunit.client.0.smithi038.stdout:2/36: dread - dd/fe zero size 2024-02-15T18:34:51.407 INFO:tasks.workunit.client.0.smithi038.stdout:0/68: write d4/f5 [1506240,70094] 0 2024-02-15T18:34:51.408 INFO:tasks.workunit.client.0.smithi038.stdout:0/69: creat d4/f16 x:0 0 0 2024-02-15T18:34:51.410 INFO:tasks.workunit.client.0.smithi038.stdout:8/15: dread d1/f2 [4194304,4194304] 0 2024-02-15T18:34:51.410 INFO:tasks.workunit.client.0.smithi038.stdout:0/70: unlink f0 0 2024-02-15T18:34:51.410 INFO:tasks.workunit.client.0.smithi038.stdout:0/71: dread - d4/f16 zero size 2024-02-15T18:34:51.412 INFO:tasks.workunit.client.0.smithi038.stdout:2/37: getdents dd 0 2024-02-15T18:34:51.412 INFO:tasks.workunit.client.0.smithi038.stdout:2/38: chown l9 1044423748 1 2024-02-15T18:34:51.413 INFO:tasks.workunit.client.0.smithi038.stdout:0/72: symlink d4/d13/l17 0 2024-02-15T18:34:51.414 INFO:tasks.workunit.client.0.smithi038.stdout:2/39: symlink dd/lf 0 2024-02-15T18:34:51.415 INFO:tasks.workunit.client.0.smithi038.stdout:8/16: dread d1/f2 [0,4194304] 0 2024-02-15T18:34:51.415 INFO:tasks.workunit.client.0.smithi038.stdout:0/73: truncate d4/ff 313571 0 2024-02-15T18:34:51.416 INFO:tasks.workunit.client.0.smithi038.stdout:2/40: rmdir dd 39 2024-02-15T18:34:51.417 INFO:tasks.workunit.client.0.smithi038.stdout:8/17: mkdir d1/d4 0 2024-02-15T18:34:51.417 INFO:tasks.workunit.client.0.smithi038.stdout:8/18: write d1/f2 [12573564,16752] 0 2024-02-15T18:34:51.418 INFO:tasks.workunit.client.0.smithi038.stdout:8/19: getdents d1/d4 0 2024-02-15T18:34:51.418 INFO:tasks.workunit.client.0.smithi038.stdout:0/74: dread - d4/f12 zero size 2024-02-15T18:34:51.418 INFO:tasks.workunit.client.0.smithi038.stdout:0/75: truncate d4/f16 245361 0 2024-02-15T18:34:51.418 INFO:tasks.workunit.client.0.smithi038.stdout:0/76: read - d4/f11 zero size 2024-02-15T18:34:51.421 INFO:tasks.workunit.client.0.smithi038.stdout:0/77: fsync d4/f16 0 2024-02-15T18:34:51.423 INFO:tasks.workunit.client.0.smithi038.stdout:0/78: stat d4/l8 0 2024-02-15T18:34:51.424 INFO:tasks.workunit.client.0.smithi038.stdout:0/79: write d4/f11 [752099,122288] 0 2024-02-15T18:34:51.424 INFO:tasks.workunit.client.0.smithi038.stdout:0/80: getdents d4/d13 0 2024-02-15T18:34:51.426 INFO:tasks.workunit.client.0.smithi038.stdout:0/81: mknod d4/d13/c18 0 2024-02-15T18:34:51.427 INFO:tasks.workunit.client.0.smithi038.stdout:0/82: dread d4/f16 [0,4194304] 0 2024-02-15T18:34:51.430 INFO:tasks.workunit.client.0.smithi038.stdout:0/83: getdents d4/d13 0 2024-02-15T18:34:51.439 INFO:tasks.workunit.client.0.smithi038.stdout:0/84: dread d4/f5 [0,4194304] 0 2024-02-15T18:34:51.439 INFO:tasks.workunit.client.0.smithi038.stdout:8/20: dwrite d1/f2 [0,4194304] 0 2024-02-15T18:34:51.439 INFO:tasks.workunit.client.0.smithi038.stdout:0/85: stat d4/c10 0 2024-02-15T18:34:51.444 INFO:tasks.workunit.client.0.smithi038.stdout:8/21: dread d1/f2 [4194304,4194304] 0 2024-02-15T18:34:51.447 INFO:tasks.workunit.client.0.smithi038.stdout:8/22: truncate d1/f2 1433921 0 2024-02-15T18:34:51.447 INFO:tasks.workunit.client.0.smithi038.stdout:8/23: fdatasync d1/f2 0 2024-02-15T18:34:51.448 INFO:tasks.workunit.client.0.smithi038.stdout:8/24: mkdir d1/d4/d5 0 2024-02-15T18:34:51.449 INFO:tasks.workunit.client.0.smithi038.stdout:8/25: mkdir d1/d4/d5/d6 0 2024-02-15T18:34:51.454 INFO:tasks.workunit.client.0.smithi038.stdout:8/26: write d1/f2 [86261,28531] 0 2024-02-15T18:34:51.455 INFO:tasks.workunit.client.0.smithi038.stdout:8/27: mkdir d1/d7 0 2024-02-15T18:34:51.471 INFO:tasks.workunit.client.0.smithi038.stdout:8/28: dwrite d1/f2 [0,4194304] 0 2024-02-15T18:34:51.473 INFO:tasks.workunit.client.0.smithi038.stdout:8/29: write d1/f2 [894662,51608] 0 2024-02-15T18:34:51.474 INFO:tasks.workunit.client.0.smithi038.stdout:8/30: mknod d1/d7/c8 0 2024-02-15T18:34:51.474 INFO:tasks.workunit.client.0.smithi038.stdout:8/31: stat d1/d7 0 2024-02-15T18:34:51.475 INFO:tasks.workunit.client.0.smithi038.stdout:8/32: symlink d1/d7/l9 0 2024-02-15T18:34:51.476 INFO:tasks.workunit.client.0.smithi038.stdout:8/33: rename d1/d7/c8 to d1/ca 0 2024-02-15T18:34:51.487 INFO:tasks.workunit.client.0.smithi038.stdout:8/34: dread d1/f2 [0,4194304] 0 2024-02-15T18:34:51.488 INFO:tasks.workunit.client.0.smithi038.stdout:8/35: write d1/f2 [1734585,40781] 0 2024-02-15T18:34:51.489 INFO:tasks.workunit.client.0.smithi038.stdout:8/36: creat d1/fb x:0 0 0 2024-02-15T18:34:51.508 INFO:tasks.workunit.client.0.smithi038.stdout:8/37: dwrite d1/f2 [0,4194304] 0 2024-02-15T18:34:51.510 INFO:tasks.workunit.client.0.smithi038.stdout:8/38: creat d1/d7/fc x:0 0 0 2024-02-15T18:34:51.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:50.316412+0000 mgr.smithi038.khesbl (mgr.14162) 335 : cluster [DBG] pgmap v277: 65 pgs: 65 active+clean; 278 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 16 MiB/s wr, 849 op/s 2024-02-15T18:34:51.526 INFO:tasks.workunit.client.0.smithi038.stdout:8/39: dwrite d1/f2 [0,4194304] 0 2024-02-15T18:34:51.542 INFO:tasks.workunit.client.0.smithi038.stdout:8/40: dwrite d1/d7/fc [0,4194304] 0 2024-02-15T18:34:51.551 INFO:tasks.workunit.client.0.smithi038.stdout:8/41: dread d1/f2 [0,4194304] 0 2024-02-15T18:34:51.551 INFO:tasks.workunit.client.0.smithi038.stdout:8/42: dread - d1/fb zero size 2024-02-15T18:34:51.552 INFO:tasks.workunit.client.0.smithi038.stdout:8/43: mkdir d1/d4/d5/d6/dd 0 2024-02-15T18:34:51.553 INFO:tasks.workunit.client.0.smithi038.stdout:8/44: symlink d1/d4/d5/d6/le 0 2024-02-15T18:34:51.554 INFO:tasks.workunit.client.0.smithi038.stdout:8/45: write d1/d7/fc [107616,9326] 0 2024-02-15T18:34:51.555 INFO:tasks.workunit.client.0.smithi038.stdout:8/46: getdents d1/d4 0 2024-02-15T18:34:51.555 INFO:tasks.workunit.client.0.smithi038.stdout:8/47: chown d1/ca 272433320 1 2024-02-15T18:34:51.555 INFO:tasks.workunit.client.0.smithi038.stdout:8/48: creat d1/d4/d5/d6/ff x:0 0 0 2024-02-15T18:34:51.556 INFO:tasks.workunit.client.0.smithi038.stdout:8/49: dread - d1/d4/d5/d6/ff zero size 2024-02-15T18:34:51.556 INFO:tasks.workunit.client.0.smithi038.stdout:8/50: mkdir d1/d4/d10 0 2024-02-15T18:34:51.561 INFO:tasks.workunit.client.0.smithi038.stdout:8/51: truncate d1/f2 1251455 0 2024-02-15T18:34:51.562 INFO:tasks.workunit.client.0.smithi038.stdout:8/52: rmdir d1/d7 39 2024-02-15T18:34:51.564 INFO:tasks.workunit.client.0.smithi038.stdout:8/53: link d1/d4/d5/d6/le d1/d7/l11 0 2024-02-15T18:34:51.564 INFO:tasks.workunit.client.0.smithi038.stdout:8/54: write d1/fb [343849,41138] 0 2024-02-15T18:34:51.564 INFO:tasks.workunit.client.0.smithi038.stdout:8/55: chown l0 3926 1 2024-02-15T18:34:51.586 INFO:tasks.workunit.client.0.smithi038.stdout:8/56: dwrite d1/d4/d5/d6/ff [0,4194304] 0 2024-02-15T18:34:51.587 INFO:tasks.workunit.client.0.smithi038.stdout:8/57: write d1/d4/d5/d6/ff [264986,4751] 0 2024-02-15T18:34:51.587 INFO:tasks.workunit.client.0.smithi038.stdout:8/58: rmdir d1/d4 39 2024-02-15T18:34:51.587 INFO:tasks.workunit.client.0.smithi038.stdout:8/59: write d1/f2 [1748498,129685] 0 2024-02-15T18:34:51.592 INFO:tasks.workunit.client.0.smithi038.stdout:8/60: dread d1/fb [0,4194304] 0 2024-02-15T18:34:51.605 INFO:tasks.workunit.client.0.smithi038.stdout:8/61: dwrite d1/fb [0,4194304] 0 2024-02-15T18:34:51.609 INFO:tasks.workunit.client.0.smithi038.stdout:8/62: truncate d1/f2 1061310 0 2024-02-15T18:34:51.610 INFO:tasks.workunit.client.0.smithi038.stdout:8/63: mknod d1/d4/d5/d6/dd/c12 0 2024-02-15T18:34:51.611 INFO:tasks.workunit.client.0.smithi038.stdout:8/64: mknod d1/c13 0 2024-02-15T18:34:51.613 INFO:tasks.workunit.client.0.smithi038.stdout:8/65: unlink d1/ca 0 2024-02-15T18:34:51.613 INFO:tasks.workunit.client.0.smithi038.stdout:8/66: readlink d1/d7/l11 0 2024-02-15T18:34:51.615 INFO:tasks.workunit.client.0.smithi038.stdout:8/67: dread d1/f2 [0,4194304] 0 2024-02-15T18:34:51.630 INFO:tasks.workunit.client.0.smithi038.stdout:8/68: dwrite d1/fb [0,4194304] 0 2024-02-15T18:34:51.631 INFO:tasks.workunit.client.0.smithi038.stdout:8/69: read d1/fb [578765,49874] 0 2024-02-15T18:34:51.633 INFO:tasks.workunit.client.0.smithi038.stdout:8/70: rmdir d1 39 2024-02-15T18:34:51.651 INFO:tasks.workunit.client.0.smithi038.stdout:8/71: dwrite d1/d7/fc [0,4194304] 0 2024-02-15T18:34:51.666 INFO:tasks.workunit.client.0.smithi038.stdout:8/72: dwrite d1/d4/d5/d6/ff [4194304,4194304] 0 2024-02-15T18:34:51.667 INFO:tasks.workunit.client.0.smithi038.stdout:8/73: creat d1/d4/d10/f14 x:0 0 0 2024-02-15T18:34:51.668 INFO:tasks.workunit.client.0.smithi038.stdout:8/74: fdatasync d1/d4/d10/f14 0 2024-02-15T18:34:51.669 INFO:tasks.workunit.client.0.smithi038.stdout:8/75: rename d1/d4/d5/d6/dd to d1/d7/d15 0 2024-02-15T18:34:51.673 INFO:tasks.workunit.client.0.smithi038.stdout:8/76: truncate d1/fb 3982913 0 2024-02-15T18:34:51.673 INFO:tasks.workunit.client.0.smithi038.stdout:8/77: rmdir d1/d4 39 2024-02-15T18:34:51.675 INFO:tasks.workunit.client.0.smithi038.stdout:8/78: unlink d1/d4/d10/f14 0 2024-02-15T18:34:51.676 INFO:tasks.workunit.client.0.smithi038.stdout:8/79: getdents d1/d7 0 2024-02-15T18:34:51.676 INFO:tasks.workunit.client.0.smithi038.stdout:8/80: chown d1/d7/d15 25227054 1 2024-02-15T18:34:51.676 INFO:tasks.workunit.client.0.smithi038.stdout:8/81: truncate d1/f2 1374986 0 2024-02-15T18:34:51.676 INFO:tasks.workunit.client.0.smithi038.stdout:8/82: rmdir d1 39 2024-02-15T18:34:51.677 INFO:tasks.workunit.client.0.smithi038.stdout:8/83: symlink d1/d7/d15/l16 0 2024-02-15T18:34:51.686 INFO:tasks.workunit.client.0.smithi038.stdout:8/84: dread d1/d7/fc [0,4194304] 0 2024-02-15T18:34:51.686 INFO:tasks.workunit.client.0.smithi038.stdout:8/85: getdents d1/d7 0 2024-02-15T18:34:51.686 INFO:tasks.workunit.client.0.smithi038.stdout:8/86: stat d1/d7/fc 0 2024-02-15T18:34:51.687 INFO:tasks.workunit.client.0.smithi038.stdout:8/87: link d1/d4/d5/d6/le d1/d4/l17 0 2024-02-15T18:34:51.688 INFO:tasks.workunit.client.0.smithi038.stdout:8/88: creat d1/d4/f18 x:0 0 0 2024-02-15T18:34:51.688 INFO:tasks.workunit.client.0.smithi038.stdout:8/89: symlink d1/l19 0 2024-02-15T18:34:51.689 INFO:tasks.workunit.client.0.smithi038.stdout:8/90: mknod d1/d7/d15/c1a 0 2024-02-15T18:34:51.689 INFO:tasks.workunit.client.0.smithi038.stdout:8/91: dread - d1/d4/f18 zero size 2024-02-15T18:34:51.705 INFO:tasks.workunit.client.0.smithi038.stdout:8/92: dwrite d1/f2 [0,4194304] 0 2024-02-15T18:34:51.705 INFO:tasks.workunit.client.0.smithi038.stdout:8/93: write d1/d7/fc [4182191,33489] 0 2024-02-15T18:34:51.711 INFO:tasks.workunit.client.0.smithi038.stdout:8/94: creat d1/f1b x:0 0 0 2024-02-15T18:34:51.712 INFO:tasks.workunit.client.0.smithi038.stdout:8/95: symlink d1/d7/l1c 0 2024-02-15T18:34:51.712 INFO:tasks.workunit.client.0.smithi038.stdout:8/96: dread - d1/d4/f18 zero size 2024-02-15T18:34:51.712 INFO:tasks.workunit.client.0.smithi038.stdout:8/97: chown d1/d4/d5 119880110 1 2024-02-15T18:34:51.715 INFO:tasks.workunit.client.0.smithi038.stdout:8/98: creat d1/d7/f1d x:0 0 0 2024-02-15T18:34:51.715 INFO:tasks.workunit.client.0.smithi038.stdout:8/99: chown d1/d4/d5/d6/ff 761 1 2024-02-15T18:34:51.715 INFO:tasks.workunit.client.0.smithi038.stdout:8/100: truncate d1/d4/f18 232706 0 2024-02-15T18:34:51.716 INFO:tasks.workunit.client.0.smithi038.stdout:8/101: creat d1/d4/d10/f1e x:0 0 0 2024-02-15T18:34:51.716 INFO:tasks.workunit.client.0.smithi038.stdout:8/102: truncate d1/d4/d10/f1e 341008 0 2024-02-15T18:34:51.721 INFO:tasks.workunit.client.0.smithi038.stdout:8/103: dread d1/fb [0,4194304] 0 2024-02-15T18:34:51.722 INFO:tasks.workunit.client.0.smithi038.stdout:8/104: mknod d1/d7/c1f 0 2024-02-15T18:34:51.739 INFO:tasks.workunit.client.0.smithi038.stdout:8/105: dwrite d1/f2 [4194304,4194304] 0 2024-02-15T18:34:51.739 INFO:tasks.workunit.client.0.smithi038.stdout:8/106: write d1/fb [4613039,66093] 0 2024-02-15T18:34:51.740 INFO:tasks.workunit.client.0.smithi038.stdout:8/107: creat d1/d4/d5/f20 x:0 0 0 2024-02-15T18:34:51.753 INFO:tasks.workunit.client.0.smithi038.stdout:8/108: dwrite d1/fb [0,4194304] 0 2024-02-15T18:34:51.755 INFO:tasks.workunit.client.0.smithi038.stdout:8/109: link d1/d7/c1f d1/d7/d15/c21 0 2024-02-15T18:34:51.755 INFO:tasks.workunit.client.0.smithi038.stdout:8/110: chown d1/d4/d10/f1e 47132 1 2024-02-15T18:34:51.755 INFO:tasks.workunit.client.0.smithi038.stdout:8/111: truncate d1/d4/d5/f20 884712 0 2024-02-15T18:34:51.756 INFO:tasks.workunit.client.0.smithi038.stdout:8/112: write d1/d4/d5/d6/ff [8236205,102765] 0 2024-02-15T18:34:51.757 INFO:tasks.workunit.client.0.smithi038.stdout:8/113: mkdir d1/d7/d22 0 2024-02-15T18:34:51.758 INFO:tasks.workunit.client.0.smithi038.stdout:8/114: creat d1/d4/d5/f23 x:0 0 0 2024-02-15T18:34:51.759 INFO:tasks.workunit.client.0.smithi038.stdout:8/115: write d1/d7/fc [2337149,75516] 0 2024-02-15T18:34:51.759 INFO:tasks.workunit.client.0.smithi038.stdout:8/116: dread - d1/f1b zero size 2024-02-15T18:34:51.760 INFO:tasks.workunit.client.0.smithi038.stdout:8/117: symlink d1/d7/l24 0 2024-02-15T18:34:51.760 INFO:tasks.workunit.client.0.smithi038.stdout:8/118: mknod d1/c25 0 2024-02-15T18:34:51.761 INFO:tasks.workunit.client.0.smithi038.stdout:8/119: mkdir d1/d7/d26 0 2024-02-15T18:34:51.762 INFO:tasks.workunit.client.0.smithi038.stdout:8/120: chown d1/d4/d5/d6/le 1256 1 2024-02-15T18:34:51.762 INFO:tasks.workunit.client.0.smithi038.stdout:8/121: creat d1/d4/d5/f27 x:0 0 0 2024-02-15T18:34:51.763 INFO:tasks.workunit.client.0.smithi038.stdout:8/122: mknod d1/d4/d5/d6/c28 0 2024-02-15T18:34:51.763 INFO:tasks.workunit.client.0.smithi038.stdout:8/123: dread - d1/d4/d5/f23 zero size 2024-02-15T18:34:51.764 INFO:tasks.workunit.client.0.smithi038.stdout:8/124: symlink d1/d7/d26/l29 0 2024-02-15T18:34:51.764 INFO:tasks.workunit.client.0.smithi038.stdout:8/125: write d1/f1b [166981,123245] 0 2024-02-15T18:34:51.776 INFO:tasks.workunit.client.0.smithi038.stdout:5/38: sync 2024-02-15T18:34:51.776 INFO:tasks.workunit.client.0.smithi038.stdout:3/23: sync 2024-02-15T18:34:51.777 INFO:tasks.workunit.client.0.smithi038.stdout:5/39: symlink d2/d3/l9 0 2024-02-15T18:34:51.778 INFO:tasks.workunit.client.0.smithi038.stdout:3/24: creat d2/f5 x:0 0 0 2024-02-15T18:34:51.785 INFO:tasks.workunit.client.0.smithi038.stdout:8/126: dwrite d1/d4/d10/f1e [0,4194304] 0 2024-02-15T18:34:51.790 INFO:tasks.workunit.client.0.smithi038.stdout:8/127: creat d1/d4/d5/d6/f2a x:0 0 0 2024-02-15T18:34:51.791 INFO:tasks.workunit.client.0.smithi038.stdout:8/128: mknod d1/d7/d15/c2b 0 2024-02-15T18:34:51.805 INFO:tasks.workunit.client.0.smithi038.stdout:5/40: dwrite d2/d3/f7 [0,4194304] 0 2024-02-15T18:34:51.806 INFO:tasks.workunit.client.0.smithi038.stdout:3/25: dwrite d2/f5 [0,4194304] 0 2024-02-15T18:34:51.814 INFO:tasks.workunit.client.0.smithi038.stdout:8/129: dread d1/d4/d10/f1e [0,4194304] 0 2024-02-15T18:34:51.814 INFO:tasks.workunit.client.0.smithi038.stdout:8/130: dread - d1/d7/f1d zero size 2024-02-15T18:34:51.814 INFO:tasks.workunit.client.0.smithi038.stdout:8/131: readlink l0 0 2024-02-15T18:34:51.814 INFO:tasks.workunit.client.0.smithi038.stdout:8/132: readlink d1/d4/d5/d6/le 0 2024-02-15T18:34:51.814 INFO:tasks.workunit.client.0.smithi038.stdout:8/133: chown d1/d4/d5/f23 232499282 1 2024-02-15T18:34:51.814 INFO:tasks.workunit.client.0.smithi038.stdout:5/41: getdents d2/d3 0 2024-02-15T18:34:51.814 INFO:tasks.workunit.client.0.smithi038.stdout:5/42: chown d2/d3/l9 3103502 1 2024-02-15T18:34:51.816 INFO:tasks.workunit.client.0.smithi038.stdout:3/26: dread d2/f5 [0,4194304] 0 2024-02-15T18:34:51.816 INFO:tasks.workunit.client.0.smithi038.stdout:5/43: write d2/d3/f7 [3558092,107726] 0 2024-02-15T18:34:51.820 INFO:tasks.workunit.client.0.smithi038.stdout:8/134: symlink d1/d7/d26/l2c 0 2024-02-15T18:34:51.822 INFO:tasks.workunit.client.0.smithi038.stdout:5/44: write d2/d3/f7 [1286460,124604] 0 2024-02-15T18:34:51.823 INFO:tasks.workunit.client.0.smithi038.stdout:0/86: sync 2024-02-15T18:34:51.823 INFO:tasks.workunit.client.0.smithi038.stdout:9/24: sync 2024-02-15T18:34:51.823 INFO:tasks.workunit.client.0.smithi038.stdout:6/46: sync 2024-02-15T18:34:51.823 INFO:tasks.workunit.client.0.smithi038.stdout:7/26: sync 2024-02-15T18:34:51.823 INFO:tasks.workunit.client.0.smithi038.stdout:1/43: sync 2024-02-15T18:34:51.823 INFO:tasks.workunit.client.0.smithi038.stdout:2/41: sync 2024-02-15T18:34:51.823 INFO:tasks.workunit.client.0.smithi038.stdout:9/25: chown d1/l8 1340 1 2024-02-15T18:34:51.823 INFO:tasks.workunit.client.0.smithi038.stdout:7/27: stat d0/f9 0 2024-02-15T18:34:51.824 INFO:tasks.workunit.client.0.smithi038.stdout:9/26: stat d1/f5 0 2024-02-15T18:34:51.824 INFO:tasks.workunit.client.0.smithi038.stdout:9/27: dread - d1/fb zero size 2024-02-15T18:34:51.824 INFO:tasks.workunit.client.0.smithi038.stdout:9/28: chown d1/da 6861 1 2024-02-15T18:34:51.824 INFO:tasks.workunit.client.0.smithi038.stdout:4/28: sync 2024-02-15T18:34:51.825 INFO:tasks.workunit.client.0.smithi038.stdout:2/42: rename dd to dd/d10 22 2024-02-15T18:34:51.826 INFO:tasks.workunit.client.0.smithi038.stdout:5/45: fdatasync d2/d3/f7 0 2024-02-15T18:34:51.826 INFO:tasks.workunit.client.0.smithi038.stdout:6/47: creat d2/fe x:0 0 0 2024-02-15T18:34:51.827 INFO:tasks.workunit.client.0.smithi038.stdout:9/29: mkdir d1/dc 0 2024-02-15T18:34:51.828 INFO:tasks.workunit.client.0.smithi038.stdout:9/30: dread - d1/f9 zero size 2024-02-15T18:34:51.828 INFO:tasks.workunit.client.0.smithi038.stdout:9/31: chown d1/l6 375 1 2024-02-15T18:34:51.829 INFO:tasks.workunit.client.0.smithi038.stdout:4/29: getdents . 0 2024-02-15T18:34:51.832 INFO:tasks.workunit.client.0.smithi038.stdout:8/135: dread d1/f2 [4194304,4194304] 0 2024-02-15T18:34:51.832 INFO:tasks.workunit.client.0.smithi038.stdout:1/44: mknod d2/cc 0 2024-02-15T18:34:51.833 INFO:tasks.workunit.client.0.smithi038.stdout:5/46: write d2/d3/f7 [275470,30521] 0 2024-02-15T18:34:51.834 INFO:tasks.workunit.client.0.smithi038.stdout:6/48: rename d2/d4/fa to d2/d4/ff 0 2024-02-15T18:34:51.834 INFO:tasks.workunit.client.0.smithi038.stdout:6/49: truncate f1 5152785 0 2024-02-15T18:34:51.834 INFO:tasks.workunit.client.0.smithi038.stdout:8/136: creat d1/d7/f2d x:0 0 0 2024-02-15T18:34:51.834 INFO:tasks.workunit.client.0.smithi038.stdout:8/137: write d1/d4/d5/f23 [671048,88249] 0 2024-02-15T18:34:51.835 INFO:tasks.workunit.client.0.smithi038.stdout:2/43: mknod dd/c11 0 2024-02-15T18:34:51.837 INFO:tasks.workunit.client.0.smithi038.stdout:6/50: mkdir d2/d10 0 2024-02-15T18:34:51.837 INFO:tasks.workunit.client.0.smithi038.stdout:6/51: chown d2/f3 12404771 1 2024-02-15T18:34:51.837 INFO:tasks.workunit.client.0.smithi038.stdout:8/138: mknod d1/d4/d10/c2e 0 2024-02-15T18:34:51.838 INFO:tasks.workunit.client.0.smithi038.stdout:1/45: truncate d2/f4 8085144 0 2024-02-15T18:34:51.840 INFO:tasks.workunit.client.0.smithi038.stdout:8/139: creat d1/d7/d15/f2f x:0 0 0 2024-02-15T18:34:51.842 INFO:tasks.workunit.client.0.smithi038.stdout:3/27: dwrite d2/f5 [0,4194304] 0 2024-02-15T18:34:51.842 INFO:tasks.workunit.client.0.smithi038.stdout:2/44: mkdir dd/d12 0 2024-02-15T18:34:51.845 INFO:tasks.workunit.client.0.smithi038.stdout:8/140: write d1/d7/fc [2437838,104250] 0 2024-02-15T18:34:51.845 INFO:tasks.workunit.client.0.smithi038.stdout:3/28: rename d2/c4 to d2/c6 0 2024-02-15T18:34:51.845 INFO:tasks.workunit.client.0.smithi038.stdout:3/29: chown d2/f5 64010035 1 2024-02-15T18:34:51.846 INFO:tasks.workunit.client.0.smithi038.stdout:2/45: dread - dd/fe zero size 2024-02-15T18:34:51.846 INFO:tasks.workunit.client.0.smithi038.stdout:2/46: read - dd/fe zero size 2024-02-15T18:34:51.846 INFO:tasks.workunit.client.0.smithi038.stdout:1/46: rename d2/c7 to d2/cd 0 2024-02-15T18:34:51.847 INFO:tasks.workunit.client.0.smithi038.stdout:2/47: write f7 [3393536,31350] 0 2024-02-15T18:34:51.847 INFO:tasks.workunit.client.0.smithi038.stdout:8/141: mkdir d1/d4/d5/d30 0 2024-02-15T18:34:51.847 INFO:tasks.workunit.client.0.smithi038.stdout:8/142: stat d1/d7 0 2024-02-15T18:34:51.848 INFO:tasks.workunit.client.0.smithi038.stdout:8/143: dread d1/d4/d5/f20 [0,4194304] 0 2024-02-15T18:34:51.849 INFO:tasks.workunit.client.0.smithi038.stdout:8/144: write d1/d4/d5/f23 [1666208,50329] 0 2024-02-15T18:34:51.849 INFO:tasks.workunit.client.0.smithi038.stdout:3/30: unlink d2/l3 0 2024-02-15T18:34:51.850 INFO:tasks.workunit.client.0.smithi038.stdout:2/48: mkdir dd/d12/d13 0 2024-02-15T18:34:51.851 INFO:tasks.workunit.client.0.smithi038.stdout:8/145: mkdir d1/d4/d5/d31 0 2024-02-15T18:34:51.851 INFO:tasks.workunit.client.0.smithi038.stdout:8/146: fsync d1/f1b 0 2024-02-15T18:34:51.853 INFO:tasks.workunit.client.0.smithi038.stdout:2/49: symlink dd/d12/d13/l14 0 2024-02-15T18:34:51.855 INFO:tasks.workunit.client.0.smithi038.stdout:8/147: rename d1/d7/l11 to d1/d4/d5/d30/l32 0 2024-02-15T18:34:51.855 INFO:tasks.workunit.client.0.smithi038.stdout:8/148: dread - d1/d4/d5/d6/f2a zero size 2024-02-15T18:34:51.855 INFO:tasks.workunit.client.0.smithi038.stdout:8/149: dread - d1/d4/d5/f27 zero size 2024-02-15T18:34:51.855 INFO:tasks.workunit.client.0.smithi038.stdout:8/150: chown d1/d4/f18 92 1 2024-02-15T18:34:51.856 INFO:tasks.workunit.client.0.smithi038.stdout:2/50: mkdir dd/d12/d15 0 2024-02-15T18:34:51.859 INFO:tasks.workunit.client.0.smithi038.stdout:8/151: rename d1/d4/f18 to d1/d4/d5/d31/f33 0 2024-02-15T18:34:51.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:50.316412+0000 mgr.smithi038.khesbl (mgr.14162) 335 : cluster [DBG] pgmap v277: 65 pgs: 65 active+clean; 278 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 16 MiB/s wr, 849 op/s 2024-02-15T18:34:51.864 INFO:tasks.workunit.client.0.smithi038.stdout:0/87: dwrite d4/fa [0,4194304] 0 2024-02-15T18:34:51.864 INFO:tasks.workunit.client.0.smithi038.stdout:7/28: dwrite d0/f1 [0,4194304] 0 2024-02-15T18:34:51.864 INFO:tasks.workunit.client.0.smithi038.stdout:9/32: dwrite d1/fb [0,4194304] 0 2024-02-15T18:34:51.864 INFO:tasks.workunit.client.0.smithi038.stdout:9/33: fsync f0 0 2024-02-15T18:34:51.869 INFO:tasks.workunit.client.0.smithi038.stdout:0/88: mkdir d4/d13/d19 0 2024-02-15T18:34:51.869 INFO:tasks.workunit.client.0.smithi038.stdout:0/89: chown l3 0 1 2024-02-15T18:34:51.870 INFO:tasks.workunit.client.0.smithi038.stdout:6/52: getdents d2 0 2024-02-15T18:34:51.870 INFO:tasks.workunit.client.0.smithi038.stdout:6/53: dread - d2/fe zero size 2024-02-15T18:34:51.872 INFO:tasks.workunit.client.0.smithi038.stdout:5/47: dread d2/d3/f7 [0,4194304] 0 2024-02-15T18:34:51.873 INFO:tasks.workunit.client.0.smithi038.stdout:5/48: read d2/d3/f7 [3027223,331] 0 2024-02-15T18:34:51.873 INFO:tasks.workunit.client.0.smithi038.stdout:3/31: truncate d2/f5 2810288 0 2024-02-15T18:34:51.874 INFO:tasks.workunit.client.0.smithi038.stdout:8/152: dwrite d1/d4/d5/f27 [0,4194304] 0 2024-02-15T18:34:51.874 INFO:tasks.workunit.client.0.smithi038.stdout:8/153: fsync d1/d4/d5/d6/f2a 0 2024-02-15T18:34:51.875 INFO:tasks.workunit.client.0.smithi038.stdout:1/47: write f1 [2474016,92345] 0 2024-02-15T18:34:51.875 INFO:tasks.workunit.client.0.smithi038.stdout:6/54: dread d2/f9 [0,4194304] 0 2024-02-15T18:34:51.875 INFO:tasks.workunit.client.0.smithi038.stdout:6/55: chown d2/d4/l7 4 1 2024-02-15T18:34:51.875 INFO:tasks.workunit.client.0.smithi038.stdout:6/56: fdatasync d2/fe 0 2024-02-15T18:34:51.884 INFO:tasks.workunit.client.0.smithi038.stdout:5/49: dread d2/d3/f7 [0,4194304] 0 2024-02-15T18:34:51.884 INFO:tasks.workunit.client.0.smithi038.stdout:5/50: write d2/d3/f7 [4790683,38807] 0 2024-02-15T18:34:51.885 INFO:tasks.workunit.client.0.smithi038.stdout:7/29: mkdir d0/db 0 2024-02-15T18:34:51.885 INFO:tasks.workunit.client.0.smithi038.stdout:9/34: symlink d1/dc/ld 0 2024-02-15T18:34:51.886 INFO:tasks.workunit.client.0.smithi038.stdout:9/35: dread - d1/f9 zero size 2024-02-15T18:34:51.886 INFO:tasks.workunit.client.0.smithi038.stdout:9/36: write d1/f9 [608105,129006] 0 2024-02-15T18:34:51.887 INFO:tasks.workunit.client.0.smithi038.stdout:0/90: dread d4/f15 [0,4194304] 0 2024-02-15T18:34:51.888 INFO:tasks.workunit.client.0.smithi038.stdout:8/154: creat d1/d7/d26/f34 x:0 0 0 2024-02-15T18:34:51.889 INFO:tasks.workunit.client.0.smithi038.stdout:8/155: write d1/d7/d15/f2f [423283,44966] 0 2024-02-15T18:34:51.889 INFO:tasks.workunit.client.0.smithi038.stdout:8/156: write d1/d4/d10/f1e [4443792,73242] 0 2024-02-15T18:34:51.889 INFO:tasks.workunit.client.0.smithi038.stdout:8/157: readlink d1/d4/d5/d30/l32 0 2024-02-15T18:34:51.889 INFO:tasks.workunit.client.0.smithi038.stdout:6/57: mknod d2/d4/c11 0 2024-02-15T18:34:51.890 INFO:tasks.workunit.client.0.smithi038.stdout:8/158: dread d1/fb [4194304,4194304] 0 2024-02-15T18:34:51.891 INFO:tasks.workunit.client.0.smithi038.stdout:8/159: truncate d1/d4/d5/f27 4871805 0 2024-02-15T18:34:51.891 INFO:tasks.workunit.client.0.smithi038.stdout:5/51: creat d2/d4/fa x:0 0 0 2024-02-15T18:34:51.891 INFO:tasks.workunit.client.0.smithi038.stdout:5/52: write d2/d4/fa [19069,52223] 0 2024-02-15T18:34:51.891 INFO:tasks.workunit.client.0.smithi038.stdout:5/53: chown d2 13 1 2024-02-15T18:34:51.891 INFO:tasks.workunit.client.0.smithi038.stdout:0/91: mknod d4/d13/c1a 0 2024-02-15T18:34:51.892 INFO:tasks.workunit.client.0.smithi038.stdout:0/92: read d4/f11 [437311,78235] 0 2024-02-15T18:34:51.893 INFO:tasks.workunit.client.0.smithi038.stdout:6/58: unlink d2/d4/l7 0 2024-02-15T18:34:51.893 INFO:tasks.workunit.client.0.smithi038.stdout:8/160: creat d1/d4/d10/f35 x:0 0 0 2024-02-15T18:34:51.895 INFO:tasks.workunit.client.0.smithi038.stdout:8/161: creat d1/d7/d15/f36 x:0 0 0 2024-02-15T18:34:51.896 INFO:tasks.workunit.client.0.smithi038.stdout:8/162: creat d1/d4/d5/f37 x:0 0 0 2024-02-15T18:34:51.896 INFO:tasks.workunit.client.0.smithi038.stdout:6/59: dread d2/f9 [0,4194304] 0 2024-02-15T18:34:51.897 INFO:tasks.workunit.client.0.smithi038.stdout:8/163: creat d1/d4/d5/d6/f38 x:0 0 0 2024-02-15T18:34:51.900 INFO:tasks.workunit.client.0.smithi038.stdout:8/164: unlink d1/d7/d15/c21 0 2024-02-15T18:34:51.900 INFO:tasks.workunit.client.0.smithi038.stdout:8/165: write d1/d4/d5/d6/f38 [277316,110942] 0 2024-02-15T18:34:51.900 INFO:tasks.workunit.client.0.smithi038.stdout:9/37: dread d1/f9 [0,4194304] 0 2024-02-15T18:34:51.905 INFO:tasks.workunit.client.0.smithi038.stdout:8/166: creat d1/d4/d5/d31/f39 x:0 0 0 2024-02-15T18:34:51.905 INFO:tasks.workunit.client.0.smithi038.stdout:9/38: creat d1/dc/fe x:0 0 0 2024-02-15T18:34:51.906 INFO:tasks.workunit.client.0.smithi038.stdout:8/167: mkdir d1/d3a 0 2024-02-15T18:34:51.906 INFO:tasks.workunit.client.0.smithi038.stdout:8/168: chown d1/d7/f1d 25979306 1 2024-02-15T18:34:51.907 INFO:tasks.workunit.client.0.smithi038.stdout:9/39: mkdir d1/df 0 2024-02-15T18:34:51.907 INFO:tasks.workunit.client.0.smithi038.stdout:9/40: write d1/f9 [878417,17309] 0 2024-02-15T18:34:51.908 INFO:tasks.workunit.client.0.smithi038.stdout:3/32: dread d2/f5 [0,4194304] 0 2024-02-15T18:34:51.908 INFO:tasks.workunit.client.0.smithi038.stdout:3/33: write d2/f5 [3520456,96789] 0 2024-02-15T18:34:51.908 INFO:tasks.workunit.client.0.smithi038.stdout:3/34: creat d2/f7 x:0 0 0 2024-02-15T18:34:51.913 INFO:tasks.workunit.client.0.smithi038.stdout:9/41: dread d1/fb [0,4194304] 0 2024-02-15T18:34:51.914 INFO:tasks.workunit.client.0.smithi038.stdout:9/42: mkdir d1/da/d10 0 2024-02-15T18:34:51.917 INFO:tasks.workunit.client.0.smithi038.stdout:7/30: dwrite d0/fa [0,4194304] 0 2024-02-15T18:34:51.917 INFO:tasks.workunit.client.0.smithi038.stdout:7/31: write d0/f9 [564624,88059] 0 2024-02-15T18:34:51.917 INFO:tasks.workunit.client.0.smithi038.stdout:1/48: dwrite f1 [4194304,4194304] 0 2024-02-15T18:34:51.917 INFO:tasks.workunit.client.0.smithi038.stdout:0/93: dwrite d4/f16 [0,4194304] 0 2024-02-15T18:34:51.918 INFO:tasks.workunit.client.0.smithi038.stdout:5/54: dread d2/d4/fa [0,4194304] 0 2024-02-15T18:34:51.919 INFO:tasks.workunit.client.0.smithi038.stdout:7/32: creat d0/db/fc x:0 0 0 2024-02-15T18:34:51.930 INFO:tasks.workunit.client.0.smithi038.stdout:0/94: dread d4/f16 [0,4194304] 0 2024-02-15T18:34:51.930 INFO:tasks.workunit.client.0.smithi038.stdout:0/95: chown d4/c10 707533 1 2024-02-15T18:34:51.932 INFO:tasks.workunit.client.0.smithi038.stdout:8/169: rmdir d1/d4/d5/d31 39 2024-02-15T18:34:51.933 INFO:tasks.workunit.client.0.smithi038.stdout:9/43: rmdir d1/dc 39 2024-02-15T18:34:51.934 INFO:tasks.workunit.client.0.smithi038.stdout:3/35: getdents d2 0 2024-02-15T18:34:51.934 INFO:tasks.workunit.client.0.smithi038.stdout:8/170: dread d1/d4/d5/d6/f38 [0,4194304] 0 2024-02-15T18:34:51.937 INFO:tasks.workunit.client.0.smithi038.stdout:7/33: symlink d0/db/ld 0 2024-02-15T18:34:51.937 INFO:tasks.workunit.client.0.smithi038.stdout:7/34: stat d0/db/fc 0 2024-02-15T18:34:51.937 INFO:tasks.workunit.client.0.smithi038.stdout:7/35: chown d0 3084 1 2024-02-15T18:34:51.937 INFO:tasks.workunit.client.0.smithi038.stdout:7/36: chown d0/f9 7445 1 2024-02-15T18:34:51.938 INFO:tasks.workunit.client.0.smithi038.stdout:8/171: rmdir d1/d4/d5/d30 39 2024-02-15T18:34:51.939 INFO:tasks.workunit.client.0.smithi038.stdout:0/96: mkdir d4/d1b 0 2024-02-15T18:34:51.939 INFO:tasks.workunit.client.0.smithi038.stdout:0/97: chown d4/f12 11614 1 2024-02-15T18:34:51.940 INFO:tasks.workunit.client.0.smithi038.stdout:0/98: rmdir d4/d13 39 2024-02-15T18:34:51.941 INFO:tasks.workunit.client.0.smithi038.stdout:8/172: dread d1/d4/d5/f23 [0,4194304] 0 2024-02-15T18:34:51.941 INFO:tasks.workunit.client.0.smithi038.stdout:8/173: fdatasync d1/f2 0 2024-02-15T18:34:51.941 INFO:tasks.workunit.client.0.smithi038.stdout:8/174: truncate d1/d7/f2d 239033 0 2024-02-15T18:34:51.942 INFO:tasks.workunit.client.0.smithi038.stdout:0/99: symlink d4/l1c 0 2024-02-15T18:34:51.942 INFO:tasks.workunit.client.0.smithi038.stdout:5/55: dwrite d2/d4/fa [0,4194304] 0 2024-02-15T18:34:51.942 INFO:tasks.workunit.client.0.smithi038.stdout:5/56: chown d2/d3/l9 26 1 2024-02-15T18:34:51.944 INFO:tasks.workunit.client.0.smithi038.stdout:7/37: write d0/f1 [2771111,104645] 0 2024-02-15T18:34:51.944 INFO:tasks.workunit.client.0.smithi038.stdout:7/38: readlink d0/d3/l6 0 2024-02-15T18:34:51.944 INFO:tasks.workunit.client.0.smithi038.stdout:8/175: symlink d1/d4/d5/d30/l3b 0 2024-02-15T18:34:51.946 INFO:tasks.workunit.client.0.smithi038.stdout:1/49: dwrite d2/f4 [0,4194304] 0 2024-02-15T18:34:51.946 INFO:tasks.workunit.client.0.smithi038.stdout:7/39: creat d0/db/fe x:0 0 0 2024-02-15T18:34:51.948 INFO:tasks.workunit.client.0.smithi038.stdout:8/176: truncate d1/d4/d5/f27 2707242 0 2024-02-15T18:34:51.956 INFO:tasks.workunit.client.0.smithi038.stdout:1/50: dread d2/f3 [0,4194304] 0 2024-02-15T18:34:51.960 INFO:tasks.workunit.client.0.smithi038.stdout:9/44: dwrite d1/f9 [0,4194304] 0 2024-02-15T18:34:51.964 INFO:tasks.workunit.client.0.smithi038.stdout:1/51: write f1 [2829775,14436] 0 2024-02-15T18:34:51.969 INFO:tasks.workunit.client.0.smithi038.stdout:5/57: dwrite d2/d3/f7 [4194304,4194304] 0 2024-02-15T18:34:51.969 INFO:tasks.workunit.client.0.smithi038.stdout:7/40: fdatasync d0/f9 0 2024-02-15T18:34:51.969 INFO:tasks.workunit.client.0.smithi038.stdout:3/36: dwrite d2/f5 [0,4194304] 0 2024-02-15T18:34:51.969 INFO:tasks.workunit.client.0.smithi038.stdout:3/37: chown d2/f7 356072 1 2024-02-15T18:34:51.970 INFO:tasks.workunit.client.0.smithi038.stdout:3/38: write d2/f7 [713596,20677] 0 2024-02-15T18:34:51.970 INFO:tasks.workunit.client.0.smithi038.stdout:3/39: write d2/f7 [1231691,103677] 0 2024-02-15T18:34:51.970 INFO:tasks.workunit.client.0.smithi038.stdout:3/40: getdents d2 0 2024-02-15T18:34:51.970 INFO:tasks.workunit.client.0.smithi038.stdout:8/177: dwrite d1/d7/d26/f34 [0,4194304] 0 2024-02-15T18:34:51.970 INFO:tasks.workunit.client.0.smithi038.stdout:7/41: write d0/f1 [1526048,108025] 0 2024-02-15T18:34:52.005 INFO:tasks.workunit.client.0.smithi038.stdout:0/100: truncate d4/f16 2760176 0 2024-02-15T18:34:52.008 INFO:tasks.workunit.client.0.smithi038.stdout:4/30: sync 2024-02-15T18:34:52.009 INFO:tasks.workunit.client.0.smithi038.stdout:4/31: truncate f3 4963770 0 2024-02-15T18:34:52.009 INFO:tasks.workunit.client.0.smithi038.stdout:4/32: getdents . 0 2024-02-15T18:34:52.010 INFO:tasks.workunit.client.0.smithi038.stdout:9/45: mkdir d1/d11 0 2024-02-15T18:34:52.026 INFO:tasks.workunit.client.0.smithi038.stdout:9/46: dwrite d1/f9 [0,4194304] 0 2024-02-15T18:34:52.026 INFO:tasks.workunit.client.0.smithi038.stdout:9/47: truncate d1/f9 4699631 0 2024-02-15T18:34:52.042 INFO:tasks.workunit.client.0.smithi038.stdout:5/58: mknod d2/d4/cb 0 2024-02-15T18:34:52.046 INFO:tasks.workunit.client.0.smithi038.stdout:3/41: rename l0 to d2/l8 0 2024-02-15T18:34:52.057 INFO:tasks.workunit.client.0.smithi038.stdout:8/178: symlink d1/l3c 0 2024-02-15T18:34:52.058 INFO:tasks.workunit.client.0.smithi038.stdout:7/42: creat d0/db/ff x:0 0 0 2024-02-15T18:34:52.058 INFO:tasks.workunit.client.0.smithi038.stdout:7/43: write d0/db/fe [919069,67210] 0 2024-02-15T18:34:52.058 INFO:tasks.workunit.client.0.smithi038.stdout:7/44: write d0/f9 [1080942,17832] 0 2024-02-15T18:34:52.064 INFO:tasks.workunit.client.0.smithi038.stdout:0/101: symlink d4/d13/d19/l1d 0 2024-02-15T18:34:52.065 INFO:tasks.workunit.client.0.smithi038.stdout:4/33: mknod cf 0 2024-02-15T18:34:52.066 INFO:tasks.workunit.client.0.smithi038.stdout:2/51: sync 2024-02-15T18:34:52.066 INFO:tasks.workunit.client.0.smithi038.stdout:6/60: sync 2024-02-15T18:34:52.067 INFO:tasks.workunit.client.0.smithi038.stdout:1/52: mknod d2/ce 0 2024-02-15T18:34:52.068 INFO:tasks.workunit.client.0.smithi038.stdout:9/48: unlink d1/dc/ld 0 2024-02-15T18:34:52.068 INFO:tasks.workunit.client.0.smithi038.stdout:9/49: write d1/dc/fe [137374,51887] 0 2024-02-15T18:34:52.069 INFO:tasks.workunit.client.0.smithi038.stdout:5/59: creat d2/fc x:0 0 0 2024-02-15T18:34:52.069 INFO:tasks.workunit.client.0.smithi038.stdout:5/60: fdatasync d2/d4/fa 0 2024-02-15T18:34:52.069 INFO:tasks.workunit.client.0.smithi038.stdout:5/61: fdatasync d2/d4/fa 0 2024-02-15T18:34:52.070 INFO:tasks.workunit.client.0.smithi038.stdout:3/42: creat d2/f9 x:0 0 0 2024-02-15T18:34:52.070 INFO:tasks.workunit.client.0.smithi038.stdout:3/43: dread - d2/f9 zero size 2024-02-15T18:34:52.070 INFO:tasks.workunit.client.0.smithi038.stdout:5/62: write d2/d4/fa [3089504,110515] 0 2024-02-15T18:34:52.070 INFO:tasks.workunit.client.0.smithi038.stdout:8/179: mkdir d1/d4/d10/d3d 0 2024-02-15T18:34:52.071 INFO:tasks.workunit.client.0.smithi038.stdout:0/102: mkdir d4/d1b/d1e 0 2024-02-15T18:34:52.071 INFO:tasks.workunit.client.0.smithi038.stdout:9/50: dread d1/fb [0,4194304] 0 2024-02-15T18:34:52.072 INFO:tasks.workunit.client.0.smithi038.stdout:4/34: unlink fa 0 2024-02-15T18:34:52.073 INFO:tasks.workunit.client.0.smithi038.stdout:2/52: mknod dd/d12/d13/c16 0 2024-02-15T18:34:52.073 INFO:tasks.workunit.client.0.smithi038.stdout:2/53: stat dd/d12/d15 0 2024-02-15T18:34:52.074 INFO:tasks.workunit.client.0.smithi038.stdout:6/61: mknod d2/d10/c12 0 2024-02-15T18:34:52.076 INFO:tasks.workunit.client.0.smithi038.stdout:1/53: creat d2/ff x:0 0 0 2024-02-15T18:34:52.077 INFO:tasks.workunit.client.0.smithi038.stdout:3/44: unlink c1 0 2024-02-15T18:34:52.078 INFO:tasks.workunit.client.0.smithi038.stdout:5/63: creat d2/fd x:0 0 0 2024-02-15T18:34:52.078 INFO:tasks.workunit.client.0.smithi038.stdout:8/180: creat d1/d7/d26/f3e x:0 0 0 2024-02-15T18:34:52.080 INFO:tasks.workunit.client.0.smithi038.stdout:9/51: symlink d1/df/l12 0 2024-02-15T18:34:52.080 INFO:tasks.workunit.client.0.smithi038.stdout:9/52: truncate d1/dc/fe 511690 0 2024-02-15T18:34:52.080 INFO:tasks.workunit.client.0.smithi038.stdout:7/45: dwrite d0/f9 [0,4194304] 0 2024-02-15T18:34:52.081 INFO:tasks.workunit.client.0.smithi038.stdout:4/35: rename cf to c10 0 2024-02-15T18:34:52.085 INFO:tasks.workunit.client.0.smithi038.stdout:0/103: unlink d4/ld 0 2024-02-15T18:34:52.087 INFO:tasks.workunit.client.0.smithi038.stdout:2/54: rename c1 to dd/c17 0 2024-02-15T18:34:52.088 INFO:tasks.workunit.client.0.smithi038.stdout:2/55: stat dd/fe 0 2024-02-15T18:34:52.088 INFO:tasks.workunit.client.0.smithi038.stdout:5/64: dread d2/d4/fa [0,4194304] 0 2024-02-15T18:34:52.091 INFO:tasks.workunit.client.0.smithi038.stdout:9/53: rmdir d1/dc 39 2024-02-15T18:34:52.093 INFO:tasks.workunit.client.0.smithi038.stdout:0/104: dread d4/f16 [0,4194304] 0 2024-02-15T18:34:52.102 INFO:tasks.workunit.client.0.smithi038.stdout:1/54: mkdir d2/d10 0 2024-02-15T18:34:52.103 INFO:tasks.workunit.client.0.smithi038.stdout:2/56: mkdir dd/d12/d18 0 2024-02-15T18:34:52.103 INFO:tasks.workunit.client.0.smithi038.stdout:2/57: write fc [473411,106172] 0 2024-02-15T18:34:52.103 INFO:tasks.workunit.client.0.smithi038.stdout:5/65: creat d2/d4/fe x:0 0 0 2024-02-15T18:34:52.103 INFO:tasks.workunit.client.0.smithi038.stdout:5/66: dread - d2/fc zero size 2024-02-15T18:34:52.104 INFO:tasks.workunit.client.0.smithi038.stdout:9/54: mkdir d1/da/d10/d13 0 2024-02-15T18:34:52.105 INFO:tasks.workunit.client.0.smithi038.stdout:0/105: unlink d4/f16 0 2024-02-15T18:34:52.105 INFO:tasks.workunit.client.0.smithi038.stdout:2/58: creat dd/d12/d13/f19 x:0 0 0 2024-02-15T18:34:52.108 INFO:tasks.workunit.client.0.smithi038.stdout:0/106: rename d4/d13/d19/l1d to d4/d1b/d1e/l1f 0 2024-02-15T18:34:52.109 INFO:tasks.workunit.client.0.smithi038.stdout:2/59: rename dd/fe to dd/d12/d15/f1a 0 2024-02-15T18:34:52.109 INFO:tasks.workunit.client.0.smithi038.stdout:2/60: chown dd/d12/d13/c16 1 1 2024-02-15T18:34:52.109 INFO:tasks.workunit.client.0.smithi038.stdout:2/61: write fc [1066602,92013] 0 2024-02-15T18:34:52.110 INFO:tasks.workunit.client.0.smithi038.stdout:0/107: creat d4/d1b/d1e/f20 x:0 0 0 2024-02-15T18:34:52.110 INFO:tasks.workunit.client.0.smithi038.stdout:0/108: fsync d4/f15 0 2024-02-15T18:34:52.110 INFO:tasks.workunit.client.0.smithi038.stdout:0/109: fdatasync d4/ff 0 2024-02-15T18:34:52.111 INFO:tasks.workunit.client.0.smithi038.stdout:8/181: dwrite d1/d4/d5/f37 [0,4194304] 0 2024-02-15T18:34:52.111 INFO:tasks.workunit.client.0.smithi038.stdout:8/182: write d1/d7/f2d [331814,18143] 0 2024-02-15T18:34:52.111 INFO:tasks.workunit.client.0.smithi038.stdout:3/45: dwrite d2/f7 [0,4194304] 0 2024-02-15T18:34:52.111 INFO:tasks.workunit.client.0.smithi038.stdout:1/55: write f1 [3901355,35057] 0 2024-02-15T18:34:52.111 INFO:tasks.workunit.client.0.smithi038.stdout:2/62: symlink dd/d12/d15/l1b 0 2024-02-15T18:34:52.112 INFO:tasks.workunit.client.0.smithi038.stdout:4/36: read f2 [1196024,96350] 0 2024-02-15T18:34:52.113 INFO:tasks.workunit.client.0.smithi038.stdout:7/46: dwrite d0/fa [0,4194304] 0 2024-02-15T18:34:52.113 INFO:tasks.workunit.client.0.smithi038.stdout:7/47: stat d0 0 2024-02-15T18:34:52.113 INFO:tasks.workunit.client.0.smithi038.stdout:7/48: fsync d0/fa 0 2024-02-15T18:34:52.113 INFO:tasks.workunit.client.0.smithi038.stdout:0/110: rename d4/d13/c1a to d4/d13/d19/c21 0 2024-02-15T18:34:52.120 INFO:tasks.workunit.client.0.smithi038.stdout:5/67: rmdir d2 39 2024-02-15T18:34:52.126 INFO:tasks.workunit.client.0.smithi038.stdout:8/183: truncate d1/d7/d26/f34 2804753 0 2024-02-15T18:34:52.129 INFO:tasks.workunit.client.0.smithi038.stdout:3/46: unlink d2/c6 0 2024-02-15T18:34:52.130 INFO:tasks.workunit.client.0.smithi038.stdout:3/47: write d2/f9 [924821,118821] 0 2024-02-15T18:34:52.130 INFO:tasks.workunit.client.0.smithi038.stdout:1/56: creat d2/f11 x:0 0 0 2024-02-15T18:34:52.132 INFO:tasks.workunit.client.0.smithi038.stdout:2/63: link fb dd/d12/d18/f1c 0 2024-02-15T18:34:52.133 INFO:tasks.workunit.client.0.smithi038.stdout:2/64: dread - dd/d12/d15/f1a zero size 2024-02-15T18:34:52.133 INFO:tasks.workunit.client.0.smithi038.stdout:4/37: mkdir d11 0 2024-02-15T18:34:52.135 INFO:tasks.workunit.client.0.smithi038.stdout:8/184: creat d1/d4/d5/d30/f3f x:0 0 0 2024-02-15T18:34:52.136 INFO:tasks.workunit.client.0.smithi038.stdout:3/48: creat d2/fa x:0 0 0 2024-02-15T18:34:52.136 INFO:tasks.workunit.client.0.smithi038.stdout:3/49: write d2/f9 [1129505,114404] 0 2024-02-15T18:34:52.136 INFO:tasks.workunit.client.0.smithi038.stdout:3/50: chown d2 3 1 2024-02-15T18:34:52.136 INFO:tasks.workunit.client.0.smithi038.stdout:4/38: stat c10 0 2024-02-15T18:34:52.138 INFO:tasks.workunit.client.0.smithi038.stdout:8/185: creat d1/f40 x:0 0 0 2024-02-15T18:34:52.138 INFO:tasks.workunit.client.0.smithi038.stdout:3/51: mkdir d2/db 0 2024-02-15T18:34:52.139 INFO:tasks.workunit.client.0.smithi038.stdout:8/186: mknod d1/d7/c41 0 2024-02-15T18:34:52.140 INFO:tasks.workunit.client.0.smithi038.stdout:8/187: chown d1/d7/d15/c2b 14882444 1 2024-02-15T18:34:52.141 INFO:tasks.workunit.client.0.smithi038.stdout:8/188: mknod d1/d7/d26/c42 0 2024-02-15T18:34:52.143 INFO:tasks.workunit.client.0.smithi038.stdout:8/189: mkdir d1/d4/d10/d43 0 2024-02-15T18:34:52.143 INFO:tasks.workunit.client.0.smithi038.stdout:8/190: fdatasync d1/f40 0 2024-02-15T18:34:52.143 INFO:tasks.workunit.client.0.smithi038.stdout:8/191: chown d1/d7/d26 1139831265 1 2024-02-15T18:34:52.143 INFO:tasks.workunit.client.0.smithi038.stdout:8/192: write d1/d4/d5/f23 [2212333,107509] 0 2024-02-15T18:34:52.143 INFO:tasks.workunit.client.0.smithi038.stdout:8/193: chown d1/d4/d5/d6/le 4221 1 2024-02-15T18:34:52.144 INFO:tasks.workunit.client.0.smithi038.stdout:8/194: mkdir d1/d4/d5/d31/d44 0 2024-02-15T18:34:52.144 INFO:tasks.workunit.client.0.smithi038.stdout:8/195: chown d1/d4/d5/d6 6 1 2024-02-15T18:34:52.144 INFO:tasks.workunit.client.0.smithi038.stdout:2/65: dread fc [0,4194304] 0 2024-02-15T18:34:52.144 INFO:tasks.workunit.client.0.smithi038.stdout:2/66: read - dd/d12/d15/f1a zero size 2024-02-15T18:34:52.145 INFO:tasks.workunit.client.0.smithi038.stdout:2/67: stat c8 0 2024-02-15T18:34:52.146 INFO:tasks.workunit.client.0.smithi038.stdout:0/111: dwrite d4/f15 [0,4194304] 0 2024-02-15T18:34:52.149 INFO:tasks.workunit.client.0.smithi038.stdout:0/112: mknod d4/d13/c22 0 2024-02-15T18:34:52.150 INFO:tasks.workunit.client.0.smithi038.stdout:0/113: getdents d4/d13/d19 0 2024-02-15T18:34:52.150 INFO:tasks.workunit.client.0.smithi038.stdout:7/49: dwrite d0/f1 [0,4194304] 0 2024-02-15T18:34:52.150 INFO:tasks.workunit.client.0.smithi038.stdout:5/68: dwrite d2/fc [0,4194304] 0 2024-02-15T18:34:52.150 INFO:tasks.workunit.client.0.smithi038.stdout:1/57: dwrite d2/ff [0,4194304] 0 2024-02-15T18:34:52.151 INFO:tasks.workunit.client.0.smithi038.stdout:0/114: mknod d4/d13/d19/c23 0 2024-02-15T18:34:52.151 INFO:tasks.workunit.client.0.smithi038.stdout:0/115: stat d4/c10 0 2024-02-15T18:34:52.153 INFO:tasks.workunit.client.0.smithi038.stdout:7/50: rename d0/d3/d8 to d0/d10 0 2024-02-15T18:34:52.157 INFO:tasks.workunit.client.0.smithi038.stdout:5/69: creat d2/d3/ff x:0 0 0 2024-02-15T18:34:52.160 INFO:tasks.workunit.client.0.smithi038.stdout:8/196: dwrite d1/d4/d5/d31/f39 [0,4194304] 0 2024-02-15T18:34:52.164 INFO:tasks.workunit.client.0.smithi038.stdout:1/58: mknod d2/d10/c12 0 2024-02-15T18:34:52.165 INFO:tasks.workunit.client.0.smithi038.stdout:7/51: truncate d0/db/fe 532598 0 2024-02-15T18:34:52.166 INFO:tasks.workunit.client.0.smithi038.stdout:7/52: fsync d0/f9 0 2024-02-15T18:34:52.168 INFO:tasks.workunit.client.0.smithi038.stdout:8/197: mknod d1/d4/d10/d43/c45 0 2024-02-15T18:34:52.168 INFO:tasks.workunit.client.0.smithi038.stdout:8/198: readlink d1/d7/d15/l16 0 2024-02-15T18:34:52.169 INFO:tasks.workunit.client.0.smithi038.stdout:7/53: unlink d0/c2 0 2024-02-15T18:34:52.169 INFO:tasks.workunit.client.0.smithi038.stdout:2/68: dwrite dd/d12/d15/f1a [0,4194304] 0 2024-02-15T18:34:52.169 INFO:tasks.workunit.client.0.smithi038.stdout:2/69: chown dd/d12/d15 0 1 2024-02-15T18:34:52.169 INFO:tasks.workunit.client.0.smithi038.stdout:8/199: dread d1/d7/d15/f2f [0,4194304] 0 2024-02-15T18:34:52.171 INFO:tasks.workunit.client.0.smithi038.stdout:0/116: rename d4/d13/d19/c21 to d4/d1b/c24 0 2024-02-15T18:34:52.172 INFO:tasks.workunit.client.0.smithi038.stdout:6/62: sync 2024-02-15T18:34:52.175 INFO:tasks.workunit.client.0.smithi038.stdout:1/59: dread f1 [0,4194304] 0 2024-02-15T18:34:52.175 INFO:tasks.workunit.client.0.smithi038.stdout:7/54: mknod d0/d3/c11 0 2024-02-15T18:34:52.176 INFO:tasks.workunit.client.0.smithi038.stdout:2/70: rmdir dd 39 2024-02-15T18:34:52.180 INFO:tasks.workunit.client.0.smithi038.stdout:6/63: mknod d2/d10/c13 0 2024-02-15T18:34:52.181 INFO:tasks.workunit.client.0.smithi038.stdout:1/60: getdents d2/d10 0 2024-02-15T18:34:52.181 INFO:tasks.workunit.client.0.smithi038.stdout:7/55: mknod d0/d3/c12 0 2024-02-15T18:34:52.183 INFO:tasks.workunit.client.0.smithi038.stdout:0/117: rename d4/d13/d19/c23 to d4/d13/d19/c25 0 2024-02-15T18:34:52.184 INFO:tasks.workunit.client.0.smithi038.stdout:0/118: chown d4/c10 3686257 1 2024-02-15T18:34:52.184 INFO:tasks.workunit.client.0.smithi038.stdout:0/119: dread - d4/f12 zero size 2024-02-15T18:34:52.184 INFO:tasks.workunit.client.0.smithi038.stdout:0/120: fsync d4/f12 0 2024-02-15T18:34:52.184 INFO:tasks.workunit.client.0.smithi038.stdout:0/121: write d4/f11 [1405583,129253] 0 2024-02-15T18:34:52.184 INFO:tasks.workunit.client.0.smithi038.stdout:6/64: mknod d2/d10/c14 0 2024-02-15T18:34:52.187 INFO:tasks.workunit.client.0.smithi038.stdout:0/122: rename d4/d13/c22 to d4/d1b/c26 0 2024-02-15T18:34:52.188 INFO:tasks.workunit.client.0.smithi038.stdout:6/65: creat d2/d10/f15 x:0 0 0 2024-02-15T18:34:52.188 INFO:tasks.workunit.client.0.smithi038.stdout:6/66: truncate d2/f3 1073077 0 2024-02-15T18:34:52.189 INFO:tasks.workunit.client.0.smithi038.stdout:1/61: link d2/l8 d2/d10/l13 0 2024-02-15T18:34:52.190 INFO:tasks.workunit.client.0.smithi038.stdout:6/67: write f1 [1435384,61106] 0 2024-02-15T18:34:52.191 INFO:tasks.workunit.client.0.smithi038.stdout:1/62: creat d2/d10/f14 x:0 0 0 2024-02-15T18:34:52.191 INFO:tasks.workunit.client.0.smithi038.stdout:1/63: truncate d2/f11 653712 0 2024-02-15T18:34:52.191 INFO:tasks.workunit.client.0.smithi038.stdout:1/64: read d2/f3 [339312,1273] 0 2024-02-15T18:34:52.192 INFO:tasks.workunit.client.0.smithi038.stdout:6/68: mknod d2/c16 0 2024-02-15T18:34:52.192 INFO:tasks.workunit.client.0.smithi038.stdout:6/69: chown d2/f3 1997287 1 2024-02-15T18:34:52.192 INFO:tasks.workunit.client.0.smithi038.stdout:6/70: fsync d2/d4/fc 0 2024-02-15T18:34:52.192 INFO:tasks.workunit.client.0.smithi038.stdout:7/56: dread d0/fa [0,4194304] 0 2024-02-15T18:34:52.192 INFO:tasks.workunit.client.0.smithi038.stdout:0/123: dread d4/ff [0,4194304] 0 2024-02-15T18:34:52.194 INFO:tasks.workunit.client.0.smithi038.stdout:0/124: rmdir d4/d13/d19 39 2024-02-15T18:34:52.194 INFO:tasks.workunit.client.0.smithi038.stdout:1/65: mknod d2/d10/c15 0 2024-02-15T18:34:52.194 INFO:tasks.workunit.client.0.smithi038.stdout:2/71: dwrite fb [0,4194304] 0 2024-02-15T18:34:52.195 INFO:tasks.workunit.client.0.smithi038.stdout:0/125: write d4/fa [1879460,38587] 0 2024-02-15T18:34:52.198 INFO:tasks.workunit.client.0.smithi038.stdout:0/126: rmdir d4/d1b/d1e 39 2024-02-15T18:34:52.199 INFO:tasks.workunit.client.0.smithi038.stdout:0/127: rmdir d4/d13/d19 39 2024-02-15T18:34:52.200 INFO:tasks.workunit.client.0.smithi038.stdout:0/128: symlink d4/d1b/d1e/l27 0 2024-02-15T18:34:52.205 INFO:tasks.workunit.client.0.smithi038.stdout:0/129: rename d4/c10 to d4/d1b/c28 0 2024-02-15T18:34:52.206 INFO:tasks.workunit.client.0.smithi038.stdout:0/130: write d4/f5 [1014250,24799] 0 2024-02-15T18:34:52.207 INFO:tasks.workunit.client.0.smithi038.stdout:9/55: sync 2024-02-15T18:34:52.208 INFO:tasks.workunit.client.0.smithi038.stdout:0/131: unlink l1 0 2024-02-15T18:34:52.209 INFO:tasks.workunit.client.0.smithi038.stdout:9/56: rename d1/f3 to d1/f14 0 2024-02-15T18:34:52.215 INFO:tasks.workunit.client.0.smithi038.stdout:0/132: write d4/f15 [491518,34076] 0 2024-02-15T18:34:52.216 INFO:tasks.workunit.client.0.smithi038.stdout:2/72: dwrite dd/d12/d13/f19 [0,4194304] 0 2024-02-15T18:34:52.216 INFO:tasks.workunit.client.0.smithi038.stdout:1/66: dread d2/ff [0,4194304] 0 2024-02-15T18:34:52.216 INFO:tasks.workunit.client.0.smithi038.stdout:1/67: fsync d2/f4 0 2024-02-15T18:34:52.216 INFO:tasks.workunit.client.0.smithi038.stdout:7/57: dwrite d0/fa [0,4194304] 0 2024-02-15T18:34:52.217 INFO:tasks.workunit.client.0.smithi038.stdout:2/73: write dd/d12/d13/f19 [3267116,85258] 0 2024-02-15T18:34:52.217 INFO:tasks.workunit.client.0.smithi038.stdout:9/57: getdents d1 0 2024-02-15T18:34:52.223 INFO:tasks.workunit.client.0.smithi038.stdout:2/74: rename dd/d12/d15 to dd/d12/d1d 0 2024-02-15T18:34:52.224 INFO:tasks.workunit.client.0.smithi038.stdout:0/133: symlink d4/l29 0 2024-02-15T18:34:52.224 INFO:tasks.workunit.client.0.smithi038.stdout:0/134: fdatasync d4/f12 0 2024-02-15T18:34:52.225 INFO:tasks.workunit.client.0.smithi038.stdout:1/68: mknod d2/c16 0 2024-02-15T18:34:52.225 INFO:tasks.workunit.client.0.smithi038.stdout:1/69: chown d2/d10/c12 64 1 2024-02-15T18:34:52.229 INFO:tasks.workunit.client.0.smithi038.stdout:2/75: creat dd/d12/d18/f1e x:0 0 0 2024-02-15T18:34:52.229 INFO:tasks.workunit.client.0.smithi038.stdout:0/135: mkdir d4/d13/d2a 0 2024-02-15T18:34:52.230 INFO:tasks.workunit.client.0.smithi038.stdout:1/70: chown d2/l5 739 1 2024-02-15T18:34:52.232 INFO:tasks.workunit.client.0.smithi038.stdout:2/76: unlink dd/d12/d18/f1e 0 2024-02-15T18:34:52.235 INFO:tasks.workunit.client.0.smithi038.stdout:0/136: rename d4/d1b/c26 to d4/d13/d2a/c2b 0 2024-02-15T18:34:52.235 INFO:tasks.workunit.client.0.smithi038.stdout:0/137: fsync d4/fa 0 2024-02-15T18:34:52.235 INFO:tasks.workunit.client.0.smithi038.stdout:0/138: chown d4/d1b/c24 12699 1 2024-02-15T18:34:52.235 INFO:tasks.workunit.client.0.smithi038.stdout:1/71: mknod d2/d10/c17 0 2024-02-15T18:34:52.235 INFO:tasks.workunit.client.0.smithi038.stdout:2/77: mkdir dd/d12/d18/d1f 0 2024-02-15T18:34:52.237 INFO:tasks.workunit.client.0.smithi038.stdout:1/72: mknod d2/d10/c18 0 2024-02-15T18:34:52.237 INFO:tasks.workunit.client.0.smithi038.stdout:0/139: mknod d4/d13/d19/c2c 0 2024-02-15T18:34:52.240 INFO:tasks.workunit.client.0.smithi038.stdout:0/140: unlink d4/l6 0 2024-02-15T18:34:52.240 INFO:tasks.workunit.client.0.smithi038.stdout:0/141: fsync d4/fa 0 2024-02-15T18:34:52.241 INFO:tasks.workunit.client.0.smithi038.stdout:0/142: creat d4/d13/d2a/f2d x:0 0 0 2024-02-15T18:34:52.242 INFO:tasks.workunit.client.0.smithi038.stdout:7/58: dwrite d0/f1 [0,4194304] 0 2024-02-15T18:34:52.242 INFO:tasks.workunit.client.0.smithi038.stdout:7/59: getdents d0/d10 0 2024-02-15T18:34:52.242 INFO:tasks.workunit.client.0.smithi038.stdout:7/60: rename d0/d10 to d0/d10/d13 22 2024-02-15T18:34:52.243 INFO:tasks.workunit.client.0.smithi038.stdout:0/143: symlink d4/l2e 0 2024-02-15T18:34:52.243 INFO:tasks.workunit.client.0.smithi038.stdout:9/58: dwrite f0 [0,4194304] 0 2024-02-15T18:34:52.243 INFO:tasks.workunit.client.0.smithi038.stdout:9/59: chown d1/l4 5191612 1 2024-02-15T18:34:52.244 INFO:tasks.workunit.client.0.smithi038.stdout:9/60: read d1/f9 [2669345,25292] 0 2024-02-15T18:34:52.244 INFO:tasks.workunit.client.0.smithi038.stdout:9/61: fdatasync d1/f14 0 2024-02-15T18:34:52.244 INFO:tasks.workunit.client.0.smithi038.stdout:9/62: chown d1/da/d10/d13 138122581 1 2024-02-15T18:34:52.244 INFO:tasks.workunit.client.0.smithi038.stdout:2/78: write fb [2124630,69294] 0 2024-02-15T18:34:52.247 INFO:tasks.workunit.client.0.smithi038.stdout:0/144: dread d4/f5 [0,4194304] 0 2024-02-15T18:34:52.247 INFO:tasks.workunit.client.0.smithi038.stdout:0/145: write d4/f5 [3862210,11367] 0 2024-02-15T18:34:52.248 INFO:tasks.workunit.client.0.smithi038.stdout:1/73: rmdir d2 39 2024-02-15T18:34:52.251 INFO:tasks.workunit.client.0.smithi038.stdout:1/74: mkdir d2/d10/d19 0 2024-02-15T18:34:52.257 INFO:tasks.workunit.client.0.smithi038.stdout:0/146: dread d4/f15 [0,4194304] 0 2024-02-15T18:34:52.257 INFO:tasks.workunit.client.0.smithi038.stdout:0/147: write d4/f15 [356901,107233] 0 2024-02-15T18:34:52.261 INFO:tasks.workunit.client.0.smithi038.stdout:7/61: dread d0/fa [0,4194304] 0 2024-02-15T18:34:52.261 INFO:tasks.workunit.client.0.smithi038.stdout:7/62: fdatasync d0/db/ff 0 2024-02-15T18:34:52.261 INFO:tasks.workunit.client.0.smithi038.stdout:1/75: read d2/f4 [821239,98666] 0 2024-02-15T18:34:52.262 INFO:tasks.workunit.client.0.smithi038.stdout:7/63: mknod d0/d3/c14 0 2024-02-15T18:34:52.263 INFO:tasks.workunit.client.0.smithi038.stdout:1/76: unlink d2/l5 0 2024-02-15T18:34:52.264 INFO:tasks.workunit.client.0.smithi038.stdout:7/64: mkdir d0/d3/d15 0 2024-02-15T18:34:52.264 INFO:tasks.workunit.client.0.smithi038.stdout:9/63: dwrite d1/f14 [0,4194304] 0 2024-02-15T18:34:52.264 INFO:tasks.workunit.client.0.smithi038.stdout:2/79: dread dd/d12/d1d/f1a [0,4194304] 0 2024-02-15T18:34:52.264 INFO:tasks.workunit.client.0.smithi038.stdout:2/80: truncate f7 4422280 0 2024-02-15T18:34:52.264 INFO:tasks.workunit.client.0.smithi038.stdout:2/81: rmdir dd 39 2024-02-15T18:34:52.266 INFO:tasks.workunit.client.0.smithi038.stdout:9/64: mknod d1/da/c15 0 2024-02-15T18:34:52.266 INFO:tasks.workunit.client.0.smithi038.stdout:9/65: write d1/fb [4236447,120604] 0 2024-02-15T18:34:52.270 INFO:tasks.workunit.client.0.smithi038.stdout:7/65: dread d0/f9 [0,4194304] 0 2024-02-15T18:34:52.271 INFO:tasks.workunit.client.0.smithi038.stdout:7/66: write d0/fa [5114927,109161] 0 2024-02-15T18:34:52.275 INFO:tasks.workunit.client.0.smithi038.stdout:0/148: dwrite d4/d1b/d1e/f20 [0,4194304] 0 2024-02-15T18:34:52.277 INFO:tasks.workunit.client.0.smithi038.stdout:1/77: dread d2/ff [0,4194304] 0 2024-02-15T18:34:52.284 INFO:tasks.workunit.client.0.smithi038.stdout:9/66: dread d1/fb [0,4194304] 0 2024-02-15T18:34:52.284 INFO:tasks.workunit.client.0.smithi038.stdout:9/67: fsync d1/fb 0 2024-02-15T18:34:52.285 INFO:tasks.workunit.client.0.smithi038.stdout:3/52: sync 2024-02-15T18:34:52.285 INFO:tasks.workunit.client.0.smithi038.stdout:4/39: sync 2024-02-15T18:34:52.285 INFO:tasks.workunit.client.0.smithi038.stdout:5/70: sync 2024-02-15T18:34:52.286 INFO:tasks.workunit.client.0.smithi038.stdout:4/40: chown cb 1 1 2024-02-15T18:34:52.286 INFO:tasks.workunit.client.0.smithi038.stdout:4/41: chown f2 657 1 2024-02-15T18:34:52.286 INFO:tasks.workunit.client.0.smithi038.stdout:9/68: rmdir d1 39 2024-02-15T18:34:52.286 INFO:tasks.workunit.client.0.smithi038.stdout:7/67: dwrite d0/db/fc [0,4194304] 0 2024-02-15T18:34:52.288 INFO:tasks.workunit.client.0.smithi038.stdout:3/53: unlink d2/f7 0 2024-02-15T18:34:52.288 INFO:tasks.workunit.client.0.smithi038.stdout:9/69: write f0 [803077,101185] 0 2024-02-15T18:34:52.288 INFO:tasks.workunit.client.0.smithi038.stdout:3/54: stat d2/fa 0 2024-02-15T18:34:52.288 INFO:tasks.workunit.client.0.smithi038.stdout:9/70: readlink d1/l6 0 2024-02-15T18:34:52.288 INFO:tasks.workunit.client.0.smithi038.stdout:4/42: mknod d11/c12 0 2024-02-15T18:34:52.289 INFO:tasks.workunit.client.0.smithi038.stdout:9/71: dread d1/f9 [4194304,4194304] 0 2024-02-15T18:34:52.291 INFO:tasks.workunit.client.0.smithi038.stdout:7/68: symlink d0/d10/l16 0 2024-02-15T18:34:52.291 INFO:tasks.workunit.client.0.smithi038.stdout:7/69: chown d0/d3 1 1 2024-02-15T18:34:52.291 INFO:tasks.workunit.client.0.smithi038.stdout:7/70: readlink d0/d10/l16 0 2024-02-15T18:34:52.291 INFO:tasks.workunit.client.0.smithi038.stdout:7/71: readlink d0/d3/l6 0 2024-02-15T18:34:52.293 INFO:tasks.workunit.client.0.smithi038.stdout:4/43: rename f9 to d11/f13 0 2024-02-15T18:34:52.293 INFO:tasks.workunit.client.0.smithi038.stdout:4/44: dread - f5 zero size 2024-02-15T18:34:52.293 INFO:tasks.workunit.client.0.smithi038.stdout:5/71: dread d2/fc [0,4194304] 0 2024-02-15T18:34:52.293 INFO:tasks.workunit.client.0.smithi038.stdout:4/45: chown f2 54 1 2024-02-15T18:34:52.295 INFO:tasks.workunit.client.0.smithi038.stdout:4/46: creat d11/f14 x:0 0 0 2024-02-15T18:34:52.295 INFO:tasks.workunit.client.0.smithi038.stdout:4/47: dread - fc zero size 2024-02-15T18:34:52.295 INFO:tasks.workunit.client.0.smithi038.stdout:4/48: dread - d11/f13 zero size 2024-02-15T18:34:52.295 INFO:tasks.workunit.client.0.smithi038.stdout:4/49: read - f7 zero size 2024-02-15T18:34:52.301 INFO:tasks.workunit.client.0.smithi038.stdout:2/82: dread fb [0,4194304] 0 2024-02-15T18:34:52.301 INFO:tasks.workunit.client.0.smithi038.stdout:2/83: readlink dd/d12/d1d/l1b 0 2024-02-15T18:34:52.301 INFO:tasks.workunit.client.0.smithi038.stdout:2/84: fdatasync fc 0 2024-02-15T18:34:52.307 INFO:tasks.workunit.client.0.smithi038.stdout:2/85: dread fb [0,4194304] 0 2024-02-15T18:34:52.309 INFO:tasks.workunit.client.0.smithi038.stdout:2/86: truncate dd/d12/d18/f1c 1688210 0 2024-02-15T18:34:52.310 INFO:tasks.workunit.client.0.smithi038.stdout:0/149: dwrite d4/fa [0,4194304] 0 2024-02-15T18:34:52.311 INFO:tasks.workunit.client.0.smithi038.stdout:2/87: rename dd/d12/d13/l14 to dd/d12/l20 0 2024-02-15T18:34:52.311 INFO:tasks.workunit.client.0.smithi038.stdout:2/88: fsync dd/d12/d1d/f1a 0 2024-02-15T18:34:52.313 INFO:tasks.workunit.client.0.smithi038.stdout:0/150: unlink d4/d1b/c24 0 2024-02-15T18:34:52.313 INFO:tasks.workunit.client.0.smithi038.stdout:9/72: dwrite d1/f14 [4194304,4194304] 0 2024-02-15T18:34:52.313 INFO:tasks.workunit.client.0.smithi038.stdout:2/89: creat dd/d12/d18/f21 x:0 0 0 2024-02-15T18:34:52.314 INFO:tasks.workunit.client.0.smithi038.stdout:1/78: dwrite d2/f3 [0,4194304] 0 2024-02-15T18:34:52.315 INFO:tasks.workunit.client.0.smithi038.stdout:4/50: dread f2 [0,4194304] 0 2024-02-15T18:34:52.315 INFO:tasks.workunit.client.0.smithi038.stdout:9/73: write d1/f14 [571767,76578] 0 2024-02-15T18:34:52.315 INFO:tasks.workunit.client.0.smithi038.stdout:9/74: write d1/dc/fe [869864,32948] 0 2024-02-15T18:34:52.316 INFO:tasks.workunit.client.0.smithi038.stdout:0/151: rename d4/d1b/d1e/l1f to d4/d13/l2f 0 2024-02-15T18:34:52.316 INFO:tasks.workunit.client.0.smithi038.stdout:2/90: write dd/d12/d13/f19 [2267228,20952] 0 2024-02-15T18:34:52.316 INFO:tasks.workunit.client.0.smithi038.stdout:2/91: write dd/d12/d18/f21 [527678,125907] 0 2024-02-15T18:34:52.316 INFO:tasks.workunit.client.0.smithi038.stdout:9/75: write d1/fb [2133813,55721] 0 2024-02-15T18:34:52.323 INFO:tasks.workunit.client.0.smithi038.stdout:7/72: dwrite d0/fa [0,4194304] 0 2024-02-15T18:34:52.323 INFO:tasks.workunit.client.0.smithi038.stdout:7/73: dread - d0/db/ff zero size 2024-02-15T18:34:52.324 INFO:tasks.workunit.client.0.smithi038.stdout:4/51: getdents d11 0 2024-02-15T18:34:52.326 INFO:tasks.workunit.client.0.smithi038.stdout:9/76: creat d1/f16 x:0 0 0 2024-02-15T18:34:52.326 INFO:tasks.workunit.client.0.smithi038.stdout:7/74: symlink d0/db/l17 0 2024-02-15T18:34:52.327 INFO:tasks.workunit.client.0.smithi038.stdout:7/75: fdatasync d0/db/fc 0 2024-02-15T18:34:52.327 INFO:tasks.workunit.client.0.smithi038.stdout:1/79: symlink d2/l1a 0 2024-02-15T18:34:52.328 INFO:tasks.workunit.client.0.smithi038.stdout:9/77: link d1/fb d1/da/f17 0 2024-02-15T18:34:52.328 INFO:tasks.workunit.client.0.smithi038.stdout:9/78: getdents d1/d11 0 2024-02-15T18:34:52.329 INFO:tasks.workunit.client.0.smithi038.stdout:7/76: creat d0/d3/d15/f18 x:0 0 0 2024-02-15T18:34:52.330 INFO:tasks.workunit.client.0.smithi038.stdout:1/80: creat d2/d10/f1b x:0 0 0 2024-02-15T18:34:52.330 INFO:tasks.workunit.client.0.smithi038.stdout:4/52: truncate f3 1460081 0 2024-02-15T18:34:52.330 INFO:tasks.workunit.client.0.smithi038.stdout:4/53: chown f4 175 1 2024-02-15T18:34:52.331 INFO:tasks.workunit.client.0.smithi038.stdout:9/79: creat d1/da/d10/d13/f18 x:0 0 0 2024-02-15T18:34:52.331 INFO:tasks.workunit.client.0.smithi038.stdout:9/80: chown d1/fb 316 1 2024-02-15T18:34:52.332 INFO:tasks.workunit.client.0.smithi038.stdout:7/77: mkdir d0/db/d19 0 2024-02-15T18:34:52.332 INFO:tasks.workunit.client.0.smithi038.stdout:0/152: dwrite d4/f12 [0,4194304] 0 2024-02-15T18:34:52.335 INFO:tasks.workunit.client.0.smithi038.stdout:2/92: dread fb [0,4194304] 0 2024-02-15T18:34:52.337 INFO:tasks.workunit.client.0.smithi038.stdout:8/200: sync 2024-02-15T18:34:52.337 INFO:tasks.workunit.client.0.smithi038.stdout:6/71: sync 2024-02-15T18:34:52.337 INFO:tasks.workunit.client.0.smithi038.stdout:6/72: chown d2 6492282 1 2024-02-15T18:34:52.337 INFO:tasks.workunit.client.0.smithi038.stdout:8/201: chown d1/d3a 12537 1 2024-02-15T18:34:52.339 INFO:tasks.workunit.client.0.smithi038.stdout:4/54: mknod d11/c15 0 2024-02-15T18:34:52.340 INFO:tasks.workunit.client.0.smithi038.stdout:4/55: read - d11/f14 zero size 2024-02-15T18:34:52.341 INFO:tasks.workunit.client.0.smithi038.stdout:9/81: rename d1/df to d1/da/d10/d19 0 2024-02-15T18:34:52.341 INFO:tasks.workunit.client.0.smithi038.stdout:7/78: creat d0/d3/f1a x:0 0 0 2024-02-15T18:34:52.342 INFO:tasks.workunit.client.0.smithi038.stdout:7/79: chown d0/d3/c14 2546 1 2024-02-15T18:34:52.349 INFO:tasks.workunit.client.0.smithi038.stdout:0/153: symlink d4/d13/l30 0 2024-02-15T18:34:52.350 INFO:tasks.workunit.client.0.smithi038.stdout:2/93: creat dd/d12/f22 x:0 0 0 2024-02-15T18:34:52.351 INFO:tasks.workunit.client.0.smithi038.stdout:5/72: sync 2024-02-15T18:34:52.351 INFO:tasks.workunit.client.0.smithi038.stdout:3/55: sync 2024-02-15T18:34:52.352 INFO:tasks.workunit.client.0.smithi038.stdout:1/81: mknod d2/c1c 0 2024-02-15T18:34:52.354 INFO:tasks.workunit.client.0.smithi038.stdout:6/73: rename d2/fe to d2/f17 0 2024-02-15T18:34:52.356 INFO:tasks.workunit.client.0.smithi038.stdout:8/202: creat d1/d4/d10/d43/f46 x:0 0 0 2024-02-15T18:34:52.356 INFO:tasks.workunit.client.0.smithi038.stdout:4/56: mknod d11/c16 0 2024-02-15T18:34:52.358 INFO:tasks.workunit.client.0.smithi038.stdout:7/80: mkdir d0/d1b 0 2024-02-15T18:34:52.359 INFO:tasks.workunit.client.0.smithi038.stdout:9/82: dwrite d1/f9 [0,4194304] 0 2024-02-15T18:34:52.359 INFO:tasks.workunit.client.0.smithi038.stdout:0/154: getdents d4/d13/d2a 0 2024-02-15T18:34:52.360 INFO:tasks.workunit.client.0.smithi038.stdout:3/56: truncate d2/f9 834478 0 2024-02-15T18:34:52.360 INFO:tasks.workunit.client.0.smithi038.stdout:1/82: truncate f1 2002910 0 2024-02-15T18:34:52.361 INFO:tasks.workunit.client.0.smithi038.stdout:1/83: dread - d2/d10/f14 zero size 2024-02-15T18:34:52.362 INFO:tasks.workunit.client.0.smithi038.stdout:6/74: rename d2/cb to d2/d10/c18 0 2024-02-15T18:34:52.363 INFO:tasks.workunit.client.0.smithi038.stdout:2/94: dread dd/d12/d18/f21 [0,4194304] 0 2024-02-15T18:34:52.363 INFO:tasks.workunit.client.0.smithi038.stdout:2/95: chown dd/d12/d1d/l1b 181430 1 2024-02-15T18:34:52.363 INFO:tasks.workunit.client.0.smithi038.stdout:8/203: mkdir d1/d47 0 2024-02-15T18:34:52.363 INFO:tasks.workunit.client.0.smithi038.stdout:8/204: getdents d1/d47 0 2024-02-15T18:34:52.364 INFO:tasks.workunit.client.0.smithi038.stdout:4/57: unlink d11/c12 0 2024-02-15T18:34:52.365 INFO:tasks.workunit.client.0.smithi038.stdout:7/81: creat d0/d10/f1c x:0 0 0 2024-02-15T18:34:52.365 INFO:tasks.workunit.client.0.smithi038.stdout:9/83: mknod d1/dc/c1a 0 2024-02-15T18:34:52.367 INFO:tasks.workunit.client.0.smithi038.stdout:6/75: mkdir d2/d4/d19 0 2024-02-15T18:34:52.367 INFO:tasks.workunit.client.0.smithi038.stdout:6/76: fdatasync d2/d4/fc 0 2024-02-15T18:34:52.368 INFO:tasks.workunit.client.0.smithi038.stdout:2/96: rename f7 to dd/d12/d13/f23 0 2024-02-15T18:34:52.368 INFO:tasks.workunit.client.0.smithi038.stdout:2/97: truncate dd/d12/d18/f1c 1724018 0 2024-02-15T18:34:52.369 INFO:tasks.workunit.client.0.smithi038.stdout:4/58: creat d11/f17 x:0 0 0 2024-02-15T18:34:52.372 INFO:tasks.workunit.client.0.smithi038.stdout:0/155: mknod d4/d13/d19/c31 0 2024-02-15T18:34:52.372 INFO:tasks.workunit.client.0.smithi038.stdout:8/205: dread d1/d4/d5/d6/ff [4194304,4194304] 0 2024-02-15T18:34:52.373 INFO:tasks.workunit.client.0.smithi038.stdout:7/82: dread d0/fa [4194304,4194304] 0 2024-02-15T18:34:52.373 INFO:tasks.workunit.client.0.smithi038.stdout:7/83: readlink d0/d10/l16 0 2024-02-15T18:34:52.374 INFO:tasks.workunit.client.0.smithi038.stdout:5/73: dwrite d2/d3/ff [0,4194304] 0 2024-02-15T18:34:52.378 INFO:tasks.workunit.client.0.smithi038.stdout:9/84: truncate d1/da/f17 2691999 0 2024-02-15T18:34:52.379 INFO:tasks.workunit.client.0.smithi038.stdout:3/57: dread d2/f5 [0,4194304] 0 2024-02-15T18:34:52.379 INFO:tasks.workunit.client.0.smithi038.stdout:3/58: chown d2 3540827 1 2024-02-15T18:34:52.379 INFO:tasks.workunit.client.0.smithi038.stdout:4/59: link f7 d11/f18 0 2024-02-15T18:34:52.380 INFO:tasks.workunit.client.0.smithi038.stdout:4/60: stat d11/f14 0 2024-02-15T18:34:52.380 INFO:tasks.workunit.client.0.smithi038.stdout:2/98: creat dd/f24 x:0 0 0 2024-02-15T18:34:52.380 INFO:tasks.workunit.client.0.smithi038.stdout:0/156: getdents d4/d1b 0 2024-02-15T18:34:52.381 INFO:tasks.workunit.client.0.smithi038.stdout:8/206: mkdir d1/d4/d10/d43/d48 0 2024-02-15T18:34:52.383 INFO:tasks.workunit.client.0.smithi038.stdout:7/84: mkdir d0/db/d19/d1d 0 2024-02-15T18:34:52.383 INFO:tasks.workunit.client.0.smithi038.stdout:8/207: dread d1/d7/d15/f2f [0,4194304] 0 2024-02-15T18:34:52.383 INFO:tasks.workunit.client.0.smithi038.stdout:8/208: write d1/d4/d5/f27 [3368632,7846] 0 2024-02-15T18:34:52.384 INFO:tasks.workunit.client.0.smithi038.stdout:1/84: dwrite d2/f3 [0,4194304] 0 2024-02-15T18:34:52.385 INFO:tasks.workunit.client.0.smithi038.stdout:1/85: dread d2/f11 [0,4194304] 0 2024-02-15T18:34:52.389 INFO:tasks.workunit.client.0.smithi038.stdout:5/74: mknod d2/d4/c10 0 2024-02-15T18:34:52.390 INFO:tasks.workunit.client.0.smithi038.stdout:3/59: unlink d2/f5 0 2024-02-15T18:34:52.390 INFO:tasks.workunit.client.0.smithi038.stdout:3/60: read d2/f9 [493157,10758] 0 2024-02-15T18:34:52.391 INFO:tasks.workunit.client.0.smithi038.stdout:4/61: creat d11/f19 x:0 0 0 2024-02-15T18:34:52.391 INFO:tasks.workunit.client.0.smithi038.stdout:4/62: chown l8 73824646 1 2024-02-15T18:34:52.392 INFO:tasks.workunit.client.0.smithi038.stdout:6/77: dwrite d2/f3 [0,4194304] 0 2024-02-15T18:34:52.392 INFO:tasks.workunit.client.0.smithi038.stdout:0/157: mkdir d4/d1b/d1e/d32 0 2024-02-15T18:34:52.395 INFO:tasks.workunit.client.0.smithi038.stdout:7/85: unlink d0/f1 0 2024-02-15T18:34:52.395 INFO:tasks.workunit.client.0.smithi038.stdout:9/85: dread f0 [0,4194304] 0 2024-02-15T18:34:52.396 INFO:tasks.workunit.client.0.smithi038.stdout:1/86: rename d2/d10/f14 to d2/d10/d19/f1d 0 2024-02-15T18:34:52.397 INFO:tasks.workunit.client.0.smithi038.stdout:1/87: write d2/ff [856398,50410] 0 2024-02-15T18:34:52.397 INFO:tasks.workunit.client.0.smithi038.stdout:1/88: fdatasync d2/d10/d19/f1d 0 2024-02-15T18:34:52.399 INFO:tasks.workunit.client.0.smithi038.stdout:5/75: unlink d2/d4/c10 0 2024-02-15T18:34:52.399 INFO:tasks.workunit.client.0.smithi038.stdout:5/76: stat d2/d3/f7 0 2024-02-15T18:34:52.399 INFO:tasks.workunit.client.0.smithi038.stdout:5/77: write d2/fd [133894,3489] 0 2024-02-15T18:34:52.400 INFO:tasks.workunit.client.0.smithi038.stdout:3/61: rmdir d2/db 0 2024-02-15T18:34:52.401 INFO:tasks.workunit.client.0.smithi038.stdout:3/62: read d2/f9 [403298,124181] 0 2024-02-15T18:34:52.402 INFO:tasks.workunit.client.0.smithi038.stdout:4/63: creat d11/f1a x:0 0 0 2024-02-15T18:34:52.402 INFO:tasks.workunit.client.0.smithi038.stdout:4/64: dread - d11/f13 zero size 2024-02-15T18:34:52.402 INFO:tasks.workunit.client.0.smithi038.stdout:4/65: write d11/f14 [121530,62012] 0 2024-02-15T18:34:52.402 INFO:tasks.workunit.client.0.smithi038.stdout:2/99: symlink dd/l25 0 2024-02-15T18:34:52.403 INFO:tasks.workunit.client.0.smithi038.stdout:8/209: dwrite d1/d4/d5/d31/f39 [4194304,4194304] 0 2024-02-15T18:34:52.405 INFO:tasks.workunit.client.0.smithi038.stdout:6/78: getdents d2 0 2024-02-15T18:34:52.405 INFO:tasks.workunit.client.0.smithi038.stdout:6/79: chown d2/f9 1289 1 2024-02-15T18:34:52.405 INFO:tasks.workunit.client.0.smithi038.stdout:6/80: fdatasync d2/d10/f15 0 2024-02-15T18:34:52.407 INFO:tasks.workunit.client.0.smithi038.stdout:0/158: readlink d4/lb 0 2024-02-15T18:34:52.407 INFO:tasks.workunit.client.0.smithi038.stdout:7/86: truncate d0/db/fc 2693970 0 2024-02-15T18:34:52.410 INFO:tasks.workunit.client.0.smithi038.stdout:9/86: truncate d1/f14 7283824 0 2024-02-15T18:34:52.410 INFO:tasks.workunit.client.0.smithi038.stdout:9/87: fsync d1/fb 0 2024-02-15T18:34:52.410 INFO:tasks.workunit.client.0.smithi038.stdout:9/88: write f0 [4411532,26046] 0 2024-02-15T18:34:52.417 INFO:tasks.workunit.client.0.smithi038.stdout:1/89: creat d2/f1e x:0 0 0 2024-02-15T18:34:52.417 INFO:tasks.workunit.client.0.smithi038.stdout:1/90: chown d2/l1a 5704 1 2024-02-15T18:34:52.418 INFO:tasks.workunit.client.0.smithi038.stdout:5/78: creat d2/d3/f11 x:0 0 0 2024-02-15T18:34:52.419 INFO:tasks.workunit.client.0.smithi038.stdout:4/66: dwrite fc [0,4194304] 0 2024-02-15T18:34:52.419 INFO:tasks.workunit.client.0.smithi038.stdout:4/67: write f5 [300460,5464] 0 2024-02-15T18:34:52.420 INFO:tasks.workunit.client.0.smithi038.stdout:3/63: symlink d2/lc 0 2024-02-15T18:34:52.420 INFO:tasks.workunit.client.0.smithi038.stdout:2/100: creat dd/d12/f26 x:0 0 0 2024-02-15T18:34:52.423 INFO:tasks.workunit.client.0.smithi038.stdout:0/159: mknod d4/d1b/d1e/c33 0 2024-02-15T18:34:52.426 INFO:tasks.workunit.client.0.smithi038.stdout:7/87: symlink d0/db/d19/d1d/l1e 0 2024-02-15T18:34:52.427 INFO:tasks.workunit.client.0.smithi038.stdout:1/91: symlink d2/d10/l1f 0 2024-02-15T18:34:52.427 INFO:tasks.workunit.client.0.smithi038.stdout:7/88: stat d0/db 0 2024-02-15T18:34:52.427 INFO:tasks.workunit.client.0.smithi038.stdout:7/89: dread - d0/d10/f1c zero size 2024-02-15T18:34:52.429 INFO:tasks.workunit.client.0.smithi038.stdout:5/79: rename d2/d4 to d2/d3/d12 0 2024-02-15T18:34:52.436 INFO:tasks.workunit.client.0.smithi038.stdout:3/64: creat d2/fd x:0 0 0 2024-02-15T18:34:52.436 INFO:tasks.workunit.client.0.smithi038.stdout:0/160: mknod d4/d1b/c34 0 2024-02-15T18:34:52.436 INFO:tasks.workunit.client.0.smithi038.stdout:5/80: mknod d2/c13 0 2024-02-15T18:34:52.438 INFO:tasks.workunit.client.0.smithi038.stdout:1/92: rename d2/d10/l1f to d2/d10/d19/l20 0 2024-02-15T18:34:52.438 INFO:tasks.workunit.client.0.smithi038.stdout:1/93: dread - d2/d10/d19/f1d zero size 2024-02-15T18:34:52.438 INFO:tasks.workunit.client.0.smithi038.stdout:1/94: write d2/f1e [1002236,65313] 0 2024-02-15T18:34:52.439 INFO:tasks.workunit.client.0.smithi038.stdout:3/65: creat d2/fe x:0 0 0 2024-02-15T18:34:52.439 INFO:tasks.workunit.client.0.smithi038.stdout:3/66: stat d2/fe 0 2024-02-15T18:34:52.439 INFO:tasks.workunit.client.0.smithi038.stdout:3/67: truncate d2/fe 2578 0 2024-02-15T18:34:52.441 INFO:tasks.workunit.client.0.smithi038.stdout:5/81: creat d2/d3/d12/f14 x:0 0 0 2024-02-15T18:34:52.444 INFO:tasks.workunit.client.0.smithi038.stdout:1/95: rename d2/lb to d2/d10/l21 0 2024-02-15T18:34:52.445 INFO:tasks.workunit.client.0.smithi038.stdout:0/161: dread d4/f5 [0,4194304] 0 2024-02-15T18:34:52.445 INFO:tasks.workunit.client.0.smithi038.stdout:0/162: write d4/f5 [3005940,102538] 0 2024-02-15T18:34:52.445 INFO:tasks.workunit.client.0.smithi038.stdout:1/96: readlink d2/l9 0 2024-02-15T18:34:52.445 INFO:tasks.workunit.client.0.smithi038.stdout:1/97: chown d2/l1a 7 1 2024-02-15T18:34:52.446 INFO:tasks.workunit.client.0.smithi038.stdout:0/163: getdents d4/d13/d19 0 2024-02-15T18:34:52.446 INFO:tasks.workunit.client.0.smithi038.stdout:0/164: fdatasync d4/ff 0 2024-02-15T18:34:52.447 INFO:tasks.workunit.client.0.smithi038.stdout:0/165: dread d4/ff [0,4194304] 0 2024-02-15T18:34:52.447 INFO:tasks.workunit.client.0.smithi038.stdout:1/98: mkdir d2/d10/d19/d22 0 2024-02-15T18:34:52.448 INFO:tasks.workunit.client.0.smithi038.stdout:0/166: read d4/f15 [3640307,126236] 0 2024-02-15T18:34:52.449 INFO:tasks.workunit.client.0.smithi038.stdout:1/99: rename d2/d10/c12 to d2/d10/d19/d22/c23 0 2024-02-15T18:34:52.450 INFO:tasks.workunit.client.0.smithi038.stdout:1/100: stat d2/d10/l21 0 2024-02-15T18:34:52.450 INFO:tasks.workunit.client.0.smithi038.stdout:8/210: dwrite d1/d4/d5/f27 [0,4194304] 0 2024-02-15T18:34:52.451 INFO:tasks.workunit.client.0.smithi038.stdout:1/101: link d2/d10/c17 d2/d10/c24 0 2024-02-15T18:34:52.453 INFO:tasks.workunit.client.0.smithi038.stdout:8/211: symlink d1/d4/d10/d43/l49 0 2024-02-15T18:34:52.455 INFO:tasks.workunit.client.0.smithi038.stdout:8/212: link d1/d7/d15/f36 d1/f4a 0 2024-02-15T18:34:52.459 INFO:tasks.workunit.client.0.smithi038.stdout:7/90: dwrite d0/d3/f7 [0,4194304] 0 2024-02-15T18:34:52.459 INFO:tasks.workunit.client.0.smithi038.stdout:6/81: dwrite d2/d4/ff [0,4194304] 0 2024-02-15T18:34:52.459 INFO:tasks.workunit.client.0.smithi038.stdout:9/89: dwrite f0 [0,4194304] 0 2024-02-15T18:34:52.459 INFO:tasks.workunit.client.0.smithi038.stdout:7/91: rmdir d0/db/d19 39 2024-02-15T18:34:52.460 INFO:tasks.workunit.client.0.smithi038.stdout:9/90: write d1/da/f17 [1604174,54982] 0 2024-02-15T18:34:52.462 INFO:tasks.workunit.client.0.smithi038.stdout:7/92: creat d0/d3/d15/f1f x:0 0 0 2024-02-15T18:34:52.463 INFO:tasks.workunit.client.0.smithi038.stdout:5/82: write d2/d3/ff [2313412,31503] 0 2024-02-15T18:34:52.463 INFO:tasks.workunit.client.0.smithi038.stdout:2/101: dwrite fb [0,4194304] 0 2024-02-15T18:34:52.464 INFO:tasks.workunit.client.0.smithi038.stdout:2/102: write dd/d12/d18/f21 [1212744,84721] 0 2024-02-15T18:34:52.464 INFO:tasks.workunit.client.0.smithi038.stdout:9/91: mknod d1/da/c1b 0 2024-02-15T18:34:52.465 INFO:tasks.workunit.client.0.smithi038.stdout:7/93: creat d0/db/f20 x:0 0 0 2024-02-15T18:34:52.465 INFO:tasks.workunit.client.0.smithi038.stdout:7/94: dread - d0/db/ff zero size 2024-02-15T18:34:52.467 INFO:tasks.workunit.client.0.smithi038.stdout:2/103: creat dd/d12/d18/d1f/f27 x:0 0 0 2024-02-15T18:34:52.468 INFO:tasks.workunit.client.0.smithi038.stdout:5/83: dread d2/d3/f7 [0,4194304] 0 2024-02-15T18:34:52.468 INFO:tasks.workunit.client.0.smithi038.stdout:4/68: dwrite f5 [0,4194304] 0 2024-02-15T18:34:52.469 INFO:tasks.workunit.client.0.smithi038.stdout:9/92: creat d1/f1c x:0 0 0 2024-02-15T18:34:52.469 INFO:tasks.workunit.client.0.smithi038.stdout:9/93: readlink d1/l4 0 2024-02-15T18:34:52.470 INFO:tasks.workunit.client.0.smithi038.stdout:7/95: creat d0/f21 x:0 0 0 2024-02-15T18:34:52.470 INFO:tasks.workunit.client.0.smithi038.stdout:7/96: fsync d0/db/ff 0 2024-02-15T18:34:52.470 INFO:tasks.workunit.client.0.smithi038.stdout:2/104: creat dd/d12/d13/f28 x:0 0 0 2024-02-15T18:34:52.477 INFO:tasks.workunit.client.0.smithi038.stdout:5/84: rename d2/d3/f11 to d2/d3/f15 0 2024-02-15T18:34:52.477 INFO:tasks.workunit.client.0.smithi038.stdout:5/85: dread - d2/d3/d12/f14 zero size 2024-02-15T18:34:52.477 INFO:tasks.workunit.client.0.smithi038.stdout:5/86: chown d2/d3/ff 106749213 1 2024-02-15T18:34:52.478 INFO:tasks.workunit.client.0.smithi038.stdout:9/94: mknod d1/da/d10/d13/c1d 0 2024-02-15T18:34:52.478 INFO:tasks.workunit.client.0.smithi038.stdout:9/95: readlink d1/l4 0 2024-02-15T18:34:52.478 INFO:tasks.workunit.client.0.smithi038.stdout:5/87: dread d2/fd [0,4194304] 0 2024-02-15T18:34:52.478 INFO:tasks.workunit.client.0.smithi038.stdout:5/88: dread - d2/d3/d12/fe zero size 2024-02-15T18:34:52.478 INFO:tasks.workunit.client.0.smithi038.stdout:1/102: dwrite d2/f11 [0,4194304] 0 2024-02-15T18:34:52.479 INFO:tasks.workunit.client.0.smithi038.stdout:1/103: chown d2/d10/d19 7 1 2024-02-15T18:34:52.479 INFO:tasks.workunit.client.0.smithi038.stdout:1/104: fdatasync d2/ff 0 2024-02-15T18:34:52.479 INFO:tasks.workunit.client.0.smithi038.stdout:7/97: rmdir d0/d10 39 2024-02-15T18:34:52.479 INFO:tasks.workunit.client.0.smithi038.stdout:7/98: chown d0/db/d19 7642745 1 2024-02-15T18:34:52.479 INFO:tasks.workunit.client.0.smithi038.stdout:7/99: dread - d0/db/f20 zero size 2024-02-15T18:34:52.482 INFO:tasks.workunit.client.0.smithi038.stdout:0/167: rename d4/d1b to d4/d13/d2a/d35 0 2024-02-15T18:34:52.483 INFO:tasks.workunit.client.0.smithi038.stdout:0/168: dread d4/ff [0,4194304] 0 2024-02-15T18:34:52.483 INFO:tasks.workunit.client.0.smithi038.stdout:4/69: dread f5 [0,4194304] 0 2024-02-15T18:34:52.488 INFO:tasks.workunit.client.0.smithi038.stdout:5/89: link d2/d3/c8 d2/d3/c16 0 2024-02-15T18:34:52.489 INFO:tasks.workunit.client.0.smithi038.stdout:4/70: dread d11/f18 [0,4194304] 0 2024-02-15T18:34:52.493 INFO:tasks.workunit.client.0.smithi038.stdout:9/96: dwrite d1/f14 [4194304,4194304] 0 2024-02-15T18:34:52.496 INFO:tasks.workunit.client.0.smithi038.stdout:2/105: rmdir dd 39 2024-02-15T18:34:52.500 INFO:tasks.workunit.client.0.smithi038.stdout:7/100: symlink d0/db/d19/d1d/l22 0 2024-02-15T18:34:52.504 INFO:tasks.workunit.client.0.smithi038.stdout:8/213: rename d1/d4/d5/d30 to d1/d7/d15/d4b 0 2024-02-15T18:34:52.504 INFO:tasks.workunit.client.0.smithi038.stdout:8/214: truncate d1/d4/d5/d6/f2a 774210 0 2024-02-15T18:34:52.505 INFO:tasks.workunit.client.0.smithi038.stdout:5/90: dwrite d2/d3/d12/fa [0,4194304] 0 2024-02-15T18:34:52.505 INFO:tasks.workunit.client.0.smithi038.stdout:0/169: symlink d4/d13/d19/l36 0 2024-02-15T18:34:52.505 INFO:tasks.workunit.client.0.smithi038.stdout:5/91: readlink d2/d3/l9 0 2024-02-15T18:34:52.505 INFO:tasks.workunit.client.0.smithi038.stdout:0/170: fdatasync d4/ff 0 2024-02-15T18:34:52.506 INFO:tasks.workunit.client.0.smithi038.stdout:4/71: creat d11/f1b x:0 0 0 2024-02-15T18:34:52.507 INFO:tasks.workunit.client.0.smithi038.stdout:9/97: creat d1/da/d10/d13/f1e x:0 0 0 2024-02-15T18:34:52.507 INFO:tasks.workunit.client.0.smithi038.stdout:9/98: dread - d1/f16 zero size 2024-02-15T18:34:52.508 INFO:tasks.workunit.client.0.smithi038.stdout:2/106: creat dd/d12/d18/d1f/f29 x:0 0 0 2024-02-15T18:34:52.510 INFO:tasks.workunit.client.0.smithi038.stdout:1/105: rename d2/l9 to d2/d10/d19/d22/l25 0 2024-02-15T18:34:52.510 INFO:tasks.workunit.client.0.smithi038.stdout:0/171: rename d4/d13/d19 to d4/d13/d19/d37 22 2024-02-15T18:34:52.510 INFO:tasks.workunit.client.0.smithi038.stdout:1/106: stat d2/f4 0 2024-02-15T18:34:52.510 INFO:tasks.workunit.client.0.smithi038.stdout:8/215: symlink d1/d4/d10/d43/l4c 0 2024-02-15T18:34:52.511 INFO:tasks.workunit.client.0.smithi038.stdout:0/172: write d4/fa [833226,54352] 0 2024-02-15T18:34:52.511 INFO:tasks.workunit.client.0.smithi038.stdout:0/173: readlink d4/lb 0 2024-02-15T18:34:52.512 INFO:tasks.workunit.client.0.smithi038.stdout:4/72: unlink f3 0 2024-02-15T18:34:52.512 INFO:tasks.workunit.client.0.smithi038.stdout:4/73: dread - d11/f13 zero size 2024-02-15T18:34:52.512 INFO:tasks.workunit.client.0.smithi038.stdout:4/74: fdatasync d11/f1b 0 2024-02-15T18:34:52.512 INFO:tasks.workunit.client.0.smithi038.stdout:4/75: dread - d11/f19 zero size 2024-02-15T18:34:52.512 INFO:tasks.workunit.client.0.smithi038.stdout:4/76: write d11/f14 [217414,73864] 0 2024-02-15T18:34:52.512 INFO:tasks.workunit.client.0.smithi038.stdout:9/99: creat d1/da/d10/f1f x:0 0 0 2024-02-15T18:34:52.512 INFO:tasks.workunit.client.0.smithi038.stdout:9/100: fsync d1/f1c 0 2024-02-15T18:34:52.513 INFO:tasks.workunit.client.0.smithi038.stdout:2/107: stat c6 0 2024-02-15T18:34:52.513 INFO:tasks.workunit.client.0.smithi038.stdout:1/107: creat d2/f26 x:0 0 0 2024-02-15T18:34:52.514 INFO:tasks.workunit.client.0.smithi038.stdout:8/216: getdents d1/d4 0 2024-02-15T18:34:52.515 INFO:tasks.workunit.client.0.smithi038.stdout:2/108: read dd/d12/d13/f19 [4154620,33564] 0 2024-02-15T18:34:52.515 INFO:tasks.workunit.client.0.smithi038.stdout:0/174: symlink d4/d13/l38 0 2024-02-15T18:34:52.517 INFO:tasks.workunit.client.0.smithi038.stdout:9/101: mkdir d1/da/d20 0 2024-02-15T18:34:52.517 INFO:tasks.workunit.client.0.smithi038.stdout:7/101: dwrite d0/d3/d15/f18 [0,4194304] 0 2024-02-15T18:34:52.517 INFO:tasks.workunit.client.0.smithi038.stdout:7/102: stat d0/d3/c11 0 2024-02-15T18:34:52.518 INFO:tasks.workunit.client.0.smithi038.stdout:7/103: fdatasync d0/db/fe 0 2024-02-15T18:34:52.521 INFO:tasks.workunit.client.0.smithi038.stdout:2/109: dread dd/d12/d13/f23 [0,4194304] 0 2024-02-15T18:34:52.522 INFO:tasks.workunit.client.0.smithi038.stdout:2/110: chown dd/d12/d13/f28 497718 1 2024-02-15T18:34:52.522 INFO:tasks.workunit.client.0.smithi038.stdout:8/217: creat d1/d4/d5/f4d x:0 0 0 2024-02-15T18:34:52.524 INFO:tasks.workunit.client.0.smithi038.stdout:9/102: creat d1/d11/f21 x:0 0 0 2024-02-15T18:34:52.529 INFO:tasks.workunit.client.0.smithi038.stdout:1/108: mkdir d2/d27 0 2024-02-15T18:34:52.529 INFO:tasks.workunit.client.0.smithi038.stdout:2/111: getdents dd/d12/d18 0 2024-02-15T18:34:52.529 INFO:tasks.workunit.client.0.smithi038.stdout:1/109: chown d2/d10/l13 272764 1 2024-02-15T18:34:52.530 INFO:tasks.workunit.client.0.smithi038.stdout:5/92: dread d2/d3/ff [0,4194304] 0 2024-02-15T18:34:52.530 INFO:tasks.workunit.client.0.smithi038.stdout:8/218: rmdir d1/d4/d5/d31/d44 0 2024-02-15T18:34:52.530 INFO:tasks.workunit.client.0.smithi038.stdout:8/219: rename d1 to d1/d3a/d4e 22 2024-02-15T18:34:52.530 INFO:tasks.workunit.client.0.smithi038.stdout:8/220: dread - d1/d4/d10/d43/f46 zero size 2024-02-15T18:34:52.530 INFO:tasks.workunit.client.0.smithi038.stdout:8/221: chown d1/d7/d15/l16 3 1 2024-02-15T18:34:52.530 INFO:tasks.workunit.client.0.smithi038.stdout:9/103: mknod d1/c22 0 2024-02-15T18:34:52.531 INFO:tasks.workunit.client.0.smithi038.stdout:2/112: symlink dd/d12/l2a 0 2024-02-15T18:34:52.531 INFO:tasks.workunit.client.0.smithi038.stdout:2/113: readlink dd/lf 0 2024-02-15T18:34:52.532 INFO:tasks.workunit.client.0.smithi038.stdout:5/93: creat d2/d3/d12/f17 x:0 0 0 2024-02-15T18:34:52.532 INFO:tasks.workunit.client.0.smithi038.stdout:5/94: dread - d2/d3/d12/f14 zero size 2024-02-15T18:34:52.533 INFO:tasks.workunit.client.0.smithi038.stdout:9/104: rename d1/l6 to d1/dc/l23 0 2024-02-15T18:34:52.534 INFO:tasks.workunit.client.0.smithi038.stdout:7/104: dread d0/fa [0,4194304] 0 2024-02-15T18:34:52.534 INFO:tasks.workunit.client.0.smithi038.stdout:7/105: stat d0/d3/d15/f1f 0 2024-02-15T18:34:52.534 INFO:tasks.workunit.client.0.smithi038.stdout:7/106: write d0/db/ff [45737,10450] 0 2024-02-15T18:34:52.534 INFO:tasks.workunit.client.0.smithi038.stdout:0/175: dwrite d4/f15 [0,4194304] 0 2024-02-15T18:34:52.535 INFO:tasks.workunit.client.0.smithi038.stdout:2/114: unlink dd/d12/d13/f23 0 2024-02-15T18:34:52.535 INFO:tasks.workunit.client.0.smithi038.stdout:2/115: chown fc 25 1 2024-02-15T18:34:52.535 INFO:tasks.workunit.client.0.smithi038.stdout:0/176: write d4/f11 [589634,67987] 0 2024-02-15T18:34:52.537 INFO:tasks.workunit.client.0.smithi038.stdout:5/95: mknod d2/d3/c18 0 2024-02-15T18:34:52.537 INFO:tasks.workunit.client.0.smithi038.stdout:5/96: chown d2/d3/f15 14 1 2024-02-15T18:34:52.537 INFO:tasks.workunit.client.0.smithi038.stdout:9/105: mkdir d1/da/d10/d24 0 2024-02-15T18:34:52.537 INFO:tasks.workunit.client.0.smithi038.stdout:8/222: dread d1/d4/d5/d31/f39 [0,4194304] 0 2024-02-15T18:34:52.538 INFO:tasks.workunit.client.0.smithi038.stdout:7/107: creat d0/d3/d15/f23 x:0 0 0 2024-02-15T18:34:52.538 INFO:tasks.workunit.client.0.smithi038.stdout:7/108: write d0/d3/d15/f18 [4780430,112696] 0 2024-02-15T18:34:52.538 INFO:tasks.workunit.client.0.smithi038.stdout:2/116: truncate dd/d12/d18/f21 891381 0 2024-02-15T18:34:52.539 INFO:tasks.workunit.client.0.smithi038.stdout:0/177: getdents d4/d13/d2a/d35/d1e 0 2024-02-15T18:34:52.540 INFO:tasks.workunit.client.0.smithi038.stdout:5/97: creat d2/d3/d12/f19 x:0 0 0 2024-02-15T18:34:52.540 INFO:tasks.workunit.client.0.smithi038.stdout:5/98: chown d2/d3 132932 1 2024-02-15T18:34:52.540 INFO:tasks.workunit.client.0.smithi038.stdout:5/99: readlink d2/d3/l9 0 2024-02-15T18:34:52.540 INFO:tasks.workunit.client.0.smithi038.stdout:5/100: readlink d2/d3/l9 0 2024-02-15T18:34:52.541 INFO:tasks.workunit.client.0.smithi038.stdout:8/223: mknod d1/d7/d26/c4f 0 2024-02-15T18:34:52.541 INFO:tasks.workunit.client.0.smithi038.stdout:8/224: fsync d1/d4/d5/f37 0 2024-02-15T18:34:52.541 INFO:tasks.workunit.client.0.smithi038.stdout:8/225: write d1/d4/d5/f20 [1491112,14605] 0 2024-02-15T18:34:52.543 INFO:tasks.workunit.client.0.smithi038.stdout:7/109: mknod d0/db/d19/c24 0 2024-02-15T18:34:52.543 INFO:tasks.workunit.client.0.smithi038.stdout:7/110: write d0/db/f20 [634481,62449] 0 2024-02-15T18:34:52.543 INFO:tasks.workunit.client.0.smithi038.stdout:7/111: chown d0/fa 0 1 2024-02-15T18:34:52.543 INFO:tasks.workunit.client.0.smithi038.stdout:7/112: chown d0 9403 1 2024-02-15T18:34:52.543 INFO:tasks.workunit.client.0.smithi038.stdout:0/178: getdents d4 0 2024-02-15T18:34:52.545 INFO:tasks.workunit.client.0.smithi038.stdout:8/226: unlink d1/d7/d15/l16 0 2024-02-15T18:34:52.545 INFO:tasks.workunit.client.0.smithi038.stdout:5/101: write d2/d3/f7 [5642728,98750] 0 2024-02-15T18:34:52.545 INFO:tasks.workunit.client.0.smithi038.stdout:1/110: dwrite d2/d10/d19/f1d [0,4194304] 0 2024-02-15T18:34:52.546 INFO:tasks.workunit.client.0.smithi038.stdout:7/113: creat d0/db/d19/f25 x:0 0 0 2024-02-15T18:34:52.546 INFO:tasks.workunit.client.0.smithi038.stdout:0/179: truncate d4/f12 635508 0 2024-02-15T18:34:52.551 INFO:tasks.workunit.client.0.smithi038.stdout:5/102: rename d2/fd to d2/f1a 0 2024-02-15T18:34:52.552 INFO:tasks.workunit.client.0.smithi038.stdout:5/103: fdatasync d2/d3/d12/fa 0 2024-02-15T18:34:52.555 INFO:tasks.workunit.client.0.smithi038.stdout:9/106: dwrite d1/f14 [0,4194304] 0 2024-02-15T18:34:52.556 INFO:tasks.workunit.client.0.smithi038.stdout:0/180: write d4/f15 [2975211,73844] 0 2024-02-15T18:34:52.556 INFO:tasks.workunit.client.0.smithi038.stdout:0/181: chown d4/l1c 8385731 1 2024-02-15T18:34:52.559 INFO:tasks.workunit.client.0.smithi038.stdout:7/114: mknod d0/db/c26 0 2024-02-15T18:34:52.561 INFO:tasks.workunit.client.0.smithi038.stdout:5/104: creat d2/d3/f1b x:0 0 0 2024-02-15T18:34:52.561 INFO:tasks.workunit.client.0.smithi038.stdout:5/105: stat d2/d3/d12 0 2024-02-15T18:34:52.569 INFO:tasks.workunit.client.0.smithi038.stdout:0/182: creat d4/d13/f39 x:0 0 0 2024-02-15T18:34:52.570 INFO:tasks.workunit.client.0.smithi038.stdout:0/183: read d4/f5 [1310051,95367] 0 2024-02-15T18:34:52.570 INFO:tasks.workunit.client.0.smithi038.stdout:0/184: stat d4/d13/l30 0 2024-02-15T18:34:52.570 INFO:tasks.workunit.client.0.smithi038.stdout:1/111: mkdir d2/d28 0 2024-02-15T18:34:52.570 INFO:tasks.workunit.client.0.smithi038.stdout:7/115: mknod d0/db/c27 0 2024-02-15T18:34:52.571 INFO:tasks.workunit.client.0.smithi038.stdout:7/116: write d0/db/f20 [1015373,130243] 0 2024-02-15T18:34:52.572 INFO:tasks.workunit.client.0.smithi038.stdout:2/117: dwrite dd/d12/d13/f19 [0,4194304] 0 2024-02-15T18:34:52.572 INFO:tasks.workunit.client.0.smithi038.stdout:0/185: symlink d4/d13/d19/l3a 0 2024-02-15T18:34:52.572 INFO:tasks.workunit.client.0.smithi038.stdout:0/186: readlink d4/d13/d19/l3a 0 2024-02-15T18:34:52.573 INFO:tasks.workunit.client.0.smithi038.stdout:1/112: unlink d2/d10/c18 0 2024-02-15T18:34:52.576 INFO:tasks.workunit.client.0.smithi038.stdout:9/107: dwrite d1/f16 [0,4194304] 0 2024-02-15T18:34:52.576 INFO:tasks.workunit.client.0.smithi038.stdout:5/106: dwrite d2/d3/d12/f17 [0,4194304] 0 2024-02-15T18:34:52.577 INFO:tasks.workunit.client.0.smithi038.stdout:2/118: mkdir dd/d12/d18/d2b 0 2024-02-15T18:34:52.577 INFO:tasks.workunit.client.0.smithi038.stdout:0/187: mkdir d4/d13/d2a/d35/d1e/d32/d3b 0 2024-02-15T18:34:52.580 INFO:tasks.workunit.client.0.smithi038.stdout:8/227: truncate d1/d4/d5/f20 1395323 0 2024-02-15T18:34:52.580 INFO:tasks.workunit.client.0.smithi038.stdout:8/228: write d1/d7/d15/f36 [927760,25413] 0 2024-02-15T18:34:52.582 INFO:tasks.workunit.client.0.smithi038.stdout:8/229: dread d1/f1b [0,4194304] 0 2024-02-15T18:34:52.584 INFO:tasks.workunit.client.0.smithi038.stdout:5/107: dread d2/d3/d12/f17 [0,4194304] 0 2024-02-15T18:34:52.588 INFO:tasks.workunit.client.0.smithi038.stdout:9/108: mknod d1/da/d10/d19/c25 0 2024-02-15T18:34:52.589 INFO:tasks.workunit.client.0.smithi038.stdout:0/188: getdents d4/d13/d19 0 2024-02-15T18:34:52.590 INFO:tasks.workunit.client.0.smithi038.stdout:0/189: write d4/f5 [426594,78728] 0 2024-02-15T18:34:52.590 INFO:tasks.workunit.client.0.smithi038.stdout:8/230: creat d1/f50 x:0 0 0 2024-02-15T18:34:52.591 INFO:tasks.workunit.client.0.smithi038.stdout:8/231: chown d1/d7/d15/f36 259 1 2024-02-15T18:34:52.592 INFO:tasks.workunit.client.0.smithi038.stdout:5/108: link d2/d3/d12/f14 d2/f1c 0 2024-02-15T18:34:52.595 INFO:tasks.workunit.client.0.smithi038.stdout:9/109: rename d1/f5 to d1/da/d20/f26 0 2024-02-15T18:34:52.595 INFO:tasks.workunit.client.0.smithi038.stdout:7/117: dwrite d0/d3/d15/f1f [0,4194304] 0 2024-02-15T18:34:52.599 INFO:tasks.workunit.client.0.smithi038.stdout:8/232: rename d1/d7/d15/f36 to d1/d7/f51 0 2024-02-15T18:34:52.599 INFO:tasks.workunit.client.0.smithi038.stdout:9/110: symlink d1/d11/l27 0 2024-02-15T18:34:52.600 INFO:tasks.workunit.client.0.smithi038.stdout:7/118: symlink d0/db/d19/l28 0 2024-02-15T18:34:52.601 INFO:tasks.workunit.client.0.smithi038.stdout:2/119: dwrite dd/d12/f26 [0,4194304] 0 2024-02-15T18:34:52.610 INFO:tasks.workunit.client.0.smithi038.stdout:9/111: dread d1/f16 [0,4194304] 0 2024-02-15T18:34:52.610 INFO:tasks.workunit.client.0.smithi038.stdout:9/112: fsync d1/da/f17 0 2024-02-15T18:34:52.613 INFO:tasks.workunit.client.0.smithi038.stdout:2/120: dread dd/d12/d13/f19 [0,4194304] 0 2024-02-15T18:34:52.623 INFO:tasks.workunit.client.0.smithi038.stdout:0/190: dwrite d4/fa [0,4194304] 0 2024-02-15T18:34:52.623 INFO:tasks.workunit.client.0.smithi038.stdout:0/191: dread - d4/d13/d2a/f2d zero size 2024-02-15T18:34:52.623 INFO:tasks.workunit.client.0.smithi038.stdout:0/192: fsync d4/ff 0 2024-02-15T18:34:52.623 INFO:tasks.workunit.client.0.smithi038.stdout:0/193: fdatasync d4/f12 0 2024-02-15T18:34:52.624 INFO:tasks.workunit.client.0.smithi038.stdout:7/119: dwrite d0/db/fe [0,4194304] 0 2024-02-15T18:34:52.624 INFO:tasks.workunit.client.0.smithi038.stdout:3/68: sync 2024-02-15T18:34:52.624 INFO:tasks.workunit.client.0.smithi038.stdout:6/82: sync 2024-02-15T18:34:52.624 INFO:tasks.workunit.client.0.smithi038.stdout:3/69: chown d2/l8 1782854 1 2024-02-15T18:34:52.625 INFO:tasks.workunit.client.0.smithi038.stdout:0/194: unlink d4/d13/l14 0 2024-02-15T18:34:52.625 INFO:tasks.workunit.client.0.smithi038.stdout:0/195: dread - d4/d13/f39 zero size 2024-02-15T18:34:52.626 INFO:tasks.workunit.client.0.smithi038.stdout:2/121: write fb [696333,9877] 0 2024-02-15T18:34:52.627 INFO:tasks.workunit.client.0.smithi038.stdout:7/120: truncate d0/d3/d15/f18 3165122 0 2024-02-15T18:34:52.627 INFO:tasks.workunit.client.0.smithi038.stdout:7/121: stat d0 0 2024-02-15T18:34:52.627 INFO:tasks.workunit.client.0.smithi038.stdout:7/122: fsync d0/f21 0 2024-02-15T18:34:52.627 INFO:tasks.workunit.client.0.smithi038.stdout:6/83: truncate d2/d4/ff 1405971 0 2024-02-15T18:34:52.628 INFO:tasks.workunit.client.0.smithi038.stdout:3/70: symlink d2/lf 0 2024-02-15T18:34:52.628 INFO:tasks.workunit.client.0.smithi038.stdout:3/71: stat d2/fa 0 2024-02-15T18:34:52.628 INFO:tasks.workunit.client.0.smithi038.stdout:3/72: dread - d2/fd zero size 2024-02-15T18:34:52.630 INFO:tasks.workunit.client.0.smithi038.stdout:0/196: unlink d4/lb 0 2024-02-15T18:34:52.630 INFO:tasks.workunit.client.0.smithi038.stdout:0/197: fsync d4/f12 0 2024-02-15T18:34:52.631 INFO:tasks.workunit.client.0.smithi038.stdout:2/122: rmdir dd/d12/d18 39 2024-02-15T18:34:52.634 INFO:tasks.workunit.client.0.smithi038.stdout:7/123: mkdir d0/d3/d15/d29 0 2024-02-15T18:34:52.634 INFO:tasks.workunit.client.0.smithi038.stdout:7/124: fsync d0/f21 0 2024-02-15T18:34:52.635 INFO:tasks.workunit.client.0.smithi038.stdout:6/84: creat d2/f1a x:0 0 0 2024-02-15T18:34:52.636 INFO:tasks.workunit.client.0.smithi038.stdout:3/73: creat d2/f10 x:0 0 0 2024-02-15T18:34:52.636 INFO:tasks.workunit.client.0.smithi038.stdout:3/74: write d2/fd [307492,5631] 0 2024-02-15T18:34:52.637 INFO:tasks.workunit.client.0.smithi038.stdout:0/198: rmdir d4/d13/d19 39 2024-02-15T18:34:52.639 INFO:tasks.workunit.client.0.smithi038.stdout:7/125: dread d0/fa [0,4194304] 0 2024-02-15T18:34:52.639 INFO:tasks.workunit.client.0.smithi038.stdout:7/126: chown d0/d10/f1c 125 1 2024-02-15T18:34:52.639 INFO:tasks.workunit.client.0.smithi038.stdout:2/123: unlink dd/d12/f22 0 2024-02-15T18:34:52.639 INFO:tasks.workunit.client.0.smithi038.stdout:2/124: stat dd/lf 0 2024-02-15T18:34:52.640 INFO:tasks.workunit.client.0.smithi038.stdout:6/85: getdents d2/d4 0 2024-02-15T18:34:52.640 INFO:tasks.workunit.client.0.smithi038.stdout:6/86: stat d2/d4/fc 0 2024-02-15T18:34:52.640 INFO:tasks.workunit.client.0.smithi038.stdout:6/87: write d2/f17 [815679,86746] 0 2024-02-15T18:34:52.644 INFO:tasks.workunit.client.0.smithi038.stdout:3/75: unlink d2/fd 0 2024-02-15T18:34:52.644 INFO:tasks.workunit.client.0.smithi038.stdout:3/76: fdatasync d2/fa 0 2024-02-15T18:34:52.644 INFO:tasks.workunit.client.0.smithi038.stdout:3/77: truncate d2/fa 653971 0 2024-02-15T18:34:52.644 INFO:tasks.workunit.client.0.smithi038.stdout:0/199: mknod d4/d13/d2a/d35/c3c 0 2024-02-15T18:34:52.644 INFO:tasks.workunit.client.0.smithi038.stdout:0/200: fdatasync d4/f12 0 2024-02-15T18:34:52.644 INFO:tasks.workunit.client.0.smithi038.stdout:0/201: fdatasync d4/f15 0 2024-02-15T18:34:52.645 INFO:tasks.workunit.client.0.smithi038.stdout:0/202: fdatasync d4/f12 0 2024-02-15T18:34:52.645 INFO:tasks.workunit.client.0.smithi038.stdout:7/127: mknod d0/db/c2a 0 2024-02-15T18:34:52.645 INFO:tasks.workunit.client.0.smithi038.stdout:0/203: dread d4/ff [0,4194304] 0 2024-02-15T18:34:52.645 INFO:tasks.workunit.client.0.smithi038.stdout:2/125: symlink dd/d12/d13/l2c 0 2024-02-15T18:34:52.646 INFO:tasks.workunit.client.0.smithi038.stdout:6/88: mknod d2/c1b 0 2024-02-15T18:34:52.646 INFO:tasks.workunit.client.0.smithi038.stdout:6/89: fsync f1 0 2024-02-15T18:34:52.647 INFO:tasks.workunit.client.0.smithi038.stdout:3/78: rename d2/fe to d2/f11 0 2024-02-15T18:34:52.649 INFO:tasks.workunit.client.0.smithi038.stdout:0/204: creat d4/d13/d2a/d35/d1e/f3d x:0 0 0 2024-02-15T18:34:52.650 INFO:tasks.workunit.client.0.smithi038.stdout:6/90: mknod d2/d10/c1c 0 2024-02-15T18:34:52.650 INFO:tasks.workunit.client.0.smithi038.stdout:6/91: getdents d2/d4/d19 0 2024-02-15T18:34:52.650 INFO:tasks.workunit.client.0.smithi038.stdout:6/92: fsync f1 0 2024-02-15T18:34:52.651 INFO:tasks.workunit.client.0.smithi038.stdout:3/79: creat d2/f12 x:0 0 0 2024-02-15T18:34:52.651 INFO:tasks.workunit.client.0.smithi038.stdout:6/93: dread f1 [4194304,4194304] 0 2024-02-15T18:34:52.651 INFO:tasks.workunit.client.0.smithi038.stdout:6/94: chown d2/d10/c12 1 1 2024-02-15T18:34:52.651 INFO:tasks.workunit.client.0.smithi038.stdout:3/80: dread d2/fa [0,4194304] 0 2024-02-15T18:34:52.651 INFO:tasks.workunit.client.0.smithi038.stdout:3/81: chown d2/f11 27 1 2024-02-15T18:34:52.651 INFO:tasks.workunit.client.0.smithi038.stdout:7/128: rmdir d0/db 39 2024-02-15T18:34:52.654 INFO:tasks.workunit.client.0.smithi038.stdout:6/95: dread d2/d4/ff [0,4194304] 0 2024-02-15T18:34:52.654 INFO:tasks.workunit.client.0.smithi038.stdout:0/205: creat d4/f3e x:0 0 0 2024-02-15T18:34:52.656 INFO:tasks.workunit.client.0.smithi038.stdout:3/82: mknod d2/c13 0 2024-02-15T18:34:52.656 INFO:tasks.workunit.client.0.smithi038.stdout:3/83: fdatasync d2/f12 0 2024-02-15T18:34:52.656 INFO:tasks.workunit.client.0.smithi038.stdout:7/129: creat d0/d3/d15/f2b x:0 0 0 2024-02-15T18:34:52.657 INFO:tasks.workunit.client.0.smithi038.stdout:6/96: creat d2/d10/f1d x:0 0 0 2024-02-15T18:34:52.657 INFO:tasks.workunit.client.0.smithi038.stdout:6/97: fdatasync d2/d10/f15 0 2024-02-15T18:34:52.658 INFO:tasks.workunit.client.0.smithi038.stdout:0/206: rename d4/d13/d19/l36 to d4/d13/d2a/d35/d1e/d32/d3b/l3f 0 2024-02-15T18:34:52.658 INFO:tasks.workunit.client.0.smithi038.stdout:3/84: creat d2/f14 x:0 0 0 2024-02-15T18:34:52.659 INFO:tasks.workunit.client.0.smithi038.stdout:6/98: rename d2/f17 to d2/d4/f1e 0 2024-02-15T18:34:52.660 INFO:tasks.workunit.client.0.smithi038.stdout:0/207: creat d4/d13/d2a/d35/d1e/f40 x:0 0 0 2024-02-15T18:34:52.660 INFO:tasks.workunit.client.0.smithi038.stdout:0/208: rename d4/d13/d2a to d4/d13/d2a/d35/d1e/d41 22 2024-02-15T18:34:52.661 INFO:tasks.workunit.client.0.smithi038.stdout:3/85: creat d2/f15 x:0 0 0 2024-02-15T18:34:52.661 INFO:tasks.workunit.client.0.smithi038.stdout:3/86: truncate d2/fa 1512388 0 2024-02-15T18:34:52.661 INFO:tasks.workunit.client.0.smithi038.stdout:3/87: stat d2/lf 0 2024-02-15T18:34:52.661 INFO:tasks.workunit.client.0.smithi038.stdout:6/99: write d2/f3 [4134556,86618] 0 2024-02-15T18:34:52.661 INFO:tasks.workunit.client.0.smithi038.stdout:7/130: dread d0/db/fc [0,4194304] 0 2024-02-15T18:34:52.661 INFO:tasks.workunit.client.0.smithi038.stdout:6/100: write d2/d4/fc [705238,32220] 0 2024-02-15T18:34:52.661 INFO:tasks.workunit.client.0.smithi038.stdout:0/209: mknod d4/d13/d2a/c42 0 2024-02-15T18:34:52.663 INFO:tasks.workunit.client.0.smithi038.stdout:3/88: symlink d2/l16 0 2024-02-15T18:34:52.663 INFO:tasks.workunit.client.0.smithi038.stdout:3/89: fsync d2/f11 0 2024-02-15T18:34:52.663 INFO:tasks.workunit.client.0.smithi038.stdout:7/131: rename d0/d10/l16 to d0/l2c 0 2024-02-15T18:34:52.666 INFO:tasks.workunit.client.0.smithi038.stdout:0/210: creat d4/d13/d19/f43 x:0 0 0 2024-02-15T18:34:52.668 INFO:tasks.workunit.client.0.smithi038.stdout:0/211: rename d4/fa to d4/d13/d2a/f44 0 2024-02-15T18:34:52.668 INFO:tasks.workunit.client.0.smithi038.stdout:0/212: read - d4/d13/d2a/d35/d1e/f40 zero size 2024-02-15T18:34:52.678 INFO:tasks.workunit.client.0.smithi038.stdout:4/77: sync 2024-02-15T18:34:52.680 INFO:tasks.workunit.client.0.smithi038.stdout:4/78: symlink d11/l1c 0 2024-02-15T18:34:52.681 INFO:tasks.workunit.client.0.smithi038.stdout:3/90: dwrite d2/f15 [0,4194304] 0 2024-02-15T18:34:52.682 INFO:tasks.workunit.client.0.smithi038.stdout:4/79: rename fc to d11/f1d 0 2024-02-15T18:34:52.682 INFO:tasks.workunit.client.0.smithi038.stdout:4/80: dread - d11/f17 zero size 2024-02-15T18:34:52.682 INFO:tasks.workunit.client.0.smithi038.stdout:4/81: fdatasync d11/f1d 0 2024-02-15T18:34:52.684 INFO:tasks.workunit.client.0.smithi038.stdout:3/91: creat d2/f17 x:0 0 0 2024-02-15T18:34:52.684 INFO:tasks.workunit.client.0.smithi038.stdout:4/82: mkdir d11/d1e 0 2024-02-15T18:34:52.685 INFO:tasks.workunit.client.0.smithi038.stdout:3/92: creat d2/f18 x:0 0 0 2024-02-15T18:34:52.685 INFO:tasks.workunit.client.0.smithi038.stdout:4/83: creat d11/f1f x:0 0 0 2024-02-15T18:34:52.696 INFO:tasks.workunit.client.0.smithi038.stdout:0/213: dwrite d4/f5 [0,4194304] 0 2024-02-15T18:34:52.696 INFO:tasks.workunit.client.0.smithi038.stdout:0/214: dread - d4/d13/d2a/d35/d1e/f3d zero size 2024-02-15T18:34:52.696 INFO:tasks.workunit.client.0.smithi038.stdout:0/215: write d4/f3e [91039,7765] 0 2024-02-15T18:34:52.697 INFO:tasks.workunit.client.0.smithi038.stdout:0/216: mknod d4/d13/d2a/d35/d1e/d32/d3b/c45 0 2024-02-15T18:34:52.697 INFO:tasks.workunit.client.0.smithi038.stdout:6/101: dwrite d2/f3 [0,4194304] 0 2024-02-15T18:34:52.698 INFO:tasks.workunit.client.0.smithi038.stdout:0/217: symlink d4/d13/d19/l46 0 2024-02-15T18:34:52.700 INFO:tasks.workunit.client.0.smithi038.stdout:0/218: unlink d4/d13/l17 0 2024-02-15T18:34:52.701 INFO:tasks.workunit.client.0.smithi038.stdout:0/219: fsync d4/d13/d2a/d35/d1e/f3d 0 2024-02-15T18:34:52.701 INFO:tasks.workunit.client.0.smithi038.stdout:6/102: dread d2/f9 [0,4194304] 0 2024-02-15T18:34:52.701 INFO:tasks.workunit.client.0.smithi038.stdout:0/220: link d4/d13/d2a/d35/d1e/d32/d3b/l3f d4/l47 0 2024-02-15T18:34:52.704 INFO:tasks.workunit.client.0.smithi038.stdout:7/132: dwrite d0/db/d19/f25 [0,4194304] 0 2024-02-15T18:34:52.704 INFO:tasks.workunit.client.0.smithi038.stdout:7/133: write d0/db/f20 [1819345,24719] 0 2024-02-15T18:34:52.704 INFO:tasks.workunit.client.0.smithi038.stdout:7/134: fsync d0/d3/f7 0 2024-02-15T18:34:52.704 INFO:tasks.workunit.client.0.smithi038.stdout:6/103: dread d2/d4/ff [0,4194304] 0 2024-02-15T18:34:52.706 INFO:tasks.workunit.client.0.smithi038.stdout:7/135: rename d0/db/d19/d1d to d0/db/d19/d2d 0 2024-02-15T18:34:52.709 INFO:tasks.workunit.client.0.smithi038.stdout:3/93: dwrite d2/f15 [0,4194304] 0 2024-02-15T18:34:52.714 INFO:tasks.workunit.client.0.smithi038.stdout:3/94: link d2/f15 d2/f19 0 2024-02-15T18:34:52.715 INFO:tasks.workunit.client.0.smithi038.stdout:3/95: dread d2/f11 [0,4194304] 0 2024-02-15T18:34:52.717 INFO:tasks.workunit.client.0.smithi038.stdout:4/84: dwrite d11/f1f [0,4194304] 0 2024-02-15T18:34:52.718 INFO:tasks.workunit.client.0.smithi038.stdout:0/221: dwrite d4/f11 [0,4194304] 0 2024-02-15T18:34:52.718 INFO:tasks.workunit.client.0.smithi038.stdout:0/222: write d4/d13/d2a/f2d [78251,5911] 0 2024-02-15T18:34:52.722 INFO:tasks.workunit.client.0.smithi038.stdout:7/136: dwrite d0/d3/f7 [0,4194304] 0 2024-02-15T18:34:52.728 INFO:tasks.workunit.client.0.smithi038.stdout:4/85: mkdir d11/d1e/d20 0 2024-02-15T18:34:52.728 INFO:tasks.workunit.client.0.smithi038.stdout:4/86: write f4 [1553805,102136] 0 2024-02-15T18:34:52.730 INFO:tasks.workunit.client.0.smithi038.stdout:0/223: mkdir d4/d13/d2a/d35/d1e/d32/d48 0 2024-02-15T18:34:52.731 INFO:tasks.workunit.client.0.smithi038.stdout:7/137: mkdir d0/d2e 0 2024-02-15T18:34:52.732 INFO:tasks.workunit.client.0.smithi038.stdout:0/224: creat d4/d13/d2a/d35/f49 x:0 0 0 2024-02-15T18:34:52.734 INFO:tasks.workunit.client.0.smithi038.stdout:7/138: mknod d0/d3/c2f 0 2024-02-15T18:34:52.736 INFO:tasks.workunit.client.0.smithi038.stdout:7/139: mkdir d0/d1b/d30 0 2024-02-15T18:34:52.737 INFO:tasks.workunit.client.0.smithi038.stdout:7/140: creat d0/d3/d15/d29/f31 x:0 0 0 2024-02-15T18:34:52.737 INFO:tasks.workunit.client.0.smithi038.stdout:7/141: write d0/f9 [4792121,95255] 0 2024-02-15T18:34:52.738 INFO:tasks.workunit.client.0.smithi038.stdout:7/142: write d0/f21 [474335,116660] 0 2024-02-15T18:34:52.740 INFO:tasks.workunit.client.0.smithi038.stdout:3/96: dwrite d2/f15 [4194304,4194304] 0 2024-02-15T18:34:52.742 INFO:tasks.workunit.client.0.smithi038.stdout:4/87: dwrite d11/f19 [0,4194304] 0 2024-02-15T18:34:52.743 INFO:tasks.workunit.client.0.smithi038.stdout:4/88: chown f4 344569184 1 2024-02-15T18:34:52.743 INFO:tasks.workunit.client.0.smithi038.stdout:4/89: chown l0 59122 1 2024-02-15T18:34:52.744 INFO:tasks.workunit.client.0.smithi038.stdout:3/97: creat d2/f1a x:0 0 0 2024-02-15T18:34:52.747 INFO:tasks.workunit.client.0.smithi038.stdout:3/98: unlink d2/f14 0 2024-02-15T18:34:52.747 INFO:tasks.workunit.client.0.smithi038.stdout:3/99: chown d2/f17 322908 1 2024-02-15T18:34:52.749 INFO:tasks.workunit.client.0.smithi038.stdout:1/113: sync 2024-02-15T18:34:52.749 INFO:tasks.workunit.client.0.smithi038.stdout:3/100: mkdir d2/d1b 0 2024-02-15T18:34:52.749 INFO:tasks.workunit.client.0.smithi038.stdout:4/90: dread d11/f18 [0,4194304] 0 2024-02-15T18:34:52.749 INFO:tasks.workunit.client.0.smithi038.stdout:4/91: write d11/f1a [198930,87949] 0 2024-02-15T18:34:52.749 INFO:tasks.workunit.client.0.smithi038.stdout:4/92: stat d11/f1d 0 2024-02-15T18:34:52.749 INFO:tasks.workunit.client.0.smithi038.stdout:4/93: read f2 [3856222,113913] 0 2024-02-15T18:34:52.750 INFO:tasks.workunit.client.0.smithi038.stdout:1/114: rename d2/ca to d2/d10/d19/d22/c29 0 2024-02-15T18:34:52.751 INFO:tasks.workunit.client.0.smithi038.stdout:4/94: symlink d11/l21 0 2024-02-15T18:34:52.751 INFO:tasks.workunit.client.0.smithi038.stdout:4/95: chown l8 17578 1 2024-02-15T18:34:52.751 INFO:tasks.workunit.client.0.smithi038.stdout:4/96: write f5 [4279797,124722] 0 2024-02-15T18:34:52.753 INFO:tasks.workunit.client.0.smithi038.stdout:1/115: truncate d2/f4 921019 0 2024-02-15T18:34:52.756 INFO:tasks.workunit.client.0.smithi038.stdout:3/101: dread d2/f19 [0,4194304] 0 2024-02-15T18:34:52.756 INFO:tasks.workunit.client.0.smithi038.stdout:3/102: write d2/f9 [1506064,117233] 0 2024-02-15T18:34:52.757 INFO:tasks.workunit.client.0.smithi038.stdout:0/225: dwrite d4/d13/d2a/d35/f49 [0,4194304] 0 2024-02-15T18:34:52.759 INFO:tasks.workunit.client.0.smithi038.stdout:1/116: write d2/f11 [1618049,68651] 0 2024-02-15T18:34:52.759 INFO:tasks.workunit.client.0.smithi038.stdout:0/226: dread d4/f3e [0,4194304] 0 2024-02-15T18:34:52.761 INFO:tasks.workunit.client.0.smithi038.stdout:7/143: dwrite d0/d10/f1c [0,4194304] 0 2024-02-15T18:34:52.763 INFO:tasks.workunit.client.0.smithi038.stdout:0/227: unlink d4/d13/d19/l3a 0 2024-02-15T18:34:52.768 INFO:tasks.workunit.client.0.smithi038.stdout:1/117: dread d2/f3 [0,4194304] 0 2024-02-15T18:34:52.771 INFO:tasks.workunit.client.0.smithi038.stdout:7/144: getdents d0/d10 0 2024-02-15T18:34:52.773 INFO:tasks.workunit.client.0.smithi038.stdout:1/118: mkdir d2/d27/d2a 0 2024-02-15T18:34:52.774 INFO:tasks.workunit.client.0.smithi038.stdout:1/119: write d2/d10/d19/f1d [2273451,1339] 0 2024-02-15T18:34:52.779 INFO:tasks.workunit.client.0.smithi038.stdout:1/120: rmdir d2/d27/d2a 0 2024-02-15T18:34:52.779 INFO:tasks.workunit.client.0.smithi038.stdout:0/228: dread d4/d13/d2a/d35/d1e/f20 [0,4194304] 0 2024-02-15T18:34:52.779 INFO:tasks.workunit.client.0.smithi038.stdout:0/229: write d4/d13/d2a/d35/d1e/f40 [546339,34241] 0 2024-02-15T18:34:52.780 INFO:tasks.workunit.client.0.smithi038.stdout:1/121: mknod d2/d10/d19/c2b 0 2024-02-15T18:34:52.781 INFO:tasks.workunit.client.0.smithi038.stdout:0/230: rename d4/d13/l30 to d4/d13/d2a/d35/d1e/d32/l4a 0 2024-02-15T18:34:52.781 INFO:tasks.workunit.client.0.smithi038.stdout:0/231: write d4/f3e [991479,124185] 0 2024-02-15T18:34:52.782 INFO:tasks.workunit.client.0.smithi038.stdout:1/122: getdents d2 0 2024-02-15T18:34:52.784 INFO:tasks.workunit.client.0.smithi038.stdout:0/232: creat d4/d13/d2a/f4b x:0 0 0 2024-02-15T18:34:52.785 INFO:tasks.workunit.client.0.smithi038.stdout:0/233: truncate d4/d13/d2a/d35/f49 1699115 0 2024-02-15T18:34:52.785 INFO:tasks.workunit.client.0.smithi038.stdout:1/123: dread d2/f1e [0,4194304] 0 2024-02-15T18:34:52.785 INFO:tasks.workunit.client.0.smithi038.stdout:7/145: dwrite d0/db/fc [0,4194304] 0 2024-02-15T18:34:52.786 INFO:tasks.workunit.client.0.smithi038.stdout:1/124: creat d2/d10/d19/d22/f2c x:0 0 0 2024-02-15T18:34:52.786 INFO:tasks.workunit.client.0.smithi038.stdout:1/125: chown d2/f26 9626049 1 2024-02-15T18:34:52.787 INFO:tasks.workunit.client.0.smithi038.stdout:0/234: write d4/f5 [3997771,18941] 0 2024-02-15T18:34:52.793 INFO:tasks.workunit.client.0.smithi038.stdout:8/233: sync 2024-02-15T18:34:52.793 INFO:tasks.workunit.client.0.smithi038.stdout:9/113: sync 2024-02-15T18:34:52.793 INFO:tasks.workunit.client.0.smithi038.stdout:5/109: sync 2024-02-15T18:34:52.793 INFO:tasks.workunit.client.0.smithi038.stdout:8/234: write d1/d4/d10/d43/f46 [203787,13130] 0 2024-02-15T18:34:52.796 INFO:tasks.workunit.client.0.smithi038.stdout:5/110: rename d2/d3/d12/fe to d2/d3/f1d 0 2024-02-15T18:34:52.796 INFO:tasks.workunit.client.0.smithi038.stdout:5/111: stat d2/d3/c18 0 2024-02-15T18:34:52.796 INFO:tasks.workunit.client.0.smithi038.stdout:5/112: write d2/d3/f15 [16752,76735] 0 2024-02-15T18:34:52.796 INFO:tasks.workunit.client.0.smithi038.stdout:5/113: chown d2/fc 0 1 2024-02-15T18:34:52.799 INFO:tasks.workunit.client.0.smithi038.stdout:8/235: write d1/f2 [2079537,19181] 0 2024-02-15T18:34:52.802 INFO:tasks.workunit.client.0.smithi038.stdout:5/114: mknod d2/c1e 0 2024-02-15T18:34:52.802 INFO:tasks.workunit.client.0.smithi038.stdout:5/115: readlink d2/d3/l9 0 2024-02-15T18:34:52.803 INFO:tasks.workunit.client.0.smithi038.stdout:8/236: mkdir d1/d4/d10/d43/d48/d52 0 2024-02-15T18:34:52.804 INFO:tasks.workunit.client.0.smithi038.stdout:5/116: mknod d2/d3/d12/c1f 0 2024-02-15T18:34:52.804 INFO:tasks.workunit.client.0.smithi038.stdout:8/237: truncate d1/d7/d15/f2f 275756 0 2024-02-15T18:34:52.805 INFO:tasks.workunit.client.0.smithi038.stdout:8/238: write d1/d7/f51 [837305,118996] 0 2024-02-15T18:34:52.805 INFO:tasks.workunit.client.0.smithi038.stdout:5/117: mkdir d2/d20 0 2024-02-15T18:34:52.806 INFO:tasks.workunit.client.0.smithi038.stdout:5/118: creat d2/d20/f21 x:0 0 0 2024-02-15T18:34:52.808 INFO:tasks.workunit.client.0.smithi038.stdout:5/119: rename d2/c1e to d2/d3/d12/c22 0 2024-02-15T18:34:52.808 INFO:tasks.workunit.client.0.smithi038.stdout:5/120: write d2/d20/f21 [771960,121765] 0 2024-02-15T18:34:52.808 INFO:tasks.workunit.client.0.smithi038.stdout:5/121: readlink d2/d3/l9 0 2024-02-15T18:34:52.809 INFO:tasks.workunit.client.0.smithi038.stdout:5/122: symlink d2/d20/l23 0 2024-02-15T18:34:52.811 INFO:tasks.workunit.client.0.smithi038.stdout:5/123: unlink d2/d3/d12/c1f 0 2024-02-15T18:34:52.813 INFO:tasks.workunit.client.0.smithi038.stdout:5/124: unlink d2/d3/f15 0 2024-02-15T18:34:52.814 INFO:tasks.workunit.client.0.smithi038.stdout:5/125: symlink d2/d20/l24 0 2024-02-15T18:34:52.814 INFO:tasks.workunit.client.0.smithi038.stdout:5/126: stat d2/d3/d12 0 2024-02-15T18:34:52.816 INFO:tasks.workunit.client.0.smithi038.stdout:5/127: link d2/c13 d2/d3/c25 0 2024-02-15T18:34:52.816 INFO:tasks.workunit.client.0.smithi038.stdout:5/128: chown d2/d20/l23 92613498 1 2024-02-15T18:34:52.816 INFO:tasks.workunit.client.0.smithi038.stdout:5/129: write d2/d3/d12/f14 [4932,31577] 0 2024-02-15T18:34:52.818 INFO:tasks.workunit.client.0.smithi038.stdout:5/130: write d2/fc [3687809,111311] 0 2024-02-15T18:34:52.819 INFO:tasks.workunit.client.0.smithi038.stdout:5/131: creat d2/d20/f26 x:0 0 0 2024-02-15T18:34:52.819 INFO:tasks.workunit.client.0.smithi038.stdout:5/132: write d2/d3/d12/f19 [116792,105818] 0 2024-02-15T18:34:52.819 INFO:tasks.workunit.client.0.smithi038.stdout:5/133: readlink d2/d20/l24 0 2024-02-15T18:34:52.819 INFO:tasks.workunit.client.0.smithi038.stdout:5/134: write d2/d3/d12/f19 [823367,10096] 0 2024-02-15T18:34:52.820 INFO:tasks.workunit.client.0.smithi038.stdout:5/135: link d2/d3/l9 d2/l27 0 2024-02-15T18:34:52.820 INFO:tasks.workunit.client.0.smithi038.stdout:5/136: dread - d2/d3/f1d zero size 2024-02-15T18:34:52.820 INFO:tasks.workunit.client.0.smithi038.stdout:5/137: creat d2/d3/d12/f28 x:0 0 0 2024-02-15T18:34:52.821 INFO:tasks.workunit.client.0.smithi038.stdout:5/138: mknod d2/c29 0 2024-02-15T18:34:52.821 INFO:tasks.workunit.client.0.smithi038.stdout:5/139: truncate d2/d20/f26 955588 0 2024-02-15T18:34:52.821 INFO:tasks.workunit.client.0.smithi038.stdout:5/140: chown d2/d3/d12 69 1 2024-02-15T18:34:52.822 INFO:tasks.workunit.client.0.smithi038.stdout:5/141: creat d2/f2a x:0 0 0 2024-02-15T18:34:52.825 INFO:tasks.workunit.client.0.smithi038.stdout:7/146: dwrite d0/fa [0,4194304] 0 2024-02-15T18:34:52.825 INFO:tasks.workunit.client.0.smithi038.stdout:1/126: dwrite d2/f26 [0,4194304] 0 2024-02-15T18:34:52.825 INFO:tasks.workunit.client.0.smithi038.stdout:9/114: dwrite d1/da/f17 [0,4194304] 0 2024-02-15T18:34:52.825 INFO:tasks.workunit.client.0.smithi038.stdout:8/239: dwrite d1/fb [4194304,4194304] 0 2024-02-15T18:34:52.827 INFO:tasks.workunit.client.0.smithi038.stdout:7/147: getdents d0/d3/d15 0 2024-02-15T18:34:52.827 INFO:tasks.workunit.client.0.smithi038.stdout:7/148: fsync d0/d3/f7 0 2024-02-15T18:34:52.827 INFO:tasks.workunit.client.0.smithi038.stdout:7/149: truncate d0/d3/f1a 28187 0 2024-02-15T18:34:52.827 INFO:tasks.workunit.client.0.smithi038.stdout:0/235: dwrite d4/d13/d2a/d35/f49 [0,4194304] 0 2024-02-15T18:34:52.827 INFO:tasks.workunit.client.0.smithi038.stdout:5/142: dread d2/d3/f7 [0,4194304] 0 2024-02-15T18:34:52.827 INFO:tasks.workunit.client.0.smithi038.stdout:0/236: write d4/ff [896152,89234] 0 2024-02-15T18:34:52.829 INFO:tasks.workunit.client.0.smithi038.stdout:1/127: dread d2/f1e [0,4194304] 0 2024-02-15T18:34:52.839 INFO:tasks.workunit.client.0.smithi038.stdout:9/115: creat d1/dc/f28 x:0 0 0 2024-02-15T18:34:52.840 INFO:tasks.workunit.client.0.smithi038.stdout:8/240: creat d1/d4/d10/d43/d48/d52/f53 x:0 0 0 2024-02-15T18:34:52.841 INFO:tasks.workunit.client.0.smithi038.stdout:7/150: creat d0/d10/f32 x:0 0 0 2024-02-15T18:34:52.842 INFO:tasks.workunit.client.0.smithi038.stdout:0/237: creat d4/d13/d2a/d35/d1e/d32/d48/f4c x:0 0 0 2024-02-15T18:34:52.842 INFO:tasks.workunit.client.0.smithi038.stdout:9/116: dread d1/f16 [0,4194304] 0 2024-02-15T18:34:52.843 INFO:tasks.workunit.client.0.smithi038.stdout:1/128: creat d2/f2d x:0 0 0 2024-02-15T18:34:52.848 INFO:tasks.workunit.client.0.smithi038.stdout:8/241: mkdir d1/d7/d26/d54 0 2024-02-15T18:34:52.849 INFO:tasks.workunit.client.0.smithi038.stdout:7/151: creat d0/d1b/d30/f33 x:0 0 0 2024-02-15T18:34:52.849 INFO:tasks.workunit.client.0.smithi038.stdout:0/238: creat d4/d13/d2a/d35/d1e/f4d x:0 0 0 2024-02-15T18:34:52.850 INFO:tasks.workunit.client.0.smithi038.stdout:0/239: chown d4/l9 176536 1 2024-02-15T18:34:52.850 INFO:tasks.workunit.client.0.smithi038.stdout:1/129: truncate d2/f11 4023089 0 2024-02-15T18:34:52.854 INFO:tasks.workunit.client.0.smithi038.stdout:7/152: truncate d0/f9 3615451 0 2024-02-15T18:34:52.877 INFO:tasks.workunit.client.0.smithi038.stdout:2/126: sync 2024-02-15T18:34:52.877 INFO:tasks.workunit.client.0.smithi038.stdout:6/104: sync 2024-02-15T18:34:52.877 INFO:tasks.workunit.client.0.smithi038.stdout:6/105: readlink - no filename 2024-02-15T18:34:52.877 INFO:tasks.workunit.client.0.smithi038.stdout:6/106: chown f1 817252 1 2024-02-15T18:34:52.877 INFO:tasks.workunit.client.0.smithi038.stdout:9/117: dwrite d1/da/d10/f1f [0,4194304] 0 2024-02-15T18:34:52.878 INFO:tasks.workunit.client.0.smithi038.stdout:2/127: rmdir dd 39 2024-02-15T18:34:52.879 INFO:tasks.workunit.client.0.smithi038.stdout:9/118: symlink d1/dc/l29 0 2024-02-15T18:34:52.879 INFO:tasks.workunit.client.0.smithi038.stdout:6/107: dread d2/d4/ff [0,4194304] 0 2024-02-15T18:34:52.879 INFO:tasks.workunit.client.0.smithi038.stdout:6/108: fdatasync d2/f3 0 2024-02-15T18:34:52.880 INFO:tasks.workunit.client.0.smithi038.stdout:2/128: getdents dd/d12/d18/d1f 0 2024-02-15T18:34:52.880 INFO:tasks.workunit.client.0.smithi038.stdout:8/242: dwrite d1/d4/d5/d6/f2a [0,4194304] 0 2024-02-15T18:34:52.880 INFO:tasks.workunit.client.0.smithi038.stdout:8/243: fsync d1/d4/d10/d43/d48/d52/f53 0 2024-02-15T18:34:52.881 INFO:tasks.workunit.client.0.smithi038.stdout:9/119: truncate f0 1950085 0 2024-02-15T18:34:52.882 INFO:tasks.workunit.client.0.smithi038.stdout:6/109: creat d2/d4/f1f x:0 0 0 2024-02-15T18:34:52.882 INFO:tasks.workunit.client.0.smithi038.stdout:2/129: truncate dd/d12/d1d/f1a 1691807 0 2024-02-15T18:34:52.884 INFO:tasks.workunit.client.0.smithi038.stdout:8/244: stat d1/d7/c1f 0 2024-02-15T18:34:52.884 INFO:tasks.workunit.client.0.smithi038.stdout:8/245: readlink d1/d4/d5/d6/le 0 2024-02-15T18:34:52.885 INFO:tasks.workunit.client.0.smithi038.stdout:9/120: mknod d1/da/d10/d24/c2a 0 2024-02-15T18:34:52.885 INFO:tasks.workunit.client.0.smithi038.stdout:9/121: stat d1/c7 0 2024-02-15T18:34:52.885 INFO:tasks.workunit.client.0.smithi038.stdout:9/122: chown d1/d11/l27 92193 1 2024-02-15T18:34:52.887 INFO:tasks.workunit.client.0.smithi038.stdout:2/130: unlink dd/c17 0 2024-02-15T18:34:52.887 INFO:tasks.workunit.client.0.smithi038.stdout:8/246: dread d1/d4/d5/f23 [0,4194304] 0 2024-02-15T18:34:52.887 INFO:tasks.workunit.client.0.smithi038.stdout:8/247: read d1/f4a [876266,59344] 0 2024-02-15T18:34:52.887 INFO:tasks.workunit.client.0.smithi038.stdout:9/123: getdents d1/da/d10/d19 0 2024-02-15T18:34:52.887 INFO:tasks.workunit.client.0.smithi038.stdout:1/130: dwrite d2/f2d [0,4194304] 0 2024-02-15T18:34:52.888 INFO:tasks.workunit.client.0.smithi038.stdout:2/131: symlink dd/d12/d18/d2b/l2d 0 2024-02-15T18:34:52.888 INFO:tasks.workunit.client.0.smithi038.stdout:2/132: chown c6 305122 1 2024-02-15T18:34:52.888 INFO:tasks.workunit.client.0.smithi038.stdout:2/133: write dd/f24 [872261,96297] 0 2024-02-15T18:34:52.889 INFO:tasks.workunit.client.0.smithi038.stdout:7/153: dwrite d0/d3/d15/f23 [0,4194304] 0 2024-02-15T18:34:52.893 INFO:tasks.workunit.client.0.smithi038.stdout:6/110: getdents d2/d4 0 2024-02-15T18:34:52.893 INFO:tasks.workunit.client.0.smithi038.stdout:6/111: write d2/f1a [519195,102423] 0 2024-02-15T18:34:52.895 INFO:tasks.workunit.client.0.smithi038.stdout:9/124: dread d1/f14 [4194304,4194304] 0 2024-02-15T18:34:52.895 INFO:tasks.workunit.client.0.smithi038.stdout:8/248: rename d1/f4a to d1/d7/d15/d4b/f55 0 2024-02-15T18:34:52.896 INFO:tasks.workunit.client.0.smithi038.stdout:8/249: dread d1/f1b [0,4194304] 0 2024-02-15T18:34:52.900 INFO:tasks.workunit.client.0.smithi038.stdout:2/134: getdents dd/d12 0 2024-02-15T18:34:52.902 INFO:tasks.workunit.client.0.smithi038.stdout:7/154: mknod d0/d3/d15/d29/c34 0 2024-02-15T18:34:52.904 INFO:tasks.workunit.client.0.smithi038.stdout:9/125: mkdir d1/dc/d2b 0 2024-02-15T18:34:52.904 INFO:tasks.workunit.client.0.smithi038.stdout:8/250: mknod d1/d7/d26/c56 0 2024-02-15T18:34:52.908 INFO:tasks.workunit.client.0.smithi038.stdout:7/155: creat d0/d10/f35 x:0 0 0 2024-02-15T18:34:52.908 INFO:tasks.workunit.client.0.smithi038.stdout:7/156: readlink d0/db/l17 0 2024-02-15T18:34:52.915 INFO:tasks.workunit.client.0.smithi038.stdout:1/131: dwrite d2/f4 [0,4194304] 0 2024-02-15T18:34:52.917 INFO:tasks.workunit.client.0.smithi038.stdout:6/112: dwrite d2/d4/fc [0,4194304] 0 2024-02-15T18:34:52.926 INFO:tasks.workunit.client.0.smithi038.stdout:6/113: rename d2/d10/f15 to d2/f20 0 2024-02-15T18:34:52.926 INFO:tasks.workunit.client.0.smithi038.stdout:6/114: dread - d2/d4/f1f zero size 2024-02-15T18:34:52.928 INFO:tasks.workunit.client.0.smithi038.stdout:6/115: link d2/c16 d2/d10/c21 0 2024-02-15T18:34:52.929 INFO:tasks.workunit.client.0.smithi038.stdout:6/116: mknod d2/c22 0 2024-02-15T18:34:52.929 INFO:tasks.workunit.client.0.smithi038.stdout:6/117: stat d2/f20 0 2024-02-15T18:34:52.932 INFO:tasks.workunit.client.0.smithi038.stdout:6/118: dread d2/f9 [0,4194304] 0 2024-02-15T18:34:52.932 INFO:tasks.workunit.client.0.smithi038.stdout:6/119: chown d2/c1b 0 1 2024-02-15T18:34:52.932 INFO:tasks.workunit.client.0.smithi038.stdout:6/120: stat d2/f20 0 2024-02-15T18:34:52.936 INFO:tasks.workunit.client.0.smithi038.stdout:8/251: dwrite d1/d4/d10/d43/f46 [0,4194304] 0 2024-02-15T18:34:52.938 INFO:tasks.workunit.client.0.smithi038.stdout:9/126: dwrite d1/fb [0,4194304] 0 2024-02-15T18:34:52.938 INFO:tasks.workunit.client.0.smithi038.stdout:2/135: dwrite dd/d12/d1d/f1a [0,4194304] 0 2024-02-15T18:34:52.938 INFO:tasks.workunit.client.0.smithi038.stdout:9/127: truncate d1/da/d10/d13/f18 432722 0 2024-02-15T18:34:52.939 INFO:tasks.workunit.client.0.smithi038.stdout:1/132: dwrite d2/d10/f1b [0,4194304] 0 2024-02-15T18:34:52.941 INFO:tasks.workunit.client.0.smithi038.stdout:9/128: creat d1/d11/f2c x:0 0 0 2024-02-15T18:34:52.944 INFO:tasks.workunit.client.0.smithi038.stdout:1/133: mkdir d2/d2e 0 2024-02-15T18:34:52.947 INFO:tasks.workunit.client.0.smithi038.stdout:9/129: mknod d1/da/d20/c2d 0 2024-02-15T18:34:52.948 INFO:tasks.workunit.client.0.smithi038.stdout:1/134: truncate d2/f26 4161654 0 2024-02-15T18:34:52.951 INFO:tasks.workunit.client.0.smithi038.stdout:1/135: mkdir d2/d10/d19/d2f 0 2024-02-15T18:34:52.952 INFO:tasks.workunit.client.0.smithi038.stdout:1/136: fdatasync d2/d10/d19/d22/f2c 0 2024-02-15T18:34:52.953 INFO:tasks.workunit.client.0.smithi038.stdout:1/137: creat d2/d10/d19/d22/f30 x:0 0 0 2024-02-15T18:34:52.953 INFO:tasks.workunit.client.0.smithi038.stdout:1/138: fdatasync d2/d10/d19/d22/f30 0 2024-02-15T18:34:52.954 INFO:tasks.workunit.client.0.smithi038.stdout:1/139: creat d2/d10/f31 x:0 0 0 2024-02-15T18:34:52.954 INFO:tasks.workunit.client.0.smithi038.stdout:1/140: write d2/d10/f31 [623749,63346] 0 2024-02-15T18:34:52.955 INFO:tasks.workunit.client.0.smithi038.stdout:6/121: dwrite d2/f1a [0,4194304] 0 2024-02-15T18:34:52.955 INFO:tasks.workunit.client.0.smithi038.stdout:6/122: truncate d2/f20 989749 0 2024-02-15T18:34:52.957 INFO:tasks.workunit.client.0.smithi038.stdout:1/141: mkdir d2/d10/d32 0 2024-02-15T18:34:52.958 INFO:tasks.workunit.client.0.smithi038.stdout:3/103: sync 2024-02-15T18:34:52.958 INFO:tasks.workunit.client.0.smithi038.stdout:5/143: sync 2024-02-15T18:34:52.958 INFO:tasks.workunit.client.0.smithi038.stdout:4/97: sync 2024-02-15T18:34:52.958 INFO:tasks.workunit.client.0.smithi038.stdout:3/104: fdatasync d2/f15 0 2024-02-15T18:34:52.958 INFO:tasks.workunit.client.0.smithi038.stdout:6/123: getdents d2 0 2024-02-15T18:34:52.959 INFO:tasks.workunit.client.0.smithi038.stdout:6/124: getdents d2/d4 0 2024-02-15T18:34:52.959 INFO:tasks.workunit.client.0.smithi038.stdout:6/125: chown d2/d4/f1f 0 1 2024-02-15T18:34:52.960 INFO:tasks.workunit.client.0.smithi038.stdout:4/98: unlink d11/f18 0 2024-02-15T18:34:52.960 INFO:tasks.workunit.client.0.smithi038.stdout:5/144: creat d2/f2b x:0 0 0 2024-02-15T18:34:52.961 INFO:tasks.workunit.client.0.smithi038.stdout:5/145: write d2/f2b [811353,26140] 0 2024-02-15T18:34:52.961 INFO:tasks.workunit.client.0.smithi038.stdout:9/130: dread d1/da/d10/f1f [0,4194304] 0 2024-02-15T18:34:52.961 INFO:tasks.workunit.client.0.smithi038.stdout:4/99: creat d11/d1e/d20/f22 x:0 0 0 2024-02-15T18:34:52.962 INFO:tasks.workunit.client.0.smithi038.stdout:5/146: getdents d2/d3 0 2024-02-15T18:34:52.963 INFO:tasks.workunit.client.0.smithi038.stdout:5/147: write d2/f2a [486987,7303] 0 2024-02-15T18:34:52.964 INFO:tasks.workunit.client.0.smithi038.stdout:5/148: dread d2/f1a [0,4194304] 0 2024-02-15T18:34:52.964 INFO:tasks.workunit.client.0.smithi038.stdout:5/149: fsync d2/d20/f21 0 2024-02-15T18:34:52.965 INFO:tasks.workunit.client.0.smithi038.stdout:8/252: dwrite d1/d4/d10/f35 [0,4194304] 0 2024-02-15T18:34:52.966 INFO:tasks.workunit.client.0.smithi038.stdout:2/136: dwrite dd/d12/d18/f1c [0,4194304] 0 2024-02-15T18:34:52.967 INFO:tasks.workunit.client.0.smithi038.stdout:9/131: creat d1/dc/d2b/f2e x:0 0 0 2024-02-15T18:34:52.969 INFO:tasks.workunit.client.0.smithi038.stdout:4/100: link d11/f1d d11/d1e/f23 0 2024-02-15T18:34:52.969 INFO:tasks.workunit.client.0.smithi038.stdout:5/150: symlink d2/d20/l2c 0 2024-02-15T18:34:52.970 INFO:tasks.workunit.client.0.smithi038.stdout:5/151: stat d2/f1a 0 2024-02-15T18:34:52.973 INFO:tasks.workunit.client.0.smithi038.stdout:1/142: write d2/f26 [2638060,24059] 0 2024-02-15T18:34:52.979 INFO:tasks.workunit.client.0.smithi038.stdout:1/143: symlink d2/d10/d19/d22/l33 0 2024-02-15T18:34:52.979 INFO:tasks.workunit.client.0.smithi038.stdout:1/144: write d2/d10/d19/d22/f30 [462018,91788] 0 2024-02-15T18:34:52.979 INFO:tasks.workunit.client.0.smithi038.stdout:2/137: dread dd/d12/f26 [0,4194304] 0 2024-02-15T18:34:52.979 INFO:tasks.workunit.client.0.smithi038.stdout:2/138: chown dd/d12/d18/f21 346058 1 2024-02-15T18:34:52.979 INFO:tasks.workunit.client.0.smithi038.stdout:2/139: stat dd/d12/d13 0 2024-02-15T18:34:52.981 INFO:tasks.workunit.client.0.smithi038.stdout:9/132: read d1/da/d10/f1f [2169920,38283] 0 2024-02-15T18:34:52.983 INFO:tasks.workunit.client.0.smithi038.stdout:4/101: dread d11/f1f [0,4194304] 0 2024-02-15T18:34:52.983 INFO:tasks.workunit.client.0.smithi038.stdout:4/102: chown d11/c16 3 1 2024-02-15T18:34:52.983 INFO:tasks.workunit.client.0.smithi038.stdout:1/145: creat d2/d28/f34 x:0 0 0 2024-02-15T18:34:52.984 INFO:tasks.workunit.client.0.smithi038.stdout:9/133: mkdir d1/da/d10/d2f 0 2024-02-15T18:34:52.985 INFO:tasks.workunit.client.0.smithi038.stdout:9/134: mknod d1/da/d10/d19/c30 0 2024-02-15T18:34:52.985 INFO:tasks.workunit.client.0.smithi038.stdout:6/126: dwrite d2/f1a [0,4194304] 0 2024-02-15T18:34:52.986 INFO:tasks.workunit.client.0.smithi038.stdout:9/135: mknod d1/da/d10/d2f/c31 0 2024-02-15T18:34:52.987 INFO:tasks.workunit.client.0.smithi038.stdout:6/127: symlink d2/l23 0 2024-02-15T18:34:52.989 INFO:tasks.workunit.client.0.smithi038.stdout:9/136: unlink d1/dc/d2b/f2e 0 2024-02-15T18:34:52.989 INFO:tasks.workunit.client.0.smithi038.stdout:6/128: mkdir d2/d24 0 2024-02-15T18:34:52.990 INFO:tasks.workunit.client.0.smithi038.stdout:9/137: mknod d1/da/d10/d2f/c32 0 2024-02-15T18:34:52.991 INFO:tasks.workunit.client.0.smithi038.stdout:9/138: creat d1/da/d10/f33 x:0 0 0 2024-02-15T18:34:52.991 INFO:tasks.workunit.client.0.smithi038.stdout:9/139: write d1/f1c [632627,13829] 0 2024-02-15T18:34:52.992 INFO:tasks.workunit.client.0.smithi038.stdout:3/105: dwrite d2/f15 [4194304,4194304] 0 2024-02-15T18:34:52.992 INFO:tasks.workunit.client.0.smithi038.stdout:3/106: chown d2/fa 6213 1 2024-02-15T18:34:52.992 INFO:tasks.workunit.client.0.smithi038.stdout:3/107: write d2/f12 [812623,71816] 0 2024-02-15T18:34:52.993 INFO:tasks.workunit.client.0.smithi038.stdout:8/253: dwrite d1/d7/f2d [0,4194304] 0 2024-02-15T18:34:52.994 INFO:tasks.workunit.client.0.smithi038.stdout:9/140: rename d1/d11/f2c to d1/dc/d2b/f34 0 2024-02-15T18:34:52.995 INFO:tasks.workunit.client.0.smithi038.stdout:6/129: dread d2/d4/f1e [0,4194304] 0 2024-02-15T18:34:52.995 INFO:tasks.workunit.client.0.smithi038.stdout:6/130: fsync d2/f20 0 2024-02-15T18:34:52.995 INFO:tasks.workunit.client.0.smithi038.stdout:6/131: readlink d2/l23 0 2024-02-15T18:34:52.995 INFO:tasks.workunit.client.0.smithi038.stdout:0/240: sync 2024-02-15T18:34:53.000 INFO:tasks.workunit.client.0.smithi038.stdout:8/254: creat d1/d7/d15/d4b/f57 x:0 0 0 2024-02-15T18:34:53.001 INFO:tasks.workunit.client.0.smithi038.stdout:9/141: mkdir d1/da/d10/d35 0 2024-02-15T18:34:53.002 INFO:tasks.workunit.client.0.smithi038.stdout:6/132: rename d2/cd to d2/d10/c25 0 2024-02-15T18:34:53.002 INFO:tasks.workunit.client.0.smithi038.stdout:6/133: readlink d2/l23 0 2024-02-15T18:34:53.002 INFO:tasks.workunit.client.0.smithi038.stdout:0/241: creat d4/d13/d2a/f4e x:0 0 0 2024-02-15T18:34:53.004 INFO:tasks.workunit.client.0.smithi038.stdout:8/255: getdents d1 0 2024-02-15T18:34:53.004 INFO:tasks.workunit.client.0.smithi038.stdout:8/256: truncate d1/d4/d10/d43/d48/d52/f53 625822 0 2024-02-15T18:34:53.004 INFO:tasks.workunit.client.0.smithi038.stdout:8/257: fdatasync d1/d4/d10/f1e 0 2024-02-15T18:34:53.004 INFO:tasks.workunit.client.0.smithi038.stdout:8/258: chown d1/d7/d15/c2b 0 1 2024-02-15T18:34:53.004 INFO:tasks.workunit.client.0.smithi038.stdout:8/259: chown d1/d4/d10/d43/f46 11546157 1 2024-02-15T18:34:53.005 INFO:tasks.workunit.client.0.smithi038.stdout:9/142: mkdir d1/da/d36 0 2024-02-15T18:34:53.005 INFO:tasks.workunit.client.0.smithi038.stdout:6/134: truncate d2/d4/ff 208738 0 2024-02-15T18:34:53.007 INFO:tasks.workunit.client.0.smithi038.stdout:0/242: rmdir d4/d13/d2a/d35/d1e/d32/d48 39 2024-02-15T18:34:53.008 INFO:tasks.workunit.client.0.smithi038.stdout:6/135: dread d2/d4/f1e [0,4194304] 0 2024-02-15T18:34:53.008 INFO:tasks.workunit.client.0.smithi038.stdout:6/136: chown d2/d10/c14 0 1 2024-02-15T18:34:53.008 INFO:tasks.workunit.client.0.smithi038.stdout:4/103: dwrite d11/d1e/f23 [0,4194304] 0 2024-02-15T18:34:53.010 INFO:tasks.workunit.client.0.smithi038.stdout:1/146: dwrite d2/d10/f31 [0,4194304] 0 2024-02-15T18:34:53.010 INFO:tasks.workunit.client.0.smithi038.stdout:8/260: rename d1/d7/d26/d54 to d1/d3a/d58 0 2024-02-15T18:34:53.010 INFO:tasks.workunit.client.0.smithi038.stdout:8/261: chown d1/d7/d15 8521662 1 2024-02-15T18:34:53.010 INFO:tasks.workunit.client.0.smithi038.stdout:8/262: stat d1/d7/d15/d4b/f55 0 2024-02-15T18:34:53.010 INFO:tasks.workunit.client.0.smithi038.stdout:9/143: truncate d1/fb 1450489 0 2024-02-15T18:34:53.011 INFO:tasks.workunit.client.0.smithi038.stdout:0/243: creat d4/d13/d2a/d35/d1e/d32/f4f x:0 0 0 2024-02-15T18:34:53.012 INFO:tasks.workunit.client.0.smithi038.stdout:1/147: read d2/f4 [3198842,43869] 0 2024-02-15T18:34:53.014 INFO:tasks.workunit.client.0.smithi038.stdout:9/144: dread d1/f16 [0,4194304] 0 2024-02-15T18:34:53.014 INFO:tasks.workunit.client.0.smithi038.stdout:9/145: chown d1/da 45076 1 2024-02-15T18:34:53.018 INFO:tasks.workunit.client.0.smithi038.stdout:6/137: truncate d2/f3 2750686 0 2024-02-15T18:34:53.021 INFO:tasks.workunit.client.0.smithi038.stdout:3/108: truncate d2/f15 6092078 0 2024-02-15T18:34:53.024 INFO:tasks.workunit.client.0.smithi038.stdout:4/104: mkdir d11/d1e/d20/d24 0 2024-02-15T18:34:53.025 INFO:tasks.workunit.client.0.smithi038.stdout:4/105: truncate d11/d1e/d20/f22 237199 0 2024-02-15T18:34:53.027 INFO:tasks.workunit.client.0.smithi038.stdout:8/263: dwrite d1/d4/d5/f27 [0,4194304] 0 2024-02-15T18:34:53.031 INFO:tasks.workunit.client.0.smithi038.stdout:4/106: dread d11/f19 [0,4194304] 0 2024-02-15T18:34:53.034 INFO:tasks.workunit.client.0.smithi038.stdout:0/244: link d4/d13/d2a/d35/d1e/f4d d4/d13/d2a/d35/d1e/f50 0 2024-02-15T18:34:53.036 INFO:tasks.workunit.client.0.smithi038.stdout:9/146: creat d1/da/d36/f37 x:0 0 0 2024-02-15T18:34:53.037 INFO:tasks.workunit.client.0.smithi038.stdout:6/138: mknod d2/d4/c26 0 2024-02-15T18:34:53.038 INFO:tasks.workunit.client.0.smithi038.stdout:3/109: rename d2/l8 to d2/l1c 0 2024-02-15T18:34:53.038 INFO:tasks.workunit.client.0.smithi038.stdout:8/264: mknod d1/d4/d10/d3d/c59 0 2024-02-15T18:34:53.039 INFO:tasks.workunit.client.0.smithi038.stdout:4/107: link d11/f1b d11/d1e/f25 0 2024-02-15T18:34:53.041 INFO:tasks.workunit.client.0.smithi038.stdout:4/108: write d11/d1e/f23 [3176594,107706] 0 2024-02-15T18:34:53.042 INFO:tasks.workunit.client.0.smithi038.stdout:9/147: rmdir d1/da/d10/d35 0 2024-02-15T18:34:53.042 INFO:tasks.workunit.client.0.smithi038.stdout:7/157: sync 2024-02-15T18:34:53.043 INFO:tasks.workunit.client.0.smithi038.stdout:7/158: chown d0/d1b/d30/f33 644067218 1 2024-02-15T18:34:53.043 INFO:tasks.workunit.client.0.smithi038.stdout:7/159: truncate d0/db/d19/f25 4345508 0 2024-02-15T18:34:53.044 INFO:tasks.workunit.client.0.smithi038.stdout:6/139: creat d2/d4/f27 x:0 0 0 2024-02-15T18:34:53.044 INFO:tasks.workunit.client.0.smithi038.stdout:7/160: dread d0/d3/f1a [0,4194304] 0 2024-02-15T18:34:53.044 INFO:tasks.workunit.client.0.smithi038.stdout:8/265: creat d1/d7/d22/f5a x:0 0 0 2024-02-15T18:34:53.046 INFO:tasks.workunit.client.0.smithi038.stdout:1/148: rmdir d2/d27 0 2024-02-15T18:34:53.046 INFO:tasks.workunit.client.0.smithi038.stdout:4/109: truncate f2 2072571 0 2024-02-15T18:34:53.047 INFO:tasks.workunit.client.0.smithi038.stdout:9/148: getdents d1/da/d20 0 2024-02-15T18:34:53.048 INFO:tasks.workunit.client.0.smithi038.stdout:9/149: write d1/f14 [3831072,686] 0 2024-02-15T18:34:53.048 INFO:tasks.workunit.client.0.smithi038.stdout:6/140: rmdir d2/d24 0 2024-02-15T18:34:53.048 INFO:tasks.workunit.client.0.smithi038.stdout:6/141: readlink d2/l23 0 2024-02-15T18:34:53.049 INFO:tasks.workunit.client.0.smithi038.stdout:8/266: mknod d1/d7/c5b 0 2024-02-15T18:34:53.050 INFO:tasks.workunit.client.0.smithi038.stdout:9/150: dread d1/dc/fe [0,4194304] 0 2024-02-15T18:34:53.052 INFO:tasks.workunit.client.0.smithi038.stdout:6/142: mkdir d2/d4/d28 0 2024-02-15T18:34:53.053 INFO:tasks.workunit.client.0.smithi038.stdout:8/267: creat d1/d7/d22/f5c x:0 0 0 2024-02-15T18:34:53.053 INFO:tasks.workunit.client.0.smithi038.stdout:8/268: fdatasync d1/f40 0 2024-02-15T18:34:53.054 INFO:tasks.workunit.client.0.smithi038.stdout:9/151: mkdir d1/dc/d38 0 2024-02-15T18:34:53.054 INFO:tasks.workunit.client.0.smithi038.stdout:9/152: chown d1/c7 8 1 2024-02-15T18:34:53.055 INFO:tasks.workunit.client.0.smithi038.stdout:3/110: dwrite d2/f17 [0,4194304] 0 2024-02-15T18:34:53.055 INFO:tasks.workunit.client.0.smithi038.stdout:3/111: chown d2/l1c 3 1 2024-02-15T18:34:53.055 INFO:tasks.workunit.client.0.smithi038.stdout:4/110: dread d11/f1d [0,4194304] 0 2024-02-15T18:34:53.055 INFO:tasks.workunit.client.0.smithi038.stdout:4/111: write d11/f1a [514548,105553] 0 2024-02-15T18:34:53.055 INFO:tasks.workunit.client.0.smithi038.stdout:8/269: creat d1/f5d x:0 0 0 2024-02-15T18:34:53.058 INFO:tasks.workunit.client.0.smithi038.stdout:7/161: dread d0/d10/f1c [0,4194304] 0 2024-02-15T18:34:53.058 INFO:tasks.workunit.client.0.smithi038.stdout:7/162: chown d0/d3/c2f 414308879 1 2024-02-15T18:34:53.059 INFO:tasks.workunit.client.0.smithi038.stdout:8/270: write d1/d4/d5/f37 [425313,75278] 0 2024-02-15T18:34:53.061 INFO:tasks.workunit.client.0.smithi038.stdout:3/112: dread d2/f12 [0,4194304] 0 2024-02-15T18:34:53.065 INFO:tasks.workunit.client.0.smithi038.stdout:9/153: mknod d1/dc/d38/c39 0 2024-02-15T18:34:53.065 INFO:tasks.workunit.client.0.smithi038.stdout:9/154: dread - d1/da/d10/f33 zero size 2024-02-15T18:34:53.067 INFO:tasks.workunit.client.0.smithi038.stdout:4/112: read f5 [341804,24220] 0 2024-02-15T18:34:53.067 INFO:tasks.workunit.client.0.smithi038.stdout:4/113: write d11/d1e/f25 [92906,39417] 0 2024-02-15T18:34:53.067 INFO:tasks.workunit.client.0.smithi038.stdout:7/163: creat d0/db/d19/d2d/f36 x:0 0 0 2024-02-15T18:34:53.067 INFO:tasks.workunit.client.0.smithi038.stdout:7/164: write d0/db/fc [4597701,7420] 0 2024-02-15T18:34:53.069 INFO:tasks.workunit.client.0.smithi038.stdout:8/271: rename d1/d4/d5/d6/c28 to d1/d4/d10/d43/d48/d52/c5e 0 2024-02-15T18:34:53.071 INFO:tasks.workunit.client.0.smithi038.stdout:6/143: dwrite d2/d4/fc [0,4194304] 0 2024-02-15T18:34:53.071 INFO:tasks.workunit.client.0.smithi038.stdout:3/113: creat d2/f1d x:0 0 0 2024-02-15T18:34:53.071 INFO:tasks.workunit.client.0.smithi038.stdout:9/155: link d1/f14 d1/dc/d2b/f3a 0 2024-02-15T18:34:53.072 INFO:tasks.workunit.client.0.smithi038.stdout:4/114: creat d11/d1e/d20/d24/f26 x:0 0 0 2024-02-15T18:34:53.072 INFO:tasks.workunit.client.0.smithi038.stdout:4/115: write d11/f1b [575165,56779] 0 2024-02-15T18:34:53.073 INFO:tasks.workunit.client.0.smithi038.stdout:8/272: rename d1/d4/d5/d6/le to d1/d4/d5/l5f 0 2024-02-15T18:34:53.074 INFO:tasks.workunit.client.0.smithi038.stdout:6/144: getdents d2/d4 0 2024-02-15T18:34:53.077 INFO:tasks.workunit.client.0.smithi038.stdout:3/114: mkdir d2/d1e 0 2024-02-15T18:34:53.077 INFO:tasks.workunit.client.0.smithi038.stdout:3/115: chown d2/f10 3698803 1 2024-02-15T18:34:53.078 INFO:tasks.workunit.client.0.smithi038.stdout:9/156: mknod d1/d11/c3b 0 2024-02-15T18:34:53.080 INFO:tasks.workunit.client.0.smithi038.stdout:4/116: unlink f2 0 2024-02-15T18:34:53.081 INFO:tasks.workunit.client.0.smithi038.stdout:8/273: getdents d1/d4/d5 0 2024-02-15T18:34:53.082 INFO:tasks.workunit.client.0.smithi038.stdout:6/145: truncate f1 1259763 0 2024-02-15T18:34:53.083 INFO:tasks.workunit.client.0.smithi038.stdout:6/146: dread d2/d4/ff [0,4194304] 0 2024-02-15T18:34:53.084 INFO:tasks.workunit.client.0.smithi038.stdout:3/116: rename d2/c13 to d2/d1e/c1f 0 2024-02-15T18:34:53.084 INFO:tasks.workunit.client.0.smithi038.stdout:3/117: chown d2/l16 1 1 2024-02-15T18:34:53.085 INFO:tasks.workunit.client.0.smithi038.stdout:9/157: symlink d1/da/d20/l3c 0 2024-02-15T18:34:53.085 INFO:tasks.workunit.client.0.smithi038.stdout:9/158: chown d1/dc/f28 221643 1 2024-02-15T18:34:53.085 INFO:tasks.workunit.client.0.smithi038.stdout:9/159: write d1/da/d36/f37 [674441,62923] 0 2024-02-15T18:34:53.085 INFO:tasks.workunit.client.0.smithi038.stdout:9/160: dread - d1/d11/f21 zero size 2024-02-15T18:34:53.085 INFO:tasks.workunit.client.0.smithi038.stdout:9/161: truncate d1/da/d20/f26 5207294 0 2024-02-15T18:34:53.085 INFO:tasks.workunit.client.0.smithi038.stdout:9/162: dread - d1/d11/f21 zero size 2024-02-15T18:34:53.086 INFO:tasks.workunit.client.0.smithi038.stdout:7/165: fsync d0/db/fc 0 2024-02-15T18:34:53.087 INFO:tasks.workunit.client.0.smithi038.stdout:4/117: link d11/f17 d11/d1e/d20/f27 0 2024-02-15T18:34:53.088 INFO:tasks.workunit.client.0.smithi038.stdout:8/274: creat d1/d3a/d58/f60 x:0 0 0 2024-02-15T18:34:53.091 INFO:tasks.workunit.client.0.smithi038.stdout:6/147: creat d2/f29 x:0 0 0 2024-02-15T18:34:53.091 INFO:tasks.workunit.client.0.smithi038.stdout:6/148: write d2/d4/f27 [992730,46716] 0 2024-02-15T18:34:53.092 INFO:tasks.workunit.client.0.smithi038.stdout:9/163: read f0 [550701,87062] 0 2024-02-15T18:34:53.092 INFO:tasks.workunit.client.0.smithi038.stdout:7/166: creat d0/d2e/f37 x:0 0 0 2024-02-15T18:34:53.092 INFO:tasks.workunit.client.0.smithi038.stdout:7/167: write d0/d3/d15/f2b [187195,57703] 0 2024-02-15T18:34:53.094 INFO:tasks.workunit.client.0.smithi038.stdout:4/118: rename f5 to d11/d1e/f28 0 2024-02-15T18:34:53.094 INFO:tasks.workunit.client.0.smithi038.stdout:8/275: creat d1/d4/d10/d43/d48/d52/f61 x:0 0 0 2024-02-15T18:34:53.095 INFO:tasks.workunit.client.0.smithi038.stdout:6/149: creat d2/f2a x:0 0 0 2024-02-15T18:34:53.095 INFO:tasks.workunit.client.0.smithi038.stdout:9/164: mkdir d1/da/d10/d13/d3d 0 2024-02-15T18:34:53.096 INFO:tasks.workunit.client.0.smithi038.stdout:7/168: getdents d0/d1b/d30 0 2024-02-15T18:34:53.098 INFO:tasks.workunit.client.0.smithi038.stdout:4/119: rename d11/d1e/f25 to d11/d1e/f29 0 2024-02-15T18:34:53.099 INFO:tasks.workunit.client.0.smithi038.stdout:3/118: dwrite d2/f1d [0,4194304] 0 2024-02-15T18:34:53.101 INFO:tasks.workunit.client.0.smithi038.stdout:3/119: write d2/f19 [4272137,107344] 0 2024-02-15T18:34:53.101 INFO:tasks.workunit.client.0.smithi038.stdout:3/120: dread - d2/f10 zero size 2024-02-15T18:34:53.101 INFO:tasks.workunit.client.0.smithi038.stdout:3/121: rename d2 to d2/d20 22 2024-02-15T18:34:53.101 INFO:tasks.workunit.client.0.smithi038.stdout:3/122: chown d2/f12 62 1 2024-02-15T18:34:53.101 INFO:tasks.workunit.client.0.smithi038.stdout:8/276: link d1/d4/d10/d43/c45 d1/d7/d15/c62 0 2024-02-15T18:34:53.106 INFO:tasks.workunit.client.0.smithi038.stdout:6/150: rmdir d2/d10 39 2024-02-15T18:34:53.106 INFO:tasks.workunit.client.0.smithi038.stdout:6/151: fdatasync d2/f3 0 2024-02-15T18:34:53.106 INFO:tasks.workunit.client.0.smithi038.stdout:7/169: mknod d0/db/d19/c38 0 2024-02-15T18:34:53.106 INFO:tasks.workunit.client.0.smithi038.stdout:7/170: readlink d0/l2c 0 2024-02-15T18:34:53.106 INFO:tasks.workunit.client.0.smithi038.stdout:7/171: truncate d0/d3/f7 4582720 0 2024-02-15T18:34:53.109 INFO:tasks.workunit.client.0.smithi038.stdout:8/277: mknod d1/d47/c63 0 2024-02-15T18:34:53.110 INFO:tasks.workunit.client.0.smithi038.stdout:6/152: unlink d2/c22 0 2024-02-15T18:34:53.111 INFO:tasks.workunit.client.0.smithi038.stdout:6/153: chown d2/d4/c26 2087087 1 2024-02-15T18:34:53.111 INFO:tasks.workunit.client.0.smithi038.stdout:7/172: symlink d0/d1b/l39 0 2024-02-15T18:34:53.112 INFO:tasks.workunit.client.0.smithi038.stdout:6/154: read d2/f1a [1572998,26780] 0 2024-02-15T18:34:53.113 INFO:tasks.workunit.client.0.smithi038.stdout:4/120: getdents d11/d1e 0 2024-02-15T18:34:53.114 INFO:tasks.workunit.client.0.smithi038.stdout:7/173: write d0/d3/d15/f2b [5572,64186] 0 2024-02-15T18:34:53.116 INFO:tasks.workunit.client.0.smithi038.stdout:7/174: symlink d0/d1b/d30/l3a 0 2024-02-15T18:34:53.116 INFO:tasks.workunit.client.0.smithi038.stdout:7/175: write d0/d3/f7 [5375710,36545] 0 2024-02-15T18:34:53.117 INFO:tasks.workunit.client.0.smithi038.stdout:7/176: creat d0/d2e/f3b x:0 0 0 2024-02-15T18:34:53.117 INFO:tasks.workunit.client.0.smithi038.stdout:7/177: stat d0/db/d19/c24 0 2024-02-15T18:34:53.118 INFO:tasks.workunit.client.0.smithi038.stdout:7/178: link d0/d3/d15/d29/c34 d0/c3c 0 2024-02-15T18:34:53.119 INFO:tasks.workunit.client.0.smithi038.stdout:7/179: symlink d0/d1b/d30/l3d 0 2024-02-15T18:34:53.120 INFO:tasks.workunit.client.0.smithi038.stdout:7/180: mknod d0/d3/c3e 0 2024-02-15T18:34:53.127 INFO:tasks.workunit.client.0.smithi038.stdout:9/165: dwrite d1/da/d36/f37 [0,4194304] 0 2024-02-15T18:34:53.129 INFO:tasks.workunit.client.0.smithi038.stdout:9/166: mknod d1/d11/c3e 0 2024-02-15T18:34:53.129 INFO:tasks.workunit.client.0.smithi038.stdout:9/167: write d1/fb [2308569,17293] 0 2024-02-15T18:34:53.141 INFO:tasks.workunit.client.0.smithi038.stdout:4/121: dwrite d11/d1e/d20/f22 [0,4194304] 0 2024-02-15T18:34:53.142 INFO:tasks.workunit.client.0.smithi038.stdout:3/123: dwrite d2/f12 [0,4194304] 0 2024-02-15T18:34:53.143 INFO:tasks.workunit.client.0.smithi038.stdout:8/278: dwrite d1/d3a/d58/f60 [0,4194304] 0 2024-02-15T18:34:53.145 INFO:tasks.workunit.client.0.smithi038.stdout:4/122: read d11/d1e/d20/f22 [1592518,54615] 0 2024-02-15T18:34:53.145 INFO:tasks.workunit.client.0.smithi038.stdout:4/123: chown d11/d1e 20374 1 2024-02-15T18:34:53.148 INFO:tasks.workunit.client.0.smithi038.stdout:7/181: dwrite d0/d1b/d30/f33 [0,4194304] 0 2024-02-15T18:34:53.150 INFO:tasks.workunit.client.0.smithi038.stdout:4/124: mknod d11/d1e/c2a 0 2024-02-15T18:34:53.157 INFO:tasks.workunit.client.0.smithi038.stdout:7/182: rename d0/d10/f32 to d0/d1b/f3f 0 2024-02-15T18:34:53.157 INFO:tasks.workunit.client.0.smithi038.stdout:7/183: dread - d0/d10/f35 zero size 2024-02-15T18:34:53.157 INFO:tasks.workunit.client.0.smithi038.stdout:7/184: readlink d0/l2c 0 2024-02-15T18:34:53.158 INFO:tasks.workunit.client.0.smithi038.stdout:4/125: mkdir d11/d1e/d20/d2b 0 2024-02-15T18:34:53.160 INFO:tasks.workunit.client.0.smithi038.stdout:7/185: dread d0/f9 [0,4194304] 0 2024-02-15T18:34:53.160 INFO:tasks.workunit.client.0.smithi038.stdout:4/126: mknod d11/d1e/d20/d24/c2c 0 2024-02-15T18:34:53.161 INFO:tasks.workunit.client.0.smithi038.stdout:7/186: creat d0/d1b/d30/f40 x:0 0 0 2024-02-15T18:34:53.161 INFO:tasks.workunit.client.0.smithi038.stdout:7/187: chown d0/db/d19/d2d/l1e 55266693 1 2024-02-15T18:34:53.166 INFO:tasks.workunit.client.0.smithi038.stdout:6/155: dwrite d2/d4/f27 [0,4194304] 0 2024-02-15T18:34:53.166 INFO:tasks.workunit.client.0.smithi038.stdout:8/279: dwrite d1/d4/d5/d6/ff [4194304,4194304] 0 2024-02-15T18:34:53.172 INFO:tasks.workunit.client.0.smithi038.stdout:8/280: dread d1/d7/fc [0,4194304] 0 2024-02-15T18:34:53.172 INFO:tasks.workunit.client.0.smithi038.stdout:8/281: write d1/d7/d15/f2f [853536,82084] 0 2024-02-15T18:34:53.172 INFO:tasks.workunit.client.0.smithi038.stdout:6/156: rename d2/d10/c1c to d2/d4/d28/c2b 0 2024-02-15T18:34:53.172 INFO:tasks.workunit.client.0.smithi038.stdout:6/157: readlink d2/l23 0 2024-02-15T18:34:53.173 INFO:tasks.workunit.client.0.smithi038.stdout:8/282: mknod d1/d7/d15/c64 0 2024-02-15T18:34:53.174 INFO:tasks.workunit.client.0.smithi038.stdout:6/158: mkdir d2/d4/d2c 0 2024-02-15T18:34:53.174 INFO:tasks.workunit.client.0.smithi038.stdout:4/127: dwrite d11/f1d [0,4194304] 0 2024-02-15T18:34:53.175 INFO:tasks.workunit.client.0.smithi038.stdout:8/283: mkdir d1/d4/d5/d31/d65 0 2024-02-15T18:34:53.175 INFO:tasks.workunit.client.0.smithi038.stdout:8/284: write d1/d4/d5/f23 [2961000,43734] 0 2024-02-15T18:34:53.175 INFO:tasks.workunit.client.0.smithi038.stdout:8/285: truncate d1/d7/d22/f5a 639950 0 2024-02-15T18:34:53.175 INFO:tasks.workunit.client.0.smithi038.stdout:8/286: write d1/d4/d5/f20 [978139,18496] 0 2024-02-15T18:34:53.175 INFO:tasks.workunit.client.0.smithi038.stdout:6/159: unlink d2/d10/c18 0 2024-02-15T18:34:53.176 INFO:tasks.workunit.client.0.smithi038.stdout:4/128: dread d11/f1b [0,4194304] 0 2024-02-15T18:34:53.179 INFO:tasks.workunit.client.0.smithi038.stdout:6/160: rename d2/f29 to d2/d4/f2d 0 2024-02-15T18:34:53.179 INFO:tasks.workunit.client.0.smithi038.stdout:6/161: chown d2/d4/d2c 3 1 2024-02-15T18:34:53.179 INFO:tasks.workunit.client.0.smithi038.stdout:4/129: rename f7 to d11/d1e/d20/d24/f2d 0 2024-02-15T18:34:53.182 INFO:tasks.workunit.client.0.smithi038.stdout:8/287: dread d1/d4/d5/d6/ff [4194304,4194304] 0 2024-02-15T18:34:53.182 INFO:tasks.workunit.client.0.smithi038.stdout:5/152: sync 2024-02-15T18:34:53.182 INFO:tasks.workunit.client.0.smithi038.stdout:2/140: sync 2024-02-15T18:34:53.185 INFO:tasks.workunit.client.0.smithi038.stdout:6/162: fsync d2/d4/f2d 0 2024-02-15T18:34:53.185 INFO:tasks.workunit.client.0.smithi038.stdout:6/163: dread - d2/d10/f1d zero size 2024-02-15T18:34:53.188 INFO:tasks.workunit.client.0.smithi038.stdout:4/130: rename d11/f19 to d11/d1e/d20/d2b/f2e 0 2024-02-15T18:34:53.188 INFO:tasks.workunit.client.0.smithi038.stdout:2/141: creat dd/d12/d1d/f2e x:0 0 0 2024-02-15T18:34:53.189 INFO:tasks.workunit.client.0.smithi038.stdout:5/153: creat d2/d3/d12/f2d x:0 0 0 2024-02-15T18:34:53.190 INFO:tasks.workunit.client.0.smithi038.stdout:8/288: rename d1/d7/d26/c42 to d1/d4/d5/d31/c66 0 2024-02-15T18:34:53.196 INFO:tasks.workunit.client.0.smithi038.stdout:4/131: dread d11/d1e/d20/d24/f2d [0,4194304] 0 2024-02-15T18:34:53.197 INFO:tasks.workunit.client.0.smithi038.stdout:2/142: dread dd/d12/f26 [0,4194304] 0 2024-02-15T18:34:53.197 INFO:tasks.workunit.client.0.smithi038.stdout:2/143: rename dd to dd/d12/d13/d2f 22 2024-02-15T18:34:53.198 INFO:tasks.workunit.client.0.smithi038.stdout:4/132: mkdir d11/d1e/d20/d2f 0 2024-02-15T18:34:53.199 INFO:tasks.workunit.client.0.smithi038.stdout:2/144: creat dd/d12/d18/d1f/f30 x:0 0 0 2024-02-15T18:34:53.199 INFO:tasks.workunit.client.0.smithi038.stdout:2/145: dread - dd/d12/d18/d1f/f29 zero size 2024-02-15T18:34:53.199 INFO:tasks.workunit.client.0.smithi038.stdout:2/146: dread - dd/d12/d18/d1f/f27 zero size 2024-02-15T18:34:53.199 INFO:tasks.workunit.client.0.smithi038.stdout:6/164: dwrite d2/d10/f1d [0,4194304] 0 2024-02-15T18:34:53.200 INFO:tasks.workunit.client.0.smithi038.stdout:2/147: creat dd/d12/d1d/f31 x:0 0 0 2024-02-15T18:34:53.200 INFO:tasks.workunit.client.0.smithi038.stdout:2/148: dread - dd/d12/d1d/f31 zero size 2024-02-15T18:34:53.203 INFO:tasks.workunit.client.0.smithi038.stdout:6/165: rename d2/d4/c26 to d2/d4/c2e 0 2024-02-15T18:34:53.205 INFO:tasks.workunit.client.0.smithi038.stdout:6/166: dread d2/d4/ff [0,4194304] 0 2024-02-15T18:34:53.205 INFO:tasks.workunit.client.0.smithi038.stdout:2/149: link c6 dd/d12/c32 0 2024-02-15T18:34:53.205 INFO:tasks.workunit.client.0.smithi038.stdout:6/167: stat d2/d4/d28/c2b 0 2024-02-15T18:34:53.206 INFO:tasks.workunit.client.0.smithi038.stdout:2/150: mkdir dd/d12/d13/d33 0 2024-02-15T18:34:53.207 INFO:tasks.workunit.client.0.smithi038.stdout:5/154: dread d2/d3/f7 [0,4194304] 0 2024-02-15T18:34:53.207 INFO:tasks.workunit.client.0.smithi038.stdout:4/133: dread d11/f14 [0,4194304] 0 2024-02-15T18:34:53.207 INFO:tasks.workunit.client.0.smithi038.stdout:2/151: link c0 dd/d12/d18/d1f/c34 0 2024-02-15T18:34:53.207 INFO:tasks.workunit.client.0.smithi038.stdout:2/152: read - dd/d12/d1d/f2e zero size 2024-02-15T18:34:53.209 INFO:tasks.workunit.client.0.smithi038.stdout:5/155: rename d2/l27 to d2/d20/l2e 0 2024-02-15T18:34:53.210 INFO:tasks.workunit.client.0.smithi038.stdout:8/289: dwrite d1/fb [8388608,4194304] 0 2024-02-15T18:34:53.210 INFO:tasks.workunit.client.0.smithi038.stdout:8/290: readlink d1/l3c 0 2024-02-15T18:34:53.210 INFO:tasks.workunit.client.0.smithi038.stdout:8/291: fdatasync d1/d7/d26/f34 0 2024-02-15T18:34:53.211 INFO:tasks.workunit.client.0.smithi038.stdout:6/168: write d2/d4/f27 [1931987,99672] 0 2024-02-15T18:34:53.211 INFO:tasks.workunit.client.0.smithi038.stdout:5/156: creat d2/d3/f2f x:0 0 0 2024-02-15T18:34:53.212 INFO:tasks.workunit.client.0.smithi038.stdout:5/157: write d2/d3/f7 [307399,100535] 0 2024-02-15T18:34:53.216 INFO:tasks.workunit.client.0.smithi038.stdout:8/292: dread d1/d7/f51 [0,4194304] 0 2024-02-15T18:34:53.216 INFO:tasks.workunit.client.0.smithi038.stdout:8/293: write d1/f50 [788251,33540] 0 2024-02-15T18:34:53.216 INFO:tasks.workunit.client.0.smithi038.stdout:6/169: creat d2/d4/f2f x:0 0 0 2024-02-15T18:34:53.217 INFO:tasks.workunit.client.0.smithi038.stdout:6/170: read d2/f20 [699470,80957] 0 2024-02-15T18:34:53.217 INFO:tasks.workunit.client.0.smithi038.stdout:6/171: truncate d2/d4/f1f 718688 0 2024-02-15T18:34:53.217 INFO:tasks.workunit.client.0.smithi038.stdout:5/158: mkdir d2/d30 0 2024-02-15T18:34:53.217 INFO:tasks.workunit.client.0.smithi038.stdout:5/159: write d2/d3/f1d [447567,34797] 0 2024-02-15T18:34:53.219 INFO:tasks.workunit.client.0.smithi038.stdout:5/160: mknod d2/d30/c31 0 2024-02-15T18:34:53.219 INFO:tasks.workunit.client.0.smithi038.stdout:5/161: truncate d2/d3/d12/f28 236075 0 2024-02-15T18:34:53.220 INFO:tasks.workunit.client.0.smithi038.stdout:8/294: write d1/fb [1313244,24834] 0 2024-02-15T18:34:53.221 INFO:tasks.workunit.client.0.smithi038.stdout:8/295: mknod d1/d4/d10/d3d/c67 0 2024-02-15T18:34:53.221 INFO:tasks.workunit.client.0.smithi038.stdout:8/296: fsync d1/d4/d10/f1e 0 2024-02-15T18:34:53.221 INFO:tasks.workunit.client.0.smithi038.stdout:8/297: stat d1/d7/c41 0 2024-02-15T18:34:53.221 INFO:tasks.workunit.client.0.smithi038.stdout:8/298: stat d1/d7/d15/c62 0 2024-02-15T18:34:53.222 INFO:tasks.workunit.client.0.smithi038.stdout:8/299: mknod d1/d7/d26/c68 0 2024-02-15T18:34:53.222 INFO:tasks.workunit.client.0.smithi038.stdout:8/300: dread - d1/d7/d15/d4b/f3f zero size 2024-02-15T18:34:53.222 INFO:tasks.workunit.client.0.smithi038.stdout:8/301: fsync d1/d7/d15/d4b/f55 0 2024-02-15T18:34:53.223 INFO:tasks.workunit.client.0.smithi038.stdout:8/302: mkdir d1/d7/d26/d69 0 2024-02-15T18:34:53.224 INFO:tasks.workunit.client.0.smithi038.stdout:8/303: mkdir d1/d7/d15/d6a 0 2024-02-15T18:34:53.224 INFO:tasks.workunit.client.0.smithi038.stdout:8/304: write d1/f40 [500990,120967] 0 2024-02-15T18:34:53.225 INFO:tasks.workunit.client.0.smithi038.stdout:8/305: creat d1/d7/d15/d6a/f6b x:0 0 0 2024-02-15T18:34:53.226 INFO:tasks.workunit.client.0.smithi038.stdout:8/306: creat d1/d3a/f6c x:0 0 0 2024-02-15T18:34:53.228 INFO:tasks.workunit.client.0.smithi038.stdout:8/307: rename d1/d47 to d1/d4/d5/d6/d6d 0 2024-02-15T18:34:53.228 INFO:tasks.workunit.client.0.smithi038.stdout:8/308: stat d1/d7/d15/d4b/f55 0 2024-02-15T18:34:53.228 INFO:tasks.workunit.client.0.smithi038.stdout:8/309: write d1/d4/d5/f20 [1077738,22129] 0 2024-02-15T18:34:53.229 INFO:tasks.workunit.client.0.smithi038.stdout:4/134: dwrite d11/f13 [0,4194304] 0 2024-02-15T18:34:53.229 INFO:tasks.workunit.client.0.smithi038.stdout:2/153: dwrite dd/d12/d1d/f1a [0,4194304] 0 2024-02-15T18:34:53.229 INFO:tasks.workunit.client.0.smithi038.stdout:8/310: creat d1/d4/d5/d6/f6e x:0 0 0 2024-02-15T18:34:53.232 INFO:tasks.workunit.client.0.smithi038.stdout:4/135: creat d11/d1e/d20/d2b/f30 x:0 0 0 2024-02-15T18:34:53.232 INFO:tasks.workunit.client.0.smithi038.stdout:4/136: truncate d11/d1e/d20/d2b/f30 456560 0 2024-02-15T18:34:53.233 INFO:tasks.workunit.client.0.smithi038.stdout:8/311: creat d1/f6f x:0 0 0 2024-02-15T18:34:53.234 INFO:tasks.workunit.client.0.smithi038.stdout:4/137: mkdir d11/d1e/d20/d31 0 2024-02-15T18:34:53.237 INFO:tasks.workunit.client.0.smithi038.stdout:8/312: rename d1/d4/d10/d43/d48/d52/f61 to d1/d4/d10/d43/f70 0 2024-02-15T18:34:53.237 INFO:tasks.workunit.client.0.smithi038.stdout:8/313: write d1/d4/d10/d43/f70 [387917,105568] 0 2024-02-15T18:34:53.238 INFO:tasks.workunit.client.0.smithi038.stdout:5/162: dwrite d2/d3/d12/fa [0,4194304] 0 2024-02-15T18:34:53.239 INFO:tasks.workunit.client.0.smithi038.stdout:8/314: mknod d1/d7/d15/d4b/c71 0 2024-02-15T18:34:53.239 INFO:tasks.workunit.client.0.smithi038.stdout:8/315: dread - d1/f5d zero size 2024-02-15T18:34:53.244 INFO:tasks.workunit.client.0.smithi038.stdout:8/316: symlink d1/d4/d5/d6/l72 0 2024-02-15T18:34:53.245 INFO:tasks.workunit.client.0.smithi038.stdout:3/124: sync 2024-02-15T18:34:53.245 INFO:tasks.workunit.client.0.smithi038.stdout:9/168: sync 2024-02-15T18:34:53.245 INFO:tasks.workunit.client.0.smithi038.stdout:7/188: sync 2024-02-15T18:34:53.245 INFO:tasks.workunit.client.0.smithi038.stdout:3/125: chown d2/lc 146324 1 2024-02-15T18:34:53.245 INFO:tasks.workunit.client.0.smithi038.stdout:3/126: readlink d2/l1c 0 2024-02-15T18:34:53.245 INFO:tasks.workunit.client.0.smithi038.stdout:0/245: sync 2024-02-15T18:34:53.245 INFO:tasks.workunit.client.0.smithi038.stdout:1/149: sync 2024-02-15T18:34:53.245 INFO:tasks.workunit.client.0.smithi038.stdout:0/246: chown d4/d13/d2a/d35/d1e/d32/d48/f4c 3 1 2024-02-15T18:34:53.246 INFO:tasks.workunit.client.0.smithi038.stdout:6/172: dread d2/d4/f27 [0,4194304] 0 2024-02-15T18:34:53.247 INFO:tasks.workunit.client.0.smithi038.stdout:8/317: mknod d1/d7/d15/d4b/c73 0 2024-02-15T18:34:53.248 INFO:tasks.workunit.client.0.smithi038.stdout:3/127: symlink d2/d1b/l21 0 2024-02-15T18:34:53.248 INFO:tasks.workunit.client.0.smithi038.stdout:8/318: chown d1/l19 7503389 1 2024-02-15T18:34:53.248 INFO:tasks.workunit.client.0.smithi038.stdout:7/189: mknod d0/d1b/c41 0 2024-02-15T18:34:53.250 INFO:tasks.workunit.client.0.smithi038.stdout:4/138: dread d11/d1e/d20/d24/f2d [0,4194304] 0 2024-02-15T18:34:53.250 INFO:tasks.workunit.client.0.smithi038.stdout:4/139: dread - d11/d1e/d20/f27 zero size 2024-02-15T18:34:53.250 INFO:tasks.workunit.client.0.smithi038.stdout:4/140: chown d11/d1e/c2a 6088 1 2024-02-15T18:34:53.250 INFO:tasks.workunit.client.0.smithi038.stdout:4/141: write d11/f17 [333493,85627] 0 2024-02-15T18:34:53.254 INFO:tasks.workunit.client.0.smithi038.stdout:2/154: dwrite dd/d12/d1d/f31 [0,4194304] 0 2024-02-15T18:34:53.254 INFO:tasks.workunit.client.0.smithi038.stdout:1/150: link d2/d10/l13 d2/l35 0 2024-02-15T18:34:53.254 INFO:tasks.workunit.client.0.smithi038.stdout:1/151: chown d2/d10/c17 89 1 2024-02-15T18:34:53.254 INFO:tasks.workunit.client.0.smithi038.stdout:7/190: stat d0/d3/l6 0 2024-02-15T18:34:53.255 INFO:tasks.workunit.client.0.smithi038.stdout:4/142: creat d11/d1e/d20/d31/f32 x:0 0 0 2024-02-15T18:34:53.256 INFO:tasks.workunit.client.0.smithi038.stdout:6/173: creat d2/d10/f30 x:0 0 0 2024-02-15T18:34:53.258 INFO:tasks.workunit.client.0.smithi038.stdout:8/319: symlink d1/l74 0 2024-02-15T18:34:53.258 INFO:tasks.workunit.client.0.smithi038.stdout:0/247: dread d4/f5 [0,4194304] 0 2024-02-15T18:34:53.259 INFO:tasks.workunit.client.0.smithi038.stdout:2/155: creat dd/d12/d1d/f35 x:0 0 0 2024-02-15T18:34:53.259 INFO:tasks.workunit.client.0.smithi038.stdout:7/191: creat d0/db/f42 x:0 0 0 2024-02-15T18:34:53.259 INFO:tasks.workunit.client.0.smithi038.stdout:4/143: creat d11/d1e/d20/d31/f33 x:0 0 0 2024-02-15T18:34:53.259 INFO:tasks.workunit.client.0.smithi038.stdout:5/163: dwrite d2/d3/d12/fa [0,4194304] 0 2024-02-15T18:34:53.260 INFO:tasks.workunit.client.0.smithi038.stdout:5/164: dread d2/f1a [0,4194304] 0 2024-02-15T18:34:53.262 INFO:tasks.workunit.client.0.smithi038.stdout:6/174: dread d2/f9 [0,4194304] 0 2024-02-15T18:34:53.262 INFO:tasks.workunit.client.0.smithi038.stdout:8/320: creat d1/d4/d5/d6/f75 x:0 0 0 2024-02-15T18:34:53.262 INFO:tasks.workunit.client.0.smithi038.stdout:0/248: rmdir d4/d13/d19 39 2024-02-15T18:34:53.262 INFO:tasks.workunit.client.0.smithi038.stdout:8/321: dread - d1/d4/d5/d6/f6e zero size 2024-02-15T18:34:53.263 INFO:tasks.workunit.client.0.smithi038.stdout:6/175: mknod d2/d4/c31 0 2024-02-15T18:34:53.265 INFO:tasks.workunit.client.0.smithi038.stdout:0/249: unlink d4/d13/d2a/d35/d1e/d32/d3b/c45 0 2024-02-15T18:34:53.265 INFO:tasks.workunit.client.0.smithi038.stdout:0/250: fdatasync d4/d13/d2a/f44 0 2024-02-15T18:34:53.266 INFO:tasks.workunit.client.0.smithi038.stdout:8/322: rename d1/d4/d5/f27 to d1/d4/d5/d31/d65/f76 0 2024-02-15T18:34:53.267 INFO:tasks.workunit.client.0.smithi038.stdout:4/144: write d11/d1e/d20/d2b/f2e [3648795,93256] 0 2024-02-15T18:34:53.268 INFO:tasks.workunit.client.0.smithi038.stdout:6/176: mknod d2/d4/d2c/c32 0 2024-02-15T18:34:53.268 INFO:tasks.workunit.client.0.smithi038.stdout:6/177: stat d2/d10/c13 0 2024-02-15T18:34:53.268 INFO:tasks.workunit.client.0.smithi038.stdout:4/145: chown d11/f17 456720 1 2024-02-15T18:34:53.268 INFO:tasks.workunit.client.0.smithi038.stdout:4/146: chown le 1787750356 1 2024-02-15T18:34:53.268 INFO:tasks.workunit.client.0.smithi038.stdout:4/147: stat d11/f13 0 2024-02-15T18:34:53.268 INFO:tasks.workunit.client.0.smithi038.stdout:1/152: dread d2/f26 [0,4194304] 0 2024-02-15T18:34:53.269 INFO:tasks.workunit.client.0.smithi038.stdout:6/178: write d2/d4/f1f [93528,33468] 0 2024-02-15T18:34:53.269 INFO:tasks.workunit.client.0.smithi038.stdout:9/169: dwrite d1/f14 [4194304,4194304] 0 2024-02-15T18:34:53.271 INFO:tasks.workunit.client.0.smithi038.stdout:6/179: unlink d2/d4/d28/c2b 0 2024-02-15T18:34:53.271 INFO:tasks.workunit.client.0.smithi038.stdout:2/156: dread dd/d12/f26 [0,4194304] 0 2024-02-15T18:34:53.272 INFO:tasks.workunit.client.0.smithi038.stdout:6/180: write d2/d4/f27 [4482334,14128] 0 2024-02-15T18:34:53.273 INFO:tasks.workunit.client.0.smithi038.stdout:0/251: dread d4/ff [0,4194304] 0 2024-02-15T18:34:53.273 INFO:tasks.workunit.client.0.smithi038.stdout:0/252: fsync d4/d13/d2a/d35/f49 0 2024-02-15T18:34:53.273 INFO:tasks.workunit.client.0.smithi038.stdout:9/170: mkdir d1/da/d36/d3f 0 2024-02-15T18:34:53.273 INFO:tasks.workunit.client.0.smithi038.stdout:9/171: fdatasync d1/da/d20/f26 0 2024-02-15T18:34:53.274 INFO:tasks.workunit.client.0.smithi038.stdout:3/128: dwrite d2/f10 [0,4194304] 0 2024-02-15T18:34:53.274 INFO:tasks.workunit.client.0.smithi038.stdout:2/157: unlink dd/d12/c32 0 2024-02-15T18:34:53.275 INFO:tasks.workunit.client.0.smithi038.stdout:6/181: symlink d2/d4/d19/l33 0 2024-02-15T18:34:53.277 INFO:tasks.workunit.client.0.smithi038.stdout:0/253: rename d4/d13/d19/c31 to d4/d13/d2a/d35/d1e/d32/d48/c51 0 2024-02-15T18:34:53.277 INFO:tasks.workunit.client.0.smithi038.stdout:9/172: truncate d1/f1c 275818 0 2024-02-15T18:34:53.281 INFO:tasks.workunit.client.0.smithi038.stdout:3/129: creat d2/f22 x:0 0 0 2024-02-15T18:34:53.281 INFO:tasks.workunit.client.0.smithi038.stdout:3/130: read - d2/f18 zero size 2024-02-15T18:34:53.282 INFO:tasks.workunit.client.0.smithi038.stdout:9/173: rename d1/l4 to d1/da/d10/d13/l40 0 2024-02-15T18:34:53.291 INFO:tasks.workunit.client.0.smithi038.stdout:0/254: truncate d4/f5 2545555 0 2024-02-15T18:34:53.291 INFO:tasks.workunit.client.0.smithi038.stdout:2/158: dread dd/d12/d1d/f31 [0,4194304] 0 2024-02-15T18:34:53.291 INFO:tasks.workunit.client.0.smithi038.stdout:3/131: symlink d2/d1e/l23 0 2024-02-15T18:34:53.292 INFO:tasks.workunit.client.0.smithi038.stdout:0/255: rename d4/d13/d2a/d35/d1e/c33 to d4/d13/d2a/d35/d1e/c52 0 2024-02-15T18:34:53.293 INFO:tasks.workunit.client.0.smithi038.stdout:2/159: creat dd/d12/f36 x:0 0 0 2024-02-15T18:34:53.294 INFO:tasks.workunit.client.0.smithi038.stdout:3/132: mkdir d2/d24 0 2024-02-15T18:34:53.295 INFO:tasks.workunit.client.0.smithi038.stdout:2/160: rename dd/d12/d18/d2b/l2d to dd/d12/d18/d2b/l37 0 2024-02-15T18:34:53.295 INFO:tasks.workunit.client.0.smithi038.stdout:2/161: write dd/d12/d13/f28 [198830,41545] 0 2024-02-15T18:34:53.295 INFO:tasks.workunit.client.0.smithi038.stdout:3/133: dread d2/fa [0,4194304] 0 2024-02-15T18:34:53.296 INFO:tasks.workunit.client.0.smithi038.stdout:2/162: symlink dd/d12/d13/d33/l38 0 2024-02-15T18:34:53.296 INFO:tasks.workunit.client.0.smithi038.stdout:2/163: write dd/d12/d18/d1f/f29 [102198,5383] 0 2024-02-15T18:34:53.296 INFO:tasks.workunit.client.0.smithi038.stdout:3/134: dread d2/fa [0,4194304] 0 2024-02-15T18:34:53.297 INFO:tasks.workunit.client.0.smithi038.stdout:3/135: mkdir d2/d25 0 2024-02-15T18:34:53.297 INFO:tasks.workunit.client.0.smithi038.stdout:3/136: stat d2/f18 0 2024-02-15T18:34:53.298 INFO:tasks.workunit.client.0.smithi038.stdout:7/192: dwrite d0/d10/f35 [0,4194304] 0 2024-02-15T18:34:53.299 INFO:tasks.workunit.client.0.smithi038.stdout:7/193: write d0/d3/d15/d29/f31 [787488,26156] 0 2024-02-15T18:34:53.299 INFO:tasks.workunit.client.0.smithi038.stdout:3/137: creat d2/d25/f26 x:0 0 0 2024-02-15T18:34:53.299 INFO:tasks.workunit.client.0.smithi038.stdout:3/138: fdatasync d2/f17 0 2024-02-15T18:34:53.300 INFO:tasks.workunit.client.0.smithi038.stdout:7/194: creat d0/d1b/d30/f43 x:0 0 0 2024-02-15T18:34:53.300 INFO:tasks.workunit.client.0.smithi038.stdout:7/195: chown d0/db/d19/d2d/l22 1 1 2024-02-15T18:34:53.301 INFO:tasks.workunit.client.0.smithi038.stdout:3/139: symlink d2/d1e/l27 0 2024-02-15T18:34:53.302 INFO:tasks.workunit.client.0.smithi038.stdout:7/196: creat d0/d3/d15/f44 x:0 0 0 2024-02-15T18:34:53.303 INFO:tasks.workunit.client.0.smithi038.stdout:7/197: stat d0/d1b/d30/f40 0 2024-02-15T18:34:53.303 INFO:tasks.workunit.client.0.smithi038.stdout:7/198: stat d0/d3/c3e 0 2024-02-15T18:34:53.303 INFO:tasks.workunit.client.0.smithi038.stdout:7/199: creat d0/d10/f45 x:0 0 0 2024-02-15T18:34:53.303 INFO:tasks.workunit.client.0.smithi038.stdout:1/153: write d2/ff [3498544,58483] 0 2024-02-15T18:34:53.305 INFO:tasks.workunit.client.0.smithi038.stdout:7/200: symlink d0/db/d19/d2d/l46 0 2024-02-15T18:34:53.305 INFO:tasks.workunit.client.0.smithi038.stdout:1/154: mkdir d2/d10/d19/d22/d36 0 2024-02-15T18:34:53.306 INFO:tasks.workunit.client.0.smithi038.stdout:7/201: mkdir d0/d3/d47 0 2024-02-15T18:34:53.306 INFO:tasks.workunit.client.0.smithi038.stdout:7/202: readlink d0/db/l17 0 2024-02-15T18:34:53.307 INFO:tasks.workunit.client.0.smithi038.stdout:5/165: dwrite d2/d3/f1b [0,4194304] 0 2024-02-15T18:34:53.307 INFO:tasks.workunit.client.0.smithi038.stdout:8/323: dwrite d1/f2 [4194304,4194304] 0 2024-02-15T18:34:53.308 INFO:tasks.workunit.client.0.smithi038.stdout:8/324: write d1/d4/d5/d31/f33 [554412,66664] 0 2024-02-15T18:34:53.308 INFO:tasks.workunit.client.0.smithi038.stdout:7/203: write d0/db/fc [199206,79012] 0 2024-02-15T18:34:53.308 INFO:tasks.workunit.client.0.smithi038.stdout:1/155: creat d2/d10/f37 x:0 0 0 2024-02-15T18:34:53.308 INFO:tasks.workunit.client.0.smithi038.stdout:7/204: getdents d0/d3/d47 0 2024-02-15T18:34:53.309 INFO:tasks.workunit.client.0.smithi038.stdout:5/166: dread d2/f1c [0,4194304] 0 2024-02-15T18:34:53.310 INFO:tasks.workunit.client.0.smithi038.stdout:1/156: write d2/f3 [122389,53742] 0 2024-02-15T18:34:53.312 INFO:tasks.workunit.client.0.smithi038.stdout:1/157: write d2/d10/d19/f1d [1816612,88502] 0 2024-02-15T18:34:53.312 INFO:tasks.workunit.client.0.smithi038.stdout:7/205: getdents d0/d2e 0 2024-02-15T18:34:53.312 INFO:tasks.workunit.client.0.smithi038.stdout:7/206: read d0/db/ff [37996,86552] 0 2024-02-15T18:34:53.312 INFO:tasks.workunit.client.0.smithi038.stdout:7/207: dread - d0/d1b/f3f zero size 2024-02-15T18:34:53.313 INFO:tasks.workunit.client.0.smithi038.stdout:6/182: dwrite d2/d4/f2d [0,4194304] 0 2024-02-15T18:34:53.313 INFO:tasks.workunit.client.0.smithi038.stdout:5/167: symlink d2/d30/l32 0 2024-02-15T18:34:53.316 INFO:tasks.workunit.client.0.smithi038.stdout:1/158: rename d2/ff to d2/d2e/f38 0 2024-02-15T18:34:53.317 INFO:tasks.workunit.client.0.smithi038.stdout:4/148: dwrite d11/d1e/f28 [4194304,4194304] 0 2024-02-15T18:34:53.319 INFO:tasks.workunit.client.0.smithi038.stdout:4/149: write d11/d1e/f28 [6210266,74432] 0 2024-02-15T18:34:53.321 INFO:tasks.workunit.client.0.smithi038.stdout:6/183: write d2/f20 [985247,24187] 0 2024-02-15T18:34:53.321 INFO:tasks.workunit.client.0.smithi038.stdout:6/184: write d2/d4/f1e [1529193,89416] 0 2024-02-15T18:34:53.329 INFO:tasks.workunit.client.0.smithi038.stdout:5/168: mkdir d2/d20/d33 0 2024-02-15T18:34:53.330 INFO:tasks.workunit.client.0.smithi038.stdout:5/169: write d2/f2b [983967,90802] 0 2024-02-15T18:34:53.334 INFO:tasks.workunit.client.0.smithi038.stdout:3/140: fsync d2/d25/f26 0 2024-02-15T18:34:53.335 INFO:tasks.workunit.client.0.smithi038.stdout:8/325: dwrite d1/d4/d5/d6/f6e [0,4194304] 0 2024-02-15T18:34:53.336 INFO:tasks.workunit.client.0.smithi038.stdout:8/326: truncate d1/d7/d22/f5a 1133766 0 2024-02-15T18:34:53.336 INFO:tasks.workunit.client.0.smithi038.stdout:8/327: fsync d1/d3a/d58/f60 0 2024-02-15T18:34:53.336 INFO:tasks.workunit.client.0.smithi038.stdout:2/164: dwrite dd/d12/d13/f28 [0,4194304] 0 2024-02-15T18:34:53.336 INFO:tasks.workunit.client.0.smithi038.stdout:2/165: dread - dd/d12/d18/d1f/f30 zero size 2024-02-15T18:34:53.338 INFO:tasks.workunit.client.0.smithi038.stdout:6/185: write d2/f1a [981617,49119] 0 2024-02-15T18:34:53.339 INFO:tasks.workunit.client.0.smithi038.stdout:0/256: dread d4/f5 [0,4194304] 0 2024-02-15T18:34:53.340 INFO:tasks.workunit.client.0.smithi038.stdout:7/208: dwrite d0/db/f20 [0,4194304] 0 2024-02-15T18:34:53.340 INFO:tasks.workunit.client.0.smithi038.stdout:7/209: fsync d0/d3/d15/f44 0 2024-02-15T18:34:53.346 INFO:tasks.workunit.client.0.smithi038.stdout:6/186: dread d2/d4/f2d [0,4194304] 0 2024-02-15T18:34:53.346 INFO:tasks.workunit.client.0.smithi038.stdout:6/187: fsync d2/d4/ff 0 2024-02-15T18:34:53.346 INFO:tasks.workunit.client.0.smithi038.stdout:1/159: dwrite f1 [0,4194304] 0 2024-02-15T18:34:53.352 INFO:tasks.workunit.client.0.smithi038.stdout:4/150: rename d11/d1e/d20/f27 to d11/d1e/d20/d24/f34 0 2024-02-15T18:34:53.352 INFO:tasks.workunit.client.0.smithi038.stdout:4/151: write d11/f1a [1255158,19089] 0 2024-02-15T18:34:53.353 INFO:tasks.workunit.client.0.smithi038.stdout:5/170: mknod d2/c34 0 2024-02-15T18:34:53.354 INFO:tasks.workunit.client.0.smithi038.stdout:3/141: mknod d2/d25/c28 0 2024-02-15T18:34:53.357 INFO:tasks.workunit.client.0.smithi038.stdout:7/210: dwrite d0/f21 [0,4194304] 0 2024-02-15T18:34:53.357 INFO:tasks.workunit.client.0.smithi038.stdout:4/152: dread d11/d1e/d20/d2b/f2e [0,4194304] 0 2024-02-15T18:34:53.357 INFO:tasks.workunit.client.0.smithi038.stdout:8/328: getdents d1/d4/d10/d43 0 2024-02-15T18:34:53.360 INFO:tasks.workunit.client.0.smithi038.stdout:2/166: getdents dd 0 2024-02-15T18:34:53.360 INFO:tasks.workunit.client.0.smithi038.stdout:2/167: write dd/d12/f26 [4952134,67619] 0 2024-02-15T18:34:53.366 INFO:tasks.workunit.client.0.smithi038.stdout:0/257: mkdir d4/d13/d53 0 2024-02-15T18:34:53.366 INFO:tasks.workunit.client.0.smithi038.stdout:0/258: fsync d4/f3e 0 2024-02-15T18:34:53.366 INFO:tasks.workunit.client.0.smithi038.stdout:0/259: readlink d4/d13/d19/l46 0 2024-02-15T18:34:53.366 INFO:tasks.workunit.client.0.smithi038.stdout:0/260: stat l3 0 2024-02-15T18:34:53.375 INFO:tasks.workunit.client.0.smithi038.stdout:6/188: symlink d2/d4/d28/l34 0 2024-02-15T18:34:53.375 INFO:tasks.workunit.client.0.smithi038.stdout:1/160: mkdir d2/d39 0 2024-02-15T18:34:53.375 INFO:tasks.workunit.client.0.smithi038.stdout:1/161: truncate d2/f1e 2052015 0 2024-02-15T18:34:53.375 INFO:tasks.workunit.client.0.smithi038.stdout:5/171: mkdir d2/d3/d12/d35 0 2024-02-15T18:34:53.375 INFO:tasks.workunit.client.0.smithi038.stdout:3/142: mkdir d2/d1e/d29 0 2024-02-15T18:34:53.376 INFO:tasks.workunit.client.0.smithi038.stdout:4/153: mkdir d11/d1e/d20/d2b/d35 0 2024-02-15T18:34:53.377 INFO:tasks.workunit.client.0.smithi038.stdout:8/329: getdents d1/d4/d5 0 2024-02-15T18:34:53.379 INFO:tasks.workunit.client.0.smithi038.stdout:5/172: dread d2/d20/f21 [0,4194304] 0 2024-02-15T18:34:53.379 INFO:tasks.workunit.client.0.smithi038.stdout:5/173: chown d2/f1a 11958 1 2024-02-15T18:34:53.379 INFO:tasks.workunit.client.0.smithi038.stdout:0/261: rename d4/f15 to d4/d13/f54 0 2024-02-15T18:34:53.379 INFO:tasks.workunit.client.0.smithi038.stdout:0/262: write d4/d13/d2a/d35/d1e/f50 [329428,30370] 0 2024-02-15T18:34:53.380 INFO:tasks.workunit.client.0.smithi038.stdout:7/211: dwrite d0/d3/d15/f44 [0,4194304] 0 2024-02-15T18:34:53.381 INFO:tasks.workunit.client.0.smithi038.stdout:7/212: fsync d0/d3/f7 0 2024-02-15T18:34:53.382 INFO:tasks.workunit.client.0.smithi038.stdout:6/189: unlink d2/d4/f2d 0 2024-02-15T18:34:53.382 INFO:tasks.workunit.client.0.smithi038.stdout:6/190: write d2/f20 [1636678,95687] 0 2024-02-15T18:34:53.382 INFO:tasks.workunit.client.0.smithi038.stdout:2/168: dwrite dd/d12/d1d/f31 [0,4194304] 0 2024-02-15T18:34:53.382 INFO:tasks.workunit.client.0.smithi038.stdout:6/191: read d2/f20 [148138,50486] 0 2024-02-15T18:34:53.382 INFO:tasks.workunit.client.0.smithi038.stdout:1/162: mknod d2/d2e/c3a 0 2024-02-15T18:34:53.382 INFO:tasks.workunit.client.0.smithi038.stdout:5/174: fsync d2/f2b 0 2024-02-15T18:34:53.383 INFO:tasks.workunit.client.0.smithi038.stdout:8/330: dread d1/d4/d5/d31/f33 [0,4194304] 0 2024-02-15T18:34:53.384 INFO:tasks.workunit.client.0.smithi038.stdout:9/174: sync 2024-02-15T18:34:53.385 INFO:tasks.workunit.client.0.smithi038.stdout:3/143: rmdir d2/d1b 39 2024-02-15T18:34:53.386 INFO:tasks.workunit.client.0.smithi038.stdout:4/154: unlink cb 0 2024-02-15T18:34:53.387 INFO:tasks.workunit.client.0.smithi038.stdout:0/263: mkdir d4/d13/d2a/d35/d1e/d32/d3b/d55 0 2024-02-15T18:34:53.387 INFO:tasks.workunit.client.0.smithi038.stdout:0/264: chown d4/l47 0 1 2024-02-15T18:34:53.387 INFO:tasks.workunit.client.0.smithi038.stdout:0/265: readlink d4/d13/l2f 0 2024-02-15T18:34:53.389 INFO:tasks.workunit.client.0.smithi038.stdout:7/213: rmdir d0/d3/d15/d29 39 2024-02-15T18:34:53.389 INFO:tasks.workunit.client.0.smithi038.stdout:0/266: dread d4/ff [0,4194304] 0 2024-02-15T18:34:53.389 INFO:tasks.workunit.client.0.smithi038.stdout:0/267: write d4/f3e [1900192,17345] 0 2024-02-15T18:34:53.390 INFO:tasks.workunit.client.0.smithi038.stdout:0/268: write d4/d13/d2a/f44 [2457340,42950] 0 2024-02-15T18:34:53.390 INFO:tasks.workunit.client.0.smithi038.stdout:0/269: chown d4/d13/d2a/d35/d1e/d32/d3b 168676050 1 2024-02-15T18:34:53.391 INFO:tasks.workunit.client.0.smithi038.stdout:2/169: link dd/d12/d1d/f1a dd/d12/f39 0 2024-02-15T18:34:53.393 INFO:tasks.workunit.client.0.smithi038.stdout:6/192: getdents d2/d10 0 2024-02-15T18:34:53.393 INFO:tasks.workunit.client.0.smithi038.stdout:6/193: write d2/d10/f30 [607310,116951] 0 2024-02-15T18:34:53.393 INFO:tasks.workunit.client.0.smithi038.stdout:6/194: chown d2/d10/f1d 486173 1 2024-02-15T18:34:53.393 INFO:tasks.workunit.client.0.smithi038.stdout:2/170: dread dd/f24 [0,4194304] 0 2024-02-15T18:34:53.394 INFO:tasks.workunit.client.0.smithi038.stdout:1/163: symlink d2/d10/d32/l3b 0 2024-02-15T18:34:53.396 INFO:tasks.workunit.client.0.smithi038.stdout:5/175: creat d2/d20/d33/f36 x:0 0 0 2024-02-15T18:34:53.396 INFO:tasks.workunit.client.0.smithi038.stdout:5/176: fsync d2/d3/d12/f17 0 2024-02-15T18:34:53.397 INFO:tasks.workunit.client.0.smithi038.stdout:7/214: dread d0/d3/d15/f18 [0,4194304] 0 2024-02-15T18:34:53.399 INFO:tasks.workunit.client.0.smithi038.stdout:8/331: mkdir d1/d4/d5/d77 0 2024-02-15T18:34:53.399 INFO:tasks.workunit.client.0.smithi038.stdout:9/175: symlink d1/dc/d2b/l41 0 2024-02-15T18:34:53.401 INFO:tasks.workunit.client.0.smithi038.stdout:3/144: mknod d2/d24/c2a 0 2024-02-15T18:34:53.402 INFO:tasks.workunit.client.0.smithi038.stdout:3/145: write d2/f15 [4380880,74353] 0 2024-02-15T18:34:53.404 INFO:tasks.workunit.client.0.smithi038.stdout:4/155: rename d11/c16 to d11/d1e/d20/d2f/c36 0 2024-02-15T18:34:53.405 INFO:tasks.workunit.client.0.smithi038.stdout:4/156: write d11/f1a [1127067,124369] 0 2024-02-15T18:34:53.405 INFO:tasks.workunit.client.0.smithi038.stdout:4/157: readlink d11/l1c 0 2024-02-15T18:34:53.405 INFO:tasks.workunit.client.0.smithi038.stdout:7/215: dread d0/db/d19/f25 [0,4194304] 0 2024-02-15T18:34:53.406 INFO:tasks.workunit.client.0.smithi038.stdout:7/216: fsync d0/d3/f7 0 2024-02-15T18:34:53.407 INFO:tasks.workunit.client.0.smithi038.stdout:6/195: rename d2/l23 to d2/l35 0 2024-02-15T18:34:53.407 INFO:tasks.workunit.client.0.smithi038.stdout:0/270: creat d4/d13/d2a/d35/d1e/d32/d48/f56 x:0 0 0 2024-02-15T18:34:53.408 INFO:tasks.workunit.client.0.smithi038.stdout:1/164: unlink d2/d10/f1b 0 2024-02-15T18:34:53.408 INFO:tasks.workunit.client.0.smithi038.stdout:5/177: getdents d2 0 2024-02-15T18:34:53.408 INFO:tasks.workunit.client.0.smithi038.stdout:5/178: chown d2/d30/c31 1180215 1 2024-02-15T18:34:53.408 INFO:tasks.workunit.client.0.smithi038.stdout:5/179: dread - d2/d3/d12/f2d zero size 2024-02-15T18:34:53.408 INFO:tasks.workunit.client.0.smithi038.stdout:5/180: chown d2/d3/d12/d35 34072 1 2024-02-15T18:34:53.409 INFO:tasks.workunit.client.0.smithi038.stdout:8/332: creat d1/d3a/d58/f78 x:0 0 0 2024-02-15T18:34:53.409 INFO:tasks.workunit.client.0.smithi038.stdout:5/181: dread d2/d3/d12/f28 [0,4194304] 0 2024-02-15T18:34:53.411 INFO:tasks.workunit.client.0.smithi038.stdout:3/146: unlink d2/f22 0 2024-02-15T18:34:53.412 INFO:tasks.workunit.client.0.smithi038.stdout:4/158: creat d11/d1e/f37 x:0 0 0 2024-02-15T18:34:53.413 INFO:tasks.workunit.client.0.smithi038.stdout:0/271: unlink d4/f3e 0 2024-02-15T18:34:53.413 INFO:tasks.workunit.client.0.smithi038.stdout:9/176: dwrite d1/fb [0,4194304] 0 2024-02-15T18:34:53.414 INFO:tasks.workunit.client.0.smithi038.stdout:1/165: mkdir d2/d28/d3c 0 2024-02-15T18:34:53.414 INFO:tasks.workunit.client.0.smithi038.stdout:1/166: fsync d2/f1e 0 2024-02-15T18:34:53.420 INFO:tasks.workunit.client.0.smithi038.stdout:6/196: creat d2/d4/d28/f36 x:0 0 0 2024-02-15T18:34:53.421 INFO:tasks.workunit.client.0.smithi038.stdout:8/333: truncate d1/d4/d5/d31/d65/f76 1229920 0 2024-02-15T18:34:53.423 INFO:tasks.workunit.client.0.smithi038.stdout:5/182: rename d2/d3/d12/f2d to d2/d20/f37 0 2024-02-15T18:34:53.425 INFO:tasks.workunit.client.0.smithi038.stdout:4/159: rmdir d11/d1e/d20 39 2024-02-15T18:34:53.427 INFO:tasks.workunit.client.0.smithi038.stdout:9/177: creat d1/da/d10/d19/f42 x:0 0 0 2024-02-15T18:34:53.427 INFO:tasks.workunit.client.0.smithi038.stdout:9/178: readlink d1/dc/d2b/l41 0 2024-02-15T18:34:53.427 INFO:tasks.workunit.client.0.smithi038.stdout:9/179: write d1/da/d10/d13/f1e [681978,15434] 0 2024-02-15T18:34:53.428 INFO:tasks.workunit.client.0.smithi038.stdout:1/167: mknod d2/d10/d19/d22/c3d 0 2024-02-15T18:34:53.428 INFO:tasks.workunit.client.0.smithi038.stdout:1/168: write d2/d10/d19/d22/f30 [719585,36327] 0 2024-02-15T18:34:53.428 INFO:tasks.workunit.client.0.smithi038.stdout:6/197: creat d2/d4/d2c/f37 x:0 0 0 2024-02-15T18:34:53.433 INFO:tasks.workunit.client.0.smithi038.stdout:8/334: mkdir d1/d4/d5/d77/d79 0 2024-02-15T18:34:53.433 INFO:tasks.workunit.client.0.smithi038.stdout:3/147: rename d2/d1e/l27 to d2/d1e/d29/l2b 0 2024-02-15T18:34:53.433 INFO:tasks.workunit.client.0.smithi038.stdout:5/183: getdents d2/d20/d33 0 2024-02-15T18:34:53.433 INFO:tasks.workunit.client.0.smithi038.stdout:5/184: fdatasync d2/d3/ff 0 2024-02-15T18:34:53.433 INFO:tasks.workunit.client.0.smithi038.stdout:5/185: dread - d2/d20/d33/f36 zero size 2024-02-15T18:34:53.433 INFO:tasks.workunit.client.0.smithi038.stdout:5/186: readlink d2/d20/l2c 0 2024-02-15T18:34:53.434 INFO:tasks.workunit.client.0.smithi038.stdout:4/160: symlink d11/d1e/d20/d2b/l38 0 2024-02-15T18:34:53.435 INFO:tasks.workunit.client.0.smithi038.stdout:9/180: mkdir d1/da/d10/d13/d3d/d43 0 2024-02-15T18:34:53.437 INFO:tasks.workunit.client.0.smithi038.stdout:8/335: rmdir d1/d7/d22 39 2024-02-15T18:34:53.437 INFO:tasks.workunit.client.0.smithi038.stdout:8/336: truncate d1/d3a/f6c 294468 0 2024-02-15T18:34:53.438 INFO:tasks.workunit.client.0.smithi038.stdout:7/217: dwrite d0/f9 [0,4194304] 0 2024-02-15T18:34:53.438 INFO:tasks.workunit.client.0.smithi038.stdout:7/218: chown d0/db/ld 7 1 2024-02-15T18:34:53.438 INFO:tasks.workunit.client.0.smithi038.stdout:7/219: truncate d0/db/f42 510749 0 2024-02-15T18:34:53.438 INFO:tasks.workunit.client.0.smithi038.stdout:2/171: dwrite fc [0,4194304] 0 2024-02-15T18:34:53.439 INFO:tasks.workunit.client.0.smithi038.stdout:3/148: creat d2/d1e/f2c x:0 0 0 2024-02-15T18:34:53.439 INFO:tasks.workunit.client.0.smithi038.stdout:3/149: stat d2/d1b/l21 0 2024-02-15T18:34:53.439 INFO:tasks.workunit.client.0.smithi038.stdout:3/150: fsync d2/f18 0 2024-02-15T18:34:53.440 INFO:tasks.workunit.client.0.smithi038.stdout:0/272: rename d4/d13/d2a/d35/d1e/c52 to d4/d13/c57 0 2024-02-15T18:34:53.441 INFO:tasks.workunit.client.0.smithi038.stdout:5/187: rename d2/f2a to d2/f38 0 2024-02-15T18:34:53.444 INFO:tasks.workunit.client.0.smithi038.stdout:4/161: mkdir d11/d1e/d20/d2f/d39 0 2024-02-15T18:34:53.446 INFO:tasks.workunit.client.0.smithi038.stdout:9/181: creat d1/da/d10/d24/f44 x:0 0 0 2024-02-15T18:34:53.446 INFO:tasks.workunit.client.0.smithi038.stdout:9/182: write d1/da/d10/d24/f44 [10583,109787] 0 2024-02-15T18:34:53.447 INFO:tasks.workunit.client.0.smithi038.stdout:1/169: dread d2/d2e/f38 [0,4194304] 0 2024-02-15T18:34:53.451 INFO:tasks.workunit.client.0.smithi038.stdout:7/220: creat d0/f48 x:0 0 0 2024-02-15T18:34:53.452 INFO:tasks.workunit.client.0.smithi038.stdout:7/221: read d0/d3/d15/f1f [2021835,28969] 0 2024-02-15T18:34:53.452 INFO:tasks.workunit.client.0.smithi038.stdout:7/222: fdatasync d0/db/d19/d2d/f36 0 2024-02-15T18:34:53.457 INFO:tasks.workunit.client.0.smithi038.stdout:7/223: dread d0/d10/f1c [0,4194304] 0 2024-02-15T18:34:53.457 INFO:tasks.workunit.client.0.smithi038.stdout:3/151: symlink d2/d24/l2d 0 2024-02-15T18:34:53.457 INFO:tasks.workunit.client.0.smithi038.stdout:3/152: chown d2/l1c 148919 1 2024-02-15T18:34:53.461 INFO:tasks.workunit.client.0.smithi038.stdout:0/273: getdents d4/d13/d2a/d35/d1e 0 2024-02-15T18:34:53.463 INFO:tasks.workunit.client.0.smithi038.stdout:5/188: creat d2/f39 x:0 0 0 2024-02-15T18:34:53.467 INFO:tasks.workunit.client.0.smithi038.stdout:4/162: mknod d11/d1e/d20/d2b/c3a 0 2024-02-15T18:34:53.468 INFO:tasks.workunit.client.0.smithi038.stdout:9/183: rename d1/da/d20/c2d to d1/da/d10/d13/d3d/d43/c45 0 2024-02-15T18:34:53.469 INFO:tasks.workunit.client.0.smithi038.stdout:1/170: mkdir d2/d10/d19/d2f/d3e 0 2024-02-15T18:34:53.470 INFO:tasks.workunit.client.0.smithi038.stdout:7/224: getdents d0 0 2024-02-15T18:34:53.471 INFO:tasks.workunit.client.0.smithi038.stdout:5/189: getdents d2/d3/d12 0 2024-02-15T18:34:53.472 INFO:tasks.workunit.client.0.smithi038.stdout:5/190: fdatasync d2/d3/f1d 0 2024-02-15T18:34:53.474 INFO:tasks.workunit.client.0.smithi038.stdout:3/153: rename d2/d1e/f2c to d2/d24/f2e 0 2024-02-15T18:34:53.474 INFO:tasks.workunit.client.0.smithi038.stdout:0/274: rename d4/d13/d2a to d4/d13/d2a/d35/d1e/d32/d58 22 2024-02-15T18:34:53.474 INFO:tasks.workunit.client.0.smithi038.stdout:0/275: write d4/d13/d2a/f4e [1018055,102986] 0 2024-02-15T18:34:53.475 INFO:tasks.workunit.client.0.smithi038.stdout:9/184: link d1/f14 d1/da/d10/d2f/f46 0 2024-02-15T18:34:53.475 INFO:tasks.workunit.client.0.smithi038.stdout:9/185: write d1/da/d10/d2f/f46 [9007735,106688] 0 2024-02-15T18:34:53.476 INFO:tasks.workunit.client.0.smithi038.stdout:1/171: mknod d2/d10/d19/c3f 0 2024-02-15T18:34:53.477 INFO:tasks.workunit.client.0.smithi038.stdout:7/225: mknod d0/db/c49 0 2024-02-15T18:34:53.477 INFO:tasks.workunit.client.0.smithi038.stdout:7/226: stat d0/d2e/f37 0 2024-02-15T18:34:53.478 INFO:tasks.workunit.client.0.smithi038.stdout:5/191: mknod d2/d30/c3a 0 2024-02-15T18:34:53.481 INFO:tasks.workunit.client.0.smithi038.stdout:9/186: creat d1/da/d10/d2f/f47 x:0 0 0 2024-02-15T18:34:53.482 INFO:tasks.workunit.client.0.smithi038.stdout:1/172: mknod d2/d28/c40 0 2024-02-15T18:34:53.482 INFO:tasks.workunit.client.0.smithi038.stdout:1/173: fdatasync d2/f3 0 2024-02-15T18:34:53.483 INFO:tasks.workunit.client.0.smithi038.stdout:3/154: rmdir d2 39 2024-02-15T18:34:53.484 INFO:tasks.workunit.client.0.smithi038.stdout:4/163: dwrite d11/f1b [0,4194304] 0 2024-02-15T18:34:53.485 INFO:tasks.workunit.client.0.smithi038.stdout:1/174: dread d2/f1e [0,4194304] 0 2024-02-15T18:34:53.485 INFO:tasks.workunit.client.0.smithi038.stdout:7/227: symlink d0/d3/d47/l4a 0 2024-02-15T18:34:53.486 INFO:tasks.workunit.client.0.smithi038.stdout:5/192: rmdir d2/d20/d33 39 2024-02-15T18:34:53.487 INFO:tasks.workunit.client.0.smithi038.stdout:9/187: mkdir d1/da/d36/d3f/d48 0 2024-02-15T18:34:53.487 INFO:tasks.workunit.client.0.smithi038.stdout:9/188: stat d1/f1c 0 2024-02-15T18:34:53.489 INFO:tasks.workunit.client.0.smithi038.stdout:1/175: symlink d2/d10/l41 0 2024-02-15T18:34:53.489 INFO:tasks.workunit.client.0.smithi038.stdout:7/228: mknod d0/d2e/c4b 0 2024-02-15T18:34:53.492 INFO:tasks.workunit.client.0.smithi038.stdout:9/189: rename d1/d11 to d1/da/d10/d24/d49 0 2024-02-15T18:34:53.492 INFO:tasks.workunit.client.0.smithi038.stdout:9/190: readlink d1/da/d10/d13/l40 0 2024-02-15T18:34:53.492 INFO:tasks.workunit.client.0.smithi038.stdout:0/276: dwrite d4/d13/d2a/d35/f49 [4194304,4194304] 0 2024-02-15T18:34:53.492 INFO:tasks.workunit.client.0.smithi038.stdout:0/277: chown d4/d13/f39 350762 1 2024-02-15T18:34:53.493 INFO:tasks.workunit.client.0.smithi038.stdout:1/176: creat d2/f42 x:0 0 0 2024-02-15T18:34:53.493 INFO:tasks.workunit.client.0.smithi038.stdout:9/191: write d1/da/d10/d2f/f46 [907267,75499] 0 2024-02-15T18:34:53.493 INFO:tasks.workunit.client.0.smithi038.stdout:3/155: write d2/f1d [868326,38319] 0 2024-02-15T18:34:53.493 INFO:tasks.workunit.client.0.smithi038.stdout:7/229: mknod d0/d2e/c4c 0 2024-02-15T18:34:53.499 INFO:tasks.workunit.client.0.smithi038.stdout:0/278: symlink d4/d13/d2a/d35/d1e/d32/d3b/l59 0 2024-02-15T18:34:53.501 INFO:tasks.workunit.client.0.smithi038.stdout:1/177: getdents d2/d28 0 2024-02-15T18:34:53.503 INFO:tasks.workunit.client.0.smithi038.stdout:3/156: creat d2/f2f x:0 0 0 2024-02-15T18:34:53.504 INFO:tasks.workunit.client.0.smithi038.stdout:7/230: creat d0/d2e/f4d x:0 0 0 2024-02-15T18:34:53.504 INFO:tasks.workunit.client.0.smithi038.stdout:7/231: fsync d0/d3/d15/f18 0 2024-02-15T18:34:53.506 INFO:tasks.workunit.client.0.smithi038.stdout:1/178: creat d2/d28/f43 x:0 0 0 2024-02-15T18:34:53.508 INFO:tasks.workunit.client.0.smithi038.stdout:7/232: rename d0/d3/d15/f1f to d0/db/d19/f4e 0 2024-02-15T18:34:53.508 INFO:tasks.workunit.client.0.smithi038.stdout:7/233: dread - d0/d2e/f3b zero size 2024-02-15T18:34:53.508 INFO:tasks.workunit.client.0.smithi038.stdout:1/179: truncate d2/f4 76675 0 2024-02-15T18:34:53.511 INFO:tasks.workunit.client.0.smithi038.stdout:4/164: dwrite d11/f1a [0,4194304] 0 2024-02-15T18:34:53.511 INFO:tasks.workunit.client.0.smithi038.stdout:4/165: chown d11/d1e/d20/d2b/c3a 29446066 1 2024-02-15T18:34:53.516 INFO:tasks.workunit.client.0.smithi038.stdout:4/166: mknod d11/d1e/c3b 0 2024-02-15T18:34:53.516 INFO:tasks.workunit.client.0.smithi038.stdout:4/167: chown d11/d1e/d20/d2f/c36 159470116 1 2024-02-15T18:34:53.516 INFO:tasks.workunit.client.0.smithi038.stdout:9/192: dwrite d1/fb [0,4194304] 0 2024-02-15T18:34:53.518 INFO:tasks.workunit.client.0.smithi038.stdout:4/168: unlink d11/d1e/d20/d24/f26 0 2024-02-15T18:34:53.518 INFO:tasks.workunit.client.0.smithi038.stdout:4/169: stat d11/d1e/d20/d2b/f30 0 2024-02-15T18:34:53.519 INFO:tasks.workunit.client.0.smithi038.stdout:9/193: getdents d1/da/d10/d24/d49 0 2024-02-15T18:34:53.520 INFO:tasks.workunit.client.0.smithi038.stdout:0/279: dwrite d4/f12 [0,4194304] 0 2024-02-15T18:34:53.520 INFO:tasks.workunit.client.0.smithi038.stdout:3/157: dread d2/f1d [0,4194304] 0 2024-02-15T18:34:53.520 INFO:tasks.workunit.client.0.smithi038.stdout:3/158: dread - d2/f18 zero size 2024-02-15T18:34:53.521 INFO:tasks.workunit.client.0.smithi038.stdout:4/170: rename d11/d1e/d20/d2b/c3a to d11/d1e/d20/d2f/d39/c3c 0 2024-02-15T18:34:53.523 INFO:tasks.workunit.client.0.smithi038.stdout:9/194: getdents d1/da/d36 0 2024-02-15T18:34:53.526 INFO:tasks.workunit.client.0.smithi038.stdout:3/159: creat d2/d25/f30 x:0 0 0 2024-02-15T18:34:53.527 INFO:tasks.workunit.client.0.smithi038.stdout:4/171: creat d11/d1e/d20/d2f/f3d x:0 0 0 2024-02-15T18:34:53.528 INFO:tasks.workunit.client.0.smithi038.stdout:3/160: creat d2/d1e/d29/f31 x:0 0 0 2024-02-15T18:34:53.528 INFO:tasks.workunit.client.0.smithi038.stdout:3/161: truncate d2/f11 786275 0 2024-02-15T18:34:53.529 INFO:tasks.workunit.client.0.smithi038.stdout:4/172: mkdir d11/d1e/d3e 0 2024-02-15T18:34:53.531 INFO:tasks.workunit.client.0.smithi038.stdout:3/162: rename d2/f11 to d2/d1e/f32 0 2024-02-15T18:34:53.534 INFO:tasks.workunit.client.0.smithi038.stdout:0/280: write d4/d13/d2a/d35/f49 [5943297,94636] 0 2024-02-15T18:34:53.534 INFO:tasks.workunit.client.0.smithi038.stdout:7/234: dwrite d0/db/f20 [0,4194304] 0 2024-02-15T18:34:53.534 INFO:tasks.workunit.client.0.smithi038.stdout:7/235: truncate d0/d2e/f37 653642 0 2024-02-15T18:34:53.536 INFO:tasks.workunit.client.0.smithi038.stdout:3/163: rmdir d2/d1b 39 2024-02-15T18:34:53.536 INFO:tasks.workunit.client.0.smithi038.stdout:3/164: fdatasync d2/fa 0 2024-02-15T18:34:53.537 INFO:tasks.workunit.client.0.smithi038.stdout:1/180: dwrite d2/f3 [0,4194304] 0 2024-02-15T18:34:53.540 INFO:tasks.workunit.client.0.smithi038.stdout:7/236: mkdir d0/d3/d15/d4f 0 2024-02-15T18:34:53.540 INFO:tasks.workunit.client.0.smithi038.stdout:3/165: mknod d2/c33 0 2024-02-15T18:34:53.541 INFO:tasks.workunit.client.0.smithi038.stdout:1/181: read d2/f11 [1900255,127575] 0 2024-02-15T18:34:53.542 INFO:tasks.workunit.client.0.smithi038.stdout:3/166: write d2/f17 [3695896,128022] 0 2024-02-15T18:34:53.544 INFO:tasks.workunit.client.0.smithi038.stdout:7/237: mkdir d0/d1b/d30/d50 0 2024-02-15T18:34:53.544 INFO:tasks.workunit.client.0.smithi038.stdout:3/167: symlink d2/d25/l34 0 2024-02-15T18:34:53.545 INFO:tasks.workunit.client.0.smithi038.stdout:7/238: creat d0/d2e/f51 x:0 0 0 2024-02-15T18:34:53.546 INFO:tasks.workunit.client.0.smithi038.stdout:3/168: symlink d2/d25/l35 0 2024-02-15T18:34:53.547 INFO:tasks.workunit.client.0.smithi038.stdout:7/239: creat d0/db/d19/d2d/f52 x:0 0 0 2024-02-15T18:34:53.548 INFO:tasks.workunit.client.0.smithi038.stdout:3/169: creat d2/d1b/f36 x:0 0 0 2024-02-15T18:34:53.549 INFO:tasks.workunit.client.0.smithi038.stdout:7/240: symlink d0/d3/d47/l53 0 2024-02-15T18:34:53.550 INFO:tasks.workunit.client.0.smithi038.stdout:7/241: chown d0/db/d19/d2d/f36 7824 1 2024-02-15T18:34:53.551 INFO:tasks.workunit.client.0.smithi038.stdout:7/242: creat d0/f54 x:0 0 0 2024-02-15T18:34:53.553 INFO:tasks.workunit.client.0.smithi038.stdout:4/173: dwrite d11/d1e/d20/d24/f34 [0,4194304] 0 2024-02-15T18:34:53.554 INFO:tasks.workunit.client.0.smithi038.stdout:0/281: dwrite d4/d13/d2a/f4e [0,4194304] 0 2024-02-15T18:34:53.554 INFO:tasks.workunit.client.0.smithi038.stdout:0/282: chown d4/d13/d19/f43 579 1 2024-02-15T18:34:53.555 INFO:tasks.workunit.client.0.smithi038.stdout:0/283: dread d4/d13/d2a/d35/d1e/f50 [0,4194304] 0 2024-02-15T18:34:53.556 INFO:tasks.workunit.client.0.smithi038.stdout:0/284: readlink d4/d13/d2a/d35/d1e/d32/l4a 0 2024-02-15T18:34:53.556 INFO:tasks.workunit.client.0.smithi038.stdout:1/182: dread d2/f2d [0,4194304] 0 2024-02-15T18:34:53.559 INFO:tasks.workunit.client.0.smithi038.stdout:7/243: dread d0/d3/f7 [0,4194304] 0 2024-02-15T18:34:53.559 INFO:tasks.workunit.client.0.smithi038.stdout:7/244: chown d0/d10/f45 43949081 1 2024-02-15T18:34:53.560 INFO:tasks.workunit.client.0.smithi038.stdout:2/172: sync 2024-02-15T18:34:53.560 INFO:tasks.workunit.client.0.smithi038.stdout:6/198: sync 2024-02-15T18:34:53.560 INFO:tasks.workunit.client.0.smithi038.stdout:8/337: sync 2024-02-15T18:34:53.560 INFO:tasks.workunit.client.0.smithi038.stdout:8/338: write d1/d7/f51 [1682438,64374] 0 2024-02-15T18:34:53.560 INFO:tasks.workunit.client.0.smithi038.stdout:2/173: write dd/d12/d18/d1f/f29 [86086,36372] 0 2024-02-15T18:34:53.563 INFO:tasks.workunit.client.0.smithi038.stdout:4/174: symlink d11/d1e/d20/d2f/d39/l3f 0 2024-02-15T18:34:53.566 INFO:tasks.workunit.client.0.smithi038.stdout:0/285: getdents d4/d13/d19 0 2024-02-15T18:34:53.567 INFO:tasks.workunit.client.0.smithi038.stdout:6/199: mkdir d2/d4/d19/d38 0 2024-02-15T18:34:53.567 INFO:tasks.workunit.client.0.smithi038.stdout:7/245: dread d0/d3/d15/d29/f31 [0,4194304] 0 2024-02-15T18:34:53.567 INFO:tasks.workunit.client.0.smithi038.stdout:3/170: dwrite d2/f1a [0,4194304] 0 2024-02-15T18:34:53.568 INFO:tasks.workunit.client.0.smithi038.stdout:8/339: creat d1/d3a/f7a x:0 0 0 2024-02-15T18:34:53.569 INFO:tasks.workunit.client.0.smithi038.stdout:0/286: rmdir d4/d13/d2a/d35/d1e/d32/d3b/d55 0 2024-02-15T18:34:53.571 INFO:tasks.workunit.client.0.smithi038.stdout:7/246: mknod d0/d1b/d30/d50/c55 0 2024-02-15T18:34:53.574 INFO:tasks.workunit.client.0.smithi038.stdout:4/175: rename d11/d1e/d20/d2f/d39 to d11/d1e/d20/d24/d40 0 2024-02-15T18:34:53.574 INFO:tasks.workunit.client.0.smithi038.stdout:3/171: dread d2/f10 [0,4194304] 0 2024-02-15T18:34:53.575 INFO:tasks.workunit.client.0.smithi038.stdout:6/200: dread d2/f20 [0,4194304] 0 2024-02-15T18:34:53.578 INFO:tasks.workunit.client.0.smithi038.stdout:0/287: symlink d4/d13/l5a 0 2024-02-15T18:34:53.580 INFO:tasks.workunit.client.0.smithi038.stdout:8/340: rename l0 to d1/d4/d5/d6/d6d/l7b 0 2024-02-15T18:34:53.580 INFO:tasks.workunit.client.0.smithi038.stdout:8/341: read - d1/f6f zero size 2024-02-15T18:34:53.580 INFO:tasks.workunit.client.0.smithi038.stdout:4/176: mknod d11/d1e/c41 0 2024-02-15T18:34:53.580 INFO:tasks.workunit.client.0.smithi038.stdout:4/177: fdatasync f4 0 2024-02-15T18:34:53.580 INFO:tasks.workunit.client.0.smithi038.stdout:4/178: chown l0 325545 1 2024-02-15T18:34:53.580 INFO:tasks.workunit.client.0.smithi038.stdout:3/172: mknod d2/c37 0 2024-02-15T18:34:53.581 INFO:tasks.workunit.client.0.smithi038.stdout:8/342: creat d1/d7/d22/f7c x:0 0 0 2024-02-15T18:34:53.583 INFO:tasks.workunit.client.0.smithi038.stdout:4/179: symlink d11/d1e/d3e/l42 0 2024-02-15T18:34:53.583 INFO:tasks.workunit.client.0.smithi038.stdout:4/180: dread - d11/d1e/d20/d31/f32 zero size 2024-02-15T18:34:53.583 INFO:tasks.workunit.client.0.smithi038.stdout:4/181: chown d11/d1e/c3b 234 1 2024-02-15T18:34:53.583 INFO:tasks.workunit.client.0.smithi038.stdout:3/173: creat d2/f38 x:0 0 0 2024-02-15T18:34:53.584 INFO:tasks.workunit.client.0.smithi038.stdout:8/343: mknod d1/d7/d22/c7d 0 2024-02-15T18:34:53.586 INFO:tasks.workunit.client.0.smithi038.stdout:1/183: dwrite d2/f3 [0,4194304] 0 2024-02-15T18:34:53.587 INFO:tasks.workunit.client.0.smithi038.stdout:4/182: mkdir d11/d1e/d43 0 2024-02-15T18:34:53.588 INFO:tasks.workunit.client.0.smithi038.stdout:3/174: link d2/lc d2/d1e/d29/l39 0 2024-02-15T18:34:53.589 INFO:tasks.workunit.client.0.smithi038.stdout:8/344: unlink d1/d7/d15/c62 0 2024-02-15T18:34:53.590 INFO:tasks.workunit.client.0.smithi038.stdout:1/184: creat d2/d10/d19/d22/d36/f44 x:0 0 0 2024-02-15T18:34:53.590 INFO:tasks.workunit.client.0.smithi038.stdout:1/185: chown d2/f3 1324890890 1 2024-02-15T18:34:53.590 INFO:tasks.workunit.client.0.smithi038.stdout:1/186: fsync d2/f2d 0 2024-02-15T18:34:53.592 INFO:tasks.workunit.client.0.smithi038.stdout:4/183: link le d11/d1e/d3e/l44 0 2024-02-15T18:34:53.592 INFO:tasks.workunit.client.0.smithi038.stdout:8/345: mknod d1/d3a/c7e 0 2024-02-15T18:34:53.596 INFO:tasks.workunit.client.0.smithi038.stdout:4/184: write f4 [1032090,16593] 0 2024-02-15T18:34:53.596 INFO:tasks.workunit.client.0.smithi038.stdout:0/288: dwrite d4/d13/d19/f43 [0,4194304] 0 2024-02-15T18:34:53.598 INFO:tasks.workunit.client.0.smithi038.stdout:0/289: write d4/d13/d2a/d35/d1e/f20 [135181,57460] 0 2024-02-15T18:34:53.598 INFO:tasks.workunit.client.0.smithi038.stdout:1/187: dread d2/f1e [0,4194304] 0 2024-02-15T18:34:53.599 INFO:tasks.workunit.client.0.smithi038.stdout:0/290: write d4/f5 [397256,76829] 0 2024-02-15T18:34:53.599 INFO:tasks.workunit.client.0.smithi038.stdout:6/201: dwrite d2/f2a [0,4194304] 0 2024-02-15T18:34:53.600 INFO:tasks.workunit.client.0.smithi038.stdout:4/185: creat d11/d1e/d43/f45 x:0 0 0 2024-02-15T18:34:53.601 INFO:tasks.workunit.client.0.smithi038.stdout:0/291: symlink d4/l5b 0 2024-02-15T18:34:53.605 INFO:tasks.workunit.client.0.smithi038.stdout:6/202: dread d2/f3 [0,4194304] 0 2024-02-15T18:34:53.605 INFO:tasks.workunit.client.0.smithi038.stdout:0/292: creat d4/d13/d19/f5c x:0 0 0 2024-02-15T18:34:53.606 INFO:tasks.workunit.client.0.smithi038.stdout:0/293: rename d4/d13/d2a/d35/d1e/f3d to d4/d13/d2a/d35/d1e/f5d 0 2024-02-15T18:34:53.615 INFO:tasks.workunit.client.0.smithi038.stdout:3/175: dwrite d2/fa [0,4194304] 0 2024-02-15T18:34:53.615 INFO:tasks.workunit.client.0.smithi038.stdout:4/186: dread d11/f1d [0,4194304] 0 2024-02-15T18:34:53.615 INFO:tasks.workunit.client.0.smithi038.stdout:8/346: dwrite d1/d7/d26/f34 [0,4194304] 0 2024-02-15T18:34:53.615 INFO:tasks.workunit.client.0.smithi038.stdout:4/187: stat d11/d1e/d20/d2b 0 2024-02-15T18:34:53.615 INFO:tasks.workunit.client.0.smithi038.stdout:8/347: chown d1/d4/d5/d6/f2a 7041 1 2024-02-15T18:34:53.616 INFO:tasks.workunit.client.0.smithi038.stdout:3/176: fsync d2/d24/f2e 0 2024-02-15T18:34:53.617 INFO:tasks.workunit.client.0.smithi038.stdout:8/348: dread d1/d4/d10/d43/f70 [0,4194304] 0 2024-02-15T18:34:53.618 INFO:tasks.workunit.client.0.smithi038.stdout:5/193: sync 2024-02-15T18:34:53.619 INFO:tasks.workunit.client.0.smithi038.stdout:4/188: rename d11/d1e/d20/d31/f33 to d11/d1e/d20/d24/f46 0 2024-02-15T18:34:53.619 INFO:tasks.workunit.client.0.smithi038.stdout:3/177: mkdir d2/d1e/d29/d3a 0 2024-02-15T18:34:53.620 INFO:tasks.workunit.client.0.smithi038.stdout:0/294: dwrite d4/d13/d2a/d35/d1e/f20 [4194304,4194304] 0 2024-02-15T18:34:53.620 INFO:tasks.workunit.client.0.smithi038.stdout:8/349: creat d1/d4/d5/d31/f7f x:0 0 0 2024-02-15T18:34:53.622 INFO:tasks.workunit.client.0.smithi038.stdout:4/189: creat d11/d1e/d3e/f47 x:0 0 0 2024-02-15T18:34:53.622 INFO:tasks.workunit.client.0.smithi038.stdout:4/190: readlink ld 0 2024-02-15T18:34:53.622 INFO:tasks.workunit.client.0.smithi038.stdout:4/191: fsync f4 0 2024-02-15T18:34:53.622 INFO:tasks.workunit.client.0.smithi038.stdout:5/194: dread d2/f38 [0,4194304] 0 2024-02-15T18:34:53.623 INFO:tasks.workunit.client.0.smithi038.stdout:0/295: mknod d4/d13/d2a/c5e 0 2024-02-15T18:34:53.625 INFO:tasks.workunit.client.0.smithi038.stdout:8/350: rmdir d1/d7 39 2024-02-15T18:34:53.626 INFO:tasks.workunit.client.0.smithi038.stdout:3/178: dread d2/f15 [0,4194304] 0 2024-02-15T18:34:53.629 INFO:tasks.workunit.client.0.smithi038.stdout:4/192: mkdir d11/d1e/d20/d31/d48 0 2024-02-15T18:34:53.630 INFO:tasks.workunit.client.0.smithi038.stdout:5/195: creat d2/d20/f3b x:0 0 0 2024-02-15T18:34:53.630 INFO:tasks.workunit.client.0.smithi038.stdout:5/196: dread - d2/d3/f2f zero size 2024-02-15T18:34:53.630 INFO:tasks.workunit.client.0.smithi038.stdout:5/197: fdatasync d2/d3/f7 0 2024-02-15T18:34:53.631 INFO:tasks.workunit.client.0.smithi038.stdout:0/296: mknod d4/d13/d2a/d35/c5f 0 2024-02-15T18:34:53.631 INFO:tasks.workunit.client.0.smithi038.stdout:8/351: mkdir d1/d4/d5/d31/d65/d80 0 2024-02-15T18:34:53.633 INFO:tasks.workunit.client.0.smithi038.stdout:4/193: read d11/d1e/d20/f22 [3450381,70960] 0 2024-02-15T18:34:53.634 INFO:tasks.workunit.client.0.smithi038.stdout:5/198: mknod d2/c3c 0 2024-02-15T18:34:53.634 INFO:tasks.workunit.client.0.smithi038.stdout:8/352: symlink d1/d4/d5/d77/d79/l81 0 2024-02-15T18:34:53.635 INFO:tasks.workunit.client.0.smithi038.stdout:4/194: symlink d11/d1e/d3e/l49 0 2024-02-15T18:34:53.637 INFO:tasks.workunit.client.0.smithi038.stdout:5/199: unlink d2/d20/f21 0 2024-02-15T18:34:53.637 INFO:tasks.workunit.client.0.smithi038.stdout:5/200: chown d2/d3/f2f 3921 1 2024-02-15T18:34:53.638 INFO:tasks.workunit.client.0.smithi038.stdout:8/353: truncate d1/fb 6328482 0 2024-02-15T18:34:53.640 INFO:tasks.workunit.client.0.smithi038.stdout:3/179: dwrite d2/f10 [0,4194304] 0 2024-02-15T18:34:53.640 INFO:tasks.workunit.client.0.smithi038.stdout:3/180: write d2/f19 [2625507,77474] 0 2024-02-15T18:34:53.642 INFO:tasks.workunit.client.0.smithi038.stdout:3/181: symlink d2/d1e/d29/l3b 0 2024-02-15T18:34:53.643 INFO:tasks.workunit.client.0.smithi038.stdout:3/182: write d2/d25/f26 [598135,115078] 0 2024-02-15T18:34:53.643 INFO:tasks.workunit.client.0.smithi038.stdout:3/183: write d2/f15 [6637248,118335] 0 2024-02-15T18:34:53.644 INFO:tasks.workunit.client.0.smithi038.stdout:3/184: getdents d2/d25 0 2024-02-15T18:34:53.646 INFO:tasks.workunit.client.0.smithi038.stdout:3/185: write d2/f19 [4468834,46986] 0 2024-02-15T18:34:53.649 INFO:tasks.workunit.client.0.smithi038.stdout:0/297: dwrite d4/d13/d2a/f44 [4194304,4194304] 0 2024-02-15T18:34:53.652 INFO:tasks.workunit.client.0.smithi038.stdout:9/195: sync 2024-02-15T18:34:53.653 INFO:tasks.workunit.client.0.smithi038.stdout:9/196: write d1/f16 [234647,53770] 0 2024-02-15T18:34:53.653 INFO:tasks.workunit.client.0.smithi038.stdout:9/197: write d1/dc/fe [887630,33330] 0 2024-02-15T18:34:53.653 INFO:tasks.workunit.client.0.smithi038.stdout:9/198: write d1/da/d36/f37 [4967875,130867] 0 2024-02-15T18:34:53.653 INFO:tasks.workunit.client.0.smithi038.stdout:9/199: stat d1/da/d10 0 2024-02-15T18:34:53.654 INFO:tasks.workunit.client.0.smithi038.stdout:0/298: creat d4/d13/d2a/d35/d1e/d32/d3b/f60 x:0 0 0 2024-02-15T18:34:53.654 INFO:tasks.workunit.client.0.smithi038.stdout:0/299: write d4/d13/d2a/d35/d1e/f4d [1222989,8128] 0 2024-02-15T18:34:53.658 INFO:tasks.workunit.client.0.smithi038.stdout:0/300: mknod d4/d13/d2a/d35/d1e/d32/c61 0 2024-02-15T18:34:53.661 INFO:tasks.workunit.client.0.smithi038.stdout:0/301: mknod d4/d13/c62 0 2024-02-15T18:34:53.661 INFO:tasks.workunit.client.0.smithi038.stdout:0/302: fdatasync d4/f11 0 2024-02-15T18:34:53.665 INFO:tasks.workunit.client.0.smithi038.stdout:4/195: dwrite d11/f1f [0,4194304] 0 2024-02-15T18:34:53.665 INFO:tasks.workunit.client.0.smithi038.stdout:0/303: dread d4/f5 [0,4194304] 0 2024-02-15T18:34:53.665 INFO:tasks.workunit.client.0.smithi038.stdout:5/201: dwrite d2/d3/d12/f14 [0,4194304] 0 2024-02-15T18:34:53.665 INFO:tasks.workunit.client.0.smithi038.stdout:3/186: rename d2/d1e/d29 to d2/d1b/d3c 0 2024-02-15T18:34:53.665 INFO:tasks.workunit.client.0.smithi038.stdout:5/202: read d2/d20/f26 [182233,50119] 0 2024-02-15T18:34:53.667 INFO:tasks.workunit.client.0.smithi038.stdout:4/196: read d11/d1e/d20/f22 [1191541,83535] 0 2024-02-15T18:34:53.667 INFO:tasks.workunit.client.0.smithi038.stdout:4/197: chown f4 0 1 2024-02-15T18:34:53.667 INFO:tasks.workunit.client.0.smithi038.stdout:4/198: chown d11/d1e/d20/d2b 8901 1 2024-02-15T18:34:53.667 INFO:tasks.workunit.client.0.smithi038.stdout:9/200: truncate d1/f16 433603 0 2024-02-15T18:34:53.676 INFO:tasks.workunit.client.0.smithi038.stdout:0/304: truncate d4/f5 2263344 0 2024-02-15T18:34:53.677 INFO:tasks.workunit.client.0.smithi038.stdout:5/203: symlink d2/d3/d12/l3d 0 2024-02-15T18:34:53.677 INFO:tasks.workunit.client.0.smithi038.stdout:5/204: chown d2/d30 10 1 2024-02-15T18:34:53.678 INFO:tasks.workunit.client.0.smithi038.stdout:4/199: mkdir d11/d1e/d43/d4a 0 2024-02-15T18:34:53.680 INFO:tasks.workunit.client.0.smithi038.stdout:0/305: getdents d4/d13/d2a/d35/d1e/d32 0 2024-02-15T18:34:53.680 INFO:tasks.workunit.client.0.smithi038.stdout:0/306: write d4/d13/f39 [607144,54138] 0 2024-02-15T18:34:53.681 INFO:tasks.workunit.client.0.smithi038.stdout:5/205: symlink d2/d20/l3e 0 2024-02-15T18:34:53.681 INFO:tasks.workunit.client.0.smithi038.stdout:5/206: chown d2/d20/l24 43 1 2024-02-15T18:34:53.681 INFO:tasks.workunit.client.0.smithi038.stdout:5/207: write d2/d3/f2f [435160,87184] 0 2024-02-15T18:34:53.681 INFO:tasks.workunit.client.0.smithi038.stdout:3/187: dwrite d2/f1a [0,4194304] 0 2024-02-15T18:34:53.681 INFO:tasks.workunit.client.0.smithi038.stdout:5/208: write d2/d3/ff [1983374,92956] 0 2024-02-15T18:34:53.682 INFO:tasks.workunit.client.0.smithi038.stdout:5/209: chown d2/c13 53647441 1 2024-02-15T18:34:53.682 INFO:tasks.workunit.client.0.smithi038.stdout:9/201: truncate d1/dc/fe 137846 0 2024-02-15T18:34:53.686 INFO:tasks.workunit.client.0.smithi038.stdout:0/307: rename d4/d13/d2a/d35/d1e/d32/f4f to d4/f63 0 2024-02-15T18:34:53.691 INFO:tasks.workunit.client.0.smithi038.stdout:3/188: symlink d2/d1b/l3d 0 2024-02-15T18:34:53.692 INFO:tasks.workunit.client.0.smithi038.stdout:5/210: creat d2/f3f x:0 0 0 2024-02-15T18:34:53.695 INFO:tasks.workunit.client.0.smithi038.stdout:5/211: symlink d2/d20/l40 0 2024-02-15T18:34:53.695 INFO:tasks.workunit.client.0.smithi038.stdout:5/212: dread - d2/d20/f37 zero size 2024-02-15T18:34:53.696 INFO:tasks.workunit.client.0.smithi038.stdout:5/213: read d2/d3/ff [218124,113370] 0 2024-02-15T18:34:53.696 INFO:tasks.workunit.client.0.smithi038.stdout:5/214: stat d2/d30/c31 0 2024-02-15T18:34:53.697 INFO:tasks.workunit.client.0.smithi038.stdout:5/215: truncate d2/d3/d12/fa 2413408 0 2024-02-15T18:34:53.699 INFO:tasks.workunit.client.0.smithi038.stdout:0/308: dwrite d4/d13/d2a/d35/d1e/d32/d48/f56 [0,4194304] 0 2024-02-15T18:34:53.701 INFO:tasks.workunit.client.0.smithi038.stdout:5/216: read d2/f1c [2013982,80119] 0 2024-02-15T18:34:53.701 INFO:tasks.workunit.client.0.smithi038.stdout:3/189: dread d2/f12 [0,4194304] 0 2024-02-15T18:34:53.714 INFO:tasks.workunit.client.0.smithi038.stdout:5/217: dread d2/d3/ff [0,4194304] 0 2024-02-15T18:34:53.720 INFO:tasks.workunit.client.0.smithi038.stdout:0/309: dwrite d4/d13/d2a/f4e [0,4194304] 0 2024-02-15T18:34:53.721 INFO:tasks.workunit.client.0.smithi038.stdout:3/190: dwrite d2/d1b/d3c/f31 [0,4194304] 0 2024-02-15T18:34:53.723 INFO:tasks.workunit.client.0.smithi038.stdout:0/310: creat d4/d13/f64 x:0 0 0 2024-02-15T18:34:53.725 INFO:tasks.workunit.client.0.smithi038.stdout:0/311: mknod d4/d13/c65 0 2024-02-15T18:34:53.726 INFO:tasks.workunit.client.0.smithi038.stdout:0/312: symlink d4/d13/d2a/d35/d1e/l66 0 2024-02-15T18:34:53.727 INFO:tasks.workunit.client.0.smithi038.stdout:3/191: dread d2/d1b/d3c/f31 [0,4194304] 0 2024-02-15T18:34:53.728 INFO:tasks.workunit.client.0.smithi038.stdout:3/192: dread - d2/d25/f30 zero size 2024-02-15T18:34:53.728 INFO:tasks.workunit.client.0.smithi038.stdout:0/313: read d4/d13/d2a/d35/d1e/d32/d48/f56 [2239004,119710] 0 2024-02-15T18:34:53.730 INFO:tasks.workunit.client.0.smithi038.stdout:5/218: dwrite d2/d3/f2f [0,4194304] 0 2024-02-15T18:34:53.730 INFO:tasks.workunit.client.0.smithi038.stdout:0/314: dread d4/f5 [0,4194304] 0 2024-02-15T18:34:53.730 INFO:tasks.workunit.client.0.smithi038.stdout:0/315: dread - d4/d13/d2a/d35/d1e/d32/d48/f4c zero size 2024-02-15T18:34:53.730 INFO:tasks.workunit.client.0.smithi038.stdout:0/316: chown d4/d13/l38 13031912 1 2024-02-15T18:34:53.731 INFO:tasks.workunit.client.0.smithi038.stdout:7/247: sync 2024-02-15T18:34:53.731 INFO:tasks.workunit.client.0.smithi038.stdout:2/174: sync 2024-02-15T18:34:53.737 INFO:tasks.workunit.client.0.smithi038.stdout:3/193: dread d2/f17 [0,4194304] 0 2024-02-15T18:34:53.737 INFO:tasks.workunit.client.0.smithi038.stdout:3/194: truncate d2/f38 94046 0 2024-02-15T18:34:53.742 INFO:tasks.workunit.client.0.smithi038.stdout:3/195: rename d2/d1b/d3c/l39 to d2/l3e 0 2024-02-15T18:34:53.742 INFO:tasks.workunit.client.0.smithi038.stdout:3/196: chown d2/d1b/d3c/l2b 216255605 1 2024-02-15T18:34:53.746 INFO:tasks.workunit.client.0.smithi038.stdout:3/197: dread d2/d1b/d3c/f31 [0,4194304] 0 2024-02-15T18:34:53.746 INFO:tasks.workunit.client.0.smithi038.stdout:3/198: dread - d2/d25/f30 zero size 2024-02-15T18:34:53.746 INFO:tasks.workunit.client.0.smithi038.stdout:3/199: getdents d2/d1b/d3c/d3a 0 2024-02-15T18:34:53.746 INFO:tasks.workunit.client.0.smithi038.stdout:3/200: chown d2/f12 5323867 1 2024-02-15T18:34:53.746 INFO:tasks.workunit.client.0.smithi038.stdout:3/201: stat d2/c33 0 2024-02-15T18:34:53.751 INFO:tasks.workunit.client.0.smithi038.stdout:5/219: dwrite d2/f3f [0,4194304] 0 2024-02-15T18:34:53.754 INFO:tasks.workunit.client.0.smithi038.stdout:5/220: symlink d2/d3/l41 0 2024-02-15T18:34:53.767 INFO:tasks.workunit.client.0.smithi038.stdout:2/175: dwrite dd/d12/f36 [0,4194304] 0 2024-02-15T18:34:53.767 INFO:tasks.workunit.client.0.smithi038.stdout:7/248: dwrite d0/d1b/f3f [0,4194304] 0 2024-02-15T18:34:53.767 INFO:tasks.workunit.client.0.smithi038.stdout:0/317: dwrite d4/f12 [0,4194304] 0 2024-02-15T18:34:53.769 INFO:tasks.workunit.client.0.smithi038.stdout:0/318: write d4/d13/d19/f43 [3920918,59585] 0 2024-02-15T18:34:53.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:52.316828+0000 mgr.smithi038.khesbl (mgr.14162) 336 : cluster [DBG] pgmap v278: 65 pgs: 65 active+clean; 305 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 5.1 MiB/s rd, 18 MiB/s wr, 761 op/s 2024-02-15T18:34:53.775 INFO:tasks.workunit.client.0.smithi038.stdout:7/249: mknod d0/d1b/c56 0 2024-02-15T18:34:53.776 INFO:tasks.workunit.client.0.smithi038.stdout:0/319: truncate d4/d13/d2a/f4e 4075582 0 2024-02-15T18:34:53.776 INFO:tasks.workunit.client.0.smithi038.stdout:0/320: fdatasync d4/d13/f39 0 2024-02-15T18:34:53.776 INFO:tasks.workunit.client.0.smithi038.stdout:7/250: rename d0/d3/d47 to d0/d3/d57 0 2024-02-15T18:34:53.778 INFO:tasks.workunit.client.0.smithi038.stdout:0/321: symlink d4/d13/d2a/d35/l67 0 2024-02-15T18:34:53.779 INFO:tasks.workunit.client.0.smithi038.stdout:7/251: link d0/db/d19/d2d/l1e d0/d3/l58 0 2024-02-15T18:34:53.780 INFO:tasks.workunit.client.0.smithi038.stdout:0/322: truncate d4/f11 1157057 0 2024-02-15T18:34:53.781 INFO:tasks.workunit.client.0.smithi038.stdout:2/176: dwrite dd/d12/d1d/f1a [0,4194304] 0 2024-02-15T18:34:53.789 INFO:tasks.workunit.client.0.smithi038.stdout:7/252: rename d0/d3 to d0/db/d59 0 2024-02-15T18:34:53.791 INFO:tasks.workunit.client.0.smithi038.stdout:7/253: symlink d0/db/d59/d15/l5a 0 2024-02-15T18:34:53.793 INFO:tasks.workunit.client.0.smithi038.stdout:7/254: creat d0/db/d59/d57/f5b x:0 0 0 2024-02-15T18:34:53.793 INFO:tasks.workunit.client.0.smithi038.stdout:7/255: fsync d0/d1b/f3f 0 2024-02-15T18:34:53.793 INFO:tasks.workunit.client.0.smithi038.stdout:7/256: read - d0/d2e/f3b zero size 2024-02-15T18:34:53.803 INFO:tasks.workunit.client.0.smithi038.stdout:0/323: dwrite d4/d13/d19/f5c [0,4194304] 0 2024-02-15T18:34:53.803 INFO:tasks.workunit.client.0.smithi038.stdout:0/324: write d4/d13/d2a/f4b [532664,55946] 0 2024-02-15T18:34:53.805 INFO:tasks.workunit.client.0.smithi038.stdout:2/177: dwrite fb [0,4194304] 0 2024-02-15T18:34:53.806 INFO:tasks.workunit.client.0.smithi038.stdout:7/257: getdents d0/db/d59/d15 0 2024-02-15T18:34:53.810 INFO:tasks.workunit.client.0.smithi038.stdout:6/203: sync 2024-02-15T18:34:53.810 INFO:tasks.workunit.client.0.smithi038.stdout:1/188: sync 2024-02-15T18:34:53.810 INFO:tasks.workunit.client.0.smithi038.stdout:8/354: sync 2024-02-15T18:34:53.810 INFO:tasks.workunit.client.0.smithi038.stdout:8/355: chown d1/l19 20 1 2024-02-15T18:34:53.812 INFO:tasks.workunit.client.0.smithi038.stdout:2/178: dread dd/d12/f39 [0,4194304] 0 2024-02-15T18:34:53.813 INFO:tasks.workunit.client.0.smithi038.stdout:6/204: mkdir d2/d4/d2c/d39 0 2024-02-15T18:34:53.815 INFO:tasks.workunit.client.0.smithi038.stdout:1/189: mknod d2/d28/d3c/c45 0 2024-02-15T18:34:53.816 INFO:tasks.workunit.client.0.smithi038.stdout:2/179: mkdir dd/d12/d3a 0 2024-02-15T18:34:53.818 INFO:tasks.workunit.client.0.smithi038.stdout:6/205: link d2/d10/f30 d2/d4/d28/f3a 0 2024-02-15T18:34:53.820 INFO:tasks.workunit.client.0.smithi038.stdout:1/190: symlink d2/d28/l46 0 2024-02-15T18:34:53.821 INFO:tasks.workunit.client.0.smithi038.stdout:2/180: getdents dd/d12/d18/d1f 0 2024-02-15T18:34:53.821 INFO:tasks.workunit.client.0.smithi038.stdout:2/181: getdents dd/d12/d18/d1f 0 2024-02-15T18:34:53.821 INFO:tasks.workunit.client.0.smithi038.stdout:6/206: symlink d2/d4/l3b 0 2024-02-15T18:34:53.822 INFO:tasks.workunit.client.0.smithi038.stdout:6/207: read d2/d4/f1e [1465677,130093] 0 2024-02-15T18:34:53.822 INFO:tasks.workunit.client.0.smithi038.stdout:1/191: mkdir d2/d10/d19/d47 0 2024-02-15T18:34:53.822 INFO:tasks.workunit.client.0.smithi038.stdout:2/182: mkdir dd/d12/d1d/d3b 0 2024-02-15T18:34:53.823 INFO:tasks.workunit.client.0.smithi038.stdout:6/208: creat d2/d4/d2c/f3c x:0 0 0 2024-02-15T18:34:53.824 INFO:tasks.workunit.client.0.smithi038.stdout:1/192: creat d2/d39/f48 x:0 0 0 2024-02-15T18:34:53.825 INFO:tasks.workunit.client.0.smithi038.stdout:1/193: mknod d2/d10/d19/d2f/d3e/c49 0 2024-02-15T18:34:53.826 INFO:tasks.workunit.client.0.smithi038.stdout:1/194: write d2/d28/f43 [65098,21439] 0 2024-02-15T18:34:53.826 INFO:tasks.workunit.client.0.smithi038.stdout:1/195: write d2/d28/f34 [526480,118850] 0 2024-02-15T18:34:53.826 INFO:tasks.workunit.client.0.smithi038.stdout:1/196: rename d2/d10/d19/d22 to d2/d10/d19/d22/d4a 22 2024-02-15T18:34:53.829 INFO:tasks.workunit.client.0.smithi038.stdout:2/183: dread dd/d12/f36 [0,4194304] 0 2024-02-15T18:34:53.830 INFO:tasks.workunit.client.0.smithi038.stdout:2/184: fsync dd/d12/d1d/f1a 0 2024-02-15T18:34:53.830 INFO:tasks.workunit.client.0.smithi038.stdout:2/185: mkdir dd/d12/d1d/d3b/d3c 0 2024-02-15T18:34:53.831 INFO:tasks.workunit.client.0.smithi038.stdout:2/186: creat dd/d12/f3d x:0 0 0 2024-02-15T18:34:53.831 INFO:tasks.workunit.client.0.smithi038.stdout:2/187: write dd/d12/d1d/f1a [5114532,75763] 0 2024-02-15T18:34:53.832 INFO:tasks.workunit.client.0.smithi038.stdout:2/188: rename dd/l25 to dd/d12/d3a/l3e 0 2024-02-15T18:34:53.833 INFO:tasks.workunit.client.0.smithi038.stdout:2/189: readlink l9 0 2024-02-15T18:34:53.834 INFO:tasks.workunit.client.0.smithi038.stdout:2/190: symlink dd/d12/d18/d2b/l3f 0 2024-02-15T18:34:53.843 INFO:tasks.workunit.client.0.smithi038.stdout:0/325: dwrite d4/ff [0,4194304] 0 2024-02-15T18:34:53.848 INFO:tasks.workunit.client.0.smithi038.stdout:2/191: read fb [972994,9336] 0 2024-02-15T18:34:53.848 INFO:tasks.workunit.client.0.smithi038.stdout:7/258: dwrite d0/fa [0,4194304] 0 2024-02-15T18:34:53.849 INFO:tasks.workunit.client.0.smithi038.stdout:8/356: dwrite d1/d4/d5/d31/d65/f76 [0,4194304] 0 2024-02-15T18:34:53.849 INFO:tasks.workunit.client.0.smithi038.stdout:6/209: dwrite d2/d4/d2c/f3c [0,4194304] 0 2024-02-15T18:34:53.850 INFO:tasks.workunit.client.0.smithi038.stdout:0/326: fdatasync d4/d13/d2a/f4b 0 2024-02-15T18:34:53.850 INFO:tasks.workunit.client.0.smithi038.stdout:2/192: mkdir dd/d12/d18/d1f/d40 0 2024-02-15T18:34:53.850 INFO:tasks.workunit.client.0.smithi038.stdout:2/193: fdatasync fb 0 2024-02-15T18:34:53.850 INFO:tasks.workunit.client.0.smithi038.stdout:1/197: write d2/d10/f31 [3526844,51851] 0 2024-02-15T18:34:53.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:52.316828+0000 mgr.smithi038.khesbl (mgr.14162) 336 : cluster [DBG] pgmap v278: 65 pgs: 65 active+clean; 305 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 5.1 MiB/s rd, 18 MiB/s wr, 761 op/s 2024-02-15T18:34:53.859 INFO:tasks.workunit.client.0.smithi038.stdout:7/259: mknod d0/d1b/c5c 0 2024-02-15T18:34:53.859 INFO:tasks.workunit.client.0.smithi038.stdout:7/260: chown d0/db/d19/f4e 521 1 2024-02-15T18:34:53.865 INFO:tasks.workunit.client.0.smithi038.stdout:8/357: symlink d1/d7/d15/d4b/l82 0 2024-02-15T18:34:53.865 INFO:tasks.workunit.client.0.smithi038.stdout:6/210: symlink d2/d10/l3d 0 2024-02-15T18:34:53.866 INFO:tasks.workunit.client.0.smithi038.stdout:2/194: truncate dd/d12/d13/f28 3542424 0 2024-02-15T18:34:53.868 INFO:tasks.workunit.client.0.smithi038.stdout:0/327: dwrite d4/f11 [0,4194304] 0 2024-02-15T18:34:53.868 INFO:tasks.workunit.client.0.smithi038.stdout:8/358: mkdir d1/d4/d10/d43/d48/d52/d83 0 2024-02-15T18:34:53.870 INFO:tasks.workunit.client.0.smithi038.stdout:1/198: dwrite d2/d28/f34 [0,4194304] 0 2024-02-15T18:34:53.870 INFO:tasks.workunit.client.0.smithi038.stdout:1/199: fdatasync d2/d28/f43 0 2024-02-15T18:34:53.870 INFO:tasks.workunit.client.0.smithi038.stdout:2/195: rmdir dd/d12/d18/d1f/d40 0 2024-02-15T18:34:53.874 INFO:tasks.workunit.client.0.smithi038.stdout:0/328: rmdir d4 39 2024-02-15T18:34:53.880 INFO:tasks.workunit.client.0.smithi038.stdout:1/200: write d2/f1e [1071912,121502] 0 2024-02-15T18:34:53.882 INFO:tasks.workunit.client.0.smithi038.stdout:2/196: creat dd/f41 x:0 0 0 2024-02-15T18:34:53.882 INFO:tasks.workunit.client.0.smithi038.stdout:2/197: chown c6 54859 1 2024-02-15T18:34:53.885 INFO:tasks.workunit.client.0.smithi038.stdout:1/201: symlink d2/d28/l4b 0 2024-02-15T18:34:53.888 INFO:tasks.workunit.client.0.smithi038.stdout:2/198: rename ca to dd/d12/d18/d2b/c42 0 2024-02-15T18:34:53.890 INFO:tasks.workunit.client.0.smithi038.stdout:1/202: rename d2/f26 to d2/d10/f4c 0 2024-02-15T18:34:53.892 INFO:tasks.workunit.client.0.smithi038.stdout:4/200: sync 2024-02-15T18:34:53.892 INFO:tasks.workunit.client.0.smithi038.stdout:9/202: sync 2024-02-15T18:34:53.902 INFO:tasks.workunit.client.0.smithi038.stdout:8/359: dwrite d1/d4/d5/d31/f39 [8388608,4194304] 0 2024-02-15T18:34:53.902 INFO:tasks.workunit.client.0.smithi038.stdout:0/329: dwrite d4/d13/f54 [0,4194304] 0 2024-02-15T18:34:53.903 INFO:tasks.workunit.client.0.smithi038.stdout:8/360: creat d1/d7/d26/f84 x:0 0 0 2024-02-15T18:34:53.903 INFO:tasks.workunit.client.0.smithi038.stdout:4/201: write d11/d1e/f29 [1756554,105567] 0 2024-02-15T18:34:53.903 INFO:tasks.workunit.client.0.smithi038.stdout:6/211: dwrite d2/f3 [0,4194304] 0 2024-02-15T18:34:53.904 INFO:tasks.workunit.client.0.smithi038.stdout:9/203: dread d1/dc/d2b/f3a [0,4194304] 0 2024-02-15T18:34:53.904 INFO:tasks.workunit.client.0.smithi038.stdout:8/361: write d1/d3a/d58/f60 [893014,84179] 0 2024-02-15T18:34:53.904 INFO:tasks.workunit.client.0.smithi038.stdout:8/362: chown d1/d7/d15/d4b/l32 2480501 1 2024-02-15T18:34:53.906 INFO:tasks.workunit.client.0.smithi038.stdout:1/203: rename d2/d10/f4c to d2/d10/d19/d22/d36/f4d 0 2024-02-15T18:34:53.906 INFO:tasks.workunit.client.0.smithi038.stdout:1/204: write f1 [542872,27005] 0 2024-02-15T18:34:53.907 INFO:tasks.workunit.client.0.smithi038.stdout:0/330: getdents d4/d13/d19 0 2024-02-15T18:34:53.911 INFO:tasks.workunit.client.0.smithi038.stdout:6/212: symlink d2/d4/d2c/d39/l3e 0 2024-02-15T18:34:53.911 INFO:tasks.workunit.client.0.smithi038.stdout:6/213: write d2/d4/f1f [1014625,82736] 0 2024-02-15T18:34:53.911 INFO:tasks.workunit.client.0.smithi038.stdout:6/214: write d2/d4/f1e [2018632,96454] 0 2024-02-15T18:34:53.912 INFO:tasks.workunit.client.0.smithi038.stdout:9/204: link d1/dc/d2b/f3a d1/dc/d2b/f4a 0 2024-02-15T18:34:53.913 INFO:tasks.workunit.client.0.smithi038.stdout:9/205: chown d1/da/d10/d19/f42 75192 1 2024-02-15T18:34:53.914 INFO:tasks.workunit.client.0.smithi038.stdout:4/202: rename d11/c15 to d11/d1e/d20/d24/d40/c4b 0 2024-02-15T18:34:53.915 INFO:tasks.workunit.client.0.smithi038.stdout:0/331: mknod d4/c68 0 2024-02-15T18:34:53.915 INFO:tasks.workunit.client.0.smithi038.stdout:0/332: fsync d4/d13/d19/f43 0 2024-02-15T18:34:53.916 INFO:tasks.workunit.client.0.smithi038.stdout:6/215: creat d2/d4/d2c/f3f x:0 0 0 2024-02-15T18:34:53.916 INFO:tasks.workunit.client.0.smithi038.stdout:6/216: write d2/d4/d2c/f37 [149526,67496] 0 2024-02-15T18:34:53.918 INFO:tasks.workunit.client.0.smithi038.stdout:0/333: rename d4/d13/d19/c25 to d4/d13/d19/c69 0 2024-02-15T18:34:53.918 INFO:tasks.workunit.client.0.smithi038.stdout:4/203: symlink d11/d1e/d43/l4c 0 2024-02-15T18:34:53.918 INFO:tasks.workunit.client.0.smithi038.stdout:4/204: fdatasync d11/d1e/f37 0 2024-02-15T18:34:53.922 INFO:tasks.workunit.client.0.smithi038.stdout:6/217: link d2/d10/c25 d2/d4/d2c/d39/c40 0 2024-02-15T18:34:53.922 INFO:tasks.workunit.client.0.smithi038.stdout:0/334: mkdir d4/d13/d2a/d35/d6a 0 2024-02-15T18:34:53.923 INFO:tasks.workunit.client.0.smithi038.stdout:6/218: write d2/d4/d28/f3a [86574,49479] 0 2024-02-15T18:34:53.923 INFO:tasks.workunit.client.0.smithi038.stdout:0/335: mknod d4/d13/d2a/d35/c6b 0 2024-02-15T18:34:53.924 INFO:tasks.workunit.client.0.smithi038.stdout:0/336: creat d4/d13/d2a/d35/d1e/d32/f6c x:0 0 0 2024-02-15T18:34:53.924 INFO:tasks.workunit.client.0.smithi038.stdout:0/337: stat d4/f5 0 2024-02-15T18:34:53.925 INFO:tasks.workunit.client.0.smithi038.stdout:0/338: symlink d4/d13/d2a/d35/d1e/d32/l6d 0 2024-02-15T18:34:53.926 INFO:tasks.workunit.client.0.smithi038.stdout:0/339: getdents d4/d13/d2a/d35/d1e/d32/d48 0 2024-02-15T18:34:53.927 INFO:tasks.workunit.client.0.smithi038.stdout:0/340: mknod d4/c6e 0 2024-02-15T18:34:53.927 INFO:tasks.workunit.client.0.smithi038.stdout:0/341: fdatasync d4/d13/f54 0 2024-02-15T18:34:53.931 INFO:tasks.workunit.client.0.smithi038.stdout:5/221: sync 2024-02-15T18:34:53.931 INFO:tasks.workunit.client.0.smithi038.stdout:3/202: sync 2024-02-15T18:34:53.931 INFO:tasks.workunit.client.0.smithi038.stdout:5/222: readlink d2/d3/l9 0 2024-02-15T18:34:53.933 INFO:tasks.workunit.client.0.smithi038.stdout:3/203: link d2/d1b/l21 d2/d1e/l3f 0 2024-02-15T18:34:53.934 INFO:tasks.workunit.client.0.smithi038.stdout:5/223: getdents d2/d3/d12 0 2024-02-15T18:34:53.935 INFO:tasks.workunit.client.0.smithi038.stdout:5/224: chown d2/d3/f1d 19580 1 2024-02-15T18:34:53.936 INFO:tasks.workunit.client.0.smithi038.stdout:1/205: dread d2/f11 [0,4194304] 0 2024-02-15T18:34:53.937 INFO:tasks.workunit.client.0.smithi038.stdout:8/363: dwrite d1/d7/f1d [0,4194304] 0 2024-02-15T18:34:53.938 INFO:tasks.workunit.client.0.smithi038.stdout:8/364: write d1/d4/d5/d31/f33 [1394956,30453] 0 2024-02-15T18:34:53.938 INFO:tasks.workunit.client.0.smithi038.stdout:8/365: stat d1/d3a 0 2024-02-15T18:34:53.938 INFO:tasks.workunit.client.0.smithi038.stdout:5/225: rmdir d2/d3/d12/d35 0 2024-02-15T18:34:53.940 INFO:tasks.workunit.client.0.smithi038.stdout:1/206: rmdir d2/d10/d19/d22 39 2024-02-15T18:34:53.942 INFO:tasks.workunit.client.0.smithi038.stdout:0/342: dread d4/f12 [0,4194304] 0 2024-02-15T18:34:53.942 INFO:tasks.workunit.client.0.smithi038.stdout:6/219: dwrite d2/f9 [0,4194304] 0 2024-02-15T18:34:53.943 INFO:tasks.workunit.client.0.smithi038.stdout:4/205: dwrite d11/f1d [0,4194304] 0 2024-02-15T18:34:53.944 INFO:tasks.workunit.client.0.smithi038.stdout:4/206: write d11/d1e/d20/d2f/f3d [50020,63401] 0 2024-02-15T18:34:53.946 INFO:tasks.workunit.client.0.smithi038.stdout:8/366: creat d1/d7/d22/f85 x:0 0 0 2024-02-15T18:34:53.946 INFO:tasks.workunit.client.0.smithi038.stdout:8/367: chown d1/f1b 633385 1 2024-02-15T18:34:53.950 INFO:tasks.workunit.client.0.smithi038.stdout:3/204: dwrite d2/fa [0,4194304] 0 2024-02-15T18:34:53.950 INFO:tasks.workunit.client.0.smithi038.stdout:3/205: truncate d2/d1b/f36 725385 0 2024-02-15T18:34:53.950 INFO:tasks.workunit.client.0.smithi038.stdout:3/206: write d2/f2f [495676,34699] 0 2024-02-15T18:34:53.951 INFO:tasks.workunit.client.0.smithi038.stdout:5/226: mknod d2/d3/c42 0 2024-02-15T18:34:53.952 INFO:tasks.workunit.client.0.smithi038.stdout:1/207: link d2/d28/d3c/c45 d2/d39/c4e 0 2024-02-15T18:34:53.953 INFO:tasks.workunit.client.0.smithi038.stdout:0/343: creat d4/d13/d2a/d35/d1e/f6f x:0 0 0 2024-02-15T18:34:53.954 INFO:tasks.workunit.client.0.smithi038.stdout:6/220: mknod d2/d4/d19/c41 0 2024-02-15T18:34:53.954 INFO:tasks.workunit.client.0.smithi038.stdout:6/221: chown d2/d10 18284844 1 2024-02-15T18:34:53.954 INFO:tasks.workunit.client.0.smithi038.stdout:1/208: read d2/d10/d19/d22/d36/f4d [3043390,71248] 0 2024-02-15T18:34:53.955 INFO:tasks.workunit.client.0.smithi038.stdout:4/207: mkdir d11/d1e/d20/d2b/d4d 0 2024-02-15T18:34:53.959 INFO:tasks.workunit.client.0.smithi038.stdout:3/207: unlink d2/f19 0 2024-02-15T18:34:53.960 INFO:tasks.workunit.client.0.smithi038.stdout:0/344: truncate d4/ff 2298636 0 2024-02-15T18:34:53.960 INFO:tasks.workunit.client.0.smithi038.stdout:6/222: stat d2/d10/c25 0 2024-02-15T18:34:53.960 INFO:tasks.workunit.client.0.smithi038.stdout:1/209: stat d2/d10/d19/d22/c29 0 2024-02-15T18:34:53.961 INFO:tasks.workunit.client.0.smithi038.stdout:3/208: symlink d2/d1b/d3c/l40 0 2024-02-15T18:34:53.963 INFO:tasks.workunit.client.0.smithi038.stdout:0/345: rename d4/d13/d2a/d35/d1e/d32/d3b/l3f to d4/d13/d2a/l70 0 2024-02-15T18:34:53.964 INFO:tasks.workunit.client.0.smithi038.stdout:5/227: write d2/d3/d12/fa [1290801,4832] 0 2024-02-15T18:34:53.964 INFO:tasks.workunit.client.0.smithi038.stdout:5/228: fsync d2/f3f 0 2024-02-15T18:34:53.967 INFO:tasks.workunit.client.0.smithi038.stdout:1/210: mkdir d2/d10/d19/d22/d36/d4f 0 2024-02-15T18:34:53.967 INFO:tasks.workunit.client.0.smithi038.stdout:0/346: dread d4/f5 [0,4194304] 0 2024-02-15T18:34:53.969 INFO:tasks.workunit.client.0.smithi038.stdout:3/209: write d2/fa [2540616,118797] 0 2024-02-15T18:34:53.971 INFO:tasks.workunit.client.0.smithi038.stdout:8/368: dwrite d1/d7/d15/d4b/f3f [0,4194304] 0 2024-02-15T18:34:53.971 INFO:tasks.workunit.client.0.smithi038.stdout:8/369: chown d1/d4/d5 570 1 2024-02-15T18:34:53.971 INFO:tasks.workunit.client.0.smithi038.stdout:8/370: truncate d1/d7/d22/f7c 129949 0 2024-02-15T18:34:53.971 INFO:tasks.workunit.client.0.smithi038.stdout:0/347: write d4/d13/d2a/f44 [283897,60504] 0 2024-02-15T18:34:53.971 INFO:tasks.workunit.client.0.smithi038.stdout:0/348: chown d4/d13/d2a/c2b 4254594 1 2024-02-15T18:34:53.971 INFO:tasks.workunit.client.0.smithi038.stdout:0/349: truncate d4/d13/f64 546506 0 2024-02-15T18:34:53.971 INFO:tasks.workunit.client.0.smithi038.stdout:0/350: rename d4 to d4/d13/d71 22 2024-02-15T18:34:53.971 INFO:tasks.workunit.client.0.smithi038.stdout:4/208: dwrite d11/d1e/d3e/f47 [0,4194304] 0 2024-02-15T18:34:53.971 INFO:tasks.workunit.client.0.smithi038.stdout:4/209: rename d11/d1e/d20 to d11/d1e/d20/d24/d40/d4e 22 2024-02-15T18:34:53.971 INFO:tasks.workunit.client.0.smithi038.stdout:0/351: write d4/d13/d2a/d35/f49 [495153,12285] 0 2024-02-15T18:34:53.972 INFO:tasks.workunit.client.0.smithi038.stdout:8/371: truncate d1/d4/d5/d31/f39 12034580 0 2024-02-15T18:34:53.973 INFO:tasks.workunit.client.0.smithi038.stdout:3/210: write d2/f15 [3449239,28730] 0 2024-02-15T18:34:53.975 INFO:tasks.workunit.client.0.smithi038.stdout:4/210: getdents d11/d1e/d43 0 2024-02-15T18:34:53.976 INFO:tasks.workunit.client.0.smithi038.stdout:4/211: chown d11/d1e/d43/d4a 3 1 2024-02-15T18:34:53.976 INFO:tasks.workunit.client.0.smithi038.stdout:0/352: mknod d4/d13/d2a/d35/d6a/c72 0 2024-02-15T18:34:53.983 INFO:tasks.workunit.client.0.smithi038.stdout:8/372: creat d1/d7/d15/d6a/f86 x:0 0 0 2024-02-15T18:34:53.983 INFO:tasks.workunit.client.0.smithi038.stdout:8/373: write d1/d7/d22/f85 [259126,72292] 0 2024-02-15T18:34:53.984 INFO:tasks.workunit.client.0.smithi038.stdout:8/374: read d1/d3a/f6c [160125,89561] 0 2024-02-15T18:34:53.984 INFO:tasks.workunit.client.0.smithi038.stdout:8/375: chown d1/d4/d5/d6/f38 498 1 2024-02-15T18:34:53.988 INFO:tasks.workunit.client.0.smithi038.stdout:3/211: mkdir d2/d1b/d3c/d41 0 2024-02-15T18:34:53.988 INFO:tasks.workunit.client.0.smithi038.stdout:4/212: mknod d11/d1e/d20/d31/c4f 0 2024-02-15T18:34:53.989 INFO:tasks.workunit.client.0.smithi038.stdout:0/353: creat d4/d13/d2a/d35/d6a/f73 x:0 0 0 2024-02-15T18:34:53.990 INFO:tasks.workunit.client.0.smithi038.stdout:8/376: truncate d1/d4/d10/f35 3500770 0 2024-02-15T18:34:53.991 INFO:tasks.workunit.client.0.smithi038.stdout:3/212: mkdir d2/d1b/d3c/d42 0 2024-02-15T18:34:53.993 INFO:tasks.workunit.client.0.smithi038.stdout:4/213: rename d11/d1e/d3e/l42 to d11/d1e/d20/d31/l50 0 2024-02-15T18:34:53.993 INFO:tasks.workunit.client.0.smithi038.stdout:4/214: chown ld 322849571 1 2024-02-15T18:34:53.994 INFO:tasks.workunit.client.0.smithi038.stdout:0/354: creat d4/d13/d2a/f74 x:0 0 0 2024-02-15T18:34:53.995 INFO:tasks.workunit.client.0.smithi038.stdout:8/377: symlink d1/d4/d5/d31/l87 0 2024-02-15T18:34:53.995 INFO:tasks.workunit.client.0.smithi038.stdout:4/215: write d11/d1e/f29 [255222,55113] 0 2024-02-15T18:34:53.995 INFO:tasks.workunit.client.0.smithi038.stdout:4/216: chown d11/d1e/d20/d24/f46 58949335 1 2024-02-15T18:34:53.995 INFO:tasks.workunit.client.0.smithi038.stdout:0/355: mknod d4/d13/d53/c75 0 2024-02-15T18:34:53.996 INFO:tasks.workunit.client.0.smithi038.stdout:0/356: rmdir d4/d13/d2a/d35/d1e/d32/d3b 39 2024-02-15T18:34:53.998 INFO:tasks.workunit.client.0.smithi038.stdout:6/223: dwrite d2/d10/f1d [0,4194304] 0 2024-02-15T18:34:53.998 INFO:tasks.workunit.client.0.smithi038.stdout:5/229: dwrite d2/f2b [0,4194304] 0 2024-02-15T18:34:53.999 INFO:tasks.workunit.client.0.smithi038.stdout:1/211: dwrite d2/d10/f31 [0,4194304] 0 2024-02-15T18:34:54.000 INFO:tasks.workunit.client.0.smithi038.stdout:0/357: rename d4/d13/d2a/f74 to d4/d13/d2a/d35/d1e/d32/f76 0 2024-02-15T18:34:54.000 INFO:tasks.workunit.client.0.smithi038.stdout:4/217: read d11/f1f [729817,51188] 0 2024-02-15T18:34:54.000 INFO:tasks.workunit.client.0.smithi038.stdout:4/218: getdents d11/d1e/d20/d31/d48 0 2024-02-15T18:34:54.002 INFO:tasks.workunit.client.0.smithi038.stdout:4/219: write d11/d1e/f28 [5184713,64864] 0 2024-02-15T18:34:54.002 INFO:tasks.workunit.client.0.smithi038.stdout:4/220: readlink d11/d1e/d43/l4c 0 2024-02-15T18:34:54.003 INFO:tasks.workunit.client.0.smithi038.stdout:6/224: dread d2/d4/ff [0,4194304] 0 2024-02-15T18:34:54.005 INFO:tasks.workunit.client.0.smithi038.stdout:5/230: creat d2/d20/f43 x:0 0 0 2024-02-15T18:34:54.008 INFO:tasks.workunit.client.0.smithi038.stdout:1/212: dread d2/d10/d19/d22/d36/f4d [0,4194304] 0 2024-02-15T18:34:54.008 INFO:tasks.workunit.client.0.smithi038.stdout:5/231: write d2/d3/f7 [5967085,79760] 0 2024-02-15T18:34:54.009 INFO:tasks.workunit.client.0.smithi038.stdout:6/225: creat d2/d4/f42 x:0 0 0 2024-02-15T18:34:54.009 INFO:tasks.workunit.client.0.smithi038.stdout:6/226: truncate d2/d4/f27 5328776 0 2024-02-15T18:34:54.009 INFO:tasks.workunit.client.0.smithi038.stdout:1/213: creat d2/d28/f50 x:0 0 0 2024-02-15T18:34:54.011 INFO:tasks.workunit.client.0.smithi038.stdout:1/214: creat d2/d10/d19/d22/d36/d4f/f51 x:0 0 0 2024-02-15T18:34:54.014 INFO:tasks.workunit.client.0.smithi038.stdout:1/215: rename d2/d10/d19/d22/c23 to d2/c52 0 2024-02-15T18:34:54.014 INFO:tasks.workunit.client.0.smithi038.stdout:1/216: fdatasync d2/d28/f50 0 2024-02-15T18:34:54.017 INFO:tasks.workunit.client.0.smithi038.stdout:8/378: dwrite d1/d4/d10/d43/d48/d52/f53 [0,4194304] 0 2024-02-15T18:34:54.017 INFO:tasks.workunit.client.0.smithi038.stdout:8/379: truncate d1/d7/d15/d6a/f86 794636 0 2024-02-15T18:34:54.017 INFO:tasks.workunit.client.0.smithi038.stdout:8/380: chown d1/d4/d10/d43/l49 10835 1 2024-02-15T18:34:54.021 INFO:tasks.workunit.client.0.smithi038.stdout:5/232: dread d2/d3/f2f [0,4194304] 0 2024-02-15T18:34:54.021 INFO:tasks.workunit.client.0.smithi038.stdout:5/233: stat d2/d30/c31 0 2024-02-15T18:34:54.024 INFO:tasks.workunit.client.0.smithi038.stdout:8/381: mkdir d1/d4/d10/d43/d48/d52/d88 0 2024-02-15T18:34:54.024 INFO:tasks.workunit.client.0.smithi038.stdout:5/234: rmdir d2/d20/d33 39 2024-02-15T18:34:54.026 INFO:tasks.workunit.client.0.smithi038.stdout:5/235: rename d2/d3/c8 to d2/c44 0 2024-02-15T18:34:54.027 INFO:tasks.workunit.client.0.smithi038.stdout:5/236: mknod d2/d3/c45 0 2024-02-15T18:34:54.028 INFO:tasks.workunit.client.0.smithi038.stdout:5/237: creat d2/d30/f46 x:0 0 0 2024-02-15T18:34:54.028 INFO:tasks.workunit.client.0.smithi038.stdout:5/238: truncate d2/d20/d33/f36 947242 0 2024-02-15T18:34:54.028 INFO:tasks.workunit.client.0.smithi038.stdout:5/239: readlink d2/d20/l24 0 2024-02-15T18:34:54.029 INFO:tasks.workunit.client.0.smithi038.stdout:4/221: dwrite d11/f13 [0,4194304] 0 2024-02-15T18:34:54.030 INFO:tasks.workunit.client.0.smithi038.stdout:6/227: dwrite d2/d4/f42 [0,4194304] 0 2024-02-15T18:34:54.030 INFO:tasks.workunit.client.0.smithi038.stdout:4/222: mknod d11/d1e/d20/d2b/c51 0 2024-02-15T18:34:54.031 INFO:tasks.workunit.client.0.smithi038.stdout:6/228: symlink d2/l43 0 2024-02-15T18:34:54.032 INFO:tasks.workunit.client.0.smithi038.stdout:4/223: getdents d11 0 2024-02-15T18:34:54.033 INFO:tasks.workunit.client.0.smithi038.stdout:4/224: stat d11/d1e/d20/d24/d40/c3c 0 2024-02-15T18:34:54.038 INFO:tasks.workunit.client.0.smithi038.stdout:1/217: dwrite d2/f1e [0,4194304] 0 2024-02-15T18:34:54.038 INFO:tasks.workunit.client.0.smithi038.stdout:6/229: read d2/f2a [2835747,3285] 0 2024-02-15T18:34:54.038 INFO:tasks.workunit.client.0.smithi038.stdout:6/230: write d2/d4/d2c/f3f [575189,39688] 0 2024-02-15T18:34:54.043 INFO:tasks.workunit.client.0.smithi038.stdout:6/231: rmdir d2/d4/d19/d38 0 2024-02-15T18:34:54.043 INFO:tasks.workunit.client.0.smithi038.stdout:6/232: read - d2/d4/d28/f36 zero size 2024-02-15T18:34:54.043 INFO:tasks.workunit.client.0.smithi038.stdout:8/382: dread d1/d4/d5/f37 [0,4194304] 0 2024-02-15T18:34:54.043 INFO:tasks.workunit.client.0.smithi038.stdout:8/383: fdatasync d1/d3a/f6c 0 2024-02-15T18:34:54.043 INFO:tasks.workunit.client.0.smithi038.stdout:8/384: chown d1/d7/d15 56209991 1 2024-02-15T18:34:54.046 INFO:tasks.workunit.client.0.smithi038.stdout:4/225: dread d11/d1e/d20/d24/f2d [4194304,4194304] 0 2024-02-15T18:34:54.046 INFO:tasks.workunit.client.0.smithi038.stdout:4/226: chown d11/d1e/d3e 9 1 2024-02-15T18:34:54.046 INFO:tasks.workunit.client.0.smithi038.stdout:5/240: dread d2/f3f [0,4194304] 0 2024-02-15T18:34:54.047 INFO:tasks.workunit.client.0.smithi038.stdout:7/261: sync 2024-02-15T18:34:54.047 INFO:tasks.workunit.client.0.smithi038.stdout:2/199: sync 2024-02-15T18:34:54.047 INFO:tasks.workunit.client.0.smithi038.stdout:5/241: chown d2/d20/d33 117100533 1 2024-02-15T18:34:54.047 INFO:tasks.workunit.client.0.smithi038.stdout:7/262: read - d0/d1b/d30/f43 zero size 2024-02-15T18:34:54.047 INFO:tasks.workunit.client.0.smithi038.stdout:7/263: chown d0/d2e/c4b 1728 1 2024-02-15T18:34:54.047 INFO:tasks.workunit.client.0.smithi038.stdout:2/200: truncate dd/d12/d18/d1f/f27 591250 0 2024-02-15T18:34:54.047 INFO:tasks.workunit.client.0.smithi038.stdout:8/385: mknod d1/d4/d10/d43/d48/d52/d88/c89 0 2024-02-15T18:34:54.050 INFO:tasks.workunit.client.0.smithi038.stdout:1/218: dread d2/f1e [0,4194304] 0 2024-02-15T18:34:54.056 INFO:tasks.workunit.client.0.smithi038.stdout:1/219: dread d2/d28/f34 [0,4194304] 0 2024-02-15T18:34:54.056 INFO:tasks.workunit.client.0.smithi038.stdout:4/227: mknod d11/d1e/d43/d4a/c52 0 2024-02-15T18:34:54.057 INFO:tasks.workunit.client.0.smithi038.stdout:5/242: getdents d2/d3/d12 0 2024-02-15T18:34:54.057 INFO:tasks.workunit.client.0.smithi038.stdout:5/243: chown d2/f2b 109852 1 2024-02-15T18:34:54.057 INFO:tasks.workunit.client.0.smithi038.stdout:5/244: dread - d2/d20/f37 zero size 2024-02-15T18:34:54.058 INFO:tasks.workunit.client.0.smithi038.stdout:5/245: read d2/d3/f1d [216260,105537] 0 2024-02-15T18:34:54.058 INFO:tasks.workunit.client.0.smithi038.stdout:7/264: mkdir d0/d1b/d30/d5d 0 2024-02-15T18:34:54.058 INFO:tasks.workunit.client.0.smithi038.stdout:7/265: stat d0/db/d59/c11 0 2024-02-15T18:34:54.058 INFO:tasks.workunit.client.0.smithi038.stdout:7/266: chown d0/d1b/d30/d5d 10149 1 2024-02-15T18:34:54.058 INFO:tasks.workunit.client.0.smithi038.stdout:2/201: symlink dd/d12/d1d/d3b/l43 0 2024-02-15T18:34:54.059 INFO:tasks.workunit.client.0.smithi038.stdout:2/202: chown dd/d12/d13/d33/l38 126036440 1 2024-02-15T18:34:54.059 INFO:tasks.workunit.client.0.smithi038.stdout:5/246: write d2/d3/f1b [3171360,77487] 0 2024-02-15T18:34:54.059 INFO:tasks.workunit.client.0.smithi038.stdout:8/386: getdents d1/d4/d5 0 2024-02-15T18:34:54.062 INFO:tasks.workunit.client.0.smithi038.stdout:1/220: mkdir d2/d10/d32/d53 0 2024-02-15T18:34:54.062 INFO:tasks.workunit.client.0.smithi038.stdout:4/228: mknod d11/d1e/d20/c53 0 2024-02-15T18:34:54.063 INFO:tasks.workunit.client.0.smithi038.stdout:2/203: mknod dd/d12/d1d/c44 0 2024-02-15T18:34:54.064 INFO:tasks.workunit.client.0.smithi038.stdout:5/247: symlink d2/d20/l47 0 2024-02-15T18:34:54.064 INFO:tasks.workunit.client.0.smithi038.stdout:5/248: write d2/d20/f43 [727205,19380] 0 2024-02-15T18:34:54.064 INFO:tasks.workunit.client.0.smithi038.stdout:5/249: write d2/f2b [4514418,79375] 0 2024-02-15T18:34:54.066 INFO:tasks.workunit.client.0.smithi038.stdout:1/221: dread d2/f2d [0,4194304] 0 2024-02-15T18:34:54.068 INFO:tasks.workunit.client.0.smithi038.stdout:8/387: symlink d1/d7/l8a 0 2024-02-15T18:34:54.068 INFO:tasks.workunit.client.0.smithi038.stdout:4/229: creat d11/d1e/d3e/f54 x:0 0 0 2024-02-15T18:34:54.071 INFO:tasks.workunit.client.0.smithi038.stdout:2/204: unlink dd/d12/d1d/f1a 0 2024-02-15T18:34:54.071 INFO:tasks.workunit.client.0.smithi038.stdout:2/205: fsync dd/d12/f26 0 2024-02-15T18:34:54.072 INFO:tasks.workunit.client.0.smithi038.stdout:5/250: mkdir d2/d48 0 2024-02-15T18:34:54.072 INFO:tasks.workunit.client.0.smithi038.stdout:7/267: dwrite d0/db/d19/d2d/f36 [0,4194304] 0 2024-02-15T18:34:54.072 INFO:tasks.workunit.client.0.smithi038.stdout:7/268: write d0/d2e/f3b [416507,49645] 0 2024-02-15T18:34:54.073 INFO:tasks.workunit.client.0.smithi038.stdout:1/222: rename d2/d10/d19/d47 to d2/d10/d19/d22/d54 0 2024-02-15T18:34:54.073 INFO:tasks.workunit.client.0.smithi038.stdout:7/269: dread d0/d2e/f37 [0,4194304] 0 2024-02-15T18:34:54.073 INFO:tasks.workunit.client.0.smithi038.stdout:1/223: read d2/f11 [1358292,60502] 0 2024-02-15T18:34:54.074 INFO:tasks.workunit.client.0.smithi038.stdout:8/388: mknod d1/d4/d5/d31/d65/d80/c8b 0 2024-02-15T18:34:54.074 INFO:tasks.workunit.client.0.smithi038.stdout:7/270: write d0/db/d19/d2d/f36 [3598234,116614] 0 2024-02-15T18:34:54.074 INFO:tasks.workunit.client.0.smithi038.stdout:4/230: link d11/d1e/d20/d2b/f30 d11/d1e/d20/d2b/d35/f55 0 2024-02-15T18:34:54.075 INFO:tasks.workunit.client.0.smithi038.stdout:1/224: dread d2/d28/f43 [0,4194304] 0 2024-02-15T18:34:54.075 INFO:tasks.workunit.client.0.smithi038.stdout:2/206: creat dd/d12/d3a/f45 x:0 0 0 2024-02-15T18:34:54.075 INFO:tasks.workunit.client.0.smithi038.stdout:2/207: truncate dd/f41 127679 0 2024-02-15T18:34:54.075 INFO:tasks.workunit.client.0.smithi038.stdout:2/208: chown dd/d12/d13/l2c 3057 1 2024-02-15T18:34:54.079 INFO:tasks.workunit.client.0.smithi038.stdout:7/271: symlink d0/l5e 0 2024-02-15T18:34:54.079 INFO:tasks.workunit.client.0.smithi038.stdout:7/272: write d0/db/fc [5137998,80412] 0 2024-02-15T18:34:54.080 INFO:tasks.workunit.client.0.smithi038.stdout:7/273: truncate d0/db/d59/d15/d29/f31 1764621 0 2024-02-15T18:34:54.080 INFO:tasks.workunit.client.0.smithi038.stdout:4/231: creat d11/d1e/f56 x:0 0 0 2024-02-15T18:34:54.081 INFO:tasks.workunit.client.0.smithi038.stdout:1/225: rmdir d2/d28/d3c 39 2024-02-15T18:34:54.082 INFO:tasks.workunit.client.0.smithi038.stdout:7/274: creat d0/f5f x:0 0 0 2024-02-15T18:34:54.086 INFO:tasks.workunit.client.0.smithi038.stdout:4/232: mkdir d11/d1e/d3e/d57 0 2024-02-15T18:34:54.089 INFO:tasks.workunit.client.0.smithi038.stdout:2/209: dread dd/d12/f39 [0,4194304] 0 2024-02-15T18:34:54.089 INFO:tasks.workunit.client.0.smithi038.stdout:1/226: creat d2/d10/d19/d22/d36/d4f/f55 x:0 0 0 2024-02-15T18:34:54.089 INFO:tasks.workunit.client.0.smithi038.stdout:1/227: chown d2/d10/d32/d53 26 1 2024-02-15T18:34:54.089 INFO:tasks.workunit.client.0.smithi038.stdout:1/228: stat d2/d28/f43 0 2024-02-15T18:34:54.090 INFO:tasks.workunit.client.0.smithi038.stdout:8/389: dwrite d1/d4/d5/d6/f75 [0,4194304] 0 2024-02-15T18:34:54.091 INFO:tasks.workunit.client.0.smithi038.stdout:7/275: unlink d0/db/c49 0 2024-02-15T18:34:54.091 INFO:tasks.workunit.client.0.smithi038.stdout:7/276: readlink d0/d1b/l39 0 2024-02-15T18:34:54.092 INFO:tasks.workunit.client.0.smithi038.stdout:4/233: truncate d11/d1e/f28 2753051 0 2024-02-15T18:34:54.093 INFO:tasks.workunit.client.0.smithi038.stdout:1/229: mknod d2/c56 0 2024-02-15T18:34:54.093 INFO:tasks.workunit.client.0.smithi038.stdout:1/230: fsync d2/d10/d19/d22/d36/f44 0 2024-02-15T18:34:54.095 INFO:tasks.workunit.client.0.smithi038.stdout:9/206: sync 2024-02-15T18:34:54.095 INFO:tasks.workunit.client.0.smithi038.stdout:9/207: dread - d1/da/d10/f33 zero size 2024-02-15T18:34:54.095 INFO:tasks.workunit.client.0.smithi038.stdout:9/208: write d1/da/d10/d24/f44 [642233,2425] 0 2024-02-15T18:34:54.101 INFO:tasks.workunit.client.0.smithi038.stdout:8/390: mknod d1/d4/d10/d3d/c8c 0 2024-02-15T18:34:54.101 INFO:tasks.workunit.client.0.smithi038.stdout:7/277: creat d0/db/d59/d57/f60 x:0 0 0 2024-02-15T18:34:54.102 INFO:tasks.workunit.client.0.smithi038.stdout:4/234: symlink d11/d1e/d20/l58 0 2024-02-15T18:34:54.103 INFO:tasks.workunit.client.0.smithi038.stdout:1/231: creat d2/d10/d19/d22/f57 x:0 0 0 2024-02-15T18:34:54.105 INFO:tasks.workunit.client.0.smithi038.stdout:9/209: mkdir d1/da/d10/d13/d3d/d4b 0 2024-02-15T18:34:54.106 INFO:tasks.workunit.client.0.smithi038.stdout:8/391: rename d1/d4/d10/d43/d48/d52/c5e to d1/d4/d5/d77/d79/c8d 0 2024-02-15T18:34:54.107 INFO:tasks.workunit.client.0.smithi038.stdout:1/232: dread d2/f2d [0,4194304] 0 2024-02-15T18:34:54.107 INFO:tasks.workunit.client.0.smithi038.stdout:7/278: mkdir d0/d1b/d30/d5d/d61 0 2024-02-15T18:34:54.107 INFO:tasks.workunit.client.0.smithi038.stdout:1/233: write d2/d2e/f38 [1985568,46303] 0 2024-02-15T18:34:54.107 INFO:tasks.workunit.client.0.smithi038.stdout:9/210: mknod d1/da/d10/d13/d3d/d43/c4c 0 2024-02-15T18:34:54.108 INFO:tasks.workunit.client.0.smithi038.stdout:7/279: mkdir d0/d1b/d30/d62 0 2024-02-15T18:34:54.108 INFO:tasks.workunit.client.0.smithi038.stdout:7/280: stat d0/d1b/l39 0 2024-02-15T18:34:54.109 INFO:tasks.workunit.client.0.smithi038.stdout:9/211: mkdir d1/da/d10/d13/d3d/d43/d4d 0 2024-02-15T18:34:54.110 INFO:tasks.workunit.client.0.smithi038.stdout:1/234: creat d2/d10/d32/d53/f58 x:0 0 0 2024-02-15T18:34:54.110 INFO:tasks.workunit.client.0.smithi038.stdout:4/235: dread d11/f1b [0,4194304] 0 2024-02-15T18:34:54.111 INFO:tasks.workunit.client.0.smithi038.stdout:2/210: dwrite dd/d12/d1d/f2e [0,4194304] 0 2024-02-15T18:34:54.111 INFO:tasks.workunit.client.0.smithi038.stdout:7/281: rename d0/d2e/c4c to d0/db/d59/d57/c63 0 2024-02-15T18:34:54.113 INFO:tasks.workunit.client.0.smithi038.stdout:9/212: creat d1/da/d10/d24/d49/f4e x:0 0 0 2024-02-15T18:34:54.113 INFO:tasks.workunit.client.0.smithi038.stdout:8/392: dread d1/d4/d5/d6/f2a [0,4194304] 0 2024-02-15T18:34:54.116 INFO:tasks.workunit.client.0.smithi038.stdout:1/235: mknod d2/d10/d19/c59 0 2024-02-15T18:34:54.121 INFO:tasks.workunit.client.0.smithi038.stdout:4/236: link d11/f1a d11/d1e/d43/d4a/f59 0 2024-02-15T18:34:54.121 INFO:tasks.workunit.client.0.smithi038.stdout:4/237: write d11/d1e/d3e/f54 [127392,101449] 0 2024-02-15T18:34:54.121 INFO:tasks.workunit.client.0.smithi038.stdout:4/238: write d11/d1e/d20/d31/f32 [141312,28201] 0 2024-02-15T18:34:54.122 INFO:tasks.workunit.client.0.smithi038.stdout:2/211: chown dd/d12/l2a 311128146 1 2024-02-15T18:34:54.123 INFO:tasks.workunit.client.0.smithi038.stdout:7/282: rename d0/db/d19/d2d/f36 to d0/db/d59/d15/f64 0 2024-02-15T18:34:54.124 INFO:tasks.workunit.client.0.smithi038.stdout:1/236: mknod d2/d28/d3c/c5a 0 2024-02-15T18:34:54.124 INFO:tasks.workunit.client.0.smithi038.stdout:4/239: mknod d11/d1e/d20/d2b/d4d/c5a 0 2024-02-15T18:34:54.128 INFO:tasks.workunit.client.0.smithi038.stdout:9/213: rename d1/da/d10/d2f to d1/dc/d4f 0 2024-02-15T18:34:54.128 INFO:tasks.workunit.client.0.smithi038.stdout:1/237: dread d2/f2d [0,4194304] 0 2024-02-15T18:34:54.128 INFO:tasks.workunit.client.0.smithi038.stdout:8/393: dwrite d1/d4/d5/f23 [0,4194304] 0 2024-02-15T18:34:54.129 INFO:tasks.workunit.client.0.smithi038.stdout:7/283: symlink d0/d2e/l65 0 2024-02-15T18:34:54.129 INFO:tasks.workunit.client.0.smithi038.stdout:7/284: stat d0/db/d19/d2d/l22 0 2024-02-15T18:34:54.129 INFO:tasks.workunit.client.0.smithi038.stdout:9/214: mkdir d1/da/d36/d50 0 2024-02-15T18:34:54.130 INFO:tasks.workunit.client.0.smithi038.stdout:2/212: dread dd/d12/d18/d1f/f27 [0,4194304] 0 2024-02-15T18:34:54.130 INFO:tasks.workunit.client.0.smithi038.stdout:2/213: stat dd/d12/d18/d1f/f27 0 2024-02-15T18:34:54.138 INFO:tasks.workunit.client.0.smithi038.stdout:8/394: link d1/d4/d5/d77/d79/l81 d1/d3a/l8e 0 2024-02-15T18:34:54.139 INFO:tasks.workunit.client.0.smithi038.stdout:7/285: rmdir d0/db 39 2024-02-15T18:34:54.142 INFO:tasks.workunit.client.0.smithi038.stdout:9/215: mknod d1/da/d10/d24/d49/c51 0 2024-02-15T18:34:54.142 INFO:tasks.workunit.client.0.smithi038.stdout:9/216: chown d1/da/d36 267 1 2024-02-15T18:34:54.144 INFO:tasks.workunit.client.0.smithi038.stdout:8/395: mkdir d1/d7/d26/d8f 0 2024-02-15T18:34:54.144 INFO:tasks.workunit.client.0.smithi038.stdout:8/396: fdatasync d1/d4/d5/f37 0 2024-02-15T18:34:54.148 INFO:tasks.workunit.client.0.smithi038.stdout:7/286: rename d0/d10 to d0/d66 0 2024-02-15T18:34:54.148 INFO:tasks.workunit.client.0.smithi038.stdout:9/217: mknod d1/da/d20/c52 0 2024-02-15T18:34:54.149 INFO:tasks.workunit.client.0.smithi038.stdout:1/238: dwrite d2/d10/d19/d22/f30 [0,4194304] 0 2024-02-15T18:34:54.149 INFO:tasks.workunit.client.0.smithi038.stdout:4/240: dwrite d11/d1e/d20/d24/f2d [0,4194304] 0 2024-02-15T18:34:54.149 INFO:tasks.workunit.client.0.smithi038.stdout:4/241: write d11/d1e/d3e/f54 [495125,3551] 0 2024-02-15T18:34:54.152 INFO:tasks.workunit.client.0.smithi038.stdout:7/287: mknod d0/c67 0 2024-02-15T18:34:54.152 INFO:tasks.workunit.client.0.smithi038.stdout:2/214: dread dd/d12/d13/f28 [0,4194304] 0 2024-02-15T18:34:54.153 INFO:tasks.workunit.client.0.smithi038.stdout:9/218: creat d1/da/d10/d24/f53 x:0 0 0 2024-02-15T18:34:54.154 INFO:tasks.workunit.client.0.smithi038.stdout:1/239: symlink d2/d2e/l5b 0 2024-02-15T18:34:54.155 INFO:tasks.workunit.client.0.smithi038.stdout:1/240: readlink d2/l6 0 2024-02-15T18:34:54.160 INFO:tasks.workunit.client.0.smithi038.stdout:4/242: getdents d11/d1e/d20/d24 0 2024-02-15T18:34:54.161 INFO:tasks.workunit.client.0.smithi038.stdout:7/288: mkdir d0/d68 0 2024-02-15T18:34:54.163 INFO:tasks.workunit.client.0.smithi038.stdout:9/219: creat d1/da/f54 x:0 0 0 2024-02-15T18:34:54.164 INFO:tasks.workunit.client.0.smithi038.stdout:1/241: mknod d2/d10/d32/d53/c5c 0 2024-02-15T18:34:54.164 INFO:tasks.workunit.client.0.smithi038.stdout:1/242: chown d2/d39 1 1 2024-02-15T18:34:54.165 INFO:tasks.workunit.client.0.smithi038.stdout:8/397: dwrite d1/d4/d5/d6/f38 [0,4194304] 0 2024-02-15T18:34:54.165 INFO:tasks.workunit.client.0.smithi038.stdout:8/398: stat d1/d4/d10/d43/d48/d52/f53 0 2024-02-15T18:34:54.165 INFO:tasks.workunit.client.0.smithi038.stdout:7/289: symlink d0/d1b/d30/d5d/d61/l69 0 2024-02-15T18:34:54.166 INFO:tasks.workunit.client.0.smithi038.stdout:2/215: unlink dd/d12/d1d/c44 0 2024-02-15T18:34:54.169 INFO:tasks.workunit.client.0.smithi038.stdout:9/220: rename d1/da/d10/f1f to d1/da/d36/d50/f55 0 2024-02-15T18:34:54.169 INFO:tasks.workunit.client.0.smithi038.stdout:9/221: chown d1/dc/d4f 110035663 1 2024-02-15T18:34:54.170 INFO:tasks.workunit.client.0.smithi038.stdout:1/243: mknod d2/d10/d19/d2f/c5d 0 2024-02-15T18:34:54.177 INFO:tasks.workunit.client.0.smithi038.stdout:8/399: dread d1/d4/d5/d6/f6e [0,4194304] 0 2024-02-15T18:34:54.178 INFO:tasks.workunit.client.0.smithi038.stdout:7/290: unlink d0/d1b/d30/f33 0 2024-02-15T18:34:54.178 INFO:tasks.workunit.client.0.smithi038.stdout:0/358: sync 2024-02-15T18:34:54.178 INFO:tasks.workunit.client.0.smithi038.stdout:0/359: chown d4/d13/d2a/d35/d1e/d32/d3b/l59 12 1 2024-02-15T18:34:54.178 INFO:tasks.workunit.client.0.smithi038.stdout:9/222: creat d1/da/d36/f56 x:0 0 0 2024-02-15T18:34:54.178 INFO:tasks.workunit.client.0.smithi038.stdout:3/213: sync 2024-02-15T18:34:54.180 INFO:tasks.workunit.client.0.smithi038.stdout:9/223: rename d1/da/d10/d24/f44 to d1/da/d36/d3f/d48/f57 0 2024-02-15T18:34:54.182 INFO:tasks.workunit.client.0.smithi038.stdout:9/224: mkdir d1/da/d20/d58 0 2024-02-15T18:34:54.185 INFO:tasks.workunit.client.0.smithi038.stdout:0/360: dread d4/d13/d2a/d35/d1e/f4d [0,4194304] 0 2024-02-15T18:34:54.185 INFO:tasks.workunit.client.0.smithi038.stdout:0/361: chown d4/d13/d2a/d35/d1e/d32/d3b 33314075 1 2024-02-15T18:34:54.185 INFO:tasks.workunit.client.0.smithi038.stdout:8/400: dread d1/d4/d5/f37 [0,4194304] 0 2024-02-15T18:34:54.186 INFO:tasks.workunit.client.0.smithi038.stdout:0/362: truncate d4/d13/d2a/f4b 400063 0 2024-02-15T18:34:54.187 INFO:tasks.workunit.client.0.smithi038.stdout:8/401: mknod d1/d4/d10/d43/d48/d52/d88/c90 0 2024-02-15T18:34:54.187 INFO:tasks.workunit.client.0.smithi038.stdout:0/363: creat d4/d13/d2a/d35/d6a/f77 x:0 0 0 2024-02-15T18:34:54.188 INFO:tasks.workunit.client.0.smithi038.stdout:8/402: mkdir d1/d4/d10/d91 0 2024-02-15T18:34:54.188 INFO:tasks.workunit.client.0.smithi038.stdout:8/403: truncate d1/d7/d15/d4b/f3f 4232296 0 2024-02-15T18:34:54.189 INFO:tasks.workunit.client.0.smithi038.stdout:4/243: dwrite d11/d1e/f37 [0,4194304] 0 2024-02-15T18:34:54.190 INFO:tasks.workunit.client.0.smithi038.stdout:0/364: symlink d4/d13/d53/l78 0 2024-02-15T18:34:54.191 INFO:tasks.workunit.client.0.smithi038.stdout:0/365: chown d4/d13/d2a/f4e 114631964 1 2024-02-15T18:34:54.191 INFO:tasks.workunit.client.0.smithi038.stdout:0/366: chown d4/d13/d19/c69 484583642 1 2024-02-15T18:34:54.191 INFO:tasks.workunit.client.0.smithi038.stdout:2/216: dwrite dd/d12/d18/f1c [0,4194304] 0 2024-02-15T18:34:54.191 INFO:tasks.workunit.client.0.smithi038.stdout:9/225: dread d1/da/f17 [0,4194304] 0 2024-02-15T18:34:54.193 INFO:tasks.workunit.client.0.smithi038.stdout:4/244: rename d11/d1e/d20/d2b/f2e to d11/d1e/d20/d31/f5b 0 2024-02-15T18:34:54.195 INFO:tasks.workunit.client.0.smithi038.stdout:0/367: mkdir d4/d13/d2a/d35/d1e/d32/d79 0 2024-02-15T18:34:54.195 INFO:tasks.workunit.client.0.smithi038.stdout:3/214: dread d2/f1a [0,4194304] 0 2024-02-15T18:34:54.196 INFO:tasks.workunit.client.0.smithi038.stdout:2/217: creat dd/f46 x:0 0 0 2024-02-15T18:34:54.198 INFO:tasks.workunit.client.0.smithi038.stdout:0/368: link d4/l5b d4/d13/d2a/d35/d1e/d32/d79/l7a 0 2024-02-15T18:34:54.198 INFO:tasks.workunit.client.0.smithi038.stdout:0/369: fdatasync d4/d13/d2a/d35/d1e/f20 0 2024-02-15T18:34:54.198 INFO:tasks.workunit.client.0.smithi038.stdout:0/370: write d4/d13/d2a/d35/d1e/f50 [268896,81595] 0 2024-02-15T18:34:54.198 INFO:tasks.workunit.client.0.smithi038.stdout:7/291: dwrite d0/d66/f35 [0,4194304] 0 2024-02-15T18:34:54.199 INFO:tasks.workunit.client.0.smithi038.stdout:3/215: rmdir d2 39 2024-02-15T18:34:54.201 INFO:tasks.workunit.client.0.smithi038.stdout:0/371: truncate d4/d13/d2a/d35/f49 52683 0 2024-02-15T18:34:54.201 INFO:tasks.workunit.client.0.smithi038.stdout:0/372: stat d4/d13/d2a/d35/d6a/f77 0 2024-02-15T18:34:54.202 INFO:tasks.workunit.client.0.smithi038.stdout:3/216: mknod d2/d24/c43 0 2024-02-15T18:34:54.202 INFO:tasks.workunit.client.0.smithi038.stdout:3/217: truncate d2/f18 217498 0 2024-02-15T18:34:54.205 INFO:tasks.workunit.client.0.smithi038.stdout:4/245: dread d11/f13 [0,4194304] 0 2024-02-15T18:34:54.205 INFO:tasks.workunit.client.0.smithi038.stdout:4/246: write d11/d1e/f56 [971221,90498] 0 2024-02-15T18:34:54.206 INFO:tasks.workunit.client.0.smithi038.stdout:2/218: rename dd/d12/d13/f19 to dd/d12/f47 0 2024-02-15T18:34:54.207 INFO:tasks.workunit.client.0.smithi038.stdout:2/219: dread dd/d12/d18/f21 [0,4194304] 0 2024-02-15T18:34:54.208 INFO:tasks.workunit.client.0.smithi038.stdout:0/373: creat d4/d13/d53/f7b x:0 0 0 2024-02-15T18:34:54.209 INFO:tasks.workunit.client.0.smithi038.stdout:2/220: dread dd/f24 [0,4194304] 0 2024-02-15T18:34:54.209 INFO:tasks.workunit.client.0.smithi038.stdout:2/221: write dd/d12/d1d/f35 [315980,65256] 0 2024-02-15T18:34:54.210 INFO:tasks.workunit.client.0.smithi038.stdout:2/222: truncate dd/f46 909167 0 2024-02-15T18:34:54.210 INFO:tasks.workunit.client.0.smithi038.stdout:8/404: dwrite d1/d4/d5/f4d [0,4194304] 0 2024-02-15T18:34:54.210 INFO:tasks.workunit.client.0.smithi038.stdout:9/226: write f0 [521378,11608] 0 2024-02-15T18:34:54.211 INFO:tasks.workunit.client.0.smithi038.stdout:3/218: creat d2/d1e/f44 x:0 0 0 2024-02-15T18:34:54.211 INFO:tasks.workunit.client.0.smithi038.stdout:3/219: chown d2/d1b/d3c/l3b 2484 1 2024-02-15T18:34:54.211 INFO:tasks.workunit.client.0.smithi038.stdout:3/220: fdatasync d2/f9 0 2024-02-15T18:34:54.212 INFO:tasks.workunit.client.0.smithi038.stdout:4/247: mknod d11/d1e/d20/d31/d48/c5c 0 2024-02-15T18:34:54.212 INFO:tasks.workunit.client.0.smithi038.stdout:4/248: fsync d11/d1e/d43/f45 0 2024-02-15T18:34:54.212 INFO:tasks.workunit.client.0.smithi038.stdout:3/221: write d2/fa [2810514,61642] 0 2024-02-15T18:34:54.212 INFO:tasks.workunit.client.0.smithi038.stdout:3/222: chown d2/f1d 24871 1 2024-02-15T18:34:54.212 INFO:tasks.workunit.client.0.smithi038.stdout:3/223: chown d2/f10 2 1 2024-02-15T18:34:54.214 INFO:tasks.workunit.client.0.smithi038.stdout:7/292: rename d0/db/d59/d57 to d0/d66/d6a 0 2024-02-15T18:34:54.214 INFO:tasks.workunit.client.0.smithi038.stdout:7/293: stat d0/db/d19/c38 0 2024-02-15T18:34:54.217 INFO:tasks.workunit.client.0.smithi038.stdout:6/233: sync 2024-02-15T18:34:54.219 INFO:tasks.workunit.client.0.smithi038.stdout:7/294: dread d0/d66/f1c [0,4194304] 0 2024-02-15T18:34:54.219 INFO:tasks.workunit.client.0.smithi038.stdout:7/295: chown d0/f9 372680908 1 2024-02-15T18:34:54.221 INFO:tasks.workunit.client.0.smithi038.stdout:7/296: dread d0/d2e/f3b [0,4194304] 0 2024-02-15T18:34:54.226 INFO:tasks.workunit.client.0.smithi038.stdout:2/223: unlink c6 0 2024-02-15T18:34:54.226 INFO:tasks.workunit.client.0.smithi038.stdout:2/224: chown dd/d12/d18/d2b/l37 2319598 1 2024-02-15T18:34:54.228 INFO:tasks.workunit.client.0.smithi038.stdout:8/405: mknod d1/d4/d10/d3d/c92 0 2024-02-15T18:34:54.229 INFO:tasks.workunit.client.0.smithi038.stdout:8/406: read d1/d4/d5/d6/f6e [1009430,85445] 0 2024-02-15T18:34:54.229 INFO:tasks.workunit.client.0.smithi038.stdout:9/227: truncate d1/fb 3136676 0 2024-02-15T18:34:54.229 INFO:tasks.workunit.client.0.smithi038.stdout:9/228: fsync d1/da/d36/d50/f55 0 2024-02-15T18:34:54.229 INFO:tasks.workunit.client.0.smithi038.stdout:9/229: read - d1/da/d36/f56 zero size 2024-02-15T18:34:54.232 INFO:tasks.workunit.client.0.smithi038.stdout:4/249: symlink d11/d1e/d20/d31/d48/l5d 0 2024-02-15T18:34:54.232 INFO:tasks.workunit.client.0.smithi038.stdout:4/250: chown l0 0 1 2024-02-15T18:34:54.232 INFO:tasks.workunit.client.0.smithi038.stdout:4/251: readlink l0 0 2024-02-15T18:34:54.234 INFO:tasks.workunit.client.0.smithi038.stdout:3/224: mknod d2/d1b/d3c/d42/c45 0 2024-02-15T18:34:54.237 INFO:tasks.workunit.client.0.smithi038.stdout:0/374: link d4/d13/d2a/d35/d6a/f77 d4/f7c 0 2024-02-15T18:34:54.239 INFO:tasks.workunit.client.0.smithi038.stdout:6/234: truncate d2/f20 636396 0 2024-02-15T18:34:54.244 INFO:tasks.workunit.client.0.smithi038.stdout:2/225: creat dd/d12/d1d/d3b/d3c/f48 x:0 0 0 2024-02-15T18:34:54.244 INFO:tasks.workunit.client.0.smithi038.stdout:9/230: symlink d1/da/d10/d13/l59 0 2024-02-15T18:34:54.247 INFO:tasks.workunit.client.0.smithi038.stdout:9/231: write d1/da/d20/f26 [602044,14897] 0 2024-02-15T18:34:54.248 INFO:tasks.workunit.client.0.smithi038.stdout:4/252: mkdir d11/d1e/d43/d5e 0 2024-02-15T18:34:54.248 INFO:tasks.workunit.client.0.smithi038.stdout:4/253: fdatasync d11/d1e/f28 0 2024-02-15T18:34:54.248 INFO:tasks.workunit.client.0.smithi038.stdout:5/251: sync 2024-02-15T18:34:54.248 INFO:tasks.workunit.client.0.smithi038.stdout:3/225: mkdir d2/d1b/d3c/d46 0 2024-02-15T18:34:54.260 INFO:tasks.workunit.client.0.smithi038.stdout:0/375: mkdir d4/d13/d2a/d35/d1e/d32/d79/d7d 0 2024-02-15T18:34:54.260 INFO:tasks.workunit.client.0.smithi038.stdout:2/226: symlink dd/d12/d13/d33/l49 0 2024-02-15T18:34:54.260 INFO:tasks.workunit.client.0.smithi038.stdout:6/235: symlink d2/l44 0 2024-02-15T18:34:54.260 INFO:tasks.workunit.client.0.smithi038.stdout:0/376: fdatasync d4/d13/d2a/d35/d1e/d32/d48/f56 0 2024-02-15T18:34:54.260 INFO:tasks.workunit.client.0.smithi038.stdout:9/232: mkdir d1/dc/d5a 0 2024-02-15T18:34:54.260 INFO:tasks.workunit.client.0.smithi038.stdout:9/233: fsync d1/dc/d2b/f4a 0 2024-02-15T18:34:54.260 INFO:tasks.workunit.client.0.smithi038.stdout:4/254: dread d11/d1e/d43/d4a/f59 [0,4194304] 0 2024-02-15T18:34:54.260 INFO:tasks.workunit.client.0.smithi038.stdout:4/255: chown d11/d1e 0 1 2024-02-15T18:34:54.262 INFO:tasks.workunit.client.0.smithi038.stdout:5/252: rename d2/d20/l2c to d2/d20/d33/l49 0 2024-02-15T18:34:54.263 INFO:tasks.workunit.client.0.smithi038.stdout:3/226: rmdir d2/d1b/d3c/d42 39 2024-02-15T18:34:54.263 INFO:tasks.workunit.client.0.smithi038.stdout:3/227: fsync d2/d1e/f32 0 2024-02-15T18:34:54.263 INFO:tasks.workunit.client.0.smithi038.stdout:3/228: chown d2/f17 10 1 2024-02-15T18:34:54.264 INFO:tasks.workunit.client.0.smithi038.stdout:2/227: truncate dd/d12/f36 4659010 0 2024-02-15T18:34:54.264 INFO:tasks.workunit.client.0.smithi038.stdout:2/228: readlink dd/d12/d13/d33/l38 0 2024-02-15T18:34:54.266 INFO:tasks.workunit.client.0.smithi038.stdout:0/377: link d4/d13/d2a/d35/c5f d4/c7e 0 2024-02-15T18:34:54.267 INFO:tasks.workunit.client.0.smithi038.stdout:6/236: creat d2/d4/d2c/d39/f45 x:0 0 0 2024-02-15T18:34:54.269 INFO:tasks.workunit.client.0.smithi038.stdout:9/234: mkdir d1/dc/d5a/d5b 0 2024-02-15T18:34:54.269 INFO:tasks.workunit.client.0.smithi038.stdout:9/235: fdatasync d1/da/d10/d24/d49/f4e 0 2024-02-15T18:34:54.269 INFO:tasks.workunit.client.0.smithi038.stdout:9/236: chown d1/f14 4 1 2024-02-15T18:34:54.270 INFO:tasks.workunit.client.0.smithi038.stdout:4/256: creat d11/d1e/d43/d4a/f5f x:0 0 0 2024-02-15T18:34:54.270 INFO:tasks.workunit.client.0.smithi038.stdout:4/257: chown d11/d1e/d20/d31/c4f 299245 1 2024-02-15T18:34:54.271 INFO:tasks.workunit.client.0.smithi038.stdout:4/258: write d11/d1e/d43/d4a/f5f [1044344,94550] 0 2024-02-15T18:34:54.273 INFO:tasks.workunit.client.0.smithi038.stdout:5/253: unlink d2/d20/d33/f36 0 2024-02-15T18:34:54.277 INFO:tasks.workunit.client.0.smithi038.stdout:1/244: sync 2024-02-15T18:34:54.277 INFO:tasks.workunit.client.0.smithi038.stdout:1/245: write d2/d10/d19/d22/d36/d4f/f51 [21025,128596] 0 2024-02-15T18:34:54.277 INFO:tasks.workunit.client.0.smithi038.stdout:3/229: symlink d2/d1e/l47 0 2024-02-15T18:34:54.277 INFO:tasks.workunit.client.0.smithi038.stdout:3/230: stat d2/fa 0 2024-02-15T18:34:54.279 INFO:tasks.workunit.client.0.smithi038.stdout:2/229: mkdir dd/d12/d1d/d3b/d3c/d4a 0 2024-02-15T18:34:54.280 INFO:tasks.workunit.client.0.smithi038.stdout:1/246: rename d2/d10/d19/l20 to d2/d2e/l5e 0 2024-02-15T18:34:54.282 INFO:tasks.workunit.client.0.smithi038.stdout:3/231: mknod d2/d1b/d3c/d41/c48 0 2024-02-15T18:34:54.282 INFO:tasks.workunit.client.0.smithi038.stdout:4/259: mkdir d11/d1e/d20/d31/d48/d60 0 2024-02-15T18:34:54.285 INFO:tasks.workunit.client.0.smithi038.stdout:2/230: creat dd/d12/d1d/d3b/f4b x:0 0 0 2024-02-15T18:34:54.285 INFO:tasks.workunit.client.0.smithi038.stdout:1/247: mkdir d2/d10/d19/d5f 0 2024-02-15T18:34:54.285 INFO:tasks.workunit.client.0.smithi038.stdout:3/232: creat d2/d24/f49 x:0 0 0 2024-02-15T18:34:54.287 INFO:tasks.workunit.client.0.smithi038.stdout:2/231: dread dd/d12/d13/f28 [0,4194304] 0 2024-02-15T18:34:54.289 INFO:tasks.workunit.client.0.smithi038.stdout:3/233: write d2/f15 [4656134,48993] 0 2024-02-15T18:34:54.290 INFO:tasks.workunit.client.0.smithi038.stdout:2/232: link dd/d12/d1d/l1b dd/l4c 0 2024-02-15T18:34:54.292 INFO:tasks.workunit.client.0.smithi038.stdout:3/234: rename d2/f10 to d2/d1b/f4a 0 2024-02-15T18:34:54.292 INFO:tasks.workunit.client.0.smithi038.stdout:2/233: creat dd/d12/d1d/d3b/d3c/f4d x:0 0 0 2024-02-15T18:34:54.297 INFO:tasks.workunit.client.0.smithi038.stdout:2/234: write dd/d12/f39 [4466242,66561] 0 2024-02-15T18:34:54.298 INFO:tasks.workunit.client.0.smithi038.stdout:2/235: stat dd/d12/d3a/l3e 0 2024-02-15T18:34:54.298 INFO:tasks.workunit.client.0.smithi038.stdout:2/236: truncate fc 5176123 0 2024-02-15T18:34:54.299 INFO:tasks.workunit.client.0.smithi038.stdout:2/237: symlink dd/d12/l4e 0 2024-02-15T18:34:54.299 INFO:tasks.workunit.client.0.smithi038.stdout:2/238: chown dd/d12/d13/c16 13 1 2024-02-15T18:34:54.309 INFO:tasks.workunit.client.0.smithi038.stdout:0/378: dwrite d4/d13/d2a/d35/d1e/d32/d48/f56 [4194304,4194304] 0 2024-02-15T18:34:54.309 INFO:tasks.workunit.client.0.smithi038.stdout:9/237: dwrite d1/da/d36/f56 [0,4194304] 0 2024-02-15T18:34:54.309 INFO:tasks.workunit.client.0.smithi038.stdout:6/237: dwrite d2/d4/f1f [0,4194304] 0 2024-02-15T18:34:54.309 INFO:tasks.workunit.client.0.smithi038.stdout:4/260: dwrite d11/d1e/f37 [4194304,4194304] 0 2024-02-15T18:34:54.310 INFO:tasks.workunit.client.0.smithi038.stdout:9/238: truncate d1/fb 94521 0 2024-02-15T18:34:54.310 INFO:tasks.workunit.client.0.smithi038.stdout:1/248: dread d2/d10/d19/d22/d36/d4f/f51 [0,4194304] 0 2024-02-15T18:34:54.310 INFO:tasks.workunit.client.0.smithi038.stdout:1/249: chown d2/d10/d32/l3b 555657 1 2024-02-15T18:34:54.310 INFO:tasks.workunit.client.0.smithi038.stdout:1/250: write f1 [570660,81138] 0 2024-02-15T18:34:54.311 INFO:tasks.workunit.client.0.smithi038.stdout:5/254: dwrite d2/f2b [0,4194304] 0 2024-02-15T18:34:54.311 INFO:tasks.workunit.client.0.smithi038.stdout:0/379: write d4/d13/d2a/d35/d1e/d32/d48/f56 [5583576,95205] 0 2024-02-15T18:34:54.311 INFO:tasks.workunit.client.0.smithi038.stdout:6/238: creat d2/f46 x:0 0 0 2024-02-15T18:34:54.312 INFO:tasks.workunit.client.0.smithi038.stdout:4/261: creat d11/d1e/d20/d2b/f61 x:0 0 0 2024-02-15T18:34:54.315 INFO:tasks.workunit.client.0.smithi038.stdout:9/239: creat d1/da/d10/d13/d3d/f5c x:0 0 0 2024-02-15T18:34:54.315 INFO:tasks.workunit.client.0.smithi038.stdout:1/251: symlink d2/d10/d19/d22/d36/l60 0 2024-02-15T18:34:54.316 INFO:tasks.workunit.client.0.smithi038.stdout:5/255: mknod d2/c4a 0 2024-02-15T18:34:54.322 INFO:tasks.workunit.client.0.smithi038.stdout:2/239: getdents dd 0 2024-02-15T18:34:54.322 INFO:tasks.workunit.client.0.smithi038.stdout:2/240: chown dd/f46 0 1 2024-02-15T18:34:54.325 INFO:tasks.workunit.client.0.smithi038.stdout:0/380: unlink d4/d13/d2a/d35/d1e/f40 0 2024-02-15T18:34:54.327 INFO:tasks.workunit.client.0.smithi038.stdout:6/239: creat d2/d4/d19/f47 x:0 0 0 2024-02-15T18:34:54.327 INFO:tasks.workunit.client.0.smithi038.stdout:4/262: mkdir d11/d1e/d43/d5e/d62 0 2024-02-15T18:34:54.328 INFO:tasks.workunit.client.0.smithi038.stdout:4/263: write d11/d1e/f29 [2489934,84071] 0 2024-02-15T18:34:54.329 INFO:tasks.workunit.client.0.smithi038.stdout:9/240: write d1/fb [552522,68221] 0 2024-02-15T18:34:54.331 INFO:tasks.workunit.client.0.smithi038.stdout:2/241: dread dd/d12/d18/f1c [0,4194304] 0 2024-02-15T18:34:54.331 INFO:tasks.workunit.client.0.smithi038.stdout:1/252: dwrite d2/d10/d19/d22/f30 [0,4194304] 0 2024-02-15T18:34:54.331 INFO:tasks.workunit.client.0.smithi038.stdout:1/253: readlink d2/d10/d19/d22/l25 0 2024-02-15T18:34:54.332 INFO:tasks.workunit.client.0.smithi038.stdout:1/254: fdatasync d2/d28/f50 0 2024-02-15T18:34:54.332 INFO:tasks.workunit.client.0.smithi038.stdout:3/235: truncate d2/f15 4094742 0 2024-02-15T18:34:54.332 INFO:tasks.workunit.client.0.smithi038.stdout:0/381: stat d4/d13/d2a/l70 0 2024-02-15T18:34:54.333 INFO:tasks.workunit.client.0.smithi038.stdout:6/240: rename d2/d10/l3d to d2/d10/l48 0 2024-02-15T18:34:54.337 INFO:tasks.workunit.client.0.smithi038.stdout:4/264: creat d11/d1e/d43/f63 x:0 0 0 2024-02-15T18:34:54.338 INFO:tasks.workunit.client.0.smithi038.stdout:9/241: symlink d1/l5d 0 2024-02-15T18:34:54.339 INFO:tasks.workunit.client.0.smithi038.stdout:9/242: write d1/da/d36/d50/f55 [109195,109886] 0 2024-02-15T18:34:54.340 INFO:tasks.workunit.client.0.smithi038.stdout:2/242: mkdir dd/d12/d18/d1f/d4f 0 2024-02-15T18:34:54.346 INFO:tasks.workunit.client.0.smithi038.stdout:1/255: symlink d2/d10/l61 0 2024-02-15T18:34:54.346 INFO:tasks.workunit.client.0.smithi038.stdout:1/256: dread - d2/f42 zero size 2024-02-15T18:34:54.350 INFO:tasks.workunit.client.0.smithi038.stdout:3/236: symlink d2/l4b 0 2024-02-15T18:34:54.350 INFO:tasks.workunit.client.0.smithi038.stdout:3/237: stat d2/d24/c43 0 2024-02-15T18:34:54.350 INFO:tasks.workunit.client.0.smithi038.stdout:0/382: mkdir d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f 0 2024-02-15T18:34:54.352 INFO:tasks.workunit.client.0.smithi038.stdout:6/241: creat d2/d10/f49 x:0 0 0 2024-02-15T18:34:54.353 INFO:tasks.workunit.client.0.smithi038.stdout:9/243: symlink d1/da/d10/d13/d3d/d43/l5e 0 2024-02-15T18:34:54.355 INFO:tasks.workunit.client.0.smithi038.stdout:1/257: unlink d2/d10/d19/d22/d36/d4f/f51 0 2024-02-15T18:34:54.355 INFO:tasks.workunit.client.0.smithi038.stdout:1/258: fsync d2/f4 0 2024-02-15T18:34:54.356 INFO:tasks.workunit.client.0.smithi038.stdout:3/238: mknod d2/d1b/d3c/d41/c4c 0 2024-02-15T18:34:54.356 INFO:tasks.workunit.client.0.smithi038.stdout:3/239: fdatasync d2/f1d 0 2024-02-15T18:34:54.357 INFO:tasks.workunit.client.0.smithi038.stdout:0/383: mkdir d4/d13/d2a/d80 0 2024-02-15T18:34:54.357 INFO:tasks.workunit.client.0.smithi038.stdout:0/384: chown d4/d13/d2a/d35/d6a 6996618 1 2024-02-15T18:34:54.357 INFO:tasks.workunit.client.0.smithi038.stdout:1/259: mkdir d2/d10/d62 0 2024-02-15T18:34:54.357 INFO:tasks.workunit.client.0.smithi038.stdout:1/260: dread - d2/d10/d19/d22/d36/d4f/f55 zero size 2024-02-15T18:34:54.358 INFO:tasks.workunit.client.0.smithi038.stdout:1/261: read d2/d10/d19/f1d [3124317,108598] 0 2024-02-15T18:34:54.359 INFO:tasks.workunit.client.0.smithi038.stdout:4/265: dwrite d11/d1e/d20/d24/f34 [0,4194304] 0 2024-02-15T18:34:54.359 INFO:tasks.workunit.client.0.smithi038.stdout:0/385: link d4/d13/d2a/c42 d4/d13/d2a/d35/d1e/d32/d79/d7d/c81 0 2024-02-15T18:34:54.360 INFO:tasks.workunit.client.0.smithi038.stdout:1/262: creat d2/d28/d3c/f63 x:0 0 0 2024-02-15T18:34:54.360 INFO:tasks.workunit.client.0.smithi038.stdout:0/386: write d4/d13/d2a/d35/d1e/f4d [112297,92904] 0 2024-02-15T18:34:54.361 INFO:tasks.workunit.client.0.smithi038.stdout:2/243: dwrite dd/d12/d1d/f31 [0,4194304] 0 2024-02-15T18:34:54.361 INFO:tasks.workunit.client.0.smithi038.stdout:1/263: write f1 [480048,112481] 0 2024-02-15T18:34:54.362 INFO:tasks.workunit.client.0.smithi038.stdout:7/297: sync 2024-02-15T18:34:54.362 INFO:tasks.workunit.client.0.smithi038.stdout:8/407: sync 2024-02-15T18:34:54.364 INFO:tasks.workunit.client.0.smithi038.stdout:0/387: creat d4/d13/d19/f82 x:0 0 0 2024-02-15T18:34:54.364 INFO:tasks.workunit.client.0.smithi038.stdout:4/266: dread d11/d1e/d20/d31/f5b [0,4194304] 0 2024-02-15T18:34:54.365 INFO:tasks.workunit.client.0.smithi038.stdout:2/244: rename dd/d12/d1d/d3b/d3c/f4d to dd/d12/d1d/f50 0 2024-02-15T18:34:54.371 INFO:tasks.workunit.client.0.smithi038.stdout:6/242: dwrite f1 [0,4194304] 0 2024-02-15T18:34:54.371 INFO:tasks.workunit.client.0.smithi038.stdout:6/243: chown d2/d4/f1e 413263 1 2024-02-15T18:34:54.371 INFO:tasks.workunit.client.0.smithi038.stdout:6/244: fdatasync d2/d4/f27 0 2024-02-15T18:34:54.371 INFO:tasks.workunit.client.0.smithi038.stdout:6/245: chown d2/d10/f30 8686 1 2024-02-15T18:34:54.371 INFO:tasks.workunit.client.0.smithi038.stdout:6/246: chown d2/d4/d28/f36 56132840 1 2024-02-15T18:34:54.371 INFO:tasks.workunit.client.0.smithi038.stdout:6/247: dread - d2/d4/f2f zero size 2024-02-15T18:34:54.371 INFO:tasks.workunit.client.0.smithi038.stdout:8/408: unlink d1/d7/d22/f7c 0 2024-02-15T18:34:54.371 INFO:tasks.workunit.client.0.smithi038.stdout:7/298: symlink d0/d1b/d30/d50/l6b 0 2024-02-15T18:34:54.373 INFO:tasks.workunit.client.0.smithi038.stdout:2/245: unlink dd/d12/d13/d33/l49 0 2024-02-15T18:34:54.373 INFO:tasks.workunit.client.0.smithi038.stdout:8/409: write d1/d7/d15/d4b/f55 [2576326,40336] 0 2024-02-15T18:34:54.373 INFO:tasks.workunit.client.0.smithi038.stdout:0/388: creat d4/d13/d2a/d35/f83 x:0 0 0 2024-02-15T18:34:54.373 INFO:tasks.workunit.client.0.smithi038.stdout:1/264: link d2/c52 d2/d28/c64 0 2024-02-15T18:34:54.374 INFO:tasks.workunit.client.0.smithi038.stdout:8/410: creat d1/d7/d15/d4b/f93 x:0 0 0 2024-02-15T18:34:54.383 INFO:tasks.workunit.client.0.smithi038.stdout:4/267: dwrite d11/d1e/d20/d24/f46 [0,4194304] 0 2024-02-15T18:34:54.384 INFO:tasks.workunit.client.0.smithi038.stdout:4/268: truncate d11/d1e/d3e/f47 5084873 0 2024-02-15T18:34:54.389 INFO:tasks.workunit.client.0.smithi038.stdout:0/389: creat d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/f84 x:0 0 0 2024-02-15T18:34:54.390 INFO:tasks.workunit.client.0.smithi038.stdout:1/265: mknod d2/d10/d32/c65 0 2024-02-15T18:34:54.391 INFO:tasks.workunit.client.0.smithi038.stdout:8/411: symlink d1/d4/d10/d43/d48/d52/d88/l94 0 2024-02-15T18:34:54.392 INFO:tasks.workunit.client.0.smithi038.stdout:8/412: stat d1/d4/d5/d31/l87 0 2024-02-15T18:34:54.398 INFO:tasks.workunit.client.0.smithi038.stdout:4/269: symlink d11/d1e/l64 0 2024-02-15T18:34:54.399 INFO:tasks.workunit.client.0.smithi038.stdout:0/390: symlink d4/d13/d2a/d80/l85 0 2024-02-15T18:34:54.400 INFO:tasks.workunit.client.0.smithi038.stdout:1/266: rename d2/d2e/l5e to d2/d10/d19/d22/d36/l66 0 2024-02-15T18:34:54.400 INFO:tasks.workunit.client.0.smithi038.stdout:1/267: dread - d2/d10/f37 zero size 2024-02-15T18:34:54.401 INFO:tasks.workunit.client.0.smithi038.stdout:6/248: dwrite d2/f3 [4194304,4194304] 0 2024-02-15T18:34:54.401 INFO:tasks.workunit.client.0.smithi038.stdout:6/249: readlink d2/d4/l3b 0 2024-02-15T18:34:54.401 INFO:tasks.workunit.client.0.smithi038.stdout:8/413: creat d1/d7/d22/f95 x:0 0 0 2024-02-15T18:34:54.402 INFO:tasks.workunit.client.0.smithi038.stdout:0/391: getdents d4/d13/d2a 0 2024-02-15T18:34:54.402 INFO:tasks.workunit.client.0.smithi038.stdout:1/268: symlink d2/d2e/l67 0 2024-02-15T18:34:54.405 INFO:tasks.workunit.client.0.smithi038.stdout:8/414: link d1/d7/d22/c7d d1/d7/d15/d4b/c96 0 2024-02-15T18:34:54.406 INFO:tasks.workunit.client.0.smithi038.stdout:1/269: unlink d2/c1c 0 2024-02-15T18:34:54.406 INFO:tasks.workunit.client.0.smithi038.stdout:7/299: dwrite d0/db/fc [4194304,4194304] 0 2024-02-15T18:34:54.407 INFO:tasks.workunit.client.0.smithi038.stdout:8/415: stat d1/d4/d5/d77/d79/l81 0 2024-02-15T18:34:54.407 INFO:tasks.workunit.client.0.smithi038.stdout:2/246: dwrite dd/d12/d1d/f50 [0,4194304] 0 2024-02-15T18:34:54.409 INFO:tasks.workunit.client.0.smithi038.stdout:1/270: creat d2/d10/d62/f68 x:0 0 0 2024-02-15T18:34:54.409 INFO:tasks.workunit.client.0.smithi038.stdout:1/271: stat d2/d10/d19/d22/l33 0 2024-02-15T18:34:54.412 INFO:tasks.workunit.client.0.smithi038.stdout:4/270: dread d11/d1e/f23 [0,4194304] 0 2024-02-15T18:34:54.413 INFO:tasks.workunit.client.0.smithi038.stdout:8/416: rename d1/d4/d10/d43/d48/d52/f53 to d1/d4/d10/d3d/f97 0 2024-02-15T18:34:54.413 INFO:tasks.workunit.client.0.smithi038.stdout:8/417: write d1/d7/d15/d6a/f6b [103347,123571] 0 2024-02-15T18:34:54.418 INFO:tasks.workunit.client.0.smithi038.stdout:6/250: dwrite d2/d4/f27 [4194304,4194304] 0 2024-02-15T18:34:54.418 INFO:tasks.workunit.client.0.smithi038.stdout:6/251: chown f1 2055506649 1 2024-02-15T18:34:54.420 INFO:tasks.workunit.client.0.smithi038.stdout:6/252: write f1 [3219779,84068] 0 2024-02-15T18:34:54.420 INFO:tasks.workunit.client.0.smithi038.stdout:2/247: symlink dd/d12/d1d/d3b/l51 0 2024-02-15T18:34:54.420 INFO:tasks.workunit.client.0.smithi038.stdout:6/253: fsync d2/d4/d2c/f3f 0 2024-02-15T18:34:54.420 INFO:tasks.workunit.client.0.smithi038.stdout:2/248: write dd/d12/d1d/f35 [1349771,93230] 0 2024-02-15T18:34:54.425 INFO:tasks.workunit.client.0.smithi038.stdout:4/271: mkdir d11/d1e/d20/d2b/d65 0 2024-02-15T18:34:54.425 INFO:tasks.workunit.client.0.smithi038.stdout:4/272: readlink d11/d1e/d20/d2b/l38 0 2024-02-15T18:34:54.425 INFO:tasks.workunit.client.0.smithi038.stdout:4/273: chown d11/d1e/d20/d24/d40 153 1 2024-02-15T18:34:54.429 INFO:tasks.workunit.client.0.smithi038.stdout:7/300: dwrite d0/d2e/f4d [0,4194304] 0 2024-02-15T18:34:54.430 INFO:tasks.workunit.client.0.smithi038.stdout:4/274: dread d11/f1d [0,4194304] 0 2024-02-15T18:34:54.430 INFO:tasks.workunit.client.0.smithi038.stdout:2/249: link l9 dd/d12/d1d/d3b/d3c/l52 0 2024-02-15T18:34:54.430 INFO:tasks.workunit.client.0.smithi038.stdout:2/250: write dd/d12/d18/d1f/f27 [1487580,109692] 0 2024-02-15T18:34:54.434 INFO:tasks.workunit.client.0.smithi038.stdout:4/275: creat d11/d1e/d20/d31/d48/d60/f66 x:0 0 0 2024-02-15T18:34:54.434 INFO:tasks.workunit.client.0.smithi038.stdout:4/276: chown d11/d1e/d20/d31/l50 237470 1 2024-02-15T18:34:54.435 INFO:tasks.workunit.client.0.smithi038.stdout:2/251: mknod dd/d12/d1d/d3b/d3c/d4a/c53 0 2024-02-15T18:34:54.436 INFO:tasks.workunit.client.0.smithi038.stdout:2/252: fdatasync dd/f24 0 2024-02-15T18:34:54.436 INFO:tasks.workunit.client.0.smithi038.stdout:2/253: mkdir dd/d12/d13/d54 0 2024-02-15T18:34:54.438 INFO:tasks.workunit.client.0.smithi038.stdout:6/254: write d2/f3 [7836163,40195] 0 2024-02-15T18:34:54.439 INFO:tasks.workunit.client.0.smithi038.stdout:1/272: dwrite d2/d28/f43 [0,4194304] 0 2024-02-15T18:34:54.440 INFO:tasks.workunit.client.0.smithi038.stdout:1/273: chown d2/d2e/c3a 30242330 1 2024-02-15T18:34:54.444 INFO:tasks.workunit.client.0.smithi038.stdout:8/418: dwrite d1/d7/f2d [0,4194304] 0 2024-02-15T18:34:54.447 INFO:tasks.workunit.client.0.smithi038.stdout:1/274: mkdir d2/d69 0 2024-02-15T18:34:54.447 INFO:tasks.workunit.client.0.smithi038.stdout:1/275: stat d2/f4 0 2024-02-15T18:34:54.448 INFO:tasks.workunit.client.0.smithi038.stdout:2/254: rmdir dd/d12/d1d/d3b 39 2024-02-15T18:34:54.452 INFO:tasks.workunit.client.0.smithi038.stdout:1/276: symlink d2/d10/d19/d22/d36/d4f/l6a 0 2024-02-15T18:34:54.453 INFO:tasks.workunit.client.0.smithi038.stdout:2/255: rename l9 to dd/d12/d1d/d3b/d3c/l55 0 2024-02-15T18:34:54.453 INFO:tasks.workunit.client.0.smithi038.stdout:8/419: write d1/d4/d5/d6/ff [2685078,80962] 0 2024-02-15T18:34:54.455 INFO:tasks.workunit.client.0.smithi038.stdout:8/420: creat d1/d4/d5/d31/d65/f98 x:0 0 0 2024-02-15T18:34:54.455 INFO:tasks.workunit.client.0.smithi038.stdout:8/421: symlink d1/d3a/l99 0 2024-02-15T18:34:54.457 INFO:tasks.workunit.client.0.smithi038.stdout:8/422: dread d1/f50 [0,4194304] 0 2024-02-15T18:34:54.462 INFO:tasks.workunit.client.0.smithi038.stdout:5/256: sync 2024-02-15T18:34:54.462 INFO:tasks.workunit.client.0.smithi038.stdout:5/257: fsync d2/f38 0 2024-02-15T18:34:54.462 INFO:tasks.workunit.client.0.smithi038.stdout:1/277: dread f1 [0,4194304] 0 2024-02-15T18:34:54.463 INFO:tasks.workunit.client.0.smithi038.stdout:6/255: dwrite d2/f1a [0,4194304] 0 2024-02-15T18:34:54.465 INFO:tasks.workunit.client.0.smithi038.stdout:5/258: rename d2/d3/d12/f17 to d2/d3/f4b 0 2024-02-15T18:34:54.465 INFO:tasks.workunit.client.0.smithi038.stdout:4/277: dwrite d11/f13 [0,4194304] 0 2024-02-15T18:34:54.465 INFO:tasks.workunit.client.0.smithi038.stdout:5/259: write d2/d3/d12/f28 [411385,66750] 0 2024-02-15T18:34:54.465 INFO:tasks.workunit.client.0.smithi038.stdout:5/260: fsync d2/fc 0 2024-02-15T18:34:54.465 INFO:tasks.workunit.client.0.smithi038.stdout:4/278: write d11/d1e/d3e/f54 [1405089,14759] 0 2024-02-15T18:34:54.465 INFO:tasks.workunit.client.0.smithi038.stdout:6/256: creat d2/d4/f4a x:0 0 0 2024-02-15T18:34:54.466 INFO:tasks.workunit.client.0.smithi038.stdout:5/261: creat d2/f4c x:0 0 0 2024-02-15T18:34:54.469 INFO:tasks.workunit.client.0.smithi038.stdout:6/257: rename d2/d4/d28/f36 to d2/d4/d2c/f4b 0 2024-02-15T18:34:54.469 INFO:tasks.workunit.client.0.smithi038.stdout:5/262: mknod d2/d3/d12/c4d 0 2024-02-15T18:34:54.472 INFO:tasks.workunit.client.0.smithi038.stdout:5/263: creat d2/d3/f4e x:0 0 0 2024-02-15T18:34:54.474 INFO:tasks.workunit.client.0.smithi038.stdout:5/264: mknod d2/d20/c4f 0 2024-02-15T18:34:54.474 INFO:tasks.workunit.client.0.smithi038.stdout:5/265: truncate d2/d3/d12/f28 642324 0 2024-02-15T18:34:54.474 INFO:tasks.workunit.client.0.smithi038.stdout:5/266: creat d2/d20/f50 x:0 0 0 2024-02-15T18:34:54.475 INFO:tasks.workunit.client.0.smithi038.stdout:5/267: readlink d2/d20/l24 0 2024-02-15T18:34:54.475 INFO:tasks.workunit.client.0.smithi038.stdout:4/279: dread d11/f1a [0,4194304] 0 2024-02-15T18:34:54.476 INFO:tasks.workunit.client.0.smithi038.stdout:5/268: unlink d2/c4a 0 2024-02-15T18:34:54.478 INFO:tasks.workunit.client.0.smithi038.stdout:4/280: dread d11/d1e/d20/d31/f32 [0,4194304] 0 2024-02-15T18:34:54.480 INFO:tasks.workunit.client.0.smithi038.stdout:8/423: dwrite d1/d4/d10/d43/f70 [0,4194304] 0 2024-02-15T18:34:54.480 INFO:tasks.workunit.client.0.smithi038.stdout:1/278: dwrite d2/d10/d19/d22/d36/f44 [0,4194304] 0 2024-02-15T18:34:54.481 INFO:tasks.workunit.client.0.smithi038.stdout:2/256: dwrite dd/d12/d1d/f50 [0,4194304] 0 2024-02-15T18:34:54.481 INFO:tasks.workunit.client.0.smithi038.stdout:2/257: read dd/f41 [4353,8472] 0 2024-02-15T18:34:54.481 INFO:tasks.workunit.client.0.smithi038.stdout:4/281: dread d11/d1e/d20/d2f/f3d [0,4194304] 0 2024-02-15T18:34:54.481 INFO:tasks.workunit.client.0.smithi038.stdout:1/279: write d2/f2d [2049659,87157] 0 2024-02-15T18:34:54.481 INFO:tasks.workunit.client.0.smithi038.stdout:1/280: write d2/d28/f50 [79250,5888] 0 2024-02-15T18:34:54.485 INFO:tasks.workunit.client.0.smithi038.stdout:6/258: rmdir d2/d4 39 2024-02-15T18:34:54.486 INFO:tasks.workunit.client.0.smithi038.stdout:4/282: dread d11/f1a [0,4194304] 0 2024-02-15T18:34:54.487 INFO:tasks.workunit.client.0.smithi038.stdout:6/259: dread d2/f20 [0,4194304] 0 2024-02-15T18:34:54.487 INFO:tasks.workunit.client.0.smithi038.stdout:8/424: mkdir d1/d4/d10/d43/d48/d9a 0 2024-02-15T18:34:54.489 INFO:tasks.workunit.client.0.smithi038.stdout:2/258: mknod dd/d12/d18/c56 0 2024-02-15T18:34:54.489 INFO:tasks.workunit.client.0.smithi038.stdout:2/259: readlink dd/d12/d1d/d3b/l51 0 2024-02-15T18:34:54.489 INFO:tasks.workunit.client.0.smithi038.stdout:2/260: truncate dd/d12/d1d/d3b/d3c/f48 199918 0 2024-02-15T18:34:54.493 INFO:tasks.workunit.client.0.smithi038.stdout:9/244: sync 2024-02-15T18:34:54.494 INFO:tasks.workunit.client.0.smithi038.stdout:1/281: getdents d2/d10/d32/d53 0 2024-02-15T18:34:54.494 INFO:tasks.workunit.client.0.smithi038.stdout:1/282: write d2/d10/d19/d22/d36/d4f/f55 [262142,30407] 0 2024-02-15T18:34:54.494 INFO:tasks.workunit.client.0.smithi038.stdout:1/283: chown d2/d10/c17 113 1 2024-02-15T18:34:54.495 INFO:tasks.workunit.client.0.smithi038.stdout:1/284: write d2/d10/d19/d22/f2c [904134,56103] 0 2024-02-15T18:34:54.496 INFO:tasks.workunit.client.0.smithi038.stdout:4/283: dread d11/d1e/f28 [0,4194304] 0 2024-02-15T18:34:54.496 INFO:tasks.workunit.client.0.smithi038.stdout:6/260: rename d2/f20 to d2/d10/f4c 0 2024-02-15T18:34:54.496 INFO:tasks.workunit.client.0.smithi038.stdout:1/285: rmdir d2/d10/d19/d2f 39 2024-02-15T18:34:54.500 INFO:tasks.workunit.client.0.smithi038.stdout:6/261: read d2/d4/fc [1618772,96713] 0 2024-02-15T18:34:54.501 INFO:tasks.workunit.client.0.smithi038.stdout:4/284: mkdir d11/d1e/d20/d2b/d4d/d67 0 2024-02-15T18:34:54.501 INFO:tasks.workunit.client.0.smithi038.stdout:8/425: rename d1/d7/f51 to d1/d4/f9b 0 2024-02-15T18:34:54.501 INFO:tasks.workunit.client.0.smithi038.stdout:4/285: dread - d11/d1e/d43/f45 zero size 2024-02-15T18:34:54.501 INFO:tasks.workunit.client.0.smithi038.stdout:1/286: mkdir d2/d28/d6b 0 2024-02-15T18:34:54.501 INFO:tasks.workunit.client.0.smithi038.stdout:1/287: chown d2/d2e 1551 1 2024-02-15T18:34:54.501 INFO:tasks.workunit.client.0.smithi038.stdout:4/286: write d11/d1e/d20/d31/f32 [593237,9868] 0 2024-02-15T18:34:54.502 INFO:tasks.workunit.client.0.smithi038.stdout:6/262: rename d2/d10/l48 to d2/d10/l4d 0 2024-02-15T18:34:54.503 INFO:tasks.workunit.client.0.smithi038.stdout:8/426: link d1/d7/l9 d1/d7/l9c 0 2024-02-15T18:34:54.505 INFO:tasks.workunit.client.0.smithi038.stdout:1/288: creat d2/d69/f6c x:0 0 0 2024-02-15T18:34:54.505 INFO:tasks.workunit.client.0.smithi038.stdout:4/287: creat d11/d1e/d3e/f68 x:0 0 0 2024-02-15T18:34:54.505 INFO:tasks.workunit.client.0.smithi038.stdout:4/288: chown d11/d1e/f37 54427616 1 2024-02-15T18:34:54.506 INFO:tasks.workunit.client.0.smithi038.stdout:4/289: chown d11/d1e/d20/d2f 70064 1 2024-02-15T18:34:54.506 INFO:tasks.workunit.client.0.smithi038.stdout:4/290: stat d11/d1e/c41 0 2024-02-15T18:34:54.507 INFO:tasks.workunit.client.0.smithi038.stdout:6/263: unlink d2/d4/c2e 0 2024-02-15T18:34:54.507 INFO:tasks.workunit.client.0.smithi038.stdout:6/264: readlink d2/d4/d28/l34 0 2024-02-15T18:34:54.508 INFO:tasks.workunit.client.0.smithi038.stdout:9/245: dwrite d1/f9 [0,4194304] 0 2024-02-15T18:34:54.508 INFO:tasks.workunit.client.0.smithi038.stdout:9/246: chown d1/da/d10/d24/f53 235399 1 2024-02-15T18:34:54.509 INFO:tasks.workunit.client.0.smithi038.stdout:2/261: rename c5 to dd/d12/d13/d54/c57 0 2024-02-15T18:34:54.509 INFO:tasks.workunit.client.0.smithi038.stdout:2/262: chown dd/d12/d18/d1f/f30 10038628 1 2024-02-15T18:34:54.509 INFO:tasks.workunit.client.0.smithi038.stdout:2/263: truncate dd/d12/d18/d1f/f27 2119746 0 2024-02-15T18:34:54.511 INFO:tasks.workunit.client.0.smithi038.stdout:8/427: unlink d1/d7/d15/d4b/l82 0 2024-02-15T18:34:54.511 INFO:tasks.workunit.client.0.smithi038.stdout:1/289: mknod d2/d10/c6d 0 2024-02-15T18:34:54.514 INFO:tasks.workunit.client.0.smithi038.stdout:8/428: dread d1/d4/d5/f20 [0,4194304] 0 2024-02-15T18:34:54.514 INFO:tasks.workunit.client.0.smithi038.stdout:8/429: dread - d1/d7/d22/f95 zero size 2024-02-15T18:34:54.514 INFO:tasks.workunit.client.0.smithi038.stdout:8/430: write d1/d7/d15/d6a/f6b [701856,25922] 0 2024-02-15T18:34:54.514 INFO:tasks.workunit.client.0.smithi038.stdout:4/291: rmdir d11/d1e/d20/d2b/d35 39 2024-02-15T18:34:54.515 INFO:tasks.workunit.client.0.smithi038.stdout:6/265: creat d2/d10/f4e x:0 0 0 2024-02-15T18:34:54.515 INFO:tasks.workunit.client.0.smithi038.stdout:6/266: fdatasync d2/d4/d19/f47 0 2024-02-15T18:34:54.518 INFO:tasks.workunit.client.0.smithi038.stdout:9/247: rename d1/da/d10/d19/c25 to d1/da/c5f 0 2024-02-15T18:34:54.519 INFO:tasks.workunit.client.0.smithi038.stdout:2/264: getdents dd/d12/d1d/d3b/d3c 0 2024-02-15T18:34:54.521 INFO:tasks.workunit.client.0.smithi038.stdout:8/431: rmdir d1/d4/d10 39 2024-02-15T18:34:54.523 INFO:tasks.workunit.client.0.smithi038.stdout:9/248: link d1/f14 d1/dc/d2b/f60 0 2024-02-15T18:34:54.524 INFO:tasks.workunit.client.0.smithi038.stdout:2/265: mkdir dd/d12/d18/d1f/d4f/d58 0 2024-02-15T18:34:54.525 INFO:tasks.workunit.client.0.smithi038.stdout:9/249: creat d1/da/d10/d13/f61 x:0 0 0 2024-02-15T18:34:54.527 INFO:tasks.workunit.client.0.smithi038.stdout:9/250: creat d1/dc/d4f/f62 x:0 0 0 2024-02-15T18:34:54.529 INFO:tasks.workunit.client.0.smithi038.stdout:8/432: dread d1/d3a/d58/f60 [0,4194304] 0 2024-02-15T18:34:54.529 INFO:tasks.workunit.client.0.smithi038.stdout:8/433: truncate d1/d7/d22/f5c 675383 0 2024-02-15T18:34:54.529 INFO:tasks.workunit.client.0.smithi038.stdout:9/251: symlink d1/dc/d38/l63 0 2024-02-15T18:34:54.529 INFO:tasks.workunit.client.0.smithi038.stdout:9/252: read - d1/da/d10/d24/d49/f4e zero size 2024-02-15T18:34:54.530 INFO:tasks.workunit.client.0.smithi038.stdout:9/253: mknod d1/da/d10/d13/d3d/d43/c64 0 2024-02-15T18:34:54.530 INFO:tasks.workunit.client.0.smithi038.stdout:1/290: dwrite d2/f42 [0,4194304] 0 2024-02-15T18:34:54.533 INFO:tasks.workunit.client.0.smithi038.stdout:9/254: creat d1/da/d20/f65 x:0 0 0 2024-02-15T18:34:54.541 INFO:tasks.workunit.client.0.smithi038.stdout:9/255: dread d1/da/d36/f37 [0,4194304] 0 2024-02-15T18:34:54.542 INFO:tasks.workunit.client.0.smithi038.stdout:1/291: rename d2/d10/d19/d2f/c5d to d2/d10/d19/c6e 0 2024-02-15T18:34:54.543 INFO:tasks.workunit.client.0.smithi038.stdout:9/256: mkdir d1/da/d20/d58/d66 0 2024-02-15T18:34:54.544 INFO:tasks.workunit.client.0.smithi038.stdout:1/292: rename d2/d10/d19/d22/l25 to d2/d28/d3c/l6f 0 2024-02-15T18:34:54.545 INFO:tasks.workunit.client.0.smithi038.stdout:9/257: getdents d1/da/d10/d24/d49 0 2024-02-15T18:34:54.546 INFO:tasks.workunit.client.0.smithi038.stdout:1/293: rename d2/d10/d19/d22/d36/d4f/f55 to d2/d10/d19/d5f/f70 0 2024-02-15T18:34:54.548 INFO:tasks.workunit.client.0.smithi038.stdout:9/258: rename d1/da/d10/d24/d49/c3e to d1/da/d10/d13/d3d/d43/d4d/c67 0 2024-02-15T18:34:54.548 INFO:tasks.workunit.client.0.smithi038.stdout:9/259: chown d1/da/d10/d13/d3d/d43/c64 41149319 1 2024-02-15T18:34:54.549 INFO:tasks.workunit.client.0.smithi038.stdout:1/294: rename d2/d28/l46 to d2/d10/d19/l71 0 2024-02-15T18:34:54.550 INFO:tasks.workunit.client.0.smithi038.stdout:1/295: creat d2/d10/d19/f72 x:0 0 0 2024-02-15T18:34:54.551 INFO:tasks.workunit.client.0.smithi038.stdout:1/296: creat d2/f73 x:0 0 0 2024-02-15T18:34:54.552 INFO:tasks.workunit.client.0.smithi038.stdout:1/297: link d2/d10/c24 d2/d10/d32/d53/c74 0 2024-02-15T18:34:54.554 INFO:tasks.workunit.client.0.smithi038.stdout:1/298: write d2/d10/d19/d22/d36/f44 [2153217,7273] 0 2024-02-15T18:34:54.554 INFO:tasks.workunit.client.0.smithi038.stdout:4/292: dwrite d11/d1e/d20/d24/f2d [4194304,4194304] 0 2024-02-15T18:34:54.554 INFO:tasks.workunit.client.0.smithi038.stdout:6/267: dwrite d2/d4/d2c/f3f [0,4194304] 0 2024-02-15T18:34:54.555 INFO:tasks.workunit.client.0.smithi038.stdout:2/266: dwrite dd/d12/d1d/f2e [4194304,4194304] 0 2024-02-15T18:34:54.555 INFO:tasks.workunit.client.0.smithi038.stdout:2/267: write dd/d12/d18/d1f/f27 [2858521,33777] 0 2024-02-15T18:34:54.555 INFO:tasks.workunit.client.0.smithi038.stdout:2/268: stat dd/d12/d3a/l3e 0 2024-02-15T18:34:54.558 INFO:tasks.workunit.client.0.smithi038.stdout:4/293: dread d11/f1d [0,4194304] 0 2024-02-15T18:34:54.558 INFO:tasks.workunit.client.0.smithi038.stdout:4/294: chown d11/d1e/d3e/d57 12384375 1 2024-02-15T18:34:54.561 INFO:tasks.workunit.client.0.smithi038.stdout:1/299: rename d2/c52 to d2/d10/d62/c75 0 2024-02-15T18:34:54.563 INFO:tasks.workunit.client.0.smithi038.stdout:2/269: write dd/d12/d1d/f31 [1970905,97006] 0 2024-02-15T18:34:54.564 INFO:tasks.workunit.client.0.smithi038.stdout:9/260: dwrite d1/fb [0,4194304] 0 2024-02-15T18:34:54.564 INFO:tasks.workunit.client.0.smithi038.stdout:9/261: write d1/da/d10/d13/d3d/f5c [877302,14412] 0 2024-02-15T18:34:54.564 INFO:tasks.workunit.client.0.smithi038.stdout:9/262: write d1/f16 [803935,8802] 0 2024-02-15T18:34:54.566 INFO:tasks.workunit.client.0.smithi038.stdout:4/295: unlink d11/f14 0 2024-02-15T18:34:54.567 INFO:tasks.workunit.client.0.smithi038.stdout:9/263: mknod d1/c68 0 2024-02-15T18:34:54.568 INFO:tasks.workunit.client.0.smithi038.stdout:4/296: dread d11/d1e/d20/d2f/f3d [0,4194304] 0 2024-02-15T18:34:54.570 INFO:tasks.workunit.client.0.smithi038.stdout:9/264: mkdir d1/dc/d69 0 2024-02-15T18:34:54.570 INFO:tasks.workunit.client.0.smithi038.stdout:9/265: fsync d1/dc/d4f/f62 0 2024-02-15T18:34:54.570 INFO:tasks.workunit.client.0.smithi038.stdout:4/297: creat d11/d1e/d43/d5e/d62/f69 x:0 0 0 2024-02-15T18:34:54.571 INFO:tasks.workunit.client.0.smithi038.stdout:1/300: dread d2/f3 [0,4194304] 0 2024-02-15T18:34:54.571 INFO:tasks.workunit.client.0.smithi038.stdout:9/266: symlink d1/da/d36/d3f/l6a 0 2024-02-15T18:34:54.571 INFO:tasks.workunit.client.0.smithi038.stdout:6/268: dwrite d2/d4/f4a [0,4194304] 0 2024-02-15T18:34:54.572 INFO:tasks.workunit.client.0.smithi038.stdout:4/298: symlink d11/d1e/d20/d2b/d4d/l6a 0 2024-02-15T18:34:54.573 INFO:tasks.workunit.client.0.smithi038.stdout:4/299: write d11/d1e/f28 [456359,39917] 0 2024-02-15T18:34:54.576 INFO:tasks.workunit.client.0.smithi038.stdout:1/301: symlink d2/d10/d19/d22/d36/d4f/l76 0 2024-02-15T18:34:54.579 INFO:tasks.workunit.client.0.smithi038.stdout:9/267: creat d1/dc/d38/f6b x:0 0 0 2024-02-15T18:34:54.579 INFO:tasks.workunit.client.0.smithi038.stdout:9/268: write d1/dc/d38/f6b [588856,2055] 0 2024-02-15T18:34:54.579 INFO:tasks.workunit.client.0.smithi038.stdout:2/270: dread dd/d12/f26 [0,4194304] 0 2024-02-15T18:34:54.585 INFO:tasks.workunit.client.0.smithi038.stdout:9/269: dread d1/dc/d2b/f3a [4194304,4194304] 0 2024-02-15T18:34:54.585 INFO:tasks.workunit.client.0.smithi038.stdout:9/270: dread - d1/dc/d2b/f34 zero size 2024-02-15T18:34:54.592 INFO:tasks.workunit.client.0.smithi038.stdout:1/302: dwrite d2/d10/d19/d22/f57 [0,4194304] 0 2024-02-15T18:34:54.592 INFO:tasks.workunit.client.0.smithi038.stdout:1/303: chown d2/d28/d3c/c5a 340930 1 2024-02-15T18:34:54.593 INFO:tasks.workunit.client.0.smithi038.stdout:4/300: creat d11/d1e/d20/f6b x:0 0 0 2024-02-15T18:34:54.593 INFO:tasks.workunit.client.0.smithi038.stdout:4/301: readlink le 0 2024-02-15T18:34:54.594 INFO:tasks.workunit.client.0.smithi038.stdout:0/392: sync 2024-02-15T18:34:54.594 INFO:tasks.workunit.client.0.smithi038.stdout:3/240: sync 2024-02-15T18:34:54.594 INFO:tasks.workunit.client.0.smithi038.stdout:0/393: read d4/d13/d2a/d35/d1e/f50 [822397,54718] 0 2024-02-15T18:34:54.595 INFO:tasks.workunit.client.0.smithi038.stdout:9/271: mkdir d1/da/d10/d13/d3d/d6c 0 2024-02-15T18:34:54.596 INFO:tasks.workunit.client.0.smithi038.stdout:6/269: rename d2/d4/d2c/f3f to d2/d4/d2c/f4f 0 2024-02-15T18:34:54.598 INFO:tasks.workunit.client.0.smithi038.stdout:1/304: mknod d2/d10/d62/c77 0 2024-02-15T18:34:54.599 INFO:tasks.workunit.client.0.smithi038.stdout:4/302: mkdir d11/d1e/d6c 0 2024-02-15T18:34:54.600 INFO:tasks.workunit.client.0.smithi038.stdout:3/241: symlink d2/d1b/d3c/d41/l4d 0 2024-02-15T18:34:54.603 INFO:tasks.workunit.client.0.smithi038.stdout:6/270: rename d2/l44 to d2/l50 0 2024-02-15T18:34:54.604 INFO:tasks.workunit.client.0.smithi038.stdout:1/305: creat d2/d10/d19/d22/d36/f78 x:0 0 0 2024-02-15T18:34:54.604 INFO:tasks.workunit.client.0.smithi038.stdout:1/306: dread - d2/d10/f37 zero size 2024-02-15T18:34:54.606 INFO:tasks.workunit.client.0.smithi038.stdout:4/303: creat d11/d1e/d20/d2b/d65/f6d x:0 0 0 2024-02-15T18:34:54.606 INFO:tasks.workunit.client.0.smithi038.stdout:4/304: chown ld 26 1 2024-02-15T18:34:54.608 INFO:tasks.workunit.client.0.smithi038.stdout:3/242: rename d2/d1b/d3c/d46 to d2/d1b/d3c/d41/d4e 0 2024-02-15T18:34:54.608 INFO:tasks.workunit.client.0.smithi038.stdout:4/305: mknod d11/d1e/d20/d31/d48/d60/c6e 0 2024-02-15T18:34:54.610 INFO:tasks.workunit.client.0.smithi038.stdout:3/243: mkdir d2/d1b/d4f 0 2024-02-15T18:34:54.611 INFO:tasks.workunit.client.0.smithi038.stdout:3/244: write d2/d1b/f4a [824604,49599] 0 2024-02-15T18:34:54.616 INFO:tasks.workunit.client.0.smithi038.stdout:3/245: creat d2/f50 x:0 0 0 2024-02-15T18:34:54.618 INFO:tasks.workunit.client.0.smithi038.stdout:3/246: creat d2/f51 x:0 0 0 2024-02-15T18:34:54.618 INFO:tasks.workunit.client.0.smithi038.stdout:3/247: chown d2/f50 218094 1 2024-02-15T18:34:54.618 INFO:tasks.workunit.client.0.smithi038.stdout:3/248: chown d2/d1b/d3c/d41/d4e 221791826 1 2024-02-15T18:34:54.618 INFO:tasks.workunit.client.0.smithi038.stdout:3/249: chown d2/d1b/f4a 16936290 1 2024-02-15T18:34:54.619 INFO:tasks.workunit.client.0.smithi038.stdout:3/250: write d2/d1b/d3c/f31 [3201783,73818] 0 2024-02-15T18:34:54.619 INFO:tasks.workunit.client.0.smithi038.stdout:3/251: fdatasync d2/f1a 0 2024-02-15T18:34:54.619 INFO:tasks.workunit.client.0.smithi038.stdout:3/252: chown d2/fa 696567595 1 2024-02-15T18:34:54.621 INFO:tasks.workunit.client.0.smithi038.stdout:3/253: rename d2/d1e/c1f to d2/d1e/c52 0 2024-02-15T18:34:54.621 INFO:tasks.workunit.client.0.smithi038.stdout:3/254: fdatasync d2/f38 0 2024-02-15T18:34:54.621 INFO:tasks.workunit.client.0.smithi038.stdout:3/255: truncate d2/f51 1030215 0 2024-02-15T18:34:54.621 INFO:tasks.workunit.client.0.smithi038.stdout:3/256: chown d2/c33 3 1 2024-02-15T18:34:54.622 INFO:tasks.workunit.client.0.smithi038.stdout:3/257: write d2/f9 [2485810,19459] 0 2024-02-15T18:34:54.642 INFO:tasks.workunit.client.0.smithi038.stdout:6/271: dwrite d2/d4/fc [4194304,4194304] 0 2024-02-15T18:34:54.642 INFO:tasks.workunit.client.0.smithi038.stdout:6/272: stat d2/d10/c14 0 2024-02-15T18:34:54.643 INFO:tasks.workunit.client.0.smithi038.stdout:2/271: dwrite dd/d12/f3d [0,4194304] 0 2024-02-15T18:34:54.643 INFO:tasks.workunit.client.0.smithi038.stdout:0/394: dwrite d4/d13/d2a/d35/d1e/d32/f6c [0,4194304] 0 2024-02-15T18:34:54.643 INFO:tasks.workunit.client.0.smithi038.stdout:0/395: read - d4/d13/d2a/d35/d1e/f5d zero size 2024-02-15T18:34:54.644 INFO:tasks.workunit.client.0.smithi038.stdout:4/306: dwrite d11/f1d [0,4194304] 0 2024-02-15T18:34:54.644 INFO:tasks.workunit.client.0.smithi038.stdout:2/272: write dd/d12/d1d/d3b/d3c/f48 [165578,46120] 0 2024-02-15T18:34:54.645 INFO:tasks.workunit.client.0.smithi038.stdout:1/307: dwrite d2/d69/f6c [0,4194304] 0 2024-02-15T18:34:54.647 INFO:tasks.workunit.client.0.smithi038.stdout:3/258: getdents d2/d1b/d3c 0 2024-02-15T18:34:54.649 INFO:tasks.workunit.client.0.smithi038.stdout:3/259: dread d2/f2f [0,4194304] 0 2024-02-15T18:34:54.651 INFO:tasks.workunit.client.0.smithi038.stdout:4/307: dread d11/d1e/d20/d24/f46 [0,4194304] 0 2024-02-15T18:34:54.651 INFO:tasks.workunit.client.0.smithi038.stdout:4/308: readlink d11/d1e/d20/d31/d48/l5d 0 2024-02-15T18:34:54.654 INFO:tasks.workunit.client.0.smithi038.stdout:2/273: dread fb [0,4194304] 0 2024-02-15T18:34:54.655 INFO:tasks.workunit.client.0.smithi038.stdout:0/396: creat d4/d13/d2a/d35/d1e/d32/d79/d7d/f86 x:0 0 0 2024-02-15T18:34:54.658 INFO:tasks.workunit.client.0.smithi038.stdout:1/308: symlink d2/d10/d19/d5f/l79 0 2024-02-15T18:34:54.658 INFO:tasks.workunit.client.0.smithi038.stdout:3/260: symlink d2/d1b/d3c/l53 0 2024-02-15T18:34:54.659 INFO:tasks.workunit.client.0.smithi038.stdout:4/309: mkdir d11/d1e/d20/d24/d40/d6f 0 2024-02-15T18:34:54.661 INFO:tasks.workunit.client.0.smithi038.stdout:0/397: symlink d4/d13/d2a/d80/l87 0 2024-02-15T18:34:54.661 INFO:tasks.workunit.client.0.smithi038.stdout:0/398: chown d4/d13/d19/c2c 1 1 2024-02-15T18:34:54.661 INFO:tasks.workunit.client.0.smithi038.stdout:0/399: truncate d4/d13/d2a/f2d 571099 0 2024-02-15T18:34:54.661 INFO:tasks.workunit.client.0.smithi038.stdout:1/309: rename d2/f42 to d2/d10/d19/d22/f7a 0 2024-02-15T18:34:54.662 INFO:tasks.workunit.client.0.smithi038.stdout:4/310: creat d11/d1e/d20/f70 x:0 0 0 2024-02-15T18:34:54.662 INFO:tasks.workunit.client.0.smithi038.stdout:1/310: write d2/f2d [3846095,128673] 0 2024-02-15T18:34:54.662 INFO:tasks.workunit.client.0.smithi038.stdout:1/311: write d2/f3 [4365429,115769] 0 2024-02-15T18:34:54.663 INFO:tasks.workunit.client.0.smithi038.stdout:2/274: rename dd/d12/d18 to dd/d12/d59 0 2024-02-15T18:34:54.663 INFO:tasks.workunit.client.0.smithi038.stdout:2/275: chown dd/d12/f47 53349 1 2024-02-15T18:34:54.664 INFO:tasks.workunit.client.0.smithi038.stdout:6/273: dwrite d2/d10/f49 [0,4194304] 0 2024-02-15T18:34:54.664 INFO:tasks.workunit.client.0.smithi038.stdout:6/274: fdatasync d2/d4/ff 0 2024-02-15T18:34:54.665 INFO:tasks.workunit.client.0.smithi038.stdout:0/400: rename d4/f7c to d4/f88 0 2024-02-15T18:34:54.667 INFO:tasks.workunit.client.0.smithi038.stdout:4/311: mkdir d11/d1e/d20/d2f/d71 0 2024-02-15T18:34:54.669 INFO:tasks.workunit.client.0.smithi038.stdout:1/312: rename d2/f3 to d2/d10/d32/d53/f7b 0 2024-02-15T18:34:54.670 INFO:tasks.workunit.client.0.smithi038.stdout:5/269: sync 2024-02-15T18:34:54.670 INFO:tasks.workunit.client.0.smithi038.stdout:7/301: sync 2024-02-15T18:34:54.672 INFO:tasks.workunit.client.0.smithi038.stdout:2/276: rmdir dd/d12/d13/d54 39 2024-02-15T18:34:54.678 INFO:tasks.workunit.client.0.smithi038.stdout:1/313: link d2/d10/d32/d53/c5c d2/d39/c7c 0 2024-02-15T18:34:54.679 INFO:tasks.workunit.client.0.smithi038.stdout:7/302: symlink d0/d1b/d30/d62/l6c 0 2024-02-15T18:34:54.679 INFO:tasks.workunit.client.0.smithi038.stdout:7/303: chown d0/db/fc 1665 1 2024-02-15T18:34:54.679 INFO:tasks.workunit.client.0.smithi038.stdout:2/277: mkdir dd/d12/d59/d1f/d4f/d5a 0 2024-02-15T18:34:54.682 INFO:tasks.workunit.client.0.smithi038.stdout:5/270: write d2/d3/d12/f14 [359719,76312] 0 2024-02-15T18:34:54.682 INFO:tasks.workunit.client.0.smithi038.stdout:3/261: dwrite d2/d1b/f36 [0,4194304] 0 2024-02-15T18:34:54.682 INFO:tasks.workunit.client.0.smithi038.stdout:5/271: fsync d2/d3/f4b 0 2024-02-15T18:34:54.682 INFO:tasks.workunit.client.0.smithi038.stdout:5/272: stat d2/d30/l32 0 2024-02-15T18:34:54.682 INFO:tasks.workunit.client.0.smithi038.stdout:3/262: write d2/f51 [1372418,84600] 0 2024-02-15T18:34:54.682 INFO:tasks.workunit.client.0.smithi038.stdout:3/263: dread - d2/f50 zero size 2024-02-15T18:34:54.682 INFO:tasks.workunit.client.0.smithi038.stdout:3/264: fdatasync d2/f17 0 2024-02-15T18:34:54.685 INFO:tasks.workunit.client.0.smithi038.stdout:5/273: mknod d2/d20/d33/c51 0 2024-02-15T18:34:54.686 INFO:tasks.workunit.client.0.smithi038.stdout:3/265: mknod d2/d1b/d4f/c54 0 2024-02-15T18:34:54.686 INFO:tasks.workunit.client.0.smithi038.stdout:3/266: chown d2/f9 3607 1 2024-02-15T18:34:54.689 INFO:tasks.workunit.client.0.smithi038.stdout:5/274: mkdir d2/d30/d52 0 2024-02-15T18:34:54.690 INFO:tasks.workunit.client.0.smithi038.stdout:3/267: mkdir d2/d1b/d3c/d3a/d55 0 2024-02-15T18:34:54.692 INFO:tasks.workunit.client.0.smithi038.stdout:5/275: link d2/c13 d2/c53 0 2024-02-15T18:34:54.692 INFO:tasks.workunit.client.0.smithi038.stdout:5/276: fsync d2/d3/f4b 0 2024-02-15T18:34:54.692 INFO:tasks.workunit.client.0.smithi038.stdout:5/277: write d2/f39 [831452,36175] 0 2024-02-15T18:34:54.693 INFO:tasks.workunit.client.0.smithi038.stdout:5/278: mknod d2/d20/c54 0 2024-02-15T18:34:54.694 INFO:tasks.workunit.client.0.smithi038.stdout:5/279: dread - d2/d20/f50 zero size 2024-02-15T18:34:54.694 INFO:tasks.workunit.client.0.smithi038.stdout:5/280: dread - d2/d3/f4e zero size 2024-02-15T18:34:54.698 INFO:tasks.workunit.client.0.smithi038.stdout:8/434: sync 2024-02-15T18:34:54.702 INFO:tasks.workunit.client.0.smithi038.stdout:8/435: symlink d1/d4/d5/d6/d6d/l9d 0 2024-02-15T18:34:54.704 INFO:tasks.workunit.client.0.smithi038.stdout:8/436: creat d1/d4/f9e x:0 0 0 2024-02-15T18:34:54.704 INFO:tasks.workunit.client.0.smithi038.stdout:8/437: truncate d1/d7/d22/f5c 1580525 0 2024-02-15T18:34:54.704 INFO:tasks.workunit.client.0.smithi038.stdout:8/438: write d1/d4/d5/d31/f7f [707240,86617] 0 2024-02-15T18:34:54.706 INFO:tasks.workunit.client.0.smithi038.stdout:6/275: dwrite d2/d4/d2c/f37 [0,4194304] 0 2024-02-15T18:34:54.706 INFO:tasks.workunit.client.0.smithi038.stdout:8/439: link d1/d7/fc d1/d4/d5/d77/f9f 0 2024-02-15T18:34:54.706 INFO:tasks.workunit.client.0.smithi038.stdout:8/440: chown d1/f40 242 1 2024-02-15T18:34:54.708 INFO:tasks.workunit.client.0.smithi038.stdout:6/276: creat d2/d4/d2c/f51 x:0 0 0 2024-02-15T18:34:54.710 INFO:tasks.workunit.client.0.smithi038.stdout:8/441: creat d1/d4/d5/d31/d65/d80/fa0 x:0 0 0 2024-02-15T18:34:54.711 INFO:tasks.workunit.client.0.smithi038.stdout:6/277: unlink d2/d4/d2c/f4b 0 2024-02-15T18:34:54.717 INFO:tasks.workunit.client.0.smithi038.stdout:6/278: rename d2/d4/d2c/d39/f45 to d2/d4/d2c/f52 0 2024-02-15T18:34:54.717 INFO:tasks.workunit.client.0.smithi038.stdout:6/279: chown d2/d10/c14 914 1 2024-02-15T18:34:54.717 INFO:tasks.workunit.client.0.smithi038.stdout:6/280: fdatasync d2/f46 0 2024-02-15T18:34:54.722 INFO:tasks.workunit.client.0.smithi038.stdout:4/312: dwrite d11/f1a [0,4194304] 0 2024-02-15T18:34:54.722 INFO:tasks.workunit.client.0.smithi038.stdout:1/314: dwrite d2/d28/f50 [0,4194304] 0 2024-02-15T18:34:54.725 INFO:tasks.workunit.client.0.smithi038.stdout:6/281: read d2/f1a [3112576,24887] 0 2024-02-15T18:34:54.725 INFO:tasks.workunit.client.0.smithi038.stdout:4/313: rename d11/d1e/d3e/l44 to d11/d1e/l72 0 2024-02-15T18:34:54.727 INFO:tasks.workunit.client.0.smithi038.stdout:8/442: dwrite d1/d4/d5/d6/ff [4194304,4194304] 0 2024-02-15T18:34:54.727 INFO:tasks.workunit.client.0.smithi038.stdout:8/443: write d1/d7/d26/f84 [623885,129625] 0 2024-02-15T18:34:54.727 INFO:tasks.workunit.client.0.smithi038.stdout:8/444: fdatasync d1/d4/d5/d31/d65/d80/fa0 0 2024-02-15T18:34:54.727 INFO:tasks.workunit.client.0.smithi038.stdout:8/445: read d1/d4/d5/d77/f9f [228224,84897] 0 2024-02-15T18:34:54.728 INFO:tasks.workunit.client.0.smithi038.stdout:8/446: read d1/f50 [709596,2552] 0 2024-02-15T18:34:54.729 INFO:tasks.workunit.client.0.smithi038.stdout:7/304: dwrite d0/d2e/f3b [0,4194304] 0 2024-02-15T18:34:54.730 INFO:tasks.workunit.client.0.smithi038.stdout:9/272: sync 2024-02-15T18:34:54.731 INFO:tasks.workunit.client.0.smithi038.stdout:6/282: creat d2/d4/d19/f53 x:0 0 0 2024-02-15T18:34:54.731 INFO:tasks.workunit.client.0.smithi038.stdout:3/268: dwrite d2/d1e/f32 [0,4194304] 0 2024-02-15T18:34:54.731 INFO:tasks.workunit.client.0.smithi038.stdout:2/278: dwrite dd/d12/d59/f1c [0,4194304] 0 2024-02-15T18:34:54.732 INFO:tasks.workunit.client.0.smithi038.stdout:4/314: mkdir d11/d1e/d20/d2b/d4d/d67/d73 0 2024-02-15T18:34:54.737 INFO:tasks.workunit.client.0.smithi038.stdout:5/281: rmdir d2/d20 39 2024-02-15T18:34:54.737 INFO:tasks.workunit.client.0.smithi038.stdout:7/305: dread d0/d66/f35 [0,4194304] 0 2024-02-15T18:34:54.737 INFO:tasks.workunit.client.0.smithi038.stdout:5/282: chown d2/d3/d12/c22 44666526 1 2024-02-15T18:34:54.737 INFO:tasks.workunit.client.0.smithi038.stdout:9/273: creat d1/da/d20/f6d x:0 0 0 2024-02-15T18:34:54.737 INFO:tasks.workunit.client.0.smithi038.stdout:6/283: mknod d2/d4/d28/c54 0 2024-02-15T18:34:54.744 INFO:tasks.workunit.client.0.smithi038.stdout:1/315: dwrite f1 [0,4194304] 0 2024-02-15T18:34:54.744 INFO:tasks.workunit.client.0.smithi038.stdout:1/316: chown d2/f4 8491119 1 2024-02-15T18:34:54.745 INFO:tasks.workunit.client.0.smithi038.stdout:5/283: rmdir d2/d30 39 2024-02-15T18:34:54.745 INFO:tasks.workunit.client.0.smithi038.stdout:6/284: dread d2/d10/f1d [0,4194304] 0 2024-02-15T18:34:54.747 INFO:tasks.workunit.client.0.smithi038.stdout:8/447: getdents d1/d4/d5/d77 0 2024-02-15T18:34:54.750 INFO:tasks.workunit.client.0.smithi038.stdout:7/306: rename d0/db/d59/d15/f2b to d0/db/d19/f6d 0 2024-02-15T18:34:54.752 INFO:tasks.workunit.client.0.smithi038.stdout:2/279: dread dd/d12/d1d/f31 [0,4194304] 0 2024-02-15T18:34:54.752 INFO:tasks.workunit.client.0.smithi038.stdout:2/280: chown dd/d12/d13/d54/c57 6820699 1 2024-02-15T18:34:54.752 INFO:tasks.workunit.client.0.smithi038.stdout:5/284: write d2/d3/ff [672094,110579] 0 2024-02-15T18:34:54.754 INFO:tasks.workunit.client.0.smithi038.stdout:1/317: unlink d2/d10/d32/l3b 0 2024-02-15T18:34:54.756 INFO:tasks.workunit.client.0.smithi038.stdout:6/285: mkdir d2/d4/d19/d55 0 2024-02-15T18:34:54.756 INFO:tasks.workunit.client.0.smithi038.stdout:6/286: readlink d2/l35 0 2024-02-15T18:34:54.756 INFO:tasks.workunit.client.0.smithi038.stdout:6/287: chown d2/d10/c21 93 1 2024-02-15T18:34:54.756 INFO:tasks.workunit.client.0.smithi038.stdout:4/315: write d11/d1e/d43/d4a/f59 [2509606,36054] 0 2024-02-15T18:34:54.757 INFO:tasks.workunit.client.0.smithi038.stdout:8/448: symlink d1/d7/d26/d69/la1 0 2024-02-15T18:34:54.757 INFO:tasks.workunit.client.0.smithi038.stdout:4/316: write d11/d1e/f29 [2266280,38760] 0 2024-02-15T18:34:54.757 INFO:tasks.workunit.client.0.smithi038.stdout:3/269: dwrite d2/d24/f2e [0,4194304] 0 2024-02-15T18:34:54.758 INFO:tasks.workunit.client.0.smithi038.stdout:7/307: creat d0/d1b/d30/d62/f6e x:0 0 0 2024-02-15T18:34:54.758 INFO:tasks.workunit.client.0.smithi038.stdout:9/274: dwrite d1/da/f54 [0,4194304] 0 2024-02-15T18:34:54.759 INFO:tasks.workunit.client.0.smithi038.stdout:2/281: symlink dd/d12/d59/d1f/d4f/d58/l5b 0 2024-02-15T18:34:54.761 INFO:tasks.workunit.client.0.smithi038.stdout:0/401: sync 2024-02-15T18:34:54.761 INFO:tasks.workunit.client.0.smithi038.stdout:0/402: read d4/d13/d2a/d35/d1e/f50 [1229733,115433] 0 2024-02-15T18:34:54.761 INFO:tasks.workunit.client.0.smithi038.stdout:0/403: fdatasync d4/d13/d2a/d35/d1e/f5d 0 2024-02-15T18:34:54.762 INFO:tasks.workunit.client.0.smithi038.stdout:6/288: dread d2/f9 [0,4194304] 0 2024-02-15T18:34:54.762 INFO:tasks.workunit.client.0.smithi038.stdout:6/289: write d2/f46 [102043,94775] 0 2024-02-15T18:34:54.762 INFO:tasks.workunit.client.0.smithi038.stdout:8/449: link d1/d4/d10/d43/f70 d1/d4/d10/d43/d48/fa2 0 2024-02-15T18:34:54.763 INFO:tasks.workunit.client.0.smithi038.stdout:8/450: chown d1/d4/d5/d31/d65/f76 3 1 2024-02-15T18:34:54.763 INFO:tasks.workunit.client.0.smithi038.stdout:8/451: chown d1/d7/d22/f5a 10780 1 2024-02-15T18:34:54.763 INFO:tasks.workunit.client.0.smithi038.stdout:4/317: mkdir d11/d74 0 2024-02-15T18:34:54.767 INFO:tasks.workunit.client.0.smithi038.stdout:7/308: unlink d0/fa 0 2024-02-15T18:34:54.771 INFO:tasks.workunit.client.0.smithi038.stdout:9/275: mknod d1/da/d36/d50/c6e 0 2024-02-15T18:34:54.771 INFO:tasks.workunit.client.0.smithi038.stdout:2/282: mknod dd/c5c 0 2024-02-15T18:34:54.774 INFO:tasks.workunit.client.0.smithi038.stdout:0/404: rename d4/c6e to d4/d13/d2a/d35/d1e/d32/d48/c89 0 2024-02-15T18:34:54.780 INFO:tasks.workunit.client.0.smithi038.stdout:8/452: symlink d1/d3a/d58/la3 0 2024-02-15T18:34:54.780 INFO:tasks.workunit.client.0.smithi038.stdout:7/309: symlink d0/d1b/d30/d5d/d61/l6f 0 2024-02-15T18:34:54.781 INFO:tasks.workunit.client.0.smithi038.stdout:9/276: creat d1/dc/d38/f6f x:0 0 0 2024-02-15T18:34:54.781 INFO:tasks.workunit.client.0.smithi038.stdout:1/318: dwrite d2/d28/d3c/f63 [0,4194304] 0 2024-02-15T18:34:54.782 INFO:tasks.workunit.client.0.smithi038.stdout:4/318: rename d11/d1e/d20/d24 to d11/d1e/d3e/d75 0 2024-02-15T18:34:54.782 INFO:tasks.workunit.client.0.smithi038.stdout:0/405: truncate d4/d13/d2a/d35/d1e/f4d 759617 0 2024-02-15T18:34:54.785 INFO:tasks.workunit.client.0.smithi038.stdout:5/285: dwrite d2/d20/f3b [0,4194304] 0 2024-02-15T18:34:54.785 INFO:tasks.workunit.client.0.smithi038.stdout:5/286: dread - d2/d3/f4e zero size 2024-02-15T18:34:54.785 INFO:tasks.workunit.client.0.smithi038.stdout:5/287: fsync d2/d20/f3b 0 2024-02-15T18:34:54.785 INFO:tasks.workunit.client.0.smithi038.stdout:8/453: creat d1/d4/d10/d91/fa4 x:0 0 0 2024-02-15T18:34:54.785 INFO:tasks.workunit.client.0.smithi038.stdout:7/310: mkdir d0/d1b/d30/d5d/d70 0 2024-02-15T18:34:54.787 INFO:tasks.workunit.client.0.smithi038.stdout:9/277: creat d1/da/d10/d13/d3d/d43/f70 x:0 0 0 2024-02-15T18:34:54.789 INFO:tasks.workunit.client.0.smithi038.stdout:5/288: write d2/d3/f7 [1268482,55312] 0 2024-02-15T18:34:54.789 INFO:tasks.workunit.client.0.smithi038.stdout:6/290: dwrite d2/d4/f1e [0,4194304] 0 2024-02-15T18:34:54.789 INFO:tasks.workunit.client.0.smithi038.stdout:6/291: readlink d2/l35 0 2024-02-15T18:34:54.790 INFO:tasks.workunit.client.0.smithi038.stdout:3/270: dwrite d2/f38 [0,4194304] 0 2024-02-15T18:34:54.790 INFO:tasks.workunit.client.0.smithi038.stdout:1/319: rmdir d2/d10/d19/d22 39 2024-02-15T18:34:54.791 INFO:tasks.workunit.client.0.smithi038.stdout:5/289: fdatasync d2/d3/ff 0 2024-02-15T18:34:54.796 INFO:tasks.workunit.client.0.smithi038.stdout:4/319: creat d11/d1e/d20/f76 x:0 0 0 2024-02-15T18:34:54.799 INFO:tasks.workunit.client.0.smithi038.stdout:7/311: creat d0/db/d59/d15/f71 x:0 0 0 2024-02-15T18:34:54.801 INFO:tasks.workunit.client.0.smithi038.stdout:2/283: truncate fb 2483545 0 2024-02-15T18:34:54.801 INFO:tasks.workunit.client.0.smithi038.stdout:2/284: chown dd/d12/d59/d1f/c34 32617918 1 2024-02-15T18:34:54.801 INFO:tasks.workunit.client.0.smithi038.stdout:2/285: readlink dd/lf 0 2024-02-15T18:34:54.801 INFO:tasks.workunit.client.0.smithi038.stdout:2/286: write dd/d12/d1d/d3b/f4b [118700,8860] 0 2024-02-15T18:34:54.802 INFO:tasks.workunit.client.0.smithi038.stdout:4/320: dread d11/d1e/f29 [0,4194304] 0 2024-02-15T18:34:54.803 INFO:tasks.workunit.client.0.smithi038.stdout:9/278: link d1/da/f17 d1/dc/f71 0 2024-02-15T18:34:54.805 INFO:tasks.workunit.client.0.smithi038.stdout:6/292: mkdir d2/d56 0 2024-02-15T18:34:54.808 INFO:tasks.workunit.client.0.smithi038.stdout:8/454: dwrite d1/f2 [0,4194304] 0 2024-02-15T18:34:54.808 INFO:tasks.workunit.client.0.smithi038.stdout:8/455: fsync d1/d3a/d58/f60 0 2024-02-15T18:34:54.808 INFO:tasks.workunit.client.0.smithi038.stdout:3/271: creat d2/d1b/d3c/d41/f56 x:0 0 0 2024-02-15T18:34:54.814 INFO:tasks.workunit.client.0.smithi038.stdout:5/290: dwrite d2/d3/d12/f28 [0,4194304] 0 2024-02-15T18:34:54.814 INFO:tasks.workunit.client.0.smithi038.stdout:7/312: creat d0/d1b/d30/d50/f72 x:0 0 0 2024-02-15T18:34:54.814 INFO:tasks.workunit.client.0.smithi038.stdout:7/313: chown d0/db/d59/c12 107837103 1 2024-02-15T18:34:54.816 INFO:tasks.workunit.client.0.smithi038.stdout:4/321: creat d11/d1e/d3e/d57/f77 x:0 0 0 2024-02-15T18:34:54.816 INFO:tasks.workunit.client.0.smithi038.stdout:4/322: chown d11/d1e/c3b 0 1 2024-02-15T18:34:54.816 INFO:tasks.workunit.client.0.smithi038.stdout:4/323: fsync d11/d1e/d20/f76 0 2024-02-15T18:34:54.817 INFO:tasks.workunit.client.0.smithi038.stdout:9/279: rename d1/f9 to d1/da/d10/d13/d3d/d4b/f72 0 2024-02-15T18:34:54.818 INFO:tasks.workunit.client.0.smithi038.stdout:6/293: mknod d2/d4/d19/c57 0 2024-02-15T18:34:54.818 INFO:tasks.workunit.client.0.smithi038.stdout:6/294: truncate d2/d10/f4e 203371 0 2024-02-15T18:34:54.822 INFO:tasks.workunit.client.0.smithi038.stdout:2/287: dwrite dd/d12/d59/f21 [0,4194304] 0 2024-02-15T18:34:54.822 INFO:tasks.workunit.client.0.smithi038.stdout:8/456: rmdir d1/d7/d22 39 2024-02-15T18:34:54.823 INFO:tasks.workunit.client.0.smithi038.stdout:3/272: rmdir d2/d1b/d3c/d3a 39 2024-02-15T18:34:54.825 INFO:tasks.workunit.client.0.smithi038.stdout:7/314: getdents d0/d1b/d30/d5d 0 2024-02-15T18:34:54.825 INFO:tasks.workunit.client.0.smithi038.stdout:7/315: readlink d0/d66/d6a/l53 0 2024-02-15T18:34:54.828 INFO:tasks.workunit.client.0.smithi038.stdout:5/291: rename d2/d3/f7 to d2/d30/f55 0 2024-02-15T18:34:54.828 INFO:tasks.workunit.client.0.smithi038.stdout:4/324: dwrite d11/d1e/d20/d31/d48/d60/f66 [0,4194304] 0 2024-02-15T18:34:54.829 INFO:tasks.workunit.client.0.smithi038.stdout:9/280: creat d1/da/d36/d50/f73 x:0 0 0 2024-02-15T18:34:54.830 INFO:tasks.workunit.client.0.smithi038.stdout:6/295: truncate d2/f9 2882321 0 2024-02-15T18:34:54.832 INFO:tasks.workunit.client.0.smithi038.stdout:7/316: creat d0/d66/d6a/f73 x:0 0 0 2024-02-15T18:34:54.835 INFO:tasks.workunit.client.0.smithi038.stdout:8/457: rename d1/d7/d15/d4b/f55 to d1/d4/d10/d43/d48/d9a/fa5 0 2024-02-15T18:34:54.835 INFO:tasks.workunit.client.0.smithi038.stdout:3/273: rename d2/d1b/d3c/d41 to d2/d1b/d3c/d41/d4e/d57 22 2024-02-15T18:34:54.836 INFO:tasks.workunit.client.0.smithi038.stdout:4/325: symlink d11/d1e/d3e/d57/l78 0 2024-02-15T18:34:54.838 INFO:tasks.workunit.client.0.smithi038.stdout:9/281: unlink d1/da/d10/d13/l40 0 2024-02-15T18:34:54.838 INFO:tasks.workunit.client.0.smithi038.stdout:6/296: creat d2/f58 x:0 0 0 2024-02-15T18:34:54.840 INFO:tasks.workunit.client.0.smithi038.stdout:7/317: rename d0/d66/f45 to d0/d1b/d30/d50/f74 0 2024-02-15T18:34:54.840 INFO:tasks.workunit.client.0.smithi038.stdout:8/458: creat d1/d7/d15/d6a/fa6 x:0 0 0 2024-02-15T18:34:54.841 INFO:tasks.workunit.client.0.smithi038.stdout:3/274: symlink d2/d1b/d3c/d41/d4e/l58 0 2024-02-15T18:34:54.842 INFO:tasks.workunit.client.0.smithi038.stdout:4/326: symlink d11/d1e/d43/l79 0 2024-02-15T18:34:54.842 INFO:tasks.workunit.client.0.smithi038.stdout:4/327: write d11/d1e/d20/d2b/d65/f6d [847187,34527] 0 2024-02-15T18:34:54.842 INFO:tasks.workunit.client.0.smithi038.stdout:4/328: fdatasync d11/d1e/d3e/d75/f34 0 2024-02-15T18:34:54.842 INFO:tasks.workunit.client.0.smithi038.stdout:4/329: truncate d11/d1e/d20/f76 76771 0 2024-02-15T18:34:54.843 INFO:tasks.workunit.client.0.smithi038.stdout:9/282: creat d1/da/d10/d13/d3d/d43/f74 x:0 0 0 2024-02-15T18:34:54.843 INFO:tasks.workunit.client.0.smithi038.stdout:9/283: fdatasync d1/da/d10/d13/f18 0 2024-02-15T18:34:54.843 INFO:tasks.workunit.client.0.smithi038.stdout:9/284: dread - d1/dc/d4f/f47 zero size 2024-02-15T18:34:54.843 INFO:tasks.workunit.client.0.smithi038.stdout:9/285: chown d1/da/d36/d3f 529899 1 2024-02-15T18:34:54.843 INFO:tasks.workunit.client.0.smithi038.stdout:6/297: rmdir d2/d4/d19 39 2024-02-15T18:34:54.844 INFO:tasks.workunit.client.0.smithi038.stdout:3/275: mknod d2/d1b/c59 0 2024-02-15T18:34:54.844 INFO:tasks.workunit.client.0.smithi038.stdout:3/276: write d2/d1e/f44 [262841,37018] 0 2024-02-15T18:34:54.846 INFO:tasks.workunit.client.0.smithi038.stdout:4/330: mkdir d11/d1e/d20/d31/d7a 0 2024-02-15T18:34:54.846 INFO:tasks.workunit.client.0.smithi038.stdout:4/331: chown d11/d1e/d20/l58 810799233 1 2024-02-15T18:34:54.846 INFO:tasks.workunit.client.0.smithi038.stdout:6/298: write d2/f3 [6558917,67341] 0 2024-02-15T18:34:54.846 INFO:tasks.workunit.client.0.smithi038.stdout:9/286: link d1/da/d36/d3f/d48/f57 d1/da/d36/d3f/f75 0 2024-02-15T18:34:54.847 INFO:tasks.workunit.client.0.smithi038.stdout:4/332: rename d11/d1e/d43/d5e/d62/f69 to d11/d1e/d20/d2f/d71/f7b 0 2024-02-15T18:34:54.848 INFO:tasks.workunit.client.0.smithi038.stdout:6/299: mkdir d2/d4/d19/d55/d59 0 2024-02-15T18:34:54.849 INFO:tasks.workunit.client.0.smithi038.stdout:9/287: unlink d1/dc/d38/f6b 0 2024-02-15T18:34:54.850 INFO:tasks.workunit.client.0.smithi038.stdout:9/288: write d1/da/d36/f37 [4048350,3798] 0 2024-02-15T18:34:54.850 INFO:tasks.workunit.client.0.smithi038.stdout:4/333: symlink d11/d74/l7c 0 2024-02-15T18:34:54.852 INFO:tasks.workunit.client.0.smithi038.stdout:6/300: unlink d2/d4/f42 0 2024-02-15T18:34:54.852 INFO:tasks.workunit.client.0.smithi038.stdout:6/301: read d2/d4/d28/f3a [681331,43773] 0 2024-02-15T18:34:54.853 INFO:tasks.workunit.client.0.smithi038.stdout:4/334: mkdir d11/d1e/d20/d2f/d7d 0 2024-02-15T18:34:54.859 INFO:tasks.workunit.client.0.smithi038.stdout:4/335: mkdir d11/d1e/d3e/d57/d7e 0 2024-02-15T18:34:54.859 INFO:tasks.workunit.client.0.smithi038.stdout:4/336: dread - d11/d1e/d3e/f68 zero size 2024-02-15T18:34:54.859 INFO:tasks.workunit.client.0.smithi038.stdout:3/277: dread d2/d1b/d3c/f31 [0,4194304] 0 2024-02-15T18:34:54.860 INFO:tasks.workunit.client.0.smithi038.stdout:3/278: read d2/d25/f26 [438522,122251] 0 2024-02-15T18:34:54.860 INFO:tasks.workunit.client.0.smithi038.stdout:5/292: dwrite d2/d3/f4e [0,4194304] 0 2024-02-15T18:34:54.861 INFO:tasks.workunit.client.0.smithi038.stdout:7/318: dwrite d0/db/d19/d2d/f52 [0,4194304] 0 2024-02-15T18:34:54.864 INFO:tasks.workunit.client.0.smithi038.stdout:8/459: dwrite d1/d4/d5/f37 [0,4194304] 0 2024-02-15T18:34:54.864 INFO:tasks.workunit.client.0.smithi038.stdout:8/460: readlink d1/d3a/l99 0 2024-02-15T18:34:54.864 INFO:tasks.workunit.client.0.smithi038.stdout:8/461: stat d1/d4/d10/d43/d48/d52/d88/c89 0 2024-02-15T18:34:54.864 INFO:tasks.workunit.client.0.smithi038.stdout:8/462: write d1/d4/d5/d31/d65/d80/fa0 [374989,49369] 0 2024-02-15T18:34:54.864 INFO:tasks.workunit.client.0.smithi038.stdout:3/279: rmdir d2/d25 39 2024-02-15T18:34:54.864 INFO:tasks.workunit.client.0.smithi038.stdout:3/280: chown d2/f2f 15839773 1 2024-02-15T18:34:54.864 INFO:tasks.workunit.client.0.smithi038.stdout:6/302: dread d2/f9 [0,4194304] 0 2024-02-15T18:34:54.866 INFO:tasks.workunit.client.0.smithi038.stdout:5/293: mknod d2/d30/c56 0 2024-02-15T18:34:54.866 INFO:tasks.workunit.client.0.smithi038.stdout:5/294: chown d2/d30 132086883 1 2024-02-15T18:34:54.871 INFO:tasks.workunit.client.0.smithi038.stdout:7/319: creat d0/d1b/d30/d5d/d61/f75 x:0 0 0 2024-02-15T18:34:54.871 INFO:tasks.workunit.client.0.smithi038.stdout:9/289: dwrite d1/dc/d2b/f3a [4194304,4194304] 0 2024-02-15T18:34:54.871 INFO:tasks.workunit.client.0.smithi038.stdout:9/290: dread - d1/da/d10/d13/d3d/d43/f70 zero size 2024-02-15T18:34:54.871 INFO:tasks.workunit.client.0.smithi038.stdout:8/463: mkdir d1/d7/d22/da7 0 2024-02-15T18:34:54.871 INFO:tasks.workunit.client.0.smithi038.stdout:6/303: stat d2/d10/l4d 0 2024-02-15T18:34:54.871 INFO:tasks.workunit.client.0.smithi038.stdout:6/304: chown d2/f9 79170421 1 2024-02-15T18:34:54.872 INFO:tasks.workunit.client.0.smithi038.stdout:9/291: write d1/dc/f71 [951493,62600] 0 2024-02-15T18:34:54.874 INFO:tasks.workunit.client.0.smithi038.stdout:7/320: dread d0/d66/f35 [0,4194304] 0 2024-02-15T18:34:54.874 INFO:tasks.workunit.client.0.smithi038.stdout:8/464: mknod d1/d4/d5/d77/ca8 0 2024-02-15T18:34:54.874 INFO:tasks.workunit.client.0.smithi038.stdout:7/321: readlink d0/d2e/l65 0 2024-02-15T18:34:54.881 INFO:tasks.workunit.client.0.smithi038.stdout:6/305: link d2/d4/d19/f47 d2/f5a 0 2024-02-15T18:34:54.883 INFO:tasks.workunit.client.0.smithi038.stdout:9/292: mkdir d1/da/d20/d58/d76 0 2024-02-15T18:34:54.885 INFO:tasks.workunit.client.0.smithi038.stdout:7/322: rmdir d0/d1b/d30/d62 39 2024-02-15T18:34:54.885 INFO:tasks.workunit.client.0.smithi038.stdout:7/323: getdents d0/d1b/d30/d5d/d70 0 2024-02-15T18:34:54.885 INFO:tasks.workunit.client.0.smithi038.stdout:7/324: fsync d0/db/f20 0 2024-02-15T18:34:54.885 INFO:tasks.workunit.client.0.smithi038.stdout:7/325: dread - d0/d66/d6a/f5b zero size 2024-02-15T18:34:54.885 INFO:tasks.workunit.client.0.smithi038.stdout:7/326: getdents d0/d1b/d30/d5d/d70 0 2024-02-15T18:34:54.886 INFO:tasks.workunit.client.0.smithi038.stdout:3/281: dwrite d2/f9 [0,4194304] 0 2024-02-15T18:34:54.887 INFO:tasks.workunit.client.0.smithi038.stdout:8/465: link d1/d4/d5/d6/f2a d1/d4/d10/d43/d48/d52/fa9 0 2024-02-15T18:34:54.887 INFO:tasks.workunit.client.0.smithi038.stdout:5/295: dwrite d2/d20/f3b [4194304,4194304] 0 2024-02-15T18:34:54.889 INFO:tasks.workunit.client.0.smithi038.stdout:8/466: dread d1/d7/d15/f2f [0,4194304] 0 2024-02-15T18:34:54.891 INFO:tasks.workunit.client.0.smithi038.stdout:5/296: dread d2/d3/d12/fa [0,4194304] 0 2024-02-15T18:34:54.894 INFO:tasks.workunit.client.0.smithi038.stdout:6/306: stat d2/c16 0 2024-02-15T18:34:54.894 INFO:tasks.workunit.client.0.smithi038.stdout:6/307: read d2/f46 [116915,14803] 0 2024-02-15T18:34:54.894 INFO:tasks.workunit.client.0.smithi038.stdout:6/308: chown d2/d10/c12 51 1 2024-02-15T18:34:54.895 INFO:tasks.workunit.client.0.smithi038.stdout:9/293: creat d1/da/d10/d24/f77 x:0 0 0 2024-02-15T18:34:54.895 INFO:tasks.workunit.client.0.smithi038.stdout:9/294: stat d1/da/d10/d19/f42 0 2024-02-15T18:34:54.895 INFO:tasks.workunit.client.0.smithi038.stdout:9/295: read - d1/da/d10/d24/d49/f21 zero size 2024-02-15T18:34:54.895 INFO:tasks.workunit.client.0.smithi038.stdout:9/296: chown d1/dc/d4f/c31 28 1 2024-02-15T18:34:54.895 INFO:tasks.workunit.client.0.smithi038.stdout:9/297: write d1/da/d10/d24/d49/f4e [922904,5178] 0 2024-02-15T18:34:54.896 INFO:tasks.workunit.client.0.smithi038.stdout:7/327: unlink d0/d1b/d30/l3d 0 2024-02-15T18:34:54.896 INFO:tasks.workunit.client.0.smithi038.stdout:7/328: dread d0/d2e/f37 [0,4194304] 0 2024-02-15T18:34:54.898 INFO:tasks.workunit.client.0.smithi038.stdout:8/467: creat d1/d4/faa x:0 0 0 2024-02-15T18:34:54.898 INFO:tasks.workunit.client.0.smithi038.stdout:8/468: dread - d1/f5d zero size 2024-02-15T18:34:54.898 INFO:tasks.workunit.client.0.smithi038.stdout:7/329: write d0/f9 [2474326,103731] 0 2024-02-15T18:34:54.899 INFO:tasks.workunit.client.0.smithi038.stdout:5/297: mkdir d2/d30/d57 0 2024-02-15T18:34:54.902 INFO:tasks.workunit.client.0.smithi038.stdout:7/330: mknod d0/d66/c76 0 2024-02-15T18:34:54.902 INFO:tasks.workunit.client.0.smithi038.stdout:5/298: symlink d2/d30/d52/l58 0 2024-02-15T18:34:54.902 INFO:tasks.workunit.client.0.smithi038.stdout:5/299: chown d2/c44 128009 1 2024-02-15T18:34:54.902 INFO:tasks.workunit.client.0.smithi038.stdout:5/300: fsync d2/d20/f37 0 2024-02-15T18:34:54.902 INFO:tasks.workunit.client.0.smithi038.stdout:6/309: dread d2/d4/f1f [0,4194304] 0 2024-02-15T18:34:54.903 INFO:tasks.workunit.client.0.smithi038.stdout:5/301: truncate d2/f2b 4085558 0 2024-02-15T18:34:54.907 INFO:tasks.workunit.client.0.smithi038.stdout:3/282: dwrite d2/f38 [0,4194304] 0 2024-02-15T18:34:54.907 INFO:tasks.workunit.client.0.smithi038.stdout:3/283: stat d2/d1b/d3c/d3a 0 2024-02-15T18:34:54.908 INFO:tasks.workunit.client.0.smithi038.stdout:7/331: dread d0/db/f42 [0,4194304] 0 2024-02-15T18:34:54.908 INFO:tasks.workunit.client.0.smithi038.stdout:0/406: sync 2024-02-15T18:34:54.908 INFO:tasks.workunit.client.0.smithi038.stdout:7/332: write d0/d66/f35 [4149043,37965] 0 2024-02-15T18:34:54.909 INFO:tasks.workunit.client.0.smithi038.stdout:7/333: stat d0/db/f42 0 2024-02-15T18:34:54.922 INFO:tasks.workunit.client.0.smithi038.stdout:8/469: dwrite d1/d4/f9e [0,4194304] 0 2024-02-15T18:34:54.922 INFO:tasks.workunit.client.0.smithi038.stdout:8/470: readlink d1/d7/l24 0 2024-02-15T18:34:54.922 INFO:tasks.workunit.client.0.smithi038.stdout:5/302: dread d2/f1c [0,4194304] 0 2024-02-15T18:34:54.923 INFO:tasks.workunit.client.0.smithi038.stdout:9/298: dwrite d1/dc/f28 [0,4194304] 0 2024-02-15T18:34:54.925 INFO:tasks.workunit.client.0.smithi038.stdout:8/471: link d1/d7/d15/d4b/f3f d1/d4/d10/d43/d48/d52/fab 0 2024-02-15T18:34:54.929 INFO:tasks.workunit.client.0.smithi038.stdout:5/303: rename d2/d3/f4e to d2/d48/f59 0 2024-02-15T18:34:54.929 INFO:tasks.workunit.client.0.smithi038.stdout:5/304: write d2/d3/d12/f14 [436787,88027] 0 2024-02-15T18:34:54.930 INFO:tasks.workunit.client.0.smithi038.stdout:5/305: write d2/d3/d12/fa [2429034,18792] 0 2024-02-15T18:34:54.930 INFO:tasks.workunit.client.0.smithi038.stdout:5/306: chown d2/f39 61026 1 2024-02-15T18:34:54.931 INFO:tasks.workunit.client.0.smithi038.stdout:8/472: getdents d1/d4 0 2024-02-15T18:34:54.931 INFO:tasks.workunit.client.0.smithi038.stdout:8/473: readlink d1/d3a/l99 0 2024-02-15T18:34:54.931 INFO:tasks.workunit.client.0.smithi038.stdout:8/474: dread d1/d7/d22/f5a [0,4194304] 0 2024-02-15T18:34:54.931 INFO:tasks.workunit.client.0.smithi038.stdout:8/475: chown d1/d7/d15/d6a/fa6 13 1 2024-02-15T18:34:54.931 INFO:tasks.workunit.client.0.smithi038.stdout:8/476: fsync d1/d4/d5/d6/f2a 0 2024-02-15T18:34:54.932 INFO:tasks.workunit.client.0.smithi038.stdout:5/307: link d2/d30/f55 d2/d20/f5a 0 2024-02-15T18:34:54.934 INFO:tasks.workunit.client.0.smithi038.stdout:5/308: unlink d2/d20/f3b 0 2024-02-15T18:34:54.934 INFO:tasks.workunit.client.0.smithi038.stdout:5/309: fdatasync d2/d3/ff 0 2024-02-15T18:34:54.936 INFO:tasks.workunit.client.0.smithi038.stdout:5/310: rename d2/f2b to d2/d30/d52/f5b 0 2024-02-15T18:34:54.939 INFO:tasks.workunit.client.0.smithi038.stdout:6/310: dwrite d2/d4/f2f [0,4194304] 0 2024-02-15T18:34:54.942 INFO:tasks.workunit.client.0.smithi038.stdout:7/334: dwrite d0/d2e/f37 [0,4194304] 0 2024-02-15T18:34:54.942 INFO:tasks.workunit.client.0.smithi038.stdout:0/407: dwrite d4/d13/d2a/d35/d1e/d32/f76 [0,4194304] 0 2024-02-15T18:34:54.942 INFO:tasks.workunit.client.0.smithi038.stdout:0/408: fdatasync d4/f11 0 2024-02-15T18:34:54.942 INFO:tasks.workunit.client.0.smithi038.stdout:0/409: fdatasync d4/d13/d2a/d35/f49 0 2024-02-15T18:34:54.942 INFO:tasks.workunit.client.0.smithi038.stdout:0/410: write d4/d13/f39 [1392200,129408] 0 2024-02-15T18:34:54.942 INFO:tasks.workunit.client.0.smithi038.stdout:0/411: dread - d4/f88 zero size 2024-02-15T18:34:54.942 INFO:tasks.workunit.client.0.smithi038.stdout:9/299: dwrite d1/fb [0,4194304] 0 2024-02-15T18:34:54.943 INFO:tasks.workunit.client.0.smithi038.stdout:6/311: rmdir d2/d4/d19/d55/d59 0 2024-02-15T18:34:54.948 INFO:tasks.workunit.client.0.smithi038.stdout:7/335: unlink d0/d2e/l65 0 2024-02-15T18:34:54.948 INFO:tasks.workunit.client.0.smithi038.stdout:7/336: chown d0/d1b/d30 1 1 2024-02-15T18:34:54.953 INFO:tasks.workunit.client.0.smithi038.stdout:7/337: dread d0/db/d19/f25 [0,4194304] 0 2024-02-15T18:34:54.954 INFO:tasks.workunit.client.0.smithi038.stdout:0/412: rename d4/d13/d2a/d35/d1e/l66 to d4/d13/d2a/d35/d6a/l8a 0 2024-02-15T18:34:54.955 INFO:tasks.workunit.client.0.smithi038.stdout:5/311: dwrite d2/d20/f37 [0,4194304] 0 2024-02-15T18:34:54.955 INFO:tasks.workunit.client.0.smithi038.stdout:1/320: sync 2024-02-15T18:34:54.956 INFO:tasks.workunit.client.0.smithi038.stdout:1/321: read d2/d2e/f38 [3971716,57112] 0 2024-02-15T18:34:54.956 INFO:tasks.workunit.client.0.smithi038.stdout:1/322: chown d2/d39/c7c 16437 1 2024-02-15T18:34:54.956 INFO:tasks.workunit.client.0.smithi038.stdout:9/300: mkdir d1/da/d20/d58/d76/d78 0 2024-02-15T18:34:54.957 INFO:tasks.workunit.client.0.smithi038.stdout:6/312: creat d2/f5b x:0 0 0 2024-02-15T18:34:54.958 INFO:tasks.workunit.client.0.smithi038.stdout:7/338: dread d0/db/d19/f25 [0,4194304] 0 2024-02-15T18:34:54.958 INFO:tasks.workunit.client.0.smithi038.stdout:6/313: write d2/d4/ff [75601,31799] 0 2024-02-15T18:34:54.958 INFO:tasks.workunit.client.0.smithi038.stdout:0/413: rename d4/d13/d2a/d35/d6a/f73 to d4/d13/d2a/d35/f8b 0 2024-02-15T18:34:54.958 INFO:tasks.workunit.client.0.smithi038.stdout:0/414: chown d4/d13/d2a/d35/d1e/d32/d3b/f60 759 1 2024-02-15T18:34:54.961 INFO:tasks.workunit.client.0.smithi038.stdout:7/339: read d0/db/d59/f7 [4168905,37387] 0 2024-02-15T18:34:54.961 INFO:tasks.workunit.client.0.smithi038.stdout:5/312: dread d2/f1c [0,4194304] 0 2024-02-15T18:34:54.961 INFO:tasks.workunit.client.0.smithi038.stdout:5/313: fdatasync d2/f38 0 2024-02-15T18:34:54.961 INFO:tasks.workunit.client.0.smithi038.stdout:1/323: creat d2/d2e/f7d x:0 0 0 2024-02-15T18:34:54.962 INFO:tasks.workunit.client.0.smithi038.stdout:9/301: symlink d1/da/d36/d3f/d48/l79 0 2024-02-15T18:34:54.963 INFO:tasks.workunit.client.0.smithi038.stdout:9/302: write d1/dc/d2b/f60 [5896186,82690] 0 2024-02-15T18:34:54.966 INFO:tasks.workunit.client.0.smithi038.stdout:6/314: mkdir d2/d4/d19/d55/d5c 0 2024-02-15T18:34:54.970 INFO:tasks.workunit.client.0.smithi038.stdout:0/415: link d4/f63 d4/d13/d2a/d35/d1e/d32/d48/f8c 0 2024-02-15T18:34:54.970 INFO:tasks.workunit.client.0.smithi038.stdout:0/416: write d4/d13/d53/f7b [219830,7732] 0 2024-02-15T18:34:54.971 INFO:tasks.workunit.client.0.smithi038.stdout:1/324: link d2/d10/d32/d53/c74 d2/d10/d19/d22/d36/c7e 0 2024-02-15T18:34:54.974 INFO:tasks.workunit.client.0.smithi038.stdout:9/303: rmdir d1/da/d36 39 2024-02-15T18:34:54.976 INFO:tasks.workunit.client.0.smithi038.stdout:6/315: creat d2/d4/d2c/d39/f5d x:0 0 0 2024-02-15T18:34:54.976 INFO:tasks.workunit.client.0.smithi038.stdout:5/314: dread d2/d30/d52/f5b [0,4194304] 0 2024-02-15T18:34:54.978 INFO:tasks.workunit.client.0.smithi038.stdout:0/417: rename d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/f84 to d4/d13/d2a/d35/d1e/d32/d3b/f8d 0 2024-02-15T18:34:54.978 INFO:tasks.workunit.client.0.smithi038.stdout:0/418: write d4/d13/d2a/d35/f83 [2762,59775] 0 2024-02-15T18:34:54.979 INFO:tasks.workunit.client.0.smithi038.stdout:1/325: truncate d2/d10/d32/d53/f7b 1410717 0 2024-02-15T18:34:54.979 INFO:tasks.workunit.client.0.smithi038.stdout:9/304: creat d1/da/d10/d13/f7a x:0 0 0 2024-02-15T18:34:54.980 INFO:tasks.workunit.client.0.smithi038.stdout:1/326: write d2/d10/d19/d22/d36/f44 [1203536,24893] 0 2024-02-15T18:34:54.981 INFO:tasks.workunit.client.0.smithi038.stdout:5/315: rename d2/d20/l23 to d2/d3/d12/l5c 0 2024-02-15T18:34:54.983 INFO:tasks.workunit.client.0.smithi038.stdout:9/305: creat d1/da/d10/d13/d3d/d43/d4d/f7b x:0 0 0 2024-02-15T18:34:54.984 INFO:tasks.workunit.client.0.smithi038.stdout:7/340: dwrite d0/d1b/f3f [0,4194304] 0 2024-02-15T18:34:54.984 INFO:tasks.workunit.client.0.smithi038.stdout:7/341: dread - d0/f48 zero size 2024-02-15T18:34:54.984 INFO:tasks.workunit.client.0.smithi038.stdout:1/327: rmdir d2/d39 39 2024-02-15T18:34:54.984 INFO:tasks.workunit.client.0.smithi038.stdout:6/316: dread d2/d10/f49 [0,4194304] 0 2024-02-15T18:34:54.984 INFO:tasks.workunit.client.0.smithi038.stdout:6/317: stat d2/c1b 0 2024-02-15T18:34:54.985 INFO:tasks.workunit.client.0.smithi038.stdout:1/328: write d2/f11 [1327599,115750] 0 2024-02-15T18:34:54.985 INFO:tasks.workunit.client.0.smithi038.stdout:1/329: write d2/d10/f37 [152592,111905] 0 2024-02-15T18:34:54.985 INFO:tasks.workunit.client.0.smithi038.stdout:1/330: truncate d2/d10/d19/f72 533843 0 2024-02-15T18:34:54.985 INFO:tasks.workunit.client.0.smithi038.stdout:5/316: link d2/d3/ff d2/d30/f5d 0 2024-02-15T18:34:54.986 INFO:tasks.workunit.client.0.smithi038.stdout:0/419: rmdir d4/d13/d2a/d35/d1e 39 2024-02-15T18:34:54.990 INFO:tasks.workunit.client.0.smithi038.stdout:9/306: rmdir d1/da/d10/d13/d3d/d43 39 2024-02-15T18:34:54.991 INFO:tasks.workunit.client.0.smithi038.stdout:7/342: symlink d0/db/l77 0 2024-02-15T18:34:54.992 INFO:tasks.workunit.client.0.smithi038.stdout:6/318: unlink d2/d4/d2c/f52 0 2024-02-15T18:34:54.992 INFO:tasks.workunit.client.0.smithi038.stdout:6/319: stat d2/d4/c31 0 2024-02-15T18:34:54.995 INFO:tasks.workunit.client.0.smithi038.stdout:1/331: creat d2/d39/f7f x:0 0 0 2024-02-15T18:34:54.995 INFO:tasks.workunit.client.0.smithi038.stdout:5/317: stat d2/c53 0 2024-02-15T18:34:54.995 INFO:tasks.workunit.client.0.smithi038.stdout:5/318: readlink d2/d30/l32 0 2024-02-15T18:34:54.996 INFO:tasks.workunit.client.0.smithi038.stdout:1/332: write d2/d10/d19/f72 [179438,91730] 0 2024-02-15T18:34:54.996 INFO:tasks.workunit.client.0.smithi038.stdout:0/420: creat d4/d13/f8e x:0 0 0 2024-02-15T18:34:54.996 INFO:tasks.workunit.client.0.smithi038.stdout:0/421: dread - d4/d13/f8e zero size 2024-02-15T18:34:54.997 INFO:tasks.workunit.client.0.smithi038.stdout:9/307: rename d1/dc/d2b/f34 to d1/da/d10/d24/f7c 0 2024-02-15T18:34:54.997 INFO:tasks.workunit.client.0.smithi038.stdout:9/308: chown d1 13 1 2024-02-15T18:34:54.997 INFO:tasks.workunit.client.0.smithi038.stdout:6/320: truncate d2/d4/fc 635898 0 2024-02-15T18:34:54.998 INFO:tasks.workunit.client.0.smithi038.stdout:5/319: mkdir d2/d3/d12/d5e 0 2024-02-15T18:34:54.999 INFO:tasks.workunit.client.0.smithi038.stdout:1/333: mkdir d2/d10/d19/d22/d54/d80 0 2024-02-15T18:34:55.006 INFO:tasks.workunit.client.0.smithi038.stdout:0/422: link d4/d13/d2a/d35/d1e/d32/d48/f4c d4/d13/d2a/d35/f8f 0 2024-02-15T18:34:55.008 INFO:tasks.workunit.client.0.smithi038.stdout:7/343: rename d0/d2e/f4d to d0/d1b/f78 0 2024-02-15T18:34:55.008 INFO:tasks.workunit.client.0.smithi038.stdout:9/309: mkdir d1/dc/d69/d7d 0 2024-02-15T18:34:55.009 INFO:tasks.workunit.client.0.smithi038.stdout:5/320: mkdir d2/d3/d5f 0 2024-02-15T18:34:55.009 INFO:tasks.workunit.client.0.smithi038.stdout:5/321: chown d2/d3/c42 4 1 2024-02-15T18:34:55.009 INFO:tasks.workunit.client.0.smithi038.stdout:1/334: stat d2/d28/d3c/l6f 0 2024-02-15T18:34:55.010 INFO:tasks.workunit.client.0.smithi038.stdout:7/344: dread d0/db/d59/d15/d29/f31 [0,4194304] 0 2024-02-15T18:34:55.011 INFO:tasks.workunit.client.0.smithi038.stdout:0/423: creat d4/d13/d2a/d35/d1e/d32/d3b/f90 x:0 0 0 2024-02-15T18:34:55.011 INFO:tasks.workunit.client.0.smithi038.stdout:9/310: read d1/f16 [38325,66827] 0 2024-02-15T18:34:55.014 INFO:tasks.workunit.client.0.smithi038.stdout:7/345: getdents d0/d1b/d30 0 2024-02-15T18:34:55.015 INFO:tasks.workunit.client.0.smithi038.stdout:9/311: symlink d1/da/d10/d13/d3d/d4b/l7e 0 2024-02-15T18:34:55.015 INFO:tasks.workunit.client.0.smithi038.stdout:7/346: mknod d0/d66/c79 0 2024-02-15T18:34:55.016 INFO:tasks.workunit.client.0.smithi038.stdout:6/321: dwrite d2/f5a [0,4194304] 0 2024-02-15T18:34:55.017 INFO:tasks.workunit.client.0.smithi038.stdout:9/312: rmdir d1/da/d20/d58/d76/d78 0 2024-02-15T18:34:55.021 INFO:tasks.workunit.client.0.smithi038.stdout:6/322: creat d2/d56/f5e x:0 0 0 2024-02-15T18:34:55.022 INFO:tasks.workunit.client.0.smithi038.stdout:9/313: getdents d1/dc/d5a 0 2024-02-15T18:34:55.022 INFO:tasks.workunit.client.0.smithi038.stdout:9/314: dread - d1/dc/d4f/f62 zero size 2024-02-15T18:34:55.022 INFO:tasks.workunit.client.0.smithi038.stdout:0/424: dread d4/ff [0,4194304] 0 2024-02-15T18:34:55.026 INFO:tasks.workunit.client.0.smithi038.stdout:2/288: sync 2024-02-15T18:34:55.026 INFO:tasks.workunit.client.0.smithi038.stdout:4/337: sync 2024-02-15T18:34:55.026 INFO:tasks.workunit.client.0.smithi038.stdout:4/338: rename d11/d1e/d3e to d11/d1e/d3e/d75/d7f 22 2024-02-15T18:34:55.026 INFO:tasks.workunit.client.0.smithi038.stdout:4/339: dread - d11/d1e/d3e/d57/f77 zero size 2024-02-15T18:34:55.026 INFO:tasks.workunit.client.0.smithi038.stdout:9/315: mkdir d1/dc/d5a/d7f 0 2024-02-15T18:34:55.028 INFO:tasks.workunit.client.0.smithi038.stdout:4/340: truncate d11/d1e/f37 5639051 0 2024-02-15T18:34:55.029 INFO:tasks.workunit.client.0.smithi038.stdout:0/425: dread d4/f12 [0,4194304] 0 2024-02-15T18:34:55.031 INFO:tasks.workunit.client.0.smithi038.stdout:1/335: dwrite d2/d10/d19/d22/f7a [0,4194304] 0 2024-02-15T18:34:55.031 INFO:tasks.workunit.client.0.smithi038.stdout:1/336: write d2/d10/d62/f68 [64502,48657] 0 2024-02-15T18:34:55.031 INFO:tasks.workunit.client.0.smithi038.stdout:2/289: dread dd/d12/d13/f28 [0,4194304] 0 2024-02-15T18:34:55.031 INFO:tasks.workunit.client.0.smithi038.stdout:2/290: truncate dd/d12/d59/d1f/f27 2907453 0 2024-02-15T18:34:55.031 INFO:tasks.workunit.client.0.smithi038.stdout:9/316: dread d1/da/d36/d3f/f75 [0,4194304] 0 2024-02-15T18:34:55.031 INFO:tasks.workunit.client.0.smithi038.stdout:9/317: chown d1/da/d20/d58 440559 1 2024-02-15T18:34:55.033 INFO:tasks.workunit.client.0.smithi038.stdout:7/347: dread d0/db/d59/d15/f23 [0,4194304] 0 2024-02-15T18:34:55.033 INFO:tasks.workunit.client.0.smithi038.stdout:7/348: chown d0/db/c2a 1 1 2024-02-15T18:34:55.034 INFO:tasks.workunit.client.0.smithi038.stdout:1/337: mknod d2/d10/d19/d22/d54/c81 0 2024-02-15T18:34:55.035 INFO:tasks.workunit.client.0.smithi038.stdout:1/338: write d2/d10/d19/f1d [2330060,77920] 0 2024-02-15T18:34:55.036 INFO:tasks.workunit.client.0.smithi038.stdout:3/284: sync 2024-02-15T18:34:55.037 INFO:tasks.workunit.client.0.smithi038.stdout:8/477: sync 2024-02-15T18:34:55.038 INFO:tasks.workunit.client.0.smithi038.stdout:1/339: mkdir d2/d10/d32/d53/d82 0 2024-02-15T18:34:55.038 INFO:tasks.workunit.client.0.smithi038.stdout:1/340: dread - d2/d10/d19/d22/d36/f78 zero size 2024-02-15T18:34:55.038 INFO:tasks.workunit.client.0.smithi038.stdout:3/285: truncate d2/f15 3498938 0 2024-02-15T18:34:55.039 INFO:tasks.workunit.client.0.smithi038.stdout:3/286: readlink d2/d24/l2d 0 2024-02-15T18:34:55.039 INFO:tasks.workunit.client.0.smithi038.stdout:1/341: creat d2/d69/f83 x:0 0 0 2024-02-15T18:34:55.040 INFO:tasks.workunit.client.0.smithi038.stdout:3/287: mknod d2/c5a 0 2024-02-15T18:34:55.045 INFO:tasks.workunit.client.0.smithi038.stdout:1/342: rmdir d2/d10/d19/d2f/d3e 39 2024-02-15T18:34:55.048 INFO:tasks.workunit.client.0.smithi038.stdout:1/343: rename d2/ce to d2/d10/d19/d22/d54/c84 0 2024-02-15T18:34:55.048 INFO:tasks.workunit.client.0.smithi038.stdout:6/323: dread d2/d4/f2f [0,4194304] 0 2024-02-15T18:34:55.048 INFO:tasks.workunit.client.0.smithi038.stdout:9/318: read f0 [1512368,62353] 0 2024-02-15T18:34:55.049 INFO:tasks.workunit.client.0.smithi038.stdout:6/324: fdatasync d2/d4/d28/f3a 0 2024-02-15T18:34:55.050 INFO:tasks.workunit.client.0.smithi038.stdout:1/344: mkdir d2/d10/d19/d2f/d85 0 2024-02-15T18:34:55.052 INFO:tasks.workunit.client.0.smithi038.stdout:9/319: creat d1/da/d36/d3f/f80 x:0 0 0 2024-02-15T18:34:55.056 INFO:tasks.workunit.client.0.smithi038.stdout:9/320: rmdir d1/dc/d5a/d5b 0 2024-02-15T18:34:55.058 INFO:tasks.workunit.client.0.smithi038.stdout:9/321: symlink d1/da/d10/d13/d3d/d43/d4d/l81 0 2024-02-15T18:34:55.059 INFO:tasks.workunit.client.0.smithi038.stdout:9/322: truncate d1/dc/d2b/f60 6145167 0 2024-02-15T18:34:55.060 INFO:tasks.workunit.client.0.smithi038.stdout:9/323: mkdir d1/da/d10/d82 0 2024-02-15T18:34:55.065 INFO:tasks.workunit.client.0.smithi038.stdout:9/324: truncate d1/dc/f71 3971845 0 2024-02-15T18:34:55.067 INFO:tasks.workunit.client.0.smithi038.stdout:2/291: dwrite dd/d12/d1d/d3b/f4b [0,4194304] 0 2024-02-15T18:34:55.074 INFO:tasks.workunit.client.0.smithi038.stdout:2/292: rename dd/d12/d13 to dd/d12/d5d 0 2024-02-15T18:34:55.080 INFO:tasks.workunit.client.0.smithi038.stdout:9/325: dread d1/da/d36/d50/f55 [0,4194304] 0 2024-02-15T18:34:55.080 INFO:tasks.workunit.client.0.smithi038.stdout:9/326: fdatasync d1/da/d10/d24/d49/f21 0 2024-02-15T18:34:55.080 INFO:tasks.workunit.client.0.smithi038.stdout:9/327: stat d1/da/d10/d24/f53 0 2024-02-15T18:34:55.080 INFO:tasks.workunit.client.0.smithi038.stdout:0/426: dwrite d4/d13/d2a/d35/f8b [0,4194304] 0 2024-02-15T18:34:55.081 INFO:tasks.workunit.client.0.smithi038.stdout:9/328: write d1/da/d10/d13/d3d/d4b/f72 [962983,34338] 0 2024-02-15T18:34:55.083 INFO:tasks.workunit.client.0.smithi038.stdout:0/427: creat d4/d13/d19/f91 x:0 0 0 2024-02-15T18:34:55.083 INFO:tasks.workunit.client.0.smithi038.stdout:0/428: truncate d4/d13/d19/f91 1018383 0 2024-02-15T18:34:55.083 INFO:tasks.workunit.client.0.smithi038.stdout:0/429: dread - d4/d13/d19/f82 zero size 2024-02-15T18:34:55.085 INFO:tasks.workunit.client.0.smithi038.stdout:0/430: unlink l3 0 2024-02-15T18:34:55.085 INFO:tasks.workunit.client.0.smithi038.stdout:0/431: truncate d4/d13/f54 4915844 0 2024-02-15T18:34:55.085 INFO:tasks.workunit.client.0.smithi038.stdout:0/432: dread - d4/d13/d2a/d35/d1e/d32/d3b/f60 zero size 2024-02-15T18:34:55.093 INFO:tasks.workunit.client.0.smithi038.stdout:6/325: dwrite d2/d4/ff [0,4194304] 0 2024-02-15T18:34:55.093 INFO:tasks.workunit.client.0.smithi038.stdout:6/326: fdatasync d2/d4/d2c/f51 0 2024-02-15T18:34:55.094 INFO:tasks.workunit.client.0.smithi038.stdout:8/478: dwrite d1/d4/d10/d91/fa4 [0,4194304] 0 2024-02-15T18:34:55.094 INFO:tasks.workunit.client.0.smithi038.stdout:7/349: dwrite d0/db/d59/f7 [0,4194304] 0 2024-02-15T18:34:55.094 INFO:tasks.workunit.client.0.smithi038.stdout:7/350: truncate d0/d1b/d30/d5d/d61/f75 598648 0 2024-02-15T18:34:55.094 INFO:tasks.workunit.client.0.smithi038.stdout:7/351: readlink d0/d1b/d30/d62/l6c 0 2024-02-15T18:34:55.094 INFO:tasks.workunit.client.0.smithi038.stdout:6/327: truncate d2/f2a 2169208 0 2024-02-15T18:34:55.094 INFO:tasks.workunit.client.0.smithi038.stdout:6/328: fdatasync d2/f46 0 2024-02-15T18:34:55.094 INFO:tasks.workunit.client.0.smithi038.stdout:6/329: stat d2/d4/d19 0 2024-02-15T18:34:55.096 INFO:tasks.workunit.client.0.smithi038.stdout:8/479: truncate d1/d7/fc 937061 0 2024-02-15T18:34:55.096 INFO:tasks.workunit.client.0.smithi038.stdout:0/433: dread d4/d13/d2a/d35/d1e/f20 [4194304,4194304] 0 2024-02-15T18:34:55.096 INFO:tasks.workunit.client.0.smithi038.stdout:0/434: stat d4/d13/d2a/d35 0 2024-02-15T18:34:55.097 INFO:tasks.workunit.client.0.smithi038.stdout:7/352: creat d0/db/f7a x:0 0 0 2024-02-15T18:34:55.103 INFO:tasks.workunit.client.0.smithi038.stdout:6/330: mkdir d2/d56/d5f 0 2024-02-15T18:34:55.103 INFO:tasks.workunit.client.0.smithi038.stdout:6/331: truncate d2/d4/f27 9356061 0 2024-02-15T18:34:55.104 INFO:tasks.workunit.client.0.smithi038.stdout:8/480: dread d1/d3a/d58/f60 [0,4194304] 0 2024-02-15T18:34:55.104 INFO:tasks.workunit.client.0.smithi038.stdout:8/481: chown d1/d7/d15/d6a 4033 1 2024-02-15T18:34:55.104 INFO:tasks.workunit.client.0.smithi038.stdout:8/482: fsync d1/d4/d5/f20 0 2024-02-15T18:34:55.104 INFO:tasks.workunit.client.0.smithi038.stdout:3/288: dwrite d2/f38 [0,4194304] 0 2024-02-15T18:34:55.105 INFO:tasks.workunit.client.0.smithi038.stdout:6/332: mknod d2/d4/d28/c60 0 2024-02-15T18:34:55.105 INFO:tasks.workunit.client.0.smithi038.stdout:1/345: dwrite d2/d10/d19/d22/f7a [0,4194304] 0 2024-02-15T18:34:55.105 INFO:tasks.workunit.client.0.smithi038.stdout:1/346: write d2/d10/d19/d22/f2c [1459988,87235] 0 2024-02-15T18:34:55.106 INFO:tasks.workunit.client.0.smithi038.stdout:6/333: read d2/d10/f49 [3213656,130772] 0 2024-02-15T18:34:55.106 INFO:tasks.workunit.client.0.smithi038.stdout:6/334: write d2/d10/f4c [739760,122518] 0 2024-02-15T18:34:55.106 INFO:tasks.workunit.client.0.smithi038.stdout:8/483: creat d1/d4/d5/d31/d65/d80/fac x:0 0 0 2024-02-15T18:34:55.107 INFO:tasks.workunit.client.0.smithi038.stdout:2/293: dwrite dd/d12/d1d/f31 [0,4194304] 0 2024-02-15T18:34:55.116 INFO:tasks.workunit.client.0.smithi038.stdout:3/289: mknod d2/d24/c5b 0 2024-02-15T18:34:55.116 INFO:tasks.workunit.client.0.smithi038.stdout:3/290: truncate d2/d25/f30 175401 0 2024-02-15T18:34:55.122 INFO:tasks.workunit.client.0.smithi038.stdout:8/484: mknod d1/d4/d10/d43/d48/d9a/cad 0 2024-02-15T18:34:55.122 INFO:tasks.workunit.client.0.smithi038.stdout:8/485: read - d1/d7/d15/d4b/f57 zero size 2024-02-15T18:34:55.122 INFO:tasks.workunit.client.0.smithi038.stdout:2/294: mkdir dd/d12/d3a/d5e 0 2024-02-15T18:34:55.123 INFO:tasks.workunit.client.0.smithi038.stdout:8/486: symlink d1/d4/d10/d43/d48/lae 0 2024-02-15T18:34:55.124 INFO:tasks.workunit.client.0.smithi038.stdout:2/295: truncate dd/f24 235043 0 2024-02-15T18:34:55.124 INFO:tasks.workunit.client.0.smithi038.stdout:2/296: chown dd/d12/d5d/d54/c57 5972751 1 2024-02-15T18:34:55.124 INFO:tasks.workunit.client.0.smithi038.stdout:2/297: chown dd/d12/d1d/d3b/d3c/f48 397733 1 2024-02-15T18:34:55.125 INFO:tasks.workunit.client.0.smithi038.stdout:2/298: write dd/d12/d5d/f28 [3222449,46333] 0 2024-02-15T18:34:55.133 INFO:tasks.workunit.client.0.smithi038.stdout:7/353: dwrite d0/db/f7a [0,4194304] 0 2024-02-15T18:34:55.133 INFO:tasks.workunit.client.0.smithi038.stdout:7/354: dread - d0/d66/d6a/f60 zero size 2024-02-15T18:34:55.136 INFO:tasks.workunit.client.0.smithi038.stdout:7/355: symlink d0/db/d59/l7b 0 2024-02-15T18:34:55.136 INFO:tasks.workunit.client.0.smithi038.stdout:7/356: write d0/d66/f1c [5061074,15279] 0 2024-02-15T18:34:55.136 INFO:tasks.workunit.client.0.smithi038.stdout:7/357: chown d0/db/d19/l28 188086 1 2024-02-15T18:34:55.138 INFO:tasks.workunit.client.0.smithi038.stdout:7/358: unlink d0/f5f 0 2024-02-15T18:34:55.138 INFO:tasks.workunit.client.0.smithi038.stdout:3/291: dread d2/d24/f2e [0,4194304] 0 2024-02-15T18:34:55.140 INFO:tasks.workunit.client.0.smithi038.stdout:3/292: link d2/d1b/f4a d2/d1b/f5c 0 2024-02-15T18:34:55.145 INFO:tasks.workunit.client.0.smithi038.stdout:3/293: dread d2/d24/f2e [0,4194304] 0 2024-02-15T18:34:55.146 INFO:tasks.workunit.client.0.smithi038.stdout:3/294: write d2/f1a [2621857,59750] 0 2024-02-15T18:34:55.153 INFO:tasks.workunit.client.0.smithi038.stdout:0/435: dwrite d4/d13/d2a/d35/d1e/d32/d48/f8c [0,4194304] 0 2024-02-15T18:34:55.153 INFO:tasks.workunit.client.0.smithi038.stdout:0/436: fsync d4/d13/d2a/f44 0 2024-02-15T18:34:55.156 INFO:tasks.workunit.client.0.smithi038.stdout:0/437: stat d4/d13/c57 0 2024-02-15T18:34:55.157 INFO:tasks.workunit.client.0.smithi038.stdout:6/335: dwrite d2/f1a [0,4194304] 0 2024-02-15T18:34:55.158 INFO:tasks.workunit.client.0.smithi038.stdout:1/347: dwrite d2/d10/d19/d22/d36/f4d [0,4194304] 0 2024-02-15T18:34:55.158 INFO:tasks.workunit.client.0.smithi038.stdout:8/487: dwrite d1/d7/d15/d6a/fa6 [0,4194304] 0 2024-02-15T18:34:55.158 INFO:tasks.workunit.client.0.smithi038.stdout:2/299: dwrite dd/d12/d3a/f45 [0,4194304] 0 2024-02-15T18:34:55.159 INFO:tasks.workunit.client.0.smithi038.stdout:1/348: write d2/d10/d19/d22/f30 [533212,80671] 0 2024-02-15T18:34:55.159 INFO:tasks.workunit.client.0.smithi038.stdout:6/336: dread d2/f46 [0,4194304] 0 2024-02-15T18:34:55.159 INFO:tasks.workunit.client.0.smithi038.stdout:0/438: creat d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/f92 x:0 0 0 2024-02-15T18:34:55.160 INFO:tasks.workunit.client.0.smithi038.stdout:6/337: write d2/f9 [862695,44419] 0 2024-02-15T18:34:55.165 INFO:tasks.workunit.client.0.smithi038.stdout:2/300: dread dd/d12/d59/d1f/f27 [0,4194304] 0 2024-02-15T18:34:55.166 INFO:tasks.workunit.client.0.smithi038.stdout:3/295: fsync d2/d1b/f4a 0 2024-02-15T18:34:55.167 INFO:tasks.workunit.client.0.smithi038.stdout:0/439: dread d4/f11 [0,4194304] 0 2024-02-15T18:34:55.167 INFO:tasks.workunit.client.0.smithi038.stdout:6/338: dread d2/f46 [0,4194304] 0 2024-02-15T18:34:55.168 INFO:tasks.workunit.client.0.smithi038.stdout:0/440: fdatasync d4/d13/d2a/d35/d1e/f4d 0 2024-02-15T18:34:55.168 INFO:tasks.workunit.client.0.smithi038.stdout:8/488: dread d1/d4/d10/d43/d48/d9a/fa5 [0,4194304] 0 2024-02-15T18:34:55.172 INFO:tasks.workunit.client.0.smithi038.stdout:5/322: sync 2024-02-15T18:34:55.172 INFO:tasks.workunit.client.0.smithi038.stdout:1/349: symlink d2/d2e/l86 0 2024-02-15T18:34:55.172 INFO:tasks.workunit.client.0.smithi038.stdout:1/350: readlink d2/d10/l21 0 2024-02-15T18:34:55.174 INFO:tasks.workunit.client.0.smithi038.stdout:1/351: write d2/d2e/f38 [2931113,130884] 0 2024-02-15T18:34:55.175 INFO:tasks.workunit.client.0.smithi038.stdout:2/301: unlink dd/d12/d59/f1c 0 2024-02-15T18:34:55.177 INFO:tasks.workunit.client.0.smithi038.stdout:3/296: creat d2/d1b/d4f/f5d x:0 0 0 2024-02-15T18:34:55.177 INFO:tasks.workunit.client.0.smithi038.stdout:3/297: stat d2/f1a 0 2024-02-15T18:34:55.178 INFO:tasks.workunit.client.0.smithi038.stdout:9/329: sync 2024-02-15T18:34:55.179 INFO:tasks.workunit.client.0.smithi038.stdout:4/341: sync 2024-02-15T18:34:55.179 INFO:tasks.workunit.client.0.smithi038.stdout:9/330: stat d1/da/d36/d50 0 2024-02-15T18:34:55.180 INFO:tasks.workunit.client.0.smithi038.stdout:6/339: symlink d2/l61 0 2024-02-15T18:34:55.181 INFO:tasks.workunit.client.0.smithi038.stdout:8/489: mknod d1/caf 0 2024-02-15T18:34:55.182 INFO:tasks.workunit.client.0.smithi038.stdout:1/352: mkdir d2/d10/d19/d22/d36/d4f/d87 0 2024-02-15T18:34:55.182 INFO:tasks.workunit.client.0.smithi038.stdout:1/353: chown d2/d10/d19/d22/d54/d80 100007049 1 2024-02-15T18:34:55.184 INFO:tasks.workunit.client.0.smithi038.stdout:3/298: rename d2/d1b/d3c/d41/l4d to d2/d1b/d4f/l5e 0 2024-02-15T18:34:55.184 INFO:tasks.workunit.client.0.smithi038.stdout:5/323: dread d2/d20/f37 [0,4194304] 0 2024-02-15T18:34:55.186 INFO:tasks.workunit.client.0.smithi038.stdout:4/342: link d11/d1e/d3e/d75/d40/l3f d11/d1e/d43/d5e/d62/l80 0 2024-02-15T18:34:55.187 INFO:tasks.workunit.client.0.smithi038.stdout:9/331: symlink d1/dc/d38/l83 0 2024-02-15T18:34:55.187 INFO:tasks.workunit.client.0.smithi038.stdout:0/441: dwrite d4/d13/d2a/d35/d1e/d32/d3b/f60 [0,4194304] 0 2024-02-15T18:34:55.188 INFO:tasks.workunit.client.0.smithi038.stdout:1/354: creat d2/d10/d62/f88 x:0 0 0 2024-02-15T18:34:55.188 INFO:tasks.workunit.client.0.smithi038.stdout:5/324: symlink d2/d3/d12/d5e/l60 0 2024-02-15T18:34:55.188 INFO:tasks.workunit.client.0.smithi038.stdout:5/325: readlink d2/d30/d52/l58 0 2024-02-15T18:34:55.190 INFO:tasks.workunit.client.0.smithi038.stdout:4/343: rmdir d11/d1e/d6c 0 2024-02-15T18:34:55.190 INFO:tasks.workunit.client.0.smithi038.stdout:4/344: chown d11/d1e/d43/d5e 44 1 2024-02-15T18:34:55.194 INFO:tasks.workunit.client.0.smithi038.stdout:1/355: creat d2/d39/f89 x:0 0 0 2024-02-15T18:34:55.194 INFO:tasks.workunit.client.0.smithi038.stdout:5/326: link d2/d30/c31 d2/d30/c61 0 2024-02-15T18:34:55.196 INFO:tasks.workunit.client.0.smithi038.stdout:4/345: mknod d11/d1e/d3e/c81 0 2024-02-15T18:34:55.196 INFO:tasks.workunit.client.0.smithi038.stdout:5/327: creat d2/d3/d12/f62 x:0 0 0 2024-02-15T18:34:55.197 INFO:tasks.workunit.client.0.smithi038.stdout:5/328: symlink d2/d20/l63 0 2024-02-15T18:34:55.203 INFO:tasks.workunit.client.0.smithi038.stdout:3/299: dread d2/d24/f2e [0,4194304] 0 2024-02-15T18:34:55.204 INFO:tasks.workunit.client.0.smithi038.stdout:3/300: mknod d2/d1b/c5f 0 2024-02-15T18:34:55.205 INFO:tasks.workunit.client.0.smithi038.stdout:3/301: write d2/d1b/d3c/f31 [3891694,11713] 0 2024-02-15T18:34:55.210 INFO:tasks.workunit.client.0.smithi038.stdout:6/340: dread d2/d4/d2c/f3c [0,4194304] 0 2024-02-15T18:34:55.212 INFO:tasks.workunit.client.0.smithi038.stdout:6/341: dread d2/f46 [0,4194304] 0 2024-02-15T18:34:55.212 INFO:tasks.workunit.client.0.smithi038.stdout:6/342: fdatasync d2/f46 0 2024-02-15T18:34:55.212 INFO:tasks.workunit.client.0.smithi038.stdout:6/343: chown d2/l50 11062608 1 2024-02-15T18:34:55.212 INFO:tasks.workunit.client.0.smithi038.stdout:1/356: dread d2/d10/d32/d53/f7b [0,4194304] 0 2024-02-15T18:34:55.212 INFO:tasks.workunit.client.0.smithi038.stdout:1/357: readlink d2/l6 0 2024-02-15T18:34:55.212 INFO:tasks.workunit.client.0.smithi038.stdout:1/358: dread - d2/d10/d62/f88 zero size 2024-02-15T18:34:55.215 INFO:tasks.workunit.client.0.smithi038.stdout:6/344: write d2/d4/d19/f47 [3747035,21044] 0 2024-02-15T18:34:55.220 INFO:tasks.workunit.client.0.smithi038.stdout:9/332: dwrite d1/da/d10/d24/f7c [0,4194304] 0 2024-02-15T18:34:55.220 INFO:tasks.workunit.client.0.smithi038.stdout:0/442: dwrite d4/d13/d19/f91 [0,4194304] 0 2024-02-15T18:34:55.221 INFO:tasks.workunit.client.0.smithi038.stdout:2/302: dwrite dd/d12/d1d/d3b/f4b [0,4194304] 0 2024-02-15T18:34:55.222 INFO:tasks.workunit.client.0.smithi038.stdout:4/346: dwrite d11/d1e/d20/d2b/f61 [0,4194304] 0 2024-02-15T18:34:55.222 INFO:tasks.workunit.client.0.smithi038.stdout:9/333: creat d1/da/d10/d13/d3d/d43/f84 x:0 0 0 2024-02-15T18:34:55.224 INFO:tasks.workunit.client.0.smithi038.stdout:8/490: dwrite d1/d4/d5/d31/d65/d80/fa0 [0,4194304] 0 2024-02-15T18:34:55.230 INFO:tasks.workunit.client.0.smithi038.stdout:0/443: dread d4/f63 [0,4194304] 0 2024-02-15T18:34:55.230 INFO:tasks.workunit.client.0.smithi038.stdout:0/444: truncate d4/d13/d2a/d35/d6a/f77 177933 0 2024-02-15T18:34:55.239 INFO:tasks.workunit.client.0.smithi038.stdout:8/491: creat d1/d7/fb0 x:0 0 0 2024-02-15T18:34:55.240 INFO:tasks.workunit.client.0.smithi038.stdout:0/445: creat d4/d13/d2a/d35/f93 x:0 0 0 2024-02-15T18:34:55.240 INFO:tasks.workunit.client.0.smithi038.stdout:9/334: symlink d1/dc/d5a/l85 0 2024-02-15T18:34:55.240 INFO:tasks.workunit.client.0.smithi038.stdout:9/335: write d1/da/d20/f6d [776124,65340] 0 2024-02-15T18:34:55.241 INFO:tasks.workunit.client.0.smithi038.stdout:8/492: mkdir d1/d4/db1 0 2024-02-15T18:34:55.242 INFO:tasks.workunit.client.0.smithi038.stdout:9/336: unlink d1/da/d10/d13/f1e 0 2024-02-15T18:34:55.242 INFO:tasks.workunit.client.0.smithi038.stdout:9/337: chown d1/da/d10/d24/d49 1463450 1 2024-02-15T18:34:55.242 INFO:tasks.workunit.client.0.smithi038.stdout:8/493: creat d1/d7/d15/fb2 x:0 0 0 2024-02-15T18:34:55.244 INFO:tasks.workunit.client.0.smithi038.stdout:8/494: write d1/d7/d22/f5c [1216968,90728] 0 2024-02-15T18:34:55.244 INFO:tasks.workunit.client.0.smithi038.stdout:8/495: stat d1/d7/f1d 0 2024-02-15T18:34:55.246 INFO:tasks.workunit.client.0.smithi038.stdout:8/496: mkdir d1/d4/d10/d43/d48/d52/d88/db3 0 2024-02-15T18:34:55.251 INFO:tasks.workunit.client.0.smithi038.stdout:4/347: dwrite d11/d1e/d20/f76 [0,4194304] 0 2024-02-15T18:34:55.251 INFO:tasks.workunit.client.0.smithi038.stdout:4/348: fdatasync d11/d1e/d20/f70 0 2024-02-15T18:34:55.251 INFO:tasks.workunit.client.0.smithi038.stdout:5/329: rmdir d2/d3/d12 39 2024-02-15T18:34:55.251 INFO:tasks.workunit.client.0.smithi038.stdout:5/330: stat d2/d3/f1d 0 2024-02-15T18:34:55.251 INFO:tasks.workunit.client.0.smithi038.stdout:1/359: dwrite d2/d10/d19/d22/d36/f44 [0,4194304] 0 2024-02-15T18:34:55.251 INFO:tasks.workunit.client.0.smithi038.stdout:2/303: dwrite dd/d12/d1d/f50 [4194304,4194304] 0 2024-02-15T18:34:55.251 INFO:tasks.workunit.client.0.smithi038.stdout:2/304: fsync dd/d12/f26 0 2024-02-15T18:34:55.251 INFO:tasks.workunit.client.0.smithi038.stdout:2/305: write dd/d12/d59/d1f/f30 [853846,126353] 0 2024-02-15T18:34:55.252 INFO:tasks.workunit.client.0.smithi038.stdout:6/345: dwrite d2/d4/fc [0,4194304] 0 2024-02-15T18:34:55.252 INFO:tasks.workunit.client.0.smithi038.stdout:6/346: readlink d2/d4/d28/l34 0 2024-02-15T18:34:55.252 INFO:tasks.workunit.client.0.smithi038.stdout:1/360: write d2/d10/d19/d22/d36/f4d [302456,95772] 0 2024-02-15T18:34:55.257 INFO:tasks.workunit.client.0.smithi038.stdout:4/349: creat d11/d1e/d3e/d57/f82 x:0 0 0 2024-02-15T18:34:55.257 INFO:tasks.workunit.client.0.smithi038.stdout:4/350: fdatasync d11/d1e/d20/d2f/f3d 0 2024-02-15T18:34:55.258 INFO:tasks.workunit.client.0.smithi038.stdout:5/331: mkdir d2/d64 0 2024-02-15T18:34:55.260 INFO:tasks.workunit.client.0.smithi038.stdout:2/306: truncate dd/d12/d59/f21 1804157 0 2024-02-15T18:34:55.261 INFO:tasks.workunit.client.0.smithi038.stdout:6/347: rename d2/d4/d2c/d39/f5d to d2/d4/d28/f62 0 2024-02-15T18:34:55.262 INFO:tasks.workunit.client.0.smithi038.stdout:2/307: mkdir dd/d12/d1d/d5f 0 2024-02-15T18:34:55.263 INFO:tasks.workunit.client.0.smithi038.stdout:9/338: dwrite d1/da/d10/d13/d3d/d43/f70 [0,4194304] 0 2024-02-15T18:34:55.264 INFO:tasks.workunit.client.0.smithi038.stdout:4/351: rename d11/d1e/c2a to d11/d1e/d43/d5e/d62/c83 0 2024-02-15T18:34:55.265 INFO:tasks.workunit.client.0.smithi038.stdout:6/348: mknod d2/d4/d19/d55/d5c/c63 0 2024-02-15T18:34:55.267 INFO:tasks.workunit.client.0.smithi038.stdout:2/308: creat dd/d12/d59/d1f/d4f/d5a/f60 x:0 0 0 2024-02-15T18:34:55.269 INFO:tasks.workunit.client.0.smithi038.stdout:9/339: link d1/f14 d1/da/d10/d13/f86 0 2024-02-15T18:34:55.269 INFO:tasks.workunit.client.0.smithi038.stdout:9/340: chown d1/da/d36/d50/c6e 196795732 1 2024-02-15T18:34:55.269 INFO:tasks.workunit.client.0.smithi038.stdout:9/341: read - d1/dc/d4f/f62 zero size 2024-02-15T18:34:55.269 INFO:tasks.workunit.client.0.smithi038.stdout:7/359: sync 2024-02-15T18:34:55.273 INFO:tasks.workunit.client.0.smithi038.stdout:8/497: dwrite d1/d7/d15/fb2 [0,4194304] 0 2024-02-15T18:34:55.273 INFO:tasks.workunit.client.0.smithi038.stdout:5/332: rename d2/d20/c4f to d2/d30/d52/c65 0 2024-02-15T18:34:55.273 INFO:tasks.workunit.client.0.smithi038.stdout:4/352: rmdir d11/d74 39 2024-02-15T18:34:55.274 INFO:tasks.workunit.client.0.smithi038.stdout:4/353: chown d11/d1e/d20/d31/f32 585253888 1 2024-02-15T18:34:55.277 INFO:tasks.workunit.client.0.smithi038.stdout:1/361: dwrite d2/d10/d19/f1d [0,4194304] 0 2024-02-15T18:34:55.282 INFO:tasks.workunit.client.0.smithi038.stdout:6/349: dwrite d2/d4/d2c/f4f [0,4194304] 0 2024-02-15T18:34:55.287 INFO:tasks.workunit.client.0.smithi038.stdout:2/309: creat dd/d12/d1d/d3b/d3c/f61 x:0 0 0 2024-02-15T18:34:55.287 INFO:tasks.workunit.client.0.smithi038.stdout:2/310: write dd/d12/d1d/d3b/d3c/f48 [1239820,103399] 0 2024-02-15T18:34:55.288 INFO:tasks.workunit.client.0.smithi038.stdout:9/342: creat d1/da/d10/d24/f87 x:0 0 0 2024-02-15T18:34:55.289 INFO:tasks.workunit.client.0.smithi038.stdout:7/360: creat d0/db/f7c x:0 0 0 2024-02-15T18:34:55.289 INFO:tasks.workunit.client.0.smithi038.stdout:7/361: read d0/db/ff [51885,16223] 0 2024-02-15T18:34:55.289 INFO:tasks.workunit.client.0.smithi038.stdout:7/362: getdents d0/d68 0 2024-02-15T18:34:55.289 INFO:tasks.workunit.client.0.smithi038.stdout:7/363: write d0/d1b/f78 [4422200,53632] 0 2024-02-15T18:34:55.290 INFO:tasks.workunit.client.0.smithi038.stdout:9/343: write d1/f16 [268532,63094] 0 2024-02-15T18:34:55.292 INFO:tasks.workunit.client.0.smithi038.stdout:8/498: symlink d1/d4/d5/d6/d6d/lb4 0 2024-02-15T18:34:55.292 INFO:tasks.workunit.client.0.smithi038.stdout:8/499: chown d1/d4/d10/d43/d48/d9a/cad 59308839 1 2024-02-15T18:34:55.292 INFO:tasks.workunit.client.0.smithi038.stdout:4/354: mknod d11/d1e/d20/c84 0 2024-02-15T18:34:55.293 INFO:tasks.workunit.client.0.smithi038.stdout:1/362: unlink d2/d10/d62/f68 0 2024-02-15T18:34:55.294 INFO:tasks.workunit.client.0.smithi038.stdout:2/311: mkdir dd/d12/d59/d62 0 2024-02-15T18:34:55.295 INFO:tasks.workunit.client.0.smithi038.stdout:9/344: mkdir d1/da/d10/d82/d88 0 2024-02-15T18:34:55.295 INFO:tasks.workunit.client.0.smithi038.stdout:9/345: stat d1/da/d10/d24/f53 0 2024-02-15T18:34:55.295 INFO:tasks.workunit.client.0.smithi038.stdout:8/500: symlink d1/d7/d22/da7/lb5 0 2024-02-15T18:34:55.296 INFO:tasks.workunit.client.0.smithi038.stdout:4/355: symlink d11/d1e/l85 0 2024-02-15T18:34:55.297 INFO:tasks.workunit.client.0.smithi038.stdout:1/363: rmdir d2/d10/d19/d22/d36 39 2024-02-15T18:34:55.298 INFO:tasks.workunit.client.0.smithi038.stdout:9/346: truncate d1/da/d10/d13/d3d/f5c 383193 0 2024-02-15T18:34:55.298 INFO:tasks.workunit.client.0.smithi038.stdout:9/347: fdatasync d1/f1c 0 2024-02-15T18:34:55.298 INFO:tasks.workunit.client.0.smithi038.stdout:9/348: read - d1/dc/d38/f6f zero size 2024-02-15T18:34:55.298 INFO:tasks.workunit.client.0.smithi038.stdout:1/364: symlink d2/d10/d32/d53/d82/l8a 0 2024-02-15T18:34:55.300 INFO:tasks.workunit.client.0.smithi038.stdout:1/365: rmdir d2/d2e 39 2024-02-15T18:34:55.301 INFO:tasks.workunit.client.0.smithi038.stdout:5/333: dwrite d2/d30/d52/f5b [0,4194304] 0 2024-02-15T18:34:55.303 INFO:tasks.workunit.client.0.smithi038.stdout:6/350: dwrite d2/d4/fc [4194304,4194304] 0 2024-02-15T18:34:55.310 INFO:tasks.workunit.client.0.smithi038.stdout:3/302: sync 2024-02-15T18:34:55.318 INFO:tasks.workunit.client.0.smithi038.stdout:5/334: getdents d2/d20 0 2024-02-15T18:34:55.318 INFO:tasks.workunit.client.0.smithi038.stdout:6/351: creat d2/d4/d19/d55/d5c/f64 x:0 0 0 2024-02-15T18:34:55.319 INFO:tasks.workunit.client.0.smithi038.stdout:3/303: rmdir d2/d1b/d3c/d3a 39 2024-02-15T18:34:55.319 INFO:tasks.workunit.client.0.smithi038.stdout:1/366: dread d2/d10/f31 [0,4194304] 0 2024-02-15T18:34:55.331 INFO:tasks.workunit.client.0.smithi038.stdout:8/501: dwrite d1/d4/d5/f23 [4194304,4194304] 0 2024-02-15T18:34:55.334 INFO:tasks.workunit.client.0.smithi038.stdout:2/312: dwrite dd/d12/d59/d1f/f30 [0,4194304] 0 2024-02-15T18:34:55.334 INFO:tasks.workunit.client.0.smithi038.stdout:2/313: stat dd/d12/d1d/d3b/l43 0 2024-02-15T18:34:55.334 INFO:tasks.workunit.client.0.smithi038.stdout:4/356: dwrite d11/d1e/d3e/f54 [0,4194304] 0 2024-02-15T18:34:55.334 INFO:tasks.workunit.client.0.smithi038.stdout:5/335: write d2/d20/f5a [608584,123772] 0 2024-02-15T18:34:55.334 INFO:tasks.workunit.client.0.smithi038.stdout:8/502: mkdir d1/d3a/d58/db6 0 2024-02-15T18:34:55.336 INFO:tasks.workunit.client.0.smithi038.stdout:9/349: dwrite f0 [0,4194304] 0 2024-02-15T18:34:55.337 INFO:tasks.workunit.client.0.smithi038.stdout:2/314: mknod dd/d12/d1d/d3b/c63 0 2024-02-15T18:34:55.338 INFO:tasks.workunit.client.0.smithi038.stdout:6/352: truncate d2/d4/d2c/f4f 3397890 0 2024-02-15T18:34:55.338 INFO:tasks.workunit.client.0.smithi038.stdout:0/446: sync 2024-02-15T18:34:55.338 INFO:tasks.workunit.client.0.smithi038.stdout:0/447: write d4/d13/d2a/d35/d1e/f6f [940592,16476] 0 2024-02-15T18:34:55.339 INFO:tasks.workunit.client.0.smithi038.stdout:4/357: link d11/d1e/d3e/d75/c2c d11/d1e/d20/d2b/d4d/d67/d73/c86 0 2024-02-15T18:34:55.341 INFO:tasks.workunit.client.0.smithi038.stdout:8/503: mkdir d1/d4/d10/d3d/db7 0 2024-02-15T18:34:55.341 INFO:tasks.workunit.client.0.smithi038.stdout:9/350: creat d1/da/d10/d13/f89 x:0 0 0 2024-02-15T18:34:55.344 INFO:tasks.workunit.client.0.smithi038.stdout:1/367: dwrite d2/d10/d19/d22/d36/f4d [0,4194304] 0 2024-02-15T18:34:55.344 INFO:tasks.workunit.client.0.smithi038.stdout:2/315: rename dd/d12/d1d/d3b/d3c/f61 to dd/d12/d59/d1f/d4f/f64 0 2024-02-15T18:34:55.344 INFO:tasks.workunit.client.0.smithi038.stdout:6/353: creat d2/d56/f65 x:0 0 0 2024-02-15T18:34:55.344 INFO:tasks.workunit.client.0.smithi038.stdout:6/354: read - d2/f58 zero size 2024-02-15T18:34:55.346 INFO:tasks.workunit.client.0.smithi038.stdout:0/448: mknod d4/d13/d2a/c94 0 2024-02-15T18:34:55.347 INFO:tasks.workunit.client.0.smithi038.stdout:3/304: dwrite d2/f18 [0,4194304] 0 2024-02-15T18:34:55.350 INFO:tasks.workunit.client.0.smithi038.stdout:5/336: truncate d2/d30/d52/f5b 3568907 0 2024-02-15T18:34:55.350 INFO:tasks.workunit.client.0.smithi038.stdout:8/504: creat d1/d4/d5/d6/d6d/fb8 x:0 0 0 2024-02-15T18:34:55.351 INFO:tasks.workunit.client.0.smithi038.stdout:9/351: mkdir d1/da/d10/d13/d3d/d43/d4d/d8a 0 2024-02-15T18:34:55.353 INFO:tasks.workunit.client.0.smithi038.stdout:1/368: getdents d2/d2e 0 2024-02-15T18:34:55.354 INFO:tasks.workunit.client.0.smithi038.stdout:2/316: mkdir dd/d12/d3a/d5e/d65 0 2024-02-15T18:34:55.358 INFO:tasks.workunit.client.0.smithi038.stdout:4/358: dwrite d11/d1e/d20/f76 [0,4194304] 0 2024-02-15T18:34:55.358 INFO:tasks.workunit.client.0.smithi038.stdout:4/359: chown d11/d1e/d20/f76 1854 1 2024-02-15T18:34:55.358 INFO:tasks.workunit.client.0.smithi038.stdout:5/337: creat d2/d20/d33/f66 x:0 0 0 2024-02-15T18:34:55.360 INFO:tasks.workunit.client.0.smithi038.stdout:8/505: mkdir d1/d4/d5/d6/db9 0 2024-02-15T18:34:55.360 INFO:tasks.workunit.client.0.smithi038.stdout:8/506: chown d1/d4/d5/d6/d6d/c63 0 1 2024-02-15T18:34:55.360 INFO:tasks.workunit.client.0.smithi038.stdout:8/507: fsync d1/f1b 0 2024-02-15T18:34:55.360 INFO:tasks.workunit.client.0.smithi038.stdout:8/508: chown d1/d7/d15/d4b/c73 1 1 2024-02-15T18:34:55.361 INFO:tasks.workunit.client.0.smithi038.stdout:4/360: write d11/f1f [2602316,40070] 0 2024-02-15T18:34:55.361 INFO:tasks.workunit.client.0.smithi038.stdout:4/361: chown d11/d1e/d20/d2b/f30 0 1 2024-02-15T18:34:55.363 INFO:tasks.workunit.client.0.smithi038.stdout:9/352: mkdir d1/da/d36/d8b 0 2024-02-15T18:34:55.363 INFO:tasks.workunit.client.0.smithi038.stdout:9/353: read - d1/da/d10/d13/f61 zero size 2024-02-15T18:34:55.363 INFO:tasks.workunit.client.0.smithi038.stdout:1/369: mkdir d2/d69/d8b 0 2024-02-15T18:34:55.365 INFO:tasks.workunit.client.0.smithi038.stdout:2/317: truncate dd/d12/f47 2463801 0 2024-02-15T18:34:55.366 INFO:tasks.workunit.client.0.smithi038.stdout:4/362: link d11/d1e/d20/d2b/c51 d11/d1e/d20/d2b/d4d/c87 0 2024-02-15T18:34:55.366 INFO:tasks.workunit.client.0.smithi038.stdout:8/509: creat d1/d4/d10/d43/d48/d52/fba x:0 0 0 2024-02-15T18:34:55.371 INFO:tasks.workunit.client.0.smithi038.stdout:2/318: truncate dd/d12/d3a/f45 90932 0 2024-02-15T18:34:55.371 INFO:tasks.workunit.client.0.smithi038.stdout:2/319: truncate dd/d12/d1d/f2e 8769626 0 2024-02-15T18:34:55.373 INFO:tasks.workunit.client.0.smithi038.stdout:4/363: rename d11/d1e/d43/l4c to d11/d1e/d3e/d57/l88 0 2024-02-15T18:34:55.373 INFO:tasks.workunit.client.0.smithi038.stdout:4/364: write d11/d1e/d20/f22 [4269849,77441] 0 2024-02-15T18:34:55.373 INFO:tasks.workunit.client.0.smithi038.stdout:4/365: read - d11/d1e/d43/f45 zero size 2024-02-15T18:34:55.374 INFO:tasks.workunit.client.0.smithi038.stdout:1/370: dread d2/d28/f34 [0,4194304] 0 2024-02-15T18:34:55.374 INFO:tasks.workunit.client.0.smithi038.stdout:1/371: chown d2/d10/d19/d22/d54/c84 2 1 2024-02-15T18:34:55.375 INFO:tasks.workunit.client.0.smithi038.stdout:8/510: mkdir d1/d4/dbb 0 2024-02-15T18:34:55.376 INFO:tasks.workunit.client.0.smithi038.stdout:2/320: mknod dd/d12/d59/d62/c66 0 2024-02-15T18:34:55.377 INFO:tasks.workunit.client.0.smithi038.stdout:4/366: creat d11/d1e/d20/d2b/d35/f89 x:0 0 0 2024-02-15T18:34:55.378 INFO:tasks.workunit.client.0.smithi038.stdout:1/372: mkdir d2/d2e/d8c 0 2024-02-15T18:34:55.378 INFO:tasks.workunit.client.0.smithi038.stdout:1/373: fdatasync d2/f1e 0 2024-02-15T18:34:55.378 INFO:tasks.workunit.client.0.smithi038.stdout:8/511: mknod d1/d4/d10/d91/cbc 0 2024-02-15T18:34:55.381 INFO:tasks.workunit.client.0.smithi038.stdout:1/374: write d2/d10/d19/d22/f30 [3218021,78781] 0 2024-02-15T18:34:55.381 INFO:tasks.workunit.client.0.smithi038.stdout:2/321: rename dd/d12/d59/d1f to dd/d12/d3a/d67 0 2024-02-15T18:34:55.381 INFO:tasks.workunit.client.0.smithi038.stdout:4/367: truncate d11/d1e/f23 2613960 0 2024-02-15T18:34:55.383 INFO:tasks.workunit.client.0.smithi038.stdout:1/375: rename d2/d10/d19/d22/d36/d4f/l6a to d2/d10/d19/d22/d36/d4f/l8d 0 2024-02-15T18:34:55.383 INFO:tasks.workunit.client.0.smithi038.stdout:1/376: write d2/d10/d19/d22/d36/f78 [840396,60130] 0 2024-02-15T18:34:55.383 INFO:tasks.workunit.client.0.smithi038.stdout:2/322: rmdir dd/d12/d3a/d67/d4f/d5a 39 2024-02-15T18:34:55.385 INFO:tasks.workunit.client.0.smithi038.stdout:4/368: mknod d11/d1e/d43/d5e/d62/c8a 0 2024-02-15T18:34:55.386 INFO:tasks.workunit.client.0.smithi038.stdout:1/377: link d2/d28/f34 d2/d69/f8e 0 2024-02-15T18:34:55.386 INFO:tasks.workunit.client.0.smithi038.stdout:2/323: mknod dd/d12/d3a/d5e/c68 0 2024-02-15T18:34:55.386 INFO:tasks.workunit.client.0.smithi038.stdout:2/324: fdatasync dd/d12/d1d/d3b/f4b 0 2024-02-15T18:34:55.389 INFO:tasks.workunit.client.0.smithi038.stdout:1/378: rename d2/d69/f8e to d2/d39/f8f 0 2024-02-15T18:34:55.389 INFO:tasks.workunit.client.0.smithi038.stdout:1/379: truncate d2/d10/d19/d22/d36/f44 4898441 0 2024-02-15T18:34:55.389 INFO:tasks.workunit.client.0.smithi038.stdout:4/369: dread d11/d1e/f29 [0,4194304] 0 2024-02-15T18:34:55.390 INFO:tasks.workunit.client.0.smithi038.stdout:6/355: dwrite d2/d4/d19/d55/d5c/f64 [0,4194304] 0 2024-02-15T18:34:55.391 INFO:tasks.workunit.client.0.smithi038.stdout:4/370: mkdir d11/d1e/d20/d2b/d4d/d67/d73/d8b 0 2024-02-15T18:34:55.393 INFO:tasks.workunit.client.0.smithi038.stdout:4/371: symlink d11/d1e/d20/d31/d48/l8c 0 2024-02-15T18:34:55.394 INFO:tasks.workunit.client.0.smithi038.stdout:3/305: dwrite d2/d25/f26 [0,4194304] 0 2024-02-15T18:34:55.395 INFO:tasks.workunit.client.0.smithi038.stdout:0/449: dwrite d4/d13/f39 [0,4194304] 0 2024-02-15T18:34:55.395 INFO:tasks.workunit.client.0.smithi038.stdout:5/338: dwrite d2/f1a [0,4194304] 0 2024-02-15T18:34:55.395 INFO:tasks.workunit.client.0.smithi038.stdout:9/354: dwrite d1/da/d10/d24/f53 [0,4194304] 0 2024-02-15T18:34:55.398 INFO:tasks.workunit.client.0.smithi038.stdout:4/372: dread d11/d1e/d20/d2b/f30 [0,4194304] 0 2024-02-15T18:34:55.405 INFO:tasks.workunit.client.0.smithi038.stdout:8/512: dwrite d1/d7/d15/d6a/f86 [0,4194304] 0 2024-02-15T18:34:55.411 INFO:tasks.workunit.client.0.smithi038.stdout:3/306: dread d2/d1e/f32 [0,4194304] 0 2024-02-15T18:34:55.412 INFO:tasks.workunit.client.0.smithi038.stdout:0/450: creat d4/d13/d53/f95 x:0 0 0 2024-02-15T18:34:55.418 INFO:tasks.workunit.client.0.smithi038.stdout:5/339: symlink d2/d20/d33/l67 0 2024-02-15T18:34:55.419 INFO:tasks.workunit.client.0.smithi038.stdout:5/340: chown d2/c34 3 1 2024-02-15T18:34:55.421 INFO:tasks.workunit.client.0.smithi038.stdout:1/380: dwrite d2/d28/d3c/f63 [0,4194304] 0 2024-02-15T18:34:55.421 INFO:tasks.workunit.client.0.smithi038.stdout:1/381: fsync d2/d10/d19/d22/d36/f4d 0 2024-02-15T18:34:55.421 INFO:tasks.workunit.client.0.smithi038.stdout:1/382: dread - d2/d39/f48 zero size 2024-02-15T18:34:55.421 INFO:tasks.workunit.client.0.smithi038.stdout:1/383: fdatasync d2/d10/d32/d53/f58 0 2024-02-15T18:34:55.422 INFO:tasks.workunit.client.0.smithi038.stdout:9/355: getdents d1/da/d10/d19 0 2024-02-15T18:34:55.423 INFO:tasks.workunit.client.0.smithi038.stdout:9/356: dread - d1/dc/d4f/f62 zero size 2024-02-15T18:34:55.423 INFO:tasks.workunit.client.0.smithi038.stdout:6/356: dwrite d2/d56/f65 [0,4194304] 0 2024-02-15T18:34:55.425 INFO:tasks.workunit.client.0.smithi038.stdout:3/307: mkdir d2/d25/d60 0 2024-02-15T18:34:55.426 INFO:tasks.workunit.client.0.smithi038.stdout:3/308: write d2/d1e/f32 [4613877,3697] 0 2024-02-15T18:34:55.427 INFO:tasks.workunit.client.0.smithi038.stdout:8/513: dread d1/d4/d10/f1e [0,4194304] 0 2024-02-15T18:34:55.427 INFO:tasks.workunit.client.0.smithi038.stdout:8/514: dread - d1/d4/d5/d31/d65/d80/fac zero size 2024-02-15T18:34:55.427 INFO:tasks.workunit.client.0.smithi038.stdout:5/341: creat d2/d20/d33/f68 x:0 0 0 2024-02-15T18:34:55.428 INFO:tasks.workunit.client.0.smithi038.stdout:4/373: dread d11/f1d [0,4194304] 0 2024-02-15T18:34:55.428 INFO:tasks.workunit.client.0.smithi038.stdout:0/451: dwrite d4/d13/d2a/d35/f8f [0,4194304] 0 2024-02-15T18:34:55.432 INFO:tasks.workunit.client.0.smithi038.stdout:9/357: getdents d1/da 0 2024-02-15T18:34:55.432 INFO:tasks.workunit.client.0.smithi038.stdout:9/358: truncate d1/da/d10/d13/f7a 83786 0 2024-02-15T18:34:55.433 INFO:tasks.workunit.client.0.smithi038.stdout:7/364: sync 2024-02-15T18:34:55.437 INFO:tasks.workunit.client.0.smithi038.stdout:8/515: creat d1/d4/d10/d43/d48/d52/fbd x:0 0 0 2024-02-15T18:34:55.437 INFO:tasks.workunit.client.0.smithi038.stdout:8/516: write d1/d3a/f7a [1047439,76513] 0 2024-02-15T18:34:55.440 INFO:tasks.workunit.client.0.smithi038.stdout:4/374: mkdir d11/d1e/d20/d31/d8d 0 2024-02-15T18:34:55.440 INFO:tasks.workunit.client.0.smithi038.stdout:5/342: creat d2/f69 x:0 0 0 2024-02-15T18:34:55.441 INFO:tasks.workunit.client.0.smithi038.stdout:4/375: write d11/d1e/d20/f6b [843102,47691] 0 2024-02-15T18:34:55.441 INFO:tasks.workunit.client.0.smithi038.stdout:0/452: creat d4/d13/d2a/d35/d1e/d32/d79/d7d/f96 x:0 0 0 2024-02-15T18:34:55.441 INFO:tasks.workunit.client.0.smithi038.stdout:0/453: fdatasync d4/d13/d2a/d35/f93 0 2024-02-15T18:34:55.441 INFO:tasks.workunit.client.0.smithi038.stdout:0/454: fdatasync d4/d13/d2a/d35/d1e/f6f 0 2024-02-15T18:34:55.442 INFO:tasks.workunit.client.0.smithi038.stdout:7/365: mkdir d0/db/d19/d7d 0 2024-02-15T18:34:55.442 INFO:tasks.workunit.client.0.smithi038.stdout:7/366: readlink d0/db/d19/l28 0 2024-02-15T18:34:55.442 INFO:tasks.workunit.client.0.smithi038.stdout:7/367: write d0/db/ff [942688,117042] 0 2024-02-15T18:34:55.442 INFO:tasks.workunit.client.0.smithi038.stdout:8/517: creat d1/d4/d5/d77/d79/fbe x:0 0 0 2024-02-15T18:34:55.444 INFO:tasks.workunit.client.0.smithi038.stdout:5/343: creat d2/d30/d52/f6a x:0 0 0 2024-02-15T18:34:55.444 INFO:tasks.workunit.client.0.smithi038.stdout:5/344: write d2/d20/d33/f66 [1042465,108658] 0 2024-02-15T18:34:55.444 INFO:tasks.workunit.client.0.smithi038.stdout:4/376: creat d11/d1e/d20/d2b/d4d/d67/f8e x:0 0 0 2024-02-15T18:34:55.445 INFO:tasks.workunit.client.0.smithi038.stdout:0/455: mkdir d4/d13/d2a/d35/d1e/d32/d48/d97 0 2024-02-15T18:34:55.446 INFO:tasks.workunit.client.0.smithi038.stdout:4/377: mkdir d11/d1e/d20/d31/d7a/d8f 0 2024-02-15T18:34:55.446 INFO:tasks.workunit.client.0.smithi038.stdout:4/378: read - d11/d1e/d43/f63 zero size 2024-02-15T18:34:55.446 INFO:tasks.workunit.client.0.smithi038.stdout:0/456: dread d4/d13/f64 [0,4194304] 0 2024-02-15T18:34:55.446 INFO:tasks.workunit.client.0.smithi038.stdout:5/345: dread d2/f38 [0,4194304] 0 2024-02-15T18:34:55.447 INFO:tasks.workunit.client.0.smithi038.stdout:4/379: creat d11/d1e/d3e/d57/f90 x:0 0 0 2024-02-15T18:34:55.447 INFO:tasks.workunit.client.0.smithi038.stdout:5/346: symlink d2/d3/d12/l6b 0 2024-02-15T18:34:55.448 INFO:tasks.workunit.client.0.smithi038.stdout:5/347: symlink d2/l6c 0 2024-02-15T18:34:55.451 INFO:tasks.workunit.client.0.smithi038.stdout:5/348: rename d2/d3/d12/f62 to d2/f6d 0 2024-02-15T18:34:55.452 INFO:tasks.workunit.client.0.smithi038.stdout:5/349: write d2/f3f [784566,72752] 0 2024-02-15T18:34:55.454 INFO:tasks.workunit.client.0.smithi038.stdout:8/518: dread d1/d7/d26/f34 [0,4194304] 0 2024-02-15T18:34:55.454 INFO:tasks.workunit.client.0.smithi038.stdout:8/519: stat d1/d4/d5/d6/d6d/fb8 0 2024-02-15T18:34:55.454 INFO:tasks.workunit.client.0.smithi038.stdout:8/520: dread - d1/d4/d10/d43/d48/d52/fbd zero size 2024-02-15T18:34:55.454 INFO:tasks.workunit.client.0.smithi038.stdout:8/521: write d1/d4/d5/d31/d65/d80/fac [979287,42069] 0 2024-02-15T18:34:55.459 INFO:tasks.workunit.client.0.smithi038.stdout:9/359: dread d1/dc/f28 [0,4194304] 0 2024-02-15T18:34:55.459 INFO:tasks.workunit.client.0.smithi038.stdout:5/350: mknod d2/d3/d5f/c6e 0 2024-02-15T18:34:55.460 INFO:tasks.workunit.client.0.smithi038.stdout:9/360: write d1/da/f17 [2443761,109535] 0 2024-02-15T18:34:55.461 INFO:tasks.workunit.client.0.smithi038.stdout:8/522: unlink d1/d4/d10/d43/d48/fa2 0 2024-02-15T18:34:55.461 INFO:tasks.workunit.client.0.smithi038.stdout:5/351: rmdir d2/d3/d12/d5e 39 2024-02-15T18:34:55.462 INFO:tasks.workunit.client.0.smithi038.stdout:9/361: creat d1/da/d20/d58/d76/f8c x:0 0 0 2024-02-15T18:34:55.462 INFO:tasks.workunit.client.0.smithi038.stdout:3/309: dwrite d2/f18 [4194304,4194304] 0 2024-02-15T18:34:55.463 INFO:tasks.workunit.client.0.smithi038.stdout:3/310: fdatasync d2/f51 0 2024-02-15T18:34:55.464 INFO:tasks.workunit.client.0.smithi038.stdout:5/352: unlink d2/f38 0 2024-02-15T18:34:55.464 INFO:tasks.workunit.client.0.smithi038.stdout:0/457: write d4/d13/d19/f5c [3281551,119434] 0 2024-02-15T18:34:55.464 INFO:tasks.workunit.client.0.smithi038.stdout:6/357: dwrite d2/d4/d2c/f4f [0,4194304] 0 2024-02-15T18:34:55.464 INFO:tasks.workunit.client.0.smithi038.stdout:6/358: stat d2/f58 0 2024-02-15T18:34:55.465 INFO:tasks.workunit.client.0.smithi038.stdout:9/362: rename d1/dc/d4f/c31 to d1/da/c8d 0 2024-02-15T18:34:55.466 INFO:tasks.workunit.client.0.smithi038.stdout:5/353: creat d2/d3/f6f x:0 0 0 2024-02-15T18:34:55.467 INFO:tasks.workunit.client.0.smithi038.stdout:9/363: write d1/fb [2984131,111267] 0 2024-02-15T18:34:55.467 INFO:tasks.workunit.client.0.smithi038.stdout:6/359: rmdir d2/d4/d19 39 2024-02-15T18:34:55.468 INFO:tasks.workunit.client.0.smithi038.stdout:7/368: dwrite d0/db/d19/f4e [4194304,4194304] 0 2024-02-15T18:34:55.468 INFO:tasks.workunit.client.0.smithi038.stdout:7/369: chown d0/d1b/d30/d62/l6c 3 1 2024-02-15T18:34:55.468 INFO:tasks.workunit.client.0.smithi038.stdout:4/380: dwrite d11/d1e/d20/f22 [0,4194304] 0 2024-02-15T18:34:55.468 INFO:tasks.workunit.client.0.smithi038.stdout:0/458: rename d4/d13/d2a/d35/d1e/f6f to d4/d13/f98 0 2024-02-15T18:34:55.469 INFO:tasks.workunit.client.0.smithi038.stdout:0/459: fdatasync d4/f11 0 2024-02-15T18:34:55.475 INFO:tasks.workunit.client.0.smithi038.stdout:9/364: creat d1/da/d10/d13/d3d/d43/d4d/d8a/f8e x:0 0 0 2024-02-15T18:34:55.475 INFO:tasks.workunit.client.0.smithi038.stdout:8/523: getdents d1/d4/d10/d43/d48 0 2024-02-15T18:34:55.479 INFO:tasks.workunit.client.0.smithi038.stdout:5/354: creat d2/d30/d52/f70 x:0 0 0 2024-02-15T18:34:55.481 INFO:tasks.workunit.client.0.smithi038.stdout:4/381: creat d11/d1e/d20/d31/f91 x:0 0 0 2024-02-15T18:34:55.482 INFO:tasks.workunit.client.0.smithi038.stdout:6/360: dread d2/f3 [4194304,4194304] 0 2024-02-15T18:34:55.482 INFO:tasks.workunit.client.0.smithi038.stdout:4/382: chown d11/d1e/d3e/d57/l88 66738 1 2024-02-15T18:34:55.482 INFO:tasks.workunit.client.0.smithi038.stdout:6/361: write d2/d4/ff [4616261,83918] 0 2024-02-15T18:34:55.482 INFO:tasks.workunit.client.0.smithi038.stdout:3/311: dwrite d2/d25/f26 [4194304,4194304] 0 2024-02-15T18:34:55.482 INFO:tasks.workunit.client.0.smithi038.stdout:5/355: dread d2/d3/d12/fa [0,4194304] 0 2024-02-15T18:34:55.482 INFO:tasks.workunit.client.0.smithi038.stdout:3/312: readlink d2/d25/l35 0 2024-02-15T18:34:55.483 INFO:tasks.workunit.client.0.smithi038.stdout:7/370: rename d0/d66/d6a to d0/db/d59/d15/d29/d7e 0 2024-02-15T18:34:55.484 INFO:tasks.workunit.client.0.smithi038.stdout:0/460: unlink d4/d13/d2a/d35/d6a/l8a 0 2024-02-15T18:34:55.486 INFO:tasks.workunit.client.0.smithi038.stdout:9/365: rmdir d1/da/d10/d82/d88 0 2024-02-15T18:34:55.486 INFO:tasks.workunit.client.0.smithi038.stdout:9/366: write d1/da/d36/d3f/d48/f57 [712358,79836] 0 2024-02-15T18:34:55.488 INFO:tasks.workunit.client.0.smithi038.stdout:8/524: link d1/d4/d5/d6/f6e d1/d4/dbb/fbf 0 2024-02-15T18:34:55.489 INFO:tasks.workunit.client.0.smithi038.stdout:4/383: link d11/d1e/d3e/d75/f34 d11/d1e/d43/d5e/d62/f92 0 2024-02-15T18:34:55.489 INFO:tasks.workunit.client.0.smithi038.stdout:4/384: fsync d11/d1e/d20/d31/f5b 0 2024-02-15T18:34:55.490 INFO:tasks.workunit.client.0.smithi038.stdout:6/362: getdents d2/d4/d2c 0 2024-02-15T18:34:55.495 INFO:tasks.workunit.client.0.smithi038.stdout:5/356: dwrite d2/d20/f50 [0,4194304] 0 2024-02-15T18:34:55.500 INFO:tasks.workunit.client.0.smithi038.stdout:3/313: truncate d2/f1a 1287119 0 2024-02-15T18:34:55.500 INFO:tasks.workunit.client.0.smithi038.stdout:3/314: write d2/d1b/f36 [4432410,31085] 0 2024-02-15T18:34:55.502 INFO:tasks.workunit.client.0.smithi038.stdout:7/371: rename d0/d1b/c56 to d0/db/d59/d15/c7f 0 2024-02-15T18:34:55.502 INFO:tasks.workunit.client.0.smithi038.stdout:0/461: rmdir d4/d13/d2a/d35/d1e/d32/d48 39 2024-02-15T18:34:55.503 INFO:tasks.workunit.client.0.smithi038.stdout:9/367: mknod d1/da/d36/c8f 0 2024-02-15T18:34:55.505 INFO:tasks.workunit.client.0.smithi038.stdout:8/525: symlink d1/d4/d10/d43/d48/d52/d88/db3/lc0 0 2024-02-15T18:34:55.506 INFO:tasks.workunit.client.0.smithi038.stdout:4/385: symlink d11/d1e/d20/l93 0 2024-02-15T18:34:55.508 INFO:tasks.workunit.client.0.smithi038.stdout:6/363: dwrite d2/d4/f27 [4194304,4194304] 0 2024-02-15T18:34:55.508 INFO:tasks.workunit.client.0.smithi038.stdout:6/364: chown d2/d4/d19/f47 368 1 2024-02-15T18:34:55.508 INFO:tasks.workunit.client.0.smithi038.stdout:6/365: fdatasync d2/d4/f27 0 2024-02-15T18:34:55.508 INFO:tasks.workunit.client.0.smithi038.stdout:6/366: fsync d2/d4/d2c/f37 0 2024-02-15T18:34:55.508 INFO:tasks.workunit.client.0.smithi038.stdout:3/315: rmdir d2/d24 39 2024-02-15T18:34:55.508 INFO:tasks.workunit.client.0.smithi038.stdout:3/316: dread - d2/d1b/d3c/d41/f56 zero size 2024-02-15T18:34:55.509 INFO:tasks.workunit.client.0.smithi038.stdout:2/325: sync 2024-02-15T18:34:55.511 INFO:tasks.workunit.client.0.smithi038.stdout:0/462: rename d4/lc to d4/d13/d2a/d35/d1e/d32/d79/l99 0 2024-02-15T18:34:55.514 INFO:tasks.workunit.client.0.smithi038.stdout:9/368: creat d1/da/d20/d58/d66/f90 x:0 0 0 2024-02-15T18:34:55.514 INFO:tasks.workunit.client.0.smithi038.stdout:9/369: write d1/da/d10/f33 [645233,85690] 0 2024-02-15T18:34:55.514 INFO:tasks.workunit.client.0.smithi038.stdout:9/370: chown d1/da/d10/d13/d3d/d43/d4d/d8a/f8e 8 1 2024-02-15T18:34:55.514 INFO:tasks.workunit.client.0.smithi038.stdout:9/371: stat d1/dc/d2b/f4a 0 2024-02-15T18:34:55.514 INFO:tasks.workunit.client.0.smithi038.stdout:9/372: read d1/f14 [543867,107543] 0 2024-02-15T18:34:55.515 INFO:tasks.workunit.client.0.smithi038.stdout:7/372: dwrite d0/d2e/f37 [0,4194304] 0 2024-02-15T18:34:55.516 INFO:tasks.workunit.client.0.smithi038.stdout:4/386: creat d11/d1e/d3e/f94 x:0 0 0 2024-02-15T18:34:55.516 INFO:tasks.workunit.client.0.smithi038.stdout:4/387: read - d11/d1e/d3e/f94 zero size 2024-02-15T18:34:55.516 INFO:tasks.workunit.client.0.smithi038.stdout:1/384: sync 2024-02-15T18:34:55.519 INFO:tasks.workunit.client.0.smithi038.stdout:6/367: mknod d2/d56/c66 0 2024-02-15T18:34:55.520 INFO:tasks.workunit.client.0.smithi038.stdout:3/317: mknod d2/d1b/d3c/d3a/d55/c61 0 2024-02-15T18:34:55.526 INFO:tasks.workunit.client.0.smithi038.stdout:4/388: creat d11/d1e/d20/d2b/d4d/f95 x:0 0 0 2024-02-15T18:34:55.526 INFO:tasks.workunit.client.0.smithi038.stdout:9/373: creat d1/da/d10/d24/f91 x:0 0 0 2024-02-15T18:34:55.526 INFO:tasks.workunit.client.0.smithi038.stdout:8/526: rename d1/d4/d5/d6/ff to d1/d7/d26/d69/fc1 0 2024-02-15T18:34:55.526 INFO:tasks.workunit.client.0.smithi038.stdout:8/527: write d1/d7/d26/f3e [414712,53507] 0 2024-02-15T18:34:55.526 INFO:tasks.workunit.client.0.smithi038.stdout:8/528: write d1/d4/f9e [4537085,14124] 0 2024-02-15T18:34:55.528 INFO:tasks.workunit.client.0.smithi038.stdout:3/318: mkdir d2/d1b/d3c/d41/d4e/d62 0 2024-02-15T18:34:55.529 INFO:tasks.workunit.client.0.smithi038.stdout:2/326: rename c0 to dd/d12/d5d/c69 0 2024-02-15T18:34:55.530 INFO:tasks.workunit.client.0.smithi038.stdout:4/389: mknod d11/d1e/d20/d2b/d4d/c96 0 2024-02-15T18:34:55.532 INFO:tasks.workunit.client.0.smithi038.stdout:8/529: rmdir d1/d3a/d58/db6 0 2024-02-15T18:34:55.533 INFO:tasks.workunit.client.0.smithi038.stdout:7/373: rename d0/d1b/d30/d62 to d0/d1b/d80 0 2024-02-15T18:34:55.534 INFO:tasks.workunit.client.0.smithi038.stdout:7/374: read d0/d66/f35 [745048,19381] 0 2024-02-15T18:34:55.534 INFO:tasks.workunit.client.0.smithi038.stdout:8/530: mknod d1/d7/d22/da7/cc2 0 2024-02-15T18:34:55.534 INFO:tasks.workunit.client.0.smithi038.stdout:8/531: getdents d1/d4/d10/d43/d48 0 2024-02-15T18:34:55.538 INFO:tasks.workunit.client.0.smithi038.stdout:4/390: dread d11/d1e/f23 [0,4194304] 0 2024-02-15T18:34:55.538 INFO:tasks.workunit.client.0.smithi038.stdout:2/327: rename dd/d12/d3a/d67/d4f/d58 to dd/d12/d59/d2b/d6a 0 2024-02-15T18:34:55.538 INFO:tasks.workunit.client.0.smithi038.stdout:9/374: dread d1/da/d10/d13/d3d/d4b/f72 [0,4194304] 0 2024-02-15T18:34:55.538 INFO:tasks.workunit.client.0.smithi038.stdout:9/375: rename d1 to d1/dc/d5a/d7f/d92 22 2024-02-15T18:34:55.538 INFO:tasks.workunit.client.0.smithi038.stdout:7/375: truncate d0/d66/f1c 1793723 0 2024-02-15T18:34:55.538 INFO:tasks.workunit.client.0.smithi038.stdout:7/376: stat d0/d68 0 2024-02-15T18:34:55.538 INFO:tasks.workunit.client.0.smithi038.stdout:6/368: dwrite d2/f46 [0,4194304] 0 2024-02-15T18:34:55.538 INFO:tasks.workunit.client.0.smithi038.stdout:8/532: symlink d1/d4/d5/d6/db9/lc3 0 2024-02-15T18:34:55.539 INFO:tasks.workunit.client.0.smithi038.stdout:9/376: mkdir d1/da/d36/d93 0 2024-02-15T18:34:55.540 INFO:tasks.workunit.client.0.smithi038.stdout:7/377: mknod d0/d2e/c81 0 2024-02-15T18:34:55.541 INFO:tasks.workunit.client.0.smithi038.stdout:1/385: dwrite d2/d28/d3c/f63 [0,4194304] 0 2024-02-15T18:34:55.541 INFO:tasks.workunit.client.0.smithi038.stdout:4/391: rename d11/d74 to d11/d1e/d20/d97 0 2024-02-15T18:34:55.543 INFO:tasks.workunit.client.0.smithi038.stdout:8/533: symlink d1/d4/d10/d43/d48/d52/lc4 0 2024-02-15T18:34:55.544 INFO:tasks.workunit.client.0.smithi038.stdout:2/328: mkdir dd/d12/d6b 0 2024-02-15T18:34:55.545 INFO:tasks.workunit.client.0.smithi038.stdout:9/377: mkdir d1/da/d20/d58/d94 0 2024-02-15T18:34:55.546 INFO:tasks.workunit.client.0.smithi038.stdout:7/378: unlink d0/db/d59/d15/d29/c34 0 2024-02-15T18:34:55.551 INFO:tasks.workunit.client.0.smithi038.stdout:3/319: dwrite d2/f50 [0,4194304] 0 2024-02-15T18:34:55.554 INFO:tasks.workunit.client.0.smithi038.stdout:1/386: link d2/d10/d19/c59 d2/d10/d19/d2f/d3e/c90 0 2024-02-15T18:34:55.556 INFO:tasks.workunit.client.0.smithi038.stdout:8/534: dread d1/d4/d10/f1e [0,4194304] 0 2024-02-15T18:34:55.556 INFO:tasks.workunit.client.0.smithi038.stdout:7/379: dread d0/d1b/d30/d5d/d61/f75 [0,4194304] 0 2024-02-15T18:34:55.557 INFO:tasks.workunit.client.0.smithi038.stdout:1/387: write d2/d28/f34 [730986,111949] 0 2024-02-15T18:34:55.557 INFO:tasks.workunit.client.0.smithi038.stdout:1/388: dread - d2/d10/d62/f88 zero size 2024-02-15T18:34:55.559 INFO:tasks.workunit.client.0.smithi038.stdout:6/369: rename d2/d4/c31 to d2/d4/d2c/d39/c67 0 2024-02-15T18:34:55.559 INFO:tasks.workunit.client.0.smithi038.stdout:6/370: chown d2/d4/d28/f62 169 1 2024-02-15T18:34:55.560 INFO:tasks.workunit.client.0.smithi038.stdout:6/371: write d2/d10/f49 [1495780,43307] 0 2024-02-15T18:34:55.562 INFO:tasks.workunit.client.0.smithi038.stdout:9/378: mkdir d1/da/d10/d24/d49/d95 0 2024-02-15T18:34:55.562 INFO:tasks.workunit.client.0.smithi038.stdout:9/379: readlink d1/da/d10/d13/d3d/d43/d4d/l81 0 2024-02-15T18:34:55.563 INFO:tasks.workunit.client.0.smithi038.stdout:3/320: mknod d2/d1b/d3c/d42/c63 0 2024-02-15T18:34:55.563 INFO:tasks.workunit.client.0.smithi038.stdout:8/535: creat d1/d4/d5/d31/d65/d80/fc5 x:0 0 0 2024-02-15T18:34:55.564 INFO:tasks.workunit.client.0.smithi038.stdout:7/380: creat d0/d1b/d30/d5d/f82 x:0 0 0 2024-02-15T18:34:55.565 INFO:tasks.workunit.client.0.smithi038.stdout:1/389: mknod d2/d10/d62/c91 0 2024-02-15T18:34:55.566 INFO:tasks.workunit.client.0.smithi038.stdout:4/392: dwrite d11/d1e/d20/d2f/f3d [0,4194304] 0 2024-02-15T18:34:55.566 INFO:tasks.workunit.client.0.smithi038.stdout:4/393: chown d11/d1e/d20/c53 16609320 1 2024-02-15T18:34:55.567 INFO:tasks.workunit.client.0.smithi038.stdout:2/329: dwrite dd/d12/f36 [4194304,4194304] 0 2024-02-15T18:34:55.567 INFO:tasks.workunit.client.0.smithi038.stdout:2/330: fdatasync dd/f41 0 2024-02-15T18:34:55.568 INFO:tasks.workunit.client.0.smithi038.stdout:9/380: unlink d1/da/d10/d13/d3d/d43/c45 0 2024-02-15T18:34:55.568 INFO:tasks.workunit.client.0.smithi038.stdout:9/381: readlink d1/da/d20/l3c 0 2024-02-15T18:34:55.568 INFO:tasks.workunit.client.0.smithi038.stdout:9/382: read - d1/dc/d4f/f47 zero size 2024-02-15T18:34:55.568 INFO:tasks.workunit.client.0.smithi038.stdout:9/383: chown d1/da/f17 136247 1 2024-02-15T18:34:55.572 INFO:tasks.workunit.client.0.smithi038.stdout:3/321: creat d2/d25/f64 x:0 0 0 2024-02-15T18:34:55.573 INFO:tasks.workunit.client.0.smithi038.stdout:8/536: rename d1/d7/d15/d4b/c71 to d1/d4/d10/cc6 0 2024-02-15T18:34:55.574 INFO:tasks.workunit.client.0.smithi038.stdout:7/381: mknod d0/d66/c83 0 2024-02-15T18:34:55.575 INFO:tasks.workunit.client.0.smithi038.stdout:1/390: mkdir d2/d10/d19/d5f/d92 0 2024-02-15T18:34:55.575 INFO:tasks.workunit.client.0.smithi038.stdout:1/391: truncate d2/d10/d19/d5f/f70 737644 0 2024-02-15T18:34:55.575 INFO:tasks.workunit.client.0.smithi038.stdout:1/392: write d2/d10/d62/f88 [309649,9255] 0 2024-02-15T18:34:55.575 INFO:tasks.workunit.client.0.smithi038.stdout:1/393: chown d2/d10/f31 178419918 1 2024-02-15T18:34:55.577 INFO:tasks.workunit.client.0.smithi038.stdout:6/372: dwrite d2/f5a [0,4194304] 0 2024-02-15T18:34:55.577 INFO:tasks.workunit.client.0.smithi038.stdout:6/373: write d2/d4/d2c/f3c [5239808,114492] 0 2024-02-15T18:34:55.578 INFO:tasks.workunit.client.0.smithi038.stdout:2/331: creat dd/d12/d1d/f6c x:0 0 0 2024-02-15T18:34:55.582 INFO:tasks.workunit.client.0.smithi038.stdout:2/332: write fc [3686425,114559] 0 2024-02-15T18:34:55.585 INFO:tasks.workunit.client.0.smithi038.stdout:9/384: creat d1/da/d36/d8b/f96 x:0 0 0 2024-02-15T18:34:55.586 INFO:tasks.workunit.client.0.smithi038.stdout:3/322: mkdir d2/d1b/d3c/d3a/d55/d65 0 2024-02-15T18:34:55.586 INFO:tasks.workunit.client.0.smithi038.stdout:3/323: chown d2/l1c 1933617 1 2024-02-15T18:34:55.587 INFO:tasks.workunit.client.0.smithi038.stdout:3/324: truncate d2/f15 3737838 0 2024-02-15T18:34:55.587 INFO:tasks.workunit.client.0.smithi038.stdout:3/325: fsync d2/f38 0 2024-02-15T18:34:55.588 INFO:tasks.workunit.client.0.smithi038.stdout:4/394: rename d11/d1e/d20/f22 to d11/d1e/d20/d2f/d71/f98 0 2024-02-15T18:34:55.588 INFO:tasks.workunit.client.0.smithi038.stdout:4/395: getdents d11/d1e/d20/d31/d7a/d8f 0 2024-02-15T18:34:55.588 INFO:tasks.workunit.client.0.smithi038.stdout:8/537: dwrite d1/d4/d5/d31/f39 [0,4194304] 0 2024-02-15T18:34:55.589 INFO:tasks.workunit.client.0.smithi038.stdout:9/385: dread d1/f16 [0,4194304] 0 2024-02-15T18:34:55.589 INFO:tasks.workunit.client.0.smithi038.stdout:9/386: stat d1/da/d10/d13/d3d 0 2024-02-15T18:34:55.591 INFO:tasks.workunit.client.0.smithi038.stdout:1/394: creat d2/d10/d62/f93 x:0 0 0 2024-02-15T18:34:55.594 INFO:tasks.workunit.client.0.smithi038.stdout:6/374: creat d2/d56/d5f/f68 x:0 0 0 2024-02-15T18:34:55.594 INFO:tasks.workunit.client.0.smithi038.stdout:6/375: fsync d2/f9 0 2024-02-15T18:34:55.594 INFO:tasks.workunit.client.0.smithi038.stdout:6/376: chown d2/d56/d5f 3682 1 2024-02-15T18:34:55.598 INFO:tasks.workunit.client.0.smithi038.stdout:3/326: rmdir d2/d1b/d3c/d41 39 2024-02-15T18:34:55.602 INFO:tasks.workunit.client.0.smithi038.stdout:9/387: creat d1/da/d10/d13/d3d/d43/f97 x:0 0 0 2024-02-15T18:34:55.604 INFO:tasks.workunit.client.0.smithi038.stdout:1/395: rename d2/d10/d19/d2f/d85 to d2/d10/d19/d22/d36/d94 0 2024-02-15T18:34:55.604 INFO:tasks.workunit.client.0.smithi038.stdout:1/396: readlink d2/d10/l41 0 2024-02-15T18:34:55.604 INFO:tasks.workunit.client.0.smithi038.stdout:1/397: truncate d2/d10/d62/f93 848607 0 2024-02-15T18:34:55.605 INFO:tasks.workunit.client.0.smithi038.stdout:6/377: creat d2/d4/d28/f69 x:0 0 0 2024-02-15T18:34:55.605 INFO:tasks.workunit.client.0.smithi038.stdout:6/378: chown d2/d4/d19/d55/d5c/f64 2575698 1 2024-02-15T18:34:55.605 INFO:tasks.workunit.client.0.smithi038.stdout:6/379: stat d2/d4/d28/l34 0 2024-02-15T18:34:55.605 INFO:tasks.workunit.client.0.smithi038.stdout:3/327: creat d2/d25/f66 x:0 0 0 2024-02-15T18:34:55.605 INFO:tasks.workunit.client.0.smithi038.stdout:3/328: write d2/f51 [2190474,100129] 0 2024-02-15T18:34:55.605 INFO:tasks.workunit.client.0.smithi038.stdout:9/388: dread d1/da/d10/d13/d3d/d4b/f72 [0,4194304] 0 2024-02-15T18:34:55.606 INFO:tasks.workunit.client.0.smithi038.stdout:7/382: link d0/db/fc d0/d1b/d30/d5d/d61/f84 0 2024-02-15T18:34:55.606 INFO:tasks.workunit.client.0.smithi038.stdout:7/383: write d0/db/d19/f6d [1291644,118137] 0 2024-02-15T18:34:55.606 INFO:tasks.workunit.client.0.smithi038.stdout:7/384: chown d0/d1b/d30/d50/f72 17 1 2024-02-15T18:34:55.607 INFO:tasks.workunit.client.0.smithi038.stdout:3/329: write d2/d25/f26 [5717177,115038] 0 2024-02-15T18:34:55.607 INFO:tasks.workunit.client.0.smithi038.stdout:6/380: creat d2/d4/d19/d55/f6a x:0 0 0 2024-02-15T18:34:55.610 INFO:tasks.workunit.client.0.smithi038.stdout:8/538: dwrite d1/d4/d5/d6/d6d/fb8 [0,4194304] 0 2024-02-15T18:34:55.612 INFO:tasks.workunit.client.0.smithi038.stdout:4/396: dwrite d11/d1e/d20/d2f/f3d [0,4194304] 0 2024-02-15T18:34:55.612 INFO:tasks.workunit.client.0.smithi038.stdout:9/389: truncate d1/fb 3314363 0 2024-02-15T18:34:55.612 INFO:tasks.workunit.client.0.smithi038.stdout:9/390: fdatasync d1/da/d10/d13/d3d/d43/d4d/d8a/f8e 0 2024-02-15T18:34:55.612 INFO:tasks.workunit.client.0.smithi038.stdout:3/330: mknod d2/d1b/d3c/d41/d4e/c67 0 2024-02-15T18:34:55.614 INFO:tasks.workunit.client.0.smithi038.stdout:5/357: sync 2024-02-15T18:34:55.614 INFO:tasks.workunit.client.0.smithi038.stdout:5/358: stat d2/d3 0 2024-02-15T18:34:55.616 INFO:tasks.workunit.client.0.smithi038.stdout:5/359: read d2/f39 [382083,38961] 0 2024-02-15T18:34:55.616 INFO:tasks.workunit.client.0.smithi038.stdout:3/331: dread d2/d1e/f44 [0,4194304] 0 2024-02-15T18:34:55.616 INFO:tasks.workunit.client.0.smithi038.stdout:3/332: stat d2/d25/d60 0 2024-02-15T18:34:55.616 INFO:tasks.workunit.client.0.smithi038.stdout:3/333: chown d2/d1b/d3c/d41/d4e/d62 21526293 1 2024-02-15T18:34:55.616 INFO:tasks.workunit.client.0.smithi038.stdout:8/539: creat d1/d7/d15/d4b/fc7 x:0 0 0 2024-02-15T18:34:55.616 INFO:tasks.workunit.client.0.smithi038.stdout:8/540: chown d1/d7/l9 39 1 2024-02-15T18:34:55.616 INFO:tasks.workunit.client.0.smithi038.stdout:8/541: fdatasync d1/d3a/d58/f60 0 2024-02-15T18:34:55.623 INFO:tasks.workunit.client.0.smithi038.stdout:8/542: dread d1/d7/d15/d4b/f3f [0,4194304] 0 2024-02-15T18:34:55.623 INFO:tasks.workunit.client.0.smithi038.stdout:8/543: dread - d1/d7/fb0 zero size 2024-02-15T18:34:55.623 INFO:tasks.workunit.client.0.smithi038.stdout:8/544: readlink d1/d7/d15/d4b/l3b 0 2024-02-15T18:34:55.623 INFO:tasks.workunit.client.0.smithi038.stdout:8/545: truncate d1/f1b 679901 0 2024-02-15T18:34:55.623 INFO:tasks.workunit.client.0.smithi038.stdout:4/397: mkdir d11/d1e/d20/d31/d8d/d99 0 2024-02-15T18:34:55.624 INFO:tasks.workunit.client.0.smithi038.stdout:8/546: read d1/f50 [34122,19923] 0 2024-02-15T18:34:55.624 INFO:tasks.workunit.client.0.smithi038.stdout:6/381: dread d2/d56/f65 [0,4194304] 0 2024-02-15T18:34:55.624 INFO:tasks.workunit.client.0.smithi038.stdout:6/382: dread - d2/d56/f5e zero size 2024-02-15T18:34:55.624 INFO:tasks.workunit.client.0.smithi038.stdout:9/391: unlink d1/dc/d2b/f4a 0 2024-02-15T18:34:55.624 INFO:tasks.workunit.client.0.smithi038.stdout:9/392: dread - d1/da/d20/d58/d76/f8c zero size 2024-02-15T18:34:55.624 INFO:tasks.workunit.client.0.smithi038.stdout:9/393: dread - d1/dc/d38/f6f zero size 2024-02-15T18:34:55.626 INFO:tasks.workunit.client.0.smithi038.stdout:5/360: rename d2/d20 to d2/d3/d5f/d71 0 2024-02-15T18:34:55.627 INFO:tasks.workunit.client.0.smithi038.stdout:5/361: write d2/d30/f46 [1009642,104275] 0 2024-02-15T18:34:55.628 INFO:tasks.workunit.client.0.smithi038.stdout:9/394: symlink d1/dc/d5a/d7f/l98 0 2024-02-15T18:34:55.629 INFO:tasks.workunit.client.0.smithi038.stdout:1/398: dwrite d2/f2d [0,4194304] 0 2024-02-15T18:34:55.630 INFO:tasks.workunit.client.0.smithi038.stdout:5/362: rmdir d2/d30/d52 39 2024-02-15T18:34:55.630 INFO:tasks.workunit.client.0.smithi038.stdout:5/363: fdatasync d2/f1c 0 2024-02-15T18:34:55.632 INFO:tasks.workunit.client.0.smithi038.stdout:9/395: write d1/dc/d2b/f3a [2872472,125331] 0 2024-02-15T18:34:55.632 INFO:tasks.workunit.client.0.smithi038.stdout:9/396: chown d1/da/d10/d13/c1d 285938099 1 2024-02-15T18:34:55.632 INFO:tasks.workunit.client.0.smithi038.stdout:9/397: chown d1/da/d10/d13/d3d/d43/d4d 1102 1 2024-02-15T18:34:55.634 INFO:tasks.workunit.client.0.smithi038.stdout:3/334: write d2/f38 [273167,73269] 0 2024-02-15T18:34:55.637 INFO:tasks.workunit.client.0.smithi038.stdout:3/335: rmdir d2/d24 39 2024-02-15T18:34:55.639 INFO:tasks.workunit.client.0.smithi038.stdout:9/398: fsync d1/f14 0 2024-02-15T18:34:55.640 INFO:tasks.workunit.client.0.smithi038.stdout:7/385: dwrite d0/f54 [0,4194304] 0 2024-02-15T18:34:55.640 INFO:tasks.workunit.client.0.smithi038.stdout:7/386: fsync d0/db/d59/d15/f44 0 2024-02-15T18:34:55.640 INFO:tasks.workunit.client.0.smithi038.stdout:7/387: chown d0/db/d59/d15/d29/d7e/f5b 7 1 2024-02-15T18:34:55.640 INFO:tasks.workunit.client.0.smithi038.stdout:3/336: creat d2/d1b/d3c/d3a/f68 x:0 0 0 2024-02-15T18:34:55.641 INFO:tasks.workunit.client.0.smithi038.stdout:9/399: dread d1/da/d10/d13/f18 [0,4194304] 0 2024-02-15T18:34:55.643 INFO:tasks.workunit.client.0.smithi038.stdout:3/337: mkdir d2/d1b/d69 0 2024-02-15T18:34:55.644 INFO:tasks.workunit.client.0.smithi038.stdout:9/400: truncate d1/da/d10/d13/d3d/d4b/f72 3424171 0 2024-02-15T18:34:55.659 INFO:tasks.workunit.client.0.smithi038.stdout:9/401: rmdir d1/dc 39 2024-02-15T18:34:55.660 INFO:tasks.workunit.client.0.smithi038.stdout:9/402: mkdir d1/da/d36/d3f/d99 0 2024-02-15T18:34:55.660 INFO:tasks.workunit.client.0.smithi038.stdout:9/403: chown d1/dc/d4f/f46 1 1 2024-02-15T18:34:55.660 INFO:tasks.workunit.client.0.smithi038.stdout:9/404: chown d1/da/d36/d50/f55 1150 1 2024-02-15T18:34:55.660 INFO:tasks.workunit.client.0.smithi038.stdout:7/388: dread d0/d66/f1c [0,4194304] 0 2024-02-15T18:34:55.661 INFO:tasks.workunit.client.0.smithi038.stdout:7/389: chown d0/db/d59/d15/d29/d7e/l4a 2 1 2024-02-15T18:34:55.661 INFO:tasks.workunit.client.0.smithi038.stdout:7/390: write d0/db/d59/d15/f64 [5147669,27257] 0 2024-02-15T18:34:55.661 INFO:tasks.workunit.client.0.smithi038.stdout:9/405: mkdir d1/da/d36/d93/d9a 0 2024-02-15T18:34:55.661 INFO:tasks.workunit.client.0.smithi038.stdout:9/406: readlink d1/dc/d5a/l85 0 2024-02-15T18:34:55.662 INFO:tasks.workunit.client.0.smithi038.stdout:7/391: symlink d0/d68/l85 0 2024-02-15T18:34:55.662 INFO:tasks.workunit.client.0.smithi038.stdout:7/392: dread - d0/d1b/d80/f6e zero size 2024-02-15T18:34:55.662 INFO:tasks.workunit.client.0.smithi038.stdout:7/393: mknod d0/d1b/d30/d5d/c86 0 2024-02-15T18:34:55.663 INFO:tasks.workunit.client.0.smithi038.stdout:7/394: chown d0/db/d19/f4e 746453323 1 2024-02-15T18:34:55.681 INFO:tasks.workunit.client.0.smithi038.stdout:4/398: dwrite d11/d1e/d3e/d57/f77 [0,4194304] 0 2024-02-15T18:34:55.681 INFO:tasks.workunit.client.0.smithi038.stdout:4/399: fdatasync d11/d1e/d20/d2b/d65/f6d 0 2024-02-15T18:34:55.682 INFO:tasks.workunit.client.0.smithi038.stdout:4/400: symlink d11/d1e/d3e/d57/l9a 0 2024-02-15T18:34:55.682 INFO:tasks.workunit.client.0.smithi038.stdout:4/401: chown d11/d1e/d3e/d75/f34 7483 1 2024-02-15T18:34:55.684 INFO:tasks.workunit.client.0.smithi038.stdout:4/402: rename d11/d1e/d20/d31/f32 to d11/d1e/d3e/d75/d40/d6f/f9b 0 2024-02-15T18:34:55.687 INFO:tasks.workunit.client.0.smithi038.stdout:4/403: dread d11/d1e/f37 [4194304,4194304] 0 2024-02-15T18:34:55.687 INFO:tasks.workunit.client.0.smithi038.stdout:6/383: dwrite d2/d10/f1d [0,4194304] 0 2024-02-15T18:34:55.687 INFO:tasks.workunit.client.0.smithi038.stdout:6/384: chown d2/f5b 45032 1 2024-02-15T18:34:55.696 INFO:tasks.workunit.client.0.smithi038.stdout:1/399: dwrite d2/d10/d19/d22/d36/f78 [0,4194304] 0 2024-02-15T18:34:55.696 INFO:tasks.workunit.client.0.smithi038.stdout:1/400: chown d2/d28/c40 8 1 2024-02-15T18:34:55.696 INFO:tasks.workunit.client.0.smithi038.stdout:9/407: dwrite d1/da/d20/f6d [0,4194304] 0 2024-02-15T18:34:55.697 INFO:tasks.workunit.client.0.smithi038.stdout:5/364: dwrite d2/f1c [0,4194304] 0 2024-02-15T18:34:55.697 INFO:tasks.workunit.client.0.smithi038.stdout:3/338: dwrite d2/d1e/f44 [0,4194304] 0 2024-02-15T18:34:55.697 INFO:tasks.workunit.client.0.smithi038.stdout:7/395: dread d0/db/f20 [0,4194304] 0 2024-02-15T18:34:55.700 INFO:tasks.workunit.client.0.smithi038.stdout:8/547: dwrite d1/f50 [0,4194304] 0 2024-02-15T18:34:55.700 INFO:tasks.workunit.client.0.smithi038.stdout:1/401: write d2/f11 [3389635,47102] 0 2024-02-15T18:34:55.700 INFO:tasks.workunit.client.0.smithi038.stdout:1/402: dread - d2/d39/f7f zero size 2024-02-15T18:34:55.702 INFO:tasks.workunit.client.0.smithi038.stdout:9/408: getdents d1/da/d10/d82 0 2024-02-15T18:34:55.702 INFO:tasks.workunit.client.0.smithi038.stdout:9/409: write d1/da/f54 [4889802,103589] 0 2024-02-15T18:34:55.704 INFO:tasks.workunit.client.0.smithi038.stdout:6/385: dread d2/d4/f4a [0,4194304] 0 2024-02-15T18:34:55.704 INFO:tasks.workunit.client.0.smithi038.stdout:5/365: mknod d2/d3/d5f/d71/c72 0 2024-02-15T18:34:55.706 INFO:tasks.workunit.client.0.smithi038.stdout:3/339: creat d2/d1b/d69/f6a x:0 0 0 2024-02-15T18:34:55.708 INFO:tasks.workunit.client.0.smithi038.stdout:7/396: mknod d0/d1b/d30/d50/c87 0 2024-02-15T18:34:55.712 INFO:tasks.workunit.client.0.smithi038.stdout:4/404: dwrite d11/d1e/d20/d2b/d35/f89 [0,4194304] 0 2024-02-15T18:34:55.713 INFO:tasks.workunit.client.0.smithi038.stdout:4/405: write d11/d1e/d20/d31/f5b [2434916,55972] 0 2024-02-15T18:34:55.713 INFO:tasks.workunit.client.0.smithi038.stdout:4/406: rename d11/d1e/d3e to d11/d1e/d3e/d57/d7e/d9c 22 2024-02-15T18:34:55.713 INFO:tasks.workunit.client.0.smithi038.stdout:4/407: stat d11/d1e/d20/d2b/d4d/d67/d73/d8b 0 2024-02-15T18:34:55.713 INFO:tasks.workunit.client.0.smithi038.stdout:4/408: write d11/d1e/d20/d2f/d71/f98 [4560674,127900] 0 2024-02-15T18:34:55.717 INFO:tasks.workunit.client.0.smithi038.stdout:1/403: mknod d2/d28/d3c/c95 0 2024-02-15T18:34:55.718 INFO:tasks.workunit.client.0.smithi038.stdout:9/410: mknod d1/da/d36/d93/d9a/c9b 0 2024-02-15T18:34:55.718 INFO:tasks.workunit.client.0.smithi038.stdout:7/397: dread d0/db/d59/d15/f44 [0,4194304] 0 2024-02-15T18:34:55.719 INFO:tasks.workunit.client.0.smithi038.stdout:7/398: fdatasync d0/d1b/f78 0 2024-02-15T18:34:55.719 INFO:tasks.workunit.client.0.smithi038.stdout:7/399: stat d0/db/d59/d15 0 2024-02-15T18:34:55.719 INFO:tasks.workunit.client.0.smithi038.stdout:6/386: creat d2/d10/f6b x:0 0 0 2024-02-15T18:34:55.719 INFO:tasks.workunit.client.0.smithi038.stdout:6/387: fdatasync d2/d4/f1e 0 2024-02-15T18:34:55.720 INFO:tasks.workunit.client.0.smithi038.stdout:6/388: fdatasync d2/f46 0 2024-02-15T18:34:55.720 INFO:tasks.workunit.client.0.smithi038.stdout:8/548: dwrite d1/f2 [0,4194304] 0 2024-02-15T18:34:55.720 INFO:tasks.workunit.client.0.smithi038.stdout:5/366: truncate d2/d3/d5f/d71/f5a 5498177 0 2024-02-15T18:34:55.720 INFO:tasks.workunit.client.0.smithi038.stdout:5/367: chown d2/d3/d5f/d71/d33 1 1 2024-02-15T18:34:55.721 INFO:tasks.workunit.client.0.smithi038.stdout:3/340: truncate d2/f38 2967353 0 2024-02-15T18:34:55.724 INFO:tasks.workunit.client.0.smithi038.stdout:8/549: dread d1/d4/d10/d43/d48/d52/fa9 [0,4194304] 0 2024-02-15T18:34:55.724 INFO:tasks.workunit.client.0.smithi038.stdout:0/463: sync 2024-02-15T18:34:55.724 INFO:tasks.workunit.client.0.smithi038.stdout:0/464: fsync d4/d13/d53/f7b 0 2024-02-15T18:34:55.724 INFO:tasks.workunit.client.0.smithi038.stdout:3/341: read d2/f51 [1841086,76293] 0 2024-02-15T18:34:55.730 INFO:tasks.workunit.client.0.smithi038.stdout:4/409: creat d11/d1e/d20/d2b/d65/f9d x:0 0 0 2024-02-15T18:34:55.730 INFO:tasks.workunit.client.0.smithi038.stdout:4/410: chown d11/d1e/d20/d2b/d4d/c5a 48199547 1 2024-02-15T18:34:55.730 INFO:tasks.workunit.client.0.smithi038.stdout:4/411: stat d11/d1e/d20/c84 0 2024-02-15T18:34:55.730 INFO:tasks.workunit.client.0.smithi038.stdout:1/404: getdents d2/d69 0 2024-02-15T18:34:55.733 INFO:tasks.workunit.client.0.smithi038.stdout:7/400: rmdir d0/d68 39 2024-02-15T18:34:55.733 INFO:tasks.workunit.client.0.smithi038.stdout:9/411: symlink d1/da/d10/d24/d49/l9c 0 2024-02-15T18:34:55.733 INFO:tasks.workunit.client.0.smithi038.stdout:9/412: write d1/da/d10/d24/f77 [379300,51863] 0 2024-02-15T18:34:55.733 INFO:tasks.workunit.client.0.smithi038.stdout:7/401: write d0/d1b/d30/d5d/f82 [9168,104778] 0 2024-02-15T18:34:55.733 INFO:tasks.workunit.client.0.smithi038.stdout:6/389: truncate d2/f3 7589917 0 2024-02-15T18:34:55.735 INFO:tasks.workunit.client.0.smithi038.stdout:8/550: link d1/d7/d26/c56 d1/d4/d10/d3d/db7/cc8 0 2024-02-15T18:34:55.736 INFO:tasks.workunit.client.0.smithi038.stdout:0/465: mkdir d4/d13/d2a/d35/d1e/d32/d48/d9a 0 2024-02-15T18:34:55.737 INFO:tasks.workunit.client.0.smithi038.stdout:2/333: sync 2024-02-15T18:34:55.738 INFO:tasks.workunit.client.0.smithi038.stdout:4/412: rmdir d11/d1e/d43/d5e/d62 39 2024-02-15T18:34:55.738 INFO:tasks.workunit.client.0.smithi038.stdout:1/405: dread d2/d10/d19/d22/d36/f4d [0,4194304] 0 2024-02-15T18:34:55.740 INFO:tasks.workunit.client.0.smithi038.stdout:9/413: getdents d1/da/d10/d13/d3d/d43/d4d/d8a 0 2024-02-15T18:34:55.741 INFO:tasks.workunit.client.0.smithi038.stdout:6/390: mkdir d2/d4/d6c 0 2024-02-15T18:34:55.745 INFO:tasks.workunit.client.0.smithi038.stdout:5/368: dwrite d2/f3f [0,4194304] 0 2024-02-15T18:34:55.746 INFO:tasks.workunit.client.0.smithi038.stdout:3/342: dread d2/f38 [0,4194304] 0 2024-02-15T18:34:55.747 INFO:tasks.workunit.client.0.smithi038.stdout:1/406: dread d2/f2d [0,4194304] 0 2024-02-15T18:34:55.747 INFO:tasks.workunit.client.0.smithi038.stdout:8/551: rmdir d1/d4/d10/d43/d48/d9a 39 2024-02-15T18:34:55.749 INFO:tasks.workunit.client.0.smithi038.stdout:2/334: rmdir dd/d12/d5d 39 2024-02-15T18:34:55.750 INFO:tasks.workunit.client.0.smithi038.stdout:6/391: dread d2/d4/f27 [4194304,4194304] 0 2024-02-15T18:34:55.750 INFO:tasks.workunit.client.0.smithi038.stdout:4/413: symlink d11/d1e/d20/d2b/d4d/l9e 0 2024-02-15T18:34:55.750 INFO:tasks.workunit.client.0.smithi038.stdout:6/392: readlink d2/d4/l3b 0 2024-02-15T18:34:55.750 INFO:tasks.workunit.client.0.smithi038.stdout:9/414: unlink d1/f16 0 2024-02-15T18:34:55.751 INFO:tasks.workunit.client.0.smithi038.stdout:3/343: creat d2/d1b/d3c/f6b x:0 0 0 2024-02-15T18:34:55.751 INFO:tasks.workunit.client.0.smithi038.stdout:1/407: link d2/d28/d3c/c95 d2/d10/d19/d2f/d3e/c96 0 2024-02-15T18:34:55.752 INFO:tasks.workunit.client.0.smithi038.stdout:8/552: symlink d1/d4/d5/d6/d6d/lc9 0 2024-02-15T18:34:55.753 INFO:tasks.workunit.client.0.smithi038.stdout:2/335: creat dd/d12/d1d/d3b/d3c/d4a/f6d x:0 0 0 2024-02-15T18:34:55.755 INFO:tasks.workunit.client.0.smithi038.stdout:6/393: rename d2/d4/d19/d55/d5c/c63 to d2/d4/d2c/c6d 0 2024-02-15T18:34:55.756 INFO:tasks.workunit.client.0.smithi038.stdout:9/415: symlink d1/dc/d38/l9d 0 2024-02-15T18:34:55.756 INFO:tasks.workunit.client.0.smithi038.stdout:3/344: mknod d2/d1b/d3c/d3a/c6c 0 2024-02-15T18:34:55.757 INFO:tasks.workunit.client.0.smithi038.stdout:1/408: getdents d2/d10/d32/d53/d82 0 2024-02-15T18:34:55.757 INFO:tasks.workunit.client.0.smithi038.stdout:1/409: readlink d2/l1a 0 2024-02-15T18:34:55.757 INFO:tasks.workunit.client.0.smithi038.stdout:1/410: chown d2/d28/d3c/l6f 0 1 2024-02-15T18:34:55.758 INFO:tasks.workunit.client.0.smithi038.stdout:8/553: symlink d1/d7/lca 0 2024-02-15T18:34:55.763 INFO:tasks.workunit.client.0.smithi038.stdout:2/336: creat dd/d12/d3a/d67/d4f/d5a/f6e x:0 0 0 2024-02-15T18:34:55.763 INFO:tasks.workunit.client.0.smithi038.stdout:5/369: dread d2/d48/f59 [0,4194304] 0 2024-02-15T18:34:55.763 INFO:tasks.workunit.client.0.smithi038.stdout:6/394: rmdir d2/d10 39 2024-02-15T18:34:55.764 INFO:tasks.workunit.client.0.smithi038.stdout:3/345: getdents d2/d1e 0 2024-02-15T18:34:55.766 INFO:tasks.workunit.client.0.smithi038.stdout:9/416: creat d1/da/d10/d24/f9e x:0 0 0 2024-02-15T18:34:55.766 INFO:tasks.workunit.client.0.smithi038.stdout:9/417: dread - d1/da/d10/d13/d3d/d43/f74 zero size 2024-02-15T18:34:55.766 INFO:tasks.workunit.client.0.smithi038.stdout:9/418: readlink d1/da/d10/d13/d3d/d43/l5e 0 2024-02-15T18:34:55.766 INFO:tasks.workunit.client.0.smithi038.stdout:8/554: creat d1/d4/d5/d6/fcb x:0 0 0 2024-02-15T18:34:55.766 INFO:tasks.workunit.client.0.smithi038.stdout:2/337: dread dd/d12/d3a/f45 [0,4194304] 0 2024-02-15T18:34:55.766 INFO:tasks.workunit.client.0.smithi038.stdout:4/414: dwrite f4 [0,4194304] 0 2024-02-15T18:34:55.766 INFO:tasks.workunit.client.0.smithi038.stdout:0/466: dwrite d4/f88 [0,4194304] 0 2024-02-15T18:34:55.766 INFO:tasks.workunit.client.0.smithi038.stdout:3/346: mknod d2/d25/c6d 0 2024-02-15T18:34:55.767 INFO:tasks.workunit.client.0.smithi038.stdout:7/402: dwrite d0/f54 [0,4194304] 0 2024-02-15T18:34:55.767 INFO:tasks.workunit.client.0.smithi038.stdout:5/370: read d2/d3/ff [949965,50152] 0 2024-02-15T18:34:55.768 INFO:tasks.workunit.client.0.smithi038.stdout:8/555: rename d1/d7/d22/da7/lb5 to d1/d7/d26/d69/lcc 0 2024-02-15T18:34:55.768 INFO:tasks.workunit.client.0.smithi038.stdout:8/556: chown d1/d4/d5/d31/l87 6554681 1 2024-02-15T18:34:55.772 INFO:tasks.workunit.client.0.smithi038.stdout:9/419: write f0 [975414,62030] 0 2024-02-15T18:34:55.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:54.317314+0000 mgr.smithi038.khesbl (mgr.14162) 337 : cluster [DBG] pgmap v279: 65 pgs: 65 active+clean; 475 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 42 MiB/s wr, 548 op/s 2024-02-15T18:34:55.774 INFO:tasks.workunit.client.0.smithi038.stdout:2/338: creat dd/d12/d1d/d3b/f6f x:0 0 0 2024-02-15T18:34:55.774 INFO:tasks.workunit.client.0.smithi038.stdout:1/411: dwrite d2/d28/f34 [4194304,4194304] 0 2024-02-15T18:34:55.774 INFO:tasks.workunit.client.0.smithi038.stdout:4/415: link d11/d1e/d20/d2b/d35/f89 d11/d1e/d20/d97/f9f 0 2024-02-15T18:34:55.782 INFO:tasks.workunit.client.0.smithi038.stdout:0/467: creat d4/d13/d2a/d35/d1e/d32/d48/d97/f9b x:0 0 0 2024-02-15T18:34:55.785 INFO:tasks.workunit.client.0.smithi038.stdout:7/403: creat d0/d1b/d30/d5d/d70/f88 x:0 0 0 2024-02-15T18:34:55.786 INFO:tasks.workunit.client.0.smithi038.stdout:5/371: truncate d2/d3/f1b 3744949 0 2024-02-15T18:34:55.787 INFO:tasks.workunit.client.0.smithi038.stdout:5/372: stat d2/d3/d12/c4d 0 2024-02-15T18:34:55.787 INFO:tasks.workunit.client.0.smithi038.stdout:5/373: fdatasync d2/d3/f2f 0 2024-02-15T18:34:55.787 INFO:tasks.workunit.client.0.smithi038.stdout:8/557: stat d1/d7/d15/d4b/c96 0 2024-02-15T18:34:55.788 INFO:tasks.workunit.client.0.smithi038.stdout:9/420: creat d1/da/d10/f9f x:0 0 0 2024-02-15T18:34:55.790 INFO:tasks.workunit.client.0.smithi038.stdout:1/412: creat d2/d10/d19/d5f/f97 x:0 0 0 2024-02-15T18:34:55.790 INFO:tasks.workunit.client.0.smithi038.stdout:4/416: truncate d11/d1e/f29 791522 0 2024-02-15T18:34:55.792 INFO:tasks.workunit.client.0.smithi038.stdout:7/404: creat d0/d66/f89 x:0 0 0 2024-02-15T18:34:55.792 INFO:tasks.workunit.client.0.smithi038.stdout:7/405: chown d0/d1b/d30/d5d/d61 2101 1 2024-02-15T18:34:55.792 INFO:tasks.workunit.client.0.smithi038.stdout:7/406: chown d0/db/l77 105059 1 2024-02-15T18:34:55.792 INFO:tasks.workunit.client.0.smithi038.stdout:7/407: truncate d0/d1b/d30/d5d/d61/f75 1063550 0 2024-02-15T18:34:55.792 INFO:tasks.workunit.client.0.smithi038.stdout:7/408: chown d0/db/d59/c14 390852 1 2024-02-15T18:34:55.793 INFO:tasks.workunit.client.0.smithi038.stdout:8/558: rename d1/f5d to d1/d4/d10/d43/d48/fcd 0 2024-02-15T18:34:55.793 INFO:tasks.workunit.client.0.smithi038.stdout:8/559: truncate d1/d4/d5/d31/f39 12579943 0 2024-02-15T18:34:55.793 INFO:tasks.workunit.client.0.smithi038.stdout:6/395: dwrite d2/d4/d28/f3a [0,4194304] 0 2024-02-15T18:34:55.794 INFO:tasks.workunit.client.0.smithi038.stdout:9/421: unlink d1/da/d10/d24/c2a 0 2024-02-15T18:34:55.795 INFO:tasks.workunit.client.0.smithi038.stdout:1/413: truncate d2/d10/f31 4126980 0 2024-02-15T18:34:55.799 INFO:tasks.workunit.client.0.smithi038.stdout:7/409: mknod d0/d1b/d30/c8a 0 2024-02-15T18:34:55.799 INFO:tasks.workunit.client.0.smithi038.stdout:4/417: rename d11/d1e/l72 to d11/d1e/d43/d5e/la0 0 2024-02-15T18:34:55.799 INFO:tasks.workunit.client.0.smithi038.stdout:3/347: dwrite d2/d25/f30 [0,4194304] 0 2024-02-15T18:34:55.800 INFO:tasks.workunit.client.0.smithi038.stdout:8/560: truncate d1/d4/dbb/fbf 4067372 0 2024-02-15T18:34:55.800 INFO:tasks.workunit.client.0.smithi038.stdout:5/374: write d2/f1c [409266,78185] 0 2024-02-15T18:34:55.800 INFO:tasks.workunit.client.0.smithi038.stdout:5/375: dread - d2/d3/d5f/d71/d33/f68 zero size 2024-02-15T18:34:55.802 INFO:tasks.workunit.client.0.smithi038.stdout:2/339: dwrite dd/d12/f36 [0,4194304] 0 2024-02-15T18:34:55.802 INFO:tasks.workunit.client.0.smithi038.stdout:2/340: fsync dd/d12/d1d/d3b/d3c/f48 0 2024-02-15T18:34:55.802 INFO:tasks.workunit.client.0.smithi038.stdout:2/341: fdatasync dd/d12/f36 0 2024-02-15T18:34:55.804 INFO:tasks.workunit.client.0.smithi038.stdout:0/468: dwrite d4/d13/f64 [0,4194304] 0 2024-02-15T18:34:55.804 INFO:tasks.workunit.client.0.smithi038.stdout:3/348: read d2/d1b/f36 [308467,61877] 0 2024-02-15T18:34:55.804 INFO:tasks.workunit.client.0.smithi038.stdout:3/349: write d2/d1b/d69/f6a [333063,90440] 0 2024-02-15T18:34:55.804 INFO:tasks.workunit.client.0.smithi038.stdout:9/422: rename d1/da/d10/d13/c1d to d1/da/d10/ca0 0 2024-02-15T18:34:55.808 INFO:tasks.workunit.client.0.smithi038.stdout:4/418: mkdir d11/d1e/d20/d2b/d4d/d67/da1 0 2024-02-15T18:34:55.811 INFO:tasks.workunit.client.0.smithi038.stdout:8/561: truncate d1/d7/d15/d6a/f86 943272 0 2024-02-15T18:34:55.812 INFO:tasks.workunit.client.0.smithi038.stdout:2/342: rmdir dd/d12/d3a/d5e 39 2024-02-15T18:34:55.812 INFO:tasks.workunit.client.0.smithi038.stdout:2/343: chown dd/d12/d3a/d67 44024 1 2024-02-15T18:34:55.816 INFO:tasks.workunit.client.0.smithi038.stdout:0/469: unlink d4/d13/d2a/d35/f93 0 2024-02-15T18:34:55.820 INFO:tasks.workunit.client.0.smithi038.stdout:1/414: dread d2/d28/f50 [0,4194304] 0 2024-02-15T18:34:55.827 INFO:tasks.workunit.client.0.smithi038.stdout:4/419: rmdir d11/d1e/d3e/d75/d40 39 2024-02-15T18:34:55.827 INFO:tasks.workunit.client.0.smithi038.stdout:4/420: read - d11/d1e/d20/d31/f91 zero size 2024-02-15T18:34:55.827 INFO:tasks.workunit.client.0.smithi038.stdout:4/421: chown d11/d1e/d20/d31/d48/c5c 138407 1 2024-02-15T18:34:55.827 INFO:tasks.workunit.client.0.smithi038.stdout:8/562: link d1/d4/d10/d43/d48/d9a/cad d1/d4/d5/d6/d6d/cce 0 2024-02-15T18:34:55.828 INFO:tasks.workunit.client.0.smithi038.stdout:2/344: mknod dd/d12/d1d/d5f/c70 0 2024-02-15T18:34:55.829 INFO:tasks.workunit.client.0.smithi038.stdout:0/470: truncate d4/d13/d2a/d35/d6a/f77 933917 0 2024-02-15T18:34:55.833 INFO:tasks.workunit.client.0.smithi038.stdout:0/471: write d4/d13/d2a/d35/d1e/d32/d48/f56 [4741159,114239] 0 2024-02-15T18:34:55.833 INFO:tasks.workunit.client.0.smithi038.stdout:4/422: mkdir d11/d1e/d20/d31/da2 0 2024-02-15T18:34:55.835 INFO:tasks.workunit.client.0.smithi038.stdout:3/350: dwrite d2/f15 [0,4194304] 0 2024-02-15T18:34:55.835 INFO:tasks.workunit.client.0.smithi038.stdout:3/351: fsync d2/d1b/d4f/f5d 0 2024-02-15T18:34:55.836 INFO:tasks.workunit.client.0.smithi038.stdout:3/352: stat d2/d25/f30 0 2024-02-15T18:34:55.836 INFO:tasks.workunit.client.0.smithi038.stdout:8/563: creat d1/d4/d10/d43/d48/d9a/fcf x:0 0 0 2024-02-15T18:34:55.836 INFO:tasks.workunit.client.0.smithi038.stdout:5/376: dwrite d2/d30/d52/f6a [0,4194304] 0 2024-02-15T18:34:55.836 INFO:tasks.workunit.client.0.smithi038.stdout:1/415: write d2/d10/d19/d22/f57 [3843558,53185] 0 2024-02-15T18:34:55.837 INFO:tasks.workunit.client.0.smithi038.stdout:0/472: read d4/d13/f64 [1860641,103967] 0 2024-02-15T18:34:55.837 INFO:tasks.workunit.client.0.smithi038.stdout:7/410: dwrite d0/db/fe [0,4194304] 0 2024-02-15T18:34:55.837 INFO:tasks.workunit.client.0.smithi038.stdout:7/411: write d0/d1b/d30/d5d/f82 [825580,5542] 0 2024-02-15T18:34:55.839 INFO:tasks.workunit.client.0.smithi038.stdout:3/353: truncate d2/d24/f2e 1705085 0 2024-02-15T18:34:55.845 INFO:tasks.workunit.client.0.smithi038.stdout:2/345: dwrite dd/f24 [0,4194304] 0 2024-02-15T18:34:55.848 INFO:tasks.workunit.client.0.smithi038.stdout:9/423: dwrite d1/da/d36/d50/f55 [0,4194304] 0 2024-02-15T18:34:55.848 INFO:tasks.workunit.client.0.smithi038.stdout:4/423: dread d11/d1e/f28 [0,4194304] 0 2024-02-15T18:34:55.848 INFO:tasks.workunit.client.0.smithi038.stdout:1/416: symlink d2/d69/d8b/l98 0 2024-02-15T18:34:55.848 INFO:tasks.workunit.client.0.smithi038.stdout:1/417: write d2/f73 [967445,33651] 0 2024-02-15T18:34:55.849 INFO:tasks.workunit.client.0.smithi038.stdout:1/418: write d2/d10/d19/d22/f2c [88624,80319] 0 2024-02-15T18:34:55.849 INFO:tasks.workunit.client.0.smithi038.stdout:8/564: dread d1/d4/d10/f35 [0,4194304] 0 2024-02-15T18:34:55.849 INFO:tasks.workunit.client.0.smithi038.stdout:2/346: dread dd/f41 [0,4194304] 0 2024-02-15T18:34:55.850 INFO:tasks.workunit.client.0.smithi038.stdout:7/412: creat d0/db/d19/f8b x:0 0 0 2024-02-15T18:34:55.850 INFO:tasks.workunit.client.0.smithi038.stdout:8/565: dread d1/d3a/f6c [0,4194304] 0 2024-02-15T18:34:55.852 INFO:tasks.workunit.client.0.smithi038.stdout:3/354: truncate d2/d1e/f32 918943 0 2024-02-15T18:34:55.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:54.317314+0000 mgr.smithi038.khesbl (mgr.14162) 337 : cluster [DBG] pgmap v279: 65 pgs: 65 active+clean; 475 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 42 MiB/s wr, 548 op/s 2024-02-15T18:34:55.859 INFO:tasks.workunit.client.0.smithi038.stdout:5/377: dwrite d2/f4c [0,4194304] 0 2024-02-15T18:34:55.864 INFO:tasks.workunit.client.0.smithi038.stdout:9/424: creat d1/fa1 x:0 0 0 2024-02-15T18:34:55.864 INFO:tasks.workunit.client.0.smithi038.stdout:4/424: symlink d11/d1e/d20/d2b/d35/la3 0 2024-02-15T18:34:55.868 INFO:tasks.workunit.client.0.smithi038.stdout:1/419: rename d2/d28/l4b to d2/d2e/d8c/l99 0 2024-02-15T18:34:55.869 INFO:tasks.workunit.client.0.smithi038.stdout:2/347: creat dd/d12/d3a/d67/d4f/f71 x:0 0 0 2024-02-15T18:34:55.870 INFO:tasks.workunit.client.0.smithi038.stdout:7/413: creat d0/db/f8c x:0 0 0 2024-02-15T18:34:55.871 INFO:tasks.workunit.client.0.smithi038.stdout:7/414: write d0/db/ff [444231,23184] 0 2024-02-15T18:34:55.871 INFO:tasks.workunit.client.0.smithi038.stdout:7/415: fdatasync d0/db/d19/f25 0 2024-02-15T18:34:55.872 INFO:tasks.workunit.client.0.smithi038.stdout:8/566: rmdir d1/d7/d26/d8f 0 2024-02-15T18:34:55.874 INFO:tasks.workunit.client.0.smithi038.stdout:3/355: mknod d2/d1b/d3c/d42/c6e 0 2024-02-15T18:34:55.877 INFO:tasks.workunit.client.0.smithi038.stdout:5/378: creat d2/d30/f73 x:0 0 0 2024-02-15T18:34:55.877 INFO:tasks.workunit.client.0.smithi038.stdout:9/425: creat d1/da/d10/d13/d3d/fa2 x:0 0 0 2024-02-15T18:34:55.877 INFO:tasks.workunit.client.0.smithi038.stdout:9/426: stat d1/da/d20/d58/d66/f90 0 2024-02-15T18:34:55.878 INFO:tasks.workunit.client.0.smithi038.stdout:4/425: mknod d11/d1e/d20/d2b/d65/ca4 0 2024-02-15T18:34:55.883 INFO:tasks.workunit.client.0.smithi038.stdout:7/416: mkdir d0/db/d59/d15/d29/d8d 0 2024-02-15T18:34:55.883 INFO:tasks.workunit.client.0.smithi038.stdout:7/417: chown d0/db/d19/d2d/l1e 12822 1 2024-02-15T18:34:55.884 INFO:tasks.workunit.client.0.smithi038.stdout:8/567: creat d1/d4/d10/d43/d48/d52/d88/db3/fd0 x:0 0 0 2024-02-15T18:34:55.885 INFO:tasks.workunit.client.0.smithi038.stdout:3/356: unlink d2/d1b/d3c/d42/c6e 0 2024-02-15T18:34:55.887 INFO:tasks.workunit.client.0.smithi038.stdout:5/379: creat d2/d48/f74 x:0 0 0 2024-02-15T18:34:55.888 INFO:tasks.workunit.client.0.smithi038.stdout:5/380: stat d2/d3/d5f/d71/f5a 0 2024-02-15T18:34:55.888 INFO:tasks.workunit.client.0.smithi038.stdout:5/381: stat d2/fc 0 2024-02-15T18:34:55.888 INFO:tasks.workunit.client.0.smithi038.stdout:9/427: creat d1/da/d10/d13/d3d/fa3 x:0 0 0 2024-02-15T18:34:55.889 INFO:tasks.workunit.client.0.smithi038.stdout:9/428: stat d1/dc/d4f 0 2024-02-15T18:34:55.889 INFO:tasks.workunit.client.0.smithi038.stdout:9/429: write d1/dc/d38/f6f [564071,127359] 0 2024-02-15T18:34:55.889 INFO:tasks.workunit.client.0.smithi038.stdout:9/430: fdatasync d1/da/d36/f56 0 2024-02-15T18:34:55.889 INFO:tasks.workunit.client.0.smithi038.stdout:9/431: dread - d1/da/d10/d24/f9e zero size 2024-02-15T18:34:55.889 INFO:tasks.workunit.client.0.smithi038.stdout:6/396: sync 2024-02-15T18:34:55.890 INFO:tasks.workunit.client.0.smithi038.stdout:8/568: link d1/d4/d5/d31/c66 d1/d4/d5/d6/db9/cd1 0 2024-02-15T18:34:55.891 INFO:tasks.workunit.client.0.smithi038.stdout:9/432: write d1/da/d36/f37 [716833,38942] 0 2024-02-15T18:34:55.891 INFO:tasks.workunit.client.0.smithi038.stdout:9/433: chown d1/da/d10/d13 455727 1 2024-02-15T18:34:55.891 INFO:tasks.workunit.client.0.smithi038.stdout:9/434: write d1/da/d10/d24/f87 [280298,25900] 0 2024-02-15T18:34:55.891 INFO:tasks.workunit.client.0.smithi038.stdout:5/382: mkdir d2/d75 0 2024-02-15T18:34:55.892 INFO:tasks.workunit.client.0.smithi038.stdout:6/397: symlink d2/d4/d2c/l6e 0 2024-02-15T18:34:55.893 INFO:tasks.workunit.client.0.smithi038.stdout:6/398: readlink d2/d4/l3b 0 2024-02-15T18:34:55.893 INFO:tasks.workunit.client.0.smithi038.stdout:8/569: mknod d1/d7/d22/da7/cd2 0 2024-02-15T18:34:55.893 INFO:tasks.workunit.client.0.smithi038.stdout:8/570: stat d1/d7/d26/f84 0 2024-02-15T18:34:55.894 INFO:tasks.workunit.client.0.smithi038.stdout:9/435: creat d1/da/d36/d93/fa4 x:0 0 0 2024-02-15T18:34:55.894 INFO:tasks.workunit.client.0.smithi038.stdout:9/436: fdatasync d1/da/d10/d24/f7c 0 2024-02-15T18:34:55.894 INFO:tasks.workunit.client.0.smithi038.stdout:9/437: chown d1/dc/d38 38927224 1 2024-02-15T18:34:55.894 INFO:tasks.workunit.client.0.smithi038.stdout:9/438: chown d1/dc/l23 455 1 2024-02-15T18:34:55.895 INFO:tasks.workunit.client.0.smithi038.stdout:6/399: mkdir d2/d4/d6f 0 2024-02-15T18:34:55.895 INFO:tasks.workunit.client.0.smithi038.stdout:6/400: stat d2/d4/d19/c41 0 2024-02-15T18:34:55.896 INFO:tasks.workunit.client.0.smithi038.stdout:8/571: symlink d1/d4/d10/d91/ld3 0 2024-02-15T18:34:55.897 INFO:tasks.workunit.client.0.smithi038.stdout:8/572: chown d1/d7/c5b 463107 1 2024-02-15T18:34:55.900 INFO:tasks.workunit.client.0.smithi038.stdout:8/573: rename d1/d4/d10/d43/d48/d52/d88/db3/lc0 to d1/d4/d5/ld4 0 2024-02-15T18:34:55.900 INFO:tasks.workunit.client.0.smithi038.stdout:8/574: truncate d1/d4/d5/d31/f33 2466358 0 2024-02-15T18:34:55.900 INFO:tasks.workunit.client.0.smithi038.stdout:8/575: chown d1/d7/d26 429969449 1 2024-02-15T18:34:55.901 INFO:tasks.workunit.client.0.smithi038.stdout:7/418: dread d0/d1b/f78 [0,4194304] 0 2024-02-15T18:34:55.901 INFO:tasks.workunit.client.0.smithi038.stdout:1/420: dwrite d2/d10/d19/d5f/f97 [0,4194304] 0 2024-02-15T18:34:55.902 INFO:tasks.workunit.client.0.smithi038.stdout:7/419: readlink d0/db/d19/d2d/l1e 0 2024-02-15T18:34:55.902 INFO:tasks.workunit.client.0.smithi038.stdout:1/421: truncate d2/d10/d19/d22/d36/f4d 4710629 0 2024-02-15T18:34:55.905 INFO:tasks.workunit.client.0.smithi038.stdout:7/420: creat d0/d1b/d30/d50/f8e x:0 0 0 2024-02-15T18:34:55.906 INFO:tasks.workunit.client.0.smithi038.stdout:1/422: mknod d2/d28/d6b/c9a 0 2024-02-15T18:34:55.906 INFO:tasks.workunit.client.0.smithi038.stdout:6/401: dread d2/d10/f1d [0,4194304] 0 2024-02-15T18:34:55.906 INFO:tasks.workunit.client.0.smithi038.stdout:1/423: dread - d2/d2e/f7d zero size 2024-02-15T18:34:55.908 INFO:tasks.workunit.client.0.smithi038.stdout:7/421: unlink d0/d1b/d30/d5d/c86 0 2024-02-15T18:34:55.908 INFO:tasks.workunit.client.0.smithi038.stdout:7/422: chown d0/db/d19/f6d 117 1 2024-02-15T18:34:55.914 INFO:tasks.workunit.client.0.smithi038.stdout:1/424: getdents d2/d10/d19/d5f 0 2024-02-15T18:34:55.914 INFO:tasks.workunit.client.0.smithi038.stdout:7/423: creat d0/db/d59/d15/d29/d8d/f8f x:0 0 0 2024-02-15T18:34:55.916 INFO:tasks.workunit.client.0.smithi038.stdout:1/425: truncate d2/d10/d19/d22/f2c 874844 0 2024-02-15T18:34:55.921 INFO:tasks.workunit.client.0.smithi038.stdout:4/426: dwrite d11/d1e/d43/f63 [0,4194304] 0 2024-02-15T18:34:55.931 INFO:tasks.workunit.client.0.smithi038.stdout:3/357: dwrite d2/fa [0,4194304] 0 2024-02-15T18:34:55.933 INFO:tasks.workunit.client.0.smithi038.stdout:3/358: mknod d2/d1b/d3c/d41/d4e/c6f 0 2024-02-15T18:34:55.933 INFO:tasks.workunit.client.0.smithi038.stdout:3/359: fdatasync d2/f9 0 2024-02-15T18:34:55.933 INFO:tasks.workunit.client.0.smithi038.stdout:3/360: dread - d2/d1b/d4f/f5d zero size 2024-02-15T18:34:55.933 INFO:tasks.workunit.client.0.smithi038.stdout:3/361: getdents d2/d1b/d3c/d41/d4e/d62 0 2024-02-15T18:34:55.934 INFO:tasks.workunit.client.0.smithi038.stdout:3/362: symlink d2/d1b/d3c/d41/l70 0 2024-02-15T18:34:55.935 INFO:tasks.workunit.client.0.smithi038.stdout:3/363: truncate d2/f17 3609451 0 2024-02-15T18:34:55.941 INFO:tasks.workunit.client.0.smithi038.stdout:2/348: dwrite fb [0,4194304] 0 2024-02-15T18:34:55.942 INFO:tasks.workunit.client.0.smithi038.stdout:2/349: creat dd/d12/d3a/f72 x:0 0 0 2024-02-15T18:34:55.943 INFO:tasks.workunit.client.0.smithi038.stdout:2/350: mkdir dd/d12/d1d/d3b/d3c/d73 0 2024-02-15T18:34:55.943 INFO:tasks.workunit.client.0.smithi038.stdout:2/351: truncate dd/d12/d1d/f50 8954370 0 2024-02-15T18:34:55.943 INFO:tasks.workunit.client.0.smithi038.stdout:2/352: chown dd/d12/d1d/d3b/f4b 216570 1 2024-02-15T18:34:55.943 INFO:tasks.workunit.client.0.smithi038.stdout:2/353: symlink dd/d12/d5d/d33/l74 0 2024-02-15T18:34:55.944 INFO:tasks.workunit.client.0.smithi038.stdout:2/354: link dd/d12/d59/d62/c66 dd/c75 0 2024-02-15T18:34:55.946 INFO:tasks.workunit.client.0.smithi038.stdout:2/355: write dd/f24 [1026228,35817] 0 2024-02-15T18:34:55.948 INFO:tasks.workunit.client.0.smithi038.stdout:2/356: rename dd/l4c to dd/d12/d1d/d3b/d3c/d4a/l76 0 2024-02-15T18:34:55.949 INFO:tasks.workunit.client.0.smithi038.stdout:5/383: dwrite d2/d48/f59 [0,4194304] 0 2024-02-15T18:34:55.950 INFO:tasks.workunit.client.0.smithi038.stdout:2/357: rename dd/f24 to dd/f77 0 2024-02-15T18:34:55.950 INFO:tasks.workunit.client.0.smithi038.stdout:2/358: truncate dd/f46 533454 0 2024-02-15T18:34:55.951 INFO:tasks.workunit.client.0.smithi038.stdout:5/384: write d2/f1a [2279573,36013] 0 2024-02-15T18:34:55.951 INFO:tasks.workunit.client.0.smithi038.stdout:2/359: write dd/d12/d3a/d67/f30 [2604138,58411] 0 2024-02-15T18:34:55.954 INFO:tasks.workunit.client.0.smithi038.stdout:7/424: dwrite d0/d1b/f3f [0,4194304] 0 2024-02-15T18:34:55.957 INFO:tasks.workunit.client.0.smithi038.stdout:9/439: dwrite d1/da/d36/d50/f73 [0,4194304] 0 2024-02-15T18:34:55.957 INFO:tasks.workunit.client.0.smithi038.stdout:8/576: dwrite d1/d4/d10/d43/d48/d52/fbd [0,4194304] 0 2024-02-15T18:34:55.959 INFO:tasks.workunit.client.0.smithi038.stdout:7/425: getdents d0/db/d59/d15/d29 0 2024-02-15T18:34:55.959 INFO:tasks.workunit.client.0.smithi038.stdout:7/426: write d0/db/d59/d15/f44 [5032439,122524] 0 2024-02-15T18:34:55.960 INFO:tasks.workunit.client.0.smithi038.stdout:1/426: dwrite d2/d2e/f38 [0,4194304] 0 2024-02-15T18:34:55.960 INFO:tasks.workunit.client.0.smithi038.stdout:1/427: write d2/f4 [4370675,93498] 0 2024-02-15T18:34:55.965 INFO:tasks.workunit.client.0.smithi038.stdout:3/364: dread d2/d25/f30 [0,4194304] 0 2024-02-15T18:34:55.967 INFO:tasks.workunit.client.0.smithi038.stdout:9/440: mknod d1/da/d10/d13/d3d/ca5 0 2024-02-15T18:34:55.967 INFO:tasks.workunit.client.0.smithi038.stdout:9/441: fdatasync d1/dc/d2b/f60 0 2024-02-15T18:34:55.971 INFO:tasks.workunit.client.0.smithi038.stdout:2/360: dread dd/d12/f36 [4194304,4194304] 0 2024-02-15T18:34:55.971 INFO:tasks.workunit.client.0.smithi038.stdout:6/402: dwrite d2/d4/f2f [0,4194304] 0 2024-02-15T18:34:55.972 INFO:tasks.workunit.client.0.smithi038.stdout:6/403: dread - d2/d4/d19/d55/f6a zero size 2024-02-15T18:34:55.972 INFO:tasks.workunit.client.0.smithi038.stdout:2/361: write dd/d12/d1d/f35 [1821897,75411] 0 2024-02-15T18:34:55.972 INFO:tasks.workunit.client.0.smithi038.stdout:0/473: sync 2024-02-15T18:34:55.979 INFO:tasks.workunit.client.0.smithi038.stdout:8/577: mkdir d1/d4/d10/d3d/db7/dd5 0 2024-02-15T18:34:55.980 INFO:tasks.workunit.client.0.smithi038.stdout:7/427: rmdir d0/d66 39 2024-02-15T18:34:55.980 INFO:tasks.workunit.client.0.smithi038.stdout:7/428: chown d0/d1b/d30/d50/f72 169 1 2024-02-15T18:34:55.982 INFO:tasks.workunit.client.0.smithi038.stdout:1/428: dwrite d2/d10/d19/d22/f7a [0,4194304] 0 2024-02-15T18:34:55.982 INFO:tasks.workunit.client.0.smithi038.stdout:1/429: chown d2/d10/d19/d22 1510663 1 2024-02-15T18:34:55.983 INFO:tasks.workunit.client.0.smithi038.stdout:1/430: readlink d2/l8 0 2024-02-15T18:34:55.983 INFO:tasks.workunit.client.0.smithi038.stdout:3/365: dwrite d2/d1b/d3c/d3a/f68 [0,4194304] 0 2024-02-15T18:34:55.988 INFO:tasks.workunit.client.0.smithi038.stdout:9/442: creat d1/dc/d2b/fa6 x:0 0 0 2024-02-15T18:34:55.988 INFO:tasks.workunit.client.0.smithi038.stdout:9/443: write d1/dc/d4f/f62 [92975,42426] 0 2024-02-15T18:34:55.988 INFO:tasks.workunit.client.0.smithi038.stdout:6/404: creat d2/d4/d19/d55/d5c/f70 x:0 0 0 2024-02-15T18:34:55.989 INFO:tasks.workunit.client.0.smithi038.stdout:9/444: dread d1/da/d10/d13/d3d/f5c [0,4194304] 0 2024-02-15T18:34:55.990 INFO:tasks.workunit.client.0.smithi038.stdout:2/362: mknod dd/d12/d1d/d3b/c78 0 2024-02-15T18:34:55.990 INFO:tasks.workunit.client.0.smithi038.stdout:2/363: truncate dd/d12/d3a/d67/f29 740693 0 2024-02-15T18:34:55.991 INFO:tasks.workunit.client.0.smithi038.stdout:0/474: rename d4/d13/d2a/d35/d1e/d32/d48/c51 to d4/d13/d2a/d35/d6a/c9c 0 2024-02-15T18:34:55.992 INFO:tasks.workunit.client.0.smithi038.stdout:8/578: getdents d1/d4/d5/d77 0 2024-02-15T18:34:55.992 INFO:tasks.workunit.client.0.smithi038.stdout:0/475: read d4/d13/f39 [631903,41709] 0 2024-02-15T18:34:55.993 INFO:tasks.workunit.client.0.smithi038.stdout:7/429: unlink d0/db/f7c 0 2024-02-15T18:34:55.994 INFO:tasks.workunit.client.0.smithi038.stdout:1/431: creat d2/d10/d19/d22/f9b x:0 0 0 2024-02-15T18:34:55.994 INFO:tasks.workunit.client.0.smithi038.stdout:1/432: chown d2/l35 62809 1 2024-02-15T18:34:55.996 INFO:tasks.workunit.client.0.smithi038.stdout:3/366: mkdir d2/d25/d71 0 2024-02-15T18:34:55.997 INFO:tasks.workunit.client.0.smithi038.stdout:6/405: creat d2/d4/d2c/d39/f71 x:0 0 0 2024-02-15T18:34:55.999 INFO:tasks.workunit.client.0.smithi038.stdout:9/445: mknod d1/da/d10/d19/ca7 0 2024-02-15T18:34:55.999 INFO:tasks.workunit.client.0.smithi038.stdout:9/446: chown d1/f14 2 1 2024-02-15T18:34:56.003 INFO:tasks.workunit.client.0.smithi038.stdout:2/364: link dd/d12/f47 dd/f79 0 2024-02-15T18:34:56.006 INFO:tasks.workunit.client.0.smithi038.stdout:8/579: rename d1/d4/d5/d31/d65/f76 to d1/d7/fd6 0 2024-02-15T18:34:56.007 INFO:tasks.workunit.client.0.smithi038.stdout:6/406: dread d2/d4/f4a [0,4194304] 0 2024-02-15T18:34:56.007 INFO:tasks.workunit.client.0.smithi038.stdout:7/430: mkdir d0/d1b/d30/d5d/d61/d90 0 2024-02-15T18:34:56.007 INFO:tasks.workunit.client.0.smithi038.stdout:1/433: getdents d2/d10/d19/d2f/d3e 0 2024-02-15T18:34:56.009 INFO:tasks.workunit.client.0.smithi038.stdout:3/367: rmdir d2/d1b/d3c 39 2024-02-15T18:34:56.009 INFO:tasks.workunit.client.0.smithi038.stdout:3/368: write d2/d25/f64 [523335,23040] 0 2024-02-15T18:34:56.010 INFO:tasks.workunit.client.0.smithi038.stdout:7/431: write d0/db/fe [1209117,84071] 0 2024-02-15T18:34:56.010 INFO:tasks.workunit.client.0.smithi038.stdout:2/365: truncate dd/d12/f39 4807878 0 2024-02-15T18:34:56.011 INFO:tasks.workunit.client.0.smithi038.stdout:6/407: getdents d2 0 2024-02-15T18:34:56.015 INFO:tasks.workunit.client.0.smithi038.stdout:1/434: unlink d2/d10/d19/d2f/d3e/c90 0 2024-02-15T18:34:56.015 INFO:tasks.workunit.client.0.smithi038.stdout:2/366: dread dd/d12/d3a/d67/f27 [0,4194304] 0 2024-02-15T18:34:56.015 INFO:tasks.workunit.client.0.smithi038.stdout:3/369: rename d2/d24/c2a to d2/c72 0 2024-02-15T18:34:56.015 INFO:tasks.workunit.client.0.smithi038.stdout:9/447: dwrite d1/dc/fe [0,4194304] 0 2024-02-15T18:34:56.015 INFO:tasks.workunit.client.0.smithi038.stdout:9/448: chown d1/da/d36/d3f 6746396 1 2024-02-15T18:34:56.015 INFO:tasks.workunit.client.0.smithi038.stdout:7/432: creat d0/db/d59/d15/d29/f91 x:0 0 0 2024-02-15T18:34:56.019 INFO:tasks.workunit.client.0.smithi038.stdout:0/476: dwrite d4/d13/d2a/d35/d1e/d32/f76 [0,4194304] 0 2024-02-15T18:34:56.019 INFO:tasks.workunit.client.0.smithi038.stdout:8/580: dread d1/d4/d10/d43/f46 [0,4194304] 0 2024-02-15T18:34:56.019 INFO:tasks.workunit.client.0.smithi038.stdout:6/408: mknod d2/c72 0 2024-02-15T18:34:56.021 INFO:tasks.workunit.client.0.smithi038.stdout:2/367: stat dd/d12/d1d/d3b/d3c/l52 0 2024-02-15T18:34:56.022 INFO:tasks.workunit.client.0.smithi038.stdout:3/370: creat d2/d1b/d3c/d41/d4e/f73 x:0 0 0 2024-02-15T18:34:56.025 INFO:tasks.workunit.client.0.smithi038.stdout:0/477: dread d4/ff [0,4194304] 0 2024-02-15T18:34:56.025 INFO:tasks.workunit.client.0.smithi038.stdout:0/478: read - d4/d13/d2a/d35/d1e/d32/d79/d7d/f86 zero size 2024-02-15T18:34:56.025 INFO:tasks.workunit.client.0.smithi038.stdout:9/449: mknod d1/da/d36/d3f/d48/ca8 0 2024-02-15T18:34:56.025 INFO:tasks.workunit.client.0.smithi038.stdout:3/371: write d2/f38 [536841,69473] 0 2024-02-15T18:34:56.026 INFO:tasks.workunit.client.0.smithi038.stdout:7/433: creat d0/d66/f92 x:0 0 0 2024-02-15T18:34:56.028 INFO:tasks.workunit.client.0.smithi038.stdout:8/581: mknod d1/d4/cd7 0 2024-02-15T18:34:56.028 INFO:tasks.workunit.client.0.smithi038.stdout:8/582: fdatasync d1/fb 0 2024-02-15T18:34:56.028 INFO:tasks.workunit.client.0.smithi038.stdout:8/583: stat d1/d4/d5 0 2024-02-15T18:34:56.030 INFO:tasks.workunit.client.0.smithi038.stdout:9/450: creat d1/da/d10/d82/fa9 x:0 0 0 2024-02-15T18:34:56.030 INFO:tasks.workunit.client.0.smithi038.stdout:9/451: chown d1/da/d20/d58/d76/f8c 60156 1 2024-02-15T18:34:56.030 INFO:tasks.workunit.client.0.smithi038.stdout:9/452: chown d1/da/d10/d24/f7c 325129 1 2024-02-15T18:34:56.032 INFO:tasks.workunit.client.0.smithi038.stdout:7/434: link d0/db/d59/f7 d0/d68/f93 0 2024-02-15T18:34:56.033 INFO:tasks.workunit.client.0.smithi038.stdout:9/453: write d1/dc/d2b/f60 [4396712,25225] 0 2024-02-15T18:34:56.033 INFO:tasks.workunit.client.0.smithi038.stdout:8/584: mkdir d1/d4/db1/dd8 0 2024-02-15T18:34:56.034 INFO:tasks.workunit.client.0.smithi038.stdout:6/409: rename d2/d4/d2c/c32 to d2/d4/d2c/d39/c73 0 2024-02-15T18:34:56.036 INFO:tasks.workunit.client.0.smithi038.stdout:8/585: getdents d1/d4/d5/d31/d65 0 2024-02-15T18:34:56.036 INFO:tasks.workunit.client.0.smithi038.stdout:1/435: dwrite d2/d10/d19/d22/f57 [0,4194304] 0 2024-02-15T18:34:56.036 INFO:tasks.workunit.client.0.smithi038.stdout:3/372: dread d2/fa [0,4194304] 0 2024-02-15T18:34:56.038 INFO:tasks.workunit.client.0.smithi038.stdout:7/435: rename d0/d1b/d30/d50/f72 to d0/db/d19/d7d/f94 0 2024-02-15T18:34:56.038 INFO:tasks.workunit.client.0.smithi038.stdout:7/436: dread - d0/db/d59/d15/f71 zero size 2024-02-15T18:34:56.042 INFO:tasks.workunit.client.0.smithi038.stdout:6/410: creat d2/d4/d19/f74 x:0 0 0 2024-02-15T18:34:56.042 INFO:tasks.workunit.client.0.smithi038.stdout:8/586: mkdir d1/d4/d10/d43/d48/d9a/dd9 0 2024-02-15T18:34:56.042 INFO:tasks.workunit.client.0.smithi038.stdout:8/587: chown d1/d4/d5/d6/f38 4044195 1 2024-02-15T18:34:56.046 INFO:tasks.workunit.client.0.smithi038.stdout:1/436: creat d2/d10/d19/d22/d36/d94/f9c x:0 0 0 2024-02-15T18:34:56.046 INFO:tasks.workunit.client.0.smithi038.stdout:3/373: mkdir d2/d25/d60/d74 0 2024-02-15T18:34:56.047 INFO:tasks.workunit.client.0.smithi038.stdout:3/374: getdents d2/d1b/d3c/d41/d4e/d62 0 2024-02-15T18:34:56.047 INFO:tasks.workunit.client.0.smithi038.stdout:8/588: dread d1/f40 [0,4194304] 0 2024-02-15T18:34:56.047 INFO:tasks.workunit.client.0.smithi038.stdout:8/589: read - d1/d7/d15/d4b/f57 zero size 2024-02-15T18:34:56.048 INFO:tasks.workunit.client.0.smithi038.stdout:8/590: write d1/d7/d15/d4b/fc7 [680912,114707] 0 2024-02-15T18:34:56.048 INFO:tasks.workunit.client.0.smithi038.stdout:6/411: stat d2/d10/f6b 0 2024-02-15T18:34:56.049 INFO:tasks.workunit.client.0.smithi038.stdout:3/375: write d2/d1b/f36 [1938553,109931] 0 2024-02-15T18:34:56.050 INFO:tasks.workunit.client.0.smithi038.stdout:1/437: rename d2/d10/d62/f88 to d2/f9d 0 2024-02-15T18:34:56.050 INFO:tasks.workunit.client.0.smithi038.stdout:1/438: write d2/d10/d19/d22/d36/d94/f9c [688929,56526] 0 2024-02-15T18:34:56.050 INFO:tasks.workunit.client.0.smithi038.stdout:2/368: dwrite dd/d12/d1d/f31 [0,4194304] 0 2024-02-15T18:34:56.050 INFO:tasks.workunit.client.0.smithi038.stdout:8/591: getdents d1 0 2024-02-15T18:34:56.051 INFO:tasks.workunit.client.0.smithi038.stdout:0/479: dwrite d4/f88 [0,4194304] 0 2024-02-15T18:34:56.051 INFO:tasks.workunit.client.0.smithi038.stdout:8/592: fdatasync d1/d3a/d58/f60 0 2024-02-15T18:34:56.051 INFO:tasks.workunit.client.0.smithi038.stdout:9/454: dwrite d1/da/d10/d13/d3d/d4b/f72 [0,4194304] 0 2024-02-15T18:34:56.051 INFO:tasks.workunit.client.0.smithi038.stdout:9/455: chown d1/fa1 1222970 1 2024-02-15T18:34:56.051 INFO:tasks.workunit.client.0.smithi038.stdout:9/456: truncate d1/da/d10/d13/d3d/fa2 824566 0 2024-02-15T18:34:56.057 INFO:tasks.workunit.client.0.smithi038.stdout:6/412: rmdir d2/d4/d2c/d39 39 2024-02-15T18:34:56.059 INFO:tasks.workunit.client.0.smithi038.stdout:0/480: dread d4/d13/d2a/d35/d1e/f20 [0,4194304] 0 2024-02-15T18:34:56.063 INFO:tasks.workunit.client.0.smithi038.stdout:4/427: sync 2024-02-15T18:34:56.063 INFO:tasks.workunit.client.0.smithi038.stdout:5/385: sync 2024-02-15T18:34:56.063 INFO:tasks.workunit.client.0.smithi038.stdout:4/428: dread - d11/d1e/d20/d31/f91 zero size 2024-02-15T18:34:56.063 INFO:tasks.workunit.client.0.smithi038.stdout:5/386: getdents d2/d30/d57 0 2024-02-15T18:34:56.063 INFO:tasks.workunit.client.0.smithi038.stdout:3/376: creat d2/d1b/d3c/d41/d4e/d62/f75 x:0 0 0 2024-02-15T18:34:56.063 INFO:tasks.workunit.client.0.smithi038.stdout:3/377: write d2/f12 [4408752,4717] 0 2024-02-15T18:34:56.066 INFO:tasks.workunit.client.0.smithi038.stdout:1/439: mknod d2/d10/d32/d53/d82/c9e 0 2024-02-15T18:34:56.066 INFO:tasks.workunit.client.0.smithi038.stdout:1/440: fdatasync d2/d28/f50 0 2024-02-15T18:34:56.068 INFO:tasks.workunit.client.0.smithi038.stdout:7/437: rename d0/d2e to d0/d95 0 2024-02-15T18:34:56.068 INFO:tasks.workunit.client.0.smithi038.stdout:7/438: dread - d0/db/d59/d15/d29/d7e/f73 zero size 2024-02-15T18:34:56.069 INFO:tasks.workunit.client.0.smithi038.stdout:7/439: fdatasync d0/d1b/d30/d5d/d61/f75 0 2024-02-15T18:34:56.070 INFO:tasks.workunit.client.0.smithi038.stdout:2/369: rename dd/d12/d1d/d5f to dd/d12/d59/d2b/d7a 0 2024-02-15T18:34:56.070 INFO:tasks.workunit.client.0.smithi038.stdout:8/593: mknod d1/d4/d5/d6/db9/cda 0 2024-02-15T18:34:56.072 INFO:tasks.workunit.client.0.smithi038.stdout:9/457: unlink d1/dc/d38/l9d 0 2024-02-15T18:34:56.072 INFO:tasks.workunit.client.0.smithi038.stdout:9/458: readlink d1/da/d36/d3f/d48/l79 0 2024-02-15T18:34:56.072 INFO:tasks.workunit.client.0.smithi038.stdout:9/459: dread - d1/da/d10/d19/f42 zero size 2024-02-15T18:34:56.072 INFO:tasks.workunit.client.0.smithi038.stdout:6/413: dread - d2/f5b zero size 2024-02-15T18:34:56.072 INFO:tasks.workunit.client.0.smithi038.stdout:4/429: symlink d11/d1e/d20/d97/la5 0 2024-02-15T18:34:56.074 INFO:tasks.workunit.client.0.smithi038.stdout:3/378: creat d2/d25/d60/f76 x:0 0 0 2024-02-15T18:34:56.075 INFO:tasks.workunit.client.0.smithi038.stdout:1/441: link d2/d10/c24 d2/d28/d3c/c9f 0 2024-02-15T18:34:56.077 INFO:tasks.workunit.client.0.smithi038.stdout:0/481: dwrite d4/d13/d53/f95 [0,4194304] 0 2024-02-15T18:34:56.077 INFO:tasks.workunit.client.0.smithi038.stdout:1/442: read d2/d10/d19/d22/d36/f4d [1817197,100966] 0 2024-02-15T18:34:56.078 INFO:tasks.workunit.client.0.smithi038.stdout:7/440: mknod d0/db/d59/c96 0 2024-02-15T18:34:56.078 INFO:tasks.workunit.client.0.smithi038.stdout:7/441: fdatasync d0/db/d59/d15/d29/f91 0 2024-02-15T18:34:56.078 INFO:tasks.workunit.client.0.smithi038.stdout:7/442: chown d0/db/d19/c38 105 1 2024-02-15T18:34:56.078 INFO:tasks.workunit.client.0.smithi038.stdout:7/443: chown d0/d66/c83 0 1 2024-02-15T18:34:56.079 INFO:tasks.workunit.client.0.smithi038.stdout:5/387: rename d2/d3/l41 to d2/d30/d57/l76 0 2024-02-15T18:34:56.079 INFO:tasks.workunit.client.0.smithi038.stdout:2/370: symlink dd/d12/d59/d2b/d7a/l7b 0 2024-02-15T18:34:56.080 INFO:tasks.workunit.client.0.smithi038.stdout:7/444: write d0/db/fc [1654794,66707] 0 2024-02-15T18:34:56.080 INFO:tasks.workunit.client.0.smithi038.stdout:7/445: read - d0/db/d59/d15/d29/d7e/f5b zero size 2024-02-15T18:34:56.080 INFO:tasks.workunit.client.0.smithi038.stdout:7/446: chown d0/db/d19/d2d 5 1 2024-02-15T18:34:56.081 INFO:tasks.workunit.client.0.smithi038.stdout:8/594: rmdir d1/d4/d10/d3d 39 2024-02-15T18:34:56.081 INFO:tasks.workunit.client.0.smithi038.stdout:8/595: fsync d1/d4/d5/d31/f7f 0 2024-02-15T18:34:56.082 INFO:tasks.workunit.client.0.smithi038.stdout:9/460: mkdir d1/da/d10/d13/d3d/d4b/daa 0 2024-02-15T18:34:56.082 INFO:tasks.workunit.client.0.smithi038.stdout:9/461: write d1/dc/d4f/f62 [446035,10612] 0 2024-02-15T18:34:56.085 INFO:tasks.workunit.client.0.smithi038.stdout:0/482: rmdir d4/d13/d2a/d35/d1e/d32/d79/d7d 39 2024-02-15T18:34:56.088 INFO:tasks.workunit.client.0.smithi038.stdout:8/596: link d1/d7/l9c d1/d7/d26/d69/ldb 0 2024-02-15T18:34:56.092 INFO:tasks.workunit.client.0.smithi038.stdout:9/462: creat d1/da/d10/d13/d3d/d4b/daa/fab x:0 0 0 2024-02-15T18:34:56.093 INFO:tasks.workunit.client.0.smithi038.stdout:6/414: mkdir d2/d56/d75 0 2024-02-15T18:34:56.101 INFO:tasks.workunit.client.0.smithi038.stdout:8/597: rename d1/d4/d5/d31/d65/d80/fa0 to d1/d7/d26/fdc 0 2024-02-15T18:34:56.101 INFO:tasks.workunit.client.0.smithi038.stdout:9/463: creat d1/da/d10/d19/fac x:0 0 0 2024-02-15T18:34:56.102 INFO:tasks.workunit.client.0.smithi038.stdout:4/430: dwrite d11/d1e/d3e/f47 [4194304,4194304] 0 2024-02-15T18:34:56.103 INFO:tasks.workunit.client.0.smithi038.stdout:8/598: creat d1/d3a/d58/fdd x:0 0 0 2024-02-15T18:34:56.103 INFO:tasks.workunit.client.0.smithi038.stdout:8/599: chown d1/d4/d10/d3d/c67 1 1 2024-02-15T18:34:56.104 INFO:tasks.workunit.client.0.smithi038.stdout:9/464: symlink d1/dc/lad 0 2024-02-15T18:34:56.104 INFO:tasks.workunit.client.0.smithi038.stdout:9/465: fsync d1/dc/fe 0 2024-02-15T18:34:56.106 INFO:tasks.workunit.client.0.smithi038.stdout:7/447: dread d0/d1b/f78 [0,4194304] 0 2024-02-15T18:34:56.106 INFO:tasks.workunit.client.0.smithi038.stdout:7/448: dread - d0/d1b/d30/f40 zero size 2024-02-15T18:34:56.110 INFO:tasks.workunit.client.0.smithi038.stdout:4/431: rmdir d11/d1e/d3e/d57 39 2024-02-15T18:34:56.111 INFO:tasks.workunit.client.0.smithi038.stdout:8/600: mknod d1/d4/db1/dd8/cde 0 2024-02-15T18:34:56.114 INFO:tasks.workunit.client.0.smithi038.stdout:4/432: mkdir d11/d1e/d20/d31/d8d/d99/da6 0 2024-02-15T18:34:56.114 INFO:tasks.workunit.client.0.smithi038.stdout:8/601: symlink d1/d4/d5/d31/ldf 0 2024-02-15T18:34:56.114 INFO:tasks.workunit.client.0.smithi038.stdout:4/433: truncate f4 4296291 0 2024-02-15T18:34:56.116 INFO:tasks.workunit.client.0.smithi038.stdout:8/602: unlink d1/d4/d10/cc6 0 2024-02-15T18:34:56.117 INFO:tasks.workunit.client.0.smithi038.stdout:4/434: symlink d11/d1e/d20/d2b/d4d/d67/da1/la7 0 2024-02-15T18:34:56.119 INFO:tasks.workunit.client.0.smithi038.stdout:4/435: symlink d11/d1e/la8 0 2024-02-15T18:34:56.119 INFO:tasks.workunit.client.0.smithi038.stdout:4/436: chown d11/d1e/d20/d2b/d35/f55 4 1 2024-02-15T18:34:56.127 INFO:tasks.workunit.client.0.smithi038.stdout:6/415: dread d2/d4/fc [4194304,4194304] 0 2024-02-15T18:34:56.127 INFO:tasks.workunit.client.0.smithi038.stdout:6/416: mkdir d2/d4/d2c/d39/d76 0 2024-02-15T18:34:56.127 INFO:tasks.workunit.client.0.smithi038.stdout:6/417: write d2/d4/d19/f53 [151272,1260] 0 2024-02-15T18:34:56.129 INFO:tasks.workunit.client.0.smithi038.stdout:6/418: link d2/d4/f1e d2/d4/d19/d55/d5c/f77 0 2024-02-15T18:34:56.130 INFO:tasks.workunit.client.0.smithi038.stdout:6/419: mkdir d2/d4/d19/d78 0 2024-02-15T18:34:56.130 INFO:tasks.workunit.client.0.smithi038.stdout:6/420: creat d2/d10/f79 x:0 0 0 2024-02-15T18:34:56.131 INFO:tasks.workunit.client.0.smithi038.stdout:6/421: fsync d2/f2a 0 2024-02-15T18:34:56.131 INFO:tasks.workunit.client.0.smithi038.stdout:6/422: write d2/d4/d19/d55/f6a [535180,20096] 0 2024-02-15T18:34:56.131 INFO:tasks.workunit.client.0.smithi038.stdout:6/423: write d2/d10/f4c [748148,103493] 0 2024-02-15T18:34:56.132 INFO:tasks.workunit.client.0.smithi038.stdout:3/379: dwrite d2/f2f [0,4194304] 0 2024-02-15T18:34:56.132 INFO:tasks.workunit.client.0.smithi038.stdout:6/424: mkdir d2/d4/d19/d78/d7a 0 2024-02-15T18:34:56.134 INFO:tasks.workunit.client.0.smithi038.stdout:3/380: symlink d2/d1b/d3c/d3a/l77 0 2024-02-15T18:34:56.134 INFO:tasks.workunit.client.0.smithi038.stdout:6/425: truncate d2/d10/f1d 1833098 0 2024-02-15T18:34:56.146 INFO:tasks.workunit.client.0.smithi038.stdout:4/437: read d11/f1f [4115330,86506] 0 2024-02-15T18:34:56.147 INFO:tasks.workunit.client.0.smithi038.stdout:4/438: symlink d11/d1e/d20/d31/d48/d60/la9 0 2024-02-15T18:34:56.149 INFO:tasks.workunit.client.0.smithi038.stdout:5/388: dwrite d2/d3/f2f [4194304,4194304] 0 2024-02-15T18:34:56.149 INFO:tasks.workunit.client.0.smithi038.stdout:2/371: dwrite dd/d12/d1d/f31 [4194304,4194304] 0 2024-02-15T18:34:56.149 INFO:tasks.workunit.client.0.smithi038.stdout:9/466: dwrite d1/dc/d4f/f47 [0,4194304] 0 2024-02-15T18:34:56.149 INFO:tasks.workunit.client.0.smithi038.stdout:4/439: creat d11/d1e/d3e/d75/d40/d6f/faa x:0 0 0 2024-02-15T18:34:56.151 INFO:tasks.workunit.client.0.smithi038.stdout:2/372: mknod dd/d12/d59/d2b/d6a/c7c 0 2024-02-15T18:34:56.154 INFO:tasks.workunit.client.0.smithi038.stdout:9/467: link d1/dc/d5a/l85 d1/da/d10/d13/d3d/d4b/lae 0 2024-02-15T18:34:56.154 INFO:tasks.workunit.client.0.smithi038.stdout:5/389: dread d2/d3/d5f/d71/f37 [0,4194304] 0 2024-02-15T18:34:56.154 INFO:tasks.workunit.client.0.smithi038.stdout:3/381: write d2/d1e/f44 [2612013,129580] 0 2024-02-15T18:34:56.155 INFO:tasks.workunit.client.0.smithi038.stdout:4/440: mknod d11/d1e/d20/d97/cab 0 2024-02-15T18:34:56.155 INFO:tasks.workunit.client.0.smithi038.stdout:4/441: chown d11/d1e/d20/d2b/f30 133 1 2024-02-15T18:34:56.155 INFO:tasks.workunit.client.0.smithi038.stdout:2/373: unlink dd/d12/l2a 0 2024-02-15T18:34:56.155 INFO:tasks.workunit.client.0.smithi038.stdout:2/374: write dd/d12/d1d/d3b/f6f [624027,60880] 0 2024-02-15T18:34:56.156 INFO:tasks.workunit.client.0.smithi038.stdout:9/468: unlink d1/da/d10/d24/d49/c51 0 2024-02-15T18:34:56.157 INFO:tasks.workunit.client.0.smithi038.stdout:0/483: dwrite d4/d13/d2a/d35/f49 [0,4194304] 0 2024-02-15T18:34:56.157 INFO:tasks.workunit.client.0.smithi038.stdout:0/484: truncate d4/d13/d2a/d35/d1e/f5d 479531 0 2024-02-15T18:34:56.157 INFO:tasks.workunit.client.0.smithi038.stdout:1/443: dwrite d2/d10/d19/d22/d36/f44 [0,4194304] 0 2024-02-15T18:34:56.157 INFO:tasks.workunit.client.0.smithi038.stdout:8/603: dwrite d1/d4/d5/d6/fcb [0,4194304] 0 2024-02-15T18:34:56.158 INFO:tasks.workunit.client.0.smithi038.stdout:8/604: readlink d1/d4/d10/d43/d48/lae 0 2024-02-15T18:34:56.158 INFO:tasks.workunit.client.0.smithi038.stdout:5/390: rename d2/c44 to d2/d3/c77 0 2024-02-15T18:34:56.158 INFO:tasks.workunit.client.0.smithi038.stdout:7/449: dwrite d0/db/d59/d15/d29/d8d/f8f [0,4194304] 0 2024-02-15T18:34:56.159 INFO:tasks.workunit.client.0.smithi038.stdout:3/382: mknod d2/d1b/d3c/d42/c78 0 2024-02-15T18:34:56.159 INFO:tasks.workunit.client.0.smithi038.stdout:9/469: dread d1/da/d10/d24/f87 [0,4194304] 0 2024-02-15T18:34:56.167 INFO:tasks.workunit.client.0.smithi038.stdout:6/426: rmdir d2/d4/d19/d55/d5c 39 2024-02-15T18:34:56.168 INFO:tasks.workunit.client.0.smithi038.stdout:5/391: write d2/d48/f59 [1054180,56868] 0 2024-02-15T18:34:56.173 INFO:tasks.workunit.client.0.smithi038.stdout:2/375: symlink dd/d12/d1d/d3b/l7d 0 2024-02-15T18:34:56.174 INFO:tasks.workunit.client.0.smithi038.stdout:0/485: symlink d4/d13/d19/l9d 0 2024-02-15T18:34:56.175 INFO:tasks.workunit.client.0.smithi038.stdout:2/376: write dd/f79 [2269500,46048] 0 2024-02-15T18:34:56.176 INFO:tasks.workunit.client.0.smithi038.stdout:4/442: dwrite d11/d1e/f37 [0,4194304] 0 2024-02-15T18:34:56.176 INFO:tasks.workunit.client.0.smithi038.stdout:1/444: unlink d2/d10/l61 0 2024-02-15T18:34:56.178 INFO:tasks.workunit.client.0.smithi038.stdout:8/605: creat d1/d4/d10/d43/d48/d52/fe0 x:0 0 0 2024-02-15T18:34:56.180 INFO:tasks.workunit.client.0.smithi038.stdout:7/450: mkdir d0/db/d59/d97 0 2024-02-15T18:34:56.186 INFO:tasks.workunit.client.0.smithi038.stdout:9/470: creat d1/dc/faf x:0 0 0 2024-02-15T18:34:56.186 INFO:tasks.workunit.client.0.smithi038.stdout:6/427: rmdir d2/d4/d19/d78 39 2024-02-15T18:34:56.190 INFO:tasks.workunit.client.0.smithi038.stdout:5/392: creat d2/d30/d57/f78 x:0 0 0 2024-02-15T18:34:56.191 INFO:tasks.workunit.client.0.smithi038.stdout:6/428: dread d2/f2a [0,4194304] 0 2024-02-15T18:34:56.192 INFO:tasks.workunit.client.0.smithi038.stdout:0/486: creat d4/d13/d2a/d35/d6a/f9e x:0 0 0 2024-02-15T18:34:56.193 INFO:tasks.workunit.client.0.smithi038.stdout:0/487: chown d4/d13/f8e 289999693 1 2024-02-15T18:34:56.194 INFO:tasks.workunit.client.0.smithi038.stdout:2/377: symlink dd/d12/d59/d2b/l7e 0 2024-02-15T18:34:56.196 INFO:tasks.workunit.client.0.smithi038.stdout:4/443: mkdir d11/d1e/d20/d31/d48/dac 0 2024-02-15T18:34:56.198 INFO:tasks.workunit.client.0.smithi038.stdout:1/445: link d2/d10/f31 d2/d2e/d8c/fa0 0 2024-02-15T18:34:56.206 INFO:tasks.workunit.client.0.smithi038.stdout:8/606: rename d1/d4/d5/d6/f38 to d1/d7/d26/fe1 0 2024-02-15T18:34:56.207 INFO:tasks.workunit.client.0.smithi038.stdout:7/451: link d0/d66/f1c d0/db/f98 0 2024-02-15T18:34:56.207 INFO:tasks.workunit.client.0.smithi038.stdout:7/452: chown d0/db/f42 2071 1 2024-02-15T18:34:56.207 INFO:tasks.workunit.client.0.smithi038.stdout:9/471: truncate d1/dc/d2b/f3a 858873 0 2024-02-15T18:34:56.208 INFO:tasks.workunit.client.0.smithi038.stdout:5/393: creat d2/d30/d57/f79 x:0 0 0 2024-02-15T18:34:56.208 INFO:tasks.workunit.client.0.smithi038.stdout:0/488: getdents d4/d13/d2a/d35/d1e/d32/d3b 0 2024-02-15T18:34:56.210 INFO:tasks.workunit.client.0.smithi038.stdout:2/378: creat dd/d12/d3a/d67/d4f/d5a/f7f x:0 0 0 2024-02-15T18:34:56.210 INFO:tasks.workunit.client.0.smithi038.stdout:6/429: dwrite d2/d4/d19/d55/d5c/f64 [0,4194304] 0 2024-02-15T18:34:56.210 INFO:tasks.workunit.client.0.smithi038.stdout:6/430: chown d2/d4/d2c/d39/c40 235 1 2024-02-15T18:34:56.215 INFO:tasks.workunit.client.0.smithi038.stdout:1/446: symlink d2/d10/d19/d22/d36/la1 0 2024-02-15T18:34:56.216 INFO:tasks.workunit.client.0.smithi038.stdout:8/607: truncate d1/d7/d26/d69/fc1 4013043 0 2024-02-15T18:34:56.219 INFO:tasks.workunit.client.0.smithi038.stdout:8/608: dread d1/d7/d26/f84 [0,4194304] 0 2024-02-15T18:34:56.219 INFO:tasks.workunit.client.0.smithi038.stdout:7/453: symlink d0/d1b/d30/d5d/d61/l99 0 2024-02-15T18:34:56.221 INFO:tasks.workunit.client.0.smithi038.stdout:4/444: dwrite d11/f17 [4194304,4194304] 0 2024-02-15T18:34:56.221 INFO:tasks.workunit.client.0.smithi038.stdout:9/472: creat d1/da/fb0 x:0 0 0 2024-02-15T18:34:56.222 INFO:tasks.workunit.client.0.smithi038.stdout:5/394: creat d2/f7a x:0 0 0 2024-02-15T18:34:56.224 INFO:tasks.workunit.client.0.smithi038.stdout:0/489: mknod d4/d13/d2a/d35/d6a/c9f 0 2024-02-15T18:34:56.227 INFO:tasks.workunit.client.0.smithi038.stdout:6/431: unlink d2/d10/c14 0 2024-02-15T18:34:56.227 INFO:tasks.workunit.client.0.smithi038.stdout:6/432: fsync d2/d10/f30 0 2024-02-15T18:34:56.227 INFO:tasks.workunit.client.0.smithi038.stdout:6/433: write d2/d4/d19/d55/f6a [1042191,9492] 0 2024-02-15T18:34:56.228 INFO:tasks.workunit.client.0.smithi038.stdout:1/447: dread d2/d10/d19/d22/f7a [0,4194304] 0 2024-02-15T18:34:56.228 INFO:tasks.workunit.client.0.smithi038.stdout:8/609: mkdir d1/d4/d10/d43/de2 0 2024-02-15T18:34:56.228 INFO:tasks.workunit.client.0.smithi038.stdout:8/610: chown d1/d4/d5/f37 0 1 2024-02-15T18:34:56.228 INFO:tasks.workunit.client.0.smithi038.stdout:8/611: chown d1/f6f 521214 1 2024-02-15T18:34:56.229 INFO:tasks.workunit.client.0.smithi038.stdout:7/454: creat d0/db/d59/d15/d29/d8d/f9a x:0 0 0 2024-02-15T18:34:56.233 INFO:tasks.workunit.client.0.smithi038.stdout:4/445: unlink d11/f1a 0 2024-02-15T18:34:56.235 INFO:tasks.workunit.client.0.smithi038.stdout:0/490: creat d4/d13/d2a/d80/fa0 x:0 0 0 2024-02-15T18:34:56.239 INFO:tasks.workunit.client.0.smithi038.stdout:2/379: rename dd/d12/d1d/f6c to dd/d12/f80 0 2024-02-15T18:34:56.239 INFO:tasks.workunit.client.0.smithi038.stdout:4/446: getdents d11/d1e/d3e 0 2024-02-15T18:34:56.240 INFO:tasks.workunit.client.0.smithi038.stdout:0/491: link d4/d13/d19/f82 d4/d13/d53/fa1 0 2024-02-15T18:34:56.242 INFO:tasks.workunit.client.0.smithi038.stdout:8/612: rename d1/d3a/d58/fdd to d1/d7/d26/d69/fe3 0 2024-02-15T18:34:56.244 INFO:tasks.workunit.client.0.smithi038.stdout:8/613: creat d1/d4/d10/d43/de2/fe4 x:0 0 0 2024-02-15T18:34:56.245 INFO:tasks.workunit.client.0.smithi038.stdout:2/380: truncate dd/d12/d59/f21 88148 0 2024-02-15T18:34:56.247 INFO:tasks.workunit.client.0.smithi038.stdout:8/614: rename d1/d4/d10/d43/d48/d52/d88 to d1/d4/db1/de5 0 2024-02-15T18:34:56.247 INFO:tasks.workunit.client.0.smithi038.stdout:8/615: fsync d1/d7/d15/d6a/f86 0 2024-02-15T18:34:56.248 INFO:tasks.workunit.client.0.smithi038.stdout:8/616: creat d1/d4/d10/d3d/fe6 x:0 0 0 2024-02-15T18:34:56.252 INFO:tasks.workunit.client.0.smithi038.stdout:1/448: dwrite d2/d10/d62/f93 [0,4194304] 0 2024-02-15T18:34:56.258 INFO:tasks.workunit.client.0.smithi038.stdout:5/395: dwrite d2/d3/d12/f28 [0,4194304] 0 2024-02-15T18:34:56.258 INFO:tasks.workunit.client.0.smithi038.stdout:4/447: dread d11/d1e/d20/d2b/d35/f89 [0,4194304] 0 2024-02-15T18:34:56.258 INFO:tasks.workunit.client.0.smithi038.stdout:4/448: chown d11/d1e/d20/d2b/d4d/l9e 881 1 2024-02-15T18:34:56.261 INFO:tasks.workunit.client.0.smithi038.stdout:5/396: creat d2/d30/d52/f7b x:0 0 0 2024-02-15T18:34:56.261 INFO:tasks.workunit.client.0.smithi038.stdout:0/492: dread d4/d13/f54 [0,4194304] 0 2024-02-15T18:34:56.261 INFO:tasks.workunit.client.0.smithi038.stdout:9/473: dwrite d1/da/d10/d24/f77 [0,4194304] 0 2024-02-15T18:34:56.261 INFO:tasks.workunit.client.0.smithi038.stdout:9/474: fdatasync d1/da/d10/d13/d3d/fa3 0 2024-02-15T18:34:56.262 INFO:tasks.workunit.client.0.smithi038.stdout:6/434: dwrite d2/f9 [0,4194304] 0 2024-02-15T18:34:56.262 INFO:tasks.workunit.client.0.smithi038.stdout:6/435: dread - d2/d4/d28/f69 zero size 2024-02-15T18:34:56.262 INFO:tasks.workunit.client.0.smithi038.stdout:6/436: dread - d2/d4/d28/f69 zero size 2024-02-15T18:34:56.263 INFO:tasks.workunit.client.0.smithi038.stdout:4/449: dread d11/d1e/f29 [0,4194304] 0 2024-02-15T18:34:56.267 INFO:tasks.workunit.client.0.smithi038.stdout:9/475: unlink d1/da/d10/f33 0 2024-02-15T18:34:56.267 INFO:tasks.workunit.client.0.smithi038.stdout:5/397: symlink d2/d3/l7c 0 2024-02-15T18:34:56.267 INFO:tasks.workunit.client.0.smithi038.stdout:5/398: truncate d2/d30/f73 122290 0 2024-02-15T18:34:56.269 INFO:tasks.workunit.client.0.smithi038.stdout:6/437: symlink d2/d4/d28/l7b 0 2024-02-15T18:34:56.269 INFO:tasks.workunit.client.0.smithi038.stdout:6/438: write f1 [4859962,100687] 0 2024-02-15T18:34:56.271 INFO:tasks.workunit.client.0.smithi038.stdout:7/455: dwrite d0/d1b/f3f [0,4194304] 0 2024-02-15T18:34:56.271 INFO:tasks.workunit.client.0.smithi038.stdout:7/456: dread - d0/d1b/d30/d5d/d70/f88 zero size 2024-02-15T18:34:56.271 INFO:tasks.workunit.client.0.smithi038.stdout:7/457: chown d0/d1b/d30 91292 1 2024-02-15T18:34:56.272 INFO:tasks.workunit.client.0.smithi038.stdout:4/450: truncate d11/d1e/d20/d2f/f3d 273624 0 2024-02-15T18:34:56.272 INFO:tasks.workunit.client.0.smithi038.stdout:2/381: dread dd/f79 [0,4194304] 0 2024-02-15T18:34:56.273 INFO:tasks.workunit.client.0.smithi038.stdout:7/458: write d0/db/d59/f7 [3208856,41825] 0 2024-02-15T18:34:56.273 INFO:tasks.workunit.client.0.smithi038.stdout:5/399: rmdir d2/d30/d57 39 2024-02-15T18:34:56.274 INFO:tasks.workunit.client.0.smithi038.stdout:9/476: creat d1/da/d10/d13/d3d/d4b/daa/fb1 x:0 0 0 2024-02-15T18:34:56.275 INFO:tasks.workunit.client.0.smithi038.stdout:6/439: unlink d2/d4/d19/f53 0 2024-02-15T18:34:56.278 INFO:tasks.workunit.client.0.smithi038.stdout:4/451: creat d11/d1e/d20/d2b/d4d/d67/d73/d8b/fad x:0 0 0 2024-02-15T18:34:56.279 INFO:tasks.workunit.client.0.smithi038.stdout:8/617: dwrite d1/d7/d15/f2f [0,4194304] 0 2024-02-15T18:34:56.279 INFO:tasks.workunit.client.0.smithi038.stdout:2/382: rename dd/d12/d5d/d33/l74 to dd/d12/d59/d2b/l81 0 2024-02-15T18:34:56.281 INFO:tasks.workunit.client.0.smithi038.stdout:0/493: dwrite d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/f92 [0,4194304] 0 2024-02-15T18:34:56.281 INFO:tasks.workunit.client.0.smithi038.stdout:0/494: chown d4/d13/f8e 213 1 2024-02-15T18:34:56.281 INFO:tasks.workunit.client.0.smithi038.stdout:2/383: write dd/d12/d1d/f2e [3910415,68079] 0 2024-02-15T18:34:56.281 INFO:tasks.workunit.client.0.smithi038.stdout:1/449: dwrite d2/f4 [0,4194304] 0 2024-02-15T18:34:56.284 INFO:tasks.workunit.client.0.smithi038.stdout:7/459: symlink d0/d68/l9b 0 2024-02-15T18:34:56.284 INFO:tasks.workunit.client.0.smithi038.stdout:7/460: write d0/db/d59/d15/d29/f91 [941021,104126] 0 2024-02-15T18:34:56.288 INFO:tasks.workunit.client.0.smithi038.stdout:3/383: sync 2024-02-15T18:34:56.289 INFO:tasks.workunit.client.0.smithi038.stdout:9/477: truncate d1/da/d20/f26 644644 0 2024-02-15T18:34:56.291 INFO:tasks.workunit.client.0.smithi038.stdout:6/440: rmdir d2 39 2024-02-15T18:34:56.295 INFO:tasks.workunit.client.0.smithi038.stdout:8/618: rmdir d1/d7/d22 39 2024-02-15T18:34:56.296 INFO:tasks.workunit.client.0.smithi038.stdout:5/400: rename d2/d3/d5f/d71/c72 to d2/d30/c7d 0 2024-02-15T18:34:56.296 INFO:tasks.workunit.client.0.smithi038.stdout:5/401: stat d2/d30/d57 0 2024-02-15T18:34:56.299 INFO:tasks.workunit.client.0.smithi038.stdout:0/495: creat d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/fa2 x:0 0 0 2024-02-15T18:34:56.299 INFO:tasks.workunit.client.0.smithi038.stdout:2/384: mkdir dd/d12/d1d/d82 0 2024-02-15T18:34:56.299 INFO:tasks.workunit.client.0.smithi038.stdout:1/450: mknod d2/d28/d6b/ca2 0 2024-02-15T18:34:56.300 INFO:tasks.workunit.client.0.smithi038.stdout:1/451: chown d2/d10/d19/d22/f7a 605236345 1 2024-02-15T18:34:56.300 INFO:tasks.workunit.client.0.smithi038.stdout:1/452: readlink d2/d2e/d8c/l99 0 2024-02-15T18:34:56.301 INFO:tasks.workunit.client.0.smithi038.stdout:7/461: truncate d0/d66/f1c 1542881 0 2024-02-15T18:34:56.301 INFO:tasks.workunit.client.0.smithi038.stdout:1/453: read d2/d10/d19/d22/d36/f78 [1359454,5590] 0 2024-02-15T18:34:56.301 INFO:tasks.workunit.client.0.smithi038.stdout:3/384: creat d2/d1b/d3c/d3a/d55/f79 x:0 0 0 2024-02-15T18:34:56.303 INFO:tasks.workunit.client.0.smithi038.stdout:9/478: unlink d1/da/d36/d3f/d48/l79 0 2024-02-15T18:34:56.303 INFO:tasks.workunit.client.0.smithi038.stdout:9/479: chown d1/da/d10/d13/d3d/d43/f97 185114 1 2024-02-15T18:34:56.305 INFO:tasks.workunit.client.0.smithi038.stdout:4/452: truncate d11/d1e/d43/d4a/f59 2035349 0 2024-02-15T18:34:56.305 INFO:tasks.workunit.client.0.smithi038.stdout:8/619: symlink d1/d4/d10/d91/le7 0 2024-02-15T18:34:56.305 INFO:tasks.workunit.client.0.smithi038.stdout:6/441: creat d2/d56/f7c x:0 0 0 2024-02-15T18:34:56.308 INFO:tasks.workunit.client.0.smithi038.stdout:6/442: mknod d2/d56/c7d 0 2024-02-15T18:34:56.308 INFO:tasks.workunit.client.0.smithi038.stdout:6/443: truncate d2/f2a 2694523 0 2024-02-15T18:34:56.308 INFO:tasks.workunit.client.0.smithi038.stdout:9/480: write d1/dc/f28 [939979,20279] 0 2024-02-15T18:34:56.309 INFO:tasks.workunit.client.0.smithi038.stdout:3/385: write d2/f12 [1249671,62898] 0 2024-02-15T18:34:56.310 INFO:tasks.workunit.client.0.smithi038.stdout:9/481: mkdir d1/dc/db2 0 2024-02-15T18:34:56.310 INFO:tasks.workunit.client.0.smithi038.stdout:9/482: fdatasync d1/da/d10/d24/d49/f4e 0 2024-02-15T18:34:56.311 INFO:tasks.workunit.client.0.smithi038.stdout:2/385: dread dd/d12/d3a/d67/f30 [0,4194304] 0 2024-02-15T18:34:56.311 INFO:tasks.workunit.client.0.smithi038.stdout:7/462: dread d0/db/fc [0,4194304] 0 2024-02-15T18:34:56.311 INFO:tasks.workunit.client.0.smithi038.stdout:2/386: stat dd/d12/d1d/d3b/f4b 0 2024-02-15T18:34:56.311 INFO:tasks.workunit.client.0.smithi038.stdout:2/387: chown dd/d12/f47 155 1 2024-02-15T18:34:56.312 INFO:tasks.workunit.client.0.smithi038.stdout:7/463: dread d0/db/f42 [0,4194304] 0 2024-02-15T18:34:56.312 INFO:tasks.workunit.client.0.smithi038.stdout:7/464: chown d0/d1b/d30/d5d/d61/l99 30 1 2024-02-15T18:34:56.312 INFO:tasks.workunit.client.0.smithi038.stdout:7/465: chown d0/d1b/d30/d5d/f82 684 1 2024-02-15T18:34:56.313 INFO:tasks.workunit.client.0.smithi038.stdout:6/444: rename d2/d4/d28/f62 to d2/f7e 0 2024-02-15T18:34:56.313 INFO:tasks.workunit.client.0.smithi038.stdout:3/386: mkdir d2/d1b/d3c/d41/d7a 0 2024-02-15T18:34:56.313 INFO:tasks.workunit.client.0.smithi038.stdout:3/387: dread - d2/d25/f66 zero size 2024-02-15T18:34:56.313 INFO:tasks.workunit.client.0.smithi038.stdout:4/453: write d11/d1e/d20/f76 [2709367,34889] 0 2024-02-15T18:34:56.313 INFO:tasks.workunit.client.0.smithi038.stdout:4/454: chown d11/d1e/l85 245 1 2024-02-15T18:34:56.313 INFO:tasks.workunit.client.0.smithi038.stdout:4/455: chown d11/d1e/f37 1 1 2024-02-15T18:34:56.314 INFO:tasks.workunit.client.0.smithi038.stdout:9/483: creat d1/da/d36/d3f/d48/fb3 x:0 0 0 2024-02-15T18:34:56.316 INFO:tasks.workunit.client.0.smithi038.stdout:7/466: getdents d0/db 0 2024-02-15T18:34:56.317 INFO:tasks.workunit.client.0.smithi038.stdout:5/402: dwrite d2/d30/f73 [0,4194304] 0 2024-02-15T18:34:56.317 INFO:tasks.workunit.client.0.smithi038.stdout:0/496: dwrite d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/f92 [0,4194304] 0 2024-02-15T18:34:56.318 INFO:tasks.workunit.client.0.smithi038.stdout:6/445: symlink d2/d4/d2c/l7f 0 2024-02-15T18:34:56.318 INFO:tasks.workunit.client.0.smithi038.stdout:6/446: chown d2/d4/f1e 3 1 2024-02-15T18:34:56.319 INFO:tasks.workunit.client.0.smithi038.stdout:0/497: write d4/f5 [1555347,83589] 0 2024-02-15T18:34:56.319 INFO:tasks.workunit.client.0.smithi038.stdout:4/456: write d11/d1e/d20/d2f/d71/f98 [868737,111127] 0 2024-02-15T18:34:56.321 INFO:tasks.workunit.client.0.smithi038.stdout:3/388: mkdir d2/d1b/d3c/d3a/d55/d7b 0 2024-02-15T18:34:56.322 INFO:tasks.workunit.client.0.smithi038.stdout:9/484: mknod d1/da/d36/cb4 0 2024-02-15T18:34:56.327 INFO:tasks.workunit.client.0.smithi038.stdout:5/403: dread d2/f1a [0,4194304] 0 2024-02-15T18:34:56.327 INFO:tasks.workunit.client.0.smithi038.stdout:0/498: dread d4/ff [0,4194304] 0 2024-02-15T18:34:56.327 INFO:tasks.workunit.client.0.smithi038.stdout:0/499: chown d4/d13/d2a/d35/d1e/f50 15 1 2024-02-15T18:34:56.327 INFO:tasks.workunit.client.0.smithi038.stdout:9/485: dread d1/dc/d2b/f60 [0,4194304] 0 2024-02-15T18:34:56.329 INFO:tasks.workunit.client.0.smithi038.stdout:9/486: dread d1/da/d10/d13/f7a [0,4194304] 0 2024-02-15T18:34:56.329 INFO:tasks.workunit.client.0.smithi038.stdout:7/467: creat d0/d1b/d30/d5d/d61/d90/f9c x:0 0 0 2024-02-15T18:34:56.329 INFO:tasks.workunit.client.0.smithi038.stdout:7/468: read - d0/d1b/d30/f43 zero size 2024-02-15T18:34:56.329 INFO:tasks.workunit.client.0.smithi038.stdout:7/469: chown d0/d1b/d30/f43 118 1 2024-02-15T18:34:56.334 INFO:tasks.workunit.client.0.smithi038.stdout:6/447: mknod d2/d4/d19/d78/d7a/c80 0 2024-02-15T18:34:56.334 INFO:tasks.workunit.client.0.smithi038.stdout:2/388: dwrite dd/d12/d1d/d3b/f4b [0,4194304] 0 2024-02-15T18:34:56.338 INFO:tasks.workunit.client.0.smithi038.stdout:4/457: rename d11/d1e/d20/d31/d48/c5c to d11/d1e/d20/d31/d48/d60/cae 0 2024-02-15T18:34:56.338 INFO:tasks.workunit.client.0.smithi038.stdout:3/389: mkdir d2/d25/d71/d7c 0 2024-02-15T18:34:56.339 INFO:tasks.workunit.client.0.smithi038.stdout:5/404: mknod d2/d30/d57/c7e 0 2024-02-15T18:34:56.341 INFO:tasks.workunit.client.0.smithi038.stdout:6/448: dread f1 [0,4194304] 0 2024-02-15T18:34:56.341 INFO:tasks.workunit.client.0.smithi038.stdout:9/487: creat d1/da/d20/d58/d76/fb5 x:0 0 0 2024-02-15T18:34:56.341 INFO:tasks.workunit.client.0.smithi038.stdout:9/488: read - d1/da/d36/d93/fa4 zero size 2024-02-15T18:34:56.343 INFO:tasks.workunit.client.0.smithi038.stdout:7/470: symlink d0/db/d59/l9d 0 2024-02-15T18:34:56.348 INFO:tasks.workunit.client.0.smithi038.stdout:2/389: unlink dd/d12/f3d 0 2024-02-15T18:34:56.348 INFO:tasks.workunit.client.0.smithi038.stdout:4/458: mkdir d11/d1e/d3e/d75/daf 0 2024-02-15T18:34:56.348 INFO:tasks.workunit.client.0.smithi038.stdout:4/459: chown d11/d1e/d43/d5e/d62/f92 6 1 2024-02-15T18:34:56.348 INFO:tasks.workunit.client.0.smithi038.stdout:3/390: symlink d2/d1b/d4f/l7d 0 2024-02-15T18:34:56.348 INFO:tasks.workunit.client.0.smithi038.stdout:3/391: chown d2/f15 73 1 2024-02-15T18:34:56.348 INFO:tasks.workunit.client.0.smithi038.stdout:3/392: fsync d2/d24/f49 0 2024-02-15T18:34:56.348 INFO:tasks.workunit.client.0.smithi038.stdout:4/460: read d11/d1e/d3e/f47 [393378,65415] 0 2024-02-15T18:34:56.348 INFO:tasks.workunit.client.0.smithi038.stdout:5/405: truncate d2/d30/f5d 1047100 0 2024-02-15T18:34:56.349 INFO:tasks.workunit.client.0.smithi038.stdout:3/393: write d2/f15 [695082,67218] 0 2024-02-15T18:34:56.349 INFO:tasks.workunit.client.0.smithi038.stdout:3/394: chown d2/d1b 374 1 2024-02-15T18:34:56.349 INFO:tasks.workunit.client.0.smithi038.stdout:6/449: rmdir d2/d4/d2c 39 2024-02-15T18:34:56.349 INFO:tasks.workunit.client.0.smithi038.stdout:0/500: truncate d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/f92 2003986 0 2024-02-15T18:34:56.350 INFO:tasks.workunit.client.0.smithi038.stdout:9/489: rmdir d1/da/d36/d3f 39 2024-02-15T18:34:56.351 INFO:tasks.workunit.client.0.smithi038.stdout:7/471: symlink d0/db/d59/l9e 0 2024-02-15T18:34:56.352 INFO:tasks.workunit.client.0.smithi038.stdout:2/390: symlink dd/d12/d59/d2b/l83 0 2024-02-15T18:34:56.352 INFO:tasks.workunit.client.0.smithi038.stdout:2/391: fsync dd/d12/d1d/d3b/d3c/f48 0 2024-02-15T18:34:56.352 INFO:tasks.workunit.client.0.smithi038.stdout:2/392: dread - dd/d12/d3a/d67/d4f/f71 zero size 2024-02-15T18:34:56.353 INFO:tasks.workunit.client.0.smithi038.stdout:5/406: rename d2/d3/d5f/d71/d33/l49 to d2/d3/d5f/d71/l7f 0 2024-02-15T18:34:56.358 INFO:tasks.workunit.client.0.smithi038.stdout:6/450: creat d2/d4/d19/d55/f81 x:0 0 0 2024-02-15T18:34:56.359 INFO:tasks.workunit.client.0.smithi038.stdout:0/501: mkdir d4/d13/d2a/d35/d1e/d32/d79/d7d/da3 0 2024-02-15T18:34:56.360 INFO:tasks.workunit.client.0.smithi038.stdout:9/490: symlink d1/da/d20/lb6 0 2024-02-15T18:34:56.362 INFO:tasks.workunit.client.0.smithi038.stdout:7/472: creat d0/d1b/d30/d5d/d70/f9f x:0 0 0 2024-02-15T18:34:56.362 INFO:tasks.workunit.client.0.smithi038.stdout:7/473: write d0/db/d59/d15/d29/f91 [1544642,128880] 0 2024-02-15T18:34:56.362 INFO:tasks.workunit.client.0.smithi038.stdout:7/474: chown d0/db/d59/d15/d29/d7e/l4a 76429 1 2024-02-15T18:34:56.363 INFO:tasks.workunit.client.0.smithi038.stdout:3/395: rename d2/d25/d71 to d2/d1e/d7e 0 2024-02-15T18:34:56.364 INFO:tasks.workunit.client.0.smithi038.stdout:5/407: creat d2/f80 x:0 0 0 2024-02-15T18:34:56.365 INFO:tasks.workunit.client.0.smithi038.stdout:6/451: unlink d2/d4/f4a 0 2024-02-15T18:34:56.366 INFO:tasks.workunit.client.0.smithi038.stdout:0/502: mknod d4/d13/d53/ca4 0 2024-02-15T18:34:56.366 INFO:tasks.workunit.client.0.smithi038.stdout:0/503: write d4/d13/d2a/d35/d1e/d32/f76 [4672255,77650] 0 2024-02-15T18:34:56.366 INFO:tasks.workunit.client.0.smithi038.stdout:0/504: truncate d4/d13/d19/f82 376811 0 2024-02-15T18:34:56.366 INFO:tasks.workunit.client.0.smithi038.stdout:0/505: getdents d4/d13/d2a/d35/d1e/d32/d48/d9a 0 2024-02-15T18:34:56.366 INFO:tasks.workunit.client.0.smithi038.stdout:9/491: creat d1/da/d36/d3f/d99/fb7 x:0 0 0 2024-02-15T18:34:56.366 INFO:tasks.workunit.client.0.smithi038.stdout:9/492: fdatasync d1/da/d10/d24/d49/f21 0 2024-02-15T18:34:56.368 INFO:tasks.workunit.client.0.smithi038.stdout:7/475: rename d0/d1b/d30/l3a to d0/d66/la0 0 2024-02-15T18:34:56.369 INFO:tasks.workunit.client.0.smithi038.stdout:3/396: mkdir d2/d7f 0 2024-02-15T18:34:56.370 INFO:tasks.workunit.client.0.smithi038.stdout:5/408: symlink d2/d30/l81 0 2024-02-15T18:34:56.370 INFO:tasks.workunit.client.0.smithi038.stdout:6/452: mknod d2/d56/c82 0 2024-02-15T18:34:56.371 INFO:tasks.workunit.client.0.smithi038.stdout:9/493: mkdir d1/da/d36/d3f/d48/db8 0 2024-02-15T18:34:56.372 INFO:tasks.workunit.client.0.smithi038.stdout:7/476: truncate d0/d66/f1c 262228 0 2024-02-15T18:34:56.373 INFO:tasks.workunit.client.0.smithi038.stdout:5/409: getdents d2/d3/d12 0 2024-02-15T18:34:56.375 INFO:tasks.workunit.client.0.smithi038.stdout:3/397: dread d2/f1a [0,4194304] 0 2024-02-15T18:34:56.377 INFO:tasks.workunit.client.0.smithi038.stdout:9/494: mkdir d1/da/d20/d58/d94/db9 0 2024-02-15T18:34:56.378 INFO:tasks.workunit.client.0.smithi038.stdout:7/477: symlink d0/d1b/d30/d5d/d61/d90/la1 0 2024-02-15T18:34:56.382 INFO:tasks.workunit.client.0.smithi038.stdout:0/506: dread d4/d13/d53/f95 [0,4194304] 0 2024-02-15T18:34:56.382 INFO:tasks.workunit.client.0.smithi038.stdout:2/393: dwrite dd/d12/d3a/f72 [0,4194304] 0 2024-02-15T18:34:56.384 INFO:tasks.workunit.client.0.smithi038.stdout:5/410: truncate d2/f1a 94841 0 2024-02-15T18:34:56.385 INFO:tasks.workunit.client.0.smithi038.stdout:3/398: truncate d2/f17 3489323 0 2024-02-15T18:34:56.386 INFO:tasks.workunit.client.0.smithi038.stdout:9/495: getdents d1/da/d10/d19 0 2024-02-15T18:34:56.386 INFO:tasks.workunit.client.0.smithi038.stdout:9/496: fdatasync d1/dc/d4f/f46 0 2024-02-15T18:34:56.386 INFO:tasks.workunit.client.0.smithi038.stdout:7/478: mknod d0/db/d59/d15/d29/d7e/ca2 0 2024-02-15T18:34:56.389 INFO:tasks.workunit.client.0.smithi038.stdout:2/394: symlink dd/d12/d59/l84 0 2024-02-15T18:34:56.389 INFO:tasks.workunit.client.0.smithi038.stdout:3/399: creat d2/d1b/d3c/d3a/d55/f80 x:0 0 0 2024-02-15T18:34:56.389 INFO:tasks.workunit.client.0.smithi038.stdout:3/400: readlink d2/d1b/d4f/l5e 0 2024-02-15T18:34:56.390 INFO:tasks.workunit.client.0.smithi038.stdout:2/395: write dd/d12/d1d/f2e [2462694,60191] 0 2024-02-15T18:34:56.391 INFO:tasks.workunit.client.0.smithi038.stdout:9/497: unlink d1/dc/d38/l83 0 2024-02-15T18:34:56.393 INFO:tasks.workunit.client.0.smithi038.stdout:7/479: link d0/d1b/d30/d5d/d61/d90/f9c d0/d1b/d30/d5d/fa3 0 2024-02-15T18:34:56.398 INFO:tasks.workunit.client.0.smithi038.stdout:3/401: creat d2/d1b/d3c/d3a/d55/f81 x:0 0 0 2024-02-15T18:34:56.400 INFO:tasks.workunit.client.0.smithi038.stdout:2/396: unlink dd/c5c 0 2024-02-15T18:34:56.400 INFO:tasks.workunit.client.0.smithi038.stdout:2/397: truncate dd/d12/d3a/d67/d4f/f64 973838 0 2024-02-15T18:34:56.401 INFO:tasks.workunit.client.0.smithi038.stdout:7/480: creat d0/db/d59/fa4 x:0 0 0 2024-02-15T18:34:56.402 INFO:tasks.workunit.client.0.smithi038.stdout:0/507: write d4/d13/d2a/d35/d1e/d32/d3b/f60 [2281335,129800] 0 2024-02-15T18:34:56.402 INFO:tasks.workunit.client.0.smithi038.stdout:6/453: dwrite d2/d4/ff [4194304,4194304] 0 2024-02-15T18:34:56.402 INFO:tasks.workunit.client.0.smithi038.stdout:5/411: dread d2/f1c [0,4194304] 0 2024-02-15T18:34:56.402 INFO:tasks.workunit.client.0.smithi038.stdout:6/454: chown d2/d4/d19/d55/f81 50608 1 2024-02-15T18:34:56.403 INFO:tasks.workunit.client.0.smithi038.stdout:3/402: symlink d2/d1b/d3c/d41/d7a/l82 0 2024-02-15T18:34:56.403 INFO:tasks.workunit.client.0.smithi038.stdout:7/481: symlink d0/db/d59/d15/d29/la5 0 2024-02-15T18:34:56.404 INFO:tasks.workunit.client.0.smithi038.stdout:3/403: read d2/d25/f64 [139456,17048] 0 2024-02-15T18:34:56.406 INFO:tasks.workunit.client.0.smithi038.stdout:6/455: read d2/d4/d28/f3a [3726546,59946] 0 2024-02-15T18:34:56.407 INFO:tasks.workunit.client.0.smithi038.stdout:0/508: rmdir d4/d13/d2a/d35/d1e/d32/d79 39 2024-02-15T18:34:56.407 INFO:tasks.workunit.client.0.smithi038.stdout:7/482: mknod d0/db/ca6 0 2024-02-15T18:34:56.409 INFO:tasks.workunit.client.0.smithi038.stdout:9/498: dwrite d1/dc/d2b/f3a [0,4194304] 0 2024-02-15T18:34:56.409 INFO:tasks.workunit.client.0.smithi038.stdout:3/404: truncate d2/d1b/f4a 2413861 0 2024-02-15T18:34:56.409 INFO:tasks.workunit.client.0.smithi038.stdout:3/405: fdatasync d2/d1b/d3c/d3a/d55/f81 0 2024-02-15T18:34:56.409 INFO:tasks.workunit.client.0.smithi038.stdout:0/509: mknod d4/d13/d2a/d35/d1e/d32/d48/ca5 0 2024-02-15T18:34:56.409 INFO:tasks.workunit.client.0.smithi038.stdout:0/510: stat d4/d13/d19/f82 0 2024-02-15T18:34:56.410 INFO:tasks.workunit.client.0.smithi038.stdout:7/483: symlink d0/db/d19/d7d/la7 0 2024-02-15T18:34:56.412 INFO:tasks.workunit.client.0.smithi038.stdout:9/499: link d1/da/d10/d13/d3d/d43/d4d/d8a/f8e d1/dc/fba 0 2024-02-15T18:34:56.413 INFO:tasks.workunit.client.0.smithi038.stdout:7/484: read d0/db/fe [3282086,83750] 0 2024-02-15T18:34:56.413 INFO:tasks.workunit.client.0.smithi038.stdout:7/485: readlink d0/db/d19/d2d/l22 0 2024-02-15T18:34:56.413 INFO:tasks.workunit.client.0.smithi038.stdout:9/500: read d1/da/d36/f56 [3406199,82695] 0 2024-02-15T18:34:56.413 INFO:tasks.workunit.client.0.smithi038.stdout:0/511: mknod d4/ca6 0 2024-02-15T18:34:56.414 INFO:tasks.workunit.client.0.smithi038.stdout:0/512: read d4/d13/d2a/d35/d1e/d32/d48/f8c [2195772,1332] 0 2024-02-15T18:34:56.414 INFO:tasks.workunit.client.0.smithi038.stdout:0/513: stat d4/d13/c18 0 2024-02-15T18:34:56.414 INFO:tasks.workunit.client.0.smithi038.stdout:7/486: dread d0/d1b/d30/d5d/d61/f75 [0,4194304] 0 2024-02-15T18:34:56.416 INFO:tasks.workunit.client.0.smithi038.stdout:9/501: link d1/da/d10/d13/f18 d1/da/d36/d3f/d48/db8/fbb 0 2024-02-15T18:34:56.417 INFO:tasks.workunit.client.0.smithi038.stdout:0/514: dread d4/d13/f98 [0,4194304] 0 2024-02-15T18:34:56.417 INFO:tasks.workunit.client.0.smithi038.stdout:0/515: write d4/d13/d2a/d80/fa0 [1027740,87866] 0 2024-02-15T18:34:56.417 INFO:tasks.workunit.client.0.smithi038.stdout:7/487: symlink d0/d1b/d30/d5d/la8 0 2024-02-15T18:34:56.418 INFO:tasks.workunit.client.0.smithi038.stdout:3/406: write d2/f51 [1708768,886] 0 2024-02-15T18:34:56.418 INFO:tasks.workunit.client.0.smithi038.stdout:9/502: symlink d1/da/d20/d58/d94/db9/lbc 0 2024-02-15T18:34:56.418 INFO:tasks.workunit.client.0.smithi038.stdout:9/503: truncate d1/da/f17 4230106 0 2024-02-15T18:34:56.420 INFO:tasks.workunit.client.0.smithi038.stdout:0/516: creat d4/d13/d53/fa7 x:0 0 0 2024-02-15T18:34:56.421 INFO:tasks.workunit.client.0.smithi038.stdout:7/488: creat d0/d1b/d80/fa9 x:0 0 0 2024-02-15T18:34:56.421 INFO:tasks.workunit.client.0.smithi038.stdout:7/489: dread - d0/d1b/d30/d5d/d70/f88 zero size 2024-02-15T18:34:56.421 INFO:tasks.workunit.client.0.smithi038.stdout:7/490: write d0/db/d59/d15/f64 [5937913,114508] 0 2024-02-15T18:34:56.421 INFO:tasks.workunit.client.0.smithi038.stdout:7/491: fsync d0/db/d59/d15/f23 0 2024-02-15T18:34:56.422 INFO:tasks.workunit.client.0.smithi038.stdout:7/492: dread d0/d66/f1c [0,4194304] 0 2024-02-15T18:34:56.423 INFO:tasks.workunit.client.0.smithi038.stdout:9/504: mknod d1/dc/d69/cbd 0 2024-02-15T18:34:56.423 INFO:tasks.workunit.client.0.smithi038.stdout:9/505: write d1/da/d10/d13/d3d/fa2 [1043793,81311] 0 2024-02-15T18:34:56.427 INFO:tasks.workunit.client.0.smithi038.stdout:8/620: sync 2024-02-15T18:34:56.427 INFO:tasks.workunit.client.0.smithi038.stdout:1/454: sync 2024-02-15T18:34:56.427 INFO:tasks.workunit.client.0.smithi038.stdout:0/517: symlink d4/d13/d2a/d35/la8 0 2024-02-15T18:34:56.429 INFO:tasks.workunit.client.0.smithi038.stdout:7/493: mkdir d0/db/d19/d7d/daa 0 2024-02-15T18:34:56.429 INFO:tasks.workunit.client.0.smithi038.stdout:7/494: write d0/db/d59/d15/d29/d7e/f5b [668740,3617] 0 2024-02-15T18:34:56.429 INFO:tasks.workunit.client.0.smithi038.stdout:0/518: dread d4/d13/d2a/d35/d1e/f5d [0,4194304] 0 2024-02-15T18:34:56.430 INFO:tasks.workunit.client.0.smithi038.stdout:1/455: rmdir d2/d10/d19/d22/d36/d94 39 2024-02-15T18:34:56.431 INFO:tasks.workunit.client.0.smithi038.stdout:1/456: write d2/d10/d19/d22/f2c [434783,70914] 0 2024-02-15T18:34:56.431 INFO:tasks.workunit.client.0.smithi038.stdout:7/495: truncate d0/d95/f3b 3577858 0 2024-02-15T18:34:56.432 INFO:tasks.workunit.client.0.smithi038.stdout:0/519: creat d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/fa9 x:0 0 0 2024-02-15T18:34:56.432 INFO:tasks.workunit.client.0.smithi038.stdout:7/496: fdatasync d0/f21 0 2024-02-15T18:34:56.436 INFO:tasks.workunit.client.0.smithi038.stdout:0/520: mkdir d4/d13/d2a/d35/d1e/d32/d3b/daa 0 2024-02-15T18:34:56.439 INFO:tasks.workunit.client.0.smithi038.stdout:5/412: dwrite d2/f4c [0,4194304] 0 2024-02-15T18:34:56.442 INFO:tasks.workunit.client.0.smithi038.stdout:3/407: fdatasync d2/f51 0 2024-02-15T18:34:56.442 INFO:tasks.workunit.client.0.smithi038.stdout:6/456: dwrite d2/d4/d2c/f4f [0,4194304] 0 2024-02-15T18:34:56.442 INFO:tasks.workunit.client.0.smithi038.stdout:6/457: chown d2/d56/c7d 112250207 1 2024-02-15T18:34:56.442 INFO:tasks.workunit.client.0.smithi038.stdout:5/413: rename d2/f4c to d2/d3/d5f/f82 0 2024-02-15T18:34:56.442 INFO:tasks.workunit.client.0.smithi038.stdout:5/414: dread - d2/f6d zero size 2024-02-15T18:34:56.442 INFO:tasks.workunit.client.0.smithi038.stdout:5/415: write d2/f1c [2962544,41721] 0 2024-02-15T18:34:56.449 INFO:tasks.workunit.client.0.smithi038.stdout:9/506: dwrite d1/da/d10/d13/d3d/d4b/f72 [0,4194304] 0 2024-02-15T18:34:56.453 INFO:tasks.workunit.client.0.smithi038.stdout:3/408: rmdir d2/d1b/d3c 39 2024-02-15T18:34:56.455 INFO:tasks.workunit.client.0.smithi038.stdout:6/458: rename d2/d10/f1d to d2/d4/d6f/f83 0 2024-02-15T18:34:56.455 INFO:tasks.workunit.client.0.smithi038.stdout:6/459: chown d2/d4/l3b 26 1 2024-02-15T18:34:56.456 INFO:tasks.workunit.client.0.smithi038.stdout:1/457: dwrite d2/d28/f34 [0,4194304] 0 2024-02-15T18:34:56.456 INFO:tasks.workunit.client.0.smithi038.stdout:8/621: dwrite d1/d4/d5/f37 [0,4194304] 0 2024-02-15T18:34:56.457 INFO:tasks.workunit.client.0.smithi038.stdout:3/409: rmdir d2/d1b/d3c/d3a/d55 39 2024-02-15T18:34:56.459 INFO:tasks.workunit.client.0.smithi038.stdout:5/416: dwrite d2/d30/d57/f79 [0,4194304] 0 2024-02-15T18:34:56.459 INFO:tasks.workunit.client.0.smithi038.stdout:5/417: stat d2/d3/d5f/d71/l47 0 2024-02-15T18:34:56.459 INFO:tasks.workunit.client.0.smithi038.stdout:5/418: write d2/d30/d52/f7b [221400,82473] 0 2024-02-15T18:34:56.460 INFO:tasks.workunit.client.0.smithi038.stdout:8/622: write d1/d7/d15/f2f [3031413,125700] 0 2024-02-15T18:34:56.460 INFO:tasks.workunit.client.0.smithi038.stdout:6/460: link d2/d4/d2c/l7f d2/d56/l84 0 2024-02-15T18:34:56.460 INFO:tasks.workunit.client.0.smithi038.stdout:8/623: readlink d1/d4/d5/ld4 0 2024-02-15T18:34:56.460 INFO:tasks.workunit.client.0.smithi038.stdout:8/624: write d1/d4/d5/d31/d65/f98 [205404,99395] 0 2024-02-15T18:34:56.460 INFO:tasks.workunit.client.0.smithi038.stdout:8/625: write d1/d7/d26/fdc [4661404,43219] 0 2024-02-15T18:34:56.466 INFO:tasks.workunit.client.0.smithi038.stdout:9/507: dwrite d1/fb [0,4194304] 0 2024-02-15T18:34:56.466 INFO:tasks.workunit.client.0.smithi038.stdout:1/458: dread d2/d10/f31 [0,4194304] 0 2024-02-15T18:34:56.467 INFO:tasks.workunit.client.0.smithi038.stdout:1/459: write d2/d39/f7f [925053,12184] 0 2024-02-15T18:34:56.467 INFO:tasks.workunit.client.0.smithi038.stdout:1/460: chown d2/d2e/f38 32 1 2024-02-15T18:34:56.467 INFO:tasks.workunit.client.0.smithi038.stdout:1/461: write d2/d39/f8f [9013036,41448] 0 2024-02-15T18:34:56.468 INFO:tasks.workunit.client.0.smithi038.stdout:8/626: rename d1/d4/d5/d77/d79/c8d to d1/d4/d10/d43/d48/ce8 0 2024-02-15T18:34:56.468 INFO:tasks.workunit.client.0.smithi038.stdout:8/627: fsync d1/d7/d15/d6a/fa6 0 2024-02-15T18:34:56.469 INFO:tasks.workunit.client.0.smithi038.stdout:4/461: sync 2024-02-15T18:34:56.470 INFO:tasks.workunit.client.0.smithi038.stdout:9/508: write d1/da/d10/d13/d3d/d43/f70 [3490180,43083] 0 2024-02-15T18:34:56.470 INFO:tasks.workunit.client.0.smithi038.stdout:9/509: readlink d1/l8 0 2024-02-15T18:34:56.470 INFO:tasks.workunit.client.0.smithi038.stdout:6/461: dread d2/f9 [0,4194304] 0 2024-02-15T18:34:56.470 INFO:tasks.workunit.client.0.smithi038.stdout:8/628: creat d1/d4/db1/fe9 x:0 0 0 2024-02-15T18:34:56.471 INFO:tasks.workunit.client.0.smithi038.stdout:8/629: dread - d1/d4/db1/de5/db3/fd0 zero size 2024-02-15T18:34:56.475 INFO:tasks.workunit.client.0.smithi038.stdout:5/419: dread d2/d30/d52/f5b [0,4194304] 0 2024-02-15T18:34:56.475 INFO:tasks.workunit.client.0.smithi038.stdout:4/462: link d11/d1e/d20/d2b/d4d/c96 d11/d1e/d20/d2f/d7d/cb0 0 2024-02-15T18:34:56.475 INFO:tasks.workunit.client.0.smithi038.stdout:4/463: fsync d11/d1e/d20/f70 0 2024-02-15T18:34:56.476 INFO:tasks.workunit.client.0.smithi038.stdout:9/510: mknod d1/da/d10/d13/d3d/d43/d4d/d8a/cbe 0 2024-02-15T18:34:56.477 INFO:tasks.workunit.client.0.smithi038.stdout:5/420: mkdir d2/d48/d83 0 2024-02-15T18:34:56.479 INFO:tasks.workunit.client.0.smithi038.stdout:8/630: write d1/d4/d10/d3d/f97 [1729529,35740] 0 2024-02-15T18:34:56.479 INFO:tasks.workunit.client.0.smithi038.stdout:4/464: unlink d11/d1e/d20/d2b/f30 0 2024-02-15T18:34:56.481 INFO:tasks.workunit.client.0.smithi038.stdout:5/421: rename d2/d3/f1d to d2/d30/d52/f84 0 2024-02-15T18:34:56.481 INFO:tasks.workunit.client.0.smithi038.stdout:5/422: write d2/f6d [491980,30417] 0 2024-02-15T18:34:56.481 INFO:tasks.workunit.client.0.smithi038.stdout:9/511: write d1/da/d10/d24/f7c [3661687,82173] 0 2024-02-15T18:34:56.481 INFO:tasks.workunit.client.0.smithi038.stdout:4/465: mkdir d11/d1e/d3e/db1 0 2024-02-15T18:34:56.483 INFO:tasks.workunit.client.0.smithi038.stdout:8/631: rename d1/d7/d15/d6a/fa6 to d1/d4/d5/d77/d79/fea 0 2024-02-15T18:34:56.484 INFO:tasks.workunit.client.0.smithi038.stdout:5/423: unlink d2/d3/d12/l5c 0 2024-02-15T18:34:56.486 INFO:tasks.workunit.client.0.smithi038.stdout:4/466: link d11/d1e/d3e/f68 d11/d1e/d20/d31/d8d/fb2 0 2024-02-15T18:34:56.487 INFO:tasks.workunit.client.0.smithi038.stdout:9/512: read d1/da/d10/d24/f77 [1811403,127020] 0 2024-02-15T18:34:56.487 INFO:tasks.workunit.client.0.smithi038.stdout:6/462: dread d2/d10/f30 [0,4194304] 0 2024-02-15T18:34:56.487 INFO:tasks.workunit.client.0.smithi038.stdout:8/632: getdents d1/d4/d5/d6/db9 0 2024-02-15T18:34:56.487 INFO:tasks.workunit.client.0.smithi038.stdout:3/410: dwrite d2/d1e/f32 [0,4194304] 0 2024-02-15T18:34:56.490 INFO:tasks.workunit.client.0.smithi038.stdout:9/513: mknod d1/da/d20/d58/d76/cbf 0 2024-02-15T18:34:56.490 INFO:tasks.workunit.client.0.smithi038.stdout:1/462: dwrite d2/d10/d19/d5f/f97 [4194304,4194304] 0 2024-02-15T18:34:56.491 INFO:tasks.workunit.client.0.smithi038.stdout:1/463: chown d2/d10/d19/d5f/l79 107388584 1 2024-02-15T18:34:56.492 INFO:tasks.workunit.client.0.smithi038.stdout:6/463: mknod d2/d56/d5f/c85 0 2024-02-15T18:34:56.493 INFO:tasks.workunit.client.0.smithi038.stdout:3/411: mknod d2/d25/d60/c83 0 2024-02-15T18:34:56.493 INFO:tasks.workunit.client.0.smithi038.stdout:6/464: rmdir d2/d4/d2c 39 2024-02-15T18:34:56.494 INFO:tasks.workunit.client.0.smithi038.stdout:1/464: write f1 [2774654,69049] 0 2024-02-15T18:34:56.494 INFO:tasks.workunit.client.0.smithi038.stdout:9/514: dread d1/f1c [0,4194304] 0 2024-02-15T18:34:56.496 INFO:tasks.workunit.client.0.smithi038.stdout:3/412: mknod d2/d1b/d3c/d3a/d55/d65/c84 0 2024-02-15T18:34:56.496 INFO:tasks.workunit.client.0.smithi038.stdout:6/465: symlink d2/d56/d5f/l86 0 2024-02-15T18:34:56.496 INFO:tasks.workunit.client.0.smithi038.stdout:9/515: write d1/fb [1481869,40366] 0 2024-02-15T18:34:56.498 INFO:tasks.workunit.client.0.smithi038.stdout:6/466: symlink d2/d4/d6f/l87 0 2024-02-15T18:34:56.498 INFO:tasks.workunit.client.0.smithi038.stdout:6/467: stat d2/d10/f4e 0 2024-02-15T18:34:56.498 INFO:tasks.workunit.client.0.smithi038.stdout:1/465: dread d2/d10/d32/d53/f7b [0,4194304] 0 2024-02-15T18:34:56.499 INFO:tasks.workunit.client.0.smithi038.stdout:3/413: symlink d2/d1b/d3c/d3a/d55/l85 0 2024-02-15T18:34:56.499 INFO:tasks.workunit.client.0.smithi038.stdout:3/414: dread - d2/d1b/d3c/d3a/d55/f79 zero size 2024-02-15T18:34:56.501 INFO:tasks.workunit.client.0.smithi038.stdout:9/516: rename d1/dc/d2b/f60 to d1/da/d10/d13/d3d/d4b/daa/fc0 0 2024-02-15T18:34:56.502 INFO:tasks.workunit.client.0.smithi038.stdout:6/468: creat d2/d4/d6f/f88 x:0 0 0 2024-02-15T18:34:56.504 INFO:tasks.workunit.client.0.smithi038.stdout:9/517: dread d1/da/d36/d3f/d48/f57 [0,4194304] 0 2024-02-15T18:34:56.504 INFO:tasks.workunit.client.0.smithi038.stdout:9/518: readlink d1/da/d10/d24/d49/l9c 0 2024-02-15T18:34:56.504 INFO:tasks.workunit.client.0.smithi038.stdout:9/519: chown d1/dc/db2 1 1 2024-02-15T18:34:56.504 INFO:tasks.workunit.client.0.smithi038.stdout:9/520: write d1/da/d20/d58/d76/f8c [169618,81900] 0 2024-02-15T18:34:56.509 INFO:tasks.workunit.client.0.smithi038.stdout:1/466: dread d2/d10/d19/d22/d36/f4d [0,4194304] 0 2024-02-15T18:34:56.509 INFO:tasks.workunit.client.0.smithi038.stdout:1/467: truncate d2/d2e/f7d 797846 0 2024-02-15T18:34:56.509 INFO:tasks.workunit.client.0.smithi038.stdout:1/468: fdatasync d2/d10/d32/d53/f58 0 2024-02-15T18:34:56.509 INFO:tasks.workunit.client.0.smithi038.stdout:1/469: chown d2/d10/d19/d22/d36/l60 3600913 1 2024-02-15T18:34:56.512 INFO:tasks.workunit.client.0.smithi038.stdout:4/467: dwrite d11/d1e/d20/d2f/f3d [0,4194304] 0 2024-02-15T18:34:56.512 INFO:tasks.workunit.client.0.smithi038.stdout:4/468: fsync d11/d1e/f56 0 2024-02-15T18:34:56.512 INFO:tasks.workunit.client.0.smithi038.stdout:5/424: dwrite d2/d3/d5f/d71/d33/f66 [0,4194304] 0 2024-02-15T18:34:56.513 INFO:tasks.workunit.client.0.smithi038.stdout:3/415: symlink d2/d1e/l86 0 2024-02-15T18:34:56.516 INFO:tasks.workunit.client.0.smithi038.stdout:1/470: creat d2/d10/fa3 x:0 0 0 2024-02-15T18:34:56.517 INFO:tasks.workunit.client.0.smithi038.stdout:4/469: link ld d11/d1e/d20/d31/d8d/lb3 0 2024-02-15T18:34:56.518 INFO:tasks.workunit.client.0.smithi038.stdout:6/469: dread d2/d4/fc [4194304,4194304] 0 2024-02-15T18:34:56.519 INFO:tasks.workunit.client.0.smithi038.stdout:9/521: dwrite d1/da/d10/d24/f9e [0,4194304] 0 2024-02-15T18:34:56.519 INFO:tasks.workunit.client.0.smithi038.stdout:5/425: creat d2/d75/f85 x:0 0 0 2024-02-15T18:34:56.523 INFO:tasks.workunit.client.0.smithi038.stdout:3/416: dread d2/f9 [0,4194304] 0 2024-02-15T18:34:56.526 INFO:tasks.workunit.client.0.smithi038.stdout:1/471: link d2/d10/c6d d2/d10/d19/d22/d54/d80/ca4 0 2024-02-15T18:34:56.530 INFO:tasks.workunit.client.0.smithi038.stdout:4/470: unlink d11/d1e/f56 0 2024-02-15T18:34:56.530 INFO:tasks.workunit.client.0.smithi038.stdout:4/471: write d11/d1e/d3e/d57/f82 [717106,34325] 0 2024-02-15T18:34:56.531 INFO:tasks.workunit.client.0.smithi038.stdout:6/470: mknod d2/d4/d6f/c89 0 2024-02-15T18:34:56.531 INFO:tasks.workunit.client.0.smithi038.stdout:6/471: chown f1 4602 1 2024-02-15T18:34:56.535 INFO:tasks.workunit.client.0.smithi038.stdout:9/522: link d1/da/d36/d50/f55 d1/da/d36/d3f/d99/fc1 0 2024-02-15T18:34:56.536 INFO:tasks.workunit.client.0.smithi038.stdout:9/523: dread - d1/da/d10/d24/f91 zero size 2024-02-15T18:34:56.542 INFO:tasks.workunit.client.0.smithi038.stdout:5/426: rmdir d2/d48 39 2024-02-15T18:34:56.543 INFO:tasks.workunit.client.0.smithi038.stdout:3/417: rmdir d2/d1b/d3c 39 2024-02-15T18:34:56.543 INFO:tasks.workunit.client.0.smithi038.stdout:3/418: stat d2/f1a 0 2024-02-15T18:34:56.543 INFO:tasks.workunit.client.0.smithi038.stdout:3/419: chown d2/d1b 3054 1 2024-02-15T18:34:56.543 INFO:tasks.workunit.client.0.smithi038.stdout:6/472: creat d2/d4/d19/d55/f8a x:0 0 0 2024-02-15T18:34:56.544 INFO:tasks.workunit.client.0.smithi038.stdout:3/420: creat d2/d1b/d3c/d41/d4e/d62/f87 x:0 0 0 2024-02-15T18:34:56.545 INFO:tasks.workunit.client.0.smithi038.stdout:1/472: dwrite d2/f9d [0,4194304] 0 2024-02-15T18:34:56.547 INFO:tasks.workunit.client.0.smithi038.stdout:6/473: rename d2/f7e to d2/d4/d2c/f8b 0 2024-02-15T18:34:56.547 INFO:tasks.workunit.client.0.smithi038.stdout:6/474: chown d2/d4/d19/f47 1358151 1 2024-02-15T18:34:56.555 INFO:tasks.workunit.client.0.smithi038.stdout:1/473: creat d2/d10/fa5 x:0 0 0 2024-02-15T18:34:56.556 INFO:tasks.workunit.client.0.smithi038.stdout:6/475: creat d2/d4/d19/d55/f8c x:0 0 0 2024-02-15T18:34:56.556 INFO:tasks.workunit.client.0.smithi038.stdout:6/476: chown d2/d56/l84 619919178 1 2024-02-15T18:34:56.556 INFO:tasks.workunit.client.0.smithi038.stdout:6/477: stat d2/d56/d5f/c85 0 2024-02-15T18:34:56.556 INFO:tasks.workunit.client.0.smithi038.stdout:6/478: chown d2/d4/d28/f3a 79286 1 2024-02-15T18:34:56.557 INFO:tasks.workunit.client.0.smithi038.stdout:2/398: sync 2024-02-15T18:34:56.557 INFO:tasks.workunit.client.0.smithi038.stdout:2/399: truncate dd/f41 466525 0 2024-02-15T18:34:56.558 INFO:tasks.workunit.client.0.smithi038.stdout:2/400: mknod dd/d12/d3a/c85 0 2024-02-15T18:34:56.559 INFO:tasks.workunit.client.0.smithi038.stdout:6/479: write d2/d4/f1f [2441895,123102] 0 2024-02-15T18:34:56.562 INFO:tasks.workunit.client.0.smithi038.stdout:4/472: dwrite d11/d1e/d3e/d75/d40/d6f/f9b [0,4194304] 0 2024-02-15T18:34:56.567 INFO:tasks.workunit.client.0.smithi038.stdout:9/524: dwrite d1/da/d36/d3f/d48/db8/fbb [0,4194304] 0 2024-02-15T18:34:56.567 INFO:tasks.workunit.client.0.smithi038.stdout:9/525: stat d1/da/d10/d13/d3d/d43/f97 0 2024-02-15T18:34:56.569 INFO:tasks.workunit.client.0.smithi038.stdout:4/473: rmdir d11/d1e/d20 39 2024-02-15T18:34:56.569 INFO:tasks.workunit.client.0.smithi038.stdout:4/474: read - d11/d1e/d20/d31/d8d/fb2 zero size 2024-02-15T18:34:56.569 INFO:tasks.workunit.client.0.smithi038.stdout:4/475: fsync d11/f13 0 2024-02-15T18:34:56.572 INFO:tasks.workunit.client.0.smithi038.stdout:9/526: link d1/da/d10/d13/d3d/d4b/l7e d1/lc2 0 2024-02-15T18:34:56.574 INFO:tasks.workunit.client.0.smithi038.stdout:9/527: creat d1/dc/d69/d7d/fc3 x:0 0 0 2024-02-15T18:34:56.575 INFO:tasks.workunit.client.0.smithi038.stdout:9/528: mknod d1/da/d10/d13/d3d/d43/cc4 0 2024-02-15T18:34:56.575 INFO:tasks.workunit.client.0.smithi038.stdout:9/529: fdatasync d1/da/d10/d13/f61 0 2024-02-15T18:34:56.576 INFO:tasks.workunit.client.0.smithi038.stdout:9/530: creat d1/da/d10/d82/fc5 x:0 0 0 2024-02-15T18:34:56.579 INFO:tasks.workunit.client.0.smithi038.stdout:9/531: rmdir d1/da/d20/d58/d66 39 2024-02-15T18:34:56.579 INFO:tasks.workunit.client.0.smithi038.stdout:9/532: stat d1/c7 0 2024-02-15T18:34:56.579 INFO:tasks.workunit.client.0.smithi038.stdout:9/533: write d1/da/d36/d3f/d48/fb3 [327821,119579] 0 2024-02-15T18:34:56.579 INFO:tasks.workunit.client.0.smithi038.stdout:3/421: dwrite d2/d1b/d4f/f5d [0,4194304] 0 2024-02-15T18:34:56.580 INFO:tasks.workunit.client.0.smithi038.stdout:9/534: read d1/dc/f71 [1102249,55455] 0 2024-02-15T18:34:56.581 INFO:tasks.workunit.client.0.smithi038.stdout:6/480: dread d2/f1a [0,4194304] 0 2024-02-15T18:34:56.581 INFO:tasks.workunit.client.0.smithi038.stdout:5/427: dwrite d2/d3/d12/f14 [0,4194304] 0 2024-02-15T18:34:56.582 INFO:tasks.workunit.client.0.smithi038.stdout:2/401: rmdir dd/d12 39 2024-02-15T18:34:56.582 INFO:tasks.workunit.client.0.smithi038.stdout:9/535: write d1/da/f54 [4374396,29542] 0 2024-02-15T18:34:56.584 INFO:tasks.workunit.client.0.smithi038.stdout:2/402: creat dd/d12/d3a/d67/f86 x:0 0 0 2024-02-15T18:34:56.584 INFO:tasks.workunit.client.0.smithi038.stdout:2/403: fdatasync dd/d12/d3a/d67/d4f/d5a/f6e 0 2024-02-15T18:34:56.587 INFO:tasks.workunit.client.0.smithi038.stdout:9/536: rename d1/da/d10/d24 to d1/da/d20/d58/d94/db9/dc6 0 2024-02-15T18:34:56.588 INFO:tasks.workunit.client.0.smithi038.stdout:2/404: creat dd/d12/d3a/d5e/f87 x:0 0 0 2024-02-15T18:34:56.589 INFO:tasks.workunit.client.0.smithi038.stdout:9/537: unlink d1/dc/d69/cbd 0 2024-02-15T18:34:56.589 INFO:tasks.workunit.client.0.smithi038.stdout:9/538: stat d1/da/d36/c8f 0 2024-02-15T18:34:56.589 INFO:tasks.workunit.client.0.smithi038.stdout:9/539: dread - d1/da/d10/d13/f61 zero size 2024-02-15T18:34:56.591 INFO:tasks.workunit.client.0.smithi038.stdout:1/474: dwrite d2/f11 [0,4194304] 0 2024-02-15T18:34:56.591 INFO:tasks.workunit.client.0.smithi038.stdout:5/428: dread d2/f6d [0,4194304] 0 2024-02-15T18:34:56.592 INFO:tasks.workunit.client.0.smithi038.stdout:9/540: creat d1/da/d20/d58/d94/fc7 x:0 0 0 2024-02-15T18:34:56.592 INFO:tasks.workunit.client.0.smithi038.stdout:5/429: write d2/d30/d52/f7b [506461,112378] 0 2024-02-15T18:34:56.593 INFO:tasks.workunit.client.0.smithi038.stdout:1/475: creat d2/d10/d62/fa6 x:0 0 0 2024-02-15T18:34:56.593 INFO:tasks.workunit.client.0.smithi038.stdout:1/476: fdatasync d2/d28/f50 0 2024-02-15T18:34:56.593 INFO:tasks.workunit.client.0.smithi038.stdout:5/430: unlink d2/d3/d12/f28 0 2024-02-15T18:34:56.594 INFO:tasks.workunit.client.0.smithi038.stdout:5/431: write d2/d3/d12/fa [1635210,40700] 0 2024-02-15T18:34:56.594 INFO:tasks.workunit.client.0.smithi038.stdout:1/477: write d2/d10/d19/d22/f7a [3411553,5990] 0 2024-02-15T18:34:56.596 INFO:tasks.workunit.client.0.smithi038.stdout:4/476: dwrite d11/d1e/d43/d4a/f59 [0,4194304] 0 2024-02-15T18:34:56.596 INFO:tasks.workunit.client.0.smithi038.stdout:4/477: write d11/d1e/d20/d31/f91 [226111,41666] 0 2024-02-15T18:34:56.596 INFO:tasks.workunit.client.0.smithi038.stdout:3/422: write d2/d1b/d4f/f5d [5214652,113525] 0 2024-02-15T18:34:56.596 INFO:tasks.workunit.client.0.smithi038.stdout:3/423: fsync d2/d1b/d3c/d3a/f68 0 2024-02-15T18:34:56.596 INFO:tasks.workunit.client.0.smithi038.stdout:4/478: read d11/d1e/d20/d97/f9f [2104926,27140] 0 2024-02-15T18:34:56.596 INFO:tasks.workunit.client.0.smithi038.stdout:4/479: rename d11/d1e to d11/d1e/d20/d31/d48/d60/db4 22 2024-02-15T18:34:56.597 INFO:tasks.workunit.client.0.smithi038.stdout:6/481: dwrite d2/d4/d19/d55/d5c/f70 [0,4194304] 0 2024-02-15T18:34:56.600 INFO:tasks.workunit.client.0.smithi038.stdout:5/432: mkdir d2/d3/d12/d5e/d86 0 2024-02-15T18:34:56.600 INFO:tasks.workunit.client.0.smithi038.stdout:4/480: mkdir d11/d1e/d20/d31/d8d/d99/db5 0 2024-02-15T18:34:56.600 INFO:tasks.workunit.client.0.smithi038.stdout:2/405: dread dd/d12/d1d/d3b/f4b [0,4194304] 0 2024-02-15T18:34:56.602 INFO:tasks.workunit.client.0.smithi038.stdout:2/406: getdents dd/d12/d59 0 2024-02-15T18:34:56.602 INFO:tasks.workunit.client.0.smithi038.stdout:2/407: fdatasync dd/d12/d3a/d67/f86 0 2024-02-15T18:34:56.602 INFO:tasks.workunit.client.0.smithi038.stdout:2/408: chown dd/d12/d59 71107 1 2024-02-15T18:34:56.610 INFO:tasks.workunit.client.0.smithi038.stdout:2/409: mknod dd/d12/d3a/d5e/c88 0 2024-02-15T18:34:56.625 INFO:tasks.workunit.client.0.smithi038.stdout:6/482: dread d2/d56/f65 [0,4194304] 0 2024-02-15T18:34:56.627 INFO:tasks.workunit.client.0.smithi038.stdout:6/483: rename d2/d4/d2c/f8b to d2/d56/f8d 0 2024-02-15T18:34:56.627 INFO:tasks.workunit.client.0.smithi038.stdout:6/484: write d2/d4/ff [9262401,26858] 0 2024-02-15T18:34:56.627 INFO:tasks.workunit.client.0.smithi038.stdout:6/485: chown d2/d4/d19/c41 515034 1 2024-02-15T18:34:56.628 INFO:tasks.workunit.client.0.smithi038.stdout:6/486: getdents d2 0 2024-02-15T18:34:56.629 INFO:tasks.workunit.client.0.smithi038.stdout:1/478: dwrite d2/d10/d32/d53/f7b [0,4194304] 0 2024-02-15T18:34:56.630 INFO:tasks.workunit.client.0.smithi038.stdout:6/487: rename d2/d4/d19/d55/d5c/f64 to d2/f8e 0 2024-02-15T18:34:56.631 INFO:tasks.workunit.client.0.smithi038.stdout:1/479: unlink d2/d28/d3c/f63 0 2024-02-15T18:34:56.631 INFO:tasks.workunit.client.0.smithi038.stdout:1/480: rmdir d2/d10/d32/d53/d82 39 2024-02-15T18:34:56.632 INFO:tasks.workunit.client.0.smithi038.stdout:1/481: symlink d2/d10/d19/la7 0 2024-02-15T18:34:56.638 INFO:tasks.workunit.client.0.smithi038.stdout:1/482: write d2/d10/d19/d22/f57 [69719,27749] 0 2024-02-15T18:34:56.638 INFO:tasks.workunit.client.0.smithi038.stdout:3/424: dwrite d2/f1d [4194304,4194304] 0 2024-02-15T18:34:56.639 INFO:tasks.workunit.client.0.smithi038.stdout:5/433: dwrite d2/f6d [0,4194304] 0 2024-02-15T18:34:56.639 INFO:tasks.workunit.client.0.smithi038.stdout:1/483: creat d2/d10/d19/d5f/fa8 x:0 0 0 2024-02-15T18:34:56.640 INFO:tasks.workunit.client.0.smithi038.stdout:5/434: write d2/fc [2411263,18016] 0 2024-02-15T18:34:56.640 INFO:tasks.workunit.client.0.smithi038.stdout:3/425: write d2/f15 [2095191,17867] 0 2024-02-15T18:34:56.641 INFO:tasks.workunit.client.0.smithi038.stdout:4/481: dwrite d11/d1e/d43/d5e/d62/f92 [0,4194304] 0 2024-02-15T18:34:56.641 INFO:tasks.workunit.client.0.smithi038.stdout:4/482: write d11/d1e/d3e/d57/f82 [1697321,71696] 0 2024-02-15T18:34:56.642 INFO:tasks.workunit.client.0.smithi038.stdout:2/410: dwrite dd/d12/d3a/d67/d4f/d5a/f6e [0,4194304] 0 2024-02-15T18:34:56.642 INFO:tasks.workunit.client.0.smithi038.stdout:6/488: dread d2/d4/d19/d55/f6a [0,4194304] 0 2024-02-15T18:34:56.642 INFO:tasks.workunit.client.0.smithi038.stdout:6/489: chown d2 13 1 2024-02-15T18:34:56.643 INFO:tasks.workunit.client.0.smithi038.stdout:2/411: fdatasync dd/d12/d3a/d67/d4f/d5a/f60 0 2024-02-15T18:34:56.643 INFO:tasks.workunit.client.0.smithi038.stdout:1/484: mkdir d2/d69/da9 0 2024-02-15T18:34:56.645 INFO:tasks.workunit.client.0.smithi038.stdout:3/426: dread d2/d1b/f4a [0,4194304] 0 2024-02-15T18:34:56.645 INFO:tasks.workunit.client.0.smithi038.stdout:3/427: chown d2/l3e 16128 1 2024-02-15T18:34:56.646 INFO:tasks.workunit.client.0.smithi038.stdout:1/485: write d2/d69/f6c [1348243,52183] 0 2024-02-15T18:34:56.652 INFO:tasks.workunit.client.0.smithi038.stdout:1/486: dread d2/d10/d19/f1d [0,4194304] 0 2024-02-15T18:34:56.653 INFO:tasks.workunit.client.0.smithi038.stdout:5/435: symlink d2/d3/d12/d5e/d86/l87 0 2024-02-15T18:34:56.653 INFO:tasks.workunit.client.0.smithi038.stdout:5/436: readlink d2/d30/l32 0 2024-02-15T18:34:56.653 INFO:tasks.workunit.client.0.smithi038.stdout:5/437: truncate d2/d48/f74 972213 0 2024-02-15T18:34:56.653 INFO:tasks.workunit.client.0.smithi038.stdout:5/438: chown d2/d3/c25 3246 1 2024-02-15T18:34:56.653 INFO:tasks.workunit.client.0.smithi038.stdout:5/439: truncate d2/d75/f85 813297 0 2024-02-15T18:34:56.655 INFO:tasks.workunit.client.0.smithi038.stdout:4/483: rename d11/d1e/d20/d2b/d65/f9d to d11/d1e/d20/d31/d8d/d99/fb6 0 2024-02-15T18:34:56.657 INFO:tasks.workunit.client.0.smithi038.stdout:2/412: mkdir dd/d12/d5d/d54/d89 0 2024-02-15T18:34:56.658 INFO:tasks.workunit.client.0.smithi038.stdout:3/428: symlink d2/d1b/d3c/d41/d4e/l88 0 2024-02-15T18:34:56.658 INFO:tasks.workunit.client.0.smithi038.stdout:3/429: write d2/d1b/d3c/d3a/d55/f80 [848872,47825] 0 2024-02-15T18:34:56.661 INFO:tasks.workunit.client.0.smithi038.stdout:1/487: mknod d2/d28/d3c/caa 0 2024-02-15T18:34:56.662 INFO:tasks.workunit.client.0.smithi038.stdout:1/488: write d2/f73 [588854,95111] 0 2024-02-15T18:34:56.662 INFO:tasks.workunit.client.0.smithi038.stdout:4/484: symlink d11/d1e/d20/d31/d48/d60/lb7 0 2024-02-15T18:34:56.664 INFO:tasks.workunit.client.0.smithi038.stdout:2/413: creat dd/d12/d5d/d54/f8a x:0 0 0 2024-02-15T18:34:56.664 INFO:tasks.workunit.client.0.smithi038.stdout:3/430: truncate d2/d1b/f36 2132740 0 2024-02-15T18:34:56.665 INFO:tasks.workunit.client.0.smithi038.stdout:5/440: dread d2/f6d [0,4194304] 0 2024-02-15T18:34:56.665 INFO:tasks.workunit.client.0.smithi038.stdout:5/441: fsync d2/d48/f59 0 2024-02-15T18:34:56.666 INFO:tasks.workunit.client.0.smithi038.stdout:5/442: write d2/f7a [228290,44061] 0 2024-02-15T18:34:56.666 INFO:tasks.workunit.client.0.smithi038.stdout:4/485: rename d11/d1e/d3e/d75/f2d to d11/d1e/d20/d2b/d65/fb8 0 2024-02-15T18:34:56.667 INFO:tasks.workunit.client.0.smithi038.stdout:3/431: truncate d2/f2f 3897034 0 2024-02-15T18:34:56.668 INFO:tasks.workunit.client.0.smithi038.stdout:6/490: dwrite d2/d56/f7c [0,4194304] 0 2024-02-15T18:34:56.669 INFO:tasks.workunit.client.0.smithi038.stdout:5/443: unlink d2/d30/d57/l76 0 2024-02-15T18:34:56.669 INFO:tasks.workunit.client.0.smithi038.stdout:5/444: dread - d2/d3/d5f/d71/d33/f68 zero size 2024-02-15T18:34:56.670 INFO:tasks.workunit.client.0.smithi038.stdout:4/486: stat d11/d1e/d20/d2b/d4d/d67/d73/c86 0 2024-02-15T18:34:56.670 INFO:tasks.workunit.client.0.smithi038.stdout:7/497: sync 2024-02-15T18:34:56.670 INFO:tasks.workunit.client.0.smithi038.stdout:0/521: sync 2024-02-15T18:34:56.672 INFO:tasks.workunit.client.0.smithi038.stdout:8/633: sync 2024-02-15T18:34:56.676 INFO:tasks.workunit.client.0.smithi038.stdout:5/445: dread d2/d3/f4b [0,4194304] 0 2024-02-15T18:34:56.676 INFO:tasks.workunit.client.0.smithi038.stdout:5/446: stat d2/d3/f6f 0 2024-02-15T18:34:56.680 INFO:tasks.workunit.client.0.smithi038.stdout:5/447: dread d2/d3/f4b [0,4194304] 0 2024-02-15T18:34:56.683 INFO:tasks.workunit.client.0.smithi038.stdout:4/487: dread d11/d1e/d3e/d57/f77 [0,4194304] 0 2024-02-15T18:34:56.683 INFO:tasks.workunit.client.0.smithi038.stdout:4/488: chown d11/d1e/d20/d2f 55 1 2024-02-15T18:34:56.683 INFO:tasks.workunit.client.0.smithi038.stdout:4/489: dread - d11/d1e/d20/d2b/d4d/f95 zero size 2024-02-15T18:34:56.685 INFO:tasks.workunit.client.0.smithi038.stdout:2/414: dwrite dd/d12/d3a/d67/f86 [0,4194304] 0 2024-02-15T18:34:56.685 INFO:tasks.workunit.client.0.smithi038.stdout:7/498: unlink d0/d66/f1c 0 2024-02-15T18:34:56.686 INFO:tasks.workunit.client.0.smithi038.stdout:2/415: read dd/f41 [35560,114626] 0 2024-02-15T18:34:56.688 INFO:tasks.workunit.client.0.smithi038.stdout:0/522: link d4/d13/d2a/d35/d1e/d32/d3b/l59 d4/d13/d2a/d35/d1e/d32/d3b/daa/lab 0 2024-02-15T18:34:56.689 INFO:tasks.workunit.client.0.smithi038.stdout:8/634: creat d1/d7/d15/d4b/feb x:0 0 0 2024-02-15T18:34:56.690 INFO:tasks.workunit.client.0.smithi038.stdout:3/432: dwrite d2/d1b/d3c/d41/d4e/d62/f87 [0,4194304] 0 2024-02-15T18:34:56.692 INFO:tasks.workunit.client.0.smithi038.stdout:5/448: unlink d2/f1c 0 2024-02-15T18:34:56.694 INFO:tasks.workunit.client.0.smithi038.stdout:0/523: write d4/d13/d53/f95 [309065,105744] 0 2024-02-15T18:34:56.696 INFO:tasks.workunit.client.0.smithi038.stdout:7/499: creat d0/d1b/d30/d5d/d61/d90/fab x:0 0 0 2024-02-15T18:34:56.697 INFO:tasks.workunit.client.0.smithi038.stdout:8/635: link d1/d4/d5/d77/d79/fbe d1/d7/d15/d4b/fec 0 2024-02-15T18:34:56.700 INFO:tasks.workunit.client.0.smithi038.stdout:3/433: creat d2/d1e/f89 x:0 0 0 2024-02-15T18:34:56.700 INFO:tasks.workunit.client.0.smithi038.stdout:3/434: truncate d2/d25/d60/f76 117873 0 2024-02-15T18:34:56.700 INFO:tasks.workunit.client.0.smithi038.stdout:2/416: dwrite dd/d12/d3a/d67/d4f/d5a/f60 [0,4194304] 0 2024-02-15T18:34:56.701 INFO:tasks.workunit.client.0.smithi038.stdout:5/449: mknod d2/d30/d52/c88 0 2024-02-15T18:34:56.701 INFO:tasks.workunit.client.0.smithi038.stdout:5/450: fsync d2/d48/f59 0 2024-02-15T18:34:56.702 INFO:tasks.workunit.client.0.smithi038.stdout:9/541: sync 2024-02-15T18:34:56.702 INFO:tasks.workunit.client.0.smithi038.stdout:5/451: write d2/d30/d57/f79 [1754041,103886] 0 2024-02-15T18:34:56.705 INFO:tasks.workunit.client.0.smithi038.stdout:4/490: rename l8 to d11/d1e/d20/d2b/d35/lb9 0 2024-02-15T18:34:56.707 INFO:tasks.workunit.client.0.smithi038.stdout:0/524: creat d4/d13/d2a/d35/d1e/fac x:0 0 0 2024-02-15T18:34:56.707 INFO:tasks.workunit.client.0.smithi038.stdout:0/525: chown d4/f5 310 1 2024-02-15T18:34:56.707 INFO:tasks.workunit.client.0.smithi038.stdout:0/526: write d4/d13/d2a/d35/d1e/d32/d79/d7d/f86 [86025,81211] 0 2024-02-15T18:34:56.709 INFO:tasks.workunit.client.0.smithi038.stdout:8/636: getdents d1/d7/d26 0 2024-02-15T18:34:56.710 INFO:tasks.workunit.client.0.smithi038.stdout:3/435: truncate d2/d1b/d3c/d3a/f68 3723284 0 2024-02-15T18:34:56.713 INFO:tasks.workunit.client.0.smithi038.stdout:4/491: dread f4 [0,4194304] 0 2024-02-15T18:34:56.714 INFO:tasks.workunit.client.0.smithi038.stdout:4/492: fsync d11/d1e/d3e/d75/d40/d6f/faa 0 2024-02-15T18:34:56.717 INFO:tasks.workunit.client.0.smithi038.stdout:2/417: mknod dd/d12/d1d/d3b/d3c/d73/c8b 0 2024-02-15T18:34:56.719 INFO:tasks.workunit.client.0.smithi038.stdout:5/452: unlink d2/d3/d5f/d71/f43 0 2024-02-15T18:34:56.719 INFO:tasks.workunit.client.0.smithi038.stdout:5/453: readlink d2/d3/d5f/d71/l3e 0 2024-02-15T18:34:56.719 INFO:tasks.workunit.client.0.smithi038.stdout:0/527: link d4/c68 d4/d13/d2a/d35/d1e/d32/d3b/cad 0 2024-02-15T18:34:56.721 INFO:tasks.workunit.client.0.smithi038.stdout:7/500: dread d0/db/d59/d15/f44 [0,4194304] 0 2024-02-15T18:34:56.721 INFO:tasks.workunit.client.0.smithi038.stdout:4/493: mkdir d11/d1e/d20/d31/d7a/d8f/dba 0 2024-02-15T18:34:56.722 INFO:tasks.workunit.client.0.smithi038.stdout:2/418: creat dd/d12/d1d/d3b/d3c/d4a/f8c x:0 0 0 2024-02-15T18:34:56.725 INFO:tasks.workunit.client.0.smithi038.stdout:0/528: write d4/ff [706434,62615] 0 2024-02-15T18:34:56.725 INFO:tasks.workunit.client.0.smithi038.stdout:0/529: write d4/d13/d2a/d35/d1e/d32/d48/f56 [9021784,10036] 0 2024-02-15T18:34:56.725 INFO:tasks.workunit.client.0.smithi038.stdout:7/501: stat d0/db/c2a 0 2024-02-15T18:34:56.725 INFO:tasks.workunit.client.0.smithi038.stdout:4/494: symlink d11/d1e/d20/d2b/d4d/d67/d73/lbb 0 2024-02-15T18:34:56.726 INFO:tasks.workunit.client.0.smithi038.stdout:7/502: rename d0/db/c2a to d0/d1b/d30/d5d/d61/cac 0 2024-02-15T18:34:56.728 INFO:tasks.workunit.client.0.smithi038.stdout:4/495: creat d11/d1e/d20/d31/d7a/fbc x:0 0 0 2024-02-15T18:34:56.729 INFO:tasks.workunit.client.0.smithi038.stdout:2/419: mkdir dd/d12/d8d 0 2024-02-15T18:34:56.729 INFO:tasks.workunit.client.0.smithi038.stdout:2/420: write dd/d12/d3a/d67/f86 [4774443,122769] 0 2024-02-15T18:34:56.729 INFO:tasks.workunit.client.0.smithi038.stdout:2/421: stat dd/d12/d3a/d67 0 2024-02-15T18:34:56.730 INFO:tasks.workunit.client.0.smithi038.stdout:7/503: write d0/d1b/d30/d5d/d61/f75 [780862,93248] 0 2024-02-15T18:34:56.730 INFO:tasks.workunit.client.0.smithi038.stdout:7/504: write d0/d1b/d30/d5d/d70/f9f [171409,128738] 0 2024-02-15T18:34:56.731 INFO:tasks.workunit.client.0.smithi038.stdout:2/422: truncate dd/d12/f36 7132267 0 2024-02-15T18:34:56.732 INFO:tasks.workunit.client.0.smithi038.stdout:4/496: rename d11/d1e/d20/d31/d8d/d99/da6 to d11/d1e/d43/d4a/dbd 0 2024-02-15T18:34:56.733 INFO:tasks.workunit.client.0.smithi038.stdout:7/505: write d0/d95/f37 [186566,61352] 0 2024-02-15T18:34:56.733 INFO:tasks.workunit.client.0.smithi038.stdout:2/423: symlink dd/d12/d59/d2b/d7a/l8e 0 2024-02-15T18:34:56.734 INFO:tasks.workunit.client.0.smithi038.stdout:7/506: write d0/d68/f93 [2184291,27556] 0 2024-02-15T18:34:56.735 INFO:tasks.workunit.client.0.smithi038.stdout:2/424: rename dd/f79 to dd/d12/d3a/d67/f8f 0 2024-02-15T18:34:56.738 INFO:tasks.workunit.client.0.smithi038.stdout:8/637: dwrite d1/d4/d5/f23 [0,4194304] 0 2024-02-15T18:34:56.738 INFO:tasks.workunit.client.0.smithi038.stdout:4/497: read d11/d1e/d43/d5e/d62/f92 [6992077,94192] 0 2024-02-15T18:34:56.738 INFO:tasks.workunit.client.0.smithi038.stdout:3/436: dwrite d2/d1b/f5c [0,4194304] 0 2024-02-15T18:34:56.738 INFO:tasks.workunit.client.0.smithi038.stdout:3/437: fsync d2/f15 0 2024-02-15T18:34:56.739 INFO:tasks.workunit.client.0.smithi038.stdout:7/507: rename d0/d1b/c5c to d0/db/d59/d15/d4f/cad 0 2024-02-15T18:34:56.739 INFO:tasks.workunit.client.0.smithi038.stdout:7/508: chown d0/db/d59/l9d 256 1 2024-02-15T18:34:56.739 INFO:tasks.workunit.client.0.smithi038.stdout:2/425: mkdir dd/d12/d8d/d90 0 2024-02-15T18:34:56.740 INFO:tasks.workunit.client.0.smithi038.stdout:5/454: dwrite d2/fc [0,4194304] 0 2024-02-15T18:34:56.745 INFO:tasks.workunit.client.0.smithi038.stdout:4/498: rename d11/d1e/d3e/d75 to d11/d1e/d20/d31/d48/dac/dbe 0 2024-02-15T18:34:56.746 INFO:tasks.workunit.client.0.smithi038.stdout:7/509: creat d0/d68/fae x:0 0 0 2024-02-15T18:34:56.747 INFO:tasks.workunit.client.0.smithi038.stdout:2/426: rename dd/d12/d5d/d54/d89 to dd/d12/d5d/d54/d91 0 2024-02-15T18:34:56.748 INFO:tasks.workunit.client.0.smithi038.stdout:4/499: write d11/d1e/d20/d31/d48/dac/dbe/f46 [3028591,74151] 0 2024-02-15T18:34:56.748 INFO:tasks.workunit.client.0.smithi038.stdout:5/455: mkdir d2/d3/d12/d89 0 2024-02-15T18:34:56.749 INFO:tasks.workunit.client.0.smithi038.stdout:5/456: stat d2/d3/d5f 0 2024-02-15T18:34:56.750 INFO:tasks.workunit.client.0.smithi038.stdout:7/510: symlink d0/d1b/d30/laf 0 2024-02-15T18:34:56.750 INFO:tasks.workunit.client.0.smithi038.stdout:7/511: read - d0/d1b/d30/d50/f8e zero size 2024-02-15T18:34:56.750 INFO:tasks.workunit.client.0.smithi038.stdout:7/512: write d0/d1b/d80/fa9 [383326,4136] 0 2024-02-15T18:34:56.750 INFO:tasks.workunit.client.0.smithi038.stdout:2/427: rename dd/d12/d1d/d3b/l51 to dd/d12/d1d/d3b/d3c/l92 0 2024-02-15T18:34:56.752 INFO:tasks.workunit.client.0.smithi038.stdout:4/500: mkdir d11/d1e/d20/d2b/d4d/d67/dbf 0 2024-02-15T18:34:56.752 INFO:tasks.workunit.client.0.smithi038.stdout:4/501: truncate d11/f13 5021665 0 2024-02-15T18:34:56.753 INFO:tasks.workunit.client.0.smithi038.stdout:4/502: truncate d11/d1e/d20/d31/d8d/fb2 282638 0 2024-02-15T18:34:56.754 INFO:tasks.workunit.client.0.smithi038.stdout:0/530: dwrite d4/f63 [0,4194304] 0 2024-02-15T18:34:56.754 INFO:tasks.workunit.client.0.smithi038.stdout:8/638: dwrite d1/d7/d26/f34 [0,4194304] 0 2024-02-15T18:34:56.755 INFO:tasks.workunit.client.0.smithi038.stdout:0/531: mknod d4/d13/d2a/d35/d1e/d32/d79/d7d/da3/cae 0 2024-02-15T18:34:56.755 INFO:tasks.workunit.client.0.smithi038.stdout:0/532: fsync d4/d13/d53/f95 0 2024-02-15T18:34:56.757 INFO:tasks.workunit.client.0.smithi038.stdout:3/438: dread d2/f12 [0,4194304] 0 2024-02-15T18:34:56.758 INFO:tasks.workunit.client.0.smithi038.stdout:2/428: write dd/d12/d3a/d67/f86 [2554570,85419] 0 2024-02-15T18:34:56.758 INFO:tasks.workunit.client.0.smithi038.stdout:2/429: read dd/f41 [201109,95769] 0 2024-02-15T18:34:56.759 INFO:tasks.workunit.client.0.smithi038.stdout:7/513: write d0/db/d19/f4e [4730907,24356] 0 2024-02-15T18:34:56.764 INFO:tasks.workunit.client.0.smithi038.stdout:8/639: dread d1/d4/d5/f23 [0,4194304] 0 2024-02-15T18:34:56.766 INFO:tasks.workunit.client.0.smithi038.stdout:8/640: dread d1/d3a/f7a [0,4194304] 0 2024-02-15T18:34:56.768 INFO:tasks.workunit.client.0.smithi038.stdout:5/457: dread d2/d3/f1b [0,4194304] 0 2024-02-15T18:34:56.769 INFO:tasks.workunit.client.0.smithi038.stdout:4/503: dread d11/d1e/f37 [0,4194304] 0 2024-02-15T18:34:56.774 INFO:tasks.workunit.client.0.smithi038.stdout:3/439: rmdir d2/d25/d60/d74 0 2024-02-15T18:34:56.774 INFO:tasks.workunit.client.0.smithi038.stdout:3/440: chown d2/d1b/d3c/d3a/d55/f80 1231 1 2024-02-15T18:34:56.775 INFO:tasks.workunit.client.0.smithi038.stdout:2/430: mkdir dd/d12/d5d/d93 0 2024-02-15T18:34:56.777 INFO:tasks.workunit.client.0.smithi038.stdout:0/533: dwrite d4/d13/d19/f82 [0,4194304] 0 2024-02-15T18:34:56.777 INFO:tasks.workunit.client.0.smithi038.stdout:7/514: creat d0/db/fb0 x:0 0 0 2024-02-15T18:34:56.783 INFO:tasks.workunit.client.0.smithi038.stdout:8/641: write d1/d4/d10/d43/f70 [3679792,4611] 0 2024-02-15T18:34:56.783 INFO:tasks.workunit.client.0.smithi038.stdout:8/642: dread - d1/d7/d22/f95 zero size 2024-02-15T18:34:56.783 INFO:tasks.workunit.client.0.smithi038.stdout:8/643: dread - d1/d4/d10/d43/de2/fe4 zero size 2024-02-15T18:34:56.783 INFO:tasks.workunit.client.0.smithi038.stdout:6/491: sync 2024-02-15T18:34:56.783 INFO:tasks.workunit.client.0.smithi038.stdout:1/489: sync 2024-02-15T18:34:56.783 INFO:tasks.workunit.client.0.smithi038.stdout:6/492: chown d2/d4/d19/c57 72 1 2024-02-15T18:34:56.784 INFO:tasks.workunit.client.0.smithi038.stdout:4/504: rename d11/d1e/d20/d31/d48/d60/c6e to d11/d1e/d20/d2b/d4d/d67/cc0 0 2024-02-15T18:34:56.784 INFO:tasks.workunit.client.0.smithi038.stdout:3/441: symlink d2/d1b/d3c/l8a 0 2024-02-15T18:34:56.787 INFO:tasks.workunit.client.0.smithi038.stdout:0/534: creat d4/d13/d2a/d35/d1e/d32/d48/d97/faf x:0 0 0 2024-02-15T18:34:56.787 INFO:tasks.workunit.client.0.smithi038.stdout:1/490: mknod d2/d10/d19/d22/d36/d4f/cab 0 2024-02-15T18:34:56.787 INFO:tasks.workunit.client.0.smithi038.stdout:1/491: write d2/d10/d19/d5f/fa8 [901262,86510] 0 2024-02-15T18:34:56.789 INFO:tasks.workunit.client.0.smithi038.stdout:6/493: write d2/d56/f7c [1384961,65420] 0 2024-02-15T18:34:56.789 INFO:tasks.workunit.client.0.smithi038.stdout:5/458: dwrite d2/f39 [0,4194304] 0 2024-02-15T18:34:56.789 INFO:tasks.workunit.client.0.smithi038.stdout:5/459: truncate d2/d30/d52/f5b 3923841 0 2024-02-15T18:34:56.790 INFO:tasks.workunit.client.0.smithi038.stdout:2/431: rename dd/d12/d59/d2b/d6a/c7c to dd/d12/d1d/c94 0 2024-02-15T18:34:56.794 INFO:tasks.workunit.client.0.smithi038.stdout:3/442: truncate d2/d1b/d4f/f5d 1034909 0 2024-02-15T18:34:56.794 INFO:tasks.workunit.client.0.smithi038.stdout:3/443: chown d2 0 1 2024-02-15T18:34:56.794 INFO:tasks.workunit.client.0.smithi038.stdout:9/542: sync 2024-02-15T18:34:56.797 INFO:tasks.workunit.client.0.smithi038.stdout:1/492: rmdir d2/d10/d19/d22/d36/d4f/d87 0 2024-02-15T18:34:56.797 INFO:tasks.workunit.client.0.smithi038.stdout:8/644: dread d1/d4/d5/f23 [4194304,4194304] 0 2024-02-15T18:34:56.799 INFO:tasks.workunit.client.0.smithi038.stdout:5/460: creat d2/d64/f8a x:0 0 0 2024-02-15T18:34:56.799 INFO:tasks.workunit.client.0.smithi038.stdout:6/494: dread d2/d4/d19/d55/d5c/f70 [0,4194304] 0 2024-02-15T18:34:56.800 INFO:tasks.workunit.client.0.smithi038.stdout:6/495: truncate d2/f2a 2858185 0 2024-02-15T18:34:56.800 INFO:tasks.workunit.client.0.smithi038.stdout:4/505: dread d11/d1e/d20/d2f/d71/f98 [0,4194304] 0 2024-02-15T18:34:56.800 INFO:tasks.workunit.client.0.smithi038.stdout:3/444: mknod d2/d1b/d3c/d3a/d55/d65/c8b 0 2024-02-15T18:34:56.800 INFO:tasks.workunit.client.0.smithi038.stdout:9/543: mknod d1/da/d36/d50/cc8 0 2024-02-15T18:34:56.800 INFO:tasks.workunit.client.0.smithi038.stdout:9/544: readlink d1/da/d20/d58/d94/db9/dc6/d49/l9c 0 2024-02-15T18:34:56.800 INFO:tasks.workunit.client.0.smithi038.stdout:9/545: readlink d1/da/d20/d58/d94/db9/dc6/d49/l9c 0 2024-02-15T18:34:56.802 INFO:tasks.workunit.client.0.smithi038.stdout:8/645: unlink d1/d4/d5/d31/d65/f98 0 2024-02-15T18:34:56.802 INFO:tasks.workunit.client.0.smithi038.stdout:8/646: stat d1/d4/d5/d77/d79/l81 0 2024-02-15T18:34:56.806 INFO:tasks.workunit.client.0.smithi038.stdout:5/461: creat d2/d30/d52/f8b x:0 0 0 2024-02-15T18:34:56.810 INFO:tasks.workunit.client.0.smithi038.stdout:6/496: truncate d2/f8e 761786 0 2024-02-15T18:34:56.812 INFO:tasks.workunit.client.0.smithi038.stdout:7/515: rename d0/d95/f37 to d0/db/fb1 0 2024-02-15T18:34:56.812 INFO:tasks.workunit.client.0.smithi038.stdout:7/516: truncate d0/d95/f51 678300 0 2024-02-15T18:34:56.812 INFO:tasks.workunit.client.0.smithi038.stdout:7/517: chown d0/d1b/d30/d5d/d61/f75 112794907 1 2024-02-15T18:34:56.812 INFO:tasks.workunit.client.0.smithi038.stdout:4/506: mknod d11/d1e/cc1 0 2024-02-15T18:34:56.812 INFO:tasks.workunit.client.0.smithi038.stdout:4/507: truncate d11/d1e/d20/d31/d8d/d99/fb6 443 0 2024-02-15T18:34:56.816 INFO:tasks.workunit.client.0.smithi038.stdout:3/445: mkdir d2/d1b/d8c 0 2024-02-15T18:34:56.816 INFO:tasks.workunit.client.0.smithi038.stdout:9/546: symlink d1/da/d20/d58/d94/db9/dc6/lc9 0 2024-02-15T18:34:56.816 INFO:tasks.workunit.client.0.smithi038.stdout:8/647: creat d1/d4/d10/d43/de2/fed x:0 0 0 2024-02-15T18:34:56.817 INFO:tasks.workunit.client.0.smithi038.stdout:5/462: creat d2/d3/d12/f8c x:0 0 0 2024-02-15T18:34:56.819 INFO:tasks.workunit.client.0.smithi038.stdout:6/497: unlink d2/d4/fc 0 2024-02-15T18:34:56.819 INFO:tasks.workunit.client.0.smithi038.stdout:9/547: getdents d1/da/d36/d3f/d48/db8 0 2024-02-15T18:34:56.819 INFO:tasks.workunit.client.0.smithi038.stdout:9/548: fsync d1/da/d10/d13/d3d/d4b/daa/fb1 0 2024-02-15T18:34:56.820 INFO:tasks.workunit.client.0.smithi038.stdout:8/648: mknod d1/d7/d15/d6a/cee 0 2024-02-15T18:34:56.820 INFO:tasks.workunit.client.0.smithi038.stdout:8/649: dread - d1/d4/d10/d43/d48/d9a/fcf zero size 2024-02-15T18:34:56.820 INFO:tasks.workunit.client.0.smithi038.stdout:8/650: readlink d1/d4/d5/d6/d6d/lc9 0 2024-02-15T18:34:56.820 INFO:tasks.workunit.client.0.smithi038.stdout:8/651: chown d1/d7/d26/c56 1 1 2024-02-15T18:34:56.821 INFO:tasks.workunit.client.0.smithi038.stdout:6/498: unlink d2/l43 0 2024-02-15T18:34:56.822 INFO:tasks.workunit.client.0.smithi038.stdout:8/652: rename d1/d4/db1/de5/db3/fd0 to d1/d4/d10/d3d/db7/dd5/fef 0 2024-02-15T18:34:56.824 INFO:tasks.workunit.client.0.smithi038.stdout:1/493: dwrite d2/d10/d19/d22/d36/f4d [0,4194304] 0 2024-02-15T18:34:56.824 INFO:tasks.workunit.client.0.smithi038.stdout:1/494: dread - d2/d39/f48 zero size 2024-02-15T18:34:56.827 INFO:tasks.workunit.client.0.smithi038.stdout:2/432: dwrite dd/d12/d3a/d67/f30 [4194304,4194304] 0 2024-02-15T18:34:56.827 INFO:tasks.workunit.client.0.smithi038.stdout:2/433: fdatasync dd/f41 0 2024-02-15T18:34:56.829 INFO:tasks.workunit.client.0.smithi038.stdout:9/549: dread d1/dc/d4f/f47 [0,4194304] 0 2024-02-15T18:34:56.829 INFO:tasks.workunit.client.0.smithi038.stdout:9/550: readlink d1/da/d36/d3f/l6a 0 2024-02-15T18:34:56.829 INFO:tasks.workunit.client.0.smithi038.stdout:0/535: dwrite d4/d13/d2a/d35/d1e/d32/d48/f8c [0,4194304] 0 2024-02-15T18:34:56.829 INFO:tasks.workunit.client.0.smithi038.stdout:0/536: fdatasync d4/d13/d53/f7b 0 2024-02-15T18:34:56.830 INFO:tasks.workunit.client.0.smithi038.stdout:2/434: write dd/f46 [40943,99288] 0 2024-02-15T18:34:56.832 INFO:tasks.workunit.client.0.smithi038.stdout:7/518: dread d0/d1b/d30/d5d/d70/f9f [0,4194304] 0 2024-02-15T18:34:56.832 INFO:tasks.workunit.client.0.smithi038.stdout:9/551: write d1/da/d20/d58/d94/db9/dc6/f7c [570138,57300] 0 2024-02-15T18:34:56.833 INFO:tasks.workunit.client.0.smithi038.stdout:1/495: mknod d2/d28/d6b/cac 0 2024-02-15T18:34:56.833 INFO:tasks.workunit.client.0.smithi038.stdout:1/496: stat d2/d10/d19/d22 0 2024-02-15T18:34:56.833 INFO:tasks.workunit.client.0.smithi038.stdout:1/497: chown d2/d10/d19/c2b 1294366217 1 2024-02-15T18:34:56.836 INFO:tasks.workunit.client.0.smithi038.stdout:2/435: rmdir dd/d12/d1d/d3b/d3c 39 2024-02-15T18:34:56.836 INFO:tasks.workunit.client.0.smithi038.stdout:2/436: truncate dd/d12/d3a/d5e/f87 160221 0 2024-02-15T18:34:56.838 INFO:tasks.workunit.client.0.smithi038.stdout:7/519: dread - d0/db/f8c zero size 2024-02-15T18:34:56.840 INFO:tasks.workunit.client.0.smithi038.stdout:9/552: symlink d1/dc/lca 0 2024-02-15T18:34:56.847 INFO:tasks.workunit.client.0.smithi038.stdout:2/437: rmdir dd/d12/d1d/d82 0 2024-02-15T18:34:56.849 INFO:tasks.workunit.client.0.smithi038.stdout:2/438: mknod dd/d12/d8d/c95 0 2024-02-15T18:34:56.853 INFO:tasks.workunit.client.0.smithi038.stdout:0/537: dread d4/d13/d19/f5c [0,4194304] 0 2024-02-15T18:34:56.855 INFO:tasks.workunit.client.0.smithi038.stdout:0/538: link d4/f12 d4/d13/d19/fb0 0 2024-02-15T18:34:56.855 INFO:tasks.workunit.client.0.smithi038.stdout:3/446: dwrite d2/d1b/f4a [4194304,4194304] 0 2024-02-15T18:34:56.855 INFO:tasks.workunit.client.0.smithi038.stdout:3/447: fdatasync d2/f1a 0 2024-02-15T18:34:56.856 INFO:tasks.workunit.client.0.smithi038.stdout:0/539: link d4/f11 d4/fb1 0 2024-02-15T18:34:56.858 INFO:tasks.workunit.client.0.smithi038.stdout:3/448: rename d2/d1e/f32 to d2/d1b/d4f/f8d 0 2024-02-15T18:34:56.860 INFO:tasks.workunit.client.0.smithi038.stdout:2/439: dread dd/d12/f39 [0,4194304] 0 2024-02-15T18:34:56.868 INFO:tasks.workunit.client.0.smithi038.stdout:5/463: dwrite d2/d30/f5d [0,4194304] 0 2024-02-15T18:34:56.868 INFO:tasks.workunit.client.0.smithi038.stdout:5/464: truncate d2/f80 97248 0 2024-02-15T18:34:56.868 INFO:tasks.workunit.client.0.smithi038.stdout:5/465: readlink d2/d3/l7c 0 2024-02-15T18:34:56.869 INFO:tasks.workunit.client.0.smithi038.stdout:8/653: dwrite d1/d7/d15/d4b/f93 [0,4194304] 0 2024-02-15T18:34:56.870 INFO:tasks.workunit.client.0.smithi038.stdout:8/654: dread - d1/d4/faa zero size 2024-02-15T18:34:56.870 INFO:tasks.workunit.client.0.smithi038.stdout:5/466: link d2/f69 d2/d30/d57/f8d 0 2024-02-15T18:34:56.872 INFO:tasks.workunit.client.0.smithi038.stdout:7/520: dwrite d0/d1b/d30/d5d/d61/d90/fab [0,4194304] 0 2024-02-15T18:34:56.872 INFO:tasks.workunit.client.0.smithi038.stdout:8/655: creat d1/d4/d5/d6/db9/ff0 x:0 0 0 2024-02-15T18:34:56.873 INFO:tasks.workunit.client.0.smithi038.stdout:8/656: truncate d1/f1b 898601 0 2024-02-15T18:34:56.873 INFO:tasks.workunit.client.0.smithi038.stdout:5/467: write d2/fc [4112517,56961] 0 2024-02-15T18:34:56.874 INFO:tasks.workunit.client.0.smithi038.stdout:9/553: dwrite d1/da/d36/d8b/f96 [0,4194304] 0 2024-02-15T18:34:56.875 INFO:tasks.workunit.client.0.smithi038.stdout:9/554: chown d1/da/d10/d13/d3d/d6c 30 1 2024-02-15T18:34:56.875 INFO:tasks.workunit.client.0.smithi038.stdout:6/499: dwrite d2/f5b [0,4194304] 0 2024-02-15T18:34:56.879 INFO:tasks.workunit.client.0.smithi038.stdout:1/498: truncate d2/d10/d19/d22/d36/f4d 2369346 0 2024-02-15T18:34:56.881 INFO:tasks.workunit.client.0.smithi038.stdout:0/540: dwrite d4/f88 [4194304,4194304] 0 2024-02-15T18:34:56.888 INFO:tasks.workunit.client.0.smithi038.stdout:2/440: dwrite dd/d12/d1d/f35 [0,4194304] 0 2024-02-15T18:34:56.888 INFO:tasks.workunit.client.0.smithi038.stdout:0/541: dread d4/d13/d19/f5c [0,4194304] 0 2024-02-15T18:34:56.888 INFO:tasks.workunit.client.0.smithi038.stdout:3/449: dwrite d2/d1b/d3c/f31 [0,4194304] 0 2024-02-15T18:34:56.888 INFO:tasks.workunit.client.0.smithi038.stdout:3/450: fdatasync d2/d1b/d3c/d3a/d55/f81 0 2024-02-15T18:34:56.888 INFO:tasks.workunit.client.0.smithi038.stdout:7/521: symlink d0/d66/lb2 0 2024-02-15T18:34:56.888 INFO:tasks.workunit.client.0.smithi038.stdout:8/657: mknod d1/d7/d15/d6a/cf1 0 2024-02-15T18:34:56.891 INFO:tasks.workunit.client.0.smithi038.stdout:5/468: rmdir d2/d3 39 2024-02-15T18:34:56.896 INFO:tasks.workunit.client.0.smithi038.stdout:9/555: mkdir d1/da/d36/d3f/d48/db8/dcb 0 2024-02-15T18:34:56.896 INFO:tasks.workunit.client.0.smithi038.stdout:9/556: fsync d1/da/d36/d3f/d48/db8/fbb 0 2024-02-15T18:34:56.896 INFO:tasks.workunit.client.0.smithi038.stdout:9/557: dread - d1/da/d10/f9f zero size 2024-02-15T18:34:56.898 INFO:tasks.workunit.client.0.smithi038.stdout:1/499: dwrite d2/d28/f34 [4194304,4194304] 0 2024-02-15T18:34:56.900 INFO:tasks.workunit.client.0.smithi038.stdout:6/500: rename d2/d56/d5f/f68 to d2/d10/f8f 0 2024-02-15T18:34:56.907 INFO:tasks.workunit.client.0.smithi038.stdout:2/441: mknod dd/d12/d1d/d3b/d3c/d4a/c96 0 2024-02-15T18:34:56.907 INFO:tasks.workunit.client.0.smithi038.stdout:2/442: write dd/f41 [782939,63312] 0 2024-02-15T18:34:56.907 INFO:tasks.workunit.client.0.smithi038.stdout:0/542: dwrite d4/d13/d2a/d35/d1e/d32/d48/d97/faf [0,4194304] 0 2024-02-15T18:34:56.908 INFO:tasks.workunit.client.0.smithi038.stdout:3/451: creat d2/d1b/f8e x:0 0 0 2024-02-15T18:34:56.909 INFO:tasks.workunit.client.0.smithi038.stdout:3/452: dread - d2/d25/f66 zero size 2024-02-15T18:34:56.909 INFO:tasks.workunit.client.0.smithi038.stdout:7/522: mknod d0/db/d59/d15/d29/d7e/cb3 0 2024-02-15T18:34:56.915 INFO:tasks.workunit.client.0.smithi038.stdout:8/658: link d1/d7/d22/f5a d1/d4/d5/d31/d65/ff2 0 2024-02-15T18:34:56.915 INFO:tasks.workunit.client.0.smithi038.stdout:8/659: chown d1/d4/d5/f23 0 1 2024-02-15T18:34:56.916 INFO:tasks.workunit.client.0.smithi038.stdout:4/508: sync 2024-02-15T18:34:56.916 INFO:tasks.workunit.client.0.smithi038.stdout:4/509: truncate d11/d1e/d3e/d57/f82 1783902 0 2024-02-15T18:34:56.916 INFO:tasks.workunit.client.0.smithi038.stdout:5/469: mkdir d2/d75/d8e 0 2024-02-15T18:34:56.916 INFO:tasks.workunit.client.0.smithi038.stdout:6/501: dread d2/d4/d2c/f4f [0,4194304] 0 2024-02-15T18:34:56.917 INFO:tasks.workunit.client.0.smithi038.stdout:9/558: mknod d1/dc/d4f/ccc 0 2024-02-15T18:34:56.917 INFO:tasks.workunit.client.0.smithi038.stdout:1/500: rmdir d2/d2e 39 2024-02-15T18:34:56.919 INFO:tasks.workunit.client.0.smithi038.stdout:3/453: creat d2/d1b/d3c/d41/d4e/d62/f8f x:0 0 0 2024-02-15T18:34:56.919 INFO:tasks.workunit.client.0.smithi038.stdout:2/443: dwrite dd/d12/d3a/d67/d4f/d5a/f60 [4194304,4194304] 0 2024-02-15T18:34:56.920 INFO:tasks.workunit.client.0.smithi038.stdout:0/543: dread d4/d13/d53/f95 [0,4194304] 0 2024-02-15T18:34:56.920 INFO:tasks.workunit.client.0.smithi038.stdout:7/523: mkdir d0/db/d59/d15/d29/d8d/db4 0 2024-02-15T18:34:56.921 INFO:tasks.workunit.client.0.smithi038.stdout:8/660: creat d1/d4/d10/d43/d48/ff3 x:0 0 0 2024-02-15T18:34:56.924 INFO:tasks.workunit.client.0.smithi038.stdout:1/501: dread d2/d10/d19/d22/f7a [0,4194304] 0 2024-02-15T18:34:56.925 INFO:tasks.workunit.client.0.smithi038.stdout:4/510: truncate d11/d1e/d20/d31/d48/dac/dbe/f34 5148484 0 2024-02-15T18:34:56.926 INFO:tasks.workunit.client.0.smithi038.stdout:5/470: rename d2/d3/f4b to d2/d3/d12/d5e/f8f 0 2024-02-15T18:34:56.927 INFO:tasks.workunit.client.0.smithi038.stdout:9/559: link d1/da/d10/d13/d3d/d4b/lae d1/da/d20/d58/lcd 0 2024-02-15T18:34:56.928 INFO:tasks.workunit.client.0.smithi038.stdout:0/544: dread d4/d13/d2a/d35/f8f [0,4194304] 0 2024-02-15T18:34:56.928 INFO:tasks.workunit.client.0.smithi038.stdout:0/545: fdatasync d4/f11 0 2024-02-15T18:34:56.928 INFO:tasks.workunit.client.0.smithi038.stdout:5/471: write d2/d30/f5d [1916732,42993] 0 2024-02-15T18:34:56.929 INFO:tasks.workunit.client.0.smithi038.stdout:3/454: getdents d2/d1b/d3c/d42 0 2024-02-15T18:34:56.931 INFO:tasks.workunit.client.0.smithi038.stdout:7/524: getdents d0/d1b/d30/d5d/d70 0 2024-02-15T18:34:56.931 INFO:tasks.workunit.client.0.smithi038.stdout:7/525: truncate d0/db/fb0 446703 0 2024-02-15T18:34:56.932 INFO:tasks.workunit.client.0.smithi038.stdout:1/502: mkdir d2/d69/da9/dad 0 2024-02-15T18:34:56.932 INFO:tasks.workunit.client.0.smithi038.stdout:1/503: stat d2/d10/d19/d2f/d3e 0 2024-02-15T18:34:56.934 INFO:tasks.workunit.client.0.smithi038.stdout:4/511: unlink d11/d1e/d20/d31/d48/dac/dbe/f46 0 2024-02-15T18:34:56.935 INFO:tasks.workunit.client.0.smithi038.stdout:6/502: rename d2/l35 to d2/d4/d2c/d39/d76/l90 0 2024-02-15T18:34:56.939 INFO:tasks.workunit.client.0.smithi038.stdout:0/546: mkdir d4/d13/d2a/d35/db2 0 2024-02-15T18:34:56.940 INFO:tasks.workunit.client.0.smithi038.stdout:3/455: dread d2/d25/f30 [0,4194304] 0 2024-02-15T18:34:56.942 INFO:tasks.workunit.client.0.smithi038.stdout:5/472: creat d2/d30/d57/f90 x:0 0 0 2024-02-15T18:34:56.942 INFO:tasks.workunit.client.0.smithi038.stdout:7/526: rmdir d0/d1b 39 2024-02-15T18:34:56.942 INFO:tasks.workunit.client.0.smithi038.stdout:1/504: truncate d2/d2e/f38 3074280 0 2024-02-15T18:34:56.943 INFO:tasks.workunit.client.0.smithi038.stdout:4/512: mknod d11/d1e/d20/d31/d48/dac/dbe/cc2 0 2024-02-15T18:34:56.943 INFO:tasks.workunit.client.0.smithi038.stdout:4/513: readlink d11/l1c 0 2024-02-15T18:34:56.945 INFO:tasks.workunit.client.0.smithi038.stdout:2/444: rename dd/d12/d1d/d3b/f6f to dd/d12/d5d/d54/f97 0 2024-02-15T18:34:56.945 INFO:tasks.workunit.client.0.smithi038.stdout:9/560: dwrite d1/fa1 [0,4194304] 0 2024-02-15T18:34:56.946 INFO:tasks.workunit.client.0.smithi038.stdout:6/503: mkdir d2/d4/d6c/d91 0 2024-02-15T18:34:56.947 INFO:tasks.workunit.client.0.smithi038.stdout:5/473: symlink d2/d3/d12/d5e/l91 0 2024-02-15T18:34:56.953 INFO:tasks.workunit.client.0.smithi038.stdout:7/527: mkdir d0/db/d19/d7d/db5 0 2024-02-15T18:34:56.953 INFO:tasks.workunit.client.0.smithi038.stdout:7/528: write d0/d1b/d30/d50/f74 [1019154,1951] 0 2024-02-15T18:34:56.953 INFO:tasks.workunit.client.0.smithi038.stdout:7/529: chown d0/db/d59/c2f 371 1 2024-02-15T18:34:56.953 INFO:tasks.workunit.client.0.smithi038.stdout:3/456: dread d2/f12 [0,4194304] 0 2024-02-15T18:34:56.955 INFO:tasks.workunit.client.0.smithi038.stdout:6/504: read d2/d4/d28/f3a [2479030,104266] 0 2024-02-15T18:34:56.955 INFO:tasks.workunit.client.0.smithi038.stdout:6/505: write d2/d10/f79 [992294,110467] 0 2024-02-15T18:34:56.974 INFO:tasks.workunit.client.0.smithi038.stdout:5/474: dwrite d2/f1a [0,4194304] 0 2024-02-15T18:34:56.974 INFO:tasks.workunit.client.0.smithi038.stdout:5/475: chown d2/d75/f85 6548212 1 2024-02-15T18:34:56.974 INFO:tasks.workunit.client.0.smithi038.stdout:5/476: stat d2/d3/d12/l3d 0 2024-02-15T18:34:56.974 INFO:tasks.workunit.client.0.smithi038.stdout:2/445: dwrite fc [4194304,4194304] 0 2024-02-15T18:34:56.975 INFO:tasks.workunit.client.0.smithi038.stdout:2/446: stat dd/d12/d1d/f2e 0 2024-02-15T18:34:56.975 INFO:tasks.workunit.client.0.smithi038.stdout:9/561: dwrite d1/da/d10/d13/d3d/d43/f97 [0,4194304] 0 2024-02-15T18:34:56.984 INFO:tasks.workunit.client.0.smithi038.stdout:1/505: creat d2/d2e/d8c/fae x:0 0 0 2024-02-15T18:34:56.984 INFO:tasks.workunit.client.0.smithi038.stdout:1/506: dread - d2/d2e/d8c/fae zero size 2024-02-15T18:34:56.985 INFO:tasks.workunit.client.0.smithi038.stdout:5/477: dread d2/d3/ff [0,4194304] 0 2024-02-15T18:34:56.985 INFO:tasks.workunit.client.0.smithi038.stdout:4/514: unlink d11/d1e/d20/d31/d48/dac/dbe/f34 0 2024-02-15T18:34:56.986 INFO:tasks.workunit.client.0.smithi038.stdout:4/515: chown d11/d1e/d20/d31/d7a/fbc 110619201 1 2024-02-15T18:34:56.986 INFO:tasks.workunit.client.0.smithi038.stdout:4/516: dread d11/d1e/d20/d31/d8d/d99/fb6 [0,4194304] 0 2024-02-15T18:34:56.986 INFO:tasks.workunit.client.0.smithi038.stdout:4/517: chown d11/d1e/d20/d2b/d4d/d67/d73/d8b/fad 0 1 2024-02-15T18:34:57.004 INFO:tasks.workunit.client.0.smithi038.stdout:1/507: dwrite d2/d10/d32/d53/f58 [0,4194304] 0 2024-02-15T18:34:57.019 INFO:tasks.workunit.client.0.smithi038.stdout:7/530: creat d0/d1b/d30/d5d/d61/fb6 x:0 0 0 2024-02-15T18:34:57.019 INFO:tasks.workunit.client.0.smithi038.stdout:7/531: write d0/db/d59/fa4 [935894,25298] 0 2024-02-15T18:34:57.020 INFO:tasks.workunit.client.0.smithi038.stdout:3/457: mknod d2/d1e/c90 0 2024-02-15T18:34:57.021 INFO:tasks.workunit.client.0.smithi038.stdout:6/506: symlink d2/l92 0 2024-02-15T18:34:57.024 INFO:tasks.workunit.client.0.smithi038.stdout:2/447: getdents dd/d12/d8d 0 2024-02-15T18:34:57.030 INFO:tasks.workunit.client.0.smithi038.stdout:9/562: creat d1/da/d20/d58/d94/db9/dc6/d49/d95/fce x:0 0 0 2024-02-15T18:34:57.036 INFO:tasks.workunit.client.0.smithi038.stdout:3/458: dread d2/d1b/d4f/f8d [0,4194304] 0 2024-02-15T18:34:57.038 INFO:tasks.workunit.client.0.smithi038.stdout:3/459: read d2/d25/f30 [1368004,58493] 0 2024-02-15T18:34:57.038 INFO:tasks.workunit.client.0.smithi038.stdout:3/460: readlink d2/d1b/d3c/l2b 0 2024-02-15T18:34:57.038 INFO:tasks.workunit.client.0.smithi038.stdout:3/461: dread - d2/d1b/d3c/d3a/d55/f81 zero size 2024-02-15T18:34:57.040 INFO:tasks.workunit.client.0.smithi038.stdout:4/518: mknod d11/d1e/d20/d31/d7a/d8f/dba/cc3 0 2024-02-15T18:34:57.041 INFO:tasks.workunit.client.0.smithi038.stdout:7/532: dwrite d0/db/d19/f4e [4194304,4194304] 0 2024-02-15T18:34:57.045 INFO:tasks.workunit.client.0.smithi038.stdout:1/508: truncate d2/d69/f6c 2005148 0 2024-02-15T18:34:57.045 INFO:tasks.workunit.client.0.smithi038.stdout:1/509: fdatasync d2/d10/d19/d22/d36/f44 0 2024-02-15T18:34:57.047 INFO:tasks.workunit.client.0.smithi038.stdout:2/448: rename dd/d12/d1d/d3b/d3c/d4a/c53 to dd/d12/d3a/d67/d4f/c98 0 2024-02-15T18:34:57.047 INFO:tasks.workunit.client.0.smithi038.stdout:2/449: chown dd/d12/d5d/d54/f97 493494 1 2024-02-15T18:34:57.049 INFO:tasks.workunit.client.0.smithi038.stdout:3/462: creat d2/d1b/d3c/d3a/f91 x:0 0 0 2024-02-15T18:34:57.050 INFO:tasks.workunit.client.0.smithi038.stdout:9/563: dwrite d1/da/d10/d13/f7a [0,4194304] 0 2024-02-15T18:34:57.050 INFO:tasks.workunit.client.0.smithi038.stdout:9/564: dread - d1/da/d10/d13/d3d/d43/d4d/d8a/f8e zero size 2024-02-15T18:34:57.050 INFO:tasks.workunit.client.0.smithi038.stdout:9/565: chown d1/da/d10/d13/d3d/d4b/l7e 11293 1 2024-02-15T18:34:57.050 INFO:tasks.workunit.client.0.smithi038.stdout:5/478: truncate d2/d30/f5d 3377746 0 2024-02-15T18:34:57.050 INFO:tasks.workunit.client.0.smithi038.stdout:5/479: read - d2/d3/f6f zero size 2024-02-15T18:34:57.051 INFO:tasks.workunit.client.0.smithi038.stdout:7/533: symlink d0/d66/lb7 0 2024-02-15T18:34:57.051 INFO:tasks.workunit.client.0.smithi038.stdout:7/534: chown d0/db/d59/d15/f23 13 1 2024-02-15T18:34:57.051 INFO:tasks.workunit.client.0.smithi038.stdout:7/535: fsync d0/d1b/d30/d5d/d61/f84 0 2024-02-15T18:34:57.051 INFO:tasks.workunit.client.0.smithi038.stdout:7/536: truncate d0/d1b/d30/d5d/d61/f75 1220609 0 2024-02-15T18:34:57.052 INFO:tasks.workunit.client.0.smithi038.stdout:7/537: readlink d0/db/d59/d15/d29/d7e/l53 0 2024-02-15T18:34:57.052 INFO:tasks.workunit.client.0.smithi038.stdout:7/538: readlink d0/db/d59/l9d 0 2024-02-15T18:34:57.052 INFO:tasks.workunit.client.0.smithi038.stdout:7/539: read - d0/db/d19/d7d/f94 zero size 2024-02-15T18:34:57.052 INFO:tasks.workunit.client.0.smithi038.stdout:5/480: dread d2/d3/d12/f19 [0,4194304] 0 2024-02-15T18:34:57.052 INFO:tasks.workunit.client.0.smithi038.stdout:1/510: creat d2/d10/d19/d22/d54/faf x:0 0 0 2024-02-15T18:34:57.053 INFO:tasks.workunit.client.0.smithi038.stdout:1/511: chown d2/d10/d19/la7 20414 1 2024-02-15T18:34:57.054 INFO:tasks.workunit.client.0.smithi038.stdout:5/481: write d2/d30/d57/f79 [2215581,17358] 0 2024-02-15T18:34:57.054 INFO:tasks.workunit.client.0.smithi038.stdout:5/482: chown d2/d30/l81 195350 1 2024-02-15T18:34:57.054 INFO:tasks.workunit.client.0.smithi038.stdout:5/483: truncate d2/d30/d57/f90 751498 0 2024-02-15T18:34:57.054 INFO:tasks.workunit.client.0.smithi038.stdout:5/484: chown d2/d3/ff 1316329 1 2024-02-15T18:34:57.054 INFO:tasks.workunit.client.0.smithi038.stdout:5/485: fdatasync d2/d30/d57/f78 0 2024-02-15T18:34:57.054 INFO:tasks.workunit.client.0.smithi038.stdout:5/486: fsync d2/fc 0 2024-02-15T18:34:57.055 INFO:tasks.workunit.client.0.smithi038.stdout:8/661: sync 2024-02-15T18:34:57.055 INFO:tasks.workunit.client.0.smithi038.stdout:0/547: sync 2024-02-15T18:34:57.057 INFO:tasks.workunit.client.0.smithi038.stdout:2/450: truncate dd/d12/d1d/d3b/f4b 2872234 0 2024-02-15T18:34:57.057 INFO:tasks.workunit.client.0.smithi038.stdout:2/451: dread - dd/d12/d1d/d3b/d3c/d4a/f8c zero size 2024-02-15T18:34:57.058 INFO:tasks.workunit.client.0.smithi038.stdout:2/452: write dd/d12/d5d/f28 [789859,71419] 0 2024-02-15T18:34:57.058 INFO:tasks.workunit.client.0.smithi038.stdout:7/540: dread d0/d66/f35 [0,4194304] 0 2024-02-15T18:34:57.058 INFO:tasks.workunit.client.0.smithi038.stdout:3/463: mkdir d2/d1b/d4f/d92 0 2024-02-15T18:34:57.058 INFO:tasks.workunit.client.0.smithi038.stdout:3/464: write d2/d25/f64 [965596,117704] 0 2024-02-15T18:34:57.061 INFO:tasks.workunit.client.0.smithi038.stdout:9/566: rmdir d1/da/d10/d13/d3d/d4b 39 2024-02-15T18:34:57.061 INFO:tasks.workunit.client.0.smithi038.stdout:9/567: fsync d1/dc/d4f/f47 0 2024-02-15T18:34:57.061 INFO:tasks.workunit.client.0.smithi038.stdout:9/568: write d1/da/d36/d93/fa4 [76029,59613] 0 2024-02-15T18:34:57.061 INFO:tasks.workunit.client.0.smithi038.stdout:9/569: chown d1/da/d20/lb6 375 1 2024-02-15T18:34:57.062 INFO:tasks.workunit.client.0.smithi038.stdout:4/519: rename f4 to d11/d1e/d20/d2b/d35/fc4 0 2024-02-15T18:34:57.064 INFO:tasks.workunit.client.0.smithi038.stdout:8/662: truncate d1/d4/d5/f23 1910952 0 2024-02-15T18:34:57.064 INFO:tasks.workunit.client.0.smithi038.stdout:8/663: stat d1/d4/d10/d3d/c92 0 2024-02-15T18:34:57.064 INFO:tasks.workunit.client.0.smithi038.stdout:0/548: truncate d4/d13/d2a/d35/d1e/f20 7165906 0 2024-02-15T18:34:57.065 INFO:tasks.workunit.client.0.smithi038.stdout:2/453: mknod dd/d12/d5d/d54/c99 0 2024-02-15T18:34:57.072 INFO:tasks.workunit.client.0.smithi038.stdout:1/512: rename d2/d10/d32/d53/f58 to d2/d2e/d8c/fb0 0 2024-02-15T18:34:57.073 INFO:tasks.workunit.client.0.smithi038.stdout:0/549: read d4/d13/d2a/d35/d1e/d32/d48/d97/faf [421932,117160] 0 2024-02-15T18:34:57.074 INFO:tasks.workunit.client.0.smithi038.stdout:0/550: write d4/f88 [8978753,8187] 0 2024-02-15T18:34:57.074 INFO:tasks.workunit.client.0.smithi038.stdout:4/520: rename d11/d1e/d43/d5e/d62/c83 to d11/d1e/d20/d31/cc5 0 2024-02-15T18:34:57.074 INFO:tasks.workunit.client.0.smithi038.stdout:4/521: write d11/d1e/d20/d31/d8d/d99/fb6 [785608,5069] 0 2024-02-15T18:34:57.074 INFO:tasks.workunit.client.0.smithi038.stdout:0/551: mkdir d4/db3 0 2024-02-15T18:34:57.075 INFO:tasks.workunit.client.0.smithi038.stdout:9/570: dread d1/da/d36/d50/f73 [0,4194304] 0 2024-02-15T18:34:57.075 INFO:tasks.workunit.client.0.smithi038.stdout:9/571: fsync d1/da/d20/d58/d76/fb5 0 2024-02-15T18:34:57.075 INFO:tasks.workunit.client.0.smithi038.stdout:9/572: dread - d1/da/d10/d13/d3d/fa3 zero size 2024-02-15T18:34:57.075 INFO:tasks.workunit.client.0.smithi038.stdout:9/573: fsync d1/da/d10/d13/f18 0 2024-02-15T18:34:57.078 INFO:tasks.workunit.client.0.smithi038.stdout:4/522: write d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/f9b [1776963,2136] 0 2024-02-15T18:34:57.078 INFO:tasks.workunit.client.0.smithi038.stdout:2/454: dread dd/d12/d1d/f31 [4194304,4194304] 0 2024-02-15T18:34:57.078 INFO:tasks.workunit.client.0.smithi038.stdout:9/574: symlink d1/dc/d69/lcf 0 2024-02-15T18:34:57.078 INFO:tasks.workunit.client.0.smithi038.stdout:2/455: chown dd/d12/d59/d2b/l37 1036595 1 2024-02-15T18:34:57.078 INFO:tasks.workunit.client.0.smithi038.stdout:0/552: getdents d4/d13/d2a/d35/d1e/d32/d79/d7d 0 2024-02-15T18:34:57.078 INFO:tasks.workunit.client.0.smithi038.stdout:0/553: write d4/d13/d2a/d35/d1e/d32/d79/d7d/f96 [777877,11033] 0 2024-02-15T18:34:57.078 INFO:tasks.workunit.client.0.smithi038.stdout:0/554: stat d4/d13/d19/c2c 0 2024-02-15T18:34:57.079 INFO:tasks.workunit.client.0.smithi038.stdout:4/523: symlink d11/d1e/d20/d2b/d4d/d67/da1/lc6 0 2024-02-15T18:34:57.079 INFO:tasks.workunit.client.0.smithi038.stdout:1/513: dread d2/d10/d19/d22/d36/f4d [0,4194304] 0 2024-02-15T18:34:57.079 INFO:tasks.workunit.client.0.smithi038.stdout:2/456: read dd/d12/d5d/d54/f97 [41527,86325] 0 2024-02-15T18:34:57.080 INFO:tasks.workunit.client.0.smithi038.stdout:9/575: link d1/da/d36/d50/cc8 d1/da/d36/d3f/d48/db8/dcb/cd0 0 2024-02-15T18:34:57.082 INFO:tasks.workunit.client.0.smithi038.stdout:1/514: mknod d2/d10/d19/d22/d36/d4f/cb1 0 2024-02-15T18:34:57.083 INFO:tasks.workunit.client.0.smithi038.stdout:2/457: symlink dd/d12/d59/d2b/d6a/l9a 0 2024-02-15T18:34:57.084 INFO:tasks.workunit.client.0.smithi038.stdout:1/515: unlink d2/l35 0 2024-02-15T18:34:57.087 INFO:tasks.workunit.client.0.smithi038.stdout:3/465: dwrite d2/d1b/d3c/d3a/d55/f80 [0,4194304] 0 2024-02-15T18:34:57.088 INFO:tasks.workunit.client.0.smithi038.stdout:5/487: dwrite d2/d3/d12/f19 [0,4194304] 0 2024-02-15T18:34:57.088 INFO:tasks.workunit.client.0.smithi038.stdout:5/488: chown d2/d30/d57/f8d 49 1 2024-02-15T18:34:57.089 INFO:tasks.workunit.client.0.smithi038.stdout:8/664: dwrite d1/d4/d10/d43/d48/d52/fab [0,4194304] 0 2024-02-15T18:34:57.089 INFO:tasks.workunit.client.0.smithi038.stdout:8/665: chown d1/d7/fc 15352949 1 2024-02-15T18:34:57.089 INFO:tasks.workunit.client.0.smithi038.stdout:4/524: rename d11/d1e/d20/d2b/d35/fc4 to d11/d1e/d43/d4a/fc7 0 2024-02-15T18:34:57.089 INFO:tasks.workunit.client.0.smithi038.stdout:7/541: dwrite d0/db/fe [0,4194304] 0 2024-02-15T18:34:57.090 INFO:tasks.workunit.client.0.smithi038.stdout:3/466: write d2/f50 [1305362,47178] 0 2024-02-15T18:34:57.094 INFO:tasks.workunit.client.0.smithi038.stdout:2/458: dread dd/d12/d3a/d67/d4f/d5a/f60 [4194304,4194304] 0 2024-02-15T18:34:57.094 INFO:tasks.workunit.client.0.smithi038.stdout:2/459: write dd/d12/d3a/d5e/f87 [697567,94709] 0 2024-02-15T18:34:57.097 INFO:tasks.workunit.client.0.smithi038.stdout:4/525: unlink d11/d1e/d3e/l49 0 2024-02-15T18:34:57.097 INFO:tasks.workunit.client.0.smithi038.stdout:4/526: chown d11/d1e/d20/d31/c4f 827 1 2024-02-15T18:34:57.097 INFO:tasks.workunit.client.0.smithi038.stdout:4/527: chown d11/d1e/d20/d31/d48/d60/f66 1 1 2024-02-15T18:34:57.097 INFO:tasks.workunit.client.0.smithi038.stdout:4/528: fdatasync d11/d1e/d20/f70 0 2024-02-15T18:34:57.100 INFO:tasks.workunit.client.0.smithi038.stdout:4/529: symlink d11/d1e/d20/d31/d48/lc8 0 2024-02-15T18:34:57.100 INFO:tasks.workunit.client.0.smithi038.stdout:5/489: rename d2/d3/d12/d5e/l60 to d2/d3/l92 0 2024-02-15T18:34:57.100 INFO:tasks.workunit.client.0.smithi038.stdout:5/490: chown d2/d30/f55 26 1 2024-02-15T18:34:57.101 INFO:tasks.workunit.client.0.smithi038.stdout:5/491: truncate d2/d30/d52/f7b 1223368 0 2024-02-15T18:34:57.101 INFO:tasks.workunit.client.0.smithi038.stdout:5/492: dread - d2/d30/d57/f78 zero size 2024-02-15T18:34:57.101 INFO:tasks.workunit.client.0.smithi038.stdout:2/460: write dd/d12/d59/f21 [4500,81916] 0 2024-02-15T18:34:57.101 INFO:tasks.workunit.client.0.smithi038.stdout:5/493: read d2/d3/d5f/d71/d33/f66 [1382935,19184] 0 2024-02-15T18:34:57.103 INFO:tasks.workunit.client.0.smithi038.stdout:3/467: write d2/f1d [3720228,19944] 0 2024-02-15T18:34:57.103 INFO:tasks.workunit.client.0.smithi038.stdout:3/468: dread - d2/d1b/d3c/d3a/d55/f79 zero size 2024-02-15T18:34:57.107 INFO:tasks.workunit.client.0.smithi038.stdout:4/530: rename d11/d1e/d43/d5e/d62/f92 to d11/d1e/d20/d2f/fc9 0 2024-02-15T18:34:57.107 INFO:tasks.workunit.client.0.smithi038.stdout:4/531: write d11/d1e/d3e/f68 [809121,112040] 0 2024-02-15T18:34:57.107 INFO:tasks.workunit.client.0.smithi038.stdout:0/555: dwrite d4/d13/d2a/d35/d1e/d32/d48/f56 [0,4194304] 0 2024-02-15T18:34:57.110 INFO:tasks.workunit.client.0.smithi038.stdout:5/494: unlink d2/d3/d5f/d71/l7f 0 2024-02-15T18:34:57.111 INFO:tasks.workunit.client.0.smithi038.stdout:3/469: creat d2/d1b/d3c/d3a/f93 x:0 0 0 2024-02-15T18:34:57.111 INFO:tasks.workunit.client.0.smithi038.stdout:3/470: write d2/d1b/d4f/f5d [1691228,82155] 0 2024-02-15T18:34:57.113 INFO:tasks.workunit.client.0.smithi038.stdout:0/556: truncate d4/ff 1667691 0 2024-02-15T18:34:57.113 INFO:tasks.workunit.client.0.smithi038.stdout:0/557: write d4/d13/d2a/d35/d1e/d32/d3b/f8d [1017271,115318] 0 2024-02-15T18:34:57.113 INFO:tasks.workunit.client.0.smithi038.stdout:5/495: truncate d2/f39 4090911 0 2024-02-15T18:34:57.116 INFO:tasks.workunit.client.0.smithi038.stdout:3/471: rename d2/d1b/d3c/d3a/d55/f79 to d2/d1b/f94 0 2024-02-15T18:34:57.120 INFO:tasks.workunit.client.0.smithi038.stdout:0/558: mknod d4/d13/d2a/d35/d1e/d32/cb4 0 2024-02-15T18:34:57.120 INFO:tasks.workunit.client.0.smithi038.stdout:5/496: unlink d2/d30/c56 0 2024-02-15T18:34:57.120 INFO:tasks.workunit.client.0.smithi038.stdout:5/497: chown d2/d30/d52/c88 47230 1 2024-02-15T18:34:57.120 INFO:tasks.workunit.client.0.smithi038.stdout:2/461: dread fc [0,4194304] 0 2024-02-15T18:34:57.120 INFO:tasks.workunit.client.0.smithi038.stdout:2/462: chown dd/d12/d59/d2b/d7a 1 1 2024-02-15T18:34:57.120 INFO:tasks.workunit.client.0.smithi038.stdout:2/463: fdatasync dd/d12/d1d/f35 0 2024-02-15T18:34:57.120 INFO:tasks.workunit.client.0.smithi038.stdout:2/464: write dd/d12/d3a/d67/d4f/d5a/f7f [822389,3551] 0 2024-02-15T18:34:57.120 INFO:tasks.workunit.client.0.smithi038.stdout:3/472: creat d2/d1b/d8c/f95 x:0 0 0 2024-02-15T18:34:57.121 INFO:tasks.workunit.client.0.smithi038.stdout:1/516: dwrite f1 [0,4194304] 0 2024-02-15T18:34:57.121 INFO:tasks.workunit.client.0.smithi038.stdout:9/576: dwrite d1/da/d10/d13/f61 [0,4194304] 0 2024-02-15T18:34:57.121 INFO:tasks.workunit.client.0.smithi038.stdout:9/577: write d1/da/d10/d13/d3d/d43/f84 [699571,73567] 0 2024-02-15T18:34:57.122 INFO:tasks.workunit.client.0.smithi038.stdout:7/542: dwrite d0/db/d19/f4e [0,4194304] 0 2024-02-15T18:34:57.124 INFO:tasks.workunit.client.0.smithi038.stdout:1/517: dread d2/d10/d19/d22/d36/f4d [0,4194304] 0 2024-02-15T18:34:57.126 INFO:tasks.workunit.client.0.smithi038.stdout:0/559: unlink d4/d13/d2a/d35/d1e/d32/l6d 0 2024-02-15T18:34:57.137 INFO:tasks.workunit.client.0.smithi038.stdout:5/498: truncate d2/d3/d5f/d71/f37 2895626 0 2024-02-15T18:34:57.138 INFO:tasks.workunit.client.0.smithi038.stdout:5/499: write d2/d3/d5f/f82 [1418473,63126] 0 2024-02-15T18:34:57.139 INFO:tasks.workunit.client.0.smithi038.stdout:2/465: mknod dd/d12/d6b/c9b 0 2024-02-15T18:34:57.139 INFO:tasks.workunit.client.0.smithi038.stdout:5/500: dread - d2/d64/f8a zero size 2024-02-15T18:34:57.139 INFO:tasks.workunit.client.0.smithi038.stdout:3/473: creat d2/d1b/d69/f96 x:0 0 0 2024-02-15T18:34:57.139 INFO:tasks.workunit.client.0.smithi038.stdout:3/474: fdatasync d2/d1b/d3c/d3a/f93 0 2024-02-15T18:34:57.143 INFO:tasks.workunit.client.0.smithi038.stdout:9/578: rename d1/da/d36/d50 to d1/dc/d5a/d7f/dd1 0 2024-02-15T18:34:57.143 INFO:tasks.workunit.client.0.smithi038.stdout:9/579: dread - d1/dc/d2b/fa6 zero size 2024-02-15T18:34:57.144 INFO:tasks.workunit.client.0.smithi038.stdout:9/580: write d1/fa1 [1349303,52340] 0 2024-02-15T18:34:57.144 INFO:tasks.workunit.client.0.smithi038.stdout:9/581: chown d1/da/d10/d13/f86 99809 1 2024-02-15T18:34:57.144 INFO:tasks.workunit.client.0.smithi038.stdout:0/560: dwrite d4/d13/d19/f5c [0,4194304] 0 2024-02-15T18:34:57.145 INFO:tasks.workunit.client.0.smithi038.stdout:7/543: unlink d0/db/d59/d15/d29/d7e/l4a 0 2024-02-15T18:34:57.146 INFO:tasks.workunit.client.0.smithi038.stdout:7/544: dread d0/db/fb0 [0,4194304] 0 2024-02-15T18:34:57.146 INFO:tasks.workunit.client.0.smithi038.stdout:1/518: truncate d2/d10/f31 3031320 0 2024-02-15T18:34:57.161 INFO:tasks.workunit.client.0.smithi038.stdout:5/501: getdents d2/d64 0 2024-02-15T18:34:57.162 INFO:tasks.workunit.client.0.smithi038.stdout:2/466: creat dd/d12/d5d/d93/f9c x:0 0 0 2024-02-15T18:34:57.164 INFO:tasks.workunit.client.0.smithi038.stdout:3/475: rmdir d2/d1b/d3c/d3a/d55 39 2024-02-15T18:34:57.164 INFO:tasks.workunit.client.0.smithi038.stdout:3/476: chown d2/d7f 19091275 1 2024-02-15T18:34:57.166 INFO:tasks.workunit.client.0.smithi038.stdout:9/582: rename d1/da/d36/f37 to d1/dc/db2/fd2 0 2024-02-15T18:34:57.167 INFO:tasks.workunit.client.0.smithi038.stdout:0/561: rmdir d4/d13/d19 39 2024-02-15T18:34:57.168 INFO:tasks.workunit.client.0.smithi038.stdout:7/545: mkdir d0/d95/db8 0 2024-02-15T18:34:57.169 INFO:tasks.workunit.client.0.smithi038.stdout:5/502: truncate d2/d3/ff 2829596 0 2024-02-15T18:34:57.170 INFO:tasks.workunit.client.0.smithi038.stdout:2/467: mkdir dd/d12/d3a/d67/d4f/d5a/d9d 0 2024-02-15T18:34:57.172 INFO:tasks.workunit.client.0.smithi038.stdout:3/477: link d2/d1b/c5f d2/d1b/d3c/d41/d4e/d62/c97 0 2024-02-15T18:34:57.177 INFO:tasks.workunit.client.0.smithi038.stdout:9/583: rmdir d1/da 39 2024-02-15T18:34:57.179 INFO:tasks.workunit.client.0.smithi038.stdout:0/562: rename d4/d13/d53/ca4 to d4/cb5 0 2024-02-15T18:34:57.179 INFO:tasks.workunit.client.0.smithi038.stdout:0/563: readlink d4/l2e 0 2024-02-15T18:34:57.181 INFO:tasks.workunit.client.0.smithi038.stdout:7/546: link d0/db/d19/c24 d0/db/d59/d15/d4f/cb9 0 2024-02-15T18:34:57.182 INFO:tasks.workunit.client.0.smithi038.stdout:5/503: link d2/d64/f8a d2/d3/d5f/d71/d33/f93 0 2024-02-15T18:34:57.183 INFO:tasks.workunit.client.0.smithi038.stdout:2/468: creat dd/d12/d3a/d67/d4f/d5a/f9e x:0 0 0 2024-02-15T18:34:57.183 INFO:tasks.workunit.client.0.smithi038.stdout:3/478: symlink d2/d1e/l98 0 2024-02-15T18:34:57.185 INFO:tasks.workunit.client.0.smithi038.stdout:9/584: rename d1/da/d36/d8b/f96 to d1/da/d20/d58/d94/db9/dc6/fd3 0 2024-02-15T18:34:57.186 INFO:tasks.workunit.client.0.smithi038.stdout:6/507: sync 2024-02-15T18:34:57.186 INFO:tasks.workunit.client.0.smithi038.stdout:2/469: dread dd/f41 [0,4194304] 0 2024-02-15T18:34:57.186 INFO:tasks.workunit.client.0.smithi038.stdout:0/564: rename d4/d13/d2a/d35/d1e/d32/f6c to d4/d13/d2a/d35/d1e/d32/fb6 0 2024-02-15T18:34:57.186 INFO:tasks.workunit.client.0.smithi038.stdout:0/565: dread - d4/d13/d2a/d35/d1e/fac zero size 2024-02-15T18:34:57.187 INFO:tasks.workunit.client.0.smithi038.stdout:7/547: creat d0/d95/fba x:0 0 0 2024-02-15T18:34:57.188 INFO:tasks.workunit.client.0.smithi038.stdout:7/548: read - d0/db/d19/f8b zero size 2024-02-15T18:34:57.188 INFO:tasks.workunit.client.0.smithi038.stdout:3/479: creat d2/d1b/d8c/f99 x:0 0 0 2024-02-15T18:34:57.189 INFO:tasks.workunit.client.0.smithi038.stdout:6/508: rename d2/d4/f27 to d2/d56/f93 0 2024-02-15T18:34:57.191 INFO:tasks.workunit.client.0.smithi038.stdout:2/470: mknod dd/d12/d3a/d5e/c9f 0 2024-02-15T18:34:57.191 INFO:tasks.workunit.client.0.smithi038.stdout:2/471: write dd/d12/d1d/d3b/d3c/f48 [1612511,116197] 0 2024-02-15T18:34:57.192 INFO:tasks.workunit.client.0.smithi038.stdout:5/504: fsync d2/d64/f8a 0 2024-02-15T18:34:57.193 INFO:tasks.workunit.client.0.smithi038.stdout:5/505: chown d2/d30/f46 1 1 2024-02-15T18:34:57.195 INFO:tasks.workunit.client.0.smithi038.stdout:9/585: dread d1/da/d20/d58/d94/db9/dc6/f77 [0,4194304] 0 2024-02-15T18:34:57.200 INFO:tasks.workunit.client.0.smithi038.stdout:5/506: dread d2/fc [0,4194304] 0 2024-02-15T18:34:57.200 INFO:tasks.workunit.client.0.smithi038.stdout:5/507: fdatasync d2/d3/d5f/d71/f37 0 2024-02-15T18:34:57.201 INFO:tasks.workunit.client.0.smithi038.stdout:7/549: link d0/db/d59/fa4 d0/db/d59/fbb 0 2024-02-15T18:34:57.203 INFO:tasks.workunit.client.0.smithi038.stdout:7/550: dread d0/d1b/d30/d50/f74 [0,4194304] 0 2024-02-15T18:34:57.204 INFO:tasks.workunit.client.0.smithi038.stdout:2/472: creat dd/d12/d3a/d67/fa0 x:0 0 0 2024-02-15T18:34:57.204 INFO:tasks.workunit.client.0.smithi038.stdout:9/586: creat d1/da/d10/d13/d3d/d6c/fd4 x:0 0 0 2024-02-15T18:34:57.208 INFO:tasks.workunit.client.0.smithi038.stdout:9/587: link d1/dc/d5a/d7f/dd1/f55 d1/da/d36/d3f/d48/db8/fd5 0 2024-02-15T18:34:57.208 INFO:tasks.workunit.client.0.smithi038.stdout:9/588: readlink d1/dc/lad 0 2024-02-15T18:34:57.210 INFO:tasks.workunit.client.0.smithi038.stdout:9/589: rename d1/dc/d38/c39 to d1/da/d36/d3f/d48/db8/dcb/cd6 0 2024-02-15T18:34:57.212 INFO:tasks.workunit.client.0.smithi038.stdout:9/590: creat d1/dc/fd7 x:0 0 0 2024-02-15T18:34:57.214 INFO:tasks.workunit.client.0.smithi038.stdout:9/591: link d1/dc/d69/d7d/fc3 d1/da/d36/d8b/fd8 0 2024-02-15T18:34:57.229 INFO:tasks.workunit.client.0.smithi038.stdout:0/566: dwrite d4/ff [0,4194304] 0 2024-02-15T18:34:57.229 INFO:tasks.workunit.client.0.smithi038.stdout:0/567: chown d4/d13/d2a/d35/d1e/d32/c61 0 1 2024-02-15T18:34:57.230 INFO:tasks.workunit.client.0.smithi038.stdout:0/568: stat d4/d13/d19/f43 0 2024-02-15T18:34:57.231 INFO:tasks.workunit.client.0.smithi038.stdout:0/569: write d4/d13/d2a/d35/d1e/d32/d48/f56 [6471273,66596] 0 2024-02-15T18:34:57.231 INFO:tasks.workunit.client.0.smithi038.stdout:0/570: mknod d4/db3/cb7 0 2024-02-15T18:34:57.247 INFO:tasks.workunit.client.0.smithi038.stdout:6/509: dwrite d2/d10/f79 [0,4194304] 0 2024-02-15T18:34:57.252 INFO:tasks.workunit.client.0.smithi038.stdout:3/480: dwrite d2/d24/f2e [0,4194304] 0 2024-02-15T18:34:57.252 INFO:tasks.workunit.client.0.smithi038.stdout:3/481: chown d2/d1b/d3c/d3a/d55 16260 1 2024-02-15T18:34:57.252 INFO:tasks.workunit.client.0.smithi038.stdout:5/508: dwrite d2/f6d [0,4194304] 0 2024-02-15T18:34:57.252 INFO:tasks.workunit.client.0.smithi038.stdout:3/482: mknod d2/d1b/d3c/d41/c9a 0 2024-02-15T18:34:57.253 INFO:tasks.workunit.client.0.smithi038.stdout:0/571: write d4/d13/d2a/d35/f49 [2229686,100312] 0 2024-02-15T18:34:57.253 INFO:tasks.workunit.client.0.smithi038.stdout:0/572: chown d4/d13/f54 89 1 2024-02-15T18:34:57.253 INFO:tasks.workunit.client.0.smithi038.stdout:0/573: write d4/d13/f8e [313749,35138] 0 2024-02-15T18:34:57.255 INFO:tasks.workunit.client.0.smithi038.stdout:7/551: dwrite d0/f54 [0,4194304] 0 2024-02-15T18:34:57.255 INFO:tasks.workunit.client.0.smithi038.stdout:3/483: symlink d2/l9b 0 2024-02-15T18:34:57.256 INFO:tasks.workunit.client.0.smithi038.stdout:9/592: dwrite d1/dc/d2b/f3a [0,4194304] 0 2024-02-15T18:34:57.257 INFO:tasks.workunit.client.0.smithi038.stdout:9/593: chown d1/dc/fe 1527051165 1 2024-02-15T18:34:57.257 INFO:tasks.workunit.client.0.smithi038.stdout:2/473: dwrite dd/d12/f80 [0,4194304] 0 2024-02-15T18:34:57.257 INFO:tasks.workunit.client.0.smithi038.stdout:2/474: chown dd/d12/d3a/d67/d4f/f71 36 1 2024-02-15T18:34:57.258 INFO:tasks.workunit.client.0.smithi038.stdout:7/552: mkdir d0/d1b/d80/dbc 0 2024-02-15T18:34:57.258 INFO:tasks.workunit.client.0.smithi038.stdout:3/484: creat d2/d1b/d4f/d92/f9c x:0 0 0 2024-02-15T18:34:57.258 INFO:tasks.workunit.client.0.smithi038.stdout:3/485: write d2/d1b/d3c/d3a/f93 [800363,35674] 0 2024-02-15T18:34:57.260 INFO:tasks.workunit.client.0.smithi038.stdout:2/475: dread dd/d12/d3a/d67/d4f/f64 [0,4194304] 0 2024-02-15T18:34:57.263 INFO:tasks.workunit.client.0.smithi038.stdout:9/594: unlink d1/da/c8d 0 2024-02-15T18:34:57.264 INFO:tasks.workunit.client.0.smithi038.stdout:9/595: write d1/da/d10/d13/d3d/d4b/f72 [3633529,57526] 0 2024-02-15T18:34:57.266 INFO:tasks.workunit.client.0.smithi038.stdout:0/574: dread d4/d13/d2a/d35/d1e/d32/f76 [0,4194304] 0 2024-02-15T18:34:57.270 INFO:tasks.workunit.client.0.smithi038.stdout:6/510: dwrite d2/d4/d2c/f37 [0,4194304] 0 2024-02-15T18:34:57.271 INFO:tasks.workunit.client.0.smithi038.stdout:3/486: mknod d2/d1e/d7e/c9d 0 2024-02-15T18:34:57.272 INFO:tasks.workunit.client.0.smithi038.stdout:2/476: mkdir dd/d12/d59/da1 0 2024-02-15T18:34:57.273 INFO:tasks.workunit.client.0.smithi038.stdout:6/511: write d2/f5b [536003,38852] 0 2024-02-15T18:34:57.274 INFO:tasks.workunit.client.0.smithi038.stdout:8/666: sync 2024-02-15T18:34:57.277 INFO:tasks.workunit.client.0.smithi038.stdout:9/596: rename d1/dc/d5a/d7f/dd1/f73 to d1/da/d10/d13/d3d/d6c/fd9 0 2024-02-15T18:34:57.278 INFO:tasks.workunit.client.0.smithi038.stdout:1/519: sync 2024-02-15T18:34:57.278 INFO:tasks.workunit.client.0.smithi038.stdout:4/532: sync 2024-02-15T18:34:57.278 INFO:tasks.workunit.client.0.smithi038.stdout:4/533: chown d11/d1e/d43/d4a/fc7 509131 1 2024-02-15T18:34:57.281 INFO:tasks.workunit.client.0.smithi038.stdout:1/520: write d2/d10/d19/f1d [722801,80469] 0 2024-02-15T18:34:57.283 INFO:tasks.workunit.client.0.smithi038.stdout:0/575: creat d4/d13/d2a/d35/d6a/fb8 x:0 0 0 2024-02-15T18:34:57.283 INFO:tasks.workunit.client.0.smithi038.stdout:2/477: truncate dd/d12/d3a/d67/f30 1778866 0 2024-02-15T18:34:57.283 INFO:tasks.workunit.client.0.smithi038.stdout:9/597: mkdir d1/da/d36/d93/dda 0 2024-02-15T18:34:57.283 INFO:tasks.workunit.client.0.smithi038.stdout:6/512: symlink d2/d4/d19/d78/d7a/l94 0 2024-02-15T18:34:57.286 INFO:tasks.workunit.client.0.smithi038.stdout:4/534: write d11/d1e/d43/d4a/f59 [3329542,34330] 0 2024-02-15T18:34:57.286 INFO:tasks.workunit.client.0.smithi038.stdout:2/478: rmdir dd/d12/d5d/d93 39 2024-02-15T18:34:57.286 INFO:tasks.workunit.client.0.smithi038.stdout:0/576: getdents d4/d13/d2a 0 2024-02-15T18:34:57.290 INFO:tasks.workunit.client.0.smithi038.stdout:2/479: write dd/d12/d3a/d67/d4f/d5a/f60 [7955978,13175] 0 2024-02-15T18:34:57.290 INFO:tasks.workunit.client.0.smithi038.stdout:2/480: chown dd/d12/d59/d2b/d7a 0 1 2024-02-15T18:34:57.290 INFO:tasks.workunit.client.0.smithi038.stdout:2/481: chown dd/d12/d1d/d3b/d3c/l55 23 1 2024-02-15T18:34:57.291 INFO:tasks.workunit.client.0.smithi038.stdout:6/513: link d2/d4/d2c/d39/c67 d2/c95 0 2024-02-15T18:34:57.291 INFO:tasks.workunit.client.0.smithi038.stdout:6/514: chown d2/d56/l84 3217 1 2024-02-15T18:34:57.293 INFO:tasks.workunit.client.0.smithi038.stdout:6/515: read d2/f1a [2063031,92402] 0 2024-02-15T18:34:57.293 INFO:tasks.workunit.client.0.smithi038.stdout:3/487: dwrite d2/fa [0,4194304] 0 2024-02-15T18:34:57.293 INFO:tasks.workunit.client.0.smithi038.stdout:3/488: stat d2/d1b/d3c/d41 0 2024-02-15T18:34:57.294 INFO:tasks.workunit.client.0.smithi038.stdout:3/489: dread - d2/d1b/d3c/f6b zero size 2024-02-15T18:34:57.295 INFO:tasks.workunit.client.0.smithi038.stdout:2/482: truncate dd/d12/d3a/d67/f86 419789 0 2024-02-15T18:34:57.297 INFO:tasks.workunit.client.0.smithi038.stdout:2/483: symlink dd/d12/d3a/d67/d4f/d5a/la2 0 2024-02-15T18:34:57.298 INFO:tasks.workunit.client.0.smithi038.stdout:4/535: dread d11/d1e/d20/d31/f5b [0,4194304] 0 2024-02-15T18:34:57.299 INFO:tasks.workunit.client.0.smithi038.stdout:3/490: write d2/d1b/d3c/d3a/d55/f80 [1538226,16986] 0 2024-02-15T18:34:57.299 INFO:tasks.workunit.client.0.smithi038.stdout:4/536: mkdir d11/d1e/d20/d2b/d4d/d67/d73/d8b/dca 0 2024-02-15T18:34:57.301 INFO:tasks.workunit.client.0.smithi038.stdout:4/537: unlink d11/d1e/f29 0 2024-02-15T18:34:57.303 INFO:tasks.workunit.client.0.smithi038.stdout:4/538: creat d11/d1e/d20/d31/d8d/d99/db5/fcb x:0 0 0 2024-02-15T18:34:57.304 INFO:tasks.workunit.client.0.smithi038.stdout:2/484: dread dd/d12/d5d/d54/f97 [0,4194304] 0 2024-02-15T18:34:57.305 INFO:tasks.workunit.client.0.smithi038.stdout:4/539: creat d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/fcc x:0 0 0 2024-02-15T18:34:57.305 INFO:tasks.workunit.client.0.smithi038.stdout:4/540: chown d11/d1e/d20/d31/d48/dac/dbe/d40 21714 1 2024-02-15T18:34:57.305 INFO:tasks.workunit.client.0.smithi038.stdout:8/667: dwrite d1/d3a/d58/f60 [0,4194304] 0 2024-02-15T18:34:57.306 INFO:tasks.workunit.client.0.smithi038.stdout:7/553: dwrite d0/db/f20 [4194304,4194304] 0 2024-02-15T18:34:57.309 INFO:tasks.workunit.client.0.smithi038.stdout:7/554: write d0/f21 [3046366,111314] 0 2024-02-15T18:34:57.312 INFO:tasks.workunit.client.0.smithi038.stdout:8/668: rename d1/d4/d5/d77/d79/fbe to d1/d7/d26/ff4 0 2024-02-15T18:34:57.312 INFO:tasks.workunit.client.0.smithi038.stdout:4/541: creat d11/d1e/d20/d2b/d35/fcd x:0 0 0 2024-02-15T18:34:57.312 INFO:tasks.workunit.client.0.smithi038.stdout:4/542: readlink d11/d1e/d20/d2b/d4d/l9e 0 2024-02-15T18:34:57.312 INFO:tasks.workunit.client.0.smithi038.stdout:8/669: chown d1/d4/d10/d91/fa4 1557701 1 2024-02-15T18:34:57.312 INFO:tasks.workunit.client.0.smithi038.stdout:8/670: chown d1/d7/d15/c2b 1 1 2024-02-15T18:34:57.312 INFO:tasks.workunit.client.0.smithi038.stdout:7/555: symlink d0/d1b/d30/d5d/d70/lbd 0 2024-02-15T18:34:57.313 INFO:tasks.workunit.client.0.smithi038.stdout:7/556: chown d0/db/d59/d15/f23 0 1 2024-02-15T18:34:57.314 INFO:tasks.workunit.client.0.smithi038.stdout:9/598: dwrite d1/da/d20/f65 [0,4194304] 0 2024-02-15T18:34:57.315 INFO:tasks.workunit.client.0.smithi038.stdout:7/557: link d0/db/d19/d2d/l22 d0/db/d59/lbe 0 2024-02-15T18:34:57.316 INFO:tasks.workunit.client.0.smithi038.stdout:8/671: dread d1/d3a/f6c [0,4194304] 0 2024-02-15T18:34:57.316 INFO:tasks.workunit.client.0.smithi038.stdout:0/577: dwrite d4/d13/d2a/d35/d1e/d32/d48/f56 [4194304,4194304] 0 2024-02-15T18:34:57.317 INFO:tasks.workunit.client.0.smithi038.stdout:0/578: fsync d4/d13/d2a/d35/d6a/fb8 0 2024-02-15T18:34:57.317 INFO:tasks.workunit.client.0.smithi038.stdout:3/491: dread d2/fa [0,4194304] 0 2024-02-15T18:34:57.317 INFO:tasks.workunit.client.0.smithi038.stdout:3/492: chown d2/d1b/d3c/d3a/d55/l85 173 1 2024-02-15T18:34:57.317 INFO:tasks.workunit.client.0.smithi038.stdout:1/521: dwrite d2/d2e/d8c/fa0 [0,4194304] 0 2024-02-15T18:34:57.317 INFO:tasks.workunit.client.0.smithi038.stdout:1/522: readlink d2/d2e/d8c/l99 0 2024-02-15T18:34:57.318 INFO:tasks.workunit.client.0.smithi038.stdout:9/599: mknod d1/da/d36/d3f/cdb 0 2024-02-15T18:34:57.320 INFO:tasks.workunit.client.0.smithi038.stdout:7/558: rename d0/d66/la0 to d0/d1b/d30/d5d/d61/d90/lbf 0 2024-02-15T18:34:57.321 INFO:tasks.workunit.client.0.smithi038.stdout:8/672: rmdir d1 39 2024-02-15T18:34:57.325 INFO:tasks.workunit.client.0.smithi038.stdout:0/579: link d4/d13/d2a/l70 d4/d13/d2a/d35/lb9 0 2024-02-15T18:34:57.326 INFO:tasks.workunit.client.0.smithi038.stdout:1/523: getdents d2/d10/d19/d22/d36/d4f 0 2024-02-15T18:34:57.327 INFO:tasks.workunit.client.0.smithi038.stdout:9/600: rmdir d1/da/d20/d58/d94 39 2024-02-15T18:34:57.327 INFO:tasks.workunit.client.0.smithi038.stdout:9/601: write d1/da/d20/d58/d94/db9/dc6/d49/d95/fce [28544,60854] 0 2024-02-15T18:34:57.328 INFO:tasks.workunit.client.0.smithi038.stdout:1/524: write d2/d10/d19/d22/f7a [3380412,82892] 0 2024-02-15T18:34:57.329 INFO:tasks.workunit.client.0.smithi038.stdout:3/493: creat d2/d1b/f9e x:0 0 0 2024-02-15T18:34:57.329 INFO:tasks.workunit.client.0.smithi038.stdout:2/485: dwrite dd/d12/d1d/d3b/d3c/d4a/f6d [0,4194304] 0 2024-02-15T18:34:57.330 INFO:tasks.workunit.client.0.smithi038.stdout:9/602: creat d1/dc/db2/fdc x:0 0 0 2024-02-15T18:34:57.330 INFO:tasks.workunit.client.0.smithi038.stdout:9/603: write d1/da/d36/d3f/f75 [792905,92408] 0 2024-02-15T18:34:57.330 INFO:tasks.workunit.client.0.smithi038.stdout:9/604: truncate d1/f14 4597996 0 2024-02-15T18:34:57.331 INFO:tasks.workunit.client.0.smithi038.stdout:9/605: write d1/dc/d69/d7d/fc3 [598471,117107] 0 2024-02-15T18:34:57.331 INFO:tasks.workunit.client.0.smithi038.stdout:9/606: dread - d1/da/d10/d13/d3d/fa3 zero size 2024-02-15T18:34:57.335 INFO:tasks.workunit.client.0.smithi038.stdout:2/486: creat dd/d12/d6b/fa3 x:0 0 0 2024-02-15T18:34:57.335 INFO:tasks.workunit.client.0.smithi038.stdout:2/487: fdatasync fc 0 2024-02-15T18:34:57.336 INFO:tasks.workunit.client.0.smithi038.stdout:0/580: write d4/d13/f39 [1128483,48348] 0 2024-02-15T18:34:57.336 INFO:tasks.workunit.client.0.smithi038.stdout:0/581: write d4/d13/d2a/d35/d1e/f4d [318306,34412] 0 2024-02-15T18:34:57.336 INFO:tasks.workunit.client.0.smithi038.stdout:0/582: chown d4/d13/d19/l9d 196883768 1 2024-02-15T18:34:57.336 INFO:tasks.workunit.client.0.smithi038.stdout:9/607: truncate d1/da/d10/d13/d3d/d43/f70 2810588 0 2024-02-15T18:34:57.336 INFO:tasks.workunit.client.0.smithi038.stdout:9/608: write d1/dc/fe [4598529,46463] 0 2024-02-15T18:34:57.340 INFO:tasks.workunit.client.0.smithi038.stdout:0/583: creat d4/d13/d2a/d35/d1e/d32/d79/d7d/fba x:0 0 0 2024-02-15T18:34:57.342 INFO:tasks.workunit.client.0.smithi038.stdout:9/609: link d1/da/d20/d58/d76/f8c d1/da/d10/d13/d3d/d43/fdd 0 2024-02-15T18:34:57.343 INFO:tasks.workunit.client.0.smithi038.stdout:4/543: dwrite d11/d1e/d20/d97/f9f [0,4194304] 0 2024-02-15T18:34:57.343 INFO:tasks.workunit.client.0.smithi038.stdout:9/610: creat d1/dc/d5a/fde x:0 0 0 2024-02-15T18:34:57.345 INFO:tasks.workunit.client.0.smithi038.stdout:6/516: dwrite d2/d56/f93 [0,4194304] 0 2024-02-15T18:34:57.345 INFO:tasks.workunit.client.0.smithi038.stdout:7/559: dwrite d0/db/d59/fa4 [0,4194304] 0 2024-02-15T18:34:57.346 INFO:tasks.workunit.client.0.smithi038.stdout:7/560: write d0/d1b/d30/f43 [275625,119603] 0 2024-02-15T18:34:57.354 INFO:tasks.workunit.client.0.smithi038.stdout:3/494: dwrite d2/d25/f26 [4194304,4194304] 0 2024-02-15T18:34:57.354 INFO:tasks.workunit.client.0.smithi038.stdout:3/495: dread - d2/d1e/f89 zero size 2024-02-15T18:34:57.367 INFO:tasks.workunit.client.0.smithi038.stdout:9/611: rename d1/da/d36/d93/dda to d1/dc/d4f/ddf 0 2024-02-15T18:34:57.367 INFO:tasks.workunit.client.0.smithi038.stdout:9/612: read - d1/da/d10/d13/d3d/d43/d4d/f7b zero size 2024-02-15T18:34:57.370 INFO:tasks.workunit.client.0.smithi038.stdout:7/561: mkdir d0/d1b/d30/d5d/d61/d90/dc0 0 2024-02-15T18:34:57.374 INFO:tasks.workunit.client.0.smithi038.stdout:1/525: dwrite d2/d10/d19/d22/f7a [0,4194304] 0 2024-02-15T18:34:57.375 INFO:tasks.workunit.client.0.smithi038.stdout:6/517: dwrite d2/f46 [4194304,4194304] 0 2024-02-15T18:34:57.376 INFO:tasks.workunit.client.0.smithi038.stdout:9/613: dread d1/da/f17 [0,4194304] 0 2024-02-15T18:34:57.377 INFO:tasks.workunit.client.0.smithi038.stdout:9/614: readlink d1/dc/l29 0 2024-02-15T18:34:57.377 INFO:tasks.workunit.client.0.smithi038.stdout:6/518: creat d2/d10/f96 x:0 0 0 2024-02-15T18:34:57.378 INFO:tasks.workunit.client.0.smithi038.stdout:4/544: dwrite d11/d1e/d20/d31/f5b [0,4194304] 0 2024-02-15T18:34:57.378 INFO:tasks.workunit.client.0.smithi038.stdout:7/562: dread d0/d66/f35 [0,4194304] 0 2024-02-15T18:34:57.379 INFO:tasks.workunit.client.0.smithi038.stdout:9/615: link d1/dc/d5a/d7f/dd1/cc8 d1/da/d36/d93/d9a/ce0 0 2024-02-15T18:34:57.380 INFO:tasks.workunit.client.0.smithi038.stdout:6/519: dread d2/f9 [0,4194304] 0 2024-02-15T18:34:57.381 INFO:tasks.workunit.client.0.smithi038.stdout:5/509: sync 2024-02-15T18:34:57.385 INFO:tasks.workunit.client.0.smithi038.stdout:7/563: mknod d0/d1b/cc1 0 2024-02-15T18:34:57.385 INFO:tasks.workunit.client.0.smithi038.stdout:9/616: getdents d1/da/d36/d3f 0 2024-02-15T18:34:57.386 INFO:tasks.workunit.client.0.smithi038.stdout:1/526: dread d2/f1e [0,4194304] 0 2024-02-15T18:34:57.386 INFO:tasks.workunit.client.0.smithi038.stdout:4/545: write d11/d1e/d20/d2b/d35/f89 [4183883,126385] 0 2024-02-15T18:34:57.386 INFO:tasks.workunit.client.0.smithi038.stdout:6/520: rename d2/d4/d2c/d39/c73 to d2/d4/d19/c97 0 2024-02-15T18:34:57.387 INFO:tasks.workunit.client.0.smithi038.stdout:7/564: mkdir d0/db/d59/d15/d29/d8d/dc2 0 2024-02-15T18:34:57.387 INFO:tasks.workunit.client.0.smithi038.stdout:7/565: readlink d0/d1b/d30/laf 0 2024-02-15T18:34:57.388 INFO:tasks.workunit.client.0.smithi038.stdout:9/617: creat d1/da/d36/d3f/d99/fe1 x:0 0 0 2024-02-15T18:34:57.391 INFO:tasks.workunit.client.0.smithi038.stdout:9/618: dread d1/da/d36/d8b/fd8 [0,4194304] 0 2024-02-15T18:34:57.392 INFO:tasks.workunit.client.0.smithi038.stdout:4/546: rename d11/d1e/d43/d4a/fc7 to d11/d1e/d3e/fce 0 2024-02-15T18:34:57.397 INFO:tasks.workunit.client.0.smithi038.stdout:5/510: dwrite d2/d30/d52/f7b [0,4194304] 0 2024-02-15T18:34:57.398 INFO:tasks.workunit.client.0.smithi038.stdout:6/521: rename d2/d4/d19/f47 to d2/d4/d6c/f98 0 2024-02-15T18:34:57.398 INFO:tasks.workunit.client.0.smithi038.stdout:9/619: symlink d1/dc/d5a/le2 0 2024-02-15T18:34:57.400 INFO:tasks.workunit.client.0.smithi038.stdout:9/620: dread d1/da/d36/d3f/d48/fb3 [0,4194304] 0 2024-02-15T18:34:57.401 INFO:tasks.workunit.client.0.smithi038.stdout:5/511: rmdir d2/d3/d12/d5e/d86 39 2024-02-15T18:34:57.402 INFO:tasks.workunit.client.0.smithi038.stdout:6/522: mkdir d2/d56/d5f/d99 0 2024-02-15T18:34:57.404 INFO:tasks.workunit.client.0.smithi038.stdout:9/621: mkdir d1/da/d20/d58/d94/db9/de3 0 2024-02-15T18:34:57.405 INFO:tasks.workunit.client.0.smithi038.stdout:5/512: symlink d2/d3/d12/l94 0 2024-02-15T18:34:57.406 INFO:tasks.workunit.client.0.smithi038.stdout:5/513: stat d2/d75/f85 0 2024-02-15T18:34:57.407 INFO:tasks.workunit.client.0.smithi038.stdout:6/523: symlink d2/d4/d2c/l9a 0 2024-02-15T18:34:57.413 INFO:tasks.workunit.client.0.smithi038.stdout:9/622: dread d1/dc/d2b/f3a [0,4194304] 0 2024-02-15T18:34:57.415 INFO:tasks.workunit.client.0.smithi038.stdout:6/524: symlink d2/d4/d2c/d39/l9b 0 2024-02-15T18:34:57.418 INFO:tasks.workunit.client.0.smithi038.stdout:9/623: rename d1/da/d20/d58/d94/db9/dc6/d49/d95 to d1/da/d20/d58/d94/db9/dc6/d49/de4 0 2024-02-15T18:34:57.418 INFO:tasks.workunit.client.0.smithi038.stdout:6/525: creat d2/d56/f9c x:0 0 0 2024-02-15T18:34:57.418 INFO:tasks.workunit.client.0.smithi038.stdout:6/526: chown d2/f3 7127188 1 2024-02-15T18:34:57.419 INFO:tasks.workunit.client.0.smithi038.stdout:9/624: getdents d1/da/d36/d3f/d48/db8/dcb 0 2024-02-15T18:34:57.419 INFO:tasks.workunit.client.0.smithi038.stdout:9/625: truncate d1/da/d36/d93/fa4 413707 0 2024-02-15T18:34:57.419 INFO:tasks.workunit.client.0.smithi038.stdout:6/527: getdents d2/d4/d19/d55 0 2024-02-15T18:34:57.420 INFO:tasks.workunit.client.0.smithi038.stdout:9/626: mknod d1/da/d10/d19/ce5 0 2024-02-15T18:34:57.421 INFO:tasks.workunit.client.0.smithi038.stdout:9/627: creat d1/da/d20/d58/d94/db9/de3/fe6 x:0 0 0 2024-02-15T18:34:57.422 INFO:tasks.workunit.client.0.smithi038.stdout:9/628: link d1/da/f17 d1/dc/db2/fe7 0 2024-02-15T18:34:57.423 INFO:tasks.workunit.client.0.smithi038.stdout:9/629: write d1/dc/db2/fd2 [3682504,16705] 0 2024-02-15T18:34:57.423 INFO:tasks.workunit.client.0.smithi038.stdout:9/630: chown d1/da/d20/d58/d76/fb5 13074 1 2024-02-15T18:34:57.424 INFO:tasks.workunit.client.0.smithi038.stdout:9/631: write d1/dc/d4f/f46 [2423816,42434] 0 2024-02-15T18:34:57.424 INFO:tasks.workunit.client.0.smithi038.stdout:9/632: fdatasync d1/dc/d5a/d7f/dd1/f55 0 2024-02-15T18:34:57.425 INFO:tasks.workunit.client.0.smithi038.stdout:9/633: mkdir d1/da/d20/d58/d94/de8 0 2024-02-15T18:34:57.427 INFO:tasks.workunit.client.0.smithi038.stdout:9/634: mknod d1/dc/d5a/ce9 0 2024-02-15T18:34:57.428 INFO:tasks.workunit.client.0.smithi038.stdout:9/635: symlink d1/da/d36/d3f/lea 0 2024-02-15T18:34:57.428 INFO:tasks.workunit.client.0.smithi038.stdout:1/527: dwrite d2/d10/d19/d5f/fa8 [0,4194304] 0 2024-02-15T18:34:57.429 INFO:tasks.workunit.client.0.smithi038.stdout:4/547: dwrite d11/d1e/d20/d2f/d71/f98 [4194304,4194304] 0 2024-02-15T18:34:57.431 INFO:tasks.workunit.client.0.smithi038.stdout:7/566: dwrite d0/db/fc [4194304,4194304] 0 2024-02-15T18:34:57.431 INFO:tasks.workunit.client.0.smithi038.stdout:5/514: dwrite d2/d48/f59 [4194304,4194304] 0 2024-02-15T18:34:57.431 INFO:tasks.workunit.client.0.smithi038.stdout:5/515: write d2/f7a [1081946,46186] 0 2024-02-15T18:34:57.431 INFO:tasks.workunit.client.0.smithi038.stdout:5/516: chown d2/d3/l92 113 1 2024-02-15T18:34:57.432 INFO:tasks.workunit.client.0.smithi038.stdout:1/528: truncate d2/f1e 3329772 0 2024-02-15T18:34:57.434 INFO:tasks.workunit.client.0.smithi038.stdout:5/517: write d2/f3f [39230,59611] 0 2024-02-15T18:34:57.435 INFO:tasks.workunit.client.0.smithi038.stdout:4/548: symlink d11/d1e/d20/d2b/lcf 0 2024-02-15T18:34:57.437 INFO:tasks.workunit.client.0.smithi038.stdout:7/567: truncate d0/d1b/d30/d50/f74 998464 0 2024-02-15T18:34:57.442 INFO:tasks.workunit.client.0.smithi038.stdout:9/636: rmdir d1/dc 39 2024-02-15T18:34:57.445 INFO:tasks.workunit.client.0.smithi038.stdout:0/584: sync 2024-02-15T18:34:57.446 INFO:tasks.workunit.client.0.smithi038.stdout:8/673: sync 2024-02-15T18:34:57.446 INFO:tasks.workunit.client.0.smithi038.stdout:2/488: sync 2024-02-15T18:34:57.446 INFO:tasks.workunit.client.0.smithi038.stdout:3/496: sync 2024-02-15T18:34:57.450 INFO:tasks.workunit.client.0.smithi038.stdout:1/529: unlink d2/d28/d3c/c5a 0 2024-02-15T18:34:57.450 INFO:tasks.workunit.client.0.smithi038.stdout:8/674: write d1/d4/d10/f1e [1265816,68143] 0 2024-02-15T18:34:57.450 INFO:tasks.workunit.client.0.smithi038.stdout:8/675: chown d1/d4/d5/d31/d65/d80/c8b 3 1 2024-02-15T18:34:57.451 INFO:tasks.workunit.client.0.smithi038.stdout:5/518: symlink d2/d3/d12/d5e/l95 0 2024-02-15T18:34:57.451 INFO:tasks.workunit.client.0.smithi038.stdout:5/519: write d2/d30/d52/f8b [726897,105038] 0 2024-02-15T18:34:57.451 INFO:tasks.workunit.client.0.smithi038.stdout:4/549: symlink d11/ld0 0 2024-02-15T18:34:57.451 INFO:tasks.workunit.client.0.smithi038.stdout:4/550: fdatasync d11/d1e/d20/d2f/d71/f98 0 2024-02-15T18:34:57.453 INFO:tasks.workunit.client.0.smithi038.stdout:7/568: symlink d0/d95/db8/lc3 0 2024-02-15T18:34:57.453 INFO:tasks.workunit.client.0.smithi038.stdout:7/569: chown d0/d1b/d30/d50/l6b 62 1 2024-02-15T18:34:57.453 INFO:tasks.workunit.client.0.smithi038.stdout:5/520: write d2/d3/f2f [4721066,53102] 0 2024-02-15T18:34:57.454 INFO:tasks.workunit.client.0.smithi038.stdout:0/585: mknod d4/cbb 0 2024-02-15T18:34:57.455 INFO:tasks.workunit.client.0.smithi038.stdout:2/489: creat dd/d12/d5d/d54/fa4 x:0 0 0 2024-02-15T18:34:57.455 INFO:tasks.workunit.client.0.smithi038.stdout:2/490: dread - dd/d12/d5d/d54/fa4 zero size 2024-02-15T18:34:57.456 INFO:tasks.workunit.client.0.smithi038.stdout:8/676: mkdir d1/d4/d5/d6/d6d/df5 0 2024-02-15T18:34:57.456 INFO:tasks.workunit.client.0.smithi038.stdout:8/677: stat d1/d4/d5/d6/d6d/lb4 0 2024-02-15T18:34:57.456 INFO:tasks.workunit.client.0.smithi038.stdout:2/491: dread - dd/d12/d3a/d67/d4f/f71 zero size 2024-02-15T18:34:57.456 INFO:tasks.workunit.client.0.smithi038.stdout:4/551: rmdir d11/d1e/d20/d2f 39 2024-02-15T18:34:57.458 INFO:tasks.workunit.client.0.smithi038.stdout:7/570: symlink d0/db/d59/d15/d29/d8d/lc4 0 2024-02-15T18:34:57.458 INFO:tasks.workunit.client.0.smithi038.stdout:0/586: stat d4/l47 0 2024-02-15T18:34:57.462 INFO:tasks.workunit.client.0.smithi038.stdout:2/492: truncate dd/d12/d3a/d67/d4f/d5a/f60 4862213 0 2024-02-15T18:34:57.462 INFO:tasks.workunit.client.0.smithi038.stdout:7/571: creat d0/db/d59/d97/fc5 x:0 0 0 2024-02-15T18:34:57.466 INFO:tasks.workunit.client.0.smithi038.stdout:2/493: unlink fc 0 2024-02-15T18:34:57.466 INFO:tasks.workunit.client.0.smithi038.stdout:7/572: link d0/db/d59/d15/f23 d0/db/d59/d15/d29/fc6 0 2024-02-15T18:34:57.467 INFO:tasks.workunit.client.0.smithi038.stdout:5/521: dread d2/d3/d5f/d71/d33/f66 [0,4194304] 0 2024-02-15T18:34:57.468 INFO:tasks.workunit.client.0.smithi038.stdout:2/494: mkdir dd/d12/d6b/da5 0 2024-02-15T18:34:57.470 INFO:tasks.workunit.client.0.smithi038.stdout:7/573: symlink d0/d1b/d30/d5d/d61/d90/lc7 0 2024-02-15T18:34:57.470 INFO:tasks.workunit.client.0.smithi038.stdout:7/574: dread - d0/d1b/d30/d5d/d70/f88 zero size 2024-02-15T18:34:57.471 INFO:tasks.workunit.client.0.smithi038.stdout:5/522: unlink d2/f7a 0 2024-02-15T18:34:57.475 INFO:tasks.workunit.client.0.smithi038.stdout:3/497: dwrite d2/d24/f49 [0,4194304] 0 2024-02-15T18:34:57.475 INFO:tasks.workunit.client.0.smithi038.stdout:1/530: dwrite d2/f9d [0,4194304] 0 2024-02-15T18:34:57.483 INFO:tasks.workunit.client.0.smithi038.stdout:9/637: dwrite d1/dc/f28 [0,4194304] 0 2024-02-15T18:34:57.483 INFO:tasks.workunit.client.0.smithi038.stdout:2/495: dread dd/d12/f80 [0,4194304] 0 2024-02-15T18:34:57.486 INFO:tasks.workunit.client.0.smithi038.stdout:9/638: mkdir d1/da/d36/d93/deb 0 2024-02-15T18:34:57.486 INFO:tasks.workunit.client.0.smithi038.stdout:9/639: fdatasync d1/da/d36/d3f/f75 0 2024-02-15T18:34:57.486 INFO:tasks.workunit.client.0.smithi038.stdout:2/496: dread dd/f41 [0,4194304] 0 2024-02-15T18:34:57.487 INFO:tasks.workunit.client.0.smithi038.stdout:2/497: mknod dd/d12/d1d/d3b/d3c/d73/ca6 0 2024-02-15T18:34:57.488 INFO:tasks.workunit.client.0.smithi038.stdout:5/523: dread d2/f39 [0,4194304] 0 2024-02-15T18:34:57.488 INFO:tasks.workunit.client.0.smithi038.stdout:5/524: chown d2/d30/l32 90876 1 2024-02-15T18:34:57.489 INFO:tasks.workunit.client.0.smithi038.stdout:2/498: creat dd/d12/d3a/d67/d4f/fa7 x:0 0 0 2024-02-15T18:34:57.493 INFO:tasks.workunit.client.0.smithi038.stdout:4/552: dwrite d11/d1e/d20/d2f/fc9 [0,4194304] 0 2024-02-15T18:34:57.494 INFO:tasks.workunit.client.0.smithi038.stdout:4/553: readlink d11/d1e/d43/d5e/la0 0 2024-02-15T18:34:57.494 INFO:tasks.workunit.client.0.smithi038.stdout:2/499: getdents dd/d12/d3a/d67 0 2024-02-15T18:34:57.494 INFO:tasks.workunit.client.0.smithi038.stdout:0/587: dwrite d4/d13/d2a/d35/d1e/d32/d48/d97/faf [4194304,4194304] 0 2024-02-15T18:34:57.494 INFO:tasks.workunit.client.0.smithi038.stdout:0/588: write d4/d13/d2a/d35/d1e/d32/d48/d97/f9b [988480,16914] 0 2024-02-15T18:34:57.495 INFO:tasks.workunit.client.0.smithi038.stdout:8/678: truncate d1/d4/d10/f1e 3455879 0 2024-02-15T18:34:57.496 INFO:tasks.workunit.client.0.smithi038.stdout:1/531: dread d2/d10/d19/d22/d36/f44 [0,4194304] 0 2024-02-15T18:34:57.496 INFO:tasks.workunit.client.0.smithi038.stdout:1/532: write d2/d39/f89 [748541,118675] 0 2024-02-15T18:34:57.496 INFO:tasks.workunit.client.0.smithi038.stdout:7/575: dwrite d0/db/d59/d15/d29/f91 [0,4194304] 0 2024-02-15T18:34:57.497 INFO:tasks.workunit.client.0.smithi038.stdout:3/498: dwrite d2/f17 [0,4194304] 0 2024-02-15T18:34:57.499 INFO:tasks.workunit.client.0.smithi038.stdout:2/500: write dd/d12/d3a/d67/f8f [1943971,74740] 0 2024-02-15T18:34:57.499 INFO:tasks.workunit.client.0.smithi038.stdout:2/501: fsync dd/d12/d1d/f31 0 2024-02-15T18:34:57.499 INFO:tasks.workunit.client.0.smithi038.stdout:2/502: dread - dd/d12/d3a/d67/fa0 zero size 2024-02-15T18:34:57.499 INFO:tasks.workunit.client.0.smithi038.stdout:2/503: truncate dd/d12/d3a/d67/d4f/f64 1008002 0 2024-02-15T18:34:57.500 INFO:tasks.workunit.client.0.smithi038.stdout:4/554: unlink d11/d1e/d20/d2b/d4d/c96 0 2024-02-15T18:34:57.502 INFO:tasks.workunit.client.0.smithi038.stdout:8/679: mkdir d1/d3a/df6 0 2024-02-15T18:34:57.502 INFO:tasks.workunit.client.0.smithi038.stdout:8/680: chown d1/l74 478382440 1 2024-02-15T18:34:57.502 INFO:tasks.workunit.client.0.smithi038.stdout:0/589: dread d4/d13/d19/f82 [0,4194304] 0 2024-02-15T18:34:57.505 INFO:tasks.workunit.client.0.smithi038.stdout:1/533: rename d2/d10/d62/c77 to d2/d28/d3c/cb2 0 2024-02-15T18:34:57.505 INFO:tasks.workunit.client.0.smithi038.stdout:7/576: truncate d0/db/fb1 134565 0 2024-02-15T18:34:57.505 INFO:tasks.workunit.client.0.smithi038.stdout:9/640: dwrite d1/da/d20/f6d [0,4194304] 0 2024-02-15T18:34:57.507 INFO:tasks.workunit.client.0.smithi038.stdout:9/641: write d1/da/d10/d13/f61 [312890,39890] 0 2024-02-15T18:34:57.507 INFO:tasks.workunit.client.0.smithi038.stdout:9/642: read - d1/da/d10/d13/d3d/d43/d4d/d8a/f8e zero size 2024-02-15T18:34:57.509 INFO:tasks.workunit.client.0.smithi038.stdout:2/504: dread dd/d12/d1d/d3b/d3c/f48 [0,4194304] 0 2024-02-15T18:34:57.514 INFO:tasks.workunit.client.0.smithi038.stdout:5/525: dwrite d2/d30/d52/f8b [0,4194304] 0 2024-02-15T18:34:57.517 INFO:tasks.workunit.client.0.smithi038.stdout:2/505: dread dd/f77 [0,4194304] 0 2024-02-15T18:34:57.519 INFO:tasks.workunit.client.0.smithi038.stdout:3/499: truncate d2/f51 1974180 0 2024-02-15T18:34:57.519 INFO:tasks.workunit.client.0.smithi038.stdout:4/555: creat d11/d1e/d20/d2b/d4d/d67/dbf/fd1 x:0 0 0 2024-02-15T18:34:57.519 INFO:tasks.workunit.client.0.smithi038.stdout:3/500: write d2/d1b/f94 [785007,119327] 0 2024-02-15T18:34:57.519 INFO:tasks.workunit.client.0.smithi038.stdout:4/556: write d11/d1e/d20/d31/d8d/d99/db5/fcb [73458,40190] 0 2024-02-15T18:34:57.519 INFO:tasks.workunit.client.0.smithi038.stdout:4/557: chown d11/d1e/d20/d2b/d4d/d67/dbf/fd1 91946 1 2024-02-15T18:34:57.520 INFO:tasks.workunit.client.0.smithi038.stdout:0/590: mkdir d4/d13/d2a/d35/d1e/d32/d48/d97/dbc 0 2024-02-15T18:34:57.520 INFO:tasks.workunit.client.0.smithi038.stdout:0/591: chown d4/d13/d2a/d35/c28 222204 1 2024-02-15T18:34:57.520 INFO:tasks.workunit.client.0.smithi038.stdout:0/592: readlink d4/d13/l2f 0 2024-02-15T18:34:57.520 INFO:tasks.workunit.client.0.smithi038.stdout:1/534: unlink d2/d28/f50 0 2024-02-15T18:34:57.521 INFO:tasks.workunit.client.0.smithi038.stdout:0/593: write d4/d13/d2a/d35/d1e/d32/d48/f4c [1761482,6019] 0 2024-02-15T18:34:57.521 INFO:tasks.workunit.client.0.smithi038.stdout:0/594: dread - d4/d13/d2a/d35/d1e/fac zero size 2024-02-15T18:34:57.522 INFO:tasks.workunit.client.0.smithi038.stdout:9/643: mknod d1/da/d10/cec 0 2024-02-15T18:34:57.525 INFO:tasks.workunit.client.0.smithi038.stdout:2/506: write dd/d12/d3a/f72 [3651500,60641] 0 2024-02-15T18:34:57.525 INFO:tasks.workunit.client.0.smithi038.stdout:8/681: dwrite d1/d4/d5/d6/f6e [0,4194304] 0 2024-02-15T18:34:57.525 INFO:tasks.workunit.client.0.smithi038.stdout:8/682: fsync d1/d4/d5/d31/d65/d80/fc5 0 2024-02-15T18:34:57.525 INFO:tasks.workunit.client.0.smithi038.stdout:5/526: rename d2/d75 to d2/d48/d83/d96 0 2024-02-15T18:34:57.527 INFO:tasks.workunit.client.0.smithi038.stdout:3/501: rmdir d2/d1b/d3c/d3a/d55/d7b 0 2024-02-15T18:34:57.531 INFO:tasks.workunit.client.0.smithi038.stdout:6/528: sync 2024-02-15T18:34:57.531 INFO:tasks.workunit.client.0.smithi038.stdout:6/529: stat d2/d56/d5f 0 2024-02-15T18:34:57.531 INFO:tasks.workunit.client.0.smithi038.stdout:6/530: chown d2/d4/d6f/c89 474 1 2024-02-15T18:34:57.531 INFO:tasks.workunit.client.0.smithi038.stdout:4/558: creat d11/d1e/d43/d5e/fd2 x:0 0 0 2024-02-15T18:34:57.532 INFO:tasks.workunit.client.0.smithi038.stdout:1/535: mknod d2/d28/d3c/cb3 0 2024-02-15T18:34:57.535 INFO:tasks.workunit.client.0.smithi038.stdout:2/507: dread dd/d12/d3a/d67/d4f/d5a/f6e [0,4194304] 0 2024-02-15T18:34:57.536 INFO:tasks.workunit.client.0.smithi038.stdout:0/595: dwrite d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/f92 [0,4194304] 0 2024-02-15T18:34:57.540 INFO:tasks.workunit.client.0.smithi038.stdout:7/577: unlink d0/db/f98 0 2024-02-15T18:34:57.543 INFO:tasks.workunit.client.0.smithi038.stdout:9/644: rmdir d1/da/d36/d93/deb 0 2024-02-15T18:34:57.544 INFO:tasks.workunit.client.0.smithi038.stdout:5/527: truncate d2/d30/d52/f5b 3255061 0 2024-02-15T18:34:57.544 INFO:tasks.workunit.client.0.smithi038.stdout:5/528: fdatasync d2/d3/d5f/f82 0 2024-02-15T18:34:57.545 INFO:tasks.workunit.client.0.smithi038.stdout:6/531: write d2/d4/d19/d55/d5c/f77 [2176494,118750] 0 2024-02-15T18:34:57.546 INFO:tasks.workunit.client.0.smithi038.stdout:3/502: mknod d2/d1b/d3c/d3a/d55/d65/c9f 0 2024-02-15T18:34:57.546 INFO:tasks.workunit.client.0.smithi038.stdout:3/503: write d2/d1b/d3c/d41/d4e/f73 [966445,41148] 0 2024-02-15T18:34:57.546 INFO:tasks.workunit.client.0.smithi038.stdout:4/559: creat d11/d1e/d20/d2f/fd3 x:0 0 0 2024-02-15T18:34:57.547 INFO:tasks.workunit.client.0.smithi038.stdout:1/536: creat d2/d69/fb4 x:0 0 0 2024-02-15T18:34:57.548 INFO:tasks.workunit.client.0.smithi038.stdout:1/537: readlink d2/d10/d32/d53/d82/l8a 0 2024-02-15T18:34:57.548 INFO:tasks.workunit.client.0.smithi038.stdout:0/596: getdents d4/d13/d2a/d35/d1e/d32/d79/d7d 0 2024-02-15T18:34:57.548 INFO:tasks.workunit.client.0.smithi038.stdout:8/683: dwrite d1/d4/d5/f37 [0,4194304] 0 2024-02-15T18:34:57.549 INFO:tasks.workunit.client.0.smithi038.stdout:7/578: truncate d0/db/d19/f6d 878106 0 2024-02-15T18:34:57.550 INFO:tasks.workunit.client.0.smithi038.stdout:5/529: creat d2/d30/d52/f97 x:0 0 0 2024-02-15T18:34:57.551 INFO:tasks.workunit.client.0.smithi038.stdout:6/532: mkdir d2/d4/d6c/d9d 0 2024-02-15T18:34:57.551 INFO:tasks.workunit.client.0.smithi038.stdout:6/533: write d2/d4/d19/d55/f81 [604072,14202] 0 2024-02-15T18:34:57.551 INFO:tasks.workunit.client.0.smithi038.stdout:3/504: mkdir d2/d1b/d3c/d3a/d55/da0 0 2024-02-15T18:34:57.552 INFO:tasks.workunit.client.0.smithi038.stdout:4/560: creat d11/d1e/d3e/d57/d7e/fd4 x:0 0 0 2024-02-15T18:34:57.553 INFO:tasks.workunit.client.0.smithi038.stdout:1/538: creat d2/d10/d32/d53/fb5 x:0 0 0 2024-02-15T18:34:57.554 INFO:tasks.workunit.client.0.smithi038.stdout:0/597: creat d4/d13/d2a/d35/db2/fbd x:0 0 0 2024-02-15T18:34:57.555 INFO:tasks.workunit.client.0.smithi038.stdout:8/684: rmdir d1/d4/d10/d43/d48 39 2024-02-15T18:34:57.556 INFO:tasks.workunit.client.0.smithi038.stdout:2/508: dwrite dd/d12/d1d/f31 [0,4194304] 0 2024-02-15T18:34:57.556 INFO:tasks.workunit.client.0.smithi038.stdout:2/509: truncate dd/d12/d3a/d67/d4f/d5a/f7f 1116545 0 2024-02-15T18:34:57.556 INFO:tasks.workunit.client.0.smithi038.stdout:2/510: fdatasync dd/d12/d3a/d67/f27 0 2024-02-15T18:34:57.563 INFO:tasks.workunit.client.0.smithi038.stdout:9/645: getdents d1/da/d36/d93 0 2024-02-15T18:34:57.566 INFO:tasks.workunit.client.0.smithi038.stdout:7/579: mkdir d0/db/d59/dc8 0 2024-02-15T18:34:57.568 INFO:tasks.workunit.client.0.smithi038.stdout:5/530: creat d2/d3/d12/d89/f98 x:0 0 0 2024-02-15T18:34:57.568 INFO:tasks.workunit.client.0.smithi038.stdout:5/531: write d2/f39 [4615884,55366] 0 2024-02-15T18:34:57.571 INFO:tasks.workunit.client.0.smithi038.stdout:6/534: creat d2/d4/d19/d78/f9e x:0 0 0 2024-02-15T18:34:57.571 INFO:tasks.workunit.client.0.smithi038.stdout:3/505: link d2/d1e/l23 d2/d1b/d4f/la1 0 2024-02-15T18:34:57.571 INFO:tasks.workunit.client.0.smithi038.stdout:4/561: symlink d11/d1e/d20/d2b/d4d/d67/da1/ld5 0 2024-02-15T18:34:57.572 INFO:tasks.workunit.client.0.smithi038.stdout:8/685: dwrite d1/d7/fd6 [4194304,4194304] 0 2024-02-15T18:34:57.573 INFO:tasks.workunit.client.0.smithi038.stdout:1/539: creat d2/d2e/fb6 x:0 0 0 2024-02-15T18:34:57.573 INFO:tasks.workunit.client.0.smithi038.stdout:1/540: write d2/d10/d19/d22/d36/f78 [4906467,105632] 0 2024-02-15T18:34:57.574 INFO:tasks.workunit.client.0.smithi038.stdout:0/598: symlink d4/d13/d2a/d35/d1e/d32/d3b/lbe 0 2024-02-15T18:34:57.580 INFO:tasks.workunit.client.0.smithi038.stdout:7/580: mkdir d0/db/d19/d7d/dc9 0 2024-02-15T18:34:57.580 INFO:tasks.workunit.client.0.smithi038.stdout:9/646: creat d1/da/fed x:0 0 0 2024-02-15T18:34:57.580 INFO:tasks.workunit.client.0.smithi038.stdout:7/581: chown d0/d1b/d30/d5d/fa3 8138 1 2024-02-15T18:34:57.580 INFO:tasks.workunit.client.0.smithi038.stdout:7/582: chown d0/db/d19/d7d/la7 20 1 2024-02-15T18:34:57.580 INFO:tasks.workunit.client.0.smithi038.stdout:9/647: truncate d1/da/d10/d82/fa9 311902 0 2024-02-15T18:34:57.580 INFO:tasks.workunit.client.0.smithi038.stdout:9/648: chown d1/dc/d4f/f47 15413 1 2024-02-15T18:34:57.582 INFO:tasks.workunit.client.0.smithi038.stdout:5/532: mkdir d2/d64/d99 0 2024-02-15T18:34:57.582 INFO:tasks.workunit.client.0.smithi038.stdout:5/533: fdatasync d2/d30/d52/f8b 0 2024-02-15T18:34:57.584 INFO:tasks.workunit.client.0.smithi038.stdout:6/535: rename d2/l50 to d2/d56/d5f/d99/l9f 0 2024-02-15T18:34:57.584 INFO:tasks.workunit.client.0.smithi038.stdout:6/536: fsync d2/f1a 0 2024-02-15T18:34:57.587 INFO:tasks.workunit.client.0.smithi038.stdout:8/686: dwrite d1/d3a/f7a [0,4194304] 0 2024-02-15T18:34:57.587 INFO:tasks.workunit.client.0.smithi038.stdout:8/687: read - d1/d3a/d58/f78 zero size 2024-02-15T18:34:57.587 INFO:tasks.workunit.client.0.smithi038.stdout:8/688: chown d1/d3a/d58 2461960 1 2024-02-15T18:34:57.589 INFO:tasks.workunit.client.0.smithi038.stdout:4/562: getdents d11/d1e/d20/d31/d8d/d99/db5 0 2024-02-15T18:34:57.589 INFO:tasks.workunit.client.0.smithi038.stdout:4/563: fsync d11/d1e/d20/d2b/d65/fb8 0 2024-02-15T18:34:57.591 INFO:tasks.workunit.client.0.smithi038.stdout:3/506: rmdir d2/d1b 39 2024-02-15T18:34:57.591 INFO:tasks.workunit.client.0.smithi038.stdout:3/507: dread - d2/d1b/d3c/d41/f56 zero size 2024-02-15T18:34:57.594 INFO:tasks.workunit.client.0.smithi038.stdout:5/534: dread d2/f3f [0,4194304] 0 2024-02-15T18:34:57.595 INFO:tasks.workunit.client.0.smithi038.stdout:8/689: dread d1/d7/d15/d4b/f3f [0,4194304] 0 2024-02-15T18:34:57.595 INFO:tasks.workunit.client.0.smithi038.stdout:8/690: write d1/d4/d5/d31/d65/d80/fc5 [215013,93550] 0 2024-02-15T18:34:57.601 INFO:tasks.workunit.client.0.smithi038.stdout:1/541: dwrite d2/d10/f31 [0,4194304] 0 2024-02-15T18:34:57.602 INFO:tasks.workunit.client.0.smithi038.stdout:9/649: mknod d1/da/d20/d58/d76/cee 0 2024-02-15T18:34:57.603 INFO:tasks.workunit.client.0.smithi038.stdout:7/583: rename d0/d66/f92 to d0/d68/fca 0 2024-02-15T18:34:57.603 INFO:tasks.workunit.client.0.smithi038.stdout:7/584: write d0/d1b/d30/d5d/d61/fb6 [1031654,103818] 0 2024-02-15T18:34:57.603 INFO:tasks.workunit.client.0.smithi038.stdout:6/537: mknod d2/d4/d19/d78/d7a/ca0 0 2024-02-15T18:34:57.605 INFO:tasks.workunit.client.0.smithi038.stdout:3/508: mknod d2/d1b/d3c/d3a/d55/ca2 0 2024-02-15T18:34:57.605 INFO:tasks.workunit.client.0.smithi038.stdout:5/535: truncate d2/d3/f1b 3167309 0 2024-02-15T18:34:57.606 INFO:tasks.workunit.client.0.smithi038.stdout:7/585: write d0/db/d59/d15/f44 [1033123,107723] 0 2024-02-15T18:34:57.606 INFO:tasks.workunit.client.0.smithi038.stdout:7/586: dread - d0/d1b/d30/d50/f8e zero size 2024-02-15T18:34:57.607 INFO:tasks.workunit.client.0.smithi038.stdout:0/599: dwrite d4/d13/d2a/d35/d1e/d32/d48/d97/f9b [0,4194304] 0 2024-02-15T18:34:57.607 INFO:tasks.workunit.client.0.smithi038.stdout:8/691: truncate d1/d7/d15/d4b/fc7 169436 0 2024-02-15T18:34:57.607 INFO:tasks.workunit.client.0.smithi038.stdout:8/692: readlink d1/d4/d10/d43/d48/d52/lc4 0 2024-02-15T18:34:57.608 INFO:tasks.workunit.client.0.smithi038.stdout:1/542: creat d2/d28/d6b/fb7 x:0 0 0 2024-02-15T18:34:57.612 INFO:tasks.workunit.client.0.smithi038.stdout:9/650: creat d1/da/d10/fef x:0 0 0 2024-02-15T18:34:57.612 INFO:tasks.workunit.client.0.smithi038.stdout:9/651: dread - d1/da/d10/d13/d3d/d43/f74 zero size 2024-02-15T18:34:57.614 INFO:tasks.workunit.client.0.smithi038.stdout:3/509: dread d2/f2f [0,4194304] 0 2024-02-15T18:34:57.614 INFO:tasks.workunit.client.0.smithi038.stdout:5/536: mkdir d2/d9a 0 2024-02-15T18:34:57.614 INFO:tasks.workunit.client.0.smithi038.stdout:6/538: creat d2/d56/d5f/fa1 x:0 0 0 2024-02-15T18:34:57.614 INFO:tasks.workunit.client.0.smithi038.stdout:5/537: truncate d2/d3/d5f/d71/d33/f93 628536 0 2024-02-15T18:34:57.614 INFO:tasks.workunit.client.0.smithi038.stdout:6/539: write d2/d56/f9c [268059,88899] 0 2024-02-15T18:34:57.614 INFO:tasks.workunit.client.0.smithi038.stdout:6/540: dread d2/d10/f4e [0,4194304] 0 2024-02-15T18:34:57.617 INFO:tasks.workunit.client.0.smithi038.stdout:7/587: symlink d0/d1b/d30/d5d/d61/d90/lcb 0 2024-02-15T18:34:57.623 INFO:tasks.workunit.client.0.smithi038.stdout:8/693: symlink d1/d4/d10/d43/d48/d52/lf7 0 2024-02-15T18:34:57.625 INFO:tasks.workunit.client.0.smithi038.stdout:4/564: rename d11/d1e/d20/d2b/d35/f55 to d11/d1e/d20/fd6 0 2024-02-15T18:34:57.626 INFO:tasks.workunit.client.0.smithi038.stdout:9/652: creat d1/dc/d69/d7d/ff0 x:0 0 0 2024-02-15T18:34:57.627 INFO:tasks.workunit.client.0.smithi038.stdout:3/510: creat d2/d1b/d3c/d41/d4e/fa3 x:0 0 0 2024-02-15T18:34:57.629 INFO:tasks.workunit.client.0.smithi038.stdout:5/538: creat d2/d3/d5f/d71/d33/f9b x:0 0 0 2024-02-15T18:34:57.629 INFO:tasks.workunit.client.0.smithi038.stdout:5/539: chown d2/d30/c3a 82 1 2024-02-15T18:34:57.629 INFO:tasks.workunit.client.0.smithi038.stdout:5/540: chown d2/f3f 47 1 2024-02-15T18:34:57.629 INFO:tasks.workunit.client.0.smithi038.stdout:5/541: dread - d2/d30/d52/f70 zero size 2024-02-15T18:34:57.631 INFO:tasks.workunit.client.0.smithi038.stdout:7/588: creat d0/d95/db8/fcc x:0 0 0 2024-02-15T18:34:57.633 INFO:tasks.workunit.client.0.smithi038.stdout:4/565: dread d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/f9b [0,4194304] 0 2024-02-15T18:34:57.633 INFO:tasks.workunit.client.0.smithi038.stdout:8/694: unlink d1/d4/d10/f35 0 2024-02-15T18:34:57.636 INFO:tasks.workunit.client.0.smithi038.stdout:0/600: dwrite d4/d13/d19/f82 [0,4194304] 0 2024-02-15T18:34:57.636 INFO:tasks.workunit.client.0.smithi038.stdout:0/601: stat d4/d13/c18 0 2024-02-15T18:34:57.637 INFO:tasks.workunit.client.0.smithi038.stdout:0/602: chown d4/d13/l2f 114336135 1 2024-02-15T18:34:57.639 INFO:tasks.workunit.client.0.smithi038.stdout:9/653: rename d1/da/d10/d13/d3d/d6c/fd9 to d1/da/d20/d58/d94/db9/ff1 0 2024-02-15T18:34:57.640 INFO:tasks.workunit.client.0.smithi038.stdout:9/654: chown d1/da/d20/d58/d94/db9 0 1 2024-02-15T18:34:57.640 INFO:tasks.workunit.client.0.smithi038.stdout:9/655: truncate d1/da/d10/d13/f89 760782 0 2024-02-15T18:34:57.640 INFO:tasks.workunit.client.0.smithi038.stdout:9/656: fsync d1/da/d20/d58/d76/f8c 0 2024-02-15T18:34:57.640 INFO:tasks.workunit.client.0.smithi038.stdout:9/657: read - d1/da/d10/d13/d3d/d4b/daa/fab zero size 2024-02-15T18:34:57.640 INFO:tasks.workunit.client.0.smithi038.stdout:3/511: link d2/f1a d2/d1b/d3c/d3a/d55/da0/fa4 0 2024-02-15T18:34:57.640 INFO:tasks.workunit.client.0.smithi038.stdout:3/512: dread - d2/d1b/f9e zero size 2024-02-15T18:34:57.640 INFO:tasks.workunit.client.0.smithi038.stdout:3/513: write d2/d1e/f89 [583710,69631] 0 2024-02-15T18:34:57.640 INFO:tasks.workunit.client.0.smithi038.stdout:6/541: dwrite d2/d4/d19/f74 [0,4194304] 0 2024-02-15T18:34:57.640 INFO:tasks.workunit.client.0.smithi038.stdout:5/542: dread d2/d30/f46 [0,4194304] 0 2024-02-15T18:34:57.640 INFO:tasks.workunit.client.0.smithi038.stdout:7/589: mkdir d0/db/d59/d15/d29/d8d/dcd 0 2024-02-15T18:34:57.641 INFO:tasks.workunit.client.0.smithi038.stdout:7/590: dread - d0/d1b/d30/d5d/d70/f88 zero size 2024-02-15T18:34:57.641 INFO:tasks.workunit.client.0.smithi038.stdout:4/566: readlink d11/d1e/d20/d2b/d35/la3 0 2024-02-15T18:34:57.641 INFO:tasks.workunit.client.0.smithi038.stdout:4/567: chown d11/d1e/d20/d2b/d4d/d67/d73/lbb 396974070 1 2024-02-15T18:34:57.642 INFO:tasks.workunit.client.0.smithi038.stdout:8/695: link d1/caf d1/d7/d15/d4b/cf8 0 2024-02-15T18:34:57.642 INFO:tasks.workunit.client.0.smithi038.stdout:8/696: dread - d1/d7/d15/d4b/feb zero size 2024-02-15T18:34:57.642 INFO:tasks.workunit.client.0.smithi038.stdout:8/697: stat d1/d4/db1/de5/c89 0 2024-02-15T18:34:57.643 INFO:tasks.workunit.client.0.smithi038.stdout:0/603: truncate d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/f92 2981436 0 2024-02-15T18:34:57.645 INFO:tasks.workunit.client.0.smithi038.stdout:0/604: dread d4/d13/d2a/d35/d1e/f50 [0,4194304] 0 2024-02-15T18:34:57.645 INFO:tasks.workunit.client.0.smithi038.stdout:0/605: truncate d4/d13/d53/fa7 428900 0 2024-02-15T18:34:57.645 INFO:tasks.workunit.client.0.smithi038.stdout:3/514: unlink d2/d1b/d3c/l2b 0 2024-02-15T18:34:57.645 INFO:tasks.workunit.client.0.smithi038.stdout:3/515: dread - d2/d1b/d69/f96 zero size 2024-02-15T18:34:57.647 INFO:tasks.workunit.client.0.smithi038.stdout:5/543: mknod d2/d3/d12/d5e/d86/c9c 0 2024-02-15T18:34:57.648 INFO:tasks.workunit.client.0.smithi038.stdout:6/542: symlink d2/d4/d19/d78/la2 0 2024-02-15T18:34:57.649 INFO:tasks.workunit.client.0.smithi038.stdout:7/591: dread - d0/f48 zero size 2024-02-15T18:34:57.650 INFO:tasks.workunit.client.0.smithi038.stdout:0/606: dread d4/d13/f8e [0,4194304] 0 2024-02-15T18:34:57.652 INFO:tasks.workunit.client.0.smithi038.stdout:3/516: mknod d2/d24/ca5 0 2024-02-15T18:34:57.652 INFO:tasks.workunit.client.0.smithi038.stdout:3/517: write d2/d1b/d69/f6a [630076,46312] 0 2024-02-15T18:34:57.652 INFO:tasks.workunit.client.0.smithi038.stdout:7/592: creat d0/d1b/d30/d5d/d61/d90/dc0/fce x:0 0 0 2024-02-15T18:34:57.653 INFO:tasks.workunit.client.0.smithi038.stdout:0/607: creat d4/d13/d2a/d35/d1e/d32/d3b/daa/fbf x:0 0 0 2024-02-15T18:34:57.654 INFO:tasks.workunit.client.0.smithi038.stdout:4/568: creat d11/d1e/d43/d4a/dbd/fd7 x:0 0 0 2024-02-15T18:34:57.654 INFO:tasks.workunit.client.0.smithi038.stdout:4/569: stat d11/d1e/d43/f45 0 2024-02-15T18:34:57.654 INFO:tasks.workunit.client.0.smithi038.stdout:4/570: fsync d11/d1e/d3e/d57/f77 0 2024-02-15T18:34:57.654 INFO:tasks.workunit.client.0.smithi038.stdout:4/571: write d11/d1e/d43/d5e/fd2 [518112,2119] 0 2024-02-15T18:34:57.656 INFO:tasks.workunit.client.0.smithi038.stdout:3/518: creat d2/d1b/d3c/d3a/d55/d65/fa6 x:0 0 0 2024-02-15T18:34:57.657 INFO:tasks.workunit.client.0.smithi038.stdout:7/593: creat d0/db/d19/d7d/db5/fcf x:0 0 0 2024-02-15T18:34:57.658 INFO:tasks.workunit.client.0.smithi038.stdout:1/543: dwrite d2/d10/d19/d22/d36/f78 [0,4194304] 0 2024-02-15T18:34:57.658 INFO:tasks.workunit.client.0.smithi038.stdout:1/544: dread - d2/d2e/d8c/fae zero size 2024-02-15T18:34:57.658 INFO:tasks.workunit.client.0.smithi038.stdout:1/545: truncate d2/f2d 4441510 0 2024-02-15T18:34:57.658 INFO:tasks.workunit.client.0.smithi038.stdout:8/698: dread d1/d4/d10/d43/d48/d52/fbd [0,4194304] 0 2024-02-15T18:34:57.659 INFO:tasks.workunit.client.0.smithi038.stdout:0/608: getdents d4/d13/d2a/d35/d1e/d32/d79 0 2024-02-15T18:34:57.659 INFO:tasks.workunit.client.0.smithi038.stdout:0/609: readlink d4/d13/d2a/d35/lb9 0 2024-02-15T18:34:57.659 INFO:tasks.workunit.client.0.smithi038.stdout:0/610: truncate d4/d13/d2a/d35/d1e/f5d 1476618 0 2024-02-15T18:34:57.660 INFO:tasks.workunit.client.0.smithi038.stdout:0/611: write d4/d13/d53/fa1 [3953575,47469] 0 2024-02-15T18:34:57.660 INFO:tasks.workunit.client.0.smithi038.stdout:9/658: dwrite d1/da/d10/d19/f42 [0,4194304] 0 2024-02-15T18:34:57.661 INFO:tasks.workunit.client.0.smithi038.stdout:9/659: chown d1/da/d36/d3f/d48/f57 19 1 2024-02-15T18:34:57.661 INFO:tasks.workunit.client.0.smithi038.stdout:4/572: unlink d11/d1e/d3e/d57/f77 0 2024-02-15T18:34:57.666 INFO:tasks.workunit.client.0.smithi038.stdout:5/544: dwrite d2/d30/d52/f97 [0,4194304] 0 2024-02-15T18:34:57.667 INFO:tasks.workunit.client.0.smithi038.stdout:5/545: write d2/d3/d12/fa [533007,39262] 0 2024-02-15T18:34:57.669 INFO:tasks.workunit.client.0.smithi038.stdout:7/594: mkdir d0/db/d19/d7d/daa/dd0 0 2024-02-15T18:34:57.672 INFO:tasks.workunit.client.0.smithi038.stdout:8/699: rename d1/d4/d5/d6/d6d/l7b to d1/d4/d10/d43/d48/lf9 0 2024-02-15T18:34:57.676 INFO:tasks.workunit.client.0.smithi038.stdout:0/612: unlink d4/d13/d2a/d35/d6a/f77 0 2024-02-15T18:34:57.679 INFO:tasks.workunit.client.0.smithi038.stdout:1/546: dwrite d2/d10/d19/d5f/f70 [0,4194304] 0 2024-02-15T18:34:57.679 INFO:tasks.workunit.client.0.smithi038.stdout:9/660: mknod d1/da/d36/d3f/d99/cf2 0 2024-02-15T18:34:57.679 INFO:tasks.workunit.client.0.smithi038.stdout:9/661: chown d1/dc/d4f 586412136 1 2024-02-15T18:34:57.679 INFO:tasks.workunit.client.0.smithi038.stdout:4/573: mkdir d11/d1e/d20/d31/d7a/dd8 0 2024-02-15T18:34:57.680 INFO:tasks.workunit.client.0.smithi038.stdout:0/613: dread d4/d13/d2a/f4b [0,4194304] 0 2024-02-15T18:34:57.681 INFO:tasks.workunit.client.0.smithi038.stdout:3/519: dwrite d2/d1b/d3c/d41/f56 [0,4194304] 0 2024-02-15T18:34:57.688 INFO:tasks.workunit.client.0.smithi038.stdout:5/546: mknod d2/d3/d5f/c9d 0 2024-02-15T18:34:57.690 INFO:tasks.workunit.client.0.smithi038.stdout:1/547: getdents d2/d10/d19/d22 0 2024-02-15T18:34:57.690 INFO:tasks.workunit.client.0.smithi038.stdout:1/548: readlink d2/l8 0 2024-02-15T18:34:57.690 INFO:tasks.workunit.client.0.smithi038.stdout:8/700: dwrite d1/d7/d15/fb2 [0,4194304] 0 2024-02-15T18:34:57.692 INFO:tasks.workunit.client.0.smithi038.stdout:9/662: rename d1/dc/d38/f6f to d1/da/d20/d58/d76/ff3 0 2024-02-15T18:34:57.693 INFO:tasks.workunit.client.0.smithi038.stdout:0/614: mkdir d4/d13/d2a/d35/d1e/d32/dc0 0 2024-02-15T18:34:57.695 INFO:tasks.workunit.client.0.smithi038.stdout:1/549: stat d2/d10/c24 0 2024-02-15T18:34:57.695 INFO:tasks.workunit.client.0.smithi038.stdout:7/595: dwrite d0/d1b/d30/f43 [0,4194304] 0 2024-02-15T18:34:57.697 INFO:tasks.workunit.client.0.smithi038.stdout:8/701: rename d1/d4/d5/f37 to d1/d4/d5/d31/d65/ffa 0 2024-02-15T18:34:57.697 INFO:tasks.workunit.client.0.smithi038.stdout:8/702: write d1/d7/fb0 [662760,31655] 0 2024-02-15T18:34:57.697 INFO:tasks.workunit.client.0.smithi038.stdout:0/615: truncate d4/d13/d19/fb0 4176330 0 2024-02-15T18:34:57.699 INFO:tasks.workunit.client.0.smithi038.stdout:1/550: creat d2/d10/d19/d5f/d92/fb8 x:0 0 0 2024-02-15T18:34:57.701 INFO:tasks.workunit.client.0.smithi038.stdout:7/596: symlink d0/db/d19/d7d/daa/dd0/ld1 0 2024-02-15T18:34:57.701 INFO:tasks.workunit.client.0.smithi038.stdout:8/703: creat d1/d7/d15/d6a/ffb x:0 0 0 2024-02-15T18:34:57.701 INFO:tasks.workunit.client.0.smithi038.stdout:8/704: chown d1/d7/d22/f95 6903172 1 2024-02-15T18:34:57.703 INFO:tasks.workunit.client.0.smithi038.stdout:4/574: dwrite d11/d1e/d20/f76 [0,4194304] 0 2024-02-15T18:34:57.703 INFO:tasks.workunit.client.0.smithi038.stdout:4/575: dread - d11/d1e/d20/f70 zero size 2024-02-15T18:34:57.704 INFO:tasks.workunit.client.0.smithi038.stdout:4/576: chown d11/d1e/d20/d2b/d4d/d67/d73/d8b/fad 5556 1 2024-02-15T18:34:57.704 INFO:tasks.workunit.client.0.smithi038.stdout:0/616: link d4/d13/d2a/d35/c6b d4/d13/cc1 0 2024-02-15T18:34:57.705 INFO:tasks.workunit.client.0.smithi038.stdout:4/577: getdents d11/d1e/d20/d2b/d4d/d67 0 2024-02-15T18:34:57.705 INFO:tasks.workunit.client.0.smithi038.stdout:7/597: read d0/d1b/d30/d5d/d61/d90/fab [3113807,117711] 0 2024-02-15T18:34:57.710 INFO:tasks.workunit.client.0.smithi038.stdout:0/617: rename d4/d13/d2a/d80 to d4/d13/d2a/d35/d1e/d32/d79/d7d/da3/dc2 0 2024-02-15T18:34:57.711 INFO:tasks.workunit.client.0.smithi038.stdout:8/705: dread d1/d7/d15/d6a/f6b [0,4194304] 0 2024-02-15T18:34:57.711 INFO:tasks.workunit.client.0.smithi038.stdout:3/520: dwrite d2/d1b/d3c/d41/d4e/d62/f8f [0,4194304] 0 2024-02-15T18:34:57.711 INFO:tasks.workunit.client.0.smithi038.stdout:4/578: getdents d11/d1e/d43/d5e/d62 0 2024-02-15T18:34:57.711 INFO:tasks.workunit.client.0.smithi038.stdout:3/521: dread - d2/d1b/d3c/d3a/f91 zero size 2024-02-15T18:34:57.711 INFO:tasks.workunit.client.0.smithi038.stdout:4/579: stat d11/f17 0 2024-02-15T18:34:57.711 INFO:tasks.workunit.client.0.smithi038.stdout:9/663: dread d1/da/d36/d3f/d48/db8/fbb [0,4194304] 0 2024-02-15T18:34:57.711 INFO:tasks.workunit.client.0.smithi038.stdout:9/664: fdatasync d1/dc/f28 0 2024-02-15T18:34:57.712 INFO:tasks.workunit.client.0.smithi038.stdout:7/598: unlink d0/db/d59/fa4 0 2024-02-15T18:34:57.713 INFO:tasks.workunit.client.0.smithi038.stdout:7/599: write d0/d1b/d30/d50/f74 [349169,9909] 0 2024-02-15T18:34:57.714 INFO:tasks.workunit.client.0.smithi038.stdout:8/706: symlink d1/d4/d5/d31/d65/d80/lfc 0 2024-02-15T18:34:57.714 INFO:tasks.workunit.client.0.smithi038.stdout:3/522: mknod d2/d1b/d4f/ca7 0 2024-02-15T18:34:57.714 INFO:tasks.workunit.client.0.smithi038.stdout:4/580: getdents d11/d1e/d20/d2b/d4d/d67/d73/d8b 0 2024-02-15T18:34:57.717 INFO:tasks.workunit.client.0.smithi038.stdout:9/665: link d1/dc/d69/lcf d1/da/d36/d3f/d48/db8/lf4 0 2024-02-15T18:34:57.721 INFO:tasks.workunit.client.0.smithi038.stdout:7/600: link d0/db/d59/d15/f44 d0/db/d19/fd2 0 2024-02-15T18:34:57.723 INFO:tasks.workunit.client.0.smithi038.stdout:1/551: dwrite d2/d10/d19/d22/f57 [0,4194304] 0 2024-02-15T18:34:57.723 INFO:tasks.workunit.client.0.smithi038.stdout:1/552: chown d2/d10/d19/d22/d54/c84 6744 1 2024-02-15T18:34:57.723 INFO:tasks.workunit.client.0.smithi038.stdout:1/553: write d2/d69/fb4 [672065,73775] 0 2024-02-15T18:34:57.725 INFO:tasks.workunit.client.0.smithi038.stdout:8/707: mknod d1/d3a/df6/cfd 0 2024-02-15T18:34:57.725 INFO:tasks.workunit.client.0.smithi038.stdout:8/708: chown d1/d4/d10/d43/l49 1 1 2024-02-15T18:34:57.725 INFO:tasks.workunit.client.0.smithi038.stdout:8/709: write d1/d4/d10/d43/d48/fcd [587107,96163] 0 2024-02-15T18:34:57.725 INFO:tasks.workunit.client.0.smithi038.stdout:3/523: mknod d2/ca8 0 2024-02-15T18:34:57.726 INFO:tasks.workunit.client.0.smithi038.stdout:0/618: dwrite d4/d13/d2a/d35/d1e/d32/d48/d97/faf [4194304,4194304] 0 2024-02-15T18:34:57.729 INFO:tasks.workunit.client.0.smithi038.stdout:2/511: sync 2024-02-15T18:34:57.730 INFO:tasks.workunit.client.0.smithi038.stdout:2/512: write dd/d12/d3a/d67/f30 [701875,66602] 0 2024-02-15T18:34:57.733 INFO:tasks.workunit.client.0.smithi038.stdout:4/581: creat d11/d1e/d20/d31/d48/dac/fd9 x:0 0 0 2024-02-15T18:34:57.733 INFO:tasks.workunit.client.0.smithi038.stdout:4/582: write d11/d1e/d20/d31/d8d/d99/db5/fcb [159738,110241] 0 2024-02-15T18:34:57.736 INFO:tasks.workunit.client.0.smithi038.stdout:9/666: unlink d1/da/d36/f56 0 2024-02-15T18:34:57.737 INFO:tasks.workunit.client.0.smithi038.stdout:1/554: creat d2/d10/d19/d5f/fb9 x:0 0 0 2024-02-15T18:34:57.737 INFO:tasks.workunit.client.0.smithi038.stdout:1/555: chown d2/d2e/fb6 6487 1 2024-02-15T18:34:57.739 INFO:tasks.workunit.client.0.smithi038.stdout:7/601: dwrite d0/d1b/d30/d5d/d61/d90/fab [0,4194304] 0 2024-02-15T18:34:57.742 INFO:tasks.workunit.client.0.smithi038.stdout:8/710: creat d1/d4/d10/d43/d48/ffe x:0 0 0 2024-02-15T18:34:57.743 INFO:tasks.workunit.client.0.smithi038.stdout:3/524: mknod d2/d24/ca9 0 2024-02-15T18:34:57.745 INFO:tasks.workunit.client.0.smithi038.stdout:0/619: readlink d4/d13/d2a/l70 0 2024-02-15T18:34:57.750 INFO:tasks.workunit.client.0.smithi038.stdout:0/620: dread d4/d13/d53/f7b [0,4194304] 0 2024-02-15T18:34:57.750 INFO:tasks.workunit.client.0.smithi038.stdout:2/513: creat dd/d12/d8d/fa8 x:0 0 0 2024-02-15T18:34:57.750 INFO:tasks.workunit.client.0.smithi038.stdout:4/583: link d11/d1e/d3e/c81 d11/d1e/d20/d2f/cda 0 2024-02-15T18:34:57.750 INFO:tasks.workunit.client.0.smithi038.stdout:9/667: rename d1/da/d20/d58/d94/db9/lbc to d1/da/d10/d13/d3d/d4b/lf5 0 2024-02-15T18:34:57.753 INFO:tasks.workunit.client.0.smithi038.stdout:4/584: write d11/d1e/d20/d2f/d71/f98 [5635688,129411] 0 2024-02-15T18:34:57.753 INFO:tasks.workunit.client.0.smithi038.stdout:4/585: chown d11/d1e/d20/d2b/d4d/d67/da1 6147485 1 2024-02-15T18:34:57.753 INFO:tasks.workunit.client.0.smithi038.stdout:1/556: getdents d2/d10/d19/d22/d54 0 2024-02-15T18:34:57.754 INFO:tasks.workunit.client.0.smithi038.stdout:4/586: read d11/f17 [1879038,10871] 0 2024-02-15T18:34:57.760 INFO:tasks.workunit.client.0.smithi038.stdout:8/711: creat d1/d4/d5/d6/db9/fff x:0 0 0 2024-02-15T18:34:57.760 INFO:tasks.workunit.client.0.smithi038.stdout:7/602: creat d0/db/d19/d7d/fd3 x:0 0 0 2024-02-15T18:34:57.763 INFO:tasks.workunit.client.0.smithi038.stdout:3/525: dwrite d2/d25/f66 [0,4194304] 0 2024-02-15T18:34:57.764 INFO:tasks.workunit.client.0.smithi038.stdout:0/621: creat d4/d13/d2a/d35/d1e/d32/d48/d9a/fc3 x:0 0 0 2024-02-15T18:34:57.764 INFO:tasks.workunit.client.0.smithi038.stdout:2/514: symlink dd/d12/d8d/d90/la9 0 2024-02-15T18:34:57.768 INFO:tasks.workunit.client.0.smithi038.stdout:1/557: mkdir d2/d10/d19/d22/d54/dba 0 2024-02-15T18:34:57.769 INFO:tasks.workunit.client.0.smithi038.stdout:8/712: rmdir d1/d4/d5/d31/d65/d80 39 2024-02-15T18:34:57.769 INFO:tasks.workunit.client.0.smithi038.stdout:8/713: chown d1/d4/db1 3144 1 2024-02-15T18:34:57.770 INFO:tasks.workunit.client.0.smithi038.stdout:7/603: creat d0/d1b/d30/d5d/fd4 x:0 0 0 2024-02-15T18:34:57.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:56.317880+0000 mgr.smithi038.khesbl (mgr.14162) 338 : cluster [DBG] pgmap v280: 65 pgs: 65 active+clean; 992 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 57 MiB/s rd, 120 MiB/s wr, 661 op/s 2024-02-15T18:34:57.776 INFO:tasks.workunit.client.0.smithi038.stdout:1/558: symlink d2/d2e/d8c/lbb 0 2024-02-15T18:34:57.776 INFO:tasks.workunit.client.0.smithi038.stdout:0/622: truncate d4/d13/d2a/d35/d1e/d32/fb6 3903710 0 2024-02-15T18:34:57.776 INFO:tasks.workunit.client.0.smithi038.stdout:1/559: dread - d2/d28/d6b/fb7 zero size 2024-02-15T18:34:57.777 INFO:tasks.workunit.client.0.smithi038.stdout:9/668: dwrite d1/dc/d5a/fde [0,4194304] 0 2024-02-15T18:34:57.777 INFO:tasks.workunit.client.0.smithi038.stdout:9/669: stat d1/dc/d5a/d7f 0 2024-02-15T18:34:57.777 INFO:tasks.workunit.client.0.smithi038.stdout:9/670: dread - d1/da/d10/d13/d3d/d6c/fd4 zero size 2024-02-15T18:34:57.778 INFO:tasks.workunit.client.0.smithi038.stdout:0/623: getdents d4/d13/d2a/d35/d1e/d32/d48/d97 0 2024-02-15T18:34:57.778 INFO:tasks.workunit.client.0.smithi038.stdout:0/624: chown d4/d13/d2a/f4e 1 1 2024-02-15T18:34:57.780 INFO:tasks.workunit.client.0.smithi038.stdout:1/560: mkdir d2/d69/da9/dbc 0 2024-02-15T18:34:57.780 INFO:tasks.workunit.client.0.smithi038.stdout:1/561: fdatasync d2/d10/d62/fa6 0 2024-02-15T18:34:57.780 INFO:tasks.workunit.client.0.smithi038.stdout:1/562: dread - d2/d39/f48 zero size 2024-02-15T18:34:57.780 INFO:tasks.workunit.client.0.smithi038.stdout:1/563: dread - d2/d10/fa3 zero size 2024-02-15T18:34:57.781 INFO:tasks.workunit.client.0.smithi038.stdout:4/587: dwrite d11/d1e/d20/d2b/d65/f6d [0,4194304] 0 2024-02-15T18:34:57.781 INFO:tasks.workunit.client.0.smithi038.stdout:4/588: write d11/d1e/d20/d2b/d4d/f95 [556766,77331] 0 2024-02-15T18:34:57.782 INFO:tasks.workunit.client.0.smithi038.stdout:7/604: mknod d0/db/d59/d15/d29/d7e/cd5 0 2024-02-15T18:34:57.787 INFO:tasks.workunit.client.0.smithi038.stdout:7/605: dread d0/d1b/d30/d5d/d61/f75 [0,4194304] 0 2024-02-15T18:34:57.788 INFO:tasks.workunit.client.0.smithi038.stdout:8/714: dwrite d1/fb [4194304,4194304] 0 2024-02-15T18:34:57.788 INFO:tasks.workunit.client.0.smithi038.stdout:2/515: dwrite dd/d12/d3a/f72 [0,4194304] 0 2024-02-15T18:34:57.789 INFO:tasks.workunit.client.0.smithi038.stdout:9/671: symlink d1/da/lf6 0 2024-02-15T18:34:57.792 INFO:tasks.workunit.client.0.smithi038.stdout:8/715: dread d1/d7/d22/f5c [0,4194304] 0 2024-02-15T18:34:57.793 INFO:tasks.workunit.client.0.smithi038.stdout:0/625: symlink d4/d13/d2a/d35/d1e/d32/d48/d97/lc4 0 2024-02-15T18:34:57.793 INFO:tasks.workunit.client.0.smithi038.stdout:0/626: readlink d4/d13/d2a/d35/d1e/d32/d48/d97/lc4 0 2024-02-15T18:34:57.794 INFO:tasks.workunit.client.0.smithi038.stdout:1/564: mknod d2/d10/d19/d22/d36/d94/cbd 0 2024-02-15T18:34:57.794 INFO:tasks.workunit.client.0.smithi038.stdout:1/565: readlink d2/d10/d19/d22/l33 0 2024-02-15T18:34:57.796 INFO:tasks.workunit.client.0.smithi038.stdout:4/589: creat d11/d1e/d20/d2b/d4d/d67/d73/fdb x:0 0 0 2024-02-15T18:34:57.798 INFO:tasks.workunit.client.0.smithi038.stdout:2/516: rmdir dd 39 2024-02-15T18:34:57.800 INFO:tasks.workunit.client.0.smithi038.stdout:9/672: mknod d1/dc/d5a/d7f/dd1/cf7 0 2024-02-15T18:34:57.800 INFO:tasks.workunit.client.0.smithi038.stdout:0/627: readlink d4/l8 0 2024-02-15T18:34:57.800 INFO:tasks.workunit.client.0.smithi038.stdout:9/673: dread - d1/dc/db2/fdc zero size 2024-02-15T18:34:57.800 INFO:tasks.workunit.client.0.smithi038.stdout:9/674: fsync d1/dc/db2/fe7 0 2024-02-15T18:34:57.802 INFO:tasks.workunit.client.0.smithi038.stdout:1/566: dread d2/d10/d19/d22/f30 [0,4194304] 0 2024-02-15T18:34:57.803 INFO:tasks.workunit.client.0.smithi038.stdout:7/606: rename d0/d1b/d30/d5d to d0/db/d59/dc8/dd6 0 2024-02-15T18:34:57.808 INFO:tasks.workunit.client.0.smithi038.stdout:4/590: dread d11/d1e/d3e/f54 [0,4194304] 0 2024-02-15T18:34:57.808 INFO:tasks.workunit.client.0.smithi038.stdout:1/567: link d2/d39/f89 d2/d28/d6b/fbe 0 2024-02-15T18:34:57.808 INFO:tasks.workunit.client.0.smithi038.stdout:1/568: readlink d2/d10/d19/la7 0 2024-02-15T18:34:57.808 INFO:tasks.workunit.client.0.smithi038.stdout:1/569: readlink d2/l1a 0 2024-02-15T18:34:57.808 INFO:tasks.workunit.client.0.smithi038.stdout:7/607: unlink d0/db/d19/f8b 0 2024-02-15T18:34:57.809 INFO:tasks.workunit.client.0.smithi038.stdout:2/517: creat dd/d12/d3a/d67/faa x:0 0 0 2024-02-15T18:34:57.810 INFO:tasks.workunit.client.0.smithi038.stdout:4/591: mkdir d11/d1e/d20/d2b/d4d/d67/d73/d8b/dca/ddc 0 2024-02-15T18:34:57.810 INFO:tasks.workunit.client.0.smithi038.stdout:4/592: fdatasync d11/d1e/d20/f76 0 2024-02-15T18:34:57.810 INFO:tasks.workunit.client.0.smithi038.stdout:2/518: symlink dd/d12/d59/d62/lab 0 2024-02-15T18:34:57.813 INFO:tasks.workunit.client.0.smithi038.stdout:7/608: read d0/db/d59/d15/f44 [3106506,17734] 0 2024-02-15T18:34:57.813 INFO:tasks.workunit.client.0.smithi038.stdout:7/609: read - d0/db/d59/dc8/dd6/d61/d90/dc0/fce zero size 2024-02-15T18:34:57.813 INFO:tasks.workunit.client.0.smithi038.stdout:8/716: dwrite d1/d4/d5/d6/fcb [0,4194304] 0 2024-02-15T18:34:57.813 INFO:tasks.workunit.client.0.smithi038.stdout:4/593: rename d11/d1e/d20/d31/d7a/d8f to d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/ddd 0 2024-02-15T18:34:57.816 INFO:tasks.workunit.client.0.smithi038.stdout:7/610: unlink d0/d1b/d30/d50/f8e 0 2024-02-15T18:34:57.817 INFO:tasks.workunit.client.0.smithi038.stdout:8/717: creat d1/d7/d22/da7/f100 x:0 0 0 2024-02-15T18:34:57.818 INFO:tasks.workunit.client.0.smithi038.stdout:8/718: chown d1/d7/d22/da7/f100 920040 1 2024-02-15T18:34:57.818 INFO:tasks.workunit.client.0.smithi038.stdout:8/719: dread d1/d4/d5/d31/d65/ff2 [0,4194304] 0 2024-02-15T18:34:57.818 INFO:tasks.workunit.client.0.smithi038.stdout:7/611: symlink d0/db/d19/d7d/ld7 0 2024-02-15T18:34:57.819 INFO:tasks.workunit.client.0.smithi038.stdout:7/612: fdatasync d0/d1b/d30/f43 0 2024-02-15T18:34:57.819 INFO:tasks.workunit.client.0.smithi038.stdout:8/720: symlink d1/d7/d22/da7/l101 0 2024-02-15T18:34:57.821 INFO:tasks.workunit.client.0.smithi038.stdout:7/613: creat d0/db/d59/d15/d29/fd8 x:0 0 0 2024-02-15T18:34:57.822 INFO:tasks.workunit.client.0.smithi038.stdout:9/675: dwrite d1/da/d10/d13/f61 [0,4194304] 0 2024-02-15T18:34:57.823 INFO:tasks.workunit.client.0.smithi038.stdout:7/614: mkdir d0/d68/dd9 0 2024-02-15T18:34:57.823 INFO:tasks.workunit.client.0.smithi038.stdout:7/615: chown d0/db/d19/fd2 0 1 2024-02-15T18:34:57.823 INFO:tasks.workunit.client.0.smithi038.stdout:9/676: rmdir d1/da/d10/d13/d3d/d43/d4d 39 2024-02-15T18:34:57.825 INFO:tasks.workunit.client.0.smithi038.stdout:9/677: creat d1/dc/d69/d7d/ff8 x:0 0 0 2024-02-15T18:34:57.826 INFO:tasks.workunit.client.0.smithi038.stdout:9/678: getdents d1/da/d10/d19 0 2024-02-15T18:34:57.826 INFO:tasks.workunit.client.0.smithi038.stdout:9/679: dread - d1/da/d10/d13/d3d/d4b/daa/fb1 zero size 2024-02-15T18:34:57.826 INFO:tasks.workunit.client.0.smithi038.stdout:9/680: truncate d1/da/d36/d3f/d99/fe1 760372 0 2024-02-15T18:34:57.831 INFO:tasks.workunit.client.0.smithi038.stdout:7/616: rename d0/db/d59/c12 to d0/cda 0 2024-02-15T18:34:57.831 INFO:tasks.workunit.client.0.smithi038.stdout:7/617: chown d0/db/d59/dc8/dd6/d61/d90/fab 3104 1 2024-02-15T18:34:57.834 INFO:tasks.workunit.client.0.smithi038.stdout:7/618: mknod d0/d1b/d80/cdb 0 2024-02-15T18:34:57.834 INFO:tasks.workunit.client.0.smithi038.stdout:7/619: chown d0/db/d59/dc8/dd6/d61/d90/lcb 27 1 2024-02-15T18:34:57.836 INFO:tasks.workunit.client.0.smithi038.stdout:7/620: mkdir d0/db/d59/d97/ddc 0 2024-02-15T18:34:57.838 INFO:tasks.workunit.client.0.smithi038.stdout:7/621: rmdir d0/db/d59/d15/d29/d8d/db4 0 2024-02-15T18:34:57.841 INFO:tasks.workunit.client.0.smithi038.stdout:0/628: dwrite d4/f11 [4194304,4194304] 0 2024-02-15T18:34:57.843 INFO:tasks.workunit.client.0.smithi038.stdout:0/629: truncate d4/d13/d2a/d35/d1e/f50 472247 0 2024-02-15T18:34:57.844 INFO:tasks.workunit.client.0.smithi038.stdout:0/630: creat d4/d13/d2a/d35/d1e/d32/d48/fc5 x:0 0 0 2024-02-15T18:34:57.845 INFO:tasks.workunit.client.0.smithi038.stdout:0/631: truncate d4/d13/f98 1299132 0 2024-02-15T18:34:57.845 INFO:tasks.workunit.client.0.smithi038.stdout:0/632: write d4/d13/d2a/d35/d1e/f20 [7384034,35801] 0 2024-02-15T18:34:57.846 INFO:tasks.workunit.client.0.smithi038.stdout:0/633: write d4/f88 [8426790,118259] 0 2024-02-15T18:34:57.847 INFO:tasks.workunit.client.0.smithi038.stdout:0/634: mkdir d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6 0 2024-02-15T18:34:57.849 INFO:tasks.workunit.client.0.smithi038.stdout:0/635: mkdir d4/db3/dc7 0 2024-02-15T18:34:57.851 INFO:tasks.workunit.client.0.smithi038.stdout:0/636: rmdir d4/d13/d2a/d35/d1e/d32/dc0 0 2024-02-15T18:34:57.851 INFO:tasks.workunit.client.0.smithi038.stdout:0/637: read - d4/d13/d2a/d35/d1e/d32/d48/fc5 zero size 2024-02-15T18:34:57.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:56.317880+0000 mgr.smithi038.khesbl (mgr.14162) 338 : cluster [DBG] pgmap v280: 65 pgs: 65 active+clean; 992 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 57 MiB/s rd, 120 MiB/s wr, 661 op/s 2024-02-15T18:34:57.859 INFO:tasks.workunit.client.0.smithi038.stdout:1/570: dwrite d2/d39/f89 [0,4194304] 0 2024-02-15T18:34:57.860 INFO:tasks.workunit.client.0.smithi038.stdout:8/721: dwrite d1/d4/d5/d31/f7f [0,4194304] 0 2024-02-15T18:34:57.860 INFO:tasks.workunit.client.0.smithi038.stdout:4/594: dwrite d11/d1e/d20/d2f/d71/f98 [0,4194304] 0 2024-02-15T18:34:57.860 INFO:tasks.workunit.client.0.smithi038.stdout:9/681: dwrite d1/da/d20/d58/d94/db9/dc6/f53 [0,4194304] 0 2024-02-15T18:34:57.867 INFO:tasks.workunit.client.0.smithi038.stdout:1/571: dread d2/d10/d19/d5f/f70 [0,4194304] 0 2024-02-15T18:34:57.876 INFO:tasks.workunit.client.0.smithi038.stdout:0/638: dwrite d4/ff [0,4194304] 0 2024-02-15T18:34:57.877 INFO:tasks.workunit.client.0.smithi038.stdout:4/595: mkdir d11/d1e/d20/d31/d48/dac/dbe/d40/dde 0 2024-02-15T18:34:57.877 INFO:tasks.workunit.client.0.smithi038.stdout:8/722: dwrite d1/d7/d22/f5a [0,4194304] 0 2024-02-15T18:34:57.880 INFO:tasks.workunit.client.0.smithi038.stdout:9/682: unlink d1/da/d36/d3f/d99/fb7 0 2024-02-15T18:34:57.885 INFO:tasks.workunit.client.0.smithi038.stdout:6/543: sync 2024-02-15T18:34:57.888 INFO:tasks.workunit.client.0.smithi038.stdout:1/572: rename d2/d10/d19/d22/f7a to d2/d2e/d8c/fbf 0 2024-02-15T18:34:57.888 INFO:tasks.workunit.client.0.smithi038.stdout:1/573: readlink d2/d10/d19/d22/d36/la1 0 2024-02-15T18:34:57.889 INFO:tasks.workunit.client.0.smithi038.stdout:4/596: creat d11/d1e/d20/d2f/d7d/fdf x:0 0 0 2024-02-15T18:34:57.891 INFO:tasks.workunit.client.0.smithi038.stdout:9/683: symlink d1/da/d10/d13/d3d/lf9 0 2024-02-15T18:34:57.891 INFO:tasks.workunit.client.0.smithi038.stdout:9/684: stat d1/da/d10/d13/d3d/d43/d4d/l81 0 2024-02-15T18:34:57.891 INFO:tasks.workunit.client.0.smithi038.stdout:6/544: creat d2/d4/d19/fa3 x:0 0 0 2024-02-15T18:34:57.893 INFO:tasks.workunit.client.0.smithi038.stdout:0/639: rename d4/d13/d2a/d35/c6b to d4/d13/d19/cc8 0 2024-02-15T18:34:57.894 INFO:tasks.workunit.client.0.smithi038.stdout:4/597: creat d11/d1e/d20/d31/d48/fe0 x:0 0 0 2024-02-15T18:34:57.895 INFO:tasks.workunit.client.0.smithi038.stdout:8/723: rename d1/d3a/l8e to d1/d4/d5/d6/l102 0 2024-02-15T18:34:57.895 INFO:tasks.workunit.client.0.smithi038.stdout:8/724: chown d1/c3 182613424 1 2024-02-15T18:34:57.895 INFO:tasks.workunit.client.0.smithi038.stdout:8/725: write d1/f1b [1559971,103455] 0 2024-02-15T18:34:57.898 INFO:tasks.workunit.client.0.smithi038.stdout:0/640: creat d4/d13/d2a/d35/d1e/d32/d48/d97/fc9 x:0 0 0 2024-02-15T18:34:57.898 INFO:tasks.workunit.client.0.smithi038.stdout:4/598: mknod d11/d1e/d20/ce1 0 2024-02-15T18:34:57.900 INFO:tasks.workunit.client.0.smithi038.stdout:6/545: rename d2/d4/ff to d2/d56/d75/fa4 0 2024-02-15T18:34:57.901 INFO:tasks.workunit.client.0.smithi038.stdout:4/599: write d11/d1e/d20/d2b/d65/fb8 [7601085,113756] 0 2024-02-15T18:34:57.901 INFO:tasks.workunit.client.0.smithi038.stdout:4/600: stat d11/d1e/d3e/f47 0 2024-02-15T18:34:57.901 INFO:tasks.workunit.client.0.smithi038.stdout:4/601: truncate d11/d1e/d20/d2b/d65/f6d 5178918 0 2024-02-15T18:34:57.901 INFO:tasks.workunit.client.0.smithi038.stdout:0/641: mkdir d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/dca 0 2024-02-15T18:34:57.901 INFO:tasks.workunit.client.0.smithi038.stdout:0/642: getdents d4/d13/d2a/d35/d1e/d32/d79 0 2024-02-15T18:34:57.902 INFO:tasks.workunit.client.0.smithi038.stdout:8/726: rename d1/d3a/c7e to d1/d4/d5/d31/d65/d80/c103 0 2024-02-15T18:34:57.902 INFO:tasks.workunit.client.0.smithi038.stdout:8/727: truncate d1/f6f 693367 0 2024-02-15T18:34:57.902 INFO:tasks.workunit.client.0.smithi038.stdout:8/728: chown d1/d4/db1/dd8/cde 1404 1 2024-02-15T18:34:57.903 INFO:tasks.workunit.client.0.smithi038.stdout:6/546: creat d2/d4/d6c/d91/fa5 x:0 0 0 2024-02-15T18:34:57.904 INFO:tasks.workunit.client.0.smithi038.stdout:0/643: unlink d4/d13/d2a/d35/c28 0 2024-02-15T18:34:57.905 INFO:tasks.workunit.client.0.smithi038.stdout:8/729: creat d1/d4/f104 x:0 0 0 2024-02-15T18:34:57.905 INFO:tasks.workunit.client.0.smithi038.stdout:9/685: dwrite d1/dc/d4f/f46 [0,4194304] 0 2024-02-15T18:34:57.906 INFO:tasks.workunit.client.0.smithi038.stdout:6/547: symlink d2/d4/d6f/la6 0 2024-02-15T18:34:57.906 INFO:tasks.workunit.client.0.smithi038.stdout:6/548: readlink d2/d4/d2c/d39/d76/l90 0 2024-02-15T18:34:57.907 INFO:tasks.workunit.client.0.smithi038.stdout:0/644: mknod d4/d13/d2a/d35/d1e/d32/d48/ccb 0 2024-02-15T18:34:57.908 INFO:tasks.workunit.client.0.smithi038.stdout:8/730: symlink d1/d4/d5/d6/d6d/df5/l105 0 2024-02-15T18:34:57.909 INFO:tasks.workunit.client.0.smithi038.stdout:9/686: symlink d1/da/d10/d13/d3d/d43/lfa 0 2024-02-15T18:34:57.911 INFO:tasks.workunit.client.0.smithi038.stdout:0/645: rmdir d4/d13/d2a/d35/db2 39 2024-02-15T18:34:57.914 INFO:tasks.workunit.client.0.smithi038.stdout:8/731: symlink d1/d4/db1/de5/db3/l106 0 2024-02-15T18:34:57.914 INFO:tasks.workunit.client.0.smithi038.stdout:8/732: fsync d1/d4/faa 0 2024-02-15T18:34:57.914 INFO:tasks.workunit.client.0.smithi038.stdout:8/733: write d1/d4/d10/d43/de2/fed [736552,120361] 0 2024-02-15T18:34:57.914 INFO:tasks.workunit.client.0.smithi038.stdout:0/646: link d4/d13/d2a/f44 d4/d13/d2a/fcc 0 2024-02-15T18:34:57.915 INFO:tasks.workunit.client.0.smithi038.stdout:0/647: stat d4/d13/d2a 0 2024-02-15T18:34:57.915 INFO:tasks.workunit.client.0.smithi038.stdout:8/734: mkdir d1/d4/d10/d43/d107 0 2024-02-15T18:34:57.916 INFO:tasks.workunit.client.0.smithi038.stdout:2/519: sync 2024-02-15T18:34:57.916 INFO:tasks.workunit.client.0.smithi038.stdout:5/547: sync 2024-02-15T18:34:57.916 INFO:tasks.workunit.client.0.smithi038.stdout:3/526: sync 2024-02-15T18:34:57.917 INFO:tasks.workunit.client.0.smithi038.stdout:6/549: dwrite d2/f8e [0,4194304] 0 2024-02-15T18:34:57.917 INFO:tasks.workunit.client.0.smithi038.stdout:6/550: chown d2/f58 758400 1 2024-02-15T18:34:57.917 INFO:tasks.workunit.client.0.smithi038.stdout:5/548: read d2/d3/d5f/d71/f5a [4024541,92960] 0 2024-02-15T18:34:57.917 INFO:tasks.workunit.client.0.smithi038.stdout:5/549: chown d2/d3/d5f/d71/d33/f66 40 1 2024-02-15T18:34:57.923 INFO:tasks.workunit.client.0.smithi038.stdout:0/648: creat d4/d13/d2a/d35/db2/fcd x:0 0 0 2024-02-15T18:34:57.923 INFO:tasks.workunit.client.0.smithi038.stdout:2/520: dread dd/d12/d1d/d3b/d3c/d4a/f6d [0,4194304] 0 2024-02-15T18:34:57.924 INFO:tasks.workunit.client.0.smithi038.stdout:8/735: mkdir d1/d4/d10/d3d/db7/d108 0 2024-02-15T18:34:57.924 INFO:tasks.workunit.client.0.smithi038.stdout:2/521: write dd/d12/d3a/d67/f8f [3010060,119757] 0 2024-02-15T18:34:57.924 INFO:tasks.workunit.client.0.smithi038.stdout:2/522: write dd/d12/f36 [7520203,25589] 0 2024-02-15T18:34:57.924 INFO:tasks.workunit.client.0.smithi038.stdout:2/523: readlink dd/d12/d8d/d90/la9 0 2024-02-15T18:34:57.924 INFO:tasks.workunit.client.0.smithi038.stdout:3/527: symlink d2/d1b/d3c/d41/laa 0 2024-02-15T18:34:57.924 INFO:tasks.workunit.client.0.smithi038.stdout:2/524: read dd/d12/d3a/d67/f86 [216880,47128] 0 2024-02-15T18:34:57.928 INFO:tasks.workunit.client.0.smithi038.stdout:6/551: creat d2/d4/d19/d55/fa7 x:0 0 0 2024-02-15T18:34:57.933 INFO:tasks.workunit.client.0.smithi038.stdout:0/649: creat d4/db3/fce x:0 0 0 2024-02-15T18:34:57.935 INFO:tasks.workunit.client.0.smithi038.stdout:3/528: unlink d2/d1b/d3c/d41/d7a/l82 0 2024-02-15T18:34:57.937 INFO:tasks.workunit.client.0.smithi038.stdout:0/650: mknod d4/d13/d2a/d35/d1e/d32/d48/ccf 0 2024-02-15T18:34:57.937 INFO:tasks.workunit.client.0.smithi038.stdout:0/651: chown d4/d13/d2a/d35/f8b 33357092 1 2024-02-15T18:34:57.939 INFO:tasks.workunit.client.0.smithi038.stdout:3/529: mkdir d2/d7f/dab 0 2024-02-15T18:34:57.941 INFO:tasks.workunit.client.0.smithi038.stdout:8/736: dread d1/d4/d5/d6/f75 [0,4194304] 0 2024-02-15T18:34:57.942 INFO:tasks.workunit.client.0.smithi038.stdout:7/622: sync 2024-02-15T18:34:57.943 INFO:tasks.workunit.client.0.smithi038.stdout:0/652: mknod d4/d13/d2a/d35/d1e/d32/d79/d7d/cd0 0 2024-02-15T18:34:57.944 INFO:tasks.workunit.client.0.smithi038.stdout:3/530: creat d2/d1b/d4f/fac x:0 0 0 2024-02-15T18:34:57.944 INFO:tasks.workunit.client.0.smithi038.stdout:8/737: mkdir d1/d4/db1/de5/db3/d109 0 2024-02-15T18:34:57.945 INFO:tasks.workunit.client.0.smithi038.stdout:0/653: creat d4/d13/d2a/d35/d1e/d32/d79/d7d/da3/dc2/fd1 x:0 0 0 2024-02-15T18:34:57.946 INFO:tasks.workunit.client.0.smithi038.stdout:3/531: mknod d2/d25/cad 0 2024-02-15T18:34:57.946 INFO:tasks.workunit.client.0.smithi038.stdout:3/532: chown d2/d1e/l23 2362 1 2024-02-15T18:34:57.946 INFO:tasks.workunit.client.0.smithi038.stdout:3/533: write d2/d1b/f94 [882422,71077] 0 2024-02-15T18:34:57.947 INFO:tasks.workunit.client.0.smithi038.stdout:8/738: creat d1/d7/d22/da7/f10a x:0 0 0 2024-02-15T18:34:57.948 INFO:tasks.workunit.client.0.smithi038.stdout:0/654: mknod d4/d13/d2a/cd2 0 2024-02-15T18:34:57.949 INFO:tasks.workunit.client.0.smithi038.stdout:3/534: creat d2/d25/d60/fae x:0 0 0 2024-02-15T18:34:57.951 INFO:tasks.workunit.client.0.smithi038.stdout:0/655: rename d4/d13/c62 to d4/d13/d2a/d35/d1e/d32/d79/d7d/da3/cd3 0 2024-02-15T18:34:57.953 INFO:tasks.workunit.client.0.smithi038.stdout:0/656: creat d4/d13/d2a/fd4 x:0 0 0 2024-02-15T18:34:57.954 INFO:tasks.workunit.client.0.smithi038.stdout:6/552: dwrite d2/f2a [0,4194304] 0 2024-02-15T18:34:57.955 INFO:tasks.workunit.client.0.smithi038.stdout:2/525: dwrite dd/d12/d5d/d54/f97 [0,4194304] 0 2024-02-15T18:34:57.955 INFO:tasks.workunit.client.0.smithi038.stdout:2/526: chown dd/d12/d59/d2b/d7a 1305718 1 2024-02-15T18:34:57.955 INFO:tasks.workunit.client.0.smithi038.stdout:6/553: creat d2/d10/fa8 x:0 0 0 2024-02-15T18:34:57.957 INFO:tasks.workunit.client.0.smithi038.stdout:2/527: getdents dd/d12/d59/d2b/d7a 0 2024-02-15T18:34:57.959 INFO:tasks.workunit.client.0.smithi038.stdout:6/554: symlink d2/d4/d2c/d39/d76/la9 0 2024-02-15T18:34:57.959 INFO:tasks.workunit.client.0.smithi038.stdout:6/555: fsync d2/d4/d2c/f4f 0 2024-02-15T18:34:57.960 INFO:tasks.workunit.client.0.smithi038.stdout:6/556: chown d2/d4/d6c/d9d 96616909 1 2024-02-15T18:34:57.960 INFO:tasks.workunit.client.0.smithi038.stdout:2/528: getdents dd/d12 0 2024-02-15T18:34:57.962 INFO:tasks.workunit.client.0.smithi038.stdout:6/557: link d2/d4/f1e d2/d4/d19/d55/d5c/faa 0 2024-02-15T18:34:57.962 INFO:tasks.workunit.client.0.smithi038.stdout:6/558: fsync d2/d4/d28/f3a 0 2024-02-15T18:34:57.964 INFO:tasks.workunit.client.0.smithi038.stdout:2/529: creat dd/d12/d3a/d67/d4f/d5a/d9d/fac x:0 0 0 2024-02-15T18:34:57.965 INFO:tasks.workunit.client.0.smithi038.stdout:2/530: creat dd/d12/d3a/d67/fad x:0 0 0 2024-02-15T18:34:57.968 INFO:tasks.workunit.client.0.smithi038.stdout:7/623: dwrite d0/f54 [0,4194304] 0 2024-02-15T18:34:57.969 INFO:tasks.workunit.client.0.smithi038.stdout:2/531: creat dd/d12/d59/d2b/fae x:0 0 0 2024-02-15T18:34:57.970 INFO:tasks.workunit.client.0.smithi038.stdout:0/657: dwrite d4/d13/f98 [0,4194304] 0 2024-02-15T18:34:57.971 INFO:tasks.workunit.client.0.smithi038.stdout:7/624: symlink d0/ldd 0 2024-02-15T18:34:57.976 INFO:tasks.workunit.client.0.smithi038.stdout:6/559: dwrite d2/d4/d28/f69 [0,4194304] 0 2024-02-15T18:34:57.979 INFO:tasks.workunit.client.0.smithi038.stdout:2/532: truncate dd/d12/d3a/d67/f8f 2938116 0 2024-02-15T18:34:57.984 INFO:tasks.workunit.client.0.smithi038.stdout:7/625: truncate d0/db/d59/d15/f18 538853 0 2024-02-15T18:34:57.990 INFO:tasks.workunit.client.0.smithi038.stdout:6/560: rename d2/d4/d19/d55/f8a to d2/d4/d19/d78/d7a/fab 0 2024-02-15T18:34:57.991 INFO:tasks.workunit.client.0.smithi038.stdout:0/658: rmdir d4/d13/d2a/d35/d1e/d32/d48/d97/dbc 0 2024-02-15T18:34:57.993 INFO:tasks.workunit.client.0.smithi038.stdout:6/561: truncate d2/d4/d2c/f3c 4059847 0 2024-02-15T18:34:57.993 INFO:tasks.workunit.client.0.smithi038.stdout:6/562: dread - d2/d4/d19/d78/f9e zero size 2024-02-15T18:34:57.994 INFO:tasks.workunit.client.0.smithi038.stdout:6/563: chown d2/d4/d19/d78 8248 1 2024-02-15T18:34:57.997 INFO:tasks.workunit.client.0.smithi038.stdout:0/659: rename d4/d13/d2a/d35/d1e/d32/d3b/daa/fbf to d4/d13/d53/fd5 0 2024-02-15T18:34:57.997 INFO:tasks.workunit.client.0.smithi038.stdout:6/564: rmdir d2/d56/d5f/d99 39 2024-02-15T18:34:57.998 INFO:tasks.workunit.client.0.smithi038.stdout:2/533: dwrite dd/d12/d3a/d67/d4f/d5a/f6e [0,4194304] 0 2024-02-15T18:34:57.999 INFO:tasks.workunit.client.0.smithi038.stdout:0/660: link d4/d13/c57 d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/cd6 0 2024-02-15T18:34:58.000 INFO:tasks.workunit.client.0.smithi038.stdout:6/565: unlink d2/d4/f2f 0 2024-02-15T18:34:58.000 INFO:tasks.workunit.client.0.smithi038.stdout:6/566: chown d2/d4/c11 26513 1 2024-02-15T18:34:58.000 INFO:tasks.workunit.client.0.smithi038.stdout:6/567: dread - d2/d10/fa8 zero size 2024-02-15T18:34:58.004 INFO:tasks.workunit.client.0.smithi038.stdout:2/534: unlink dd/d12/l20 0 2024-02-15T18:34:58.004 INFO:tasks.workunit.client.0.smithi038.stdout:2/535: write dd/d12/d3a/d67/d4f/d5a/f7f [2071763,115131] 0 2024-02-15T18:34:58.004 INFO:tasks.workunit.client.0.smithi038.stdout:7/626: dwrite d0/db/d59/d15/f23 [0,4194304] 0 2024-02-15T18:34:58.005 INFO:tasks.workunit.client.0.smithi038.stdout:0/661: creat d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/fd7 x:0 0 0 2024-02-15T18:34:58.005 INFO:tasks.workunit.client.0.smithi038.stdout:0/662: fdatasync d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/f92 0 2024-02-15T18:34:58.006 INFO:tasks.workunit.client.0.smithi038.stdout:7/627: dread d0/db/fb1 [0,4194304] 0 2024-02-15T18:34:58.006 INFO:tasks.workunit.client.0.smithi038.stdout:7/628: write d0/d95/db8/fcc [928869,128512] 0 2024-02-15T18:34:58.007 INFO:tasks.workunit.client.0.smithi038.stdout:6/568: write d2/d4/d19/d55/d5c/faa [2917597,105684] 0 2024-02-15T18:34:58.007 INFO:tasks.workunit.client.0.smithi038.stdout:2/536: unlink dd/d12/d1d/d3b/d3c/d4a/c96 0 2024-02-15T18:34:58.010 INFO:tasks.workunit.client.0.smithi038.stdout:7/629: symlink d0/db/d19/d7d/db5/lde 0 2024-02-15T18:34:58.010 INFO:tasks.workunit.client.0.smithi038.stdout:7/630: write d0/f48 [270511,57757] 0 2024-02-15T18:34:58.012 INFO:tasks.workunit.client.0.smithi038.stdout:2/537: creat dd/d12/d8d/d90/faf x:0 0 0 2024-02-15T18:34:58.012 INFO:tasks.workunit.client.0.smithi038.stdout:7/631: mkdir d0/db/d19/d7d/daa/dd0/ddf 0 2024-02-15T18:34:58.012 INFO:tasks.workunit.client.0.smithi038.stdout:7/632: dread - d0/db/d59/d97/fc5 zero size 2024-02-15T18:34:58.019 INFO:tasks.workunit.client.0.smithi038.stdout:2/538: rename dd/d12/d1d/d3b/d3c/d4a/f6d to dd/d12/d5d/d33/fb0 0 2024-02-15T18:34:58.021 INFO:tasks.workunit.client.0.smithi038.stdout:2/539: creat dd/d12/d5d/fb1 x:0 0 0 2024-02-15T18:34:58.022 INFO:tasks.workunit.client.0.smithi038.stdout:2/540: mkdir dd/d12/d59/db2 0 2024-02-15T18:34:58.022 INFO:tasks.workunit.client.0.smithi038.stdout:2/541: dread - dd/d12/d3a/d67/d4f/f71 zero size 2024-02-15T18:34:58.022 INFO:tasks.workunit.client.0.smithi038.stdout:1/574: sync 2024-02-15T18:34:58.023 INFO:tasks.workunit.client.0.smithi038.stdout:6/569: dwrite f1 [0,4194304] 0 2024-02-15T18:34:58.024 INFO:tasks.workunit.client.0.smithi038.stdout:1/575: getdents d2/d2e 0 2024-02-15T18:34:58.030 INFO:tasks.workunit.client.0.smithi038.stdout:6/570: rename d2/d56/d75/fa4 to d2/d4/d6c/fac 0 2024-02-15T18:34:58.030 INFO:tasks.workunit.client.0.smithi038.stdout:6/571: fsync d2/d4/d2c/f37 0 2024-02-15T18:34:58.031 INFO:tasks.workunit.client.0.smithi038.stdout:6/572: mknod d2/d4/d28/cad 0 2024-02-15T18:34:58.040 INFO:tasks.workunit.client.0.smithi038.stdout:6/573: dread d2/d56/f93 [0,4194304] 0 2024-02-15T18:34:58.041 INFO:tasks.workunit.client.0.smithi038.stdout:6/574: creat d2/d4/d19/d78/fae x:0 0 0 2024-02-15T18:34:58.043 INFO:tasks.workunit.client.0.smithi038.stdout:6/575: unlink d2/d4/d19/d55/d5c/f77 0 2024-02-15T18:34:58.044 INFO:tasks.workunit.client.0.smithi038.stdout:6/576: creat d2/d56/faf x:0 0 0 2024-02-15T18:34:58.044 INFO:tasks.workunit.client.0.smithi038.stdout:6/577: fdatasync d2/d4/f1f 0 2024-02-15T18:34:58.049 INFO:tasks.workunit.client.0.smithi038.stdout:2/542: dwrite fb [0,4194304] 0 2024-02-15T18:34:58.049 INFO:tasks.workunit.client.0.smithi038.stdout:1/576: dwrite d2/d2e/d8c/fae [0,4194304] 0 2024-02-15T18:34:58.055 INFO:tasks.workunit.client.0.smithi038.stdout:1/577: rename d2/d28 to d2/d10/d19/d22/d54/dc0 0 2024-02-15T18:34:58.057 INFO:tasks.workunit.client.0.smithi038.stdout:1/578: dread d2/d10/d19/d22/f2c [0,4194304] 0 2024-02-15T18:34:58.057 INFO:tasks.workunit.client.0.smithi038.stdout:1/579: truncate d2/d10/d32/d53/fb5 426359 0 2024-02-15T18:34:58.059 INFO:tasks.workunit.client.0.smithi038.stdout:1/580: mkdir d2/d10/d19/d22/d36/d94/dc1 0 2024-02-15T18:34:58.066 INFO:tasks.workunit.client.0.smithi038.stdout:2/543: dwrite dd/d12/d3a/d5e/f87 [0,4194304] 0 2024-02-15T18:34:58.067 INFO:tasks.workunit.client.0.smithi038.stdout:4/602: sync 2024-02-15T18:34:58.069 INFO:tasks.workunit.client.0.smithi038.stdout:2/544: rmdir dd/d12/d5d 39 2024-02-15T18:34:58.069 INFO:tasks.workunit.client.0.smithi038.stdout:2/545: read - dd/d12/d5d/d54/f8a zero size 2024-02-15T18:34:58.071 INFO:tasks.workunit.client.0.smithi038.stdout:4/603: unlink d11/d1e/d20/d2b/d4d/l6a 0 2024-02-15T18:34:58.071 INFO:tasks.workunit.client.0.smithi038.stdout:2/546: creat dd/d12/d1d/d3b/d3c/d4a/fb3 x:0 0 0 2024-02-15T18:34:58.072 INFO:tasks.workunit.client.0.smithi038.stdout:4/604: unlink d11/l1c 0 2024-02-15T18:34:58.073 INFO:tasks.workunit.client.0.smithi038.stdout:2/547: mknod dd/cb4 0 2024-02-15T18:34:58.073 INFO:tasks.workunit.client.0.smithi038.stdout:4/605: mknod d11/d1e/d20/d31/d8d/ce2 0 2024-02-15T18:34:58.074 INFO:tasks.workunit.client.0.smithi038.stdout:4/606: creat d11/d1e/d20/d2b/d4d/d67/d73/fe3 x:0 0 0 2024-02-15T18:34:58.075 INFO:tasks.workunit.client.0.smithi038.stdout:4/607: symlink d11/d1e/d20/d2b/d4d/d67/d73/d8b/dca/le4 0 2024-02-15T18:34:58.076 INFO:tasks.workunit.client.0.smithi038.stdout:4/608: fsync d11/d1e/d43/d4a/f5f 0 2024-02-15T18:34:58.077 INFO:tasks.workunit.client.0.smithi038.stdout:4/609: rename d11/d1e/d20/d31/d8d/d99/fb6 to d11/d1e/d20/d31/d7a/dd8/fe5 0 2024-02-15T18:34:58.080 INFO:tasks.workunit.client.0.smithi038.stdout:4/610: write d11/d1e/d3e/f47 [3833955,58255] 0 2024-02-15T18:34:58.080 INFO:tasks.workunit.client.0.smithi038.stdout:1/581: dwrite d2/d10/d19/d22/d54/faf [0,4194304] 0 2024-02-15T18:34:58.084 INFO:tasks.workunit.client.0.smithi038.stdout:4/611: link d11/f1b d11/d1e/d20/d31/d48/d60/fe6 0 2024-02-15T18:34:58.084 INFO:tasks.workunit.client.0.smithi038.stdout:4/612: chown d11/d1e/d20/d2b/d65/ca4 1340 1 2024-02-15T18:34:58.087 INFO:tasks.workunit.client.0.smithi038.stdout:1/582: unlink d2/d10/d19/d22/d36/f4d 0 2024-02-15T18:34:58.087 INFO:tasks.workunit.client.0.smithi038.stdout:4/613: mkdir d11/d1e/d20/d31/d8d/de7 0 2024-02-15T18:34:58.088 INFO:tasks.workunit.client.0.smithi038.stdout:4/614: truncate d11/d1e/d20/d31/d48/d60/fe6 491784 0 2024-02-15T18:34:58.089 INFO:tasks.workunit.client.0.smithi038.stdout:4/615: creat d11/d1e/fe8 x:0 0 0 2024-02-15T18:34:58.089 INFO:tasks.workunit.client.0.smithi038.stdout:4/616: write d11/d1e/d20/d2b/d4d/d67/d73/fe3 [584098,103945] 0 2024-02-15T18:34:58.090 INFO:tasks.workunit.client.0.smithi038.stdout:4/617: rename le to d11/d1e/d20/d31/d8d/de7/le9 0 2024-02-15T18:34:58.094 INFO:tasks.workunit.client.0.smithi038.stdout:2/548: dwrite dd/d12/d59/f21 [0,4194304] 0 2024-02-15T18:34:58.096 INFO:tasks.workunit.client.0.smithi038.stdout:1/583: write d2/d2e/d8c/fb0 [520226,78202] 0 2024-02-15T18:34:58.096 INFO:tasks.workunit.client.0.smithi038.stdout:1/584: fsync d2/d10/fa3 0 2024-02-15T18:34:58.096 INFO:tasks.workunit.client.0.smithi038.stdout:1/585: readlink d2/d2e/l67 0 2024-02-15T18:34:58.101 INFO:tasks.workunit.client.0.smithi038.stdout:1/586: write d2/d10/d19/d22/d54/faf [4269194,98981] 0 2024-02-15T18:34:58.102 INFO:tasks.workunit.client.0.smithi038.stdout:2/549: getdents dd/d12/d3a/d67 0 2024-02-15T18:34:58.103 INFO:tasks.workunit.client.0.smithi038.stdout:4/618: dwrite d11/d1e/d43/d4a/dbd/fd7 [0,4194304] 0 2024-02-15T18:34:58.107 INFO:tasks.workunit.client.0.smithi038.stdout:2/550: mknod dd/d12/d1d/d3b/d3c/d4a/cb5 0 2024-02-15T18:34:58.111 INFO:tasks.workunit.client.0.smithi038.stdout:4/619: mknod d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/cea 0 2024-02-15T18:34:58.113 INFO:tasks.workunit.client.0.smithi038.stdout:1/587: rename d2/d10/d32/d53/fb5 to d2/d10/d19/d2f/d3e/fc2 0 2024-02-15T18:34:58.116 INFO:tasks.workunit.client.0.smithi038.stdout:1/588: mkdir d2/d10/d19/d22/d54/dc3 0 2024-02-15T18:34:58.119 INFO:tasks.workunit.client.0.smithi038.stdout:9/687: sync 2024-02-15T18:34:58.119 INFO:tasks.workunit.client.0.smithi038.stdout:0/663: sync 2024-02-15T18:34:58.119 INFO:tasks.workunit.client.0.smithi038.stdout:3/535: sync 2024-02-15T18:34:58.119 INFO:tasks.workunit.client.0.smithi038.stdout:5/550: sync 2024-02-15T18:34:58.119 INFO:tasks.workunit.client.0.smithi038.stdout:8/739: sync 2024-02-15T18:34:58.119 INFO:tasks.workunit.client.0.smithi038.stdout:9/688: stat d1/dc/d5a/fde 0 2024-02-15T18:34:58.122 INFO:tasks.workunit.client.0.smithi038.stdout:3/536: rename d2/d24/c43 to d2/d1e/d7e/caf 0 2024-02-15T18:34:58.122 INFO:tasks.workunit.client.0.smithi038.stdout:0/664: creat d4/d13/d2a/d35/d1e/d32/d48/d97/fd8 x:0 0 0 2024-02-15T18:34:58.123 INFO:tasks.workunit.client.0.smithi038.stdout:3/537: creat d2/d1b/d3c/d3a/fb0 x:0 0 0 2024-02-15T18:34:58.124 INFO:tasks.workunit.client.0.smithi038.stdout:0/665: mknod d4/d13/d2a/d35/cd9 0 2024-02-15T18:34:58.125 INFO:tasks.workunit.client.0.smithi038.stdout:8/740: dread d1/d7/d26/f84 [0,4194304] 0 2024-02-15T18:34:58.125 INFO:tasks.workunit.client.0.smithi038.stdout:3/538: creat d2/d1b/d3c/d3a/d55/fb1 x:0 0 0 2024-02-15T18:34:58.126 INFO:tasks.workunit.client.0.smithi038.stdout:0/666: mkdir d4/d13/d2a/d35/d1e/d32/d79/d7d/dda 0 2024-02-15T18:34:58.127 INFO:tasks.workunit.client.0.smithi038.stdout:8/741: creat d1/d4/d10/d43/d48/f10b x:0 0 0 2024-02-15T18:34:58.128 INFO:tasks.workunit.client.0.smithi038.stdout:3/539: getdents d2/d1b/d3c/d3a/d55/da0 0 2024-02-15T18:34:58.128 INFO:tasks.workunit.client.0.smithi038.stdout:0/667: mkdir d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/ddb 0 2024-02-15T18:34:58.129 INFO:tasks.workunit.client.0.smithi038.stdout:8/742: creat d1/d7/d22/f10c x:0 0 0 2024-02-15T18:34:58.129 INFO:tasks.workunit.client.0.smithi038.stdout:8/743: write d1/d7/d22/da7/f10a [984415,115541] 0 2024-02-15T18:34:58.130 INFO:tasks.workunit.client.0.smithi038.stdout:2/551: dwrite dd/d12/d3a/d67/fa0 [0,4194304] 0 2024-02-15T18:34:58.131 INFO:tasks.workunit.client.0.smithi038.stdout:9/689: dread d1/fa1 [0,4194304] 0 2024-02-15T18:34:58.131 INFO:tasks.workunit.client.0.smithi038.stdout:9/690: chown d1/da/d20/d58/d94/db9/dc6/d49/f4e 165931995 1 2024-02-15T18:34:58.133 INFO:tasks.workunit.client.0.smithi038.stdout:2/552: creat dd/d12/d3a/d67/d4f/fb6 x:0 0 0 2024-02-15T18:34:58.133 INFO:tasks.workunit.client.0.smithi038.stdout:2/553: getdents dd/d12/d5d/d54/d91 0 2024-02-15T18:34:58.133 INFO:tasks.workunit.client.0.smithi038.stdout:2/554: truncate dd/d12/d5d/fb1 283953 0 2024-02-15T18:34:58.133 INFO:tasks.workunit.client.0.smithi038.stdout:3/540: dread d2/d1b/d69/f6a [0,4194304] 0 2024-02-15T18:34:58.133 INFO:tasks.workunit.client.0.smithi038.stdout:9/691: truncate d1/dc/db2/fd2 1657727 0 2024-02-15T18:34:58.134 INFO:tasks.workunit.client.0.smithi038.stdout:4/620: dwrite d11/d1e/f37 [0,4194304] 0 2024-02-15T18:34:58.134 INFO:tasks.workunit.client.0.smithi038.stdout:4/621: stat d11/d1e/d20/d2b/d35 0 2024-02-15T18:34:58.135 INFO:tasks.workunit.client.0.smithi038.stdout:9/692: mkdir d1/da/d10/d13/d3d/dfb 0 2024-02-15T18:34:58.136 INFO:tasks.workunit.client.0.smithi038.stdout:0/668: write d4/d13/d2a/d35/d1e/d32/d3b/f60 [4048864,107811] 0 2024-02-15T18:34:58.136 INFO:tasks.workunit.client.0.smithi038.stdout:0/669: fdatasync d4/d13/d19/f82 0 2024-02-15T18:34:58.137 INFO:tasks.workunit.client.0.smithi038.stdout:4/622: creat d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/ddd/feb x:0 0 0 2024-02-15T18:34:58.137 INFO:tasks.workunit.client.0.smithi038.stdout:2/555: dread dd/d12/d5d/d33/fb0 [0,4194304] 0 2024-02-15T18:34:58.138 INFO:tasks.workunit.client.0.smithi038.stdout:9/693: unlink d1/dc/d2b/fa6 0 2024-02-15T18:34:58.139 INFO:tasks.workunit.client.0.smithi038.stdout:8/744: dread d1/d7/d26/f34 [0,4194304] 0 2024-02-15T18:34:58.139 INFO:tasks.workunit.client.0.smithi038.stdout:8/745: write d1/d4/d10/d3d/db7/dd5/fef [935152,102569] 0 2024-02-15T18:34:58.139 INFO:tasks.workunit.client.0.smithi038.stdout:8/746: readlink d1/d4/d10/d43/d48/lae 0 2024-02-15T18:34:58.140 INFO:tasks.workunit.client.0.smithi038.stdout:8/747: read d1/d7/d22/f5c [1271501,104846] 0 2024-02-15T18:34:58.140 INFO:tasks.workunit.client.0.smithi038.stdout:0/670: rename d4/d13/d53/f95 to d4/d13/d2a/d35/db2/fdc 0 2024-02-15T18:34:58.140 INFO:tasks.workunit.client.0.smithi038.stdout:3/541: write d2/f15 [2676809,90598] 0 2024-02-15T18:34:58.141 INFO:tasks.workunit.client.0.smithi038.stdout:4/623: mkdir d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/ddd/dec 0 2024-02-15T18:34:58.141 INFO:tasks.workunit.client.0.smithi038.stdout:9/694: write d1/dc/d2b/f3a [150020,119987] 0 2024-02-15T18:34:58.141 INFO:tasks.workunit.client.0.smithi038.stdout:8/748: creat d1/d4/d5/d77/d79/f10d x:0 0 0 2024-02-15T18:34:58.141 INFO:tasks.workunit.client.0.smithi038.stdout:4/624: chown d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/ddd/dec 0 1 2024-02-15T18:34:58.141 INFO:tasks.workunit.client.0.smithi038.stdout:9/695: write d1/da/d10/d13/d3d/fa2 [1871304,108027] 0 2024-02-15T18:34:58.141 INFO:tasks.workunit.client.0.smithi038.stdout:4/625: write d11/d1e/d20/d31/d48/fe0 [169570,124198] 0 2024-02-15T18:34:58.141 INFO:tasks.workunit.client.0.smithi038.stdout:9/696: dread - d1/da/d36/d3f/f80 zero size 2024-02-15T18:34:58.142 INFO:tasks.workunit.client.0.smithi038.stdout:9/697: read - d1/da/d10/d13/d3d/d43/f74 zero size 2024-02-15T18:34:58.144 INFO:tasks.workunit.client.0.smithi038.stdout:2/556: dread dd/d12/d5d/f28 [0,4194304] 0 2024-02-15T18:34:58.144 INFO:tasks.workunit.client.0.smithi038.stdout:2/557: dread - dd/d12/d8d/fa8 zero size 2024-02-15T18:34:58.148 INFO:tasks.workunit.client.0.smithi038.stdout:5/551: dwrite d2/d30/d57/f8d [0,4194304] 0 2024-02-15T18:34:58.148 INFO:tasks.workunit.client.0.smithi038.stdout:1/589: dwrite d2/d10/d19/d5f/fa8 [0,4194304] 0 2024-02-15T18:34:58.148 INFO:tasks.workunit.client.0.smithi038.stdout:1/590: readlink d2/d10/d19/d22/d36/l60 0 2024-02-15T18:34:58.148 INFO:tasks.workunit.client.0.smithi038.stdout:1/591: chown d2/d2e/f7d 0 1 2024-02-15T18:34:58.158 INFO:tasks.workunit.client.0.smithi038.stdout:8/749: creat d1/d4/d10/d43/d48/d52/d83/f10e x:0 0 0 2024-02-15T18:34:58.160 INFO:tasks.workunit.client.0.smithi038.stdout:0/671: truncate d4/f63 2672030 0 2024-02-15T18:34:58.161 INFO:tasks.workunit.client.0.smithi038.stdout:0/672: chown d4/d13/d53/fd5 13 1 2024-02-15T18:34:58.164 INFO:tasks.workunit.client.0.smithi038.stdout:3/542: dwrite d2/d1b/d3c/d3a/d55/d65/fa6 [0,4194304] 0 2024-02-15T18:34:58.164 INFO:tasks.workunit.client.0.smithi038.stdout:3/543: dread - d2/d1b/f9e zero size 2024-02-15T18:34:58.168 INFO:tasks.workunit.client.0.smithi038.stdout:7/633: sync 2024-02-15T18:34:58.169 INFO:tasks.workunit.client.0.smithi038.stdout:6/578: sync 2024-02-15T18:34:58.172 INFO:tasks.workunit.client.0.smithi038.stdout:4/626: creat d11/d1e/d20/d2b/d4d/d67/d73/d8b/dca/fed x:0 0 0 2024-02-15T18:34:58.172 INFO:tasks.workunit.client.0.smithi038.stdout:4/627: chown d11/d1e/d20/d97/la5 313755 1 2024-02-15T18:34:58.173 INFO:tasks.workunit.client.0.smithi038.stdout:4/628: read d11/d1e/d20/d2b/d65/f6d [1093585,62842] 0 2024-02-15T18:34:58.173 INFO:tasks.workunit.client.0.smithi038.stdout:4/629: truncate d11/d1e/d43/f63 4232200 0 2024-02-15T18:34:58.173 INFO:tasks.workunit.client.0.smithi038.stdout:9/698: rmdir d1/da/d10/d19 39 2024-02-15T18:34:58.176 INFO:tasks.workunit.client.0.smithi038.stdout:2/558: creat dd/d12/d1d/d3b/d3c/d4a/fb7 x:0 0 0 2024-02-15T18:34:58.178 INFO:tasks.workunit.client.0.smithi038.stdout:5/552: rename d2/f69 to d2/d3/f9e 0 2024-02-15T18:34:58.178 INFO:tasks.workunit.client.0.smithi038.stdout:5/553: truncate d2/d3/f6f 838761 0 2024-02-15T18:34:58.179 INFO:tasks.workunit.client.0.smithi038.stdout:0/673: rmdir d4/d13/d2a/d35/d1e/d32/d79/d7d/da3 39 2024-02-15T18:34:58.181 INFO:tasks.workunit.client.0.smithi038.stdout:3/544: creat d2/d24/fb2 x:0 0 0 2024-02-15T18:34:58.181 INFO:tasks.workunit.client.0.smithi038.stdout:3/545: fdatasync d2/d1b/d4f/fac 0 2024-02-15T18:34:58.182 INFO:tasks.workunit.client.0.smithi038.stdout:3/546: chown d2/d1b/d8c/f99 1 1 2024-02-15T18:34:58.182 INFO:tasks.workunit.client.0.smithi038.stdout:3/547: dread - d2/d1b/d4f/d92/f9c zero size 2024-02-15T18:34:58.183 INFO:tasks.workunit.client.0.smithi038.stdout:6/579: creat d2/d56/d5f/d99/fb0 x:0 0 0 2024-02-15T18:34:58.184 INFO:tasks.workunit.client.0.smithi038.stdout:9/699: creat d1/da/d36/d3f/d48/ffc x:0 0 0 2024-02-15T18:34:58.184 INFO:tasks.workunit.client.0.smithi038.stdout:4/630: read d11/d1e/f37 [2533192,110121] 0 2024-02-15T18:34:58.184 INFO:tasks.workunit.client.0.smithi038.stdout:6/580: read d2/d10/f4c [497798,62109] 0 2024-02-15T18:34:58.188 INFO:tasks.workunit.client.0.smithi038.stdout:1/592: dwrite d2/d2e/d8c/fb0 [0,4194304] 0 2024-02-15T18:34:58.188 INFO:tasks.workunit.client.0.smithi038.stdout:1/593: stat d2/d10/d19/d22 0 2024-02-15T18:34:58.188 INFO:tasks.workunit.client.0.smithi038.stdout:1/594: write d2/d10/d19/d5f/d92/fb8 [5652,106322] 0 2024-02-15T18:34:58.189 INFO:tasks.workunit.client.0.smithi038.stdout:5/554: truncate d2/d30/d52/f6a 1086804 0 2024-02-15T18:34:58.189 INFO:tasks.workunit.client.0.smithi038.stdout:5/555: chown d2/d30 167 1 2024-02-15T18:34:58.189 INFO:tasks.workunit.client.0.smithi038.stdout:3/548: creat d2/d1b/d3c/d41/fb3 x:0 0 0 2024-02-15T18:34:58.189 INFO:tasks.workunit.client.0.smithi038.stdout:2/559: dread dd/d12/d59/f21 [0,4194304] 0 2024-02-15T18:34:58.189 INFO:tasks.workunit.client.0.smithi038.stdout:2/560: dread - dd/d12/d5d/d93/f9c zero size 2024-02-15T18:34:58.189 INFO:tasks.workunit.client.0.smithi038.stdout:9/700: mkdir d1/da/d10/d13/d3d/d43/d4d/dfd 0 2024-02-15T18:34:58.190 INFO:tasks.workunit.client.0.smithi038.stdout:1/595: truncate d2/d10/d19/d22/d36/d94/f9c 729188 0 2024-02-15T18:34:58.192 INFO:tasks.workunit.client.0.smithi038.stdout:7/634: rename d0/db/l17 to d0/db/d59/d15/le0 0 2024-02-15T18:34:58.192 INFO:tasks.workunit.client.0.smithi038.stdout:7/635: dread - d0/db/d59/dc8/dd6/fa3 zero size 2024-02-15T18:34:58.192 INFO:tasks.workunit.client.0.smithi038.stdout:7/636: chown d0/db/d59/dc8/dd6/d61/cac 1693682 1 2024-02-15T18:34:58.194 INFO:tasks.workunit.client.0.smithi038.stdout:5/556: link d2/d3/d5f/d71/f5a d2/d30/f9f 0 2024-02-15T18:34:58.196 INFO:tasks.workunit.client.0.smithi038.stdout:3/549: unlink d2/f1d 0 2024-02-15T18:34:58.200 INFO:tasks.workunit.client.0.smithi038.stdout:9/701: truncate d1/da/d20/d58/d94/db9/dc6/d49/f4e 396325 0 2024-02-15T18:34:58.200 INFO:tasks.workunit.client.0.smithi038.stdout:9/702: truncate d1/da/d10/d13/d3d/d43/d4d/d8a/f8e 711564 0 2024-02-15T18:34:58.200 INFO:tasks.workunit.client.0.smithi038.stdout:7/637: creat d0/d68/fe1 x:0 0 0 2024-02-15T18:34:58.203 INFO:tasks.workunit.client.0.smithi038.stdout:2/561: dread dd/d12/d3a/d67/d4f/d5a/f60 [0,4194304] 0 2024-02-15T18:34:58.204 INFO:tasks.workunit.client.0.smithi038.stdout:6/581: dread d2/f2a [0,4194304] 0 2024-02-15T18:34:58.204 INFO:tasks.workunit.client.0.smithi038.stdout:8/750: dwrite d1/f6f [0,4194304] 0 2024-02-15T18:34:58.204 INFO:tasks.workunit.client.0.smithi038.stdout:6/582: chown d2/d4/d2c/f3c 9881 1 2024-02-15T18:34:58.206 INFO:tasks.workunit.client.0.smithi038.stdout:8/751: dread d1/d3a/f6c [0,4194304] 0 2024-02-15T18:34:58.206 INFO:tasks.workunit.client.0.smithi038.stdout:9/703: symlink d1/da/d20/d58/d76/lfe 0 2024-02-15T18:34:58.208 INFO:tasks.workunit.client.0.smithi038.stdout:8/752: write d1/d7/d22/da7/f10a [835059,8548] 0 2024-02-15T18:34:58.208 INFO:tasks.workunit.client.0.smithi038.stdout:7/638: creat d0/db/d59/dc8/dd6/d61/fe2 x:0 0 0 2024-02-15T18:34:58.211 INFO:tasks.workunit.client.0.smithi038.stdout:2/562: creat dd/d12/d1d/d3b/fb8 x:0 0 0 2024-02-15T18:34:58.212 INFO:tasks.workunit.client.0.smithi038.stdout:9/704: symlink d1/da/d36/lff 0 2024-02-15T18:34:58.213 INFO:tasks.workunit.client.0.smithi038.stdout:0/674: dwrite d4/f63 [0,4194304] 0 2024-02-15T18:34:58.213 INFO:tasks.workunit.client.0.smithi038.stdout:7/639: creat d0/db/d59/d15/fe3 x:0 0 0 2024-02-15T18:34:58.213 INFO:tasks.workunit.client.0.smithi038.stdout:8/753: symlink d1/d4/d5/d6/d6d/df5/l10f 0 2024-02-15T18:34:58.214 INFO:tasks.workunit.client.0.smithi038.stdout:2/563: mknod dd/d12/d5d/d54/d91/cb9 0 2024-02-15T18:34:58.215 INFO:tasks.workunit.client.0.smithi038.stdout:8/754: write d1/d4/d10/d43/d48/d9a/fa5 [2046051,113682] 0 2024-02-15T18:34:58.216 INFO:tasks.workunit.client.0.smithi038.stdout:9/705: link d1/da/d10/d13/d3d/d4b/lf5 d1/da/d36/d3f/d99/l100 0 2024-02-15T18:34:58.216 INFO:tasks.workunit.client.0.smithi038.stdout:0/675: truncate d4/d13/d2a/d35/d1e/d32/d48/f4c 2429274 0 2024-02-15T18:34:58.217 INFO:tasks.workunit.client.0.smithi038.stdout:2/564: creat dd/d12/d3a/d5e/d65/fba x:0 0 0 2024-02-15T18:34:58.218 INFO:tasks.workunit.client.0.smithi038.stdout:8/755: rmdir d1/d4/d10/d43/d48/d9a 39 2024-02-15T18:34:58.220 INFO:tasks.workunit.client.0.smithi038.stdout:2/565: creat dd/d12/d59/db2/fbb x:0 0 0 2024-02-15T18:34:58.221 INFO:tasks.workunit.client.0.smithi038.stdout:9/706: dread d1/da/d10/d13/d3d/d43/f70 [0,4194304] 0 2024-02-15T18:34:58.222 INFO:tasks.workunit.client.0.smithi038.stdout:4/631: dwrite d11/d1e/d20/d31/f91 [0,4194304] 0 2024-02-15T18:34:58.222 INFO:tasks.workunit.client.0.smithi038.stdout:8/756: symlink d1/d4/d10/d43/d48/d9a/dd9/l110 0 2024-02-15T18:34:58.223 INFO:tasks.workunit.client.0.smithi038.stdout:0/676: mknod d4/cdd 0 2024-02-15T18:34:58.223 INFO:tasks.workunit.client.0.smithi038.stdout:0/677: readlink d4/d13/d19/l46 0 2024-02-15T18:34:58.225 INFO:tasks.workunit.client.0.smithi038.stdout:9/707: creat d1/da/d10/d13/d3d/d43/d4d/dfd/f101 x:0 0 0 2024-02-15T18:34:58.225 INFO:tasks.workunit.client.0.smithi038.stdout:8/757: dread d1/f40 [0,4194304] 0 2024-02-15T18:34:58.226 INFO:tasks.workunit.client.0.smithi038.stdout:8/758: readlink d1/d4/d5/d31/d65/d80/lfc 0 2024-02-15T18:34:58.228 INFO:tasks.workunit.client.0.smithi038.stdout:5/557: rename d2/d3/d5f/d71 to d2/d3/da0 0 2024-02-15T18:34:58.229 INFO:tasks.workunit.client.0.smithi038.stdout:3/550: dwrite d2/f9 [0,4194304] 0 2024-02-15T18:34:58.231 INFO:tasks.workunit.client.0.smithi038.stdout:0/678: stat d4/cb5 0 2024-02-15T18:34:58.233 INFO:tasks.workunit.client.0.smithi038.stdout:1/596: dwrite d2/d39/f48 [0,4194304] 0 2024-02-15T18:34:58.233 INFO:tasks.workunit.client.0.smithi038.stdout:7/640: dread d0/db/d59/dc8/dd6/d61/d90/fab [0,4194304] 0 2024-02-15T18:34:58.233 INFO:tasks.workunit.client.0.smithi038.stdout:1/597: chown d2/d10/fa3 25314217 1 2024-02-15T18:34:58.233 INFO:tasks.workunit.client.0.smithi038.stdout:7/641: fsync d0/d1b/d80/fa9 0 2024-02-15T18:34:58.233 INFO:tasks.workunit.client.0.smithi038.stdout:4/632: rename d11/d1e/d20/d31/d8d/de7/le9 to d11/d1e/lee 0 2024-02-15T18:34:58.233 INFO:tasks.workunit.client.0.smithi038.stdout:4/633: truncate d11/d1e/d20/d31/d48/dac/fd9 689972 0 2024-02-15T18:34:58.233 INFO:tasks.workunit.client.0.smithi038.stdout:4/634: write d11/f1f [5101665,120493] 0 2024-02-15T18:34:58.234 INFO:tasks.workunit.client.0.smithi038.stdout:5/558: mknod d2/d30/d57/ca1 0 2024-02-15T18:34:58.234 INFO:tasks.workunit.client.0.smithi038.stdout:6/583: dwrite d2/d56/f9c [0,4194304] 0 2024-02-15T18:34:58.235 INFO:tasks.workunit.client.0.smithi038.stdout:6/584: write d2/d4/d2c/d39/f71 [240152,78414] 0 2024-02-15T18:34:58.235 INFO:tasks.workunit.client.0.smithi038.stdout:6/585: write d2/d56/d5f/fa1 [922767,112536] 0 2024-02-15T18:34:58.235 INFO:tasks.workunit.client.0.smithi038.stdout:6/586: dread - d2/d4/d19/d78/fae zero size 2024-02-15T18:34:58.236 INFO:tasks.workunit.client.0.smithi038.stdout:6/587: read d2/d4/d2c/f3c [2150139,43520] 0 2024-02-15T18:34:58.237 INFO:tasks.workunit.client.0.smithi038.stdout:1/598: read d2/d10/f31 [929122,3317] 0 2024-02-15T18:34:58.237 INFO:tasks.workunit.client.0.smithi038.stdout:1/599: write d2/d2e/fb6 [822209,13556] 0 2024-02-15T18:34:58.237 INFO:tasks.workunit.client.0.smithi038.stdout:7/642: rmdir d0/d1b/d30 39 2024-02-15T18:34:58.237 INFO:tasks.workunit.client.0.smithi038.stdout:7/643: chown d0/db/d59/dc8/dd6/d61/d90/dc0 841612341 1 2024-02-15T18:34:58.237 INFO:tasks.workunit.client.0.smithi038.stdout:1/600: write d2/d39/f7f [502541,118012] 0 2024-02-15T18:34:58.238 INFO:tasks.workunit.client.0.smithi038.stdout:1/601: write d2/d2e/d8c/fae [4889941,50297] 0 2024-02-15T18:34:58.239 INFO:tasks.workunit.client.0.smithi038.stdout:9/708: rename d1/dc/d4f/c32 to d1/da/d36/d3f/c102 0 2024-02-15T18:34:58.239 INFO:tasks.workunit.client.0.smithi038.stdout:2/566: dwrite dd/d12/d59/d2b/fae [0,4194304] 0 2024-02-15T18:34:58.249 INFO:tasks.workunit.client.0.smithi038.stdout:2/567: dread dd/d12/f36 [4194304,4194304] 0 2024-02-15T18:34:58.249 INFO:tasks.workunit.client.0.smithi038.stdout:2/568: chown dd/d12/d3a/d67/d4f/d5a/f7f 933 1 2024-02-15T18:34:58.250 INFO:tasks.workunit.client.0.smithi038.stdout:4/635: symlink d11/d1e/d20/d2f/lef 0 2024-02-15T18:34:58.250 INFO:tasks.workunit.client.0.smithi038.stdout:4/636: readlink d11/d1e/d20/d31/d8d/lb3 0 2024-02-15T18:34:58.252 INFO:tasks.workunit.client.0.smithi038.stdout:4/637: write d11/d1e/d3e/fce [338392,64764] 0 2024-02-15T18:34:58.253 INFO:tasks.workunit.client.0.smithi038.stdout:6/588: unlink d2/d4/d2c/f51 0 2024-02-15T18:34:58.254 INFO:tasks.workunit.client.0.smithi038.stdout:7/644: mknod d0/db/d59/d15/d29/d7e/ce4 0 2024-02-15T18:34:58.254 INFO:tasks.workunit.client.0.smithi038.stdout:7/645: chown d0/d1b/f78 63400 1 2024-02-15T18:34:58.255 INFO:tasks.workunit.client.0.smithi038.stdout:1/602: mknod d2/d10/d19/d22/d36/d94/cc4 0 2024-02-15T18:34:58.255 INFO:tasks.workunit.client.0.smithi038.stdout:1/603: stat d2/d10/d19/d5f/f97 0 2024-02-15T18:34:58.259 INFO:tasks.workunit.client.0.smithi038.stdout:3/551: rename d2/d1b/d3c/d3a/f93 to d2/d1b/d4f/fb4 0 2024-02-15T18:34:58.259 INFO:tasks.workunit.client.0.smithi038.stdout:9/709: mknod d1/da/d36/d3f/d99/c103 0 2024-02-15T18:34:58.270 INFO:tasks.workunit.client.0.smithi038.stdout:4/638: mknod d11/d1e/d20/d2b/d4d/d67/da1/cf0 0 2024-02-15T18:34:58.270 INFO:tasks.workunit.client.0.smithi038.stdout:4/639: dread - d11/d1e/d3e/f94 zero size 2024-02-15T18:34:58.270 INFO:tasks.workunit.client.0.smithi038.stdout:4/640: readlink d11/d1e/d20/d2b/d35/la3 0 2024-02-15T18:34:58.270 INFO:tasks.workunit.client.0.smithi038.stdout:4/641: dread - d11/d1e/d20/d2b/d4d/d67/dbf/fd1 zero size 2024-02-15T18:34:58.270 INFO:tasks.workunit.client.0.smithi038.stdout:4/642: write d11/d1e/d20/d2b/d4d/d67/dbf/fd1 [195985,71823] 0 2024-02-15T18:34:58.270 INFO:tasks.workunit.client.0.smithi038.stdout:7/646: creat d0/db/d19/d7d/fe5 x:0 0 0 2024-02-15T18:34:58.270 INFO:tasks.workunit.client.0.smithi038.stdout:1/604: truncate d2/d10/d19/d5f/f97 21330 0 2024-02-15T18:34:58.271 INFO:tasks.workunit.client.0.smithi038.stdout:3/552: getdents d2/d1b/d8c 0 2024-02-15T18:34:58.272 INFO:tasks.workunit.client.0.smithi038.stdout:9/710: creat d1/da/d10/f104 x:0 0 0 2024-02-15T18:34:58.273 INFO:tasks.workunit.client.0.smithi038.stdout:4/643: symlink d11/d1e/d20/d31/d8d/d99/db5/lf1 0 2024-02-15T18:34:58.273 INFO:tasks.workunit.client.0.smithi038.stdout:7/647: write d0/f9 [4815647,83550] 0 2024-02-15T18:34:58.273 INFO:tasks.workunit.client.0.smithi038.stdout:7/648: fdatasync d0/d1b/f78 0 2024-02-15T18:34:58.277 INFO:tasks.workunit.client.0.smithi038.stdout:0/679: rename d4/fb1 to d4/d13/d2a/d35/d6a/fde 0 2024-02-15T18:34:58.278 INFO:tasks.workunit.client.0.smithi038.stdout:3/553: stat d2/lc 0 2024-02-15T18:34:58.279 INFO:tasks.workunit.client.0.smithi038.stdout:9/711: truncate d1/da/d10/d13/d3d/d43/fdd 236697 0 2024-02-15T18:34:58.279 INFO:tasks.workunit.client.0.smithi038.stdout:3/554: dread d2/d1b/d69/f6a [0,4194304] 0 2024-02-15T18:34:58.281 INFO:tasks.workunit.client.0.smithi038.stdout:4/644: link d11/d1e/d20/d2f/f3d d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/ddd/ff2 0 2024-02-15T18:34:58.283 INFO:tasks.workunit.client.0.smithi038.stdout:7/649: mkdir d0/db/d19/de6 0 2024-02-15T18:34:58.283 INFO:tasks.workunit.client.0.smithi038.stdout:7/650: chown d0/db/d59/d15/fe3 0 1 2024-02-15T18:34:58.283 INFO:tasks.workunit.client.0.smithi038.stdout:1/605: symlink d2/d10/d19/d22/d54/lc5 0 2024-02-15T18:34:58.284 INFO:tasks.workunit.client.0.smithi038.stdout:6/589: rename d2/d56/l84 to d2/d56/d5f/d99/lb1 0 2024-02-15T18:34:58.284 INFO:tasks.workunit.client.0.smithi038.stdout:4/645: dread d11/d1e/d20/d31/d48/d60/fe6 [0,4194304] 0 2024-02-15T18:34:58.285 INFO:tasks.workunit.client.0.smithi038.stdout:0/680: creat d4/d13/d2a/d35/d6a/fdf x:0 0 0 2024-02-15T18:34:58.287 INFO:tasks.workunit.client.0.smithi038.stdout:9/712: mkdir d1/da/d10/d13/d3d/d43/d4d/d105 0 2024-02-15T18:34:58.287 INFO:tasks.workunit.client.0.smithi038.stdout:9/713: chown d1/da/d20/c52 703715840 1 2024-02-15T18:34:58.287 INFO:tasks.workunit.client.0.smithi038.stdout:9/714: chown d1/da/d20/f65 183 1 2024-02-15T18:34:58.287 INFO:tasks.workunit.client.0.smithi038.stdout:7/651: dread d0/db/d59/dc8/dd6/f82 [0,4194304] 0 2024-02-15T18:34:58.289 INFO:tasks.workunit.client.0.smithi038.stdout:3/555: symlink d2/d1b/d4f/d92/lb5 0 2024-02-15T18:34:58.291 INFO:tasks.workunit.client.0.smithi038.stdout:6/590: symlink d2/d56/d5f/d99/lb2 0 2024-02-15T18:34:58.293 INFO:tasks.workunit.client.0.smithi038.stdout:0/681: symlink d4/d13/d2a/d35/d1e/d32/d79/d7d/le0 0 2024-02-15T18:34:58.294 INFO:tasks.workunit.client.0.smithi038.stdout:9/715: creat d1/da/d20/f106 x:0 0 0 2024-02-15T18:34:58.295 INFO:tasks.workunit.client.0.smithi038.stdout:7/652: mknod d0/d68/ce7 0 2024-02-15T18:34:58.295 INFO:tasks.workunit.client.0.smithi038.stdout:3/556: creat d2/fb6 x:0 0 0 2024-02-15T18:34:58.296 INFO:tasks.workunit.client.0.smithi038.stdout:3/557: truncate d2/d1b/d3c/d3a/f91 267107 0 2024-02-15T18:34:58.296 INFO:tasks.workunit.client.0.smithi038.stdout:3/558: chown d2/d1b/d3c/f6b 126757339 1 2024-02-15T18:34:58.296 INFO:tasks.workunit.client.0.smithi038.stdout:7/653: write d0/db/d19/f25 [712136,10669] 0 2024-02-15T18:34:58.297 INFO:tasks.workunit.client.0.smithi038.stdout:1/606: creat d2/d10/d19/d22/d54/d80/fc6 x:0 0 0 2024-02-15T18:34:58.297 INFO:tasks.workunit.client.0.smithi038.stdout:1/607: stat d2/d10/d19/d22/d54/dc0/d3c/caa 0 2024-02-15T18:34:58.298 INFO:tasks.workunit.client.0.smithi038.stdout:6/591: mkdir d2/d10/db3 0 2024-02-15T18:34:58.298 INFO:tasks.workunit.client.0.smithi038.stdout:6/592: write d2/d56/faf [728343,84096] 0 2024-02-15T18:34:58.300 INFO:tasks.workunit.client.0.smithi038.stdout:0/682: creat d4/d13/d2a/d35/fe1 x:0 0 0 2024-02-15T18:34:58.301 INFO:tasks.workunit.client.0.smithi038.stdout:1/608: read d2/d2e/f7d [568089,67179] 0 2024-02-15T18:34:58.301 INFO:tasks.workunit.client.0.smithi038.stdout:7/654: dread d0/d1b/f78 [0,4194304] 0 2024-02-15T18:34:58.305 INFO:tasks.workunit.client.0.smithi038.stdout:9/716: rename d1/dc/d5a/d7f/dd1 to d1/da/d36/d3f/d48/d107 0 2024-02-15T18:34:58.305 INFO:tasks.workunit.client.0.smithi038.stdout:9/717: dread - d1/da/d20/d58/d94/db9/dc6/f91 zero size 2024-02-15T18:34:58.305 INFO:tasks.workunit.client.0.smithi038.stdout:9/718: write d1/da/d36/d3f/f75 [1539795,16458] 0 2024-02-15T18:34:58.307 INFO:tasks.workunit.client.0.smithi038.stdout:0/683: dread d4/d13/d19/f82 [0,4194304] 0 2024-02-15T18:34:58.307 INFO:tasks.workunit.client.0.smithi038.stdout:4/646: dwrite d11/d1e/d43/f63 [0,4194304] 0 2024-02-15T18:34:58.308 INFO:tasks.workunit.client.0.smithi038.stdout:3/559: rmdir d2/d7f 39 2024-02-15T18:34:58.311 INFO:tasks.workunit.client.0.smithi038.stdout:4/647: dread d11/d1e/d20/d31/d7a/dd8/fe5 [0,4194304] 0 2024-02-15T18:34:58.312 INFO:tasks.workunit.client.0.smithi038.stdout:1/609: mknod d2/d10/d19/d22/d36/d4f/cc7 0 2024-02-15T18:34:58.312 INFO:tasks.workunit.client.0.smithi038.stdout:6/593: rmdir d2/d10/db3 0 2024-02-15T18:34:58.312 INFO:tasks.workunit.client.0.smithi038.stdout:6/594: fsync d2/d4/d19/d78/d7a/fab 0 2024-02-15T18:34:58.314 INFO:tasks.workunit.client.0.smithi038.stdout:9/719: creat d1/da/d10/d13/d3d/d4b/daa/f108 x:0 0 0 2024-02-15T18:34:58.316 INFO:tasks.workunit.client.0.smithi038.stdout:3/560: mknod d2/d1b/d3c/cb7 0 2024-02-15T18:34:58.316 INFO:tasks.workunit.client.0.smithi038.stdout:3/561: dread - d2/d1b/d3c/f6b zero size 2024-02-15T18:34:58.317 INFO:tasks.workunit.client.0.smithi038.stdout:4/648: link d11/d1e/d20/d31/d48/lc8 d11/d1e/d20/d31/lf3 0 2024-02-15T18:34:58.321 INFO:tasks.workunit.client.0.smithi038.stdout:6/595: rename d2/d56 to d2/d4/d2c/d39/db4 0 2024-02-15T18:34:58.321 INFO:tasks.workunit.client.0.smithi038.stdout:6/596: fsync d2/d4/d19/d78/fae 0 2024-02-15T18:34:58.321 INFO:tasks.workunit.client.0.smithi038.stdout:6/597: chown d2/l61 2399803 1 2024-02-15T18:34:58.321 INFO:tasks.workunit.client.0.smithi038.stdout:4/649: rmdir d11/d1e/d20/d2b/d4d/d67 39 2024-02-15T18:34:58.321 INFO:tasks.workunit.client.0.smithi038.stdout:4/650: dread - d11/d1e/d20/d2f/d71/f7b zero size 2024-02-15T18:34:58.322 INFO:tasks.workunit.client.0.smithi038.stdout:7/655: dwrite d0/db/d59/dc8/dd6/d61/f84 [4194304,4194304] 0 2024-02-15T18:34:58.322 INFO:tasks.workunit.client.0.smithi038.stdout:7/656: fsync d0/db/d59/dc8/dd6/f82 0 2024-02-15T18:34:58.323 INFO:tasks.workunit.client.0.smithi038.stdout:6/598: symlink d2/d4/d19/d55/lb5 0 2024-02-15T18:34:58.328 INFO:tasks.workunit.client.0.smithi038.stdout:9/720: dread d1/dc/d4f/f62 [0,4194304] 0 2024-02-15T18:34:58.328 INFO:tasks.workunit.client.0.smithi038.stdout:4/651: dread d11/d1e/d20/d2f/fc9 [0,4194304] 0 2024-02-15T18:34:58.330 INFO:tasks.workunit.client.0.smithi038.stdout:7/657: creat d0/db/d59/d15/d29/fe8 x:0 0 0 2024-02-15T18:34:58.330 INFO:tasks.workunit.client.0.smithi038.stdout:7/658: chown d0/db/d59/d15/d29/d7e/ca2 5518548 1 2024-02-15T18:34:58.331 INFO:tasks.workunit.client.0.smithi038.stdout:6/599: rename d2/d4/d6c/d91 to d2/d10/db6 0 2024-02-15T18:34:58.332 INFO:tasks.workunit.client.0.smithi038.stdout:6/600: stat d2/d4/d28/c54 0 2024-02-15T18:34:58.334 INFO:tasks.workunit.client.0.smithi038.stdout:9/721: link d1/da/d10/d13/d3d/d43/d4d/l81 d1/da/d10/d13/d3d/dfb/l109 0 2024-02-15T18:34:58.334 INFO:tasks.workunit.client.0.smithi038.stdout:9/722: read - d1/da/d10/d13/d3d/d43/f74 zero size 2024-02-15T18:34:58.335 INFO:tasks.workunit.client.0.smithi038.stdout:7/659: rmdir d0/db/d59/d97 39 2024-02-15T18:34:58.336 INFO:tasks.workunit.client.0.smithi038.stdout:6/601: mknod d2/d4/d2c/cb7 0 2024-02-15T18:34:58.336 INFO:tasks.workunit.client.0.smithi038.stdout:9/723: symlink d1/da/d10/d13/d3d/d6c/l10a 0 2024-02-15T18:34:58.337 INFO:tasks.workunit.client.0.smithi038.stdout:6/602: dread d2/d4/d6f/f83 [0,4194304] 0 2024-02-15T18:34:58.338 INFO:tasks.workunit.client.0.smithi038.stdout:9/724: link d1/da/d10/f104 d1/da/d36/f10b 0 2024-02-15T18:34:58.340 INFO:tasks.workunit.client.0.smithi038.stdout:4/652: write d11/d1e/f28 [586085,119265] 0 2024-02-15T18:34:58.340 INFO:tasks.workunit.client.0.smithi038.stdout:7/660: rename d0/db/d59/l7b to d0/db/d19/d2d/le9 0 2024-02-15T18:34:58.340 INFO:tasks.workunit.client.0.smithi038.stdout:7/661: rename d0 to d0/db/d59/dc8/dd6/d61/d90/dea 22 2024-02-15T18:34:58.341 INFO:tasks.workunit.client.0.smithi038.stdout:0/684: dwrite d4/d13/d2a/f44 [0,4194304] 0 2024-02-15T18:34:58.341 INFO:tasks.workunit.client.0.smithi038.stdout:0/685: truncate d4/d13/d2a/d35/db2/fcd 391921 0 2024-02-15T18:34:58.341 INFO:tasks.workunit.client.0.smithi038.stdout:4/653: creat d11/d1e/d20/d31/d8d/d99/ff4 x:0 0 0 2024-02-15T18:34:58.342 INFO:tasks.workunit.client.0.smithi038.stdout:4/654: rmdir d11/d1e/d20/d2b/d4d/d67/d73/d8b 39 2024-02-15T18:34:58.343 INFO:tasks.workunit.client.0.smithi038.stdout:4/655: chown d11/d1e/d20/d2f/fc9 65317 1 2024-02-15T18:34:58.343 INFO:tasks.workunit.client.0.smithi038.stdout:4/656: fdatasync d11/d1e/d20/d2f/d7d/fdf 0 2024-02-15T18:34:58.343 INFO:tasks.workunit.client.0.smithi038.stdout:4/657: fdatasync d11/d1e/d20/d2b/d65/f6d 0 2024-02-15T18:34:58.344 INFO:tasks.workunit.client.0.smithi038.stdout:1/610: dwrite d2/f11 [0,4194304] 0 2024-02-15T18:34:58.349 INFO:tasks.workunit.client.0.smithi038.stdout:4/658: creat d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/ff5 x:0 0 0 2024-02-15T18:34:58.351 INFO:tasks.workunit.client.0.smithi038.stdout:4/659: read d11/d1e/d20/f76 [3716976,84442] 0 2024-02-15T18:34:58.355 INFO:tasks.workunit.client.0.smithi038.stdout:1/611: symlink d2/d10/d19/d22/d36/lc8 0 2024-02-15T18:34:58.356 INFO:tasks.workunit.client.0.smithi038.stdout:4/660: write d11/d1e/f28 [2237180,23099] 0 2024-02-15T18:34:58.359 INFO:tasks.workunit.client.0.smithi038.stdout:4/661: link d11/d1e/d20/d31/d48/d60/la9 d11/d1e/d20/d31/d48/dac/dbe/d40/lf6 0 2024-02-15T18:34:58.359 INFO:tasks.workunit.client.0.smithi038.stdout:4/662: fsync d11/d1e/d20/d2b/d35/fcd 0 2024-02-15T18:34:58.359 INFO:tasks.workunit.client.0.smithi038.stdout:4/663: stat d11/d1e/d20/d31/d48/d60/fe6 0 2024-02-15T18:34:58.360 INFO:tasks.workunit.client.0.smithi038.stdout:4/664: creat d11/d1e/d20/d2b/d4d/d67/da1/ff7 x:0 0 0 2024-02-15T18:34:58.361 INFO:tasks.workunit.client.0.smithi038.stdout:4/665: rename d11/d1e/d20/d31/lf3 to d11/d1e/d20/d31/d48/lf8 0 2024-02-15T18:34:58.362 INFO:tasks.workunit.client.0.smithi038.stdout:4/666: mknod d11/d1e/d3e/d57/cf9 0 2024-02-15T18:34:58.363 INFO:tasks.workunit.client.0.smithi038.stdout:4/667: rename d11/d1e/d3e/db1 to d11/d1e/d20/d2b/d4d/dfa 0 2024-02-15T18:34:58.363 INFO:tasks.workunit.client.0.smithi038.stdout:4/668: chown d11/d1e/d43/d4a/c52 462819 1 2024-02-15T18:34:58.365 INFO:tasks.workunit.client.0.smithi038.stdout:9/725: dwrite d1/da/d36/d93/fa4 [0,4194304] 0 2024-02-15T18:34:58.365 INFO:tasks.workunit.client.0.smithi038.stdout:4/669: creat d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/ddd/dec/ffb x:0 0 0 2024-02-15T18:34:58.367 INFO:tasks.workunit.client.0.smithi038.stdout:4/670: creat d11/d1e/d20/d2f/d71/ffc x:0 0 0 2024-02-15T18:34:58.367 INFO:tasks.workunit.client.0.smithi038.stdout:9/726: symlink d1/da/d10/d13/d3d/d4b/l10c 0 2024-02-15T18:34:58.368 INFO:tasks.workunit.client.0.smithi038.stdout:4/671: creat d11/d1e/d20/d31/ffd x:0 0 0 2024-02-15T18:34:58.371 INFO:tasks.workunit.client.0.smithi038.stdout:7/662: dwrite d0/db/ff [0,4194304] 0 2024-02-15T18:34:58.371 INFO:tasks.workunit.client.0.smithi038.stdout:6/603: dwrite d2/d4/d2c/d39/db4/f8d [0,4194304] 0 2024-02-15T18:34:58.371 INFO:tasks.workunit.client.0.smithi038.stdout:9/727: rename d1/da/d36/d3f/d48/db8/lf4 to d1/da/d10/d13/d3d/dfb/l10d 0 2024-02-15T18:34:58.372 INFO:tasks.workunit.client.0.smithi038.stdout:1/612: dread d2/d10/d62/f93 [0,4194304] 0 2024-02-15T18:34:58.372 INFO:tasks.workunit.client.0.smithi038.stdout:4/672: dread d11/d1e/d20/d2b/d4d/f95 [0,4194304] 0 2024-02-15T18:34:58.372 INFO:tasks.workunit.client.0.smithi038.stdout:0/686: dwrite d4/d13/d2a/f4b [0,4194304] 0 2024-02-15T18:34:58.372 INFO:tasks.workunit.client.0.smithi038.stdout:9/728: write d1/da/d20/f65 [3816632,55128] 0 2024-02-15T18:34:58.373 INFO:tasks.workunit.client.0.smithi038.stdout:4/673: write d11/d1e/d43/d4a/dbd/fd7 [476790,38036] 0 2024-02-15T18:34:58.373 INFO:tasks.workunit.client.0.smithi038.stdout:4/674: stat d11/d1e/d20/d97/f9f 0 2024-02-15T18:34:58.375 INFO:tasks.workunit.client.0.smithi038.stdout:5/559: sync 2024-02-15T18:34:58.375 INFO:tasks.workunit.client.0.smithi038.stdout:2/569: sync 2024-02-15T18:34:58.375 INFO:tasks.workunit.client.0.smithi038.stdout:8/759: sync 2024-02-15T18:34:58.376 INFO:tasks.workunit.client.0.smithi038.stdout:2/570: write dd/d12/d59/db2/fbb [19778,44834] 0 2024-02-15T18:34:58.376 INFO:tasks.workunit.client.0.smithi038.stdout:2/571: fsync dd/d12/d1d/d3b/d3c/f48 0 2024-02-15T18:34:58.376 INFO:tasks.workunit.client.0.smithi038.stdout:2/572: write dd/d12/d59/f21 [3853232,91615] 0 2024-02-15T18:34:58.376 INFO:tasks.workunit.client.0.smithi038.stdout:2/573: read dd/d12/d3a/d67/f27 [2681174,15165] 0 2024-02-15T18:34:58.383 INFO:tasks.workunit.client.0.smithi038.stdout:7/663: creat d0/db/d59/d15/d29/feb x:0 0 0 2024-02-15T18:34:58.383 INFO:tasks.workunit.client.0.smithi038.stdout:7/664: truncate d0/d95/f51 1199140 0 2024-02-15T18:34:58.383 INFO:tasks.workunit.client.0.smithi038.stdout:6/604: creat d2/d10/fb8 x:0 0 0 2024-02-15T18:34:58.385 INFO:tasks.workunit.client.0.smithi038.stdout:0/687: creat d4/db3/fe2 x:0 0 0 2024-02-15T18:34:58.385 INFO:tasks.workunit.client.0.smithi038.stdout:0/688: fdatasync d4/d13/d2a/d35/d1e/d32/d48/fc5 0 2024-02-15T18:34:58.386 INFO:tasks.workunit.client.0.smithi038.stdout:9/729: rmdir d1/da/d10/d13 39 2024-02-15T18:34:58.387 INFO:tasks.workunit.client.0.smithi038.stdout:1/613: dwrite d2/d10/d19/d2f/d3e/fc2 [0,4194304] 0 2024-02-15T18:34:58.394 INFO:tasks.workunit.client.0.smithi038.stdout:5/560: rename d2/d3/da0/l47 to d2/d30/la2 0 2024-02-15T18:34:58.395 INFO:tasks.workunit.client.0.smithi038.stdout:5/561: dread d2/d48/d83/d96/f85 [0,4194304] 0 2024-02-15T18:34:58.397 INFO:tasks.workunit.client.0.smithi038.stdout:8/760: symlink d1/d3a/df6/l111 0 2024-02-15T18:34:58.398 INFO:tasks.workunit.client.0.smithi038.stdout:8/761: read d1/d7/d22/f5a [3462965,3110] 0 2024-02-15T18:34:58.398 INFO:tasks.workunit.client.0.smithi038.stdout:8/762: dread - d1/d4/d10/d43/d48/d52/fba zero size 2024-02-15T18:34:58.398 INFO:tasks.workunit.client.0.smithi038.stdout:8/763: chown d1/d7/d22/da7 627317 1 2024-02-15T18:34:58.398 INFO:tasks.workunit.client.0.smithi038.stdout:4/675: rmdir d11/d1e 39 2024-02-15T18:34:58.398 INFO:tasks.workunit.client.0.smithi038.stdout:4/676: fdatasync d11/f13 0 2024-02-15T18:34:58.399 INFO:tasks.workunit.client.0.smithi038.stdout:3/562: sync 2024-02-15T18:34:58.402 INFO:tasks.workunit.client.0.smithi038.stdout:2/574: mkdir dd/d12/d3a/d67/d4f/d5a/dbc 0 2024-02-15T18:34:58.403 INFO:tasks.workunit.client.0.smithi038.stdout:9/730: dwrite d1/da/d10/d13/f61 [0,4194304] 0 2024-02-15T18:34:58.403 INFO:tasks.workunit.client.0.smithi038.stdout:7/665: readlink d0/l5e 0 2024-02-15T18:34:58.403 INFO:tasks.workunit.client.0.smithi038.stdout:9/731: write d1/dc/d4f/f47 [701501,82241] 0 2024-02-15T18:34:58.404 INFO:tasks.workunit.client.0.smithi038.stdout:6/605: symlink d2/d4/d2c/d39/db4/d5f/d99/lb9 0 2024-02-15T18:34:58.404 INFO:tasks.workunit.client.0.smithi038.stdout:6/606: write d2/d10/f4e [1200849,1054] 0 2024-02-15T18:34:58.407 INFO:tasks.workunit.client.0.smithi038.stdout:1/614: truncate d2/d2e/d8c/fae 3559122 0 2024-02-15T18:34:58.407 INFO:tasks.workunit.client.0.smithi038.stdout:0/689: creat d4/d13/d2a/d35/d1e/d32/d3b/daa/fe3 x:0 0 0 2024-02-15T18:34:58.407 INFO:tasks.workunit.client.0.smithi038.stdout:1/615: dread - d2/d10/fa3 zero size 2024-02-15T18:34:58.409 INFO:tasks.workunit.client.0.smithi038.stdout:8/764: creat d1/d4/d10/d3d/db7/f112 x:0 0 0 2024-02-15T18:34:58.409 INFO:tasks.workunit.client.0.smithi038.stdout:8/765: chown d1/d7/d15/d4b/fec 7921 1 2024-02-15T18:34:58.409 INFO:tasks.workunit.client.0.smithi038.stdout:4/677: rename d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/ddd/dec to d11/d1e/d43/d5e/dfe 0 2024-02-15T18:34:58.409 INFO:tasks.workunit.client.0.smithi038.stdout:7/666: symlink d0/db/d59/dc8/dd6/d61/d90/lec 0 2024-02-15T18:34:58.411 INFO:tasks.workunit.client.0.smithi038.stdout:4/678: read d11/d1e/d43/f63 [2594217,60030] 0 2024-02-15T18:34:58.411 INFO:tasks.workunit.client.0.smithi038.stdout:4/679: chown d11/d1e/d20/d2b/d4d/f95 21127 1 2024-02-15T18:34:58.411 INFO:tasks.workunit.client.0.smithi038.stdout:9/732: mkdir d1/dc/d38/d10e 0 2024-02-15T18:34:58.412 INFO:tasks.workunit.client.0.smithi038.stdout:3/563: dread d2/f38 [0,4194304] 0 2024-02-15T18:34:58.412 INFO:tasks.workunit.client.0.smithi038.stdout:2/575: write dd/d12/d1d/f2e [5207701,54140] 0 2024-02-15T18:34:58.412 INFO:tasks.workunit.client.0.smithi038.stdout:2/576: write dd/d12/d3a/d67/f86 [828427,117811] 0 2024-02-15T18:34:58.412 INFO:tasks.workunit.client.0.smithi038.stdout:2/577: read - dd/d12/d3a/d67/d4f/d5a/f9e zero size 2024-02-15T18:34:58.415 INFO:tasks.workunit.client.0.smithi038.stdout:0/690: mkdir d4/d13/d2a/d35/d6a/de4 0 2024-02-15T18:34:58.416 INFO:tasks.workunit.client.0.smithi038.stdout:1/616: unlink d2/cd 0 2024-02-15T18:34:58.417 INFO:tasks.workunit.client.0.smithi038.stdout:8/766: creat d1/d4/d10/d43/de2/f113 x:0 0 0 2024-02-15T18:34:58.418 INFO:tasks.workunit.client.0.smithi038.stdout:7/667: symlink d0/db/d19/d7d/db5/led 0 2024-02-15T18:34:58.421 INFO:tasks.workunit.client.0.smithi038.stdout:3/564: mkdir d2/db8 0 2024-02-15T18:34:58.422 INFO:tasks.workunit.client.0.smithi038.stdout:0/691: creat d4/d13/d2a/d35/d1e/fe5 x:0 0 0 2024-02-15T18:34:58.422 INFO:tasks.workunit.client.0.smithi038.stdout:0/692: stat d4/d13/d53/fa7 0 2024-02-15T18:34:58.422 INFO:tasks.workunit.client.0.smithi038.stdout:3/565: write d2/d1b/f9e [500576,40235] 0 2024-02-15T18:34:58.422 INFO:tasks.workunit.client.0.smithi038.stdout:3/566: stat d2/l9b 0 2024-02-15T18:34:58.422 INFO:tasks.workunit.client.0.smithi038.stdout:3/567: getdents d2/d7f/dab 0 2024-02-15T18:34:58.422 INFO:tasks.workunit.client.0.smithi038.stdout:0/693: write d4/d13/d2a/d35/d1e/d32/d48/fc5 [120310,79404] 0 2024-02-15T18:34:58.422 INFO:tasks.workunit.client.0.smithi038.stdout:0/694: write d4/db3/fe2 [84281,7438] 0 2024-02-15T18:34:58.422 INFO:tasks.workunit.client.0.smithi038.stdout:0/695: write d4/d13/d2a/d35/d1e/f20 [7633535,104338] 0 2024-02-15T18:34:58.422 INFO:tasks.workunit.client.0.smithi038.stdout:1/617: getdents d2/d10/d19/d22/d36 0 2024-02-15T18:34:58.423 INFO:tasks.workunit.client.0.smithi038.stdout:7/668: symlink d0/db/d59/dc8/dd6/d70/lee 0 2024-02-15T18:34:58.426 INFO:tasks.workunit.client.0.smithi038.stdout:3/568: mkdir d2/d24/db9 0 2024-02-15T18:34:58.427 INFO:tasks.workunit.client.0.smithi038.stdout:0/696: chown d4/c7e 1462363222 1 2024-02-15T18:34:58.428 INFO:tasks.workunit.client.0.smithi038.stdout:6/607: dwrite d2/d4/d19/d55/fa7 [0,4194304] 0 2024-02-15T18:34:58.428 INFO:tasks.workunit.client.0.smithi038.stdout:7/669: creat d0/db/d59/d15/fef x:0 0 0 2024-02-15T18:34:58.430 INFO:tasks.workunit.client.0.smithi038.stdout:0/697: link d4/d13/d2a/fcc d4/d13/d2a/d35/db2/fe6 0 2024-02-15T18:34:58.430 INFO:tasks.workunit.client.0.smithi038.stdout:0/698: fdatasync d4/d13/d2a/d35/d1e/fe5 0 2024-02-15T18:34:58.434 INFO:tasks.workunit.client.0.smithi038.stdout:6/608: symlink d2/d4/d6c/d9d/lba 0 2024-02-15T18:34:58.434 INFO:tasks.workunit.client.0.smithi038.stdout:7/670: mknod d0/db/d19/d7d/daa/dd0/cf0 0 2024-02-15T18:34:58.434 INFO:tasks.workunit.client.0.smithi038.stdout:5/562: dwrite d2/d30/d52/f6a [0,4194304] 0 2024-02-15T18:34:58.434 INFO:tasks.workunit.client.0.smithi038.stdout:1/618: dread d2/d10/d19/d22/d54/faf [0,4194304] 0 2024-02-15T18:34:58.436 INFO:tasks.workunit.client.0.smithi038.stdout:5/563: write d2/d3/da0/d33/f93 [456786,61624] 0 2024-02-15T18:34:58.436 INFO:tasks.workunit.client.0.smithi038.stdout:6/609: mknod d2/d4/d6f/cbb 0 2024-02-15T18:34:58.436 INFO:tasks.workunit.client.0.smithi038.stdout:6/610: truncate d2/d4/d2c/d39/db4/d5f/d99/fb0 389888 0 2024-02-15T18:34:58.439 INFO:tasks.workunit.client.0.smithi038.stdout:7/671: rmdir d0/db/d59/d15/d29/d8d/dc2 0 2024-02-15T18:34:58.440 INFO:tasks.workunit.client.0.smithi038.stdout:1/619: mknod d2/d10/d19/d22/d54/dba/cc9 0 2024-02-15T18:34:58.441 INFO:tasks.workunit.client.0.smithi038.stdout:7/672: creat d0/db/d59/dc8/dd6/d61/ff1 x:0 0 0 2024-02-15T18:34:58.442 INFO:tasks.workunit.client.0.smithi038.stdout:1/620: rename d2/d10/d19/la7 to d2/d10/d19/d22/lca 0 2024-02-15T18:34:58.444 INFO:tasks.workunit.client.0.smithi038.stdout:1/621: mkdir d2/d10/d19/d22/d54/dc0/dcb 0 2024-02-15T18:34:58.446 INFO:tasks.workunit.client.0.smithi038.stdout:1/622: write d2/d69/f83 [614628,77971] 0 2024-02-15T18:34:58.447 INFO:tasks.workunit.client.0.smithi038.stdout:9/733: dwrite d1/da/d36/d3f/d99/fc1 [0,4194304] 0 2024-02-15T18:34:58.450 INFO:tasks.workunit.client.0.smithi038.stdout:4/680: dwrite d11/f13 [0,4194304] 0 2024-02-15T18:34:58.451 INFO:tasks.workunit.client.0.smithi038.stdout:9/734: mknod d1/da/d10/c10f 0 2024-02-15T18:34:58.452 INFO:tasks.workunit.client.0.smithi038.stdout:4/681: mknod d11/d1e/d3e/cff 0 2024-02-15T18:34:58.452 INFO:tasks.workunit.client.0.smithi038.stdout:4/682: chown d11/f1d 344268108 1 2024-02-15T18:34:58.452 INFO:tasks.workunit.client.0.smithi038.stdout:4/683: write d11/d1e/d43/d4a/f59 [4479669,114090] 0 2024-02-15T18:34:58.462 INFO:tasks.workunit.client.0.smithi038.stdout:2/578: dwrite dd/d12/d6b/fa3 [0,4194304] 0 2024-02-15T18:34:58.462 INFO:tasks.workunit.client.0.smithi038.stdout:3/569: dwrite d2/d1b/d3c/d3a/f91 [0,4194304] 0 2024-02-15T18:34:58.468 INFO:tasks.workunit.client.0.smithi038.stdout:2/579: rename dd/d12/d59/l84 to dd/d12/d3a/d67/d4f/d5a/d9d/lbd 0 2024-02-15T18:34:58.469 INFO:tasks.workunit.client.0.smithi038.stdout:2/580: stat dd/d12/d5d/d33 0 2024-02-15T18:34:58.469 INFO:tasks.workunit.client.0.smithi038.stdout:2/581: fsync dd/d12/d3a/d67/fad 0 2024-02-15T18:34:58.478 INFO:tasks.workunit.client.0.smithi038.stdout:0/699: dwrite d4/f63 [0,4194304] 0 2024-02-15T18:34:58.478 INFO:tasks.workunit.client.0.smithi038.stdout:0/700: readlink d4/d13/d2a/d35/d1e/d32/d3b/l59 0 2024-02-15T18:34:58.478 INFO:tasks.workunit.client.0.smithi038.stdout:5/564: dwrite d2/d3/d12/f19 [0,4194304] 0 2024-02-15T18:34:58.478 INFO:tasks.workunit.client.0.smithi038.stdout:6/611: getdents d2/d4/d6c/d9d 0 2024-02-15T18:34:58.479 INFO:tasks.workunit.client.0.smithi038.stdout:5/565: write d2/d3/ff [1581032,74607] 0 2024-02-15T18:34:58.480 INFO:tasks.workunit.client.0.smithi038.stdout:3/570: write d2/d1b/f36 [20093,108498] 0 2024-02-15T18:34:58.481 INFO:tasks.workunit.client.0.smithi038.stdout:1/623: dwrite d2/d10/d19/d22/d36/d94/f9c [0,4194304] 0 2024-02-15T18:34:58.485 INFO:tasks.workunit.client.0.smithi038.stdout:9/735: getdents d1/da/d10 0 2024-02-15T18:34:58.488 INFO:tasks.workunit.client.0.smithi038.stdout:2/582: dwrite dd/d12/f80 [4194304,4194304] 0 2024-02-15T18:34:58.491 INFO:tasks.workunit.client.0.smithi038.stdout:6/612: mknod d2/d4/d19/d78/d7a/cbc 0 2024-02-15T18:34:58.493 INFO:tasks.workunit.client.0.smithi038.stdout:5/566: mknod d2/d30/ca3 0 2024-02-15T18:34:58.494 INFO:tasks.workunit.client.0.smithi038.stdout:3/571: rename d2/c5a to d2/d1b/d69/cba 0 2024-02-15T18:34:58.569 INFO:tasks.workunit.client.0.smithi038.stdout:2/583: symlink dd/d12/d6b/da5/lbe 0 2024-02-15T18:34:58.569 INFO:tasks.workunit.client.0.smithi038.stdout:2/584: dread - dd/d12/d1d/d3b/d3c/d4a/f8c zero size 2024-02-15T18:34:58.569 INFO:tasks.workunit.client.0.smithi038.stdout:5/567: mknod d2/d3/d12/d5e/d86/ca4 0 2024-02-15T18:34:58.569 INFO:tasks.workunit.client.0.smithi038.stdout:5/568: fsync d2/d3/da0/f5a 0 2024-02-15T18:34:58.569 INFO:tasks.workunit.client.0.smithi038.stdout:1/624: mknod d2/d10/d19/d2f/d3e/ccc 0 2024-02-15T18:34:58.570 INFO:tasks.workunit.client.0.smithi038.stdout:2/585: getdents dd/d12/d1d/d3b/d3c/d4a 0 2024-02-15T18:34:58.570 INFO:tasks.workunit.client.0.smithi038.stdout:5/569: creat d2/fa5 x:0 0 0 2024-02-15T18:34:58.570 INFO:tasks.workunit.client.0.smithi038.stdout:1/625: mknod d2/d10/d19/d22/d54/dc3/ccd 0 2024-02-15T18:34:58.570 INFO:tasks.workunit.client.0.smithi038.stdout:5/570: write d2/d3/f2f [814830,87455] 0 2024-02-15T18:34:58.570 INFO:tasks.workunit.client.0.smithi038.stdout:2/586: getdents dd/d12/d59/d2b 0 2024-02-15T18:34:58.570 INFO:tasks.workunit.client.0.smithi038.stdout:0/701: dwrite d4/d13/d2a/d35/d1e/f50 [0,4194304] 0 2024-02-15T18:34:58.570 INFO:tasks.workunit.client.0.smithi038.stdout:1/626: unlink d2/d2e/c3a 0 2024-02-15T18:34:58.570 INFO:tasks.workunit.client.0.smithi038.stdout:1/627: fsync d2/f73 0 2024-02-15T18:34:58.570 INFO:tasks.workunit.client.0.smithi038.stdout:3/572: read d2/d1b/d3c/d3a/d55/d65/fa6 [2481176,46752] 0 2024-02-15T18:34:58.570 INFO:tasks.workunit.client.0.smithi038.stdout:9/736: dwrite d1/da/d20/d58/d94/fc7 [0,4194304] 0 2024-02-15T18:34:58.570 INFO:tasks.workunit.client.0.smithi038.stdout:6/613: dread d2/d4/d28/f3a [0,4194304] 0 2024-02-15T18:34:58.570 INFO:tasks.workunit.client.0.smithi038.stdout:9/737: chown d1/da/d20/d58/d94/db9/dc6/f77 1683 1 2024-02-15T18:34:58.570 INFO:tasks.workunit.client.0.smithi038.stdout:2/587: link dd/d12/d59/db2/fbb dd/d12/d3a/d5e/d65/fbf 0 2024-02-15T18:34:58.570 INFO:tasks.workunit.client.0.smithi038.stdout:0/702: fsync d4/f5 0 2024-02-15T18:34:58.570 INFO:tasks.workunit.client.0.smithi038.stdout:1/628: dread d2/d10/d19/d22/d54/dc0/f43 [0,4194304] 0 2024-02-15T18:34:58.570 INFO:tasks.workunit.client.0.smithi038.stdout:3/573: mkdir d2/d1b/d3c/d3a/dbb 0 2024-02-15T18:34:58.571 INFO:tasks.workunit.client.0.smithi038.stdout:5/571: dread d2/d3/d12/f14 [0,4194304] 0 2024-02-15T18:34:58.571 INFO:tasks.workunit.client.0.smithi038.stdout:6/614: mkdir d2/d4/d6c/d9d/dbd 0 2024-02-15T18:34:58.571 INFO:tasks.workunit.client.0.smithi038.stdout:5/572: dread d2/f80 [0,4194304] 0 2024-02-15T18:34:58.571 INFO:tasks.workunit.client.0.smithi038.stdout:9/738: rename d1/c7 to d1/da/d10/d13/d3d/d43/c110 0 2024-02-15T18:34:58.571 INFO:tasks.workunit.client.0.smithi038.stdout:9/739: write d1/da/f54 [4704340,88044] 0 2024-02-15T18:34:58.571 INFO:tasks.workunit.client.0.smithi038.stdout:1/629: dread d2/d39/f8f [4194304,4194304] 0 2024-02-15T18:34:58.571 INFO:tasks.workunit.client.0.smithi038.stdout:3/574: creat d2/d1e/d7e/fbc x:0 0 0 2024-02-15T18:34:58.571 INFO:tasks.workunit.client.0.smithi038.stdout:5/573: mknod d2/d3/da0/d33/ca6 0 2024-02-15T18:34:58.571 INFO:tasks.workunit.client.0.smithi038.stdout:0/703: rename d4/d13/d2a/d35/d1e/l27 to d4/d13/d2a/d35/d1e/d32/d48/d9a/le7 0 2024-02-15T18:34:58.571 INFO:tasks.workunit.client.0.smithi038.stdout:3/575: creat d2/d1b/d3c/d42/fbd x:0 0 0 2024-02-15T18:34:58.571 INFO:tasks.workunit.client.0.smithi038.stdout:5/574: mknod d2/d48/d83/d96/d8e/ca7 0 2024-02-15T18:34:58.571 INFO:tasks.workunit.client.0.smithi038.stdout:0/704: mknod d4/d13/d2a/d35/db2/ce8 0 2024-02-15T18:34:58.571 INFO:tasks.workunit.client.0.smithi038.stdout:5/575: write d2/d30/d52/f8b [1574868,3836] 0 2024-02-15T18:34:58.571 INFO:tasks.workunit.client.0.smithi038.stdout:5/576: fsync d2/f1a 0 2024-02-15T18:34:58.571 INFO:tasks.workunit.client.0.smithi038.stdout:1/630: read d2/d10/d19/d22/d54/dc0/f43 [413179,57166] 0 2024-02-15T18:34:58.572 INFO:tasks.workunit.client.0.smithi038.stdout:5/577: write d2/fc [3414507,55996] 0 2024-02-15T18:34:58.572 INFO:tasks.workunit.client.0.smithi038.stdout:5/578: chown d2/d3/d12/fa 5160739 1 2024-02-15T18:34:58.572 INFO:tasks.workunit.client.0.smithi038.stdout:3/576: rename d2/d1b/d3c/d41/l70 to d2/d1b/lbe 0 2024-02-15T18:34:58.572 INFO:tasks.workunit.client.0.smithi038.stdout:1/631: rmdir d2/d10/d19/d22/d54/dc0/dcb 0 2024-02-15T18:34:58.572 INFO:tasks.workunit.client.0.smithi038.stdout:1/632: write d2/d10/d19/d22/f57 [5216714,73160] 0 2024-02-15T18:34:58.572 INFO:tasks.workunit.client.0.smithi038.stdout:5/579: creat d2/d3/d12/fa8 x:0 0 0 2024-02-15T18:34:58.572 INFO:tasks.workunit.client.0.smithi038.stdout:5/580: read - d2/fa5 zero size 2024-02-15T18:34:58.572 INFO:tasks.workunit.client.0.smithi038.stdout:1/633: symlink d2/d10/d19/d22/d54/d80/lce 0 2024-02-15T18:34:58.572 INFO:tasks.workunit.client.0.smithi038.stdout:5/581: write d2/d3/d12/d5e/f8f [4052621,125181] 0 2024-02-15T18:34:58.572 INFO:tasks.workunit.client.0.smithi038.stdout:5/582: creat d2/d3/d12/d5e/d86/fa9 x:0 0 0 2024-02-15T18:34:58.572 INFO:tasks.workunit.client.0.smithi038.stdout:5/583: write d2/f1a [411739,39351] 0 2024-02-15T18:34:58.572 INFO:tasks.workunit.client.0.smithi038.stdout:0/705: dread d4/d13/d2a/d35/d1e/d32/d48/d97/faf [4194304,4194304] 0 2024-02-15T18:34:58.572 INFO:tasks.workunit.client.0.smithi038.stdout:5/584: creat d2/d3/d12/d5e/d86/faa x:0 0 0 2024-02-15T18:34:58.572 INFO:tasks.workunit.client.0.smithi038.stdout:5/585: write d2/d3/d12/f14 [4628633,20304] 0 2024-02-15T18:34:58.573 INFO:tasks.workunit.client.0.smithi038.stdout:5/586: stat d2/d64/d99 0 2024-02-15T18:34:58.573 INFO:tasks.workunit.client.0.smithi038.stdout:0/706: rename d4/d13/d2a/d35/d6a/c72 to d4/d13/d2a/d35/d1e/d32/d48/d9a/ce9 0 2024-02-15T18:34:58.573 INFO:tasks.workunit.client.0.smithi038.stdout:0/707: fdatasync d4/d13/d19/fb0 0 2024-02-15T18:34:58.573 INFO:tasks.workunit.client.0.smithi038.stdout:5/587: unlink d2/d3/da0/f37 0 2024-02-15T18:34:58.573 INFO:tasks.workunit.client.0.smithi038.stdout:0/708: truncate d4/d13/d19/f91 1758417 0 2024-02-15T18:34:58.573 INFO:tasks.workunit.client.0.smithi038.stdout:5/588: symlink d2/d3/da0/d33/lab 0 2024-02-15T18:34:58.573 INFO:tasks.workunit.client.0.smithi038.stdout:0/709: mknod d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/ddb/cea 0 2024-02-15T18:34:58.573 INFO:tasks.workunit.client.0.smithi038.stdout:5/589: link d2/d3/d12/d5e/f8f d2/d64/d99/fac 0 2024-02-15T18:34:58.573 INFO:tasks.workunit.client.0.smithi038.stdout:5/590: chown d2/d3/d12/d5e/l91 4 1 2024-02-15T18:34:58.573 INFO:tasks.workunit.client.0.smithi038.stdout:5/591: write d2/d64/f8a [1569127,74215] 0 2024-02-15T18:34:58.573 INFO:tasks.workunit.client.0.smithi038.stdout:6/615: dwrite d2/d10/f49 [4194304,4194304] 0 2024-02-15T18:34:58.573 INFO:tasks.workunit.client.0.smithi038.stdout:9/740: dwrite d1/dc/d4f/f62 [0,4194304] 0 2024-02-15T18:34:58.573 INFO:tasks.workunit.client.0.smithi038.stdout:0/710: creat d4/d13/d2a/d35/feb x:0 0 0 2024-02-15T18:34:58.573 INFO:tasks.workunit.client.0.smithi038.stdout:1/634: dwrite d2/d2e/d8c/fa0 [4194304,4194304] 0 2024-02-15T18:34:58.573 INFO:tasks.workunit.client.0.smithi038.stdout:1/635: stat d2/d10/fa3 0 2024-02-15T18:34:58.573 INFO:tasks.workunit.client.0.smithi038.stdout:6/616: symlink d2/d4/d19/d78/d7a/lbe 0 2024-02-15T18:34:58.574 INFO:tasks.workunit.client.0.smithi038.stdout:6/617: dread - d2/d10/fa8 zero size 2024-02-15T18:34:58.574 INFO:tasks.workunit.client.0.smithi038.stdout:6/618: readlink d2/d4/d6c/d9d/lba 0 2024-02-15T18:34:58.574 INFO:tasks.workunit.client.0.smithi038.stdout:5/592: write d2/d30/d52/f6a [1677419,89847] 0 2024-02-15T18:34:58.574 INFO:tasks.workunit.client.0.smithi038.stdout:1/636: dread d2/d39/f48 [0,4194304] 0 2024-02-15T18:34:58.574 INFO:tasks.workunit.client.0.smithi038.stdout:9/741: creat d1/da/d10/d13/d3d/d4b/daa/f111 x:0 0 0 2024-02-15T18:34:58.574 INFO:tasks.workunit.client.0.smithi038.stdout:9/742: chown d1/da/d10/d13/d3d/d43/c64 119717537 1 2024-02-15T18:34:58.574 INFO:tasks.workunit.client.0.smithi038.stdout:0/711: dwrite d4/d13/d2a/f4e [0,4194304] 0 2024-02-15T18:34:58.579 INFO:tasks.workunit.client.0.smithi038.stdout:6/619: unlink d2/d4/d19/c97 0 2024-02-15T18:34:58.580 INFO:tasks.workunit.client.0.smithi038.stdout:5/593: creat d2/d30/d52/fad x:0 0 0 2024-02-15T18:34:58.581 INFO:tasks.workunit.client.0.smithi038.stdout:1/637: creat d2/d10/d19/fcf x:0 0 0 2024-02-15T18:34:58.582 INFO:tasks.workunit.client.0.smithi038.stdout:9/743: mknod d1/da/d10/d13/d3d/dfb/c112 0 2024-02-15T18:34:58.582 INFO:tasks.workunit.client.0.smithi038.stdout:9/744: truncate d1/da/d10/d13/d3d/d4b/daa/fab 426269 0 2024-02-15T18:34:58.582 INFO:tasks.workunit.client.0.smithi038.stdout:9/745: write d1/da/d36/d3f/d48/fb3 [424918,109460] 0 2024-02-15T18:34:58.582 INFO:tasks.workunit.client.0.smithi038.stdout:9/746: fdatasync d1/da/d10/d13/f61 0 2024-02-15T18:34:58.582 INFO:tasks.workunit.client.0.smithi038.stdout:1/638: write d2/f1e [96016,14512] 0 2024-02-15T18:34:58.583 INFO:tasks.workunit.client.0.smithi038.stdout:6/620: mkdir d2/d4/d19/d78/dbf 0 2024-02-15T18:34:58.585 INFO:tasks.workunit.client.0.smithi038.stdout:9/747: mknod d1/da/d10/d13/d3d/d43/d4d/c113 0 2024-02-15T18:34:58.585 INFO:tasks.workunit.client.0.smithi038.stdout:9/748: truncate f0 4934534 0 2024-02-15T18:34:58.585 INFO:tasks.workunit.client.0.smithi038.stdout:9/749: fdatasync d1/da/d10/d13/d3d/d4b/f72 0 2024-02-15T18:34:58.586 INFO:tasks.workunit.client.0.smithi038.stdout:9/750: write d1/da/d10/d13/d3d/d43/d4d/d8a/f8e [1080896,27843] 0 2024-02-15T18:34:58.586 INFO:tasks.workunit.client.0.smithi038.stdout:9/751: write d1/da/d10/d19/fac [770267,5351] 0 2024-02-15T18:34:58.586 INFO:tasks.workunit.client.0.smithi038.stdout:9/752: truncate d1/da/d36/d3f/d48/fb3 1211078 0 2024-02-15T18:34:58.586 INFO:tasks.workunit.client.0.smithi038.stdout:9/753: dread - d1/da/d20/f106 zero size 2024-02-15T18:34:58.587 INFO:tasks.workunit.client.0.smithi038.stdout:1/639: mkdir d2/d2e/dd0 0 2024-02-15T18:34:58.588 INFO:tasks.workunit.client.0.smithi038.stdout:6/621: mkdir d2/d4/d19/d55/dc0 0 2024-02-15T18:34:58.589 INFO:tasks.workunit.client.0.smithi038.stdout:6/622: creat d2/d4/d19/fc1 x:0 0 0 2024-02-15T18:34:58.596 INFO:tasks.workunit.client.0.smithi038.stdout:6/623: mkdir d2/d4/d2c/d39/d76/dc2 0 2024-02-15T18:34:58.596 INFO:tasks.workunit.client.0.smithi038.stdout:0/712: dwrite d4/d13/d2a/d35/d6a/fdf [0,4194304] 0 2024-02-15T18:34:58.600 INFO:tasks.workunit.client.0.smithi038.stdout:5/594: dread d2/d30/d52/f8b [0,4194304] 0 2024-02-15T18:34:58.602 INFO:tasks.workunit.client.0.smithi038.stdout:5/595: creat d2/d3/da0/fae x:0 0 0 2024-02-15T18:34:58.605 INFO:tasks.workunit.client.0.smithi038.stdout:0/713: dread d4/d13/d2a/f4b [0,4194304] 0 2024-02-15T18:34:58.609 INFO:tasks.workunit.client.0.smithi038.stdout:9/754: dwrite d1/da/d20/f65 [0,4194304] 0 2024-02-15T18:34:58.609 INFO:tasks.workunit.client.0.smithi038.stdout:9/755: chown d1/da/d10/d82 116 1 2024-02-15T18:34:58.609 INFO:tasks.workunit.client.0.smithi038.stdout:5/596: dread d2/d64/d99/fac [0,4194304] 0 2024-02-15T18:34:58.609 INFO:tasks.workunit.client.0.smithi038.stdout:1/640: dwrite d2/d10/d19/d22/f2c [0,4194304] 0 2024-02-15T18:34:58.609 INFO:tasks.workunit.client.0.smithi038.stdout:1/641: fdatasync d2/d10/d19/d22/d54/d80/fc6 0 2024-02-15T18:34:58.611 INFO:tasks.workunit.client.0.smithi038.stdout:5/597: mknod d2/d30/d52/caf 0 2024-02-15T18:34:58.612 INFO:tasks.workunit.client.0.smithi038.stdout:5/598: dread d2/d3/da0/f26 [0,4194304] 0 2024-02-15T18:34:58.612 INFO:tasks.workunit.client.0.smithi038.stdout:5/599: chown d2/d48/f74 142151 1 2024-02-15T18:34:58.612 INFO:tasks.workunit.client.0.smithi038.stdout:5/600: read - d2/d3/d12/f8c zero size 2024-02-15T18:34:58.615 INFO:tasks.workunit.client.0.smithi038.stdout:5/601: symlink d2/d64/lb0 0 2024-02-15T18:34:58.615 INFO:tasks.workunit.client.0.smithi038.stdout:5/602: chown d2/d3/d12/d5e/d86/l87 118 1 2024-02-15T18:34:58.616 INFO:tasks.workunit.client.0.smithi038.stdout:5/603: chown d2/d3/d5f/f82 4165 1 2024-02-15T18:34:58.616 INFO:tasks.workunit.client.0.smithi038.stdout:5/604: chown d2/d30/d52/fad 449515 1 2024-02-15T18:34:58.616 INFO:tasks.workunit.client.0.smithi038.stdout:8/767: sync 2024-02-15T18:34:58.616 INFO:tasks.workunit.client.0.smithi038.stdout:9/756: dread d1/da/d10/d13/d3d/d4b/daa/fc0 [0,4194304] 0 2024-02-15T18:34:58.616 INFO:tasks.workunit.client.0.smithi038.stdout:5/605: symlink d2/d30/d52/lb1 0 2024-02-15T18:34:58.616 INFO:tasks.workunit.client.0.smithi038.stdout:5/606: chown d2/d64/f8a 111013 1 2024-02-15T18:34:58.617 INFO:tasks.workunit.client.0.smithi038.stdout:6/624: dwrite d2/d4/d2c/d39/db4/faf [0,4194304] 0 2024-02-15T18:34:58.617 INFO:tasks.workunit.client.0.smithi038.stdout:8/768: link d1/c13 d1/d4/d10/d43/d107/c114 0 2024-02-15T18:34:58.618 INFO:tasks.workunit.client.0.smithi038.stdout:9/757: getdents d1/da/d36/d3f/d48 0 2024-02-15T18:34:58.620 INFO:tasks.workunit.client.0.smithi038.stdout:5/607: truncate d2/d30/d57/f8d 485994 0 2024-02-15T18:34:58.624 INFO:tasks.workunit.client.0.smithi038.stdout:0/714: dread d4/d13/d19/fb0 [0,4194304] 0 2024-02-15T18:34:58.627 INFO:tasks.workunit.client.0.smithi038.stdout:8/769: creat d1/d4/d5/d31/d65/f115 x:0 0 0 2024-02-15T18:34:58.627 INFO:tasks.workunit.client.0.smithi038.stdout:9/758: dread d1/da/d36/d3f/d48/d107/f55 [0,4194304] 0 2024-02-15T18:34:58.628 INFO:tasks.workunit.client.0.smithi038.stdout:9/759: chown d1/da/d36/cb4 1712 1 2024-02-15T18:34:58.628 INFO:tasks.workunit.client.0.smithi038.stdout:5/608: mknod d2/d3/d12/d5e/cb2 0 2024-02-15T18:34:58.628 INFO:tasks.workunit.client.0.smithi038.stdout:2/588: sync 2024-02-15T18:34:58.628 INFO:tasks.workunit.client.0.smithi038.stdout:4/684: sync 2024-02-15T18:34:58.629 INFO:tasks.workunit.client.0.smithi038.stdout:7/673: sync 2024-02-15T18:34:58.629 INFO:tasks.workunit.client.0.smithi038.stdout:2/589: truncate dd/d12/d5d/d33/fb0 4952897 0 2024-02-15T18:34:58.629 INFO:tasks.workunit.client.0.smithi038.stdout:4/685: write d11/d1e/d20/d2b/d4d/d67/d73/fe3 [1285545,62864] 0 2024-02-15T18:34:58.630 INFO:tasks.workunit.client.0.smithi038.stdout:8/770: mknod d1/d7/c116 0 2024-02-15T18:34:58.631 INFO:tasks.workunit.client.0.smithi038.stdout:5/609: rename d2/d30/l32 to d2/d30/lb3 0 2024-02-15T18:34:58.632 INFO:tasks.workunit.client.0.smithi038.stdout:7/674: mknod d0/db/d19/d7d/dc9/cf2 0 2024-02-15T18:34:58.632 INFO:tasks.workunit.client.0.smithi038.stdout:7/675: readlink d0/db/d59/dc8/dd6/d61/l6f 0 2024-02-15T18:34:58.632 INFO:tasks.workunit.client.0.smithi038.stdout:7/676: read - d0/db/d59/dc8/dd6/d61/d90/dc0/fce zero size 2024-02-15T18:34:58.633 INFO:tasks.workunit.client.0.smithi038.stdout:4/686: dread d11/d1e/d3e/d57/f82 [0,4194304] 0 2024-02-15T18:34:58.633 INFO:tasks.workunit.client.0.smithi038.stdout:4/687: chown d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/ff5 59592 1 2024-02-15T18:34:58.633 INFO:tasks.workunit.client.0.smithi038.stdout:4/688: fsync d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/ddd/feb 0 2024-02-15T18:34:58.633 INFO:tasks.workunit.client.0.smithi038.stdout:4/689: chown d11/f1d 43 1 2024-02-15T18:34:58.633 INFO:tasks.workunit.client.0.smithi038.stdout:4/690: fsync d11/d1e/d20/d2f/f3d 0 2024-02-15T18:34:58.633 INFO:tasks.workunit.client.0.smithi038.stdout:8/771: dread d1/d7/fb0 [0,4194304] 0 2024-02-15T18:34:58.633 INFO:tasks.workunit.client.0.smithi038.stdout:0/715: mknod d4/cec 0 2024-02-15T18:34:58.634 INFO:tasks.workunit.client.0.smithi038.stdout:5/610: getdents d2/d48/d83 0 2024-02-15T18:34:58.634 INFO:tasks.workunit.client.0.smithi038.stdout:0/716: chown d4/d13/d19/fb0 353476111 1 2024-02-15T18:34:58.634 INFO:tasks.workunit.client.0.smithi038.stdout:5/611: chown d2/d3/d12/l6b 23103 1 2024-02-15T18:34:58.634 INFO:tasks.workunit.client.0.smithi038.stdout:7/677: rmdir d0/db/d59/d15/d29/d7e 39 2024-02-15T18:34:58.634 INFO:tasks.workunit.client.0.smithi038.stdout:7/678: write d0/d1b/d30/d50/f74 [1759559,112777] 0 2024-02-15T18:34:58.636 INFO:tasks.workunit.client.0.smithi038.stdout:9/760: dread d1/da/d20/d58/d94/db9/dc6/f9e [0,4194304] 0 2024-02-15T18:34:58.636 INFO:tasks.workunit.client.0.smithi038.stdout:9/761: write d1/da/d20/d58/d94/db9/dc6/d49/f21 [507765,48748] 0 2024-02-15T18:34:58.636 INFO:tasks.workunit.client.0.smithi038.stdout:4/691: link d11/d1e/d20/d31/d7a/dd8/fe5 d11/f100 0 2024-02-15T18:34:58.636 INFO:tasks.workunit.client.0.smithi038.stdout:4/692: write d11/d1e/d20/d31/d8d/d99/db5/fcb [711247,40419] 0 2024-02-15T18:34:58.637 INFO:tasks.workunit.client.0.smithi038.stdout:8/772: unlink d1/d4/d5/d6/d6d/df5/l105 0 2024-02-15T18:34:58.642 INFO:tasks.workunit.client.0.smithi038.stdout:5/612: dread d2/d3/d5f/f82 [0,4194304] 0 2024-02-15T18:34:58.642 INFO:tasks.workunit.client.0.smithi038.stdout:0/717: truncate d4/d13/d19/f82 857509 0 2024-02-15T18:34:58.643 INFO:tasks.workunit.client.0.smithi038.stdout:8/773: rename d1/d7/d15/d6a/f86 to d1/d3a/d58/f117 0 2024-02-15T18:34:58.644 INFO:tasks.workunit.client.0.smithi038.stdout:8/774: fsync d1/d7/d22/da7/f100 0 2024-02-15T18:34:58.644 INFO:tasks.workunit.client.0.smithi038.stdout:5/613: creat d2/d3/da0/fb4 x:0 0 0 2024-02-15T18:34:58.645 INFO:tasks.workunit.client.0.smithi038.stdout:0/718: creat d4/d13/d2a/d35/d1e/fed x:0 0 0 2024-02-15T18:34:58.648 INFO:tasks.workunit.client.0.smithi038.stdout:8/775: rename d1/d4/d10/d43/d48/d52/d83/f10e to d1/d4/d10/d91/f118 0 2024-02-15T18:34:58.648 INFO:tasks.workunit.client.0.smithi038.stdout:8/776: chown d1/d3a/df6 64904 1 2024-02-15T18:34:58.654 INFO:tasks.workunit.client.0.smithi038.stdout:5/614: dread d2/d30/d52/f7b [0,4194304] 0 2024-02-15T18:34:58.654 INFO:tasks.workunit.client.0.smithi038.stdout:0/719: creat d4/d13/d2a/d35/d1e/d32/d79/d7d/dda/fee x:0 0 0 2024-02-15T18:34:58.655 INFO:tasks.workunit.client.0.smithi038.stdout:5/615: write d2/d3/d5f/f82 [1597362,30738] 0 2024-02-15T18:34:58.656 INFO:tasks.workunit.client.0.smithi038.stdout:8/777: creat d1/d4/d10/d43/f119 x:0 0 0 2024-02-15T18:34:58.657 INFO:tasks.workunit.client.0.smithi038.stdout:0/720: mknod d4/d13/cef 0 2024-02-15T18:34:58.657 INFO:tasks.workunit.client.0.smithi038.stdout:0/721: chown d4/d13/d2a/d35/d1e/d32/fb6 67021623 1 2024-02-15T18:34:58.658 INFO:tasks.workunit.client.0.smithi038.stdout:5/616: mknod d2/d48/d83/cb5 0 2024-02-15T18:34:58.659 INFO:tasks.workunit.client.0.smithi038.stdout:0/722: mknod d4/d13/d2a/d35/d1e/d32/d79/d7d/da3/dc2/cf0 0 2024-02-15T18:34:58.659 INFO:tasks.workunit.client.0.smithi038.stdout:0/723: chown d4/d13/d2a/d35/d1e/d32/d79/d7d/c81 989 1 2024-02-15T18:34:58.659 INFO:tasks.workunit.client.0.smithi038.stdout:2/590: dwrite dd/d12/d3a/d67/d4f/d5a/f6e [4194304,4194304] 0 2024-02-15T18:34:58.659 INFO:tasks.workunit.client.0.smithi038.stdout:5/617: mkdir d2/d64/d99/db6 0 2024-02-15T18:34:58.660 INFO:tasks.workunit.client.0.smithi038.stdout:5/618: dread d2/d30/d57/f90 [0,4194304] 0 2024-02-15T18:34:58.661 INFO:tasks.workunit.client.0.smithi038.stdout:0/724: getdents d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f 0 2024-02-15T18:34:58.662 INFO:tasks.workunit.client.0.smithi038.stdout:2/591: creat dd/d12/d3a/d67/fc0 x:0 0 0 2024-02-15T18:34:58.662 INFO:tasks.workunit.client.0.smithi038.stdout:5/619: symlink d2/d3/da0/d33/lb7 0 2024-02-15T18:34:58.664 INFO:tasks.workunit.client.0.smithi038.stdout:0/725: rename d4/d13/d2a/d35/d1e/d32/d79/d7d/da3/dc2/l87 to d4/d13/d2a/d35/d6a/lf1 0 2024-02-15T18:34:58.664 INFO:tasks.workunit.client.0.smithi038.stdout:0/726: write d4/d13/d2a/d35/fe1 [484754,60650] 0 2024-02-15T18:34:58.664 INFO:tasks.workunit.client.0.smithi038.stdout:0/727: chown d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f 3443030 1 2024-02-15T18:34:58.664 INFO:tasks.workunit.client.0.smithi038.stdout:2/592: rmdir dd/d12/d59/db2 39 2024-02-15T18:34:58.665 INFO:tasks.workunit.client.0.smithi038.stdout:0/728: mknod d4/d13/d2a/d35/d1e/d32/cf2 0 2024-02-15T18:34:58.666 INFO:tasks.workunit.client.0.smithi038.stdout:2/593: truncate dd/d12/d3a/d67/f8f 2590774 0 2024-02-15T18:34:58.667 INFO:tasks.workunit.client.0.smithi038.stdout:2/594: creat dd/d12/d3a/d67/fc1 x:0 0 0 2024-02-15T18:34:58.667 INFO:tasks.workunit.client.0.smithi038.stdout:8/778: write d1/d4/d5/d6/f75 [1135877,92107] 0 2024-02-15T18:34:58.668 INFO:tasks.workunit.client.0.smithi038.stdout:5/620: write d2/d30/f9f [528717,121588] 0 2024-02-15T18:34:58.669 INFO:tasks.workunit.client.0.smithi038.stdout:8/779: symlink d1/d4/d10/d3d/db7/l11a 0 2024-02-15T18:34:58.669 INFO:tasks.workunit.client.0.smithi038.stdout:8/780: chown d1/d4/d10/d3d/fe6 3782926 1 2024-02-15T18:34:58.669 INFO:tasks.workunit.client.0.smithi038.stdout:8/781: write d1/d7/d15/d4b/fc7 [431158,113794] 0 2024-02-15T18:34:58.669 INFO:tasks.workunit.client.0.smithi038.stdout:8/782: chown d1/d4/d10/d43/d48/d52 5717042 1 2024-02-15T18:34:58.669 INFO:tasks.workunit.client.0.smithi038.stdout:8/783: fdatasync d1/d4/d5/d6/f6e 0 2024-02-15T18:34:58.670 INFO:tasks.workunit.client.0.smithi038.stdout:5/621: symlink d2/d3/d5f/lb8 0 2024-02-15T18:34:58.673 INFO:tasks.workunit.client.0.smithi038.stdout:0/729: dread d4/f5 [0,4194304] 0 2024-02-15T18:34:58.673 INFO:tasks.workunit.client.0.smithi038.stdout:9/762: dwrite d1/da/d10/d13/d3d/d43/f74 [0,4194304] 0 2024-02-15T18:34:58.673 INFO:tasks.workunit.client.0.smithi038.stdout:9/763: stat d1/da/lf6 0 2024-02-15T18:34:58.673 INFO:tasks.workunit.client.0.smithi038.stdout:9/764: write d1/da/d36/d3f/f80 [868834,55937] 0 2024-02-15T18:34:58.673 INFO:tasks.workunit.client.0.smithi038.stdout:7/679: dwrite d0/db/fb1 [0,4194304] 0 2024-02-15T18:34:58.673 INFO:tasks.workunit.client.0.smithi038.stdout:7/680: read - d0/db/d59/d15/d29/fe8 zero size 2024-02-15T18:34:58.673 INFO:tasks.workunit.client.0.smithi038.stdout:8/784: rename d1/d4/d10/d43/de2 to d1/d7/d26/d11b 0 2024-02-15T18:34:58.674 INFO:tasks.workunit.client.0.smithi038.stdout:4/693: dwrite d11/f1f [0,4194304] 0 2024-02-15T18:34:58.677 INFO:tasks.workunit.client.0.smithi038.stdout:0/730: creat d4/d13/d2a/d35/d1e/d32/d3b/ff3 x:0 0 0 2024-02-15T18:34:58.681 INFO:tasks.workunit.client.0.smithi038.stdout:8/785: dread d1/fb [4194304,4194304] 0 2024-02-15T18:34:58.685 INFO:tasks.workunit.client.0.smithi038.stdout:7/681: dread d0/db/d59/fbb [0,4194304] 0 2024-02-15T18:34:58.693 INFO:tasks.workunit.client.0.smithi038.stdout:2/595: dwrite dd/d12/d3a/d67/d4f/d5a/f7f [0,4194304] 0 2024-02-15T18:34:58.693 INFO:tasks.workunit.client.0.smithi038.stdout:5/622: dread d2/f6d [0,4194304] 0 2024-02-15T18:34:58.695 INFO:tasks.workunit.client.0.smithi038.stdout:5/623: read d2/d30/f73 [2558092,117352] 0 2024-02-15T18:34:58.695 INFO:tasks.workunit.client.0.smithi038.stdout:0/731: mkdir d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/df4 0 2024-02-15T18:34:58.695 INFO:tasks.workunit.client.0.smithi038.stdout:0/732: chown d4/d13/d2a/d35/d1e/d32/d79/d7d/da3/dc2/l85 5305747 1 2024-02-15T18:34:58.696 INFO:tasks.workunit.client.0.smithi038.stdout:8/786: truncate d1/d7/d26/fe1 2944105 0 2024-02-15T18:34:58.696 INFO:tasks.workunit.client.0.smithi038.stdout:7/682: chown d0/db/d59/c96 889300 1 2024-02-15T18:34:58.696 INFO:tasks.workunit.client.0.smithi038.stdout:7/683: fdatasync d0/db/d59/d15/fe3 0 2024-02-15T18:34:58.698 INFO:tasks.workunit.client.0.smithi038.stdout:2/596: rmdir dd/d12/d5d/d54/d91 39 2024-02-15T18:34:58.698 INFO:tasks.workunit.client.0.smithi038.stdout:2/597: dread - dd/d12/d1d/d3b/d3c/d4a/f8c zero size 2024-02-15T18:34:58.698 INFO:tasks.workunit.client.0.smithi038.stdout:2/598: fdatasync dd/d12/d1d/d3b/d3c/f48 0 2024-02-15T18:34:58.698 INFO:tasks.workunit.client.0.smithi038.stdout:9/765: dwrite d1/da/d36/d3f/d48/d107/f55 [0,4194304] 0 2024-02-15T18:34:58.699 INFO:tasks.workunit.client.0.smithi038.stdout:5/624: getdents d2/d30 0 2024-02-15T18:34:58.699 INFO:tasks.workunit.client.0.smithi038.stdout:5/625: chown d2/d3/da0/d33/f66 173261809 1 2024-02-15T18:34:58.702 INFO:tasks.workunit.client.0.smithi038.stdout:8/787: mkdir d1/d4/d5/d6/d6d/df5/d11c 0 2024-02-15T18:34:58.702 INFO:tasks.workunit.client.0.smithi038.stdout:0/733: dread d4/d13/f64 [0,4194304] 0 2024-02-15T18:34:58.704 INFO:tasks.workunit.client.0.smithi038.stdout:7/684: rename d0/db/d59/d15/d29/feb to d0/d1b/ff3 0 2024-02-15T18:34:58.705 INFO:tasks.workunit.client.0.smithi038.stdout:3/577: sync 2024-02-15T18:34:58.707 INFO:tasks.workunit.client.0.smithi038.stdout:9/766: rmdir d1/dc/d38 39 2024-02-15T18:34:58.708 INFO:tasks.workunit.client.0.smithi038.stdout:5/626: truncate d2/d64/d99/fac 3927665 0 2024-02-15T18:34:58.708 INFO:tasks.workunit.client.0.smithi038.stdout:5/627: getdents d2/d64/d99/db6 0 2024-02-15T18:34:58.709 INFO:tasks.workunit.client.0.smithi038.stdout:8/788: creat d1/d4/d10/d3d/db7/d108/f11d x:0 0 0 2024-02-15T18:34:58.709 INFO:tasks.workunit.client.0.smithi038.stdout:0/734: rmdir d4 39 2024-02-15T18:34:58.709 INFO:tasks.workunit.client.0.smithi038.stdout:0/735: chown d4/d13/d2a/d35/db2/fe6 152678 1 2024-02-15T18:34:58.710 INFO:tasks.workunit.client.0.smithi038.stdout:7/685: mknod d0/d1b/d30/cf4 0 2024-02-15T18:34:58.710 INFO:tasks.workunit.client.0.smithi038.stdout:7/686: fdatasync d0/db/d59/d15/f71 0 2024-02-15T18:34:58.712 INFO:tasks.workunit.client.0.smithi038.stdout:8/789: unlink d1/d7/d15/fb2 0 2024-02-15T18:34:58.713 INFO:tasks.workunit.client.0.smithi038.stdout:3/578: rmdir d2/d1b/d3c/d41/d7a 0 2024-02-15T18:34:58.715 INFO:tasks.workunit.client.0.smithi038.stdout:3/579: unlink d2/d24/ca9 0 2024-02-15T18:34:58.717 INFO:tasks.workunit.client.0.smithi038.stdout:3/580: link d2/f50 d2/d1e/d7e/fbf 0 2024-02-15T18:34:58.717 INFO:tasks.workunit.client.0.smithi038.stdout:3/581: fsync d2/fb6 0 2024-02-15T18:34:58.717 INFO:tasks.workunit.client.0.smithi038.stdout:3/582: chown d2/d1b/d3c/d3a/d55/l85 18 1 2024-02-15T18:34:58.718 INFO:tasks.workunit.client.0.smithi038.stdout:9/767: dread d1/da/d10/d13/f61 [0,4194304] 0 2024-02-15T18:34:58.718 INFO:tasks.workunit.client.0.smithi038.stdout:9/768: dread - d1/da/d20/d58/d94/db9/dc6/f91 zero size 2024-02-15T18:34:58.718 INFO:tasks.workunit.client.0.smithi038.stdout:5/628: dread d2/d30/d57/f8d [0,4194304] 0 2024-02-15T18:34:58.718 INFO:tasks.workunit.client.0.smithi038.stdout:5/629: chown d2/d3/d12/f14 26443962 1 2024-02-15T18:34:58.721 INFO:tasks.workunit.client.0.smithi038.stdout:9/769: rename d1/da/d10/d13/d3d/d43/d4d/l81 to d1/da/d36/d3f/l114 0 2024-02-15T18:34:58.726 INFO:tasks.workunit.client.0.smithi038.stdout:0/736: dread d4/d13/d2a/d35/d1e/d32/f76 [0,4194304] 0 2024-02-15T18:34:58.726 INFO:tasks.workunit.client.0.smithi038.stdout:3/583: read d2/f38 [2505046,114601] 0 2024-02-15T18:34:58.727 INFO:tasks.workunit.client.0.smithi038.stdout:2/599: dwrite dd/d12/d1d/f50 [8388608,4194304] 0 2024-02-15T18:34:58.729 INFO:tasks.workunit.client.0.smithi038.stdout:5/630: dread d2/fc [0,4194304] 0 2024-02-15T18:34:58.730 INFO:tasks.workunit.client.0.smithi038.stdout:2/600: mkdir dd/d12/d5d/dc2 0 2024-02-15T18:34:58.731 INFO:tasks.workunit.client.0.smithi038.stdout:5/631: mkdir d2/d3/db9 0 2024-02-15T18:34:58.732 INFO:tasks.workunit.client.0.smithi038.stdout:5/632: chown d2/d3/l92 253169 1 2024-02-15T18:34:58.732 INFO:tasks.workunit.client.0.smithi038.stdout:0/737: dread d4/d13/d2a/d35/d1e/d32/f76 [0,4194304] 0 2024-02-15T18:34:58.733 INFO:tasks.workunit.client.0.smithi038.stdout:2/601: symlink dd/d12/d1d/d3b/d3c/d73/lc3 0 2024-02-15T18:34:58.735 INFO:tasks.workunit.client.0.smithi038.stdout:2/602: unlink dd/d12/d3a/d67/d4f/d5a/f9e 0 2024-02-15T18:34:58.737 INFO:tasks.workunit.client.0.smithi038.stdout:3/584: dread d2/f15 [0,4194304] 0 2024-02-15T18:34:58.739 INFO:tasks.workunit.client.0.smithi038.stdout:3/585: creat d2/d1b/d3c/d42/fc0 x:0 0 0 2024-02-15T18:34:58.740 INFO:tasks.workunit.client.0.smithi038.stdout:9/770: dwrite d1/da/d20/d58/d76/ff3 [0,4194304] 0 2024-02-15T18:34:58.740 INFO:tasks.workunit.client.0.smithi038.stdout:9/771: fdatasync d1/da/d10/f104 0 2024-02-15T18:34:58.740 INFO:tasks.workunit.client.0.smithi038.stdout:9/772: fdatasync d1/da/d10/d13/d3d/d4b/daa/fc0 0 2024-02-15T18:34:58.741 INFO:tasks.workunit.client.0.smithi038.stdout:3/586: dread d2/d1b/d69/f6a [0,4194304] 0 2024-02-15T18:34:58.741 INFO:tasks.workunit.client.0.smithi038.stdout:3/587: write d2/d1b/d3c/d3a/d55/fb1 [285871,44208] 0 2024-02-15T18:34:58.741 INFO:tasks.workunit.client.0.smithi038.stdout:3/588: chown d2/d1b/d4f/f8d 924 1 2024-02-15T18:34:58.744 INFO:tasks.workunit.client.0.smithi038.stdout:8/790: dwrite d1/d4/d5/d6/f75 [0,4194304] 0 2024-02-15T18:34:58.744 INFO:tasks.workunit.client.0.smithi038.stdout:8/791: readlink d1/d4/d5/d31/ldf 0 2024-02-15T18:34:58.744 INFO:tasks.workunit.client.0.smithi038.stdout:8/792: dread - d1/d4/faa zero size 2024-02-15T18:34:58.746 INFO:tasks.workunit.client.0.smithi038.stdout:9/773: truncate d1/da/d10/d13/d3d/d4b/daa/fab 272150 0 2024-02-15T18:34:58.749 INFO:tasks.workunit.client.0.smithi038.stdout:8/793: mknod d1/d4/d5/d31/d65/d80/c11e 0 2024-02-15T18:34:58.751 INFO:tasks.workunit.client.0.smithi038.stdout:8/794: creat d1/d4/d10/d3d/f11f x:0 0 0 2024-02-15T18:34:58.751 INFO:tasks.workunit.client.0.smithi038.stdout:8/795: chown d1/d4/d5/d6/d6d/c63 12 1 2024-02-15T18:34:58.753 INFO:tasks.workunit.client.0.smithi038.stdout:8/796: creat d1/d4/db1/de5/db3/d109/f120 x:0 0 0 2024-02-15T18:34:58.754 INFO:tasks.workunit.client.0.smithi038.stdout:3/589: dread d2/d1b/d3c/d41/d4e/d62/f8f [0,4194304] 0 2024-02-15T18:34:58.755 INFO:tasks.workunit.client.0.smithi038.stdout:8/797: creat d1/d3a/d58/f121 x:0 0 0 2024-02-15T18:34:58.755 INFO:tasks.workunit.client.0.smithi038.stdout:8/798: fdatasync d1/d7/d26/d11b/fed 0 2024-02-15T18:34:58.755 INFO:tasks.workunit.client.0.smithi038.stdout:8/799: stat d1/d4/d10/d3d/db7/d108 0 2024-02-15T18:34:58.756 INFO:tasks.workunit.client.0.smithi038.stdout:8/800: creat d1/d7/d26/d11b/f122 x:0 0 0 2024-02-15T18:34:58.758 INFO:tasks.workunit.client.0.smithi038.stdout:8/801: mkdir d1/d4/d5/d6/d6d/df5/d123 0 2024-02-15T18:34:58.758 INFO:tasks.workunit.client.0.smithi038.stdout:8/802: fsync d1/d4/d10/d3d/f97 0 2024-02-15T18:34:58.758 INFO:tasks.workunit.client.0.smithi038.stdout:8/803: fdatasync d1/d4/d10/d3d/f11f 0 2024-02-15T18:34:58.758 INFO:tasks.workunit.client.0.smithi038.stdout:8/804: chown d1/d4/d5/d77/ca8 2329 1 2024-02-15T18:34:58.759 INFO:tasks.workunit.client.0.smithi038.stdout:8/805: symlink d1/d4/d10/d3d/db7/d108/l124 0 2024-02-15T18:34:58.767 INFO:tasks.workunit.client.0.smithi038.stdout:5/633: dwrite d2/f3f [0,4194304] 0 2024-02-15T18:34:58.768 INFO:tasks.workunit.client.0.smithi038.stdout:5/634: symlink d2/d3/da0/lba 0 2024-02-15T18:34:58.768 INFO:tasks.workunit.client.0.smithi038.stdout:5/635: fdatasync d2/d48/f59 0 2024-02-15T18:34:58.775 INFO:tasks.workunit.client.0.smithi038.stdout:3/590: dread d2/d1b/d4f/f8d [0,4194304] 0 2024-02-15T18:34:58.776 INFO:tasks.workunit.client.0.smithi038.stdout:3/591: rmdir d2/d1b/d3c/d41 39 2024-02-15T18:34:58.776 INFO:tasks.workunit.client.0.smithi038.stdout:3/592: truncate d2/f51 3001325 0 2024-02-15T18:34:58.777 INFO:tasks.workunit.client.0.smithi038.stdout:3/593: rmdir d2/d1b/d8c 39 2024-02-15T18:34:58.778 INFO:tasks.workunit.client.0.smithi038.stdout:3/594: mkdir d2/d7f/dc1 0 2024-02-15T18:34:58.778 INFO:tasks.workunit.client.0.smithi038.stdout:3/595: fsync d2/d24/f2e 0 2024-02-15T18:34:58.779 INFO:tasks.workunit.client.0.smithi038.stdout:3/596: rmdir d2/d1e 39 2024-02-15T18:34:58.779 INFO:tasks.workunit.client.0.smithi038.stdout:3/597: chown d2/l1c 55218 1 2024-02-15T18:34:58.780 INFO:tasks.workunit.client.0.smithi038.stdout:8/806: dwrite d1/d7/d26/d11b/f122 [0,4194304] 0 2024-02-15T18:34:58.780 INFO:tasks.workunit.client.0.smithi038.stdout:8/807: fdatasync d1/d7/d22/f10c 0 2024-02-15T18:34:58.781 INFO:tasks.workunit.client.0.smithi038.stdout:9/774: dwrite d1/da/d10/d13/d3d/f5c [0,4194304] 0 2024-02-15T18:34:58.784 INFO:tasks.workunit.client.0.smithi038.stdout:1/642: sync 2024-02-15T18:34:58.785 INFO:tasks.workunit.client.0.smithi038.stdout:8/808: dread d1/d7/d22/f5c [0,4194304] 0 2024-02-15T18:34:58.788 INFO:tasks.workunit.client.0.smithi038.stdout:0/738: dwrite d4/d13/d2a/d35/d1e/d32/fb6 [0,4194304] 0 2024-02-15T18:34:58.788 INFO:tasks.workunit.client.0.smithi038.stdout:5/636: dread d2/d3/f2f [4194304,4194304] 0 2024-02-15T18:34:58.788 INFO:tasks.workunit.client.0.smithi038.stdout:5/637: chown d2/d3/d5f/c6e 1577 1 2024-02-15T18:34:58.788 INFO:tasks.workunit.client.0.smithi038.stdout:0/739: dread - d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/fa9 zero size 2024-02-15T18:34:58.788 INFO:tasks.workunit.client.0.smithi038.stdout:3/598: read d2/f50 [1904515,105615] 0 2024-02-15T18:34:58.788 INFO:tasks.workunit.client.0.smithi038.stdout:2/603: dwrite dd/d12/d3a/d67/f86 [0,4194304] 0 2024-02-15T18:34:58.792 INFO:tasks.workunit.client.0.smithi038.stdout:9/775: unlink d1/da/d10/d13/d3d/d43/d4d/c113 0 2024-02-15T18:34:58.796 INFO:tasks.workunit.client.0.smithi038.stdout:5/638: creat d2/d3/d12/d5e/fbb x:0 0 0 2024-02-15T18:34:58.800 INFO:tasks.workunit.client.0.smithi038.stdout:3/599: truncate d2/d1b/d3c/d41/d4e/d62/f8f 4024879 0 2024-02-15T18:34:58.800 INFO:tasks.workunit.client.0.smithi038.stdout:3/600: truncate d2/d25/f64 1469473 0 2024-02-15T18:34:58.800 INFO:tasks.workunit.client.0.smithi038.stdout:9/776: symlink d1/da/d20/d58/d94/db9/de3/l115 0 2024-02-15T18:34:58.800 INFO:tasks.workunit.client.0.smithi038.stdout:9/777: fdatasync d1/da/d20/d58/d94/db9/dc6/f9e 0 2024-02-15T18:34:58.800 INFO:tasks.workunit.client.0.smithi038.stdout:9/778: readlink d1/dc/d69/lcf 0 2024-02-15T18:34:58.800 INFO:tasks.workunit.client.0.smithi038.stdout:0/740: truncate d4/d13/d2a/d35/d1e/d32/d48/d97/f9b 3951703 0 2024-02-15T18:34:58.802 INFO:tasks.workunit.client.0.smithi038.stdout:3/601: rename d2/d1b/d3c/d3a/d55/c61 to d2/d25/cc2 0 2024-02-15T18:34:58.803 INFO:tasks.workunit.client.0.smithi038.stdout:9/779: creat d1/da/d20/d58/d66/f116 x:0 0 0 2024-02-15T18:34:58.803 INFO:tasks.workunit.client.0.smithi038.stdout:9/780: readlink d1/da/d36/d3f/l114 0 2024-02-15T18:34:58.805 INFO:tasks.workunit.client.0.smithi038.stdout:0/741: mkdir d4/d13/d2a/d35/d1e/d32/d79/d7d/dda/df5 0 2024-02-15T18:34:58.805 INFO:tasks.workunit.client.0.smithi038.stdout:0/742: write d4/d13/d2a/d35/d1e/d32/d79/d7d/f86 [884952,21556] 0 2024-02-15T18:34:58.805 INFO:tasks.workunit.client.0.smithi038.stdout:0/743: symlink d4/d13/d2a/d35/db2/lf6 0 2024-02-15T18:34:58.808 INFO:tasks.workunit.client.0.smithi038.stdout:0/744: rename d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/ddb/cea to d4/d13/d2a/d35/d1e/d32/d79/d7d/da3/cf7 0 2024-02-15T18:34:58.810 INFO:tasks.workunit.client.0.smithi038.stdout:0/745: write d4/d13/d2a/f4b [3073313,42395] 0 2024-02-15T18:34:58.813 INFO:tasks.workunit.client.0.smithi038.stdout:0/746: rename d4/d13/d2a/d35/d1e/d32/d79/d7d/da3/dc2 to d4/d13/d2a/d35/d1e/d32/d3b/daa/df8 0 2024-02-15T18:34:58.814 INFO:tasks.workunit.client.0.smithi038.stdout:0/747: mknod d4/d13/d2a/d35/d1e/d32/d48/d9a/cf9 0 2024-02-15T18:34:58.814 INFO:tasks.workunit.client.0.smithi038.stdout:0/748: chown d4/d13/d19/c69 23153990 1 2024-02-15T18:34:58.817 INFO:tasks.workunit.client.0.smithi038.stdout:1/643: dwrite d2/d10/d19/d22/f57 [0,4194304] 0 2024-02-15T18:34:58.818 INFO:tasks.workunit.client.0.smithi038.stdout:1/644: mknod d2/d10/d19/d22/d36/d94/dc1/cd1 0 2024-02-15T18:34:58.820 INFO:tasks.workunit.client.0.smithi038.stdout:8/809: dwrite d1/d7/d22/f85 [0,4194304] 0 2024-02-15T18:34:58.822 INFO:tasks.workunit.client.0.smithi038.stdout:8/810: creat d1/d7/f125 x:0 0 0 2024-02-15T18:34:58.823 INFO:tasks.workunit.client.0.smithi038.stdout:8/811: creat d1/d4/d10/d43/f126 x:0 0 0 2024-02-15T18:34:58.823 INFO:tasks.workunit.client.0.smithi038.stdout:8/812: truncate d1/d7/d15/d6a/f6b 1613760 0 2024-02-15T18:34:58.825 INFO:tasks.workunit.client.0.smithi038.stdout:2/604: dwrite dd/d12/d5d/d93/f9c [0,4194304] 0 2024-02-15T18:34:58.825 INFO:tasks.workunit.client.0.smithi038.stdout:2/605: chown dd/d12/d5d/d54 11 1 2024-02-15T18:34:58.827 INFO:tasks.workunit.client.0.smithi038.stdout:9/781: dwrite d1/dc/f71 [0,4194304] 0 2024-02-15T18:34:58.830 INFO:tasks.workunit.client.0.smithi038.stdout:0/749: dwrite d4/d13/d2a/d35/d1e/f4d [0,4194304] 0 2024-02-15T18:34:58.831 INFO:tasks.workunit.client.0.smithi038.stdout:1/645: dread d2/f9d [0,4194304] 0 2024-02-15T18:34:58.831 INFO:tasks.workunit.client.0.smithi038.stdout:1/646: chown d2/d10/d19/d22/d54/d80/fc6 60 1 2024-02-15T18:34:58.835 INFO:tasks.workunit.client.0.smithi038.stdout:8/813: dread d1/d4/d5/d6/fcb [0,4194304] 0 2024-02-15T18:34:58.838 INFO:tasks.workunit.client.0.smithi038.stdout:2/606: rmdir dd/d12/d5d/d54 39 2024-02-15T18:34:58.842 INFO:tasks.workunit.client.0.smithi038.stdout:0/750: symlink d4/d13/d2a/d35/d1e/d32/d3b/lfa 0 2024-02-15T18:34:58.844 INFO:tasks.workunit.client.0.smithi038.stdout:6/625: sync 2024-02-15T18:34:58.844 INFO:tasks.workunit.client.0.smithi038.stdout:6/626: chown d2/d4/d19/d78/f9e 45484862 1 2024-02-15T18:34:58.844 INFO:tasks.workunit.client.0.smithi038.stdout:4/694: sync 2024-02-15T18:34:58.844 INFO:tasks.workunit.client.0.smithi038.stdout:1/647: creat d2/d10/d62/fd2 x:0 0 0 2024-02-15T18:34:58.845 INFO:tasks.workunit.client.0.smithi038.stdout:1/648: chown d2/d10/d19/d22/d54/dc0/d6b/ca2 51800222 1 2024-02-15T18:34:58.845 INFO:tasks.workunit.client.0.smithi038.stdout:1/649: chown d2/d10/d62 366 1 2024-02-15T18:34:58.845 INFO:tasks.workunit.client.0.smithi038.stdout:8/814: getdents d1/d4/db1/de5/db3 0 2024-02-15T18:34:58.845 INFO:tasks.workunit.client.0.smithi038.stdout:0/751: symlink d4/db3/lfb 0 2024-02-15T18:34:58.846 INFO:tasks.workunit.client.0.smithi038.stdout:6/627: unlink d2/d4/d28/c60 0 2024-02-15T18:34:58.847 INFO:tasks.workunit.client.0.smithi038.stdout:6/628: stat d2/d4/d2c/d39/c40 0 2024-02-15T18:34:58.848 INFO:tasks.workunit.client.0.smithi038.stdout:9/782: dwrite d1/da/fb0 [0,4194304] 0 2024-02-15T18:34:58.848 INFO:tasks.workunit.client.0.smithi038.stdout:9/783: chown d1/da/d36/d93 129866918 1 2024-02-15T18:34:58.849 INFO:tasks.workunit.client.0.smithi038.stdout:8/815: unlink d1/d7/d22/f5c 0 2024-02-15T18:34:58.850 INFO:tasks.workunit.client.0.smithi038.stdout:0/752: mkdir d4/d13/d19/dfc 0 2024-02-15T18:34:58.851 INFO:tasks.workunit.client.0.smithi038.stdout:4/695: mknod d11/d1e/d20/d2b/d35/c101 0 2024-02-15T18:34:58.852 INFO:tasks.workunit.client.0.smithi038.stdout:6/629: write d2/f1a [2743472,66772] 0 2024-02-15T18:34:58.854 INFO:tasks.workunit.client.0.smithi038.stdout:9/784: rename d1/da/d36/d3f/d99/c103 to d1/da/d10/d13/d3d/d43/d4d/c117 0 2024-02-15T18:34:58.854 INFO:tasks.workunit.client.0.smithi038.stdout:2/607: dwrite dd/d12/d5d/f28 [0,4194304] 0 2024-02-15T18:34:58.854 INFO:tasks.workunit.client.0.smithi038.stdout:2/608: read - dd/d12/d1d/d3b/d3c/d4a/f8c zero size 2024-02-15T18:34:58.854 INFO:tasks.workunit.client.0.smithi038.stdout:1/650: symlink d2/d10/d19/d2f/d3e/ld3 0 2024-02-15T18:34:58.855 INFO:tasks.workunit.client.0.smithi038.stdout:8/816: creat d1/d7/d26/d69/f127 x:0 0 0 2024-02-15T18:34:58.858 INFO:tasks.workunit.client.0.smithi038.stdout:0/753: symlink d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/ddb/lfd 0 2024-02-15T18:34:58.858 INFO:tasks.workunit.client.0.smithi038.stdout:0/754: write d4/d13/d2a/d35/d6a/fb8 [909007,76082] 0 2024-02-15T18:34:58.859 INFO:tasks.workunit.client.0.smithi038.stdout:6/630: mkdir d2/d4/d19/d78/dc3 0 2024-02-15T18:34:58.860 INFO:tasks.workunit.client.0.smithi038.stdout:4/696: rename d11/d1e/d20/d2b/d4d/d67/d73/d8b to d11/d1e/d20/d31/d48/d102 0 2024-02-15T18:34:58.860 INFO:tasks.workunit.client.0.smithi038.stdout:4/697: write d11/d1e/d20/d31/f91 [4763376,108611] 0 2024-02-15T18:34:58.861 INFO:tasks.workunit.client.0.smithi038.stdout:9/785: truncate d1/da/f54 3306678 0 2024-02-15T18:34:58.862 INFO:tasks.workunit.client.0.smithi038.stdout:2/609: link dd/d12/f80 dd/d12/d5d/d93/fc4 0 2024-02-15T18:34:58.864 INFO:tasks.workunit.client.0.smithi038.stdout:2/610: dread dd/d12/d3a/d67/d4f/f64 [0,4194304] 0 2024-02-15T18:34:58.865 INFO:tasks.workunit.client.0.smithi038.stdout:4/698: dread d11/f1d [0,4194304] 0 2024-02-15T18:34:58.865 INFO:tasks.workunit.client.0.smithi038.stdout:4/699: write d11/d1e/d20/fd6 [1043654,67581] 0 2024-02-15T18:34:58.866 INFO:tasks.workunit.client.0.smithi038.stdout:6/631: mkdir d2/d4/d19/d78/dc4 0 2024-02-15T18:34:58.866 INFO:tasks.workunit.client.0.smithi038.stdout:0/755: mkdir d4/d13/d2a/d35/d1e/d32/d3b/daa/df8/dfe 0 2024-02-15T18:34:58.866 INFO:tasks.workunit.client.0.smithi038.stdout:0/756: fdatasync d4/db3/fce 0 2024-02-15T18:34:58.866 INFO:tasks.workunit.client.0.smithi038.stdout:0/757: write d4/d13/f8e [844731,49623] 0 2024-02-15T18:34:58.866 INFO:tasks.workunit.client.0.smithi038.stdout:9/786: creat d1/da/d20/d58/d94/db9/de3/f118 x:0 0 0 2024-02-15T18:34:58.866 INFO:tasks.workunit.client.0.smithi038.stdout:9/787: stat d1/da/d36/d3f/d48/fb3 0 2024-02-15T18:34:58.868 INFO:tasks.workunit.client.0.smithi038.stdout:2/611: link dd/d12/d3a/d67/f27 dd/fc5 0 2024-02-15T18:34:58.869 INFO:tasks.workunit.client.0.smithi038.stdout:4/700: creat d11/d1e/d20/d97/f103 x:0 0 0 2024-02-15T18:34:58.869 INFO:tasks.workunit.client.0.smithi038.stdout:4/701: readlink d11/d1e/d20/d2b/d4d/l9e 0 2024-02-15T18:34:58.869 INFO:tasks.workunit.client.0.smithi038.stdout:4/702: write d11/d1e/d20/d97/f103 [811297,27581] 0 2024-02-15T18:34:58.870 INFO:tasks.workunit.client.0.smithi038.stdout:6/632: mknod d2/d4/d6f/cc5 0 2024-02-15T18:34:58.870 INFO:tasks.workunit.client.0.smithi038.stdout:6/633: truncate d2/d4/d19/fc1 3516 0 2024-02-15T18:34:58.870 INFO:tasks.workunit.client.0.smithi038.stdout:6/634: write d2/d4/d2c/f3c [4144008,83499] 0 2024-02-15T18:34:58.870 INFO:tasks.workunit.client.0.smithi038.stdout:6/635: write d2/d4/d2c/d39/db4/f5e [691187,104152] 0 2024-02-15T18:34:58.870 INFO:tasks.workunit.client.0.smithi038.stdout:0/758: creat d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/ddb/fff x:0 0 0 2024-02-15T18:34:58.871 INFO:tasks.workunit.client.0.smithi038.stdout:0/759: read d4/db3/fe2 [26045,14300] 0 2024-02-15T18:34:58.872 INFO:tasks.workunit.client.0.smithi038.stdout:9/788: unlink d1/da/d20/f6d 0 2024-02-15T18:34:58.879 INFO:tasks.workunit.client.0.smithi038.stdout:6/636: symlink d2/d4/d28/lc6 0 2024-02-15T18:34:58.880 INFO:tasks.workunit.client.0.smithi038.stdout:2/612: dread dd/d12/d3a/d67/d4f/d5a/f7f [0,4194304] 0 2024-02-15T18:34:58.880 INFO:tasks.workunit.client.0.smithi038.stdout:2/613: truncate dd/d12/d3a/d67/f27 3892140 0 2024-02-15T18:34:58.880 INFO:tasks.workunit.client.0.smithi038.stdout:2/614: chown dd/d12/l4e 248543738 1 2024-02-15T18:34:58.880 INFO:tasks.workunit.client.0.smithi038.stdout:0/760: mkdir d4/d13/d53/d100 0 2024-02-15T18:34:58.880 INFO:tasks.workunit.client.0.smithi038.stdout:0/761: readlink d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/ddb/lfd 0 2024-02-15T18:34:58.880 INFO:tasks.workunit.client.0.smithi038.stdout:0/762: read - d4/d13/d2a/d35/d6a/f9e zero size 2024-02-15T18:34:58.881 INFO:tasks.workunit.client.0.smithi038.stdout:2/615: write dd/fc5 [92835,46951] 0 2024-02-15T18:34:58.881 INFO:tasks.workunit.client.0.smithi038.stdout:9/789: link d1/da/d10/d19/c30 d1/da/d10/d13/c119 0 2024-02-15T18:34:58.881 INFO:tasks.workunit.client.0.smithi038.stdout:8/817: dwrite d1/d7/d22/f10c [0,4194304] 0 2024-02-15T18:34:58.881 INFO:tasks.workunit.client.0.smithi038.stdout:8/818: chown d1/d4/d5/d6/f75 67909 1 2024-02-15T18:34:58.881 INFO:tasks.workunit.client.0.smithi038.stdout:8/819: chown d1/d4/d10 26 1 2024-02-15T18:34:58.882 INFO:tasks.workunit.client.0.smithi038.stdout:6/637: mkdir d2/d4/d6c/d9d/dc7 0 2024-02-15T18:34:58.882 INFO:tasks.workunit.client.0.smithi038.stdout:2/616: creat dd/d12/d5d/d54/d91/fc6 x:0 0 0 2024-02-15T18:34:58.883 INFO:tasks.workunit.client.0.smithi038.stdout:9/790: creat d1/da/d36/d8b/f11a x:0 0 0 2024-02-15T18:34:58.883 INFO:tasks.workunit.client.0.smithi038.stdout:1/651: dwrite d2/d10/d19/d5f/fa8 [4194304,4194304] 0 2024-02-15T18:34:58.886 INFO:tasks.workunit.client.0.smithi038.stdout:4/703: dread d11/d1e/d20/fd6 [0,4194304] 0 2024-02-15T18:34:58.886 INFO:tasks.workunit.client.0.smithi038.stdout:8/820: creat d1/d4/d5/d31/d65/f128 x:0 0 0 2024-02-15T18:34:58.886 INFO:tasks.workunit.client.0.smithi038.stdout:4/704: write d11/d1e/d20/d31/ffd [885866,107239] 0 2024-02-15T18:34:58.886 INFO:tasks.workunit.client.0.smithi038.stdout:1/652: dread d2/d69/f83 [0,4194304] 0 2024-02-15T18:34:58.887 INFO:tasks.workunit.client.0.smithi038.stdout:6/638: mkdir d2/d4/d19/d55/d5c/dc8 0 2024-02-15T18:34:58.888 INFO:tasks.workunit.client.0.smithi038.stdout:2/617: rmdir dd/d12/d5d 39 2024-02-15T18:34:58.890 INFO:tasks.workunit.client.0.smithi038.stdout:0/763: dread d4/d13/d2a/d35/f49 [0,4194304] 0 2024-02-15T18:34:58.902 INFO:tasks.workunit.client.0.smithi038.stdout:9/791: link d1/da/d10/d13/d3d/d6c/fd4 d1/da/d20/d58/f11b 0 2024-02-15T18:34:58.903 INFO:tasks.workunit.client.0.smithi038.stdout:9/792: read d1/da/d10/d13/f86 [1828944,104301] 0 2024-02-15T18:34:58.903 INFO:tasks.workunit.client.0.smithi038.stdout:8/821: link d1/d4/d5/d31/d65/f128 d1/d4/d5/d6/db9/f129 0 2024-02-15T18:34:58.903 INFO:tasks.workunit.client.0.smithi038.stdout:8/822: write d1/f40 [1613606,40182] 0 2024-02-15T18:34:58.907 INFO:tasks.workunit.client.0.smithi038.stdout:5/639: sync 2024-02-15T18:34:58.907 INFO:tasks.workunit.client.0.smithi038.stdout:3/602: sync 2024-02-15T18:34:58.907 INFO:tasks.workunit.client.0.smithi038.stdout:7/687: sync 2024-02-15T18:34:58.907 INFO:tasks.workunit.client.0.smithi038.stdout:5/640: chown d2/d30/d52/caf 10 1 2024-02-15T18:34:58.907 INFO:tasks.workunit.client.0.smithi038.stdout:4/705: mknod d11/d1e/d20/d31/d48/d102/c104 0 2024-02-15T18:34:58.907 INFO:tasks.workunit.client.0.smithi038.stdout:4/706: getdents d11/d1e/d20/d31/d48/dac/dbe/d40/dde 0 2024-02-15T18:34:58.910 INFO:tasks.workunit.client.0.smithi038.stdout:2/618: rename dd/d12/l4e to dd/d12/d5d/lc7 0 2024-02-15T18:34:58.910 INFO:tasks.workunit.client.0.smithi038.stdout:1/653: mkdir d2/d39/dd4 0 2024-02-15T18:34:58.914 INFO:tasks.workunit.client.0.smithi038.stdout:2/619: symlink dd/d12/d1d/lc8 0 2024-02-15T18:34:58.915 INFO:tasks.workunit.client.0.smithi038.stdout:8/823: dread d1/d4/dbb/fbf [0,4194304] 0 2024-02-15T18:34:58.917 INFO:tasks.workunit.client.0.smithi038.stdout:2/620: write dd/d12/d5d/f28 [3514878,107459] 0 2024-02-15T18:34:58.917 INFO:tasks.workunit.client.0.smithi038.stdout:9/793: rename d1/da/d36/d3f/d48 to d1/da/d36/d93/d9a/d11c 0 2024-02-15T18:34:58.918 INFO:tasks.workunit.client.0.smithi038.stdout:9/794: truncate d1/da/d20/d58/d76/fb5 1026139 0 2024-02-15T18:34:58.918 INFO:tasks.workunit.client.0.smithi038.stdout:6/639: rmdir d2/d4/d2c/d39/db4/d75 0 2024-02-15T18:34:58.922 INFO:tasks.workunit.client.0.smithi038.stdout:0/764: mknod d4/c101 0 2024-02-15T18:34:58.922 INFO:tasks.workunit.client.0.smithi038.stdout:3/603: rename d2/d1b/f5c to d2/d1b/d3c/d42/fc3 0 2024-02-15T18:34:58.922 INFO:tasks.workunit.client.0.smithi038.stdout:0/765: dread - d4/d13/d2a/d35/d1e/d32/d79/d7d/dda/fee zero size 2024-02-15T18:34:58.923 INFO:tasks.workunit.client.0.smithi038.stdout:6/640: truncate d2/d4/d19/d55/d5c/f70 500374 0 2024-02-15T18:34:58.926 INFO:tasks.workunit.client.0.smithi038.stdout:8/824: dread d1/d7/f1d [0,4194304] 0 2024-02-15T18:34:58.926 INFO:tasks.workunit.client.0.smithi038.stdout:6/641: symlink d2/d4/d6c/d9d/lc9 0 2024-02-15T18:34:58.927 INFO:tasks.workunit.client.0.smithi038.stdout:6/642: link d2/d4/d2c/l6e d2/d4/d19/d78/dc4/lca 0 2024-02-15T18:34:58.930 INFO:tasks.workunit.client.0.smithi038.stdout:6/643: write d2/d4/d2c/d39/db4/f9c [1061842,53554] 0 2024-02-15T18:34:58.932 INFO:tasks.workunit.client.0.smithi038.stdout:6/644: creat d2/d4/d6c/fcb x:0 0 0 2024-02-15T18:34:58.932 INFO:tasks.workunit.client.0.smithi038.stdout:6/645: dread - d2/d4/d19/d78/f9e zero size 2024-02-15T18:34:58.935 INFO:tasks.workunit.client.0.smithi038.stdout:6/646: symlink d2/d4/d2c/d39/lcc 0 2024-02-15T18:34:58.935 INFO:tasks.workunit.client.0.smithi038.stdout:0/766: write d4/d13/f39 [4163330,17924] 0 2024-02-15T18:34:58.935 INFO:tasks.workunit.client.0.smithi038.stdout:0/767: truncate d4/d13/d2a/f2d 620372 0 2024-02-15T18:34:58.935 INFO:tasks.workunit.client.0.smithi038.stdout:0/768: write d4/d13/d19/f82 [1479073,4400] 0 2024-02-15T18:34:58.937 INFO:tasks.workunit.client.0.smithi038.stdout:3/604: dread d2/d24/f2e [0,4194304] 0 2024-02-15T18:34:58.940 INFO:tasks.workunit.client.0.smithi038.stdout:4/707: dwrite d11/d1e/d20/d2b/d65/fb8 [8388608,4194304] 0 2024-02-15T18:34:58.941 INFO:tasks.workunit.client.0.smithi038.stdout:3/605: symlink d2/d25/d60/lc4 0 2024-02-15T18:34:58.941 INFO:tasks.workunit.client.0.smithi038.stdout:4/708: mkdir d11/d1e/d3e/d57/d7e/d105 0 2024-02-15T18:34:58.942 INFO:tasks.workunit.client.0.smithi038.stdout:0/769: read d4/d13/f39 [1889999,124406] 0 2024-02-15T18:34:58.944 INFO:tasks.workunit.client.0.smithi038.stdout:4/709: mknod d11/d1e/d20/d31/d8d/d99/c106 0 2024-02-15T18:34:58.944 INFO:tasks.workunit.client.0.smithi038.stdout:4/710: write d11/d1e/d20/d2b/d4d/d67/d73/fe3 [2057007,118981] 0 2024-02-15T18:34:58.948 INFO:tasks.workunit.client.0.smithi038.stdout:5/641: dwrite d2/d30/f55 [0,4194304] 0 2024-02-15T18:34:58.948 INFO:tasks.workunit.client.0.smithi038.stdout:4/711: mkdir d11/d1e/d20/d2b/d4d/d67/d107 0 2024-02-15T18:34:58.948 INFO:tasks.workunit.client.0.smithi038.stdout:4/712: readlink d11/d1e/l85 0 2024-02-15T18:34:58.950 INFO:tasks.workunit.client.0.smithi038.stdout:6/647: dwrite d2/d10/f30 [4194304,4194304] 0 2024-02-15T18:34:58.951 INFO:tasks.workunit.client.0.smithi038.stdout:8/825: dwrite d1/d4/d5/d6/db9/f129 [0,4194304] 0 2024-02-15T18:34:58.951 INFO:tasks.workunit.client.0.smithi038.stdout:8/826: dread - d1/d4/d10/d43/d48/f10b zero size 2024-02-15T18:34:58.959 INFO:tasks.workunit.client.0.smithi038.stdout:8/827: mknod d1/d7/d26/d69/c12a 0 2024-02-15T18:34:58.959 INFO:tasks.workunit.client.0.smithi038.stdout:8/828: truncate d1/d3a/d58/f117 1835532 0 2024-02-15T18:34:58.959 INFO:tasks.workunit.client.0.smithi038.stdout:6/648: dread d2/d4/d2c/d39/f71 [0,4194304] 0 2024-02-15T18:34:58.960 INFO:tasks.workunit.client.0.smithi038.stdout:8/829: write d1/d7/d26/fe1 [3330179,6644] 0 2024-02-15T18:34:58.962 INFO:tasks.workunit.client.0.smithi038.stdout:6/649: mkdir d2/d4/d2c/d39/db4/dcd 0 2024-02-15T18:34:58.962 INFO:tasks.workunit.client.0.smithi038.stdout:6/650: chown d2/d4/d2c/d39/db4/d5f/d99/l9f 10266 1 2024-02-15T18:34:58.963 INFO:tasks.workunit.client.0.smithi038.stdout:8/830: mkdir d1/d3a/df6/d12b 0 2024-02-15T18:34:58.963 INFO:tasks.workunit.client.0.smithi038.stdout:8/831: fsync d1/d4/d5/d31/d65/d80/fc5 0 2024-02-15T18:34:58.964 INFO:tasks.workunit.client.0.smithi038.stdout:6/651: getdents d2/d10 0 2024-02-15T18:34:58.966 INFO:tasks.workunit.client.0.smithi038.stdout:8/832: mkdir d1/d4/d10/d43/d48/d52/d12c 0 2024-02-15T18:34:58.966 INFO:tasks.workunit.client.0.smithi038.stdout:8/833: stat d1/d4/db1 0 2024-02-15T18:34:58.966 INFO:tasks.workunit.client.0.smithi038.stdout:8/834: truncate d1/d4/d10/d43/d48/ffe 277235 0 2024-02-15T18:34:58.966 INFO:tasks.workunit.client.0.smithi038.stdout:6/652: creat d2/d4/d2c/d39/d76/dc2/fce x:0 0 0 2024-02-15T18:34:58.967 INFO:tasks.workunit.client.0.smithi038.stdout:8/835: getdents d1/d4/d5/d6/d6d 0 2024-02-15T18:34:58.967 INFO:tasks.workunit.client.0.smithi038.stdout:8/836: write d1/f6f [4487750,93992] 0 2024-02-15T18:34:58.969 INFO:tasks.workunit.client.0.smithi038.stdout:6/653: unlink d2/d4/d19/d78/d7a/fab 0 2024-02-15T18:34:58.979 INFO:tasks.workunit.client.0.smithi038.stdout:3/606: dwrite d2/d1b/d3c/d41/d4e/fa3 [0,4194304] 0 2024-02-15T18:34:58.980 INFO:tasks.workunit.client.0.smithi038.stdout:0/770: dwrite d4/d13/d2a/d35/d1e/d32/d48/d97/faf [0,4194304] 0 2024-02-15T18:34:58.980 INFO:tasks.workunit.client.0.smithi038.stdout:5/642: dwrite d2/d3/d12/fa8 [0,4194304] 0 2024-02-15T18:34:58.980 INFO:tasks.workunit.client.0.smithi038.stdout:5/643: fdatasync d2/d3/da0/f26 0 2024-02-15T18:34:58.980 INFO:tasks.workunit.client.0.smithi038.stdout:4/713: dwrite d11/d1e/d43/d5e/dfe/ffb [0,4194304] 0 2024-02-15T18:34:58.985 INFO:tasks.workunit.client.0.smithi038.stdout:8/837: dwrite d1/d7/d15/d4b/fec [0,4194304] 0 2024-02-15T18:34:58.990 INFO:tasks.workunit.client.0.smithi038.stdout:3/607: symlink d2/d1b/d8c/lc5 0 2024-02-15T18:34:58.990 INFO:tasks.workunit.client.0.smithi038.stdout:3/608: chown d2/d1b/d3c/d41/c4c 28 1 2024-02-15T18:34:58.990 INFO:tasks.workunit.client.0.smithi038.stdout:0/771: symlink d4/d13/d2a/d35/d1e/d32/d79/d7d/dda/df5/l102 0 2024-02-15T18:34:58.990 INFO:tasks.workunit.client.0.smithi038.stdout:0/772: chown d4/d13/d2a/d35/f8f 87816 1 2024-02-15T18:34:58.991 INFO:tasks.workunit.client.0.smithi038.stdout:0/773: fsync d4/d13/d2a/d35/f49 0 2024-02-15T18:34:58.993 INFO:tasks.workunit.client.0.smithi038.stdout:4/714: creat d11/d1e/d20/d31/d48/d102/f108 x:0 0 0 2024-02-15T18:34:58.995 INFO:tasks.workunit.client.0.smithi038.stdout:8/838: creat d1/d7/d26/d69/f12d x:0 0 0 2024-02-15T18:34:58.997 INFO:tasks.workunit.client.0.smithi038.stdout:3/609: rename d2/d25/f66 to d2/d1b/d3c/d3a/d55/da0/fc6 0 2024-02-15T18:34:58.997 INFO:tasks.workunit.client.0.smithi038.stdout:3/610: chown d2/d1b/d3c/d41/f56 9 1 2024-02-15T18:34:58.998 INFO:tasks.workunit.client.0.smithi038.stdout:0/774: mknod d4/db3/c103 0 2024-02-15T18:34:58.999 INFO:tasks.workunit.client.0.smithi038.stdout:0/775: getdents d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6 0 2024-02-15T18:34:59.000 INFO:tasks.workunit.client.0.smithi038.stdout:0/776: rmdir d4/d13/d19 39 2024-02-15T18:34:59.000 INFO:tasks.workunit.client.0.smithi038.stdout:6/654: dwrite d2/d4/d6f/f88 [0,4194304] 0 2024-02-15T18:34:59.001 INFO:tasks.workunit.client.0.smithi038.stdout:4/715: dread d11/d1e/d20/d97/f103 [0,4194304] 0 2024-02-15T18:34:59.001 INFO:tasks.workunit.client.0.smithi038.stdout:0/777: write d4/d13/d2a/d35/db2/fe6 [1258008,99440] 0 2024-02-15T18:34:59.003 INFO:tasks.workunit.client.0.smithi038.stdout:6/655: mknod d2/d4/d2c/d39/d76/ccf 0 2024-02-15T18:34:59.004 INFO:tasks.workunit.client.0.smithi038.stdout:0/778: mknod d4/d13/d2a/d35/d1e/d32/d79/d7d/dda/c104 0 2024-02-15T18:34:59.004 INFO:tasks.workunit.client.0.smithi038.stdout:6/656: write f1 [5056779,108758] 0 2024-02-15T18:34:59.004 INFO:tasks.workunit.client.0.smithi038.stdout:6/657: chown d2/d4/d19/fc1 1815 1 2024-02-15T18:34:59.006 INFO:tasks.workunit.client.0.smithi038.stdout:0/779: link d4/db3/fce d4/d13/d2a/d35/d1e/d32/d3b/f105 0 2024-02-15T18:34:59.007 INFO:tasks.workunit.client.0.smithi038.stdout:0/780: write d4/d13/d2a/d35/d1e/d32/d79/d7d/f96 [56439,62994] 0 2024-02-15T18:34:59.007 INFO:tasks.workunit.client.0.smithi038.stdout:0/781: write d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/fa2 [279436,41626] 0 2024-02-15T18:34:59.009 INFO:tasks.workunit.client.0.smithi038.stdout:0/782: unlink d4/d13/d2a/d35/d1e/f5d 0 2024-02-15T18:34:59.010 INFO:tasks.workunit.client.0.smithi038.stdout:8/839: dread d1/d4/d5/d6/db9/f129 [0,4194304] 0 2024-02-15T18:34:59.011 INFO:tasks.workunit.client.0.smithi038.stdout:5/644: dwrite d2/d3/f1b [0,4194304] 0 2024-02-15T18:34:59.011 INFO:tasks.workunit.client.0.smithi038.stdout:5/645: fsync d2/d30/d52/f84 0 2024-02-15T18:34:59.013 INFO:tasks.workunit.client.0.smithi038.stdout:3/611: dread d2/d1b/d3c/d42/fc3 [4194304,4194304] 0 2024-02-15T18:34:59.014 INFO:tasks.workunit.client.0.smithi038.stdout:3/612: chown d2/d1b/d4f/fac 4 1 2024-02-15T18:34:59.014 INFO:tasks.workunit.client.0.smithi038.stdout:3/613: dread - d2/d1b/d3c/d42/fbd zero size 2024-02-15T18:34:59.014 INFO:tasks.workunit.client.0.smithi038.stdout:5/646: dread d2/d48/d83/d96/f85 [0,4194304] 0 2024-02-15T18:34:59.017 INFO:tasks.workunit.client.0.smithi038.stdout:5/647: dread d2/d64/d99/fac [0,4194304] 0 2024-02-15T18:34:59.023 INFO:tasks.workunit.client.0.smithi038.stdout:0/783: link d4/l2e d4/d13/d53/l106 0 2024-02-15T18:34:59.024 INFO:tasks.workunit.client.0.smithi038.stdout:5/648: mkdir d2/d64/d99/dbc 0 2024-02-15T18:34:59.025 INFO:tasks.workunit.client.0.smithi038.stdout:0/784: symlink d4/d13/d2a/l107 0 2024-02-15T18:34:59.027 INFO:tasks.workunit.client.0.smithi038.stdout:0/785: creat d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/f108 x:0 0 0 2024-02-15T18:34:59.027 INFO:tasks.workunit.client.0.smithi038.stdout:0/786: chown d4/d13/d2a/d35/d1e/d32/d79 12610547 1 2024-02-15T18:34:59.028 INFO:tasks.workunit.client.0.smithi038.stdout:8/840: dwrite d1/d4/d5/d31/d65/ffa [0,4194304] 0 2024-02-15T18:34:59.034 INFO:tasks.workunit.client.0.smithi038.stdout:8/841: creat d1/d4/db1/de5/db3/f12e x:0 0 0 2024-02-15T18:34:59.035 INFO:tasks.workunit.client.0.smithi038.stdout:7/688: sync 2024-02-15T18:34:59.035 INFO:tasks.workunit.client.0.smithi038.stdout:7/689: fdatasync d0/db/d19/f6d 0 2024-02-15T18:34:59.036 INFO:tasks.workunit.client.0.smithi038.stdout:8/842: rename d1/d4/d10/d3d/db7/dd5 to d1/d3a/d58/d12f 0 2024-02-15T18:34:59.036 INFO:tasks.workunit.client.0.smithi038.stdout:8/843: write d1/d4/d10/d43/f126 [845828,103443] 0 2024-02-15T18:34:59.036 INFO:tasks.workunit.client.0.smithi038.stdout:8/844: write d1/d7/fb0 [841826,71561] 0 2024-02-15T18:34:59.042 INFO:tasks.workunit.client.0.smithi038.stdout:5/649: dwrite d2/d48/f74 [0,4194304] 0 2024-02-15T18:34:59.045 INFO:tasks.workunit.client.0.smithi038.stdout:5/650: rename d2/d3/da0/l2e to d2/d64/d99/dbc/lbd 0 2024-02-15T18:34:59.047 INFO:tasks.workunit.client.0.smithi038.stdout:5/651: creat d2/d48/d83/d96/d8e/fbe x:0 0 0 2024-02-15T18:34:59.050 INFO:tasks.workunit.client.0.smithi038.stdout:5/652: mknod d2/d3/db9/cbf 0 2024-02-15T18:34:59.052 INFO:tasks.workunit.client.0.smithi038.stdout:7/690: dread d0/db/d59/d15/f44 [0,4194304] 0 2024-02-15T18:34:59.059 INFO:tasks.workunit.client.0.smithi038.stdout:8/845: dwrite d1/d7/d15/d4b/f57 [0,4194304] 0 2024-02-15T18:34:59.060 INFO:tasks.workunit.client.0.smithi038.stdout:8/846: chown d1/d4/d10/d43/d48/f10b 3762 1 2024-02-15T18:34:59.060 INFO:tasks.workunit.client.0.smithi038.stdout:8/847: truncate d1/d4/d10/d43/d48/d52/fba 199809 0 2024-02-15T18:34:59.062 INFO:tasks.workunit.client.0.smithi038.stdout:8/848: rename d1/d4/d5/d6/d6d/c63 to d1/d4/d5/d77/d79/c130 0 2024-02-15T18:34:59.063 INFO:tasks.workunit.client.0.smithi038.stdout:8/849: symlink d1/d4/d5/d31/d65/l131 0 2024-02-15T18:34:59.063 INFO:tasks.workunit.client.0.smithi038.stdout:8/850: symlink d1/d4/db1/dd8/l132 0 2024-02-15T18:34:59.063 INFO:tasks.workunit.client.0.smithi038.stdout:8/851: dread - d1/d7/d22/da7/f100 zero size 2024-02-15T18:34:59.066 INFO:tasks.workunit.client.0.smithi038.stdout:5/653: dwrite d2/f80 [0,4194304] 0 2024-02-15T18:34:59.072 INFO:tasks.workunit.client.0.smithi038.stdout:5/654: symlink d2/d48/d83/d96/d8e/lc0 0 2024-02-15T18:34:59.074 INFO:tasks.workunit.client.0.smithi038.stdout:5/655: rename d2/d64/d99/dbc/lbd to d2/d3/d5f/lc1 0 2024-02-15T18:34:59.074 INFO:tasks.workunit.client.0.smithi038.stdout:5/656: fsync d2/fa5 0 2024-02-15T18:34:59.075 INFO:tasks.workunit.client.0.smithi038.stdout:5/657: read d2/d3/d12/fa [164443,20747] 0 2024-02-15T18:34:59.075 INFO:tasks.workunit.client.0.smithi038.stdout:5/658: dread - d2/d3/da0/fb4 zero size 2024-02-15T18:34:59.079 INFO:tasks.workunit.client.0.smithi038.stdout:8/852: dwrite d1/d7/d26/d11b/fed [0,4194304] 0 2024-02-15T18:34:59.079 INFO:tasks.workunit.client.0.smithi038.stdout:8/853: stat d1/d4/d10/d3d/db7 0 2024-02-15T18:34:59.079 INFO:tasks.workunit.client.0.smithi038.stdout:8/854: readlink d1/d7/l9c 0 2024-02-15T18:34:59.079 INFO:tasks.workunit.client.0.smithi038.stdout:8/855: chown d1/d4/d5/d31/d65/d80/c103 787248 1 2024-02-15T18:34:59.079 INFO:tasks.workunit.client.0.smithi038.stdout:8/856: write d1/d4/d10/d43/d48/d52/fbd [4495099,36464] 0 2024-02-15T18:34:59.082 INFO:tasks.workunit.client.0.smithi038.stdout:8/857: rename d1/d7/d26/fdc to d1/d7/d22/f133 0 2024-02-15T18:34:59.082 INFO:tasks.workunit.client.0.smithi038.stdout:8/858: stat d1/d4/d10/d43/d48/d52/d83 0 2024-02-15T18:34:59.082 INFO:tasks.workunit.client.0.smithi038.stdout:8/859: write d1/d4/d5/d31/d65/d80/fac [913480,39566] 0 2024-02-15T18:34:59.091 INFO:tasks.workunit.client.0.smithi038.stdout:5/659: dwrite d2/d3/da0/fb4 [0,4194304] 0 2024-02-15T18:34:59.095 INFO:tasks.workunit.client.0.smithi038.stdout:5/660: rename d2/d30/f9f to d2/d3/d12/d5e/d86/fc2 0 2024-02-15T18:34:59.100 INFO:tasks.workunit.client.0.smithi038.stdout:5/661: rename d2/d3/da0/f26 to d2/d3/d5f/fc3 0 2024-02-15T18:34:59.100 INFO:tasks.workunit.client.0.smithi038.stdout:5/662: rmdir d2/d3/d5f 39 2024-02-15T18:34:59.100 INFO:tasks.workunit.client.0.smithi038.stdout:5/663: fsync d2/f3f 0 2024-02-15T18:34:59.101 INFO:tasks.workunit.client.0.smithi038.stdout:5/664: read d2/d3/f9e [284822,67683] 0 2024-02-15T18:34:59.101 INFO:tasks.workunit.client.0.smithi038.stdout:5/665: write d2/d48/d83/d96/d8e/fbe [286082,49408] 0 2024-02-15T18:34:59.102 INFO:tasks.workunit.client.0.smithi038.stdout:5/666: unlink d2/d30/f46 0 2024-02-15T18:34:59.102 INFO:tasks.workunit.client.0.smithi038.stdout:5/667: chown d2/f80 2403 1 2024-02-15T18:34:59.103 INFO:tasks.workunit.client.0.smithi038.stdout:8/860: dwrite d1/d7/d22/f133 [0,4194304] 0 2024-02-15T18:34:59.107 INFO:tasks.workunit.client.0.smithi038.stdout:8/861: creat d1/d4/d10/d43/d48/d52/f134 x:0 0 0 2024-02-15T18:34:59.107 INFO:tasks.workunit.client.0.smithi038.stdout:8/862: chown d1/d4/d5/d31/d65/l131 852845780 1 2024-02-15T18:34:59.107 INFO:tasks.workunit.client.0.smithi038.stdout:8/863: write d1/d4/d10/d43/d48/d52/fe0 [86786,899] 0 2024-02-15T18:34:59.107 INFO:tasks.workunit.client.0.smithi038.stdout:8/864: mknod d1/d4/d10/d43/d48/d9a/dd9/c135 0 2024-02-15T18:34:59.109 INFO:tasks.workunit.client.0.smithi038.stdout:8/865: getdents d1/d4/d10/d3d/db7/d108 0 2024-02-15T18:34:59.113 INFO:tasks.workunit.client.0.smithi038.stdout:9/795: sync 2024-02-15T18:34:59.114 INFO:tasks.workunit.client.0.smithi038.stdout:2/621: sync 2024-02-15T18:34:59.114 INFO:tasks.workunit.client.0.smithi038.stdout:1/654: sync 2024-02-15T18:34:59.114 INFO:tasks.workunit.client.0.smithi038.stdout:1/655: fsync d2/f2d 0 2024-02-15T18:34:59.114 INFO:tasks.workunit.client.0.smithi038.stdout:9/796: write d1/dc/d69/d7d/ff8 [188832,88703] 0 2024-02-15T18:34:59.115 INFO:tasks.workunit.client.0.smithi038.stdout:2/622: mknod dd/d12/d3a/d67/d4f/d5a/cc9 0 2024-02-15T18:34:59.115 INFO:tasks.workunit.client.0.smithi038.stdout:1/656: mkdir d2/d10/d19/d22/d36/d94/dc1/dd5 0 2024-02-15T18:34:59.115 INFO:tasks.workunit.client.0.smithi038.stdout:1/657: stat d2/d10/d19/c3f 0 2024-02-15T18:34:59.116 INFO:tasks.workunit.client.0.smithi038.stdout:2/623: creat dd/d12/d8d/fca x:0 0 0 2024-02-15T18:34:59.116 INFO:tasks.workunit.client.0.smithi038.stdout:2/624: dread - dd/d12/d3a/d67/d4f/fb6 zero size 2024-02-15T18:34:59.119 INFO:tasks.workunit.client.0.smithi038.stdout:1/658: dread d2/d39/f7f [0,4194304] 0 2024-02-15T18:34:59.121 INFO:tasks.workunit.client.0.smithi038.stdout:1/659: creat d2/d10/d19/d2f/fd6 x:0 0 0 2024-02-15T18:34:59.123 INFO:tasks.workunit.client.0.smithi038.stdout:8/866: dwrite d1/d7/d22/f95 [0,4194304] 0 2024-02-15T18:34:59.123 INFO:tasks.workunit.client.0.smithi038.stdout:8/867: chown d1/d7/d26 123 1 2024-02-15T18:34:59.126 INFO:tasks.workunit.client.0.smithi038.stdout:8/868: dread d1/d7/d26/f84 [0,4194304] 0 2024-02-15T18:34:59.130 INFO:tasks.workunit.client.0.smithi038.stdout:5/668: dwrite d2/d30/d52/f70 [0,4194304] 0 2024-02-15T18:34:59.131 INFO:tasks.workunit.client.0.smithi038.stdout:5/669: write d2/d3/da0/d33/f93 [1928516,58345] 0 2024-02-15T18:34:59.132 INFO:tasks.workunit.client.0.smithi038.stdout:5/670: symlink d2/d3/d5f/lc4 0 2024-02-15T18:34:59.135 INFO:tasks.workunit.client.0.smithi038.stdout:9/797: dwrite d1/f1c [0,4194304] 0 2024-02-15T18:34:59.135 INFO:tasks.workunit.client.0.smithi038.stdout:9/798: truncate d1/da/d10/d13/d3d/d4b/daa/fb1 1041011 0 2024-02-15T18:34:59.136 INFO:tasks.workunit.client.0.smithi038.stdout:1/660: dread d2/d39/f48 [0,4194304] 0 2024-02-15T18:34:59.136 INFO:tasks.workunit.client.0.smithi038.stdout:8/869: dread d1/d4/d5/d77/f9f [0,4194304] 0 2024-02-15T18:34:59.136 INFO:tasks.workunit.client.0.smithi038.stdout:9/799: write d1/da/d20/d58/d76/ff3 [645377,18370] 0 2024-02-15T18:34:59.137 INFO:tasks.workunit.client.0.smithi038.stdout:5/671: write d2/d30/d52/f6a [2475869,79717] 0 2024-02-15T18:34:59.139 INFO:tasks.workunit.client.0.smithi038.stdout:1/661: mknod d2/d10/d19/d22/d54/dc3/cd7 0 2024-02-15T18:34:59.139 INFO:tasks.workunit.client.0.smithi038.stdout:1/662: write d2/d10/d62/fd2 [446887,111696] 0 2024-02-15T18:34:59.139 INFO:tasks.workunit.client.0.smithi038.stdout:8/870: rmdir d1/d7/d26 39 2024-02-15T18:34:59.140 INFO:tasks.workunit.client.0.smithi038.stdout:8/871: write d1/d7/d26/d11b/fed [4598064,105328] 0 2024-02-15T18:34:59.140 INFO:tasks.workunit.client.0.smithi038.stdout:8/872: fdatasync d1/d7/d26/f84 0 2024-02-15T18:34:59.142 INFO:tasks.workunit.client.0.smithi038.stdout:9/800: rename d1/da/d20/d58/d94/fc7 to d1/da/d10/d13/d3d/d43/d4d/dfd/f11d 0 2024-02-15T18:34:59.142 INFO:tasks.workunit.client.0.smithi038.stdout:9/801: chown d1/da/d36/d3f/c102 4592 1 2024-02-15T18:34:59.142 INFO:tasks.workunit.client.0.smithi038.stdout:9/802: fsync d1/da/d20/d58/d94/db9/dc6/d49/de4/fce 0 2024-02-15T18:34:59.142 INFO:tasks.workunit.client.0.smithi038.stdout:2/625: dwrite dd/d12/d8d/fa8 [0,4194304] 0 2024-02-15T18:34:59.142 INFO:tasks.workunit.client.0.smithi038.stdout:2/626: dread - dd/d12/d5d/d54/fa4 zero size 2024-02-15T18:34:59.142 INFO:tasks.workunit.client.0.smithi038.stdout:2/627: chown dd/d12/d5d/d93 3323 1 2024-02-15T18:34:59.143 INFO:tasks.workunit.client.0.smithi038.stdout:5/672: rmdir d2/d48/d83 39 2024-02-15T18:34:59.143 INFO:tasks.workunit.client.0.smithi038.stdout:5/673: chown d2/d48/d83/d96/d8e/lc0 1317 1 2024-02-15T18:34:59.144 INFO:tasks.workunit.client.0.smithi038.stdout:9/803: mknod d1/da/d20/d58/d94/db9/dc6/c11e 0 2024-02-15T18:34:59.145 INFO:tasks.workunit.client.0.smithi038.stdout:8/873: dread d1/d4/d5/d31/d65/f128 [0,4194304] 0 2024-02-15T18:34:59.146 INFO:tasks.workunit.client.0.smithi038.stdout:8/874: write d1/d4/d5/d77/f9f [977676,47523] 0 2024-02-15T18:34:59.148 INFO:tasks.workunit.client.0.smithi038.stdout:9/804: rename d1/da/d36/d8b/f11a to d1/da/d20/d58/f11f 0 2024-02-15T18:34:59.148 INFO:tasks.workunit.client.0.smithi038.stdout:9/805: dread - d1/da/d10/fef zero size 2024-02-15T18:34:59.150 INFO:tasks.workunit.client.0.smithi038.stdout:5/674: dread d2/d30/d57/f79 [0,4194304] 0 2024-02-15T18:34:59.152 INFO:tasks.workunit.client.0.smithi038.stdout:8/875: creat d1/d3a/df6/d12b/f136 x:0 0 0 2024-02-15T18:34:59.153 INFO:tasks.workunit.client.0.smithi038.stdout:9/806: creat d1/da/d20/d58/d94/f120 x:0 0 0 2024-02-15T18:34:59.154 INFO:tasks.workunit.client.0.smithi038.stdout:5/675: symlink d2/d30/lc5 0 2024-02-15T18:34:59.156 INFO:tasks.workunit.client.0.smithi038.stdout:2/628: dread dd/d12/d3a/d5e/f87 [0,4194304] 0 2024-02-15T18:34:59.156 INFO:tasks.workunit.client.0.smithi038.stdout:2/629: chown dd/d12/d1d/d3b/d3c/d73 5038 1 2024-02-15T18:34:59.156 INFO:tasks.workunit.client.0.smithi038.stdout:8/876: unlink d1/d7/d26/d11b/f122 0 2024-02-15T18:34:59.157 INFO:tasks.workunit.client.0.smithi038.stdout:8/877: chown d1/l3c 417 1 2024-02-15T18:34:59.157 INFO:tasks.workunit.client.0.smithi038.stdout:8/878: chown d1/d4/d5/d6/d6d/df5/l10f 11265 1 2024-02-15T18:34:59.157 INFO:tasks.workunit.client.0.smithi038.stdout:8/879: chown d1/d4/d10/f1e 11552061 1 2024-02-15T18:34:59.157 INFO:tasks.workunit.client.0.smithi038.stdout:8/880: chown d1/d7/l9c 10008391 1 2024-02-15T18:34:59.157 INFO:tasks.workunit.client.0.smithi038.stdout:8/881: readlink d1/d4/d5/d31/d65/d80/lfc 0 2024-02-15T18:34:59.157 INFO:tasks.workunit.client.0.smithi038.stdout:8/882: write d1/d3a/f6c [313993,44080] 0 2024-02-15T18:34:59.157 INFO:tasks.workunit.client.0.smithi038.stdout:1/663: dwrite d2/d2e/f38 [0,4194304] 0 2024-02-15T18:34:59.157 INFO:tasks.workunit.client.0.smithi038.stdout:9/807: mkdir d1/da/d10/d13/d3d/d43/d4d/d105/d121 0 2024-02-15T18:34:59.158 INFO:tasks.workunit.client.0.smithi038.stdout:5/676: link d2/d30/la2 d2/d64/d99/db6/lc6 0 2024-02-15T18:34:59.162 INFO:tasks.workunit.client.0.smithi038.stdout:2/630: mkdir dd/d12/d5d/d54/dcb 0 2024-02-15T18:34:59.164 INFO:tasks.workunit.client.0.smithi038.stdout:2/631: dread dd/d12/d3a/f45 [0,4194304] 0 2024-02-15T18:34:59.165 INFO:tasks.workunit.client.0.smithi038.stdout:2/632: write dd/d12/d5d/fb1 [615740,115550] 0 2024-02-15T18:34:59.165 INFO:tasks.workunit.client.0.smithi038.stdout:5/677: mknod d2/d64/d99/db6/cc7 0 2024-02-15T18:34:59.165 INFO:tasks.workunit.client.0.smithi038.stdout:5/678: chown d2/d64/d99/db6/lc6 80573 1 2024-02-15T18:34:59.165 INFO:tasks.workunit.client.0.smithi038.stdout:5/679: chown d2/c34 79714103 1 2024-02-15T18:34:59.165 INFO:tasks.workunit.client.0.smithi038.stdout:5/680: truncate d2/d30/d52/fad 295950 0 2024-02-15T18:34:59.165 INFO:tasks.workunit.client.0.smithi038.stdout:8/883: dread d1/d7/d22/f85 [0,4194304] 0 2024-02-15T18:34:59.166 INFO:tasks.workunit.client.0.smithi038.stdout:5/681: write d2/d3/d12/d5e/d86/fc2 [3216781,83262] 0 2024-02-15T18:34:59.166 INFO:tasks.workunit.client.0.smithi038.stdout:1/664: rename d2/d10/d19/d22/d54/d80/fc6 to d2/d10/d19/d22/d54/fd8 0 2024-02-15T18:34:59.166 INFO:tasks.workunit.client.0.smithi038.stdout:8/884: write d1/d4/d5/d31/d65/f128 [3737821,124739] 0 2024-02-15T18:34:59.167 INFO:tasks.workunit.client.0.smithi038.stdout:2/633: mknod dd/d12/d5d/d33/ccc 0 2024-02-15T18:34:59.168 INFO:tasks.workunit.client.0.smithi038.stdout:5/682: unlink d2/d30/d57/f79 0 2024-02-15T18:34:59.168 INFO:tasks.workunit.client.0.smithi038.stdout:1/665: dread d2/d10/d19/d5f/f97 [0,4194304] 0 2024-02-15T18:34:59.172 INFO:tasks.workunit.client.0.smithi038.stdout:8/885: unlink d1/f6f 0 2024-02-15T18:34:59.173 INFO:tasks.workunit.client.0.smithi038.stdout:8/886: fsync d1/d4/dbb/fbf 0 2024-02-15T18:34:59.173 INFO:tasks.workunit.client.0.smithi038.stdout:2/634: mkdir dd/d12/d3a/d5e/d65/dcd 0 2024-02-15T18:34:59.173 INFO:tasks.workunit.client.0.smithi038.stdout:9/808: dwrite d1/da/fed [0,4194304] 0 2024-02-15T18:34:59.173 INFO:tasks.workunit.client.0.smithi038.stdout:5/683: creat d2/d3/da0/d33/fc8 x:0 0 0 2024-02-15T18:34:59.173 INFO:tasks.workunit.client.0.smithi038.stdout:8/887: truncate d1/d3a/f7a 3977652 0 2024-02-15T18:34:59.174 INFO:tasks.workunit.client.0.smithi038.stdout:2/635: rmdir dd 39 2024-02-15T18:34:59.174 INFO:tasks.workunit.client.0.smithi038.stdout:2/636: chown dd/d12/d59/db2 771066 1 2024-02-15T18:34:59.174 INFO:tasks.workunit.client.0.smithi038.stdout:2/637: stat dd/d12/d8d/d90/faf 0 2024-02-15T18:34:59.174 INFO:tasks.workunit.client.0.smithi038.stdout:2/638: stat dd/d12/d3a/c85 0 2024-02-15T18:34:59.174 INFO:tasks.workunit.client.0.smithi038.stdout:2/639: chown dd/d12/d59/d2b/d7a 6 1 2024-02-15T18:34:59.179 INFO:tasks.workunit.client.0.smithi038.stdout:2/640: read dd/d12/d3a/d5e/f87 [383000,14708] 0 2024-02-15T18:34:59.184 INFO:tasks.workunit.client.0.smithi038.stdout:7/691: sync 2024-02-15T18:34:59.184 INFO:tasks.workunit.client.0.smithi038.stdout:3/614: sync 2024-02-15T18:34:59.184 INFO:tasks.workunit.client.0.smithi038.stdout:0/787: sync 2024-02-15T18:34:59.184 INFO:tasks.workunit.client.0.smithi038.stdout:6/658: sync 2024-02-15T18:34:59.184 INFO:tasks.workunit.client.0.smithi038.stdout:4/716: sync 2024-02-15T18:34:59.184 INFO:tasks.workunit.client.0.smithi038.stdout:0/788: read - d4/d13/d2a/d35/d1e/d32/d3b/daa/df8/fd1 zero size 2024-02-15T18:34:59.184 INFO:tasks.workunit.client.0.smithi038.stdout:7/692: write d0/d68/fe1 [943627,10326] 0 2024-02-15T18:34:59.184 INFO:tasks.workunit.client.0.smithi038.stdout:0/789: chown d4/db3/lfb 645007294 1 2024-02-15T18:34:59.186 INFO:tasks.workunit.client.0.smithi038.stdout:9/809: creat d1/da/d20/d58/d94/db9/dc6/f122 x:0 0 0 2024-02-15T18:34:59.186 INFO:tasks.workunit.client.0.smithi038.stdout:9/810: dread - d1/da/d10/d13/d3d/d43/d4d/f7b zero size 2024-02-15T18:34:59.194 INFO:tasks.workunit.client.0.smithi038.stdout:8/888: dwrite d1/d4/d5/d31/d65/ffa [4194304,4194304] 0 2024-02-15T18:34:59.196 INFO:tasks.workunit.client.0.smithi038.stdout:5/684: getdents d2/d30/d57 0 2024-02-15T18:34:59.200 INFO:tasks.workunit.client.0.smithi038.stdout:2/641: rmdir dd/d12/d59/d2b 39 2024-02-15T18:34:59.200 INFO:tasks.workunit.client.0.smithi038.stdout:6/659: dread d2/f5b [0,4194304] 0 2024-02-15T18:34:59.200 INFO:tasks.workunit.client.0.smithi038.stdout:6/660: chown d2/d4/d19/d78/d7a/ca0 1263 1 2024-02-15T18:34:59.200 INFO:tasks.workunit.client.0.smithi038.stdout:7/693: link d0/db/d59/d15/d4f/cb9 d0/db/d19/d7d/cf5 0 2024-02-15T18:34:59.201 INFO:tasks.workunit.client.0.smithi038.stdout:0/790: mkdir d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/d109 0 2024-02-15T18:34:59.203 INFO:tasks.workunit.client.0.smithi038.stdout:8/889: mknod d1/d4/d10/d43/c137 0 2024-02-15T18:34:59.205 INFO:tasks.workunit.client.0.smithi038.stdout:5/685: rename d2/d3/ff to d2/d3/d12/d89/fc9 0 2024-02-15T18:34:59.205 INFO:tasks.workunit.client.0.smithi038.stdout:5/686: truncate d2/f80 4591643 0 2024-02-15T18:34:59.206 INFO:tasks.workunit.client.0.smithi038.stdout:6/661: mkdir d2/d10/dd0 0 2024-02-15T18:34:59.207 INFO:tasks.workunit.client.0.smithi038.stdout:6/662: write d2/f58 [71659,7592] 0 2024-02-15T18:34:59.207 INFO:tasks.workunit.client.0.smithi038.stdout:6/663: chown d2/d4/d19/d55/f6a 28600 1 2024-02-15T18:34:59.207 INFO:tasks.workunit.client.0.smithi038.stdout:6/664: stat d2/d4/d19/c41 0 2024-02-15T18:34:59.207 INFO:tasks.workunit.client.0.smithi038.stdout:5/687: rename d2/d3/d12/d5e/d86/l87 to d2/d64/lca 0 2024-02-15T18:34:59.209 INFO:tasks.workunit.client.0.smithi038.stdout:6/665: truncate d2/d4/d2c/d39/db4/f93 3525779 0 2024-02-15T18:34:59.209 INFO:tasks.workunit.client.0.smithi038.stdout:6/666: read d2/f58 [23809,96123] 0 2024-02-15T18:34:59.210 INFO:tasks.workunit.client.0.smithi038.stdout:5/688: creat d2/d3/db9/fcb x:0 0 0 2024-02-15T18:34:59.210 INFO:tasks.workunit.client.0.smithi038.stdout:5/689: fdatasync d2/d3/da0/d33/fc8 0 2024-02-15T18:34:59.212 INFO:tasks.workunit.client.0.smithi038.stdout:6/667: unlink d2/d4/d2c/d39/db4/d5f/d99/lb2 0 2024-02-15T18:34:59.212 INFO:tasks.workunit.client.0.smithi038.stdout:6/668: write d2/d10/db6/fa5 [464034,43888] 0 2024-02-15T18:34:59.216 INFO:tasks.workunit.client.0.smithi038.stdout:3/615: dwrite d2/d1b/f94 [0,4194304] 0 2024-02-15T18:34:59.216 INFO:tasks.workunit.client.0.smithi038.stdout:3/616: chown d2/d25/c28 25030524 1 2024-02-15T18:34:59.220 INFO:tasks.workunit.client.0.smithi038.stdout:3/617: unlink d2/d25/f30 0 2024-02-15T18:34:59.220 INFO:tasks.workunit.client.0.smithi038.stdout:3/618: truncate d2/d24/f49 4347958 0 2024-02-15T18:34:59.227 INFO:tasks.workunit.client.0.smithi038.stdout:8/890: dread d1/d7/d22/f85 [0,4194304] 0 2024-02-15T18:34:59.227 INFO:tasks.workunit.client.0.smithi038.stdout:8/891: readlink d1/d4/d5/d31/d65/l131 0 2024-02-15T18:34:59.227 INFO:tasks.workunit.client.0.smithi038.stdout:9/811: dwrite d1/dc/d4f/f47 [0,4194304] 0 2024-02-15T18:34:59.227 INFO:tasks.workunit.client.0.smithi038.stdout:8/892: write d1/d4/d10/d43/d48/d52/f134 [498786,110876] 0 2024-02-15T18:34:59.227 INFO:tasks.workunit.client.0.smithi038.stdout:8/893: write d1/d7/d22/da7/f100 [92991,69610] 0 2024-02-15T18:34:59.227 INFO:tasks.workunit.client.0.smithi038.stdout:8/894: write d1/d4/f9b [77821,100878] 0 2024-02-15T18:34:59.228 INFO:tasks.workunit.client.0.smithi038.stdout:9/812: symlink d1/dc/d5a/d7f/l123 0 2024-02-15T18:34:59.251 INFO:tasks.workunit.client.0.smithi038.stdout:4/717: dwrite d11/d1e/d20/d2f/fd3 [0,4194304] 0 2024-02-15T18:34:59.252 INFO:tasks.workunit.client.0.smithi038.stdout:4/718: read d11/f17 [2289393,68561] 0 2024-02-15T18:34:59.254 INFO:tasks.workunit.client.0.smithi038.stdout:0/791: write d4/f12 [1165150,21327] 0 2024-02-15T18:34:59.256 INFO:tasks.workunit.client.0.smithi038.stdout:4/719: truncate d11/d1e/d3e/f54 460488 0 2024-02-15T18:34:59.256 INFO:tasks.workunit.client.0.smithi038.stdout:4/720: stat d11/d1e/d20/d2b/d4d/d67/d73 0 2024-02-15T18:34:59.256 INFO:tasks.workunit.client.0.smithi038.stdout:0/792: truncate d4/d13/d2a/d35/d6a/fdf 3821215 0 2024-02-15T18:34:59.257 INFO:tasks.workunit.client.0.smithi038.stdout:0/793: read d4/d13/d2a/d35/f49 [152571,32163] 0 2024-02-15T18:34:59.257 INFO:tasks.workunit.client.0.smithi038.stdout:4/721: mknod d11/d1e/d20/d31/d48/d102/c109 0 2024-02-15T18:34:59.258 INFO:tasks.workunit.client.0.smithi038.stdout:4/722: getdents d11/d1e/d20/d2b/d65 0 2024-02-15T18:34:59.261 INFO:tasks.workunit.client.0.smithi038.stdout:2/642: dwrite dd/d12/d3a/d5e/d65/fba [0,4194304] 0 2024-02-15T18:34:59.262 INFO:tasks.workunit.client.0.smithi038.stdout:8/895: fdatasync d1/d4/d10/d43/d48/d9a/fa5 0 2024-02-15T18:34:59.262 INFO:tasks.workunit.client.0.smithi038.stdout:7/694: dwrite d0/db/d59/d15/d29/f31 [0,4194304] 0 2024-02-15T18:34:59.263 INFO:tasks.workunit.client.0.smithi038.stdout:9/813: dwrite d1/dc/d4f/f46 [0,4194304] 0 2024-02-15T18:34:59.263 INFO:tasks.workunit.client.0.smithi038.stdout:9/814: chown f0 761 1 2024-02-15T18:34:59.263 INFO:tasks.workunit.client.0.smithi038.stdout:9/815: fsync d1/da/d10/fef 0 2024-02-15T18:34:59.263 INFO:tasks.workunit.client.0.smithi038.stdout:6/669: dwrite d2/d4/d28/f3a [0,4194304] 0 2024-02-15T18:34:59.264 INFO:tasks.workunit.client.0.smithi038.stdout:6/670: chown d2/f3 1357851 1 2024-02-15T18:34:59.264 INFO:tasks.workunit.client.0.smithi038.stdout:5/690: dwrite d2/d3/d12/d5e/d86/fa9 [0,4194304] 0 2024-02-15T18:34:59.264 INFO:tasks.workunit.client.0.smithi038.stdout:6/671: chown d2/d4/d6c/d9d/lc9 2313840 1 2024-02-15T18:34:59.264 INFO:tasks.workunit.client.0.smithi038.stdout:0/794: dread d4/d13/d2a/d35/d1e/d32/d48/f56 [4194304,4194304] 0 2024-02-15T18:34:59.264 INFO:tasks.workunit.client.0.smithi038.stdout:0/795: chown d4/d13/d19/f43 12 1 2024-02-15T18:34:59.275 INFO:tasks.workunit.client.0.smithi038.stdout:9/816: getdents d1/da/d10/d13/d3d/d4b 0 2024-02-15T18:34:59.276 INFO:tasks.workunit.client.0.smithi038.stdout:6/672: getdents d2/d4/d2c/d39 0 2024-02-15T18:34:59.277 INFO:tasks.workunit.client.0.smithi038.stdout:0/796: dread d4/d13/f54 [0,4194304] 0 2024-02-15T18:34:59.278 INFO:tasks.workunit.client.0.smithi038.stdout:5/691: rename d2/d30/d52 to d2/dcc 0 2024-02-15T18:34:59.279 INFO:tasks.workunit.client.0.smithi038.stdout:5/692: readlink d2/d48/d83/d96/d8e/lc0 0 2024-02-15T18:34:59.279 INFO:tasks.workunit.client.0.smithi038.stdout:5/693: fdatasync d2/dcc/f97 0 2024-02-15T18:34:59.279 INFO:tasks.workunit.client.0.smithi038.stdout:4/723: dwrite d11/d1e/d43/d5e/dfe/ffb [0,4194304] 0 2024-02-15T18:34:59.289 INFO:tasks.workunit.client.0.smithi038.stdout:9/817: creat d1/f124 x:0 0 0 2024-02-15T18:34:59.291 INFO:tasks.workunit.client.0.smithi038.stdout:6/673: unlink d2/d4/d19/d55/f81 0 2024-02-15T18:34:59.292 INFO:tasks.workunit.client.0.smithi038.stdout:0/797: rename d4/d13/cc1 to d4/d13/d19/dfc/c10a 0 2024-02-15T18:34:59.292 INFO:tasks.workunit.client.0.smithi038.stdout:6/674: dread d2/d4/d6f/f83 [0,4194304] 0 2024-02-15T18:34:59.293 INFO:tasks.workunit.client.0.smithi038.stdout:6/675: readlink d2/l92 0 2024-02-15T18:34:59.293 INFO:tasks.workunit.client.0.smithi038.stdout:5/694: creat d2/d9a/fcd x:0 0 0 2024-02-15T18:34:59.295 INFO:tasks.workunit.client.0.smithi038.stdout:4/724: link d11/d1e/d20/d2b/d4d/f95 d11/d1e/d20/d31/d8d/de7/f10a 0 2024-02-15T18:34:59.296 INFO:tasks.workunit.client.0.smithi038.stdout:9/818: symlink d1/da/d20/d58/d76/l125 0 2024-02-15T18:34:59.296 INFO:tasks.workunit.client.0.smithi038.stdout:1/666: sync 2024-02-15T18:34:59.296 INFO:tasks.workunit.client.0.smithi038.stdout:1/667: dread - d2/d10/d19/fcf zero size 2024-02-15T18:34:59.297 INFO:tasks.workunit.client.0.smithi038.stdout:6/676: rmdir d2/d4/d6c 39 2024-02-15T18:34:59.297 INFO:tasks.workunit.client.0.smithi038.stdout:5/695: creat d2/d3/d5f/fce x:0 0 0 2024-02-15T18:34:59.299 INFO:tasks.workunit.client.0.smithi038.stdout:9/819: getdents d1/da/d20/d58/d94/db9/dc6 0 2024-02-15T18:34:59.299 INFO:tasks.workunit.client.0.smithi038.stdout:5/696: dread d2/d30/d57/f90 [0,4194304] 0 2024-02-15T18:34:59.300 INFO:tasks.workunit.client.0.smithi038.stdout:5/697: truncate d2/dcc/f5b 2492153 0 2024-02-15T18:34:59.302 INFO:tasks.workunit.client.0.smithi038.stdout:1/668: link d2/d10/fa5 d2/d10/d19/d22/d54/fd9 0 2024-02-15T18:34:59.302 INFO:tasks.workunit.client.0.smithi038.stdout:1/669: chown d2/d10/d32/c65 1873374 1 2024-02-15T18:34:59.304 INFO:tasks.workunit.client.0.smithi038.stdout:5/698: rename d2/d3/d12/d5e/d86 to d2/d3/db9/dcf 0 2024-02-15T18:34:59.306 INFO:tasks.workunit.client.0.smithi038.stdout:1/670: truncate d2/f1e 13412 0 2024-02-15T18:34:59.309 INFO:tasks.workunit.client.0.smithi038.stdout:1/671: rename d2/d2e/d8c/fb0 to d2/d10/d19/d22/fda 0 2024-02-15T18:34:59.309 INFO:tasks.workunit.client.0.smithi038.stdout:1/672: chown d2/d10/f31 11334 1 2024-02-15T18:34:59.314 INFO:tasks.workunit.client.0.smithi038.stdout:0/798: dwrite d4/d13/d19/f43 [0,4194304] 0 2024-02-15T18:34:59.315 INFO:tasks.workunit.client.0.smithi038.stdout:0/799: mknod d4/db3/c10b 0 2024-02-15T18:34:59.317 INFO:tasks.workunit.client.0.smithi038.stdout:0/800: mkdir d4/d13/d53/d10c 0 2024-02-15T18:34:59.326 INFO:tasks.workunit.client.0.smithi038.stdout:6/677: dwrite d2/d4/d2c/d39/d76/dc2/fce [0,4194304] 0 2024-02-15T18:34:59.326 INFO:tasks.workunit.client.0.smithi038.stdout:6/678: chown d2 614 1 2024-02-15T18:34:59.326 INFO:tasks.workunit.client.0.smithi038.stdout:6/679: write d2/d4/d19/d55/f8c [672492,6702] 0 2024-02-15T18:34:59.326 INFO:tasks.workunit.client.0.smithi038.stdout:6/680: fdatasync d2/d4/d2c/f4f 0 2024-02-15T18:34:59.329 INFO:tasks.workunit.client.0.smithi038.stdout:4/725: dwrite d11/d1e/d20/d31/d48/d102/dca/fed [0,4194304] 0 2024-02-15T18:34:59.331 INFO:tasks.workunit.client.0.smithi038.stdout:4/726: rmdir d11/d1e/d20/d31/d8d/d99 39 2024-02-15T18:34:59.331 INFO:tasks.workunit.client.0.smithi038.stdout:4/727: write d11/d1e/d3e/d57/f90 [861582,48474] 0 2024-02-15T18:34:59.334 INFO:tasks.workunit.client.0.smithi038.stdout:6/681: write d2/d4/d2c/f4f [3628632,81490] 0 2024-02-15T18:34:59.334 INFO:tasks.workunit.client.0.smithi038.stdout:6/682: chown d2/d4/d2c/d39/d76/l90 18316363 1 2024-02-15T18:34:59.334 INFO:tasks.workunit.client.0.smithi038.stdout:6/683: write d2/d4/d2c/d39/db4/d5f/d99/fb0 [996565,81478] 0 2024-02-15T18:34:59.336 INFO:tasks.workunit.client.0.smithi038.stdout:4/728: creat d11/d1e/d3e/d57/f10b x:0 0 0 2024-02-15T18:34:59.337 INFO:tasks.workunit.client.0.smithi038.stdout:6/684: creat d2/d10/fd1 x:0 0 0 2024-02-15T18:34:59.338 INFO:tasks.workunit.client.0.smithi038.stdout:0/801: dwrite d4/d13/d2a/d35/db2/fcd [0,4194304] 0 2024-02-15T18:34:59.338 INFO:tasks.workunit.client.0.smithi038.stdout:0/802: getdents d4/d13/d2a/d35/d1e/d32/d79 0 2024-02-15T18:34:59.338 INFO:tasks.workunit.client.0.smithi038.stdout:4/729: link d11/d1e/d20/d2b/d35/fcd d11/d1e/d20/d2b/d4d/d67/d73/f10c 0 2024-02-15T18:34:59.340 INFO:tasks.workunit.client.0.smithi038.stdout:4/730: write d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/f9b [532772,37543] 0 2024-02-15T18:34:59.342 INFO:tasks.workunit.client.0.smithi038.stdout:6/685: unlink d2/d4/d19/fa3 0 2024-02-15T18:34:59.344 INFO:tasks.workunit.client.0.smithi038.stdout:0/803: mknod d4/d13/d2a/d35/d1e/d32/d79/c10d 0 2024-02-15T18:34:59.346 INFO:tasks.workunit.client.0.smithi038.stdout:4/731: mknod d11/d1e/d20/d31/d48/dac/c10d 0 2024-02-15T18:34:59.348 INFO:tasks.workunit.client.0.smithi038.stdout:4/732: unlink d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/fcc 0 2024-02-15T18:34:59.348 INFO:tasks.workunit.client.0.smithi038.stdout:1/673: dwrite d2/f9d [4194304,4194304] 0 2024-02-15T18:34:59.349 INFO:tasks.workunit.client.0.smithi038.stdout:4/733: getdents d11/d1e/d20/d2b 0 2024-02-15T18:34:59.349 INFO:tasks.workunit.client.0.smithi038.stdout:4/734: dread - d11/d1e/fe8 zero size 2024-02-15T18:34:59.349 INFO:tasks.workunit.client.0.smithi038.stdout:6/686: fsync d2/d4/d2c/d39/db4/d5f/d99/fb0 0 2024-02-15T18:34:59.349 INFO:tasks.workunit.client.0.smithi038.stdout:9/820: dwrite d1/da/d36/d93/d9a/d11c/db8/fbb [0,4194304] 0 2024-02-15T18:34:59.349 INFO:tasks.workunit.client.0.smithi038.stdout:5/699: dwrite d2/d48/f59 [0,4194304] 0 2024-02-15T18:34:59.350 INFO:tasks.workunit.client.0.smithi038.stdout:1/674: creat d2/d39/fdb x:0 0 0 2024-02-15T18:34:59.350 INFO:tasks.workunit.client.0.smithi038.stdout:5/700: read d2/dcc/f7b [549844,101728] 0 2024-02-15T18:34:59.352 INFO:tasks.workunit.client.0.smithi038.stdout:4/735: rename d11/d1e/d3e/f68 to d11/d1e/d20/d2b/d4d/d67/dbf/f10e 0 2024-02-15T18:34:59.354 INFO:tasks.workunit.client.0.smithi038.stdout:9/821: creat d1/da/d10/d13/d3d/d4b/f126 x:0 0 0 2024-02-15T18:34:59.356 INFO:tasks.workunit.client.0.smithi038.stdout:1/675: getdents d2/d10/d62 0 2024-02-15T18:34:59.356 INFO:tasks.workunit.client.0.smithi038.stdout:1/676: fsync d2/d2e/f7d 0 2024-02-15T18:34:59.356 INFO:tasks.workunit.client.0.smithi038.stdout:1/677: dread - d2/d10/d19/d22/f9b zero size 2024-02-15T18:34:59.356 INFO:tasks.workunit.client.0.smithi038.stdout:1/678: chown d2/d10/d19/d5f/fb9 15701 1 2024-02-15T18:34:59.356 INFO:tasks.workunit.client.0.smithi038.stdout:0/804: dwrite d4/d13/d2a/d35/d1e/d32/d48/d9a/fc3 [0,4194304] 0 2024-02-15T18:34:59.356 INFO:tasks.workunit.client.0.smithi038.stdout:0/805: dread - d4/d13/d2a/d35/d1e/d32/d79/d7d/dda/fee zero size 2024-02-15T18:34:59.357 INFO:tasks.workunit.client.0.smithi038.stdout:9/822: link d1/dc/d5a/d7f/l98 d1/da/d20/d58/d94/db9/dc6/l127 0 2024-02-15T18:34:59.358 INFO:tasks.workunit.client.0.smithi038.stdout:6/687: dread d2/d4/d6f/f88 [0,4194304] 0 2024-02-15T18:34:59.358 INFO:tasks.workunit.client.0.smithi038.stdout:1/679: mknod d2/d10/d19/d22/d54/dc3/cdc 0 2024-02-15T18:34:59.364 INFO:tasks.workunit.client.0.smithi038.stdout:0/806: mknod d4/d13/d53/c10e 0 2024-02-15T18:34:59.366 INFO:tasks.workunit.client.0.smithi038.stdout:5/701: dread d2/d48/f59 [0,4194304] 0 2024-02-15T18:34:59.369 INFO:tasks.workunit.client.0.smithi038.stdout:6/688: truncate d2/d4/d2c/d39/db4/d5f/fa1 254409 0 2024-02-15T18:34:59.369 INFO:tasks.workunit.client.0.smithi038.stdout:6/689: fsync d2/d4/d2c/d39/db4/f93 0 2024-02-15T18:34:59.369 INFO:tasks.workunit.client.0.smithi038.stdout:6/690: readlink d2/d4/d2c/d39/l3e 0 2024-02-15T18:34:59.369 INFO:tasks.workunit.client.0.smithi038.stdout:5/702: mkdir d2/d3/dd0 0 2024-02-15T18:34:59.373 INFO:tasks.workunit.client.0.smithi038.stdout:9/823: rename d1/da/d10/d13/d3d/d4b/f126 to d1/da/d36/d93/d9a/d11c/db8/f128 0 2024-02-15T18:34:59.373 INFO:tasks.workunit.client.0.smithi038.stdout:9/824: chown d1/da/d20/d58/d94/db9/dc6/f87 0 1 2024-02-15T18:34:59.374 INFO:tasks.workunit.client.0.smithi038.stdout:6/691: creat d2/d4/d2c/d39/d76/fd2 x:0 0 0 2024-02-15T18:34:59.374 INFO:tasks.workunit.client.0.smithi038.stdout:6/692: write d2/d10/fd1 [449669,50538] 0 2024-02-15T18:34:59.374 INFO:tasks.workunit.client.0.smithi038.stdout:6/693: stat d2/d10/f4e 0 2024-02-15T18:34:59.375 INFO:tasks.workunit.client.0.smithi038.stdout:9/825: rmdir d1/da/d36/d93/d9a/d11c 39 2024-02-15T18:34:59.375 INFO:tasks.workunit.client.0.smithi038.stdout:9/826: chown d1/da/d10/d13/f61 12 1 2024-02-15T18:34:59.375 INFO:tasks.workunit.client.0.smithi038.stdout:9/827: getdents d1/dc/d4f/ddf 0 2024-02-15T18:34:59.375 INFO:tasks.workunit.client.0.smithi038.stdout:0/807: dread d4/d13/d2a/fcc [4194304,4194304] 0 2024-02-15T18:34:59.376 INFO:tasks.workunit.client.0.smithi038.stdout:6/694: symlink d2/d4/d19/d55/d5c/dc8/ld3 0 2024-02-15T18:34:59.378 INFO:tasks.workunit.client.0.smithi038.stdout:9/828: rename d1/da/d10/ca0 to d1/da/d20/d58/d66/c129 0 2024-02-15T18:34:59.379 INFO:tasks.workunit.client.0.smithi038.stdout:9/829: dread - d1/dc/db2/fdc zero size 2024-02-15T18:34:59.379 INFO:tasks.workunit.client.0.smithi038.stdout:0/808: mkdir d4/d13/d2a/d35/d1e/d32/d10f 0 2024-02-15T18:34:59.381 INFO:tasks.workunit.client.0.smithi038.stdout:6/695: mknod d2/d4/d19/d55/cd4 0 2024-02-15T18:34:59.381 INFO:tasks.workunit.client.0.smithi038.stdout:0/809: truncate d4/d13/d2a/d35/d1e/d32/d79/d7d/f86 473330 0 2024-02-15T18:34:59.381 INFO:tasks.workunit.client.0.smithi038.stdout:0/810: write d4/f12 [4856473,5406] 0 2024-02-15T18:34:59.383 INFO:tasks.workunit.client.0.smithi038.stdout:0/811: creat d4/f110 x:0 0 0 2024-02-15T18:34:59.384 INFO:tasks.workunit.client.0.smithi038.stdout:0/812: mkdir d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/df4/d111 0 2024-02-15T18:34:59.386 INFO:tasks.workunit.client.0.smithi038.stdout:0/813: creat d4/db3/dc7/f112 x:0 0 0 2024-02-15T18:34:59.386 INFO:tasks.workunit.client.0.smithi038.stdout:0/814: truncate d4/d13/d2a/d35/d1e/fac 579318 0 2024-02-15T18:34:59.386 INFO:tasks.workunit.client.0.smithi038.stdout:4/736: dwrite d11/f100 [0,4194304] 0 2024-02-15T18:34:59.387 INFO:tasks.workunit.client.0.smithi038.stdout:1/680: dwrite d2/d10/d19/d22/d54/dc0/d6b/fbe [0,4194304] 0 2024-02-15T18:34:59.388 INFO:tasks.workunit.client.0.smithi038.stdout:6/696: write d2/d4/d2c/f4f [1727366,93040] 0 2024-02-15T18:34:59.388 INFO:tasks.workunit.client.0.smithi038.stdout:0/815: read d4/d13/f98 [976815,119782] 0 2024-02-15T18:34:59.390 INFO:tasks.workunit.client.0.smithi038.stdout:9/830: dread d1/da/d10/d13/f86 [0,4194304] 0 2024-02-15T18:34:59.390 INFO:tasks.workunit.client.0.smithi038.stdout:9/831: write d1/da/fed [5014619,52735] 0 2024-02-15T18:34:59.390 INFO:tasks.workunit.client.0.smithi038.stdout:4/737: dread d11/d1e/d20/d31/d48/fe0 [0,4194304] 0 2024-02-15T18:34:59.390 INFO:tasks.workunit.client.0.smithi038.stdout:9/832: chown d1/da/d20/d58/d76/ff3 69228477 1 2024-02-15T18:34:59.390 INFO:tasks.workunit.client.0.smithi038.stdout:4/738: chown d11/d1e/d20/d31/d8d/d99/ff4 0 1 2024-02-15T18:34:59.390 INFO:tasks.workunit.client.0.smithi038.stdout:0/816: write d4/d13/f98 [598302,42386] 0 2024-02-15T18:34:59.391 INFO:tasks.workunit.client.0.smithi038.stdout:4/739: write d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/f9b [2314498,116152] 0 2024-02-15T18:34:59.391 INFO:tasks.workunit.client.0.smithi038.stdout:4/740: dread - d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/faa zero size 2024-02-15T18:34:59.391 INFO:tasks.workunit.client.0.smithi038.stdout:4/741: stat d11/d1e/d20/l93 0 2024-02-15T18:34:59.392 INFO:tasks.workunit.client.0.smithi038.stdout:5/703: dwrite d2/d3/db9/dcf/fc2 [0,4194304] 0 2024-02-15T18:34:59.393 INFO:tasks.workunit.client.0.smithi038.stdout:5/704: write d2/d3/da0/d33/f93 [1214290,66933] 0 2024-02-15T18:34:59.393 INFO:tasks.workunit.client.0.smithi038.stdout:5/705: chown d2/d3/d5f/fce 23701421 1 2024-02-15T18:34:59.403 INFO:tasks.workunit.client.0.smithi038.stdout:1/681: mkdir d2/d69/da9/dbc/ddd 0 2024-02-15T18:34:59.405 INFO:tasks.workunit.client.0.smithi038.stdout:6/697: symlink d2/d4/d2c/d39/db4/ld5 0 2024-02-15T18:34:59.405 INFO:tasks.workunit.client.0.smithi038.stdout:6/698: chown d2/d4/d28/lc6 1 1 2024-02-15T18:34:59.406 INFO:tasks.workunit.client.0.smithi038.stdout:0/817: mknod d4/d13/d2a/d35/d1e/d32/d3b/daa/c113 0 2024-02-15T18:34:59.406 INFO:tasks.workunit.client.0.smithi038.stdout:0/818: dread - d4/db3/fce zero size 2024-02-15T18:34:59.408 INFO:tasks.workunit.client.0.smithi038.stdout:4/742: getdents d11/d1e/d20/d2f/d71 0 2024-02-15T18:34:59.409 INFO:tasks.workunit.client.0.smithi038.stdout:4/743: chown d11/d1e/d20/d2b/d4d/l9e 23111 1 2024-02-15T18:34:59.409 INFO:tasks.workunit.client.0.smithi038.stdout:4/744: write d11/d1e/d20/d31/d8d/de7/f10a [1587619,56383] 0 2024-02-15T18:34:59.409 INFO:tasks.workunit.client.0.smithi038.stdout:0/819: dread d4/f5 [0,4194304] 0 2024-02-15T18:34:59.409 INFO:tasks.workunit.client.0.smithi038.stdout:3/619: sync 2024-02-15T18:34:59.409 INFO:tasks.workunit.client.0.smithi038.stdout:2/643: sync 2024-02-15T18:34:59.409 INFO:tasks.workunit.client.0.smithi038.stdout:8/896: sync 2024-02-15T18:34:59.409 INFO:tasks.workunit.client.0.smithi038.stdout:7/695: sync 2024-02-15T18:34:59.410 INFO:tasks.workunit.client.0.smithi038.stdout:7/696: fsync d0/d95/f51 0 2024-02-15T18:34:59.410 INFO:tasks.workunit.client.0.smithi038.stdout:5/706: mknod d2/d48/d83/d96/cd1 0 2024-02-15T18:34:59.412 INFO:tasks.workunit.client.0.smithi038.stdout:1/682: mknod d2/d10/d19/d22/d36/d94/cde 0 2024-02-15T18:34:59.412 INFO:tasks.workunit.client.0.smithi038.stdout:1/683: truncate d2/f9d 9169857 0 2024-02-15T18:34:59.412 INFO:tasks.workunit.client.0.smithi038.stdout:1/684: write d2/d10/d19/d5f/fb9 [152296,68571] 0 2024-02-15T18:34:59.412 INFO:tasks.workunit.client.0.smithi038.stdout:4/745: dread d11/d1e/d3e/d57/f82 [0,4194304] 0 2024-02-15T18:34:59.412 INFO:tasks.workunit.client.0.smithi038.stdout:1/685: stat d2/d10/d19/d5f/d92/fb8 0 2024-02-15T18:34:59.412 INFO:tasks.workunit.client.0.smithi038.stdout:4/746: write d11/d1e/d20/d31/d48/d60/fe6 [1352141,74358] 0 2024-02-15T18:34:59.412 INFO:tasks.workunit.client.0.smithi038.stdout:6/699: creat d2/d4/d19/d78/fd6 x:0 0 0 2024-02-15T18:34:59.412 INFO:tasks.workunit.client.0.smithi038.stdout:1/686: read d2/d10/f37 [85305,50013] 0 2024-02-15T18:34:59.413 INFO:tasks.workunit.client.0.smithi038.stdout:6/700: write d2/d4/d2c/f4f [936568,85384] 0 2024-02-15T18:34:59.414 INFO:tasks.workunit.client.0.smithi038.stdout:0/820: link d4/d13/d2a/d35/d1e/d32/d48/d97/fd8 d4/d13/d53/d100/f114 0 2024-02-15T18:34:59.414 INFO:tasks.workunit.client.0.smithi038.stdout:0/821: chown d4/d13/d53/l78 244732 1 2024-02-15T18:34:59.416 INFO:tasks.workunit.client.0.smithi038.stdout:3/620: link d2/lc d2/d1b/d3c/d3a/d55/lc7 0 2024-02-15T18:34:59.418 INFO:tasks.workunit.client.0.smithi038.stdout:5/707: dread d2/d64/f8a [0,4194304] 0 2024-02-15T18:34:59.418 INFO:tasks.workunit.client.0.smithi038.stdout:5/708: chown d2/d3/da0/fae 7 1 2024-02-15T18:34:59.418 INFO:tasks.workunit.client.0.smithi038.stdout:3/621: dread d2/d1b/d4f/fb4 [0,4194304] 0 2024-02-15T18:34:59.419 INFO:tasks.workunit.client.0.smithi038.stdout:9/833: truncate d1/da/fed 4342223 0 2024-02-15T18:34:59.422 INFO:tasks.workunit.client.0.smithi038.stdout:2/644: rmdir dd/d12/d3a/d67 39 2024-02-15T18:34:59.423 INFO:tasks.workunit.client.0.smithi038.stdout:9/834: dread d1/da/d10/d13/d3d/d43/fdd [0,4194304] 0 2024-02-15T18:34:59.426 INFO:tasks.workunit.client.0.smithi038.stdout:8/897: creat d1/d4/d10/d43/d48/f138 x:0 0 0 2024-02-15T18:34:59.427 INFO:tasks.workunit.client.0.smithi038.stdout:7/697: creat d0/d95/db8/ff6 x:0 0 0 2024-02-15T18:34:59.428 INFO:tasks.workunit.client.0.smithi038.stdout:3/622: dread d2/d1b/d4f/f8d [0,4194304] 0 2024-02-15T18:34:59.428 INFO:tasks.workunit.client.0.smithi038.stdout:3/623: chown d2/d24/c5b 560 1 2024-02-15T18:34:59.428 INFO:tasks.workunit.client.0.smithi038.stdout:3/624: chown d2/d1b/l21 7 1 2024-02-15T18:34:59.428 INFO:tasks.workunit.client.0.smithi038.stdout:4/747: creat d11/d1e/d20/d2b/d4d/d67/d73/f10f x:0 0 0 2024-02-15T18:34:59.428 INFO:tasks.workunit.client.0.smithi038.stdout:7/698: write d0/d95/db8/fcc [266770,56609] 0 2024-02-15T18:34:59.428 INFO:tasks.workunit.client.0.smithi038.stdout:0/822: dwrite d4/d13/d2a/d35/d1e/f20 [4194304,4194304] 0 2024-02-15T18:34:59.429 INFO:tasks.workunit.client.0.smithi038.stdout:1/687: creat d2/d10/d19/d5f/fdf x:0 0 0 2024-02-15T18:34:59.430 INFO:tasks.workunit.client.0.smithi038.stdout:6/701: mkdir d2/d4/d6c/d9d/dd7 0 2024-02-15T18:34:59.431 INFO:tasks.workunit.client.0.smithi038.stdout:5/709: truncate d2/f80 4341178 0 2024-02-15T18:34:59.431 INFO:tasks.workunit.client.0.smithi038.stdout:0/823: write d4/ff [2327161,109802] 0 2024-02-15T18:34:59.431 INFO:tasks.workunit.client.0.smithi038.stdout:0/824: write d4/d13/f8e [596402,48372] 0 2024-02-15T18:34:59.438 INFO:tasks.workunit.client.0.smithi038.stdout:2/645: mkdir dd/d12/d59/dce 0 2024-02-15T18:34:59.438 INFO:tasks.workunit.client.0.smithi038.stdout:9/835: symlink d1/da/d36/d93/d9a/l12a 0 2024-02-15T18:34:59.438 INFO:tasks.workunit.client.0.smithi038.stdout:8/898: unlink d1/d7/d26/c56 0 2024-02-15T18:34:59.438 INFO:tasks.workunit.client.0.smithi038.stdout:3/625: mknod d2/d1b/d3c/d41/d4e/d62/cc8 0 2024-02-15T18:34:59.438 INFO:tasks.workunit.client.0.smithi038.stdout:9/836: fdatasync d1/da/d36/d93/d9a/d11c/f57 0 2024-02-15T18:34:59.438 INFO:tasks.workunit.client.0.smithi038.stdout:8/899: fsync d1/d4/d10/d3d/f97 0 2024-02-15T18:34:59.438 INFO:tasks.workunit.client.0.smithi038.stdout:3/626: fsync d2/d1b/d3c/d41/d4e/d62/f75 0 2024-02-15T18:34:59.438 INFO:tasks.workunit.client.0.smithi038.stdout:9/837: readlink d1/da/d20/d58/d94/db9/dc6/lc9 0 2024-02-15T18:34:59.438 INFO:tasks.workunit.client.0.smithi038.stdout:8/900: readlink d1/d4/db1/de5/l94 0 2024-02-15T18:34:59.438 INFO:tasks.workunit.client.0.smithi038.stdout:4/748: creat d11/d1e/d3e/d57/d7e/d105/f110 x:0 0 0 2024-02-15T18:34:59.438 INFO:tasks.workunit.client.0.smithi038.stdout:4/749: fsync d11/f100 0 2024-02-15T18:34:59.439 INFO:tasks.workunit.client.0.smithi038.stdout:7/699: rename d0/db/d59/d15/f18 to d0/db/d59/d15/d29/d7e/ff7 0 2024-02-15T18:34:59.440 INFO:tasks.workunit.client.0.smithi038.stdout:1/688: creat d2/d10/fe0 x:0 0 0 2024-02-15T18:34:59.441 INFO:tasks.workunit.client.0.smithi038.stdout:6/702: creat d2/d4/d19/d78/fd8 x:0 0 0 2024-02-15T18:34:59.442 INFO:tasks.workunit.client.0.smithi038.stdout:6/703: write d2/d4/d2c/d39/d76/fd2 [303851,1424] 0 2024-02-15T18:34:59.445 INFO:tasks.workunit.client.0.smithi038.stdout:2/646: symlink dd/d12/d6b/da5/lcf 0 2024-02-15T18:34:59.446 INFO:tasks.workunit.client.0.smithi038.stdout:9/838: creat d1/da/d36/d8b/f12b x:0 0 0 2024-02-15T18:34:59.447 INFO:tasks.workunit.client.0.smithi038.stdout:5/710: dwrite d2/fc [4194304,4194304] 0 2024-02-15T18:34:59.447 INFO:tasks.workunit.client.0.smithi038.stdout:5/711: stat d2/d3/d12/l94 0 2024-02-15T18:34:59.447 INFO:tasks.workunit.client.0.smithi038.stdout:5/712: write d2/d30/d57/f90 [699297,35337] 0 2024-02-15T18:34:59.448 INFO:tasks.workunit.client.0.smithi038.stdout:8/901: creat d1/d4/d10/d3d/db7/f139 x:0 0 0 2024-02-15T18:34:59.448 INFO:tasks.workunit.client.0.smithi038.stdout:8/902: stat d1/d7/d15/c2b 0 2024-02-15T18:34:59.449 INFO:tasks.workunit.client.0.smithi038.stdout:4/750: mkdir d11/d1e/d20/d2b/d4d/d67/dbf/d111 0 2024-02-15T18:34:59.451 INFO:tasks.workunit.client.0.smithi038.stdout:2/647: dread dd/fc5 [0,4194304] 0 2024-02-15T18:34:59.455 INFO:tasks.workunit.client.0.smithi038.stdout:0/825: rename d4/d13/d2a/d35/d1e/d32/cb4 to d4/d13/d2a/d35/d1e/d32/d48/c115 0 2024-02-15T18:34:59.455 INFO:tasks.workunit.client.0.smithi038.stdout:7/700: fsync d0/db/d59/f1a 0 2024-02-15T18:34:59.455 INFO:tasks.workunit.client.0.smithi038.stdout:0/826: fdatasync d4/d13/f54 0 2024-02-15T18:34:59.455 INFO:tasks.workunit.client.0.smithi038.stdout:7/701: truncate d0/db/d59/d15/d29/d7e/ff7 1393145 0 2024-02-15T18:34:59.455 INFO:tasks.workunit.client.0.smithi038.stdout:0/827: dread - d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/fd7 zero size 2024-02-15T18:34:59.456 INFO:tasks.workunit.client.0.smithi038.stdout:1/689: creat d2/d39/dd4/fe1 x:0 0 0 2024-02-15T18:34:59.456 INFO:tasks.workunit.client.0.smithi038.stdout:1/690: chown d2/d10/d32/d53/d82 506 1 2024-02-15T18:34:59.456 INFO:tasks.workunit.client.0.smithi038.stdout:6/704: creat d2/d4/d19/d55/fd9 x:0 0 0 2024-02-15T18:34:59.459 INFO:tasks.workunit.client.0.smithi038.stdout:5/713: symlink d2/d3/d12/d89/ld2 0 2024-02-15T18:34:59.460 INFO:tasks.workunit.client.0.smithi038.stdout:2/648: symlink dd/d12/d8d/ld0 0 2024-02-15T18:34:59.460 INFO:tasks.workunit.client.0.smithi038.stdout:2/649: readlink dd/d12/d5d/d33/l38 0 2024-02-15T18:34:59.460 INFO:tasks.workunit.client.0.smithi038.stdout:2/650: chown dd/d12/d5d/d93/f9c 3138770 1 2024-02-15T18:34:59.460 INFO:tasks.workunit.client.0.smithi038.stdout:2/651: readlink dd/d12/d8d/d90/la9 0 2024-02-15T18:34:59.461 INFO:tasks.workunit.client.0.smithi038.stdout:2/652: write dd/d12/d1d/f50 [2812830,5544] 0 2024-02-15T18:34:59.461 INFO:tasks.workunit.client.0.smithi038.stdout:3/627: rename d2/d1b/d3c/d41/d4e/d62/cc8 to d2/d24/db9/cc9 0 2024-02-15T18:34:59.462 INFO:tasks.workunit.client.0.smithi038.stdout:0/828: truncate d4/d13/f54 953382 0 2024-02-15T18:34:59.464 INFO:tasks.workunit.client.0.smithi038.stdout:7/702: mknod d0/db/d59/dc8/dd6/d70/cf8 0 2024-02-15T18:34:59.464 INFO:tasks.workunit.client.0.smithi038.stdout:1/691: truncate d2/d10/d19/d22/d36/d94/f9c 1324959 0 2024-02-15T18:34:59.468 INFO:tasks.workunit.client.0.smithi038.stdout:8/903: chown d1/d4/d10/d3d/db7/d108 56 1 2024-02-15T18:34:59.470 INFO:tasks.workunit.client.0.smithi038.stdout:6/705: creat d2/d4/d2c/d39/fda x:0 0 0 2024-02-15T18:34:59.478 INFO:tasks.workunit.client.0.smithi038.stdout:3/628: dwrite d2/d1b/d3c/d41/fb3 [0,4194304] 0 2024-02-15T18:34:59.483 INFO:tasks.workunit.client.0.smithi038.stdout:5/714: creat d2/d48/d83/d96/fd3 x:0 0 0 2024-02-15T18:34:59.491 INFO:tasks.workunit.client.0.smithi038.stdout:9/839: rename d1/da/d20/d58/d94/db9/dc6/l127 to d1/dc/d38/l12c 0 2024-02-15T18:34:59.493 INFO:tasks.workunit.client.0.smithi038.stdout:0/829: link d4/d13/d19/c2c d4/d13/d2a/d35/d1e/d32/d48/c116 0 2024-02-15T18:34:59.494 INFO:tasks.workunit.client.0.smithi038.stdout:1/692: mkdir d2/d69/da9/dad/de2 0 2024-02-15T18:34:59.497 INFO:tasks.workunit.client.0.smithi038.stdout:8/904: rmdir d1/d4/d5/d6/d6d/df5/d11c 0 2024-02-15T18:34:59.498 INFO:tasks.workunit.client.0.smithi038.stdout:6/706: rename d2/d4/d2c/d39/c67 to d2/d4/d2c/d39/cdb 0 2024-02-15T18:34:59.503 INFO:tasks.workunit.client.0.smithi038.stdout:3/629: link d2/d25/c28 d2/d1b/cca 0 2024-02-15T18:34:59.503 INFO:tasks.workunit.client.0.smithi038.stdout:3/630: stat d2/d1b/l3d 0 2024-02-15T18:34:59.504 INFO:tasks.workunit.client.0.smithi038.stdout:5/715: mknod d2/d3/db9/dcf/cd4 0 2024-02-15T18:34:59.507 INFO:tasks.workunit.client.0.smithi038.stdout:9/840: creat d1/da/d20/f12d x:0 0 0 2024-02-15T18:34:59.508 INFO:tasks.workunit.client.0.smithi038.stdout:0/830: creat d4/d13/d2a/d35/d1e/d32/d3b/f117 x:0 0 0 2024-02-15T18:34:59.512 INFO:tasks.workunit.client.0.smithi038.stdout:1/693: symlink d2/d10/d19/d22/d36/le3 0 2024-02-15T18:34:59.512 INFO:tasks.workunit.client.0.smithi038.stdout:8/905: symlink d1/d7/l13a 0 2024-02-15T18:34:59.512 INFO:tasks.workunit.client.0.smithi038.stdout:5/716: symlink d2/ld5 0 2024-02-15T18:34:59.512 INFO:tasks.workunit.client.0.smithi038.stdout:1/694: getdents d2/d10/d19/d22/d36/d94/dc1/dd5 0 2024-02-15T18:34:59.512 INFO:tasks.workunit.client.0.smithi038.stdout:1/695: stat d2/d39/f89 0 2024-02-15T18:34:59.512 INFO:tasks.workunit.client.0.smithi038.stdout:8/906: write d1/d4/d5/d31/d65/d80/fc5 [1057422,115508] 0 2024-02-15T18:34:59.513 INFO:tasks.workunit.client.0.smithi038.stdout:7/703: rename d0/db/d19/de6 to d0/db/d59/df9 0 2024-02-15T18:34:59.513 INFO:tasks.workunit.client.0.smithi038.stdout:9/841: creat d1/da/d10/d13/d3d/d4b/f12e x:0 0 0 2024-02-15T18:34:59.513 INFO:tasks.workunit.client.0.smithi038.stdout:9/842: fsync d1/da/d36/d93/fa4 0 2024-02-15T18:34:59.513 INFO:tasks.workunit.client.0.smithi038.stdout:3/631: dread d2/d1b/d3c/d3a/f91 [0,4194304] 0 2024-02-15T18:34:59.515 INFO:tasks.workunit.client.0.smithi038.stdout:5/717: link d2/d3/d12/f14 d2/dcc/fd6 0 2024-02-15T18:34:59.515 INFO:tasks.workunit.client.0.smithi038.stdout:6/707: dwrite d2/f9 [4194304,4194304] 0 2024-02-15T18:34:59.515 INFO:tasks.workunit.client.0.smithi038.stdout:6/708: dread - d2/d10/f6b zero size 2024-02-15T18:34:59.516 INFO:tasks.workunit.client.0.smithi038.stdout:1/696: rename d2/d10/d32/d53/d82/c9e to d2/d10/d32/d53/d82/ce4 0 2024-02-15T18:34:59.517 INFO:tasks.workunit.client.0.smithi038.stdout:9/843: rename d1/da/d36/d93/d9a/d11c/d107/c6e to d1/da/d10/d13/d3d/d4b/c12f 0 2024-02-15T18:34:59.517 INFO:tasks.workunit.client.0.smithi038.stdout:9/844: dread - d1/da/d36/d8b/f12b zero size 2024-02-15T18:34:59.518 INFO:tasks.workunit.client.0.smithi038.stdout:8/907: dread d1/d4/d5/d31/f33 [0,4194304] 0 2024-02-15T18:34:59.518 INFO:tasks.workunit.client.0.smithi038.stdout:8/908: write d1/d4/d10/d43/f119 [378195,93469] 0 2024-02-15T18:34:59.518 INFO:tasks.workunit.client.0.smithi038.stdout:8/909: readlink d1/d4/d5/d6/l72 0 2024-02-15T18:34:59.519 INFO:tasks.workunit.client.0.smithi038.stdout:8/910: dread - d1/d4/faa zero size 2024-02-15T18:34:59.519 INFO:tasks.workunit.client.0.smithi038.stdout:5/718: write d2/d3/da0/d33/f93 [1381599,94468] 0 2024-02-15T18:34:59.519 INFO:tasks.workunit.client.0.smithi038.stdout:5/719: write d2/dcc/f8b [4899315,72428] 0 2024-02-15T18:34:59.519 INFO:tasks.workunit.client.0.smithi038.stdout:6/709: mkdir d2/d4/d2c/d39/db4/ddc 0 2024-02-15T18:34:59.520 INFO:tasks.workunit.client.0.smithi038.stdout:9/845: symlink d1/da/d36/d93/d9a/l130 0 2024-02-15T18:34:59.521 INFO:tasks.workunit.client.0.smithi038.stdout:8/911: mkdir d1/d7/d15/d4b/d13b 0 2024-02-15T18:34:59.521 INFO:tasks.workunit.client.0.smithi038.stdout:8/912: fdatasync d1/d7/d22/f10c 0 2024-02-15T18:34:59.527 INFO:tasks.workunit.client.0.smithi038.stdout:9/846: write d1/da/f17 [3024091,3340] 0 2024-02-15T18:34:59.532 INFO:tasks.workunit.client.0.smithi038.stdout:9/847: rename d1/da/d10/d13/d3d/d4b/daa/fc0 to d1/dc/d69/d7d/f131 0 2024-02-15T18:34:59.532 INFO:tasks.workunit.client.0.smithi038.stdout:9/848: chown d1/da/d10/d13/d3d/d43/f70 74 1 2024-02-15T18:34:59.534 INFO:tasks.workunit.client.0.smithi038.stdout:1/697: dread d2/d2e/f38 [0,4194304] 0 2024-02-15T18:34:59.538 INFO:tasks.workunit.client.0.smithi038.stdout:1/698: symlink d2/d69/le5 0 2024-02-15T18:34:59.538 INFO:tasks.workunit.client.0.smithi038.stdout:1/699: fdatasync d2/f4 0 2024-02-15T18:34:59.538 INFO:tasks.workunit.client.0.smithi038.stdout:1/700: write d2/d10/fa5 [336030,17901] 0 2024-02-15T18:34:59.539 INFO:tasks.workunit.client.0.smithi038.stdout:3/632: dwrite d2/d1b/d3c/d3a/d55/fb1 [0,4194304] 0 2024-02-15T18:34:59.542 INFO:tasks.workunit.client.0.smithi038.stdout:1/701: rmdir d2/d39/dd4 39 2024-02-15T18:34:59.542 INFO:tasks.workunit.client.0.smithi038.stdout:1/702: fsync d2/d10/d19/d5f/d92/fb8 0 2024-02-15T18:34:59.543 INFO:tasks.workunit.client.0.smithi038.stdout:3/633: mknod d2/d24/ccb 0 2024-02-15T18:34:59.544 INFO:tasks.workunit.client.0.smithi038.stdout:1/703: link d2/d10/d19/d5f/fa8 d2/d10/d19/d22/d36/d94/dc1/fe6 0 2024-02-15T18:34:59.545 INFO:tasks.workunit.client.0.smithi038.stdout:1/704: read d2/d39/f7f [742290,126476] 0 2024-02-15T18:34:59.545 INFO:tasks.workunit.client.0.smithi038.stdout:1/705: dread - d2/d10/d19/fcf zero size 2024-02-15T18:34:59.545 INFO:tasks.workunit.client.0.smithi038.stdout:3/634: link d2/d1b/d3c/d3a/d55/da0/fc6 d2/d25/fcc 0 2024-02-15T18:34:59.548 INFO:tasks.workunit.client.0.smithi038.stdout:7/704: dwrite d0/db/d59/d15/f71 [0,4194304] 0 2024-02-15T18:34:59.549 INFO:tasks.workunit.client.0.smithi038.stdout:7/705: dread d0/db/d59/f1a [0,4194304] 0 2024-02-15T18:34:59.550 INFO:tasks.workunit.client.0.smithi038.stdout:7/706: mkdir d0/d68/dfa 0 2024-02-15T18:34:59.550 INFO:tasks.workunit.client.0.smithi038.stdout:7/707: chown d0/d66/c76 7730 1 2024-02-15T18:34:59.552 INFO:tasks.workunit.client.0.smithi038.stdout:3/635: dread d2/d1b/d3c/d42/fc3 [0,4194304] 0 2024-02-15T18:34:59.552 INFO:tasks.workunit.client.0.smithi038.stdout:8/913: fsync d1/d4/d10/d43/f119 0 2024-02-15T18:34:59.552 INFO:tasks.workunit.client.0.smithi038.stdout:6/710: dwrite d2/d4/d6f/f88 [4194304,4194304] 0 2024-02-15T18:34:59.553 INFO:tasks.workunit.client.0.smithi038.stdout:3/636: mknod d2/d24/db9/ccd 0 2024-02-15T18:34:59.554 INFO:tasks.workunit.client.0.smithi038.stdout:3/637: write d2/d1b/d3c/d3a/d55/f80 [5007167,4101] 0 2024-02-15T18:34:59.557 INFO:tasks.workunit.client.0.smithi038.stdout:9/849: dread d1/da/d36/d93/d9a/d11c/d107/f55 [0,4194304] 0 2024-02-15T18:34:59.557 INFO:tasks.workunit.client.0.smithi038.stdout:5/720: dwrite d2/d3/da0/f5a [0,4194304] 0 2024-02-15T18:34:59.557 INFO:tasks.workunit.client.0.smithi038.stdout:5/721: fdatasync d2/dcc/fad 0 2024-02-15T18:34:59.557 INFO:tasks.workunit.client.0.smithi038.stdout:5/722: dread - d2/d3/da0/d33/f9b zero size 2024-02-15T18:34:59.557 INFO:tasks.workunit.client.0.smithi038.stdout:1/706: write d2/d10/d19/d5f/fa8 [7459647,114279] 0 2024-02-15T18:34:59.557 INFO:tasks.workunit.client.0.smithi038.stdout:1/707: readlink d2/d10/d19/d22/d36/le3 0 2024-02-15T18:34:59.560 INFO:tasks.workunit.client.0.smithi038.stdout:8/914: getdents d1/d4/d5/d6/d6d 0 2024-02-15T18:34:59.560 INFO:tasks.workunit.client.0.smithi038.stdout:8/915: write d1/d4/d5/d77/d79/fea [4316669,69817] 0 2024-02-15T18:34:59.560 INFO:tasks.workunit.client.0.smithi038.stdout:8/916: truncate d1/d4/d10/d43/d48/d52/fe0 557782 0 2024-02-15T18:34:59.560 INFO:tasks.workunit.client.0.smithi038.stdout:8/917: stat d1/d4/d5/f20 0 2024-02-15T18:34:59.561 INFO:tasks.workunit.client.0.smithi038.stdout:6/711: symlink d2/d4/d2c/d39/d76/ldd 0 2024-02-15T18:34:59.561 INFO:tasks.workunit.client.0.smithi038.stdout:6/712: truncate d2/d4/d2c/d39/db4/d5f/fa1 508670 0 2024-02-15T18:34:59.563 INFO:tasks.workunit.client.0.smithi038.stdout:3/638: unlink d2/d1b/d3c/d3a/l77 0 2024-02-15T18:34:59.563 INFO:tasks.workunit.client.0.smithi038.stdout:6/713: dread d2/d10/f4e [0,4194304] 0 2024-02-15T18:34:59.564 INFO:tasks.workunit.client.0.smithi038.stdout:9/850: unlink d1/dc/d5a/le2 0 2024-02-15T18:34:59.564 INFO:tasks.workunit.client.0.smithi038.stdout:9/851: getdents d1/da/d36/d93/d9a/d11c/db8/dcb 0 2024-02-15T18:34:59.564 INFO:tasks.workunit.client.0.smithi038.stdout:9/852: chown d1/da/d36/f10b 3 1 2024-02-15T18:34:59.567 INFO:tasks.workunit.client.0.smithi038.stdout:5/723: unlink d2/d9a/fcd 0 2024-02-15T18:34:59.567 INFO:tasks.workunit.client.0.smithi038.stdout:5/724: stat d2/d3/d12/l6b 0 2024-02-15T18:34:59.567 INFO:tasks.workunit.client.0.smithi038.stdout:5/725: chown d2/d3/da0/lba 18666 1 2024-02-15T18:34:59.570 INFO:tasks.workunit.client.0.smithi038.stdout:1/708: rename d2/d10/d19/d22/d54/dc0/f34 to d2/d10/d19/d22/d36/d94/dc1/dd5/fe7 0 2024-02-15T18:34:59.570 INFO:tasks.workunit.client.0.smithi038.stdout:1/709: chown d2/d10/d19/d22/d54/faf 1 1 2024-02-15T18:34:59.571 INFO:tasks.workunit.client.0.smithi038.stdout:8/918: symlink d1/d4/d5/d6/d6d/df5/d123/l13c 0 2024-02-15T18:34:59.571 INFO:tasks.workunit.client.0.smithi038.stdout:8/919: fdatasync d1/d7/d26/d69/f127 0 2024-02-15T18:34:59.573 INFO:tasks.workunit.client.0.smithi038.stdout:1/710: creat d2/d10/d19/d22/fe8 x:0 0 0 2024-02-15T18:34:59.573 INFO:tasks.workunit.client.0.smithi038.stdout:1/711: fdatasync d2/d2e/f38 0 2024-02-15T18:34:59.574 INFO:tasks.workunit.client.0.smithi038.stdout:8/920: mknod d1/d4/d5/c13d 0 2024-02-15T18:34:59.574 INFO:tasks.workunit.client.0.smithi038.stdout:8/921: readlink d1/d4/d5/d6/db9/lc3 0 2024-02-15T18:34:59.577 INFO:tasks.workunit.client.0.smithi038.stdout:1/712: rename d2/d39/dd4/fe1 to d2/d2e/d8c/fe9 0 2024-02-15T18:34:59.581 INFO:tasks.workunit.client.0.smithi038.stdout:1/713: creat d2/d69/da9/dad/de2/fea x:0 0 0 2024-02-15T18:34:59.581 INFO:tasks.workunit.client.0.smithi038.stdout:1/714: fdatasync d2/d10/f31 0 2024-02-15T18:34:59.581 INFO:tasks.workunit.client.0.smithi038.stdout:1/715: fsync d2/d10/d62/fd2 0 2024-02-15T18:34:59.581 INFO:tasks.workunit.client.0.smithi038.stdout:1/716: fsync d2/d10/d19/d22/d54/faf 0 2024-02-15T18:34:59.583 INFO:tasks.workunit.client.0.smithi038.stdout:3/639: dread d2/d1b/f94 [0,4194304] 0 2024-02-15T18:34:59.584 INFO:tasks.workunit.client.0.smithi038.stdout:7/708: dwrite d0/db/d59/f7 [0,4194304] 0 2024-02-15T18:34:59.584 INFO:tasks.workunit.client.0.smithi038.stdout:6/714: dread d2/d4/d19/d55/d5c/faa [0,4194304] 0 2024-02-15T18:34:59.584 INFO:tasks.workunit.client.0.smithi038.stdout:6/715: fsync d2/d10/f49 0 2024-02-15T18:34:59.584 INFO:tasks.workunit.client.0.smithi038.stdout:6/716: fsync d2/d4/f1e 0 2024-02-15T18:34:59.586 INFO:tasks.workunit.client.0.smithi038.stdout:3/640: mkdir d2/db8/dce 0 2024-02-15T18:34:59.587 INFO:tasks.workunit.client.0.smithi038.stdout:7/709: symlink d0/db/d19/d7d/db5/lfb 0 2024-02-15T18:34:59.588 INFO:tasks.workunit.client.0.smithi038.stdout:6/717: getdents d2/d4/d6c 0 2024-02-15T18:34:59.588 INFO:tasks.workunit.client.0.smithi038.stdout:5/726: dwrite d2/d3/da0/d33/f68 [0,4194304] 0 2024-02-15T18:34:59.589 INFO:tasks.workunit.client.0.smithi038.stdout:6/718: read d2/f5b [460734,58077] 0 2024-02-15T18:34:59.589 INFO:tasks.workunit.client.0.smithi038.stdout:3/641: rmdir d2/d1b/d3c/d3a/d55/d65 39 2024-02-15T18:34:59.589 INFO:tasks.workunit.client.0.smithi038.stdout:3/642: chown d2/d1e/l98 0 1 2024-02-15T18:34:59.592 INFO:tasks.workunit.client.0.smithi038.stdout:7/710: creat d0/db/d19/d2d/ffc x:0 0 0 2024-02-15T18:34:59.603 INFO:tasks.workunit.client.0.smithi038.stdout:5/727: symlink d2/d48/d83/ld7 0 2024-02-15T18:34:59.606 INFO:tasks.workunit.client.0.smithi038.stdout:7/711: mkdir d0/d1b/dfd 0 2024-02-15T18:34:59.606 INFO:tasks.workunit.client.0.smithi038.stdout:3/643: getdents d2/d1b/d3c/d3a/d55/d65 0 2024-02-15T18:34:59.608 INFO:tasks.workunit.client.0.smithi038.stdout:6/719: dwrite d2/d4/d6c/fac [8388608,4194304] 0 2024-02-15T18:34:59.608 INFO:tasks.workunit.client.0.smithi038.stdout:7/712: rename d0/db/d19/d7d/dc9/cf2 to d0/db/d59/dc8/dd6/d61/d90/cfe 0 2024-02-15T18:34:59.608 INFO:tasks.workunit.client.0.smithi038.stdout:7/713: chown d0/d1b/l39 8679468 1 2024-02-15T18:34:59.609 INFO:tasks.workunit.client.0.smithi038.stdout:7/714: fsync d0/d68/fca 0 2024-02-15T18:34:59.609 INFO:tasks.workunit.client.0.smithi038.stdout:7/715: fdatasync d0/db/d59/d15/d29/d7e/f5b 0 2024-02-15T18:34:59.609 INFO:tasks.workunit.client.0.smithi038.stdout:4/751: sync 2024-02-15T18:34:59.609 INFO:tasks.workunit.client.0.smithi038.stdout:4/752: chown d11/d1e/d20/d31/d48/d102/c104 241 1 2024-02-15T18:34:59.609 INFO:tasks.workunit.client.0.smithi038.stdout:6/720: creat d2/d4/d19/d55/fde x:0 0 0 2024-02-15T18:34:59.609 INFO:tasks.workunit.client.0.smithi038.stdout:6/721: chown d2/d4/d19/f74 2457951 1 2024-02-15T18:34:59.610 INFO:tasks.workunit.client.0.smithi038.stdout:7/716: mknod d0/db/d59/d15/d29/d8d/dcd/cff 0 2024-02-15T18:34:59.611 INFO:tasks.workunit.client.0.smithi038.stdout:4/753: creat d11/d1e/d43/d5e/f112 x:0 0 0 2024-02-15T18:34:59.611 INFO:tasks.workunit.client.0.smithi038.stdout:4/754: write d11/d1e/d20/d2b/d4d/d67/d73/fe3 [3183600,35331] 0 2024-02-15T18:34:59.611 INFO:tasks.workunit.client.0.smithi038.stdout:6/722: mknod d2/d4/d2c/d39/d76/cdf 0 2024-02-15T18:34:59.615 INFO:tasks.workunit.client.0.smithi038.stdout:5/728: dread d2/dcc/f5b [0,4194304] 0 2024-02-15T18:34:59.615 INFO:tasks.workunit.client.0.smithi038.stdout:5/729: mkdir d2/d3/db9/dcf/dd8 0 2024-02-15T18:34:59.617 INFO:tasks.workunit.client.0.smithi038.stdout:5/730: creat d2/d3/d12/d5e/fd9 x:0 0 0 2024-02-15T18:34:59.618 INFO:tasks.workunit.client.0.smithi038.stdout:7/717: dread d0/f9 [0,4194304] 0 2024-02-15T18:34:59.618 INFO:tasks.workunit.client.0.smithi038.stdout:7/718: fsync d0/db/d59/dc8/dd6/fd4 0 2024-02-15T18:34:59.619 INFO:tasks.workunit.client.0.smithi038.stdout:7/719: mknod d0/db/d59/dc8/dd6/d61/d90/c100 0 2024-02-15T18:34:59.620 INFO:tasks.workunit.client.0.smithi038.stdout:6/723: dread d2/d4/d2c/d39/db4/f93 [0,4194304] 0 2024-02-15T18:34:59.621 INFO:tasks.workunit.client.0.smithi038.stdout:7/720: creat d0/db/d59/dc8/dd6/d61/f101 x:0 0 0 2024-02-15T18:34:59.621 INFO:tasks.workunit.client.0.smithi038.stdout:7/721: write d0/db/d19/d7d/fd3 [491470,95020] 0 2024-02-15T18:34:59.622 INFO:tasks.workunit.client.0.smithi038.stdout:6/724: creat d2/d4/d2c/d39/fe0 x:0 0 0 2024-02-15T18:34:59.622 INFO:tasks.workunit.client.0.smithi038.stdout:7/722: truncate d0/d95/f51 313345 0 2024-02-15T18:34:59.622 INFO:tasks.workunit.client.0.smithi038.stdout:4/755: dread d11/d1e/d43/d5e/dfe/ffb [0,4194304] 0 2024-02-15T18:34:59.623 INFO:tasks.workunit.client.0.smithi038.stdout:6/725: getdents d2/d4/d19/d78 0 2024-02-15T18:34:59.624 INFO:tasks.workunit.client.0.smithi038.stdout:4/756: getdents d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/ddd/dba 0 2024-02-15T18:34:59.624 INFO:tasks.workunit.client.0.smithi038.stdout:4/757: chown d11/l21 153081 1 2024-02-15T18:34:59.624 INFO:tasks.workunit.client.0.smithi038.stdout:4/758: write d11/d1e/d43/d5e/dfe/ffb [3891662,108993] 0 2024-02-15T18:34:59.624 INFO:tasks.workunit.client.0.smithi038.stdout:6/726: truncate d2/d4/d19/d55/d5c/faa 482385 0 2024-02-15T18:34:59.626 INFO:tasks.workunit.client.0.smithi038.stdout:4/759: unlink d11/d1e/d20/d2b/l38 0 2024-02-15T18:34:59.628 INFO:tasks.workunit.client.0.smithi038.stdout:7/723: read d0/db/d59/d15/d29/d8d/f8f [2209986,82831] 0 2024-02-15T18:34:59.628 INFO:tasks.workunit.client.0.smithi038.stdout:5/731: write d2/d48/f74 [1108038,81358] 0 2024-02-15T18:34:59.628 INFO:tasks.workunit.client.0.smithi038.stdout:5/732: write d2/dcc/fd6 [5486923,106890] 0 2024-02-15T18:34:59.628 INFO:tasks.workunit.client.0.smithi038.stdout:4/760: rename d11/d1e/d20/d31/d48/lf8 to d11/d1e/d43/d5e/d62/l113 0 2024-02-15T18:34:59.628 INFO:tasks.workunit.client.0.smithi038.stdout:4/761: chown d11/d1e/d43/d5e/fd2 2087 1 2024-02-15T18:34:59.628 INFO:tasks.workunit.client.0.smithi038.stdout:4/762: truncate d11/d1e/d20/d2b/d4d/d67/dbf/fd1 824016 0 2024-02-15T18:34:59.629 INFO:tasks.workunit.client.0.smithi038.stdout:3/644: dwrite d2/d1b/d3c/d41/f56 [0,4194304] 0 2024-02-15T18:34:59.630 INFO:tasks.workunit.client.0.smithi038.stdout:7/724: mkdir d0/db/d59/d15/d29/d8d/d102 0 2024-02-15T18:34:59.630 INFO:tasks.workunit.client.0.smithi038.stdout:7/725: write d0/db/d59/d15/fe3 [211311,137] 0 2024-02-15T18:34:59.630 INFO:tasks.workunit.client.0.smithi038.stdout:7/726: dread - d0/db/d59/dc8/dd6/d61/fe2 zero size 2024-02-15T18:34:59.630 INFO:tasks.workunit.client.0.smithi038.stdout:7/727: fsync d0/db/d59/d15/d29/d7e/f73 0 2024-02-15T18:34:59.630 INFO:tasks.workunit.client.0.smithi038.stdout:7/728: truncate d0/db/d59/d15/d29/d7e/f5b 1359763 0 2024-02-15T18:34:59.631 INFO:tasks.workunit.client.0.smithi038.stdout:5/733: creat d2/d3/dd0/fda x:0 0 0 2024-02-15T18:34:59.631 INFO:tasks.workunit.client.0.smithi038.stdout:5/734: chown d2/d3/d5f/fce 1 1 2024-02-15T18:34:59.631 INFO:tasks.workunit.client.0.smithi038.stdout:5/735: write d2/d30/d57/f78 [1040356,30698] 0 2024-02-15T18:34:59.638 INFO:tasks.workunit.client.0.smithi038.stdout:3/645: dread d2/d1b/d3c/d41/d4e/fa3 [0,4194304] 0 2024-02-15T18:34:59.640 INFO:tasks.workunit.client.0.smithi038.stdout:7/729: rename d0/f9 to d0/db/d59/dc8/f103 0 2024-02-15T18:34:59.640 INFO:tasks.workunit.client.0.smithi038.stdout:7/730: write d0/d68/fca [340471,106029] 0 2024-02-15T18:34:59.643 INFO:tasks.workunit.client.0.smithi038.stdout:3/646: rmdir d2/d1b/d3c/d41 39 2024-02-15T18:34:59.643 INFO:tasks.workunit.client.0.smithi038.stdout:3/647: chown d2/f51 3685663 1 2024-02-15T18:34:59.643 INFO:tasks.workunit.client.0.smithi038.stdout:3/648: fdatasync d2/d1b/d3c/f31 0 2024-02-15T18:34:59.644 INFO:tasks.workunit.client.0.smithi038.stdout:7/731: creat d0/d66/f104 x:0 0 0 2024-02-15T18:34:59.644 INFO:tasks.workunit.client.0.smithi038.stdout:7/732: readlink d0/db/d59/dc8/dd6/d61/l69 0 2024-02-15T18:34:59.649 INFO:tasks.workunit.client.0.smithi038.stdout:3/649: mkdir d2/db8/dcf 0 2024-02-15T18:34:59.650 INFO:tasks.workunit.client.0.smithi038.stdout:4/763: dwrite d11/d1e/d43/d5e/dfe/ffb [4194304,4194304] 0 2024-02-15T18:34:59.651 INFO:tasks.workunit.client.0.smithi038.stdout:3/650: creat d2/d1b/d3c/d3a/d55/d65/fd0 x:0 0 0 2024-02-15T18:34:59.652 INFO:tasks.workunit.client.0.smithi038.stdout:4/764: dread d11/d1e/d20/d2b/d65/f6d [4194304,4194304] 0 2024-02-15T18:34:59.653 INFO:tasks.workunit.client.0.smithi038.stdout:3/651: creat d2/d1e/fd1 x:0 0 0 2024-02-15T18:34:59.653 INFO:tasks.workunit.client.0.smithi038.stdout:3/652: chown d2/d25/cc2 6207 1 2024-02-15T18:34:59.654 INFO:tasks.workunit.client.0.smithi038.stdout:4/765: mknod d11/d1e/d20/d31/d48/dac/c114 0 2024-02-15T18:34:59.655 INFO:tasks.workunit.client.0.smithi038.stdout:4/766: creat d11/d1e/d20/d31/d48/dac/dbe/d40/dde/f115 x:0 0 0 2024-02-15T18:34:59.656 INFO:tasks.workunit.client.0.smithi038.stdout:7/733: dwrite d0/d68/fae [0,4194304] 0 2024-02-15T18:34:59.658 INFO:tasks.workunit.client.0.smithi038.stdout:7/734: write d0/d1b/f78 [2174423,104683] 0 2024-02-15T18:34:59.659 INFO:tasks.workunit.client.0.smithi038.stdout:4/767: write d11/f1f [3986561,57720] 0 2024-02-15T18:34:59.659 INFO:tasks.workunit.client.0.smithi038.stdout:4/768: chown d11/ld0 251471870 1 2024-02-15T18:34:59.663 INFO:tasks.workunit.client.0.smithi038.stdout:3/653: getdents d2/d1b/d3c/d3a/d55/d65 0 2024-02-15T18:34:59.664 INFO:tasks.workunit.client.0.smithi038.stdout:7/735: rename d0/db/d59/dc8/dd6/d70/f88 to d0/db/d59/d97/f105 0 2024-02-15T18:34:59.666 INFO:tasks.workunit.client.0.smithi038.stdout:3/654: rmdir d2/d1b/d3c/d41/d4e 39 2024-02-15T18:34:59.668 INFO:tasks.workunit.client.0.smithi038.stdout:3/655: creat d2/d1b/d3c/d3a/d55/da0/fd2 x:0 0 0 2024-02-15T18:34:59.668 INFO:tasks.workunit.client.0.smithi038.stdout:3/656: readlink d2/d25/l34 0 2024-02-15T18:34:59.669 INFO:tasks.workunit.client.0.smithi038.stdout:3/657: mkdir d2/d1b/d69/dd3 0 2024-02-15T18:34:59.669 INFO:tasks.workunit.client.0.smithi038.stdout:3/658: readlink d2/l9b 0 2024-02-15T18:34:59.678 INFO:tasks.workunit.client.0.smithi038.stdout:3/659: dread d2/f51 [0,4194304] 0 2024-02-15T18:34:59.679 INFO:tasks.workunit.client.0.smithi038.stdout:4/769: dwrite d11/d1e/d43/d5e/f112 [0,4194304] 0 2024-02-15T18:34:59.679 INFO:tasks.workunit.client.0.smithi038.stdout:4/770: chown d11/d1e/d20/d31/d7a/fbc 57056 1 2024-02-15T18:34:59.679 INFO:tasks.workunit.client.0.smithi038.stdout:4/771: chown d11/d1e/d20/d2b/d4d/d67/d73/f10c 3997 1 2024-02-15T18:34:59.692 INFO:tasks.workunit.client.0.smithi038.stdout:3/660: dwrite d2/d1b/d69/f96 [0,4194304] 0 2024-02-15T18:34:59.694 INFO:tasks.workunit.client.0.smithi038.stdout:3/661: truncate d2/d1b/d3c/d3a/d55/da0/fc6 2878108 0 2024-02-15T18:34:59.695 INFO:tasks.workunit.client.0.smithi038.stdout:3/662: read - d2/d1e/d7e/fbc zero size 2024-02-15T18:34:59.695 INFO:tasks.workunit.client.0.smithi038.stdout:3/663: write d2/d1b/d3c/d3a/f91 [4483468,52861] 0 2024-02-15T18:34:59.695 INFO:tasks.workunit.client.0.smithi038.stdout:3/664: readlink d2/d1e/l98 0 2024-02-15T18:34:59.695 INFO:tasks.workunit.client.0.smithi038.stdout:3/665: chown d2/d1b/d8c/f99 104221438 1 2024-02-15T18:34:59.711 INFO:tasks.workunit.client.0.smithi038.stdout:3/666: dwrite d2/d1e/f44 [0,4194304] 0 2024-02-15T18:34:59.714 INFO:tasks.workunit.client.0.smithi038.stdout:3/667: truncate d2/f1a 533823 0 2024-02-15T18:34:59.724 INFO:tasks.workunit.client.0.smithi038.stdout:3/668: dread d2/d1e/d7e/fbf [0,4194304] 0 2024-02-15T18:34:59.724 INFO:tasks.workunit.client.0.smithi038.stdout:3/669: truncate d2/d1b/f8e 397982 0 2024-02-15T18:34:59.725 INFO:tasks.workunit.client.0.smithi038.stdout:3/670: creat d2/d1b/d3c/fd4 x:0 0 0 2024-02-15T18:34:59.726 INFO:tasks.workunit.client.0.smithi038.stdout:3/671: dread - d2/d25/d60/fae zero size 2024-02-15T18:34:59.728 INFO:tasks.workunit.client.0.smithi038.stdout:3/672: dread d2/d1b/d4f/f5d [0,4194304] 0 2024-02-15T18:34:59.739 INFO:tasks.workunit.client.0.smithi038.stdout:1/717: sync 2024-02-15T18:34:59.739 INFO:tasks.workunit.client.0.smithi038.stdout:8/922: sync 2024-02-15T18:34:59.739 INFO:tasks.workunit.client.0.smithi038.stdout:2/653: sync 2024-02-15T18:34:59.739 INFO:tasks.workunit.client.0.smithi038.stdout:8/923: readlink d1/d4/d5/d31/d65/l131 0 2024-02-15T18:34:59.739 INFO:tasks.workunit.client.0.smithi038.stdout:2/654: chown dd/d12/d59/d2b/l81 385 1 2024-02-15T18:34:59.739 INFO:tasks.workunit.client.0.smithi038.stdout:0/831: sync 2024-02-15T18:34:59.739 INFO:tasks.workunit.client.0.smithi038.stdout:9/853: sync 2024-02-15T18:34:59.739 INFO:tasks.workunit.client.0.smithi038.stdout:9/854: write d1/da/d10/d13/d3d/d4b/daa/f108 [491244,44844] 0 2024-02-15T18:34:59.740 INFO:tasks.workunit.client.0.smithi038.stdout:2/655: read dd/d12/d1d/f2e [7081723,25400] 0 2024-02-15T18:34:59.740 INFO:tasks.workunit.client.0.smithi038.stdout:1/718: mkdir d2/d10/d19/d22/deb 0 2024-02-15T18:34:59.740 INFO:tasks.workunit.client.0.smithi038.stdout:9/855: dread d1/dc/d69/d7d/fc3 [0,4194304] 0 2024-02-15T18:34:59.741 INFO:tasks.workunit.client.0.smithi038.stdout:9/856: truncate d1/da/d20/d58/f11b 783744 0 2024-02-15T18:34:59.741 INFO:tasks.workunit.client.0.smithi038.stdout:8/924: mkdir d1/d7/d15/d13e 0 2024-02-15T18:34:59.742 INFO:tasks.workunit.client.0.smithi038.stdout:0/832: mkdir d4/d13/d2a/d35/d1e/d32/d79/d7d/da3/d118 0 2024-02-15T18:34:59.742 INFO:tasks.workunit.client.0.smithi038.stdout:0/833: chown d4/db3/c103 62 1 2024-02-15T18:34:59.743 INFO:tasks.workunit.client.0.smithi038.stdout:2/656: link dd/d12/d3a/d67/f27 dd/d12/d59/d2b/fd1 0 2024-02-15T18:34:59.744 INFO:tasks.workunit.client.0.smithi038.stdout:2/657: fsync dd/d12/d5d/d33/fb0 0 2024-02-15T18:34:59.744 INFO:tasks.workunit.client.0.smithi038.stdout:1/719: mknod d2/d69/da9/cec 0 2024-02-15T18:34:59.745 INFO:tasks.workunit.client.0.smithi038.stdout:9/857: creat d1/da/d10/d13/f132 x:0 0 0 2024-02-15T18:34:59.745 INFO:tasks.workunit.client.0.smithi038.stdout:8/925: mkdir d1/d4/d10/d43/d48/d9a/d13f 0 2024-02-15T18:34:59.747 INFO:tasks.workunit.client.0.smithi038.stdout:0/834: rename d4/d13/d2a/d35/d1e/d32/d48/d9a/cf9 to d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/df4/c119 0 2024-02-15T18:34:59.747 INFO:tasks.workunit.client.0.smithi038.stdout:1/720: mkdir d2/d10/d32/d53/d82/ded 0 2024-02-15T18:34:59.747 INFO:tasks.workunit.client.0.smithi038.stdout:1/721: truncate d2/d10/d19/d22/d54/dc0/d6b/fb7 999845 0 2024-02-15T18:34:59.748 INFO:tasks.workunit.client.0.smithi038.stdout:9/858: symlink d1/da/d36/d93/d9a/d11c/db8/dcb/l133 0 2024-02-15T18:34:59.748 INFO:tasks.workunit.client.0.smithi038.stdout:0/835: readlink d4/l47 0 2024-02-15T18:34:59.749 INFO:tasks.workunit.client.0.smithi038.stdout:1/722: write d2/f11 [1034580,37912] 0 2024-02-15T18:34:59.749 INFO:tasks.workunit.client.0.smithi038.stdout:1/723: chown d2/d69/da9/cec 5674 1 2024-02-15T18:34:59.750 INFO:tasks.workunit.client.0.smithi038.stdout:9/859: creat d1/da/d36/f134 x:0 0 0 2024-02-15T18:34:59.750 INFO:tasks.workunit.client.0.smithi038.stdout:9/860: fdatasync d1/dc/db2/fd2 0 2024-02-15T18:34:59.751 INFO:tasks.workunit.client.0.smithi038.stdout:0/836: unlink d4/d13/d53/fa1 0 2024-02-15T18:34:59.751 INFO:tasks.workunit.client.0.smithi038.stdout:0/837: write d4/d13/d2a/d35/feb [550720,4140] 0 2024-02-15T18:34:59.751 INFO:tasks.workunit.client.0.smithi038.stdout:1/724: getdents d2/d10/d19/d22/d54/dc3 0 2024-02-15T18:34:59.753 INFO:tasks.workunit.client.0.smithi038.stdout:9/861: creat d1/da/d10/d13/d3d/dfb/f135 x:0 0 0 2024-02-15T18:34:59.753 INFO:tasks.workunit.client.0.smithi038.stdout:9/862: getdents d1/dc/d4f/ddf 0 2024-02-15T18:34:59.753 INFO:tasks.workunit.client.0.smithi038.stdout:9/863: fsync d1/da/d20/d58/d94/db9/dc6/d49/de4/fce 0 2024-02-15T18:34:59.753 INFO:tasks.workunit.client.0.smithi038.stdout:9/864: fsync d1/da/d36/d8b/f12b 0 2024-02-15T18:34:59.755 INFO:tasks.workunit.client.0.smithi038.stdout:0/838: mkdir d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/d11a 0 2024-02-15T18:34:59.755 INFO:tasks.workunit.client.0.smithi038.stdout:0/839: chown d4/d13/d2a/d35/d1e/d32/d48/d97/fc9 109 1 2024-02-15T18:34:59.755 INFO:tasks.workunit.client.0.smithi038.stdout:0/840: write d4/d13/d53/f7b [515469,113280] 0 2024-02-15T18:34:59.755 INFO:tasks.workunit.client.0.smithi038.stdout:0/841: chown d4/d13/d2a/d35/d1e/d32/d3b/ff3 56778206 1 2024-02-15T18:34:59.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:34:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:34:58.318300+0000 mgr.smithi038.khesbl (mgr.14162) 339 : cluster [DBG] pgmap v281: 65 pgs: 65 active+clean; 1.2 GiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 75 MiB/s rd, 153 MiB/s wr, 659 op/s 2024-02-15T18:34:59.775 INFO:tasks.workunit.client.0.smithi038.stdout:2/658: dwrite dd/d12/d59/f21 [0,4194304] 0 2024-02-15T18:34:59.776 INFO:tasks.workunit.client.0.smithi038.stdout:2/659: mkdir dd/d12/d5d/dc2/dd2 0 2024-02-15T18:34:59.776 INFO:tasks.workunit.client.0.smithi038.stdout:2/660: dread - dd/d12/d3a/d67/fc1 zero size 2024-02-15T18:34:59.776 INFO:tasks.workunit.client.0.smithi038.stdout:2/661: write dd/d12/d3a/d67/d4f/fb6 [382511,81728] 0 2024-02-15T18:34:59.777 INFO:tasks.workunit.client.0.smithi038.stdout:2/662: symlink dd/d12/d59/d62/ld3 0 2024-02-15T18:34:59.778 INFO:tasks.workunit.client.0.smithi038.stdout:8/926: dwrite d1/d7/d26/d69/fc1 [4194304,4194304] 0 2024-02-15T18:34:59.779 INFO:tasks.workunit.client.0.smithi038.stdout:2/663: symlink dd/d12/d6b/ld4 0 2024-02-15T18:34:59.779 INFO:tasks.workunit.client.0.smithi038.stdout:2/664: chown dd/d12/d1d/d3b/l7d 0 1 2024-02-15T18:34:59.779 INFO:tasks.workunit.client.0.smithi038.stdout:2/665: write dd/d12/d3a/d67/f29 [1439855,37133] 0 2024-02-15T18:34:59.780 INFO:tasks.workunit.client.0.smithi038.stdout:8/927: creat d1/d4/d10/d3d/f140 x:0 0 0 2024-02-15T18:34:59.780 INFO:tasks.workunit.client.0.smithi038.stdout:8/928: chown d1/d4/d10/d43/d48/d9a/dd9/c135 1792 1 2024-02-15T18:34:59.780 INFO:tasks.workunit.client.0.smithi038.stdout:8/929: write d1/d3a/d58/f117 [2678484,78788] 0 2024-02-15T18:34:59.780 INFO:tasks.workunit.client.0.smithi038.stdout:2/666: unlink dd/d12/d8d/ld0 0 2024-02-15T18:34:59.781 INFO:tasks.workunit.client.0.smithi038.stdout:8/930: truncate d1/d4/d10/f1e 1060390 0 2024-02-15T18:34:59.782 INFO:tasks.workunit.client.0.smithi038.stdout:2/667: rename dd/d12/d1d/d3b/d3c/d73/lc3 to dd/d12/d59/d2b/d6a/ld5 0 2024-02-15T18:34:59.783 INFO:tasks.workunit.client.0.smithi038.stdout:2/668: readlink dd/d12/d1d/lc8 0 2024-02-15T18:34:59.784 INFO:tasks.workunit.client.0.smithi038.stdout:2/669: link dd/d12/f47 dd/d12/d5d/d33/fd6 0 2024-02-15T18:34:59.784 INFO:tasks.workunit.client.0.smithi038.stdout:2/670: chown dd/d12/d3a/d5e/f87 44424752 1 2024-02-15T18:34:59.788 INFO:tasks.workunit.client.0.smithi038.stdout:2/671: dread dd/d12/d3a/d67/d4f/d5a/f7f [0,4194304] 0 2024-02-15T18:34:59.790 INFO:tasks.workunit.client.0.smithi038.stdout:2/672: mkdir dd/d12/d5d/d33/dd7 0 2024-02-15T18:34:59.790 INFO:tasks.workunit.client.0.smithi038.stdout:2/673: rmdir dd/d12/d3a/d67 39 2024-02-15T18:34:59.797 INFO:tasks.workunit.client.0.smithi038.stdout:9/865: dwrite d1/dc/d4f/f47 [4194304,4194304] 0 2024-02-15T18:34:59.798 INFO:tasks.workunit.client.0.smithi038.stdout:1/725: dwrite d2/d10/d19/d5f/fb9 [0,4194304] 0 2024-02-15T18:34:59.798 INFO:tasks.workunit.client.0.smithi038.stdout:8/931: dread d1/d7/d15/d6a/f6b [0,4194304] 0 2024-02-15T18:34:59.799 INFO:tasks.workunit.client.0.smithi038.stdout:1/726: fdatasync d2/d10/d19/d22/f9b 0 2024-02-15T18:34:59.799 INFO:tasks.workunit.client.0.smithi038.stdout:1/727: truncate d2/d10/d19/f72 710792 0 2024-02-15T18:34:59.799 INFO:tasks.workunit.client.0.smithi038.stdout:8/932: write d1/d4/faa [453121,96268] 0 2024-02-15T18:34:59.799 INFO:tasks.workunit.client.0.smithi038.stdout:0/842: dwrite d4/d13/d2a/d35/d1e/d32/d48/d97/f9b [0,4194304] 0 2024-02-15T18:34:59.802 INFO:tasks.workunit.client.0.smithi038.stdout:1/728: symlink d2/d10/d19/d22/d36/d4f/lee 0 2024-02-15T18:34:59.802 INFO:tasks.workunit.client.0.smithi038.stdout:1/729: dread - d2/d10/d19/d2f/fd6 zero size 2024-02-15T18:34:59.802 INFO:tasks.workunit.client.0.smithi038.stdout:8/933: symlink d1/d4/d5/d6/l141 0 2024-02-15T18:34:59.806 INFO:tasks.workunit.client.0.smithi038.stdout:8/934: write d1/d4/d10/d43/d48/d52/fbd [1583240,69546] 0 2024-02-15T18:34:59.806 INFO:tasks.workunit.client.0.smithi038.stdout:2/674: dwrite dd/d12/d59/f21 [4194304,4194304] 0 2024-02-15T18:34:59.807 INFO:tasks.workunit.client.0.smithi038.stdout:8/935: stat d1/d4/d10/d43/d48/d52/fa9 0 2024-02-15T18:34:59.814 INFO:tasks.workunit.client.0.smithi038.stdout:1/730: read d2/d69/fb4 [585736,66062] 0 2024-02-15T18:34:59.814 INFO:tasks.workunit.client.0.smithi038.stdout:1/731: read - d2/d2e/d8c/fe9 zero size 2024-02-15T18:34:59.814 INFO:tasks.workunit.client.0.smithi038.stdout:1/732: truncate d2/d10/d19/d22/d54/fd9 1377965 0 2024-02-15T18:34:59.819 INFO:tasks.workunit.client.0.smithi038.stdout:9/866: dwrite d1/da/d10/d13/f18 [0,4194304] 0 2024-02-15T18:34:59.819 INFO:tasks.workunit.client.0.smithi038.stdout:9/867: stat d1/da/d20/d58/d94/db9/dc6/f9e 0 2024-02-15T18:34:59.820 INFO:tasks.workunit.client.0.smithi038.stdout:9/868: dread d1/da/d20/d58/d94/db9/dc6/d49/f4e [0,4194304] 0 2024-02-15T18:34:59.824 INFO:tasks.workunit.client.0.smithi038.stdout:0/843: dwrite d4/d13/f39 [4194304,4194304] 0 2024-02-15T18:34:59.834 INFO:tasks.workunit.client.0.smithi038.stdout:8/936: getdents d1/d7 0 2024-02-15T18:34:59.834 INFO:tasks.workunit.client.0.smithi038.stdout:8/937: readlink d1/d4/d5/d6/l72 0 2024-02-15T18:34:59.835 INFO:tasks.workunit.client.0.smithi038.stdout:2/675: rename dd/d12/d5d/d54/d91 to dd/d12/d6b/da5/dd8 0 2024-02-15T18:34:59.836 INFO:tasks.workunit.client.0.smithi038.stdout:1/733: symlink d2/d2e/d8c/lef 0 2024-02-15T18:34:59.837 INFO:tasks.workunit.client.0.smithi038.stdout:1/734: write d2/d10/d19/f72 [433979,40253] 0 2024-02-15T18:34:59.837 INFO:tasks.workunit.client.0.smithi038.stdout:8/938: write d1/d4/d10/d43/f46 [312293,75689] 0 2024-02-15T18:34:59.840 INFO:tasks.workunit.client.0.smithi038.stdout:9/869: dwrite d1/da/d20/d58/d94/db9/ff1 [0,4194304] 0 2024-02-15T18:34:59.840 INFO:tasks.workunit.client.0.smithi038.stdout:1/735: dread d2/f2d [0,4194304] 0 2024-02-15T18:34:59.840 INFO:tasks.workunit.client.0.smithi038.stdout:1/736: fdatasync d2/d10/d19/d2f/d3e/fc2 0 2024-02-15T18:34:59.843 INFO:tasks.workunit.client.0.smithi038.stdout:7/736: sync 2024-02-15T18:34:59.843 INFO:tasks.workunit.client.0.smithi038.stdout:4/772: sync 2024-02-15T18:34:59.843 INFO:tasks.workunit.client.0.smithi038.stdout:6/727: sync 2024-02-15T18:34:59.843 INFO:tasks.workunit.client.0.smithi038.stdout:5/736: sync 2024-02-15T18:34:59.843 INFO:tasks.workunit.client.0.smithi038.stdout:5/737: truncate d2/d3/da0/fae 114796 0 2024-02-15T18:34:59.845 INFO:tasks.workunit.client.0.smithi038.stdout:3/673: sync 2024-02-15T18:34:59.846 INFO:tasks.workunit.client.0.smithi038.stdout:0/844: mkdir d4/d13/d2a/d35/d1e/d32/d3b/d11b 0 2024-02-15T18:34:59.846 INFO:tasks.workunit.client.0.smithi038.stdout:0/845: write d4/d13/d2a/d35/d1e/d32/d3b/ff3 [824174,7655] 0 2024-02-15T18:34:59.846 INFO:tasks.workunit.client.0.smithi038.stdout:2/676: mknod dd/cd9 0 2024-02-15T18:34:59.848 INFO:tasks.workunit.client.0.smithi038.stdout:8/939: link d1/d4/d5/d6/d6d/df5/d123/l13c d1/d7/d15/d6a/l142 0 2024-02-15T18:34:59.848 INFO:tasks.workunit.client.0.smithi038.stdout:8/940: chown d1/d4/d5/d6/d6d/df5/d123 14782 1 2024-02-15T18:34:59.850 INFO:tasks.workunit.client.0.smithi038.stdout:9/870: unlink d1/da/d10/d13/d3d/d4b/daa/fb1 0 2024-02-15T18:34:59.851 INFO:tasks.workunit.client.0.smithi038.stdout:1/737: mkdir d2/d10/d32/d53/df0 0 2024-02-15T18:34:59.851 INFO:tasks.workunit.client.0.smithi038.stdout:7/737: creat d0/d95/db8/f106 x:0 0 0 2024-02-15T18:34:59.852 INFO:tasks.workunit.client.0.smithi038.stdout:6/728: truncate d2/d4/f1f 2213177 0 2024-02-15T18:34:59.852 INFO:tasks.workunit.client.0.smithi038.stdout:6/729: chown d2/d4/d19/d55/fd9 84 1 2024-02-15T18:34:59.852 INFO:tasks.workunit.client.0.smithi038.stdout:7/738: write d0/db/d59/fbb [2255201,76727] 0 2024-02-15T18:34:59.853 INFO:tasks.workunit.client.0.smithi038.stdout:7/739: chown d0/db/fe 651539 1 2024-02-15T18:34:59.853 INFO:tasks.workunit.client.0.smithi038.stdout:7/740: dread - d0/db/d59/dc8/dd6/fa3 zero size 2024-02-15T18:34:59.853 INFO:tasks.workunit.client.0.smithi038.stdout:5/738: rmdir d2/d48/d83 39 2024-02-15T18:34:59.853 INFO:tasks.workunit.client.0.smithi038.stdout:5/739: readlink d2/d3/d12/l6b 0 2024-02-15T18:34:59.853 INFO:tasks.workunit.client.0.smithi038.stdout:6/730: read d2/d10/f4c [596203,99796] 0 2024-02-15T18:34:59.854 INFO:tasks.workunit.client.0.smithi038.stdout:3/674: truncate d2/d1b/d3c/d41/d4e/fa3 1982625 0 2024-02-15T18:34:59.854 INFO:tasks.workunit.client.0.smithi038.stdout:2/677: creat dd/d12/d6b/da5/fda x:0 0 0 2024-02-15T18:34:59.854 INFO:tasks.workunit.client.0.smithi038.stdout:2/678: truncate dd/d12/d3a/d67/fc1 44221 0 2024-02-15T18:34:59.854 INFO:tasks.workunit.client.0.smithi038.stdout:2/679: truncate fb 4762356 0 2024-02-15T18:34:59.856 INFO:tasks.workunit.client.0.smithi038.stdout:8/941: mkdir d1/d3a/d58/d12f/d143 0 2024-02-15T18:34:59.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:34:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:34:58.318300+0000 mgr.smithi038.khesbl (mgr.14162) 339 : cluster [DBG] pgmap v281: 65 pgs: 65 active+clean; 1.2 GiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 75 MiB/s rd, 153 MiB/s wr, 659 op/s 2024-02-15T18:34:59.858 INFO:tasks.workunit.client.0.smithi038.stdout:8/942: dread d1/d7/fb0 [0,4194304] 0 2024-02-15T18:34:59.861 INFO:tasks.workunit.client.0.smithi038.stdout:9/871: symlink d1/da/d36/d8b/l136 0 2024-02-15T18:34:59.861 INFO:tasks.workunit.client.0.smithi038.stdout:0/846: dwrite d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/fd7 [0,4194304] 0 2024-02-15T18:34:59.862 INFO:tasks.workunit.client.0.smithi038.stdout:1/738: creat d2/d10/d19/d22/d54/dc0/d6b/ff1 x:0 0 0 2024-02-15T18:34:59.863 INFO:tasks.workunit.client.0.smithi038.stdout:4/773: dwrite d11/d1e/d20/d2b/d4d/d67/d73/fe3 [0,4194304] 0 2024-02-15T18:34:59.864 INFO:tasks.workunit.client.0.smithi038.stdout:4/774: fdatasync d11/d1e/d20/d2b/d4d/d67/d73/fdb 0 2024-02-15T18:34:59.865 INFO:tasks.workunit.client.0.smithi038.stdout:5/740: creat d2/d30/d57/fdb x:0 0 0 2024-02-15T18:34:59.866 INFO:tasks.workunit.client.0.smithi038.stdout:6/731: unlink d2/f1a 0 2024-02-15T18:34:59.866 INFO:tasks.workunit.client.0.smithi038.stdout:6/732: chown d2/d4/d19/d78/d7a 62 1 2024-02-15T18:34:59.866 INFO:tasks.workunit.client.0.smithi038.stdout:3/675: dread d2/d1b/d3c/d41/d4e/fa3 [0,4194304] 0 2024-02-15T18:34:59.866 INFO:tasks.workunit.client.0.smithi038.stdout:3/676: chown d2/d1b/d3c/d3a/fb0 0 1 2024-02-15T18:34:59.867 INFO:tasks.workunit.client.0.smithi038.stdout:2/680: truncate dd/d12/d1d/f31 3261860 0 2024-02-15T18:34:59.869 INFO:tasks.workunit.client.0.smithi038.stdout:9/872: creat d1/da/d36/d3f/f137 x:0 0 0 2024-02-15T18:34:59.871 INFO:tasks.workunit.client.0.smithi038.stdout:0/847: link d4/d13/d2a/d35/d1e/d32/d79/l99 d4/d13/d19/l11c 0 2024-02-15T18:34:59.875 INFO:tasks.workunit.client.0.smithi038.stdout:1/739: creat d2/d10/d19/d22/d36/d94/dc1/ff2 x:0 0 0 2024-02-15T18:34:59.877 INFO:tasks.workunit.client.0.smithi038.stdout:7/741: rename d0/c3c to d0/db/d59/dc8/dd6/d61/d90/c107 0 2024-02-15T18:34:59.877 INFO:tasks.workunit.client.0.smithi038.stdout:7/742: chown d0/d1b/d30 230 1 2024-02-15T18:34:59.879 INFO:tasks.workunit.client.0.smithi038.stdout:4/775: creat d11/d1e/d20/d2b/d4d/f116 x:0 0 0 2024-02-15T18:34:59.879 INFO:tasks.workunit.client.0.smithi038.stdout:4/776: dread - d11/d1e/d20/d31/d7a/fbc zero size 2024-02-15T18:34:59.882 INFO:tasks.workunit.client.0.smithi038.stdout:3/677: getdents d2/d25 0 2024-02-15T18:34:59.882 INFO:tasks.workunit.client.0.smithi038.stdout:3/678: fdatasync d2/f1a 0 2024-02-15T18:34:59.883 INFO:tasks.workunit.client.0.smithi038.stdout:2/681: symlink dd/d12/d59/d62/ldb 0 2024-02-15T18:34:59.884 INFO:tasks.workunit.client.0.smithi038.stdout:9/873: unlink d1/da/d36/d3f/f80 0 2024-02-15T18:34:59.885 INFO:tasks.workunit.client.0.smithi038.stdout:1/740: unlink d2/d10/d19/d22/f9b 0 2024-02-15T18:34:59.885 INFO:tasks.workunit.client.0.smithi038.stdout:1/741: write d2/d2e/d8c/fae [4391668,101636] 0 2024-02-15T18:34:59.887 INFO:tasks.workunit.client.0.smithi038.stdout:4/777: rmdir d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/ddd/dba 39 2024-02-15T18:34:59.887 INFO:tasks.workunit.client.0.smithi038.stdout:5/741: rename d2/c3c to d2/d48/d83/cdc 0 2024-02-15T18:34:59.888 INFO:tasks.workunit.client.0.smithi038.stdout:3/679: mkdir d2/d1b/d3c/d41/dd5 0 2024-02-15T18:34:59.889 INFO:tasks.workunit.client.0.smithi038.stdout:6/733: dwrite d2/d4/d19/d55/d5c/faa [0,4194304] 0 2024-02-15T18:34:59.890 INFO:tasks.workunit.client.0.smithi038.stdout:9/874: symlink d1/da/d20/d58/d94/db9/de3/l138 0 2024-02-15T18:34:59.891 INFO:tasks.workunit.client.0.smithi038.stdout:8/943: dwrite d1/d7/d26/fe1 [0,4194304] 0 2024-02-15T18:34:59.894 INFO:tasks.workunit.client.0.smithi038.stdout:7/743: rename d0/d95/db8 to d0/d1b/dfd/d108 0 2024-02-15T18:34:59.894 INFO:tasks.workunit.client.0.smithi038.stdout:7/744: chown d0/db/d59/dc8/dd6/d61/l69 43480334 1 2024-02-15T18:34:59.894 INFO:tasks.workunit.client.0.smithi038.stdout:0/848: dwrite d4/d13/d2a/d35/d1e/d32/d79/d7d/dda/fee [0,4194304] 0 2024-02-15T18:34:59.895 INFO:tasks.workunit.client.0.smithi038.stdout:3/680: dread d2/d1b/d3c/d3a/d55/da0/fa4 [0,4194304] 0 2024-02-15T18:34:59.895 INFO:tasks.workunit.client.0.smithi038.stdout:3/681: write d2/d1b/d3c/f6b [832451,12161] 0 2024-02-15T18:34:59.895 INFO:tasks.workunit.client.0.smithi038.stdout:3/682: dread - d2/d1b/d3c/d3a/d55/f81 zero size 2024-02-15T18:34:59.895 INFO:tasks.workunit.client.0.smithi038.stdout:5/742: mknod d2/d48/d83/d96/cdd 0 2024-02-15T18:34:59.896 INFO:tasks.workunit.client.0.smithi038.stdout:6/734: link d2/d10/fb8 d2/d4/d2c/d39/db4/fe1 0 2024-02-15T18:34:59.897 INFO:tasks.workunit.client.0.smithi038.stdout:9/875: creat d1/da/d20/d58/f139 x:0 0 0 2024-02-15T18:34:59.898 INFO:tasks.workunit.client.0.smithi038.stdout:8/944: mkdir d1/d7/d15/d144 0 2024-02-15T18:34:59.900 INFO:tasks.workunit.client.0.smithi038.stdout:7/745: mkdir d0/db/d59/d15/d109 0 2024-02-15T18:34:59.903 INFO:tasks.workunit.client.0.smithi038.stdout:0/849: rename d4/d13/d2a/d35/d1e/d32/d3b/lfa to d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/df4/l11d 0 2024-02-15T18:34:59.903 INFO:tasks.workunit.client.0.smithi038.stdout:0/850: read d4/d13/d2a/d35/d1e/fac [139309,17913] 0 2024-02-15T18:34:59.904 INFO:tasks.workunit.client.0.smithi038.stdout:5/743: link d2/fa5 d2/d3/d5f/fde 0 2024-02-15T18:34:59.904 INFO:tasks.workunit.client.0.smithi038.stdout:5/744: readlink d2/d3/l7c 0 2024-02-15T18:34:59.911 INFO:tasks.workunit.client.0.smithi038.stdout:9/876: truncate d1/dc/fba 980973 0 2024-02-15T18:34:59.911 INFO:tasks.workunit.client.0.smithi038.stdout:8/945: creat d1/d4/db1/de5/db3/f145 x:0 0 0 2024-02-15T18:34:59.911 INFO:tasks.workunit.client.0.smithi038.stdout:7/746: mkdir d0/db/d59/d15/d29/d8d/d102/d10a 0 2024-02-15T18:34:59.911 INFO:tasks.workunit.client.0.smithi038.stdout:3/683: rename d2/d1b/d3c/d41/d4e/l88 to d2/d25/d60/ld6 0 2024-02-15T18:34:59.911 INFO:tasks.workunit.client.0.smithi038.stdout:0/851: symlink d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/l11e 0 2024-02-15T18:34:59.911 INFO:tasks.workunit.client.0.smithi038.stdout:0/852: write d4/d13/d2a/d35/d1e/d32/d3b/ff3 [868937,31052] 0 2024-02-15T18:34:59.912 INFO:tasks.workunit.client.0.smithi038.stdout:8/946: creat d1/d7/d22/da7/f146 x:0 0 0 2024-02-15T18:34:59.912 INFO:tasks.workunit.client.0.smithi038.stdout:8/947: chown d1/d7/d26/d69/fe3 3 1 2024-02-15T18:34:59.913 INFO:tasks.workunit.client.0.smithi038.stdout:3/684: rename d2/f17 to d2/db8/dcf/fd7 0 2024-02-15T18:34:59.914 INFO:tasks.workunit.client.0.smithi038.stdout:0/853: write d4/d13/d19/f82 [1556642,10963] 0 2024-02-15T18:34:59.917 INFO:tasks.workunit.client.0.smithi038.stdout:0/854: rename d4/d13/d19/l46 to d4/d13/d2a/d35/d1e/d32/d48/l11f 0 2024-02-15T18:34:59.917 INFO:tasks.workunit.client.0.smithi038.stdout:0/855: dread - d4/d13/d53/d100/f114 zero size 2024-02-15T18:34:59.919 INFO:tasks.workunit.client.0.smithi038.stdout:0/856: mkdir d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/ddb/d120 0 2024-02-15T18:34:59.921 INFO:tasks.workunit.client.0.smithi038.stdout:0/857: link d4/db3/fe2 d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/df4/f121 0 2024-02-15T18:34:59.923 INFO:tasks.workunit.client.0.smithi038.stdout:7/747: dread d0/db/d59/d15/fe3 [0,4194304] 0 2024-02-15T18:34:59.926 INFO:tasks.workunit.client.0.smithi038.stdout:0/858: creat d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/dca/f122 x:0 0 0 2024-02-15T18:34:59.927 INFO:tasks.workunit.client.0.smithi038.stdout:7/748: mkdir d0/db/d59/d15/d29/d8d/dcd/d10b 0 2024-02-15T18:34:59.928 INFO:tasks.workunit.client.0.smithi038.stdout:0/859: mkdir d4/d13/d19/dfc/d123 0 2024-02-15T18:34:59.928 INFO:tasks.workunit.client.0.smithi038.stdout:7/749: rmdir d0/db/d59/d15/d29/d8d/d102 39 2024-02-15T18:34:59.931 INFO:tasks.workunit.client.0.smithi038.stdout:0/860: rmdir d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/d109 0 2024-02-15T18:34:59.931 INFO:tasks.workunit.client.0.smithi038.stdout:9/877: dread d1/da/d10/d13/f18 [0,4194304] 0 2024-02-15T18:34:59.931 INFO:tasks.workunit.client.0.smithi038.stdout:0/861: fsync d4/ff 0 2024-02-15T18:34:59.931 INFO:tasks.workunit.client.0.smithi038.stdout:9/878: fdatasync d1/da/d10/d13/d3d/d43/fdd 0 2024-02-15T18:34:59.932 INFO:tasks.workunit.client.0.smithi038.stdout:9/879: dread d1/da/d10/d82/fa9 [0,4194304] 0 2024-02-15T18:34:59.932 INFO:tasks.workunit.client.0.smithi038.stdout:9/880: fsync d1/da/d10/d13/d3d/d6c/fd4 0 2024-02-15T18:34:59.933 INFO:tasks.workunit.client.0.smithi038.stdout:7/750: unlink d0/d1b/d80/l6c 0 2024-02-15T18:34:59.935 INFO:tasks.workunit.client.0.smithi038.stdout:7/751: creat d0/db/d59/d15/d29/d8d/f10c x:0 0 0 2024-02-15T18:34:59.935 INFO:tasks.workunit.client.0.smithi038.stdout:7/752: dread - d0/db/d59/dc8/dd6/fd4 zero size 2024-02-15T18:34:59.941 INFO:tasks.workunit.client.0.smithi038.stdout:1/742: dwrite d2/d2e/d8c/fa0 [8388608,4194304] 0 2024-02-15T18:34:59.942 INFO:tasks.workunit.client.0.smithi038.stdout:1/743: fdatasync d2/d10/d32/d53/f7b 0 2024-02-15T18:34:59.942 INFO:tasks.workunit.client.0.smithi038.stdout:2/682: dwrite dd/d12/d3a/d67/fa0 [0,4194304] 0 2024-02-15T18:34:59.943 INFO:tasks.workunit.client.0.smithi038.stdout:4/778: dwrite d11/d1e/d43/f63 [0,4194304] 0 2024-02-15T18:34:59.943 INFO:tasks.workunit.client.0.smithi038.stdout:5/745: dwrite d2/d30/f73 [0,4194304] 0 2024-02-15T18:34:59.946 INFO:tasks.workunit.client.0.smithi038.stdout:1/744: mkdir d2/d69/da9/dad/df3 0 2024-02-15T18:34:59.949 INFO:tasks.workunit.client.0.smithi038.stdout:6/735: dwrite d2/d4/d2c/d39/d76/dc2/fce [0,4194304] 0 2024-02-15T18:34:59.952 INFO:tasks.workunit.client.0.smithi038.stdout:2/683: creat dd/d12/d6b/da5/dd8/fdc x:0 0 0 2024-02-15T18:34:59.952 INFO:tasks.workunit.client.0.smithi038.stdout:2/684: chown dd/d12/d5d 221133 1 2024-02-15T18:34:59.952 INFO:tasks.workunit.client.0.smithi038.stdout:2/685: chown dd/f77 0 1 2024-02-15T18:34:59.952 INFO:tasks.workunit.client.0.smithi038.stdout:4/779: write d11/d1e/f37 [468645,78875] 0 2024-02-15T18:34:59.953 INFO:tasks.workunit.client.0.smithi038.stdout:3/685: dwrite d2/d1b/d8c/f99 [0,4194304] 0 2024-02-15T18:34:59.953 INFO:tasks.workunit.client.0.smithi038.stdout:5/746: mkdir d2/d3/db9/dcf/ddf 0 2024-02-15T18:34:59.953 INFO:tasks.workunit.client.0.smithi038.stdout:5/747: chown d2/c29 508681 1 2024-02-15T18:34:59.953 INFO:tasks.workunit.client.0.smithi038.stdout:1/745: creat d2/d10/d19/d22/d54/dc0/d6b/ff4 x:0 0 0 2024-02-15T18:34:59.954 INFO:tasks.workunit.client.0.smithi038.stdout:1/746: readlink d2/d10/d19/d22/d36/lc8 0 2024-02-15T18:34:59.955 INFO:tasks.workunit.client.0.smithi038.stdout:6/736: unlink d2/d4/d2c/d39/db4/f5e 0 2024-02-15T18:34:59.955 INFO:tasks.workunit.client.0.smithi038.stdout:6/737: chown d2/d4/d2c/d39/d76/fd2 4086 1 2024-02-15T18:34:59.955 INFO:tasks.workunit.client.0.smithi038.stdout:2/686: getdents dd/d12/d5d/dc2 0 2024-02-15T18:34:59.956 INFO:tasks.workunit.client.0.smithi038.stdout:4/780: creat d11/d1e/d20/d2b/d4d/d67/da1/f117 x:0 0 0 2024-02-15T18:34:59.958 INFO:tasks.workunit.client.0.smithi038.stdout:3/686: creat d2/d1b/d69/fd8 x:0 0 0 2024-02-15T18:34:59.958 INFO:tasks.workunit.client.0.smithi038.stdout:0/862: dwrite d4/f11 [0,4194304] 0 2024-02-15T18:34:59.959 INFO:tasks.workunit.client.0.smithi038.stdout:8/948: dwrite d1/d4/d5/f4d [0,4194304] 0 2024-02-15T18:34:59.960 INFO:tasks.workunit.client.0.smithi038.stdout:8/949: chown d1/d4/d10/d3d/c92 0 1 2024-02-15T18:34:59.960 INFO:tasks.workunit.client.0.smithi038.stdout:9/881: dwrite d1/da/d20/d58/d94/db9/dc6/fd3 [0,4194304] 0 2024-02-15T18:34:59.961 INFO:tasks.workunit.client.0.smithi038.stdout:5/748: creat d2/d3/db9/dcf/ddf/fe0 x:0 0 0 2024-02-15T18:34:59.961 INFO:tasks.workunit.client.0.smithi038.stdout:1/747: unlink d2/d39/f89 0 2024-02-15T18:34:59.961 INFO:tasks.workunit.client.0.smithi038.stdout:5/749: chown d2/dcc 25605 1 2024-02-15T18:34:59.962 INFO:tasks.workunit.client.0.smithi038.stdout:6/738: mknod d2/d4/d6f/ce2 0 2024-02-15T18:34:59.963 INFO:tasks.workunit.client.0.smithi038.stdout:5/750: write d2/d48/f59 [8242531,111233] 0 2024-02-15T18:34:59.963 INFO:tasks.workunit.client.0.smithi038.stdout:5/751: fsync d2/d64/f8a 0 2024-02-15T18:34:59.964 INFO:tasks.workunit.client.0.smithi038.stdout:0/863: dread d4/d13/f64 [0,4194304] 0 2024-02-15T18:34:59.964 INFO:tasks.workunit.client.0.smithi038.stdout:0/864: chown d4/d13/d2a/d35/d1e/d32/d79/d7d/f96 248 1 2024-02-15T18:34:59.966 INFO:tasks.workunit.client.0.smithi038.stdout:7/753: dwrite d0/d1b/d30/d50/f74 [0,4194304] 0 2024-02-15T18:34:59.966 INFO:tasks.workunit.client.0.smithi038.stdout:7/754: dread - d0/db/d59/d15/d29/d7e/f73 zero size 2024-02-15T18:34:59.966 INFO:tasks.workunit.client.0.smithi038.stdout:2/687: rename dd/d12/d8d/d90/faf to dd/d12/d1d/d3b/d3c/d4a/fdd 0 2024-02-15T18:34:59.976 INFO:tasks.workunit.client.0.smithi038.stdout:0/865: dread d4/f12 [0,4194304] 0 2024-02-15T18:34:59.976 INFO:tasks.workunit.client.0.smithi038.stdout:0/866: dread - d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/dca/f122 zero size 2024-02-15T18:34:59.976 INFO:tasks.workunit.client.0.smithi038.stdout:8/950: creat d1/d4/d5/d6/d6d/f147 x:0 0 0 2024-02-15T18:34:59.979 INFO:tasks.workunit.client.0.smithi038.stdout:9/882: mkdir d1/dc/db2/d13a 0 2024-02-15T18:34:59.980 INFO:tasks.workunit.client.0.smithi038.stdout:1/748: creat d2/d10/d19/ff5 x:0 0 0 2024-02-15T18:34:59.982 INFO:tasks.workunit.client.0.smithi038.stdout:3/687: dwrite d2/f1a [0,4194304] 0 2024-02-15T18:34:59.982 INFO:tasks.workunit.client.0.smithi038.stdout:6/739: mknod d2/d4/d19/ce3 0 2024-02-15T18:34:59.982 INFO:tasks.workunit.client.0.smithi038.stdout:3/688: write d2/f15 [1797526,37146] 0 2024-02-15T18:34:59.984 INFO:tasks.workunit.client.0.smithi038.stdout:5/752: link d2/d3/d12/fa8 d2/d30/d57/fe1 0 2024-02-15T18:34:59.984 INFO:tasks.workunit.client.0.smithi038.stdout:4/781: dwrite d11/d1e/d20/d2f/d71/f98 [4194304,4194304] 0 2024-02-15T18:34:59.984 INFO:tasks.workunit.client.0.smithi038.stdout:5/753: fsync d2/d3/da0/f50 0 2024-02-15T18:34:59.987 INFO:tasks.workunit.client.0.smithi038.stdout:7/755: rename d0/db/d59/d15/fef to d0/db/d59/df9/f10d 0 2024-02-15T18:34:59.988 INFO:tasks.workunit.client.0.smithi038.stdout:2/688: mkdir dd/d12/d6b/dde 0 2024-02-15T18:34:59.990 INFO:tasks.workunit.client.0.smithi038.stdout:1/749: symlink d2/d10/d19/lf6 0 2024-02-15T18:34:59.990 INFO:tasks.workunit.client.0.smithi038.stdout:9/883: creat d1/da/d10/f13b x:0 0 0 2024-02-15T18:34:59.990 INFO:tasks.workunit.client.0.smithi038.stdout:1/750: readlink d2/d69/le5 0 2024-02-15T18:34:59.990 INFO:tasks.workunit.client.0.smithi038.stdout:7/756: dread d0/db/d59/dc8/f103 [0,4194304] 0 2024-02-15T18:34:59.990 INFO:tasks.workunit.client.0.smithi038.stdout:9/884: write d1/dc/d5a/fde [4894292,110820] 0 2024-02-15T18:34:59.993 INFO:tasks.workunit.client.0.smithi038.stdout:6/740: link d2/d4/d19/c57 d2/d4/d19/d78/d7a/ce4 0 2024-02-15T18:34:59.993 INFO:tasks.workunit.client.0.smithi038.stdout:3/689: truncate d2/d1b/d3c/d41/d4e/fa3 249645 0 2024-02-15T18:34:59.996 INFO:tasks.workunit.client.0.smithi038.stdout:5/754: creat d2/d48/d83/fe2 x:0 0 0 2024-02-15T18:34:59.996 INFO:tasks.workunit.client.0.smithi038.stdout:5/755: write d2/d3/da0/d33/fc8 [87023,67885] 0 2024-02-15T18:34:59.996 INFO:tasks.workunit.client.0.smithi038.stdout:0/867: dwrite d4/d13/d2a/d35/db2/fdc [0,4194304] 0 2024-02-15T18:34:59.997 INFO:tasks.workunit.client.0.smithi038.stdout:8/951: rename d1/d7/d15/d4b/f57 to d1/d7/d22/f148 0 2024-02-15T18:34:59.997 INFO:tasks.workunit.client.0.smithi038.stdout:8/952: fdatasync d1/d4/db1/de5/db3/f12e 0 2024-02-15T18:34:59.997 INFO:tasks.workunit.client.0.smithi038.stdout:8/953: write d1/d7/d22/da7/f100 [279382,54427] 0 2024-02-15T18:34:59.999 INFO:tasks.workunit.client.0.smithi038.stdout:2/689: symlink dd/d12/d3a/d67/d4f/ldf 0 2024-02-15T18:35:00.001 INFO:tasks.workunit.client.0.smithi038.stdout:1/751: link d2/d10/d19/d22/c29 d2/cf7 0 2024-02-15T18:35:00.001 INFO:tasks.workunit.client.0.smithi038.stdout:1/752: chown d2/d10/d19 241 1 2024-02-15T18:35:00.002 INFO:tasks.workunit.client.0.smithi038.stdout:6/741: creat d2/d4/d19/d78/dbf/fe5 x:0 0 0 2024-02-15T18:35:00.003 INFO:tasks.workunit.client.0.smithi038.stdout:5/756: mkdir d2/d3/db9/de3 0 2024-02-15T18:35:00.008 INFO:tasks.workunit.client.0.smithi038.stdout:0/868: stat d4/d13/d2a/d35/d1e/d32/d79/l99 0 2024-02-15T18:35:00.012 INFO:tasks.workunit.client.0.smithi038.stdout:3/690: dwrite d2/d1b/d69/f96 [4194304,4194304] 0 2024-02-15T18:35:00.013 INFO:tasks.workunit.client.0.smithi038.stdout:4/782: rename d11/d1e/d3e/d57 to d11/d1e/d20/d31/da2/d118 0 2024-02-15T18:35:00.014 INFO:tasks.workunit.client.0.smithi038.stdout:8/954: mknod d1/d7/d26/d69/c149 0 2024-02-15T18:35:00.015 INFO:tasks.workunit.client.0.smithi038.stdout:2/690: mknod dd/d12/d59/d2b/d6a/ce0 0 2024-02-15T18:35:00.015 INFO:tasks.workunit.client.0.smithi038.stdout:2/691: write dd/d12/d5d/fb1 [804259,57271] 0 2024-02-15T18:35:00.016 INFO:tasks.workunit.client.0.smithi038.stdout:1/753: dwrite d2/d10/d62/fd2 [0,4194304] 0 2024-02-15T18:35:00.016 INFO:tasks.workunit.client.0.smithi038.stdout:1/754: chown d2/d10/d19/d5f/d92/fb8 299815387 1 2024-02-15T18:35:00.016 INFO:tasks.workunit.client.0.smithi038.stdout:6/742: creat d2/d4/d6c/d9d/dd7/fe6 x:0 0 0 2024-02-15T18:35:00.020 INFO:tasks.workunit.client.0.smithi038.stdout:0/869: creat d4/d13/d53/f124 x:0 0 0 2024-02-15T18:35:00.024 INFO:tasks.workunit.client.0.smithi038.stdout:3/691: getdents d2 0 2024-02-15T18:35:00.028 INFO:tasks.workunit.client.0.smithi038.stdout:7/757: rename d0/db/d59/df9 to d0/db/d59/dc8/dd6/d61/d10e 0 2024-02-15T18:35:00.029 INFO:tasks.workunit.client.0.smithi038.stdout:2/692: mkdir dd/d12/d5d/d54/dcb/de1 0 2024-02-15T18:35:00.031 INFO:tasks.workunit.client.0.smithi038.stdout:1/755: mknod d2/d10/d32/d53/df0/cf8 0 2024-02-15T18:35:00.033 INFO:tasks.workunit.client.0.smithi038.stdout:6/743: unlink d2/d10/f79 0 2024-02-15T18:35:00.040 INFO:tasks.workunit.client.0.smithi038.stdout:0/870: creat d4/db3/f125 x:0 0 0 2024-02-15T18:35:00.040 INFO:tasks.workunit.client.0.smithi038.stdout:9/885: rename d1/da/d36/d93/d9a to d1/da/d20/d58/d94/d13c 0 2024-02-15T18:35:00.040 INFO:tasks.workunit.client.0.smithi038.stdout:2/693: mknod dd/d12/d59/d62/ce2 0 2024-02-15T18:35:00.040 INFO:tasks.workunit.client.0.smithi038.stdout:2/694: stat dd/d12/d3a 0 2024-02-15T18:35:00.040 INFO:tasks.workunit.client.0.smithi038.stdout:2/695: dread - dd/d12/d1d/d3b/fb8 zero size 2024-02-15T18:35:00.041 INFO:tasks.workunit.client.0.smithi038.stdout:5/757: truncate d2/d3/da0/d33/fc8 120867 0 2024-02-15T18:35:00.042 INFO:tasks.workunit.client.0.smithi038.stdout:5/758: readlink d2/d3/da0/d33/lb7 0 2024-02-15T18:35:00.042 INFO:tasks.workunit.client.0.smithi038.stdout:5/759: chown d2/d64/d99/db6/cc7 147 1 2024-02-15T18:35:00.042 INFO:tasks.workunit.client.0.smithi038.stdout:5/760: truncate d2/d3/f6f 1597999 0 2024-02-15T18:35:00.042 INFO:tasks.workunit.client.0.smithi038.stdout:3/692: dread d2/d1b/d69/f96 [4194304,4194304] 0 2024-02-15T18:35:00.042 INFO:tasks.workunit.client.0.smithi038.stdout:8/955: dwrite d1/d4/d10/d91/f118 [0,4194304] 0 2024-02-15T18:35:00.042 INFO:tasks.workunit.client.0.smithi038.stdout:9/886: read d1/dc/d69/d7d/fc3 [141193,99269] 0 2024-02-15T18:35:00.043 INFO:tasks.workunit.client.0.smithi038.stdout:3/693: truncate d2/d1b/f8e 747815 0 2024-02-15T18:35:00.043 INFO:tasks.workunit.client.0.smithi038.stdout:1/756: mknod d2/d69/da9/dbc/ddd/cf9 0 2024-02-15T18:35:00.045 INFO:tasks.workunit.client.0.smithi038.stdout:4/783: rename d11/d1e/f23 to d11/d1e/d20/d2b/d4d/f119 0 2024-02-15T18:35:00.045 INFO:tasks.workunit.client.0.smithi038.stdout:4/784: chown d11/d1e/d20/d31/d48/d60 471402 1 2024-02-15T18:35:00.045 INFO:tasks.workunit.client.0.smithi038.stdout:4/785: write d11/d1e/d20/d2b/d4d/f119 [2618086,54700] 0 2024-02-15T18:35:00.045 INFO:tasks.workunit.client.0.smithi038.stdout:7/758: dread d0/db/d59/d15/d29/fc6 [0,4194304] 0 2024-02-15T18:35:00.045 INFO:tasks.workunit.client.0.smithi038.stdout:7/759: truncate d0/db/fe 4623605 0 2024-02-15T18:35:00.045 INFO:tasks.workunit.client.0.smithi038.stdout:7/760: fsync d0/db/d19/d2d/ffc 0 2024-02-15T18:35:00.046 INFO:tasks.workunit.client.0.smithi038.stdout:4/786: write d11/d1e/d43/f63 [3891798,35338] 0 2024-02-15T18:35:00.048 INFO:tasks.workunit.client.0.smithi038.stdout:2/696: creat dd/d12/d59/db2/fe3 x:0 0 0 2024-02-15T18:35:00.048 INFO:tasks.workunit.client.0.smithi038.stdout:2/697: fsync dd/d12/d1d/d3b/d3c/d4a/fb7 0 2024-02-15T18:35:00.049 INFO:tasks.workunit.client.0.smithi038.stdout:6/744: dwrite d2/d10/f49 [4194304,4194304] 0 2024-02-15T18:35:00.049 INFO:tasks.workunit.client.0.smithi038.stdout:6/745: chown d2/f3 1753 1 2024-02-15T18:35:00.050 INFO:tasks.workunit.client.0.smithi038.stdout:3/694: creat d2/d1e/fd9 x:0 0 0 2024-02-15T18:35:00.051 INFO:tasks.workunit.client.0.smithi038.stdout:8/956: creat d1/d4/d10/d43/d48/d52/d12c/f14a x:0 0 0 2024-02-15T18:35:00.052 INFO:tasks.workunit.client.0.smithi038.stdout:9/887: symlink d1/da/d10/l13d 0 2024-02-15T18:35:00.052 INFO:tasks.workunit.client.0.smithi038.stdout:9/888: write d1/da/d20/d58/d94/db9/de3/f118 [21993,58299] 0 2024-02-15T18:35:00.052 INFO:tasks.workunit.client.0.smithi038.stdout:9/889: read - d1/f124 zero size 2024-02-15T18:35:00.052 INFO:tasks.workunit.client.0.smithi038.stdout:9/890: truncate d1/dc/d69/d7d/ff8 871554 0 2024-02-15T18:35:00.053 INFO:tasks.workunit.client.0.smithi038.stdout:1/757: truncate d2/d2e/d8c/fae 2399377 0 2024-02-15T18:35:00.053 INFO:tasks.workunit.client.0.smithi038.stdout:1/758: truncate d2/d10/d19/d5f/d92/fb8 147986 0 2024-02-15T18:35:00.055 INFO:tasks.workunit.client.0.smithi038.stdout:5/761: rename d2/d48/d83/d96/d8e to d2/d3/db9/de3/de4 0 2024-02-15T18:35:00.056 INFO:tasks.workunit.client.0.smithi038.stdout:7/761: symlink d0/d1b/dfd/l10f 0 2024-02-15T18:35:00.058 INFO:tasks.workunit.client.0.smithi038.stdout:4/787: creat d11/d1e/d20/d31/d7a/dd8/f11a x:0 0 0 2024-02-15T18:35:00.058 INFO:tasks.workunit.client.0.smithi038.stdout:4/788: fdatasync d11/d1e/d20/d2f/f3d 0 2024-02-15T18:35:00.058 INFO:tasks.workunit.client.0.smithi038.stdout:4/789: readlink d11/d1e/d20/d31/da2/d118/l88 0 2024-02-15T18:35:00.059 INFO:tasks.workunit.client.0.smithi038.stdout:2/698: unlink dd/d12/d8d/fca 0 2024-02-15T18:35:00.059 INFO:tasks.workunit.client.0.smithi038.stdout:6/746: truncate d2/d4/d6c/f98 672702 0 2024-02-15T18:35:00.060 INFO:tasks.workunit.client.0.smithi038.stdout:2/699: write dd/d12/d3a/d67/f8f [182301,129000] 0 2024-02-15T18:35:00.060 INFO:tasks.workunit.client.0.smithi038.stdout:3/695: creat d2/db8/dcf/fda x:0 0 0 2024-02-15T18:35:00.060 INFO:tasks.workunit.client.0.smithi038.stdout:3/696: write d2/d1b/d3c/d42/fbd [979154,90955] 0 2024-02-15T18:35:00.061 INFO:tasks.workunit.client.0.smithi038.stdout:3/697: chown d2/d1b/cca 68 1 2024-02-15T18:35:00.061 INFO:tasks.workunit.client.0.smithi038.stdout:3/698: stat d2/d1b/d3c/d3a/d55/d65/c84 0 2024-02-15T18:35:00.061 INFO:tasks.workunit.client.0.smithi038.stdout:8/957: truncate d1/d7/d26/ff4 528034 0 2024-02-15T18:35:00.061 INFO:tasks.workunit.client.0.smithi038.stdout:8/958: fsync d1/d4/d10/d43/d48/fcd 0 2024-02-15T18:35:00.063 INFO:tasks.workunit.client.0.smithi038.stdout:9/891: unlink d1/dc/db2/fdc 0 2024-02-15T18:35:00.063 INFO:tasks.workunit.client.0.smithi038.stdout:9/892: chown d1/da/d10/d13/f132 365183 1 2024-02-15T18:35:00.063 INFO:tasks.workunit.client.0.smithi038.stdout:9/893: write d1/f124 [1043670,104310] 0 2024-02-15T18:35:00.064 INFO:tasks.workunit.client.0.smithi038.stdout:1/759: creat d2/d10/d32/d53/d82/ffa x:0 0 0 2024-02-15T18:35:00.068 INFO:tasks.workunit.client.0.smithi038.stdout:0/871: dwrite d4/d13/d2a/d35/d1e/d32/d3b/f60 [0,4194304] 0 2024-02-15T18:35:00.068 INFO:tasks.workunit.client.0.smithi038.stdout:5/762: truncate d2/dcc/f8b 4272781 0 2024-02-15T18:35:00.068 INFO:tasks.workunit.client.0.smithi038.stdout:0/872: fdatasync d4/d13/d2a/d35/d1e/d32/d48/d97/fc9 0 2024-02-15T18:35:00.076 INFO:tasks.workunit.client.0.smithi038.stdout:4/790: mknod d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/ddd/dba/c11b 0 2024-02-15T18:35:00.076 INFO:tasks.workunit.client.0.smithi038.stdout:2/700: getdents dd/d12/d59/d62 0 2024-02-15T18:35:00.076 INFO:tasks.workunit.client.0.smithi038.stdout:2/701: chown dd/d12/d59/d2b/d6a/l5b 64580188 1 2024-02-15T18:35:00.079 INFO:tasks.workunit.client.0.smithi038.stdout:4/791: write d11/f1b [106903,6604] 0 2024-02-15T18:35:00.079 INFO:tasks.workunit.client.0.smithi038.stdout:3/699: mkdir d2/d1b/d69/ddb 0 2024-02-15T18:35:00.080 INFO:tasks.workunit.client.0.smithi038.stdout:5/763: write d2/dcc/f6a [2822687,67637] 0 2024-02-15T18:35:00.081 INFO:tasks.workunit.client.0.smithi038.stdout:8/959: rename d1/d4/db1/de5/db3/f145 to d1/d4/d10/d43/d48/d9a/dd9/f14b 0 2024-02-15T18:35:00.081 INFO:tasks.workunit.client.0.smithi038.stdout:8/960: chown d1/d7/l1c 14599712 1 2024-02-15T18:35:00.083 INFO:tasks.workunit.client.0.smithi038.stdout:1/760: mknod d2/d2e/d8c/cfb 0 2024-02-15T18:35:00.084 INFO:tasks.workunit.client.0.smithi038.stdout:7/762: symlink d0/l110 0 2024-02-15T18:35:00.084 INFO:tasks.workunit.client.0.smithi038.stdout:4/792: getdents d11/d1e/d43 0 2024-02-15T18:35:00.087 INFO:tasks.workunit.client.0.smithi038.stdout:3/700: symlink d2/d7f/dab/ldc 0 2024-02-15T18:35:00.087 INFO:tasks.workunit.client.0.smithi038.stdout:3/701: write d2/d24/fb2 [1006282,47285] 0 2024-02-15T18:35:00.087 INFO:tasks.workunit.client.0.smithi038.stdout:3/702: fsync d2/d1b/f9e 0 2024-02-15T18:35:00.087 INFO:tasks.workunit.client.0.smithi038.stdout:5/764: symlink d2/d3/dd0/le5 0 2024-02-15T18:35:00.088 INFO:tasks.workunit.client.0.smithi038.stdout:8/961: mknod d1/d4/d5/d6/d6d/df5/c14c 0 2024-02-15T18:35:00.089 INFO:tasks.workunit.client.0.smithi038.stdout:6/747: dwrite d2/d4/d2c/d39/db4/f7c [0,4194304] 0 2024-02-15T18:35:00.090 INFO:tasks.workunit.client.0.smithi038.stdout:6/748: write d2/d4/d2c/d39/db4/d5f/fa1 [319709,1076] 0 2024-02-15T18:35:00.090 INFO:tasks.workunit.client.0.smithi038.stdout:6/749: chown d2/d4/d2c/l9a 1791526102 1 2024-02-15T18:35:00.091 INFO:tasks.workunit.client.0.smithi038.stdout:9/894: dwrite d1/da/d20/d58/d94/db9/dc6/d49/f21 [0,4194304] 0 2024-02-15T18:35:00.092 INFO:tasks.workunit.client.0.smithi038.stdout:0/873: dwrite d4/d13/d2a/d35/db2/fbd [0,4194304] 0 2024-02-15T18:35:00.096 INFO:tasks.workunit.client.0.smithi038.stdout:1/761: rmdir d2/d10/d19 39 2024-02-15T18:35:00.096 INFO:tasks.workunit.client.0.smithi038.stdout:4/793: mkdir d11/d1e/d20/d2b/d4d/d67/da1/d11c 0 2024-02-15T18:35:00.096 INFO:tasks.workunit.client.0.smithi038.stdout:1/762: fdatasync d2/d10/d19/d22/d54/fd8 0 2024-02-15T18:35:00.096 INFO:tasks.workunit.client.0.smithi038.stdout:7/763: getdents d0/db/d59/dc8/dd6/d61/d10e 0 2024-02-15T18:35:00.096 INFO:tasks.workunit.client.0.smithi038.stdout:7/764: write d0/db/d59/dc8/dd6/d61/d90/dc0/fce [698023,56463] 0 2024-02-15T18:35:00.096 INFO:tasks.workunit.client.0.smithi038.stdout:3/703: link d2/db8/dcf/fda d2/d7f/dc1/fdd 0 2024-02-15T18:35:00.098 INFO:tasks.workunit.client.0.smithi038.stdout:8/962: creat d1/d3a/df6/d12b/f14d x:0 0 0 2024-02-15T18:35:00.100 INFO:tasks.workunit.client.0.smithi038.stdout:9/895: creat d1/da/d20/d58/d94/d13c/d11c/db8/dcb/f13e x:0 0 0 2024-02-15T18:35:00.102 INFO:tasks.workunit.client.0.smithi038.stdout:9/896: write d1/da/d20/d58/d94/d13c/d11c/fb3 [816769,55925] 0 2024-02-15T18:35:00.107 INFO:tasks.workunit.client.0.smithi038.stdout:0/874: unlink d4/d13/d2a/d35/d1e/fe5 0 2024-02-15T18:35:00.110 INFO:tasks.workunit.client.0.smithi038.stdout:2/702: dwrite dd/d12/d5d/d54/f97 [0,4194304] 0 2024-02-15T18:35:00.113 INFO:tasks.workunit.client.0.smithi038.stdout:7/765: link d0/db/d59/dc8/dd6/fd4 d0/db/d19/d7d/f111 0 2024-02-15T18:35:00.114 INFO:tasks.workunit.client.0.smithi038.stdout:4/794: rmdir d11/d1e/d20/d31/d48/dac/dbe/d40/dde 39 2024-02-15T18:35:00.116 INFO:tasks.workunit.client.0.smithi038.stdout:3/704: creat d2/d1b/d4f/d92/fde x:0 0 0 2024-02-15T18:35:00.118 INFO:tasks.workunit.client.0.smithi038.stdout:0/875: creat d4/d13/d2a/d35/d1e/d32/d3b/daa/df8/f126 x:0 0 0 2024-02-15T18:35:00.121 INFO:tasks.workunit.client.0.smithi038.stdout:5/765: dwrite d2/d3/db9/de3/de4/fbe [0,4194304] 0 2024-02-15T18:35:00.121 INFO:tasks.workunit.client.0.smithi038.stdout:2/703: rmdir dd/d12/d6b/da5 39 2024-02-15T18:35:00.121 INFO:tasks.workunit.client.0.smithi038.stdout:2/704: write dd/d12/d5d/d54/f97 [4381750,94020] 0 2024-02-15T18:35:00.121 INFO:tasks.workunit.client.0.smithi038.stdout:2/705: truncate dd/d12/d3a/d67/d4f/d5a/d9d/fac 681384 0 2024-02-15T18:35:00.125 INFO:tasks.workunit.client.0.smithi038.stdout:4/795: mkdir d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/d11d 0 2024-02-15T18:35:00.128 INFO:tasks.workunit.client.0.smithi038.stdout:9/897: dwrite d1/f1c [0,4194304] 0 2024-02-15T18:35:00.130 INFO:tasks.workunit.client.0.smithi038.stdout:6/750: dwrite d2/d4/d2c/d39/db4/f8d [4194304,4194304] 0 2024-02-15T18:35:00.130 INFO:tasks.workunit.client.0.smithi038.stdout:6/751: dread - d2/d4/d6c/fcb zero size 2024-02-15T18:35:00.131 INFO:tasks.workunit.client.0.smithi038.stdout:6/752: write d2/d4/d19/d55/fde [654325,102110] 0 2024-02-15T18:35:00.131 INFO:tasks.workunit.client.0.smithi038.stdout:5/766: mkdir d2/d3/d12/de6 0 2024-02-15T18:35:00.131 INFO:tasks.workunit.client.0.smithi038.stdout:2/706: mkdir dd/d12/d5d/de4 0 2024-02-15T18:35:00.132 INFO:tasks.workunit.client.0.smithi038.stdout:8/963: dwrite d1/d4/d5/d31/f33 [0,4194304] 0 2024-02-15T18:35:00.132 INFO:tasks.workunit.client.0.smithi038.stdout:8/964: getdents d1/d4/d5/d77 0 2024-02-15T18:35:00.134 INFO:tasks.workunit.client.0.smithi038.stdout:2/707: dread dd/f46 [0,4194304] 0 2024-02-15T18:35:00.134 INFO:tasks.workunit.client.0.smithi038.stdout:2/708: readlink dd/lf 0 2024-02-15T18:35:00.141 INFO:tasks.workunit.client.0.smithi038.stdout:6/753: truncate d2/d4/d2c/f4f 2645193 0 2024-02-15T18:35:00.142 INFO:tasks.workunit.client.0.smithi038.stdout:5/767: creat d2/d3/d12/fe7 x:0 0 0 2024-02-15T18:35:00.144 INFO:tasks.workunit.client.0.smithi038.stdout:4/796: write d11/d1e/d20/d2b/d35/f89 [2065493,90575] 0 2024-02-15T18:35:00.144 INFO:tasks.workunit.client.0.smithi038.stdout:8/965: dread d1/d7/d22/da7/f10a [0,4194304] 0 2024-02-15T18:35:00.145 INFO:tasks.workunit.client.0.smithi038.stdout:2/709: unlink dd/d12/d3a/d67/f27 0 2024-02-15T18:35:00.145 INFO:tasks.workunit.client.0.smithi038.stdout:3/705: dwrite d2/d1b/d3c/d41/d4e/d62/f87 [4194304,4194304] 0 2024-02-15T18:35:00.147 INFO:tasks.workunit.client.0.smithi038.stdout:7/766: dwrite d0/db/d59/d15/d29/d8d/f10c [0,4194304] 0 2024-02-15T18:35:00.147 INFO:tasks.workunit.client.0.smithi038.stdout:0/876: dwrite d4/f11 [0,4194304] 0 2024-02-15T18:35:00.148 INFO:tasks.workunit.client.0.smithi038.stdout:0/877: dread d4/d13/d53/fa7 [0,4194304] 0 2024-02-15T18:35:00.148 INFO:tasks.workunit.client.0.smithi038.stdout:0/878: stat d4/d13/d2a/d35/d1e/d32/d3b/daa/df8/fd1 0 2024-02-15T18:35:00.156 INFO:tasks.workunit.client.0.smithi038.stdout:4/797: dread d11/d1e/d43/d5e/f112 [0,4194304] 0 2024-02-15T18:35:00.159 INFO:tasks.workunit.client.0.smithi038.stdout:4/798: dread d11/d1e/d20/d31/d8d/d99/db5/fcb [0,4194304] 0 2024-02-15T18:35:00.168 INFO:tasks.workunit.client.0.smithi038.stdout:4/799: dread d11/d1e/d20/d2b/d4d/d67/d73/fe3 [0,4194304] 0 2024-02-15T18:35:00.178 INFO:tasks.workunit.client.0.smithi038.stdout:6/754: mknod d2/d4/d2c/d39/db4/ce7 0 2024-02-15T18:35:00.180 INFO:tasks.workunit.client.0.smithi038.stdout:5/768: rename d2/f3f to d2/d64/d99/db6/fe8 0 2024-02-15T18:35:00.183 INFO:tasks.workunit.client.0.smithi038.stdout:2/710: symlink dd/d12/d6b/le5 0 2024-02-15T18:35:00.186 INFO:tasks.workunit.client.0.smithi038.stdout:0/879: mkdir d4/d13/d2a/d35/d1e/d32/d79/d7d/da3/d118/d127 0 2024-02-15T18:35:00.187 INFO:tasks.workunit.client.0.smithi038.stdout:4/800: rmdir d11/d1e/d20/d31/d8d 39 2024-02-15T18:35:00.187 INFO:tasks.workunit.client.0.smithi038.stdout:4/801: fsync d11/d1e/d20/d2b/f61 0 2024-02-15T18:35:00.187 INFO:tasks.workunit.client.0.smithi038.stdout:4/802: stat d11/d1e/d3e 0 2024-02-15T18:35:00.187 INFO:tasks.workunit.client.0.smithi038.stdout:4/803: fdatasync d11/f1d 0 2024-02-15T18:35:00.187 INFO:tasks.workunit.client.0.smithi038.stdout:4/804: read - d11/d1e/d20/d2b/d4d/d67/d73/f10f zero size 2024-02-15T18:35:00.188 INFO:tasks.workunit.client.0.smithi038.stdout:4/805: write d11/d1e/d20/d31/da2/d118/f90 [306603,102289] 0 2024-02-15T18:35:00.189 INFO:tasks.workunit.client.0.smithi038.stdout:6/755: rmdir d2/d4/d6c 39 2024-02-15T18:35:00.189 INFO:tasks.workunit.client.0.smithi038.stdout:5/769: dread d2/f6d [0,4194304] 0 2024-02-15T18:35:00.193 INFO:tasks.workunit.client.0.smithi038.stdout:8/966: rename d1/d4/d5/d77/f9f to d1/d4/d10/d43/d48/f14e 0 2024-02-15T18:35:00.197 INFO:tasks.workunit.client.0.smithi038.stdout:0/880: truncate d4/d13/d2a/d35/d1e/d32/fb6 840534 0 2024-02-15T18:35:00.198 INFO:tasks.workunit.client.0.smithi038.stdout:4/806: mkdir d11/d1e/d20/d31/d48/d11e 0 2024-02-15T18:35:00.199 INFO:tasks.workunit.client.0.smithi038.stdout:6/756: symlink d2/d4/d6c/le8 0 2024-02-15T18:35:00.199 INFO:tasks.workunit.client.0.smithi038.stdout:6/757: dread - d2/d4/d6c/fcb zero size 2024-02-15T18:35:00.199 INFO:tasks.workunit.client.0.smithi038.stdout:5/770: symlink d2/d3/le9 0 2024-02-15T18:35:00.203 INFO:tasks.workunit.client.0.smithi038.stdout:7/767: rename d0/db/d59/dc8/dd6/d61/d90/lcb to d0/db/d19/d7d/daa/l112 0 2024-02-15T18:35:00.204 INFO:tasks.workunit.client.0.smithi038.stdout:8/967: creat d1/d7/f14f x:0 0 0 2024-02-15T18:35:00.204 INFO:tasks.workunit.client.0.smithi038.stdout:8/968: truncate d1/d7/d26/d11b/fed 4734388 0 2024-02-15T18:35:00.205 INFO:tasks.workunit.client.0.smithi038.stdout:2/711: symlink dd/d12/d59/d62/le6 0 2024-02-15T18:35:00.206 INFO:tasks.workunit.client.0.smithi038.stdout:4/807: creat d11/d1e/d20/d31/d48/dac/dbe/d40/f11f x:0 0 0 2024-02-15T18:35:00.209 INFO:tasks.workunit.client.0.smithi038.stdout:2/712: creat dd/d12/d59/d2b/d7a/fe7 x:0 0 0 2024-02-15T18:35:00.210 INFO:tasks.workunit.client.0.smithi038.stdout:0/881: symlink d4/d13/d2a/d35/d1e/d32/d48/d97/l128 0 2024-02-15T18:35:00.214 INFO:tasks.workunit.client.0.smithi038.stdout:6/758: dwrite d2/d10/f6b [0,4194304] 0 2024-02-15T18:35:00.214 INFO:tasks.workunit.client.0.smithi038.stdout:2/713: rename dd/d12/d8d to dd/d12/d3a/d67/d4f/d5a/dbc/de8 0 2024-02-15T18:35:00.214 INFO:tasks.workunit.client.0.smithi038.stdout:8/969: dread d1/d4/d5/d6/fcb [0,4194304] 0 2024-02-15T18:35:00.214 INFO:tasks.workunit.client.0.smithi038.stdout:8/970: chown d1/d4/d5/d6/d6d 6262 1 2024-02-15T18:35:00.214 INFO:tasks.workunit.client.0.smithi038.stdout:0/882: dread d4/d13/d2a/d35/d1e/d32/d3b/f8d [0,4194304] 0 2024-02-15T18:35:00.218 INFO:tasks.workunit.client.0.smithi038.stdout:4/808: dread d11/d1e/d43/d5e/dfe/ffb [0,4194304] 0 2024-02-15T18:35:00.219 INFO:tasks.workunit.client.0.smithi038.stdout:6/759: mkdir d2/d4/d2c/d39/de9 0 2024-02-15T18:35:00.219 INFO:tasks.workunit.client.0.smithi038.stdout:8/971: write d1/d4/d5/d6/f6e [942860,92061] 0 2024-02-15T18:35:00.220 INFO:tasks.workunit.client.0.smithi038.stdout:0/883: creat d4/db3/f129 x:0 0 0 2024-02-15T18:35:00.221 INFO:tasks.workunit.client.0.smithi038.stdout:4/809: creat d11/d1e/d43/d5e/d62/f120 x:0 0 0 2024-02-15T18:35:00.221 INFO:tasks.workunit.client.0.smithi038.stdout:7/768: dwrite d0/db/d59/d15/d29/d8d/f10c [0,4194304] 0 2024-02-15T18:35:00.223 INFO:tasks.workunit.client.0.smithi038.stdout:6/760: creat d2/d4/d19/d78/dc3/fea x:0 0 0 2024-02-15T18:35:00.224 INFO:tasks.workunit.client.0.smithi038.stdout:6/761: write d2/f5b [1934990,13598] 0 2024-02-15T18:35:00.225 INFO:tasks.workunit.client.0.smithi038.stdout:8/972: rename d1/d4/db1/de5/db3/d109 to d1/d4/d10/d43/d48/d52/d150 0 2024-02-15T18:35:00.226 INFO:tasks.workunit.client.0.smithi038.stdout:0/884: link d4/d13/d19/l9d d4/d13/d19/dfc/l12a 0 2024-02-15T18:35:00.228 INFO:tasks.workunit.client.0.smithi038.stdout:4/810: creat d11/d1e/d20/d31/d48/d102/f121 x:0 0 0 2024-02-15T18:35:00.228 INFO:tasks.workunit.client.0.smithi038.stdout:4/811: chown d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/ff5 1 1 2024-02-15T18:35:00.228 INFO:tasks.workunit.client.0.smithi038.stdout:4/812: fsync d11/d1e/d43/d5e/f112 0 2024-02-15T18:35:00.228 INFO:tasks.workunit.client.0.smithi038.stdout:4/813: truncate d11/d1e/d20/d2b/d4d/d67/d73/f10c 58794 0 2024-02-15T18:35:00.228 INFO:tasks.workunit.client.0.smithi038.stdout:4/814: chown d11/d1e/d20/d31/d48/dac/dbe 195496446 1 2024-02-15T18:35:00.228 INFO:tasks.workunit.client.0.smithi038.stdout:7/769: mknod d0/db/d59/d15/d29/d8d/dcd/c113 0 2024-02-15T18:35:00.229 INFO:tasks.workunit.client.0.smithi038.stdout:6/762: symlink d2/leb 0 2024-02-15T18:35:00.231 INFO:tasks.workunit.client.0.smithi038.stdout:0/885: mkdir d4/d13/d2a/d35/d1e/d12b 0 2024-02-15T18:35:00.232 INFO:tasks.workunit.client.0.smithi038.stdout:7/770: link d0/db/d19/fd2 d0/db/d59/d15/d29/f114 0 2024-02-15T18:35:00.234 INFO:tasks.workunit.client.0.smithi038.stdout:8/973: write d1/d7/d22/f85 [3145400,75915] 0 2024-02-15T18:35:00.236 INFO:tasks.workunit.client.0.smithi038.stdout:8/974: mkdir d1/d4/d5/d151 0 2024-02-15T18:35:00.236 INFO:tasks.workunit.client.0.smithi038.stdout:8/975: stat d1/d7/d26/c68 0 2024-02-15T18:35:00.237 INFO:tasks.workunit.client.0.smithi038.stdout:2/714: dwrite dd/d12/d1d/d3b/d3c/d4a/fb3 [0,4194304] 0 2024-02-15T18:35:00.237 INFO:tasks.workunit.client.0.smithi038.stdout:0/886: write d4/ff [3473840,118199] 0 2024-02-15T18:35:00.237 INFO:tasks.workunit.client.0.smithi038.stdout:0/887: chown d4/db3/dc7/f112 4 1 2024-02-15T18:35:00.237 INFO:tasks.workunit.client.0.smithi038.stdout:7/771: write d0/d1b/d30/f43 [1783424,120068] 0 2024-02-15T18:35:00.238 INFO:tasks.workunit.client.0.smithi038.stdout:8/976: mknod d1/d3a/c152 0 2024-02-15T18:35:00.239 INFO:tasks.workunit.client.0.smithi038.stdout:7/772: dread d0/db/d59/dc8/dd6/f82 [0,4194304] 0 2024-02-15T18:35:00.240 INFO:tasks.workunit.client.0.smithi038.stdout:8/977: dread d1/d7/d22/f133 [4194304,4194304] 0 2024-02-15T18:35:00.241 INFO:tasks.workunit.client.0.smithi038.stdout:8/978: read - d1/d4/d10/d43/d48/d9a/fcf zero size 2024-02-15T18:35:00.243 INFO:tasks.workunit.client.0.smithi038.stdout:2/715: mknod dd/d12/d3a/d67/d4f/ce9 0 2024-02-15T18:35:00.243 INFO:tasks.workunit.client.0.smithi038.stdout:4/815: dread d11/d1e/d20/d31/d7a/dd8/fe5 [0,4194304] 0 2024-02-15T18:35:00.243 INFO:tasks.workunit.client.0.smithi038.stdout:4/816: truncate d11/d1e/d20/d31/d7a/fbc 541413 0 2024-02-15T18:35:00.243 INFO:tasks.workunit.client.0.smithi038.stdout:6/763: read d2/d4/d2c/d39/db4/f7c [739015,67303] 0 2024-02-15T18:35:00.244 INFO:tasks.workunit.client.0.smithi038.stdout:0/888: creat d4/d13/d2a/d35/d1e/f12c x:0 0 0 2024-02-15T18:35:00.254 INFO:tasks.workunit.client.0.smithi038.stdout:7/773: symlink d0/db/d59/d15/d4f/l115 0 2024-02-15T18:35:00.256 INFO:tasks.workunit.client.0.smithi038.stdout:4/817: dread d11/d1e/d20/d2b/d35/f89 [0,4194304] 0 2024-02-15T18:35:00.258 INFO:tasks.workunit.client.0.smithi038.stdout:8/979: mkdir d1/d4/db1/de5/d153 0 2024-02-15T18:35:00.261 INFO:tasks.workunit.client.0.smithi038.stdout:0/889: write d4/d13/d2a/d35/db2/fdc [3275850,40850] 0 2024-02-15T18:35:00.261 INFO:tasks.workunit.client.0.smithi038.stdout:2/716: dwrite fb [0,4194304] 0 2024-02-15T18:35:00.262 INFO:tasks.workunit.client.0.smithi038.stdout:6/764: creat d2/d4/d2c/d39/db4/d5f/d99/fec x:0 0 0 2024-02-15T18:35:00.263 INFO:tasks.workunit.client.0.smithi038.stdout:7/774: truncate d0/db/d59/dc8/f103 563939 0 2024-02-15T18:35:00.265 INFO:tasks.workunit.client.0.smithi038.stdout:0/890: getdents d4/d13/d2a/d35/d1e/d32/d3b/daa 0 2024-02-15T18:35:00.266 INFO:tasks.workunit.client.0.smithi038.stdout:2/717: mkdir dd/d12/d59/dce/dea 0 2024-02-15T18:35:00.269 INFO:tasks.workunit.client.0.smithi038.stdout:0/891: dread d4/d13/f64 [0,4194304] 0 2024-02-15T18:35:00.271 INFO:tasks.workunit.client.0.smithi038.stdout:7/775: unlink d0/d68/fe1 0 2024-02-15T18:35:00.273 INFO:tasks.workunit.client.0.smithi038.stdout:0/892: mknod d4/d13/d2a/c12d 0 2024-02-15T18:35:00.275 INFO:tasks.workunit.client.0.smithi038.stdout:1/763: sync 2024-02-15T18:35:00.282 INFO:tasks.workunit.client.0.smithi038.stdout:2/718: dread dd/d12/d5d/d54/f97 [0,4194304] 0 2024-02-15T18:35:00.282 INFO:tasks.workunit.client.0.smithi038.stdout:7/776: creat d0/d1b/dfd/f116 x:0 0 0 2024-02-15T18:35:00.282 INFO:tasks.workunit.client.0.smithi038.stdout:2/719: chown dd/d12/d5d/d33/l38 6 1 2024-02-15T18:35:00.286 INFO:tasks.workunit.client.0.smithi038.stdout:1/764: mkdir d2/d10/d62/dfc 0 2024-02-15T18:35:00.286 INFO:tasks.workunit.client.0.smithi038.stdout:4/818: dwrite d11/d1e/d20/d31/d8d/fb2 [0,4194304] 0 2024-02-15T18:35:00.293 INFO:tasks.workunit.client.0.smithi038.stdout:8/980: dwrite d1/d4/d5/d6/fcb [0,4194304] 0 2024-02-15T18:35:00.294 INFO:tasks.workunit.client.0.smithi038.stdout:8/981: stat d1/d4/d10/d91/cbc 0 2024-02-15T18:35:00.294 INFO:tasks.workunit.client.0.smithi038.stdout:8/982: truncate d1/d4/d10/d43/d48/d9a/fa5 2621243 0 2024-02-15T18:35:00.294 INFO:tasks.workunit.client.0.smithi038.stdout:8/983: chown d1/d7/d22/da7/cd2 2969 1 2024-02-15T18:35:00.294 INFO:tasks.workunit.client.0.smithi038.stdout:8/984: write d1/d4/d10/d43/d48/d52/fba [168218,44861] 0 2024-02-15T18:35:00.294 INFO:tasks.workunit.client.0.smithi038.stdout:2/720: dread dd/d12/d59/d2b/fd1 [0,4194304] 0 2024-02-15T18:35:00.295 INFO:tasks.workunit.client.0.smithi038.stdout:7/777: dread d0/db/fb1 [0,4194304] 0 2024-02-15T18:35:00.295 INFO:tasks.workunit.client.0.smithi038.stdout:7/778: truncate d0/f48 1121485 0 2024-02-15T18:35:00.295 INFO:tasks.workunit.client.0.smithi038.stdout:7/779: readlink d0/db/d19/d7d/db5/lde 0 2024-02-15T18:35:00.295 INFO:tasks.workunit.client.0.smithi038.stdout:8/985: write d1/d4/d5/d6/db9/f129 [4009021,10978] 0 2024-02-15T18:35:00.299 INFO:tasks.workunit.client.0.smithi038.stdout:2/721: write dd/d12/d3a/d5e/d65/fba [93280,57805] 0 2024-02-15T18:35:00.299 INFO:tasks.workunit.client.0.smithi038.stdout:6/765: dwrite d2/d4/d2c/d39/d76/dc2/fce [0,4194304] 0 2024-02-15T18:35:00.299 INFO:tasks.workunit.client.0.smithi038.stdout:6/766: truncate d2/f58 628667 0 2024-02-15T18:35:00.299 INFO:tasks.workunit.client.0.smithi038.stdout:7/780: creat d0/d66/f117 x:0 0 0 2024-02-15T18:35:00.299 INFO:tasks.workunit.client.0.smithi038.stdout:8/986: symlink d1/d4/d10/d43/l154 0 2024-02-15T18:35:00.299 INFO:tasks.workunit.client.0.smithi038.stdout:4/819: dread d11/d1e/d20/d2b/f61 [0,4194304] 0 2024-02-15T18:35:00.302 INFO:tasks.workunit.client.0.smithi038.stdout:6/767: write d2/f5a [524653,4232] 0 2024-02-15T18:35:00.302 INFO:tasks.workunit.client.0.smithi038.stdout:6/768: dread - d2/d4/d19/d78/fae zero size 2024-02-15T18:35:00.307 INFO:tasks.workunit.client.0.smithi038.stdout:1/765: dwrite d2/d10/d19/d22/d36/d94/dc1/dd5/fe7 [4194304,4194304] 0 2024-02-15T18:35:00.315 INFO:tasks.workunit.client.0.smithi038.stdout:2/722: dwrite dd/d12/d3a/d5e/d65/fbf [0,4194304] 0 2024-02-15T18:35:00.316 INFO:tasks.workunit.client.0.smithi038.stdout:7/781: mknod d0/d1b/d80/dbc/c118 0 2024-02-15T18:35:00.325 INFO:tasks.workunit.client.0.smithi038.stdout:8/987: symlink d1/d4/db1/de5/db3/l155 0 2024-02-15T18:35:00.328 INFO:tasks.workunit.client.0.smithi038.stdout:6/769: rmdir d2/d4/d6f 39 2024-02-15T18:35:00.328 INFO:tasks.workunit.client.0.smithi038.stdout:6/770: chown d2/d4/d2c/d39/db4/f8d 38250 1 2024-02-15T18:35:00.328 INFO:tasks.workunit.client.0.smithi038.stdout:1/766: creat d2/d10/d19/d22/deb/ffd x:0 0 0 2024-02-15T18:35:00.329 INFO:tasks.workunit.client.0.smithi038.stdout:6/771: write d2/d4/d19/d55/f6a [565467,125177] 0 2024-02-15T18:35:00.330 INFO:tasks.workunit.client.0.smithi038.stdout:6/772: read d2/d4/d19/d55/f6a [488444,130237] 0 2024-02-15T18:35:00.330 INFO:tasks.workunit.client.0.smithi038.stdout:6/773: readlink d2/l61 0 2024-02-15T18:35:00.333 INFO:tasks.workunit.client.0.smithi038.stdout:2/723: link dd/d12/d1d/d3b/d3c/d4a/fdd dd/d12/d59/d2b/d6a/feb 0 2024-02-15T18:35:00.333 INFO:tasks.workunit.client.0.smithi038.stdout:7/782: creat d0/d66/f119 x:0 0 0 2024-02-15T18:35:00.334 INFO:tasks.workunit.client.0.smithi038.stdout:8/988: creat d1/d7/d15/d4b/f156 x:0 0 0 2024-02-15T18:35:00.335 INFO:tasks.workunit.client.0.smithi038.stdout:6/774: creat d2/d4/d6c/fed x:0 0 0 2024-02-15T18:35:00.335 INFO:tasks.workunit.client.0.smithi038.stdout:6/775: write d2/d10/f96 [604679,116217] 0 2024-02-15T18:35:00.337 INFO:tasks.workunit.client.0.smithi038.stdout:2/724: dread dd/d12/d5d/fb1 [0,4194304] 0 2024-02-15T18:35:00.337 INFO:tasks.workunit.client.0.smithi038.stdout:2/725: truncate dd/d12/d5d/d54/fa4 1043268 0 2024-02-15T18:35:00.337 INFO:tasks.workunit.client.0.smithi038.stdout:2/726: stat dd/d12/d3a/d5e/d65 0 2024-02-15T18:35:00.337 INFO:tasks.workunit.client.0.smithi038.stdout:8/989: creat d1/d4/d5/d31/f157 x:0 0 0 2024-02-15T18:35:00.341 INFO:tasks.workunit.client.0.smithi038.stdout:9/898: sync 2024-02-15T18:35:00.341 INFO:tasks.workunit.client.0.smithi038.stdout:9/899: readlink d1/dc/l29 0 2024-02-15T18:35:00.341 INFO:tasks.workunit.client.0.smithi038.stdout:9/900: chown d1/da/d20/d58/d94/db9/dc6/c11e 46 1 2024-02-15T18:35:00.341 INFO:tasks.workunit.client.0.smithi038.stdout:6/776: creat d2/d4/d2c/d39/db4/d5f/d99/fee x:0 0 0 2024-02-15T18:35:00.341 INFO:tasks.workunit.client.0.smithi038.stdout:6/777: fsync d2/d4/d19/d55/d5c/faa 0 2024-02-15T18:35:00.344 INFO:tasks.workunit.client.0.smithi038.stdout:5/771: sync 2024-02-15T18:35:00.344 INFO:tasks.workunit.client.0.smithi038.stdout:3/706: sync 2024-02-15T18:35:00.344 INFO:tasks.workunit.client.0.smithi038.stdout:3/707: dread - d2/d1e/d7e/fbc zero size 2024-02-15T18:35:00.345 INFO:tasks.workunit.client.0.smithi038.stdout:5/772: read d2/d30/f73 [115023,21518] 0 2024-02-15T18:35:00.345 INFO:tasks.workunit.client.0.smithi038.stdout:5/773: dread - d2/d48/d83/fe2 zero size 2024-02-15T18:35:00.345 INFO:tasks.workunit.client.0.smithi038.stdout:5/774: readlink d2/d3/da0/d33/lb7 0 2024-02-15T18:35:00.345 INFO:tasks.workunit.client.0.smithi038.stdout:5/775: fsync d2/d3/d5f/fce 0 2024-02-15T18:35:00.346 INFO:tasks.workunit.client.0.smithi038.stdout:7/783: dread d0/db/f7a [0,4194304] 0 2024-02-15T18:35:00.347 INFO:tasks.workunit.client.0.smithi038.stdout:8/990: getdents d1/d7/d26/d69 0 2024-02-15T18:35:00.347 INFO:tasks.workunit.client.0.smithi038.stdout:8/991: dread - d1/d3a/df6/d12b/f136 zero size 2024-02-15T18:35:00.356 INFO:tasks.workunit.client.0.smithi038.stdout:3/708: dread d2/db8/dcf/fd7 [0,4194304] 0 2024-02-15T18:35:00.356 INFO:tasks.workunit.client.0.smithi038.stdout:9/901: creat d1/dc/d4f/f13f x:0 0 0 2024-02-15T18:35:00.356 INFO:tasks.workunit.client.0.smithi038.stdout:6/778: rename d2/d10/f30 to d2/d4/d19/d55/fef 0 2024-02-15T18:35:00.357 INFO:tasks.workunit.client.0.smithi038.stdout:9/902: creat d1/da/d20/d58/d94/de8/f140 x:0 0 0 2024-02-15T18:35:00.359 INFO:tasks.workunit.client.0.smithi038.stdout:0/893: sync 2024-02-15T18:35:00.361 INFO:tasks.workunit.client.0.smithi038.stdout:0/894: write d4/f5 [2029306,86359] 0 2024-02-15T18:35:00.361 INFO:tasks.workunit.client.0.smithi038.stdout:0/895: fdatasync d4/d13/d2a/d35/d1e/f12c 0 2024-02-15T18:35:00.361 INFO:tasks.workunit.client.0.smithi038.stdout:9/903: mkdir d1/da/d20/d58/d141 0 2024-02-15T18:35:00.362 INFO:tasks.workunit.client.0.smithi038.stdout:9/904: truncate d1/dc/fd7 897375 0 2024-02-15T18:35:00.362 INFO:tasks.workunit.client.0.smithi038.stdout:0/896: mkdir d4/d13/d2a/d35/d1e/d32/d48/d12e 0 2024-02-15T18:35:00.364 INFO:tasks.workunit.client.0.smithi038.stdout:9/905: mknod d1/da/d10/d13/d3d/d4b/daa/c142 0 2024-02-15T18:35:00.364 INFO:tasks.workunit.client.0.smithi038.stdout:0/897: rename d4/d13/d2a/d35/d1e/d32/d3b/lbe to d4/d13/d2a/d35/d1e/d32/d79/d7d/dda/l12f 0 2024-02-15T18:35:00.365 INFO:tasks.workunit.client.0.smithi038.stdout:9/906: mknod d1/da/d20/d58/d94/d13c/d11c/db8/c143 0 2024-02-15T18:35:00.366 INFO:tasks.workunit.client.0.smithi038.stdout:0/898: creat d4/d13/d2a/d35/db2/f130 x:0 0 0 2024-02-15T18:35:00.366 INFO:tasks.workunit.client.0.smithi038.stdout:6/779: write d2/d4/d19/d55/d5c/faa [952717,74627] 0 2024-02-15T18:35:00.366 INFO:tasks.workunit.client.0.smithi038.stdout:8/992: dread d1/d4/d5/d6/db9/f129 [0,4194304] 0 2024-02-15T18:35:00.366 INFO:tasks.workunit.client.0.smithi038.stdout:8/993: stat d1/d4/d5/d31/d65/d80/fac 0 2024-02-15T18:35:00.366 INFO:tasks.workunit.client.0.smithi038.stdout:9/907: creat d1/da/d10/d13/d3d/d43/d4d/d105/d121/f144 x:0 0 0 2024-02-15T18:35:00.369 INFO:tasks.workunit.client.0.smithi038.stdout:6/780: rmdir d2/d4/d2c/d39/db4/d5f 39 2024-02-15T18:35:00.369 INFO:tasks.workunit.client.0.smithi038.stdout:6/781: fdatasync d2/d10/f49 0 2024-02-15T18:35:00.369 INFO:tasks.workunit.client.0.smithi038.stdout:9/908: mkdir d1/da/d36/d93/d145 0 2024-02-15T18:35:00.371 INFO:tasks.workunit.client.0.smithi038.stdout:2/727: dwrite dd/d12/d3a/d67/fa0 [0,4194304] 0 2024-02-15T18:35:00.372 INFO:tasks.workunit.client.0.smithi038.stdout:9/909: mkdir d1/da/d36/d8b/d146 0 2024-02-15T18:35:00.373 INFO:tasks.workunit.client.0.smithi038.stdout:5/776: dwrite d2/d3/da0/d33/fc8 [0,4194304] 0 2024-02-15T18:35:00.374 INFO:tasks.workunit.client.0.smithi038.stdout:2/728: creat dd/d12/d5d/d54/dcb/fec x:0 0 0 2024-02-15T18:35:00.374 INFO:tasks.workunit.client.0.smithi038.stdout:2/729: truncate dd/d12/d1d/d3b/d3c/d4a/f8c 226605 0 2024-02-15T18:35:00.377 INFO:tasks.workunit.client.0.smithi038.stdout:6/782: dread d2/d4/d6f/f83 [0,4194304] 0 2024-02-15T18:35:00.377 INFO:tasks.workunit.client.0.smithi038.stdout:9/910: creat d1/da/d10/f147 x:0 0 0 2024-02-15T18:35:00.378 INFO:tasks.workunit.client.0.smithi038.stdout:9/911: truncate d1/da/d20/d58/d94/d13c/d11c/ffc 264116 0 2024-02-15T18:35:00.383 INFO:tasks.workunit.client.0.smithi038.stdout:8/994: dread d1/d4/d10/d3d/f97 [0,4194304] 0 2024-02-15T18:35:00.384 INFO:tasks.workunit.client.0.smithi038.stdout:3/709: dwrite d2/d25/fcc [0,4194304] 0 2024-02-15T18:35:00.386 INFO:tasks.workunit.client.0.smithi038.stdout:7/784: dwrite d0/db/d19/f4e [4194304,4194304] 0 2024-02-15T18:35:00.386 INFO:tasks.workunit.client.0.smithi038.stdout:7/785: fsync d0/db/d19/fd2 0 2024-02-15T18:35:00.388 INFO:tasks.workunit.client.0.smithi038.stdout:8/995: truncate d1/d7/d26/fe1 1210542 0 2024-02-15T18:35:00.388 INFO:tasks.workunit.client.0.smithi038.stdout:3/710: dread d2/d1b/f4a [0,4194304] 0 2024-02-15T18:35:00.391 INFO:tasks.workunit.client.0.smithi038.stdout:7/786: symlink d0/db/d19/l11a 0 2024-02-15T18:35:00.392 INFO:tasks.workunit.client.0.smithi038.stdout:3/711: unlink d2/d1b/d3c/d41/fb3 0 2024-02-15T18:35:00.393 INFO:tasks.workunit.client.0.smithi038.stdout:6/783: dread d2/d4/d2c/f4f [0,4194304] 0 2024-02-15T18:35:00.395 INFO:tasks.workunit.client.0.smithi038.stdout:3/712: creat d2/d1b/d3c/d3a/fdf x:0 0 0 2024-02-15T18:35:00.397 INFO:tasks.workunit.client.0.smithi038.stdout:6/784: unlink d2/d4/d2c/d39/db4/ld5 0 2024-02-15T18:35:00.397 INFO:tasks.workunit.client.0.smithi038.stdout:6/785: dread - d2/d4/d19/d78/dc3/fea zero size 2024-02-15T18:35:00.401 INFO:tasks.workunit.client.0.smithi038.stdout:3/713: unlink d2/d1b/d3c/d3a/f91 0 2024-02-15T18:35:00.404 INFO:tasks.workunit.client.0.smithi038.stdout:2/730: dwrite dd/d12/d59/d2b/d6a/feb [0,4194304] 0 2024-02-15T18:35:00.408 INFO:tasks.workunit.client.0.smithi038.stdout:2/731: rmdir dd/d12/d59/db2 39 2024-02-15T18:35:00.409 INFO:tasks.workunit.client.0.smithi038.stdout:6/786: write d2/f5b [623435,16871] 0 2024-02-15T18:35:00.409 INFO:tasks.workunit.client.0.smithi038.stdout:6/787: fsync d2/d4/d2c/d39/db4/f8d 0 2024-02-15T18:35:00.410 INFO:tasks.workunit.client.0.smithi038.stdout:6/788: mkdir d2/d4/d19/d78/dc4/df0 0 2024-02-15T18:35:00.411 INFO:tasks.workunit.client.0.smithi038.stdout:0/899: dwrite d4/d13/d2a/d35/d1e/f4d [0,4194304] 0 2024-02-15T18:35:00.411 INFO:tasks.workunit.client.0.smithi038.stdout:0/900: dread - d4/d13/d2a/d35/d1e/d32/d3b/f105 zero size 2024-02-15T18:35:00.411 INFO:tasks.workunit.client.0.smithi038.stdout:0/901: fdatasync d4/d13/d2a/d35/d1e/d32/d79/d7d/fba 0 2024-02-15T18:35:00.412 INFO:tasks.workunit.client.0.smithi038.stdout:0/902: creat d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/ddb/d120/f131 x:0 0 0 2024-02-15T18:35:00.413 INFO:tasks.workunit.client.0.smithi038.stdout:6/789: write d2/d10/db6/fa5 [368038,43050] 0 2024-02-15T18:35:00.413 INFO:tasks.workunit.client.0.smithi038.stdout:3/714: dread d2/d1b/f36 [0,4194304] 0 2024-02-15T18:35:00.413 INFO:tasks.workunit.client.0.smithi038.stdout:3/715: dread - d2/d1e/fd9 zero size 2024-02-15T18:35:00.414 INFO:tasks.workunit.client.0.smithi038.stdout:6/790: rename d2/d10/f4c to d2/d4/d2c/d39/ff1 0 2024-02-15T18:35:00.414 INFO:tasks.workunit.client.0.smithi038.stdout:6/791: readlink d2/d4/d2c/d39/d76/la9 0 2024-02-15T18:35:00.415 INFO:tasks.workunit.client.0.smithi038.stdout:3/716: mkdir d2/d1b/d3c/d3a/de0 0 2024-02-15T18:35:00.415 INFO:tasks.workunit.client.0.smithi038.stdout:6/792: write d2/d4/d19/d55/f8c [315448,118741] 0 2024-02-15T18:35:00.419 INFO:tasks.workunit.client.0.smithi038.stdout:6/793: write d2/d4/d19/f74 [3921934,130034] 0 2024-02-15T18:35:00.420 INFO:tasks.workunit.client.0.smithi038.stdout:6/794: fsync f1 0 2024-02-15T18:35:00.420 INFO:tasks.workunit.client.0.smithi038.stdout:6/795: write d2/d4/d19/d78/dbf/fe5 [969330,82700] 0 2024-02-15T18:35:00.420 INFO:tasks.workunit.client.0.smithi038.stdout:6/796: truncate d2/d4/d19/d55/f8c 1463590 0 2024-02-15T18:35:00.420 INFO:tasks.workunit.client.0.smithi038.stdout:0/903: dread d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/fd7 [0,4194304] 0 2024-02-15T18:35:00.420 INFO:tasks.workunit.client.0.smithi038.stdout:9/912: dwrite d1/fb [4194304,4194304] 0 2024-02-15T18:35:00.422 INFO:tasks.workunit.client.0.smithi038.stdout:8/996: dwrite d1/d4/faa [0,4194304] 0 2024-02-15T18:35:00.422 INFO:tasks.workunit.client.0.smithi038.stdout:8/997: stat d1/d4/d5/d6/db9 0 2024-02-15T18:35:00.422 INFO:tasks.workunit.client.0.smithi038.stdout:7/787: dwrite d0/db/d19/f4e [0,4194304] 0 2024-02-15T18:35:00.422 INFO:tasks.workunit.client.0.smithi038.stdout:7/788: truncate d0/db/d59/d15/d29/d8d/f9a 621544 0 2024-02-15T18:35:00.423 INFO:tasks.workunit.client.0.smithi038.stdout:6/797: link d2/d4/d2c/c6d d2/d4/d2c/cf2 0 2024-02-15T18:35:00.423 INFO:tasks.workunit.client.0.smithi038.stdout:9/913: rmdir d1/da/d20/d58/d94/db9/dc6/d49 39 2024-02-15T18:35:00.423 INFO:tasks.workunit.client.0.smithi038.stdout:9/914: chown d1/da/d20/d58/d94/db9/dc6/d49 27200772 1 2024-02-15T18:35:00.423 INFO:tasks.workunit.client.0.smithi038.stdout:9/915: fsync d1/da/d20/d58/d94/d13c/d11c/f57 0 2024-02-15T18:35:00.424 INFO:tasks.workunit.client.0.smithi038.stdout:0/904: truncate d4/d13/d2a/d35/d1e/d32/d48/d97/faf 5768812 0 2024-02-15T18:35:00.424 INFO:tasks.workunit.client.0.smithi038.stdout:0/905: write d4/d13/d2a/d35/d6a/f9e [445974,23865] 0 2024-02-15T18:35:00.428 INFO:tasks.workunit.client.0.smithi038.stdout:7/789: creat d0/d1b/f11b x:0 0 0 2024-02-15T18:35:00.428 INFO:tasks.workunit.client.0.smithi038.stdout:7/790: chown d0/d1b/d30/d50/c55 735 1 2024-02-15T18:35:00.431 INFO:tasks.workunit.client.0.smithi038.stdout:8/998: dread d1/d7/d22/f5a [0,4194304] 0 2024-02-15T18:35:00.431 INFO:tasks.workunit.client.0.smithi038.stdout:0/906: write d4/d13/d2a/f44 [933144,77399] 0 2024-02-15T18:35:00.440 INFO:tasks.workunit.client.0.smithi038.stdout:9/916: dwrite d1/da/d36/d93/fa4 [0,4194304] 0 2024-02-15T18:35:00.453 INFO:tasks.workunit.client.0.smithi038.stdout:3/717: rmdir d2/d1b/d3c/d3a 39 2024-02-15T18:35:00.454 INFO:tasks.workunit.client.0.smithi038.stdout:3/718: chown d2/d1b/cca 3915 1 2024-02-15T18:35:00.456 INFO:tasks.workunit.client.0.smithi038.stdout:6/798: link d2/d10/f8f d2/d4/d2c/d39/db4/ff3 0 2024-02-15T18:35:00.460 INFO:tasks.workunit.client.0.smithi038.stdout:7/791: rename d0/db/fe to d0/d1b/dfd/d108/f11c 0 2024-02-15T18:35:00.460 INFO:tasks.workunit.client.0.smithi038.stdout:7/792: truncate d0/db/d59/dc8/dd6/fd4 489778 0 2024-02-15T18:35:00.460 INFO:tasks.workunit.client.0.smithi038.stdout:7/793: readlink d0/db/d19/d7d/la7 0 2024-02-15T18:35:00.460 INFO:tasks.workunit.client.0.smithi038.stdout:0/907: rename d4/d13/d2a/d35/d1e/d32 to d4/d13/d2a/d35/d1e/d32/d79/d132 22 2024-02-15T18:35:00.461 INFO:tasks.workunit.client.0.smithi038.stdout:8/999: rmdir d1/d7/d22/da7 39 2024-02-15T18:35:00.464 INFO:tasks.workunit.client.0.smithi038.stdout:3/719: mkdir d2/d7f/dc1/de1 0 2024-02-15T18:35:00.464 INFO:tasks.workunit.client.0.smithi038.stdout:6/799: mknod d2/d4/d2c/d39/d76/dc2/cf4 0 2024-02-15T18:35:00.464 INFO:tasks.workunit.client.0.smithi038.stdout:7/794: creat d0/f11d x:0 0 0 2024-02-15T18:35:00.464 INFO:tasks.workunit.client.0.smithi038.stdout:7/795: chown d0/db/d19/d2d/f52 2 1 2024-02-15T18:35:00.466 INFO:tasks.workunit.client.0.smithi038.stdout:9/917: rename d1/da/d20/d58/d94/db9/de3/l138 to d1/da/d10/d82/l148 0 2024-02-15T18:35:00.467 INFO:tasks.workunit.client.0.smithi038.stdout:0/908: mkdir d4/d13/d2a/d35/d1e/d32/d133 0 2024-02-15T18:35:00.468 INFO:tasks.workunit.client.0.smithi038.stdout:3/720: creat d2/d1b/d3c/d3a/fe2 x:0 0 0 2024-02-15T18:35:00.468 INFO:tasks.workunit.client.0.smithi038.stdout:3/721: readlink d2/l9b 0 2024-02-15T18:35:00.470 INFO:tasks.workunit.client.0.smithi038.stdout:4/820: sync 2024-02-15T18:35:00.470 INFO:tasks.workunit.client.0.smithi038.stdout:4/821: readlink d11/d1e/d20/d31/d48/l8c 0 2024-02-15T18:35:00.473 INFO:tasks.workunit.client.0.smithi038.stdout:6/800: creat d2/d4/d6c/ff5 x:0 0 0 2024-02-15T18:35:00.473 INFO:tasks.workunit.client.0.smithi038.stdout:6/801: fsync d2/f2a 0 2024-02-15T18:35:00.474 INFO:tasks.workunit.client.0.smithi038.stdout:7/796: rename d0/ldd to d0/d1b/d30/d50/l11e 0 2024-02-15T18:35:00.474 INFO:tasks.workunit.client.0.smithi038.stdout:7/797: chown d0/d68/dfa 317722 1 2024-02-15T18:35:00.475 INFO:tasks.workunit.client.0.smithi038.stdout:7/798: chown d0/d1b/dfd/f116 5308 1 2024-02-15T18:35:00.475 INFO:tasks.workunit.client.0.smithi038.stdout:9/918: creat d1/da/d20/d58/d66/f149 x:0 0 0 2024-02-15T18:35:00.475 INFO:tasks.workunit.client.0.smithi038.stdout:9/919: write d1/da/d36/f10b [654674,125810] 0 2024-02-15T18:35:00.476 INFO:tasks.workunit.client.0.smithi038.stdout:0/909: creat d4/f134 x:0 0 0 2024-02-15T18:35:00.477 INFO:tasks.workunit.client.0.smithi038.stdout:3/722: mkdir d2/d1e/de3 0 2024-02-15T18:35:00.477 INFO:tasks.workunit.client.0.smithi038.stdout:3/723: write d2/d25/fcc [4809940,46411] 0 2024-02-15T18:35:00.482 INFO:tasks.workunit.client.0.smithi038.stdout:6/802: rmdir d2/d4/d6c/d9d/dd7 39 2024-02-15T18:35:00.482 INFO:tasks.workunit.client.0.smithi038.stdout:7/799: rename d0/db/d59/d15/d29/d7e/f73 to d0/db/d59/d15/d29/d8d/d102/d10a/f11f 0 2024-02-15T18:35:00.482 INFO:tasks.workunit.client.0.smithi038.stdout:7/800: chown d0/db/d59/d15/d29/d7e/f5b 116 1 2024-02-15T18:35:00.482 INFO:tasks.workunit.client.0.smithi038.stdout:9/920: rename d1/da/d20/d58/d141 to d1/da/d20/d58/d141/d14a 22 2024-02-15T18:35:00.484 INFO:tasks.workunit.client.0.smithi038.stdout:3/724: dread d2/db8/dcf/fd7 [0,4194304] 0 2024-02-15T18:35:00.484 INFO:tasks.workunit.client.0.smithi038.stdout:3/725: chown d2/d1b/d3c/d3a/d55/fb1 451246 1 2024-02-15T18:35:00.484 INFO:tasks.workunit.client.0.smithi038.stdout:3/726: write d2/d1b/d3c/d3a/fe2 [831283,35541] 0 2024-02-15T18:35:00.484 INFO:tasks.workunit.client.0.smithi038.stdout:3/727: dread - d2/d1b/d3c/d3a/fdf zero size 2024-02-15T18:35:00.485 INFO:tasks.workunit.client.0.smithi038.stdout:6/803: symlink d2/d4/d19/d78/dc4/df0/lf6 0 2024-02-15T18:35:00.485 INFO:tasks.workunit.client.0.smithi038.stdout:7/801: chown d0/db/d59/c14 3291 1 2024-02-15T18:35:00.487 INFO:tasks.workunit.client.0.smithi038.stdout:6/804: dread d2/d4/d6f/f83 [0,4194304] 0 2024-02-15T18:35:00.488 INFO:tasks.workunit.client.0.smithi038.stdout:9/921: rename d1/da/d20/d58/d94/d13c/l130 to d1/dc/d2b/l14b 0 2024-02-15T18:35:00.489 INFO:tasks.workunit.client.0.smithi038.stdout:9/922: dread - d1/da/d20/d58/d94/de8/f140 zero size 2024-02-15T18:35:00.490 INFO:tasks.workunit.client.0.smithi038.stdout:3/728: link d2/d1e/f89 d2/d1b/d3c/d3a/d55/da0/fe4 0 2024-02-15T18:35:00.492 INFO:tasks.workunit.client.0.smithi038.stdout:7/802: unlink d0/db/d19/f6d 0 2024-02-15T18:35:00.492 INFO:tasks.workunit.client.0.smithi038.stdout:7/803: fsync d0/db/d59/d15/d29/fe8 0 2024-02-15T18:35:00.493 INFO:tasks.workunit.client.0.smithi038.stdout:4/822: dwrite d11/d1e/d20/d2f/d71/f7b [0,4194304] 0 2024-02-15T18:35:00.500 INFO:tasks.workunit.client.0.smithi038.stdout:3/729: mknod d2/d1b/d69/ce5 0 2024-02-15T18:35:00.500 INFO:tasks.workunit.client.0.smithi038.stdout:3/730: read - d2/d1e/d7e/fbc zero size 2024-02-15T18:35:00.500 INFO:tasks.workunit.client.0.smithi038.stdout:3/731: chown d2/d1b/d4f/d92/fde 3461918 1 2024-02-15T18:35:00.501 INFO:tasks.workunit.client.0.smithi038.stdout:0/910: dwrite d4/d13/d2a/d35/db2/fbd [0,4194304] 0 2024-02-15T18:35:00.501 INFO:tasks.workunit.client.0.smithi038.stdout:7/804: symlink d0/db/d59/d15/d29/d8d/l120 0 2024-02-15T18:35:00.502 INFO:tasks.workunit.client.0.smithi038.stdout:7/805: truncate d0/d66/f104 231250 0 2024-02-15T18:35:00.503 INFO:tasks.workunit.client.0.smithi038.stdout:4/823: mknod d11/d1e/d43/d5e/c122 0 2024-02-15T18:35:00.504 INFO:tasks.workunit.client.0.smithi038.stdout:7/806: symlink d0/d66/l121 0 2024-02-15T18:35:00.504 INFO:tasks.workunit.client.0.smithi038.stdout:4/824: rmdir d11/d1e/d20/d31/d8d/de7 39 2024-02-15T18:35:00.505 INFO:tasks.workunit.client.0.smithi038.stdout:7/807: mkdir d0/db/d59/dc8/dd6/d70/d122 0 2024-02-15T18:35:00.505 INFO:tasks.workunit.client.0.smithi038.stdout:7/808: chown d0/db 2 1 2024-02-15T18:35:00.506 INFO:tasks.workunit.client.0.smithi038.stdout:4/825: creat d11/d1e/d20/d31/d8d/d99/f123 x:0 0 0 2024-02-15T18:35:00.506 INFO:tasks.workunit.client.0.smithi038.stdout:4/826: write d11/d1e/d20/d31/d48/fe0 [367301,82310] 0 2024-02-15T18:35:00.507 INFO:tasks.workunit.client.0.smithi038.stdout:7/809: mkdir d0/db/d59/dc8/dd6/d61/d90/dc0/d123 0 2024-02-15T18:35:00.507 INFO:tasks.workunit.client.0.smithi038.stdout:7/810: write d0/db/fb0 [811941,48722] 0 2024-02-15T18:35:00.507 INFO:tasks.workunit.client.0.smithi038.stdout:7/811: fdatasync d0/d1b/f11b 0 2024-02-15T18:35:00.508 INFO:tasks.workunit.client.0.smithi038.stdout:0/911: dread d4/d13/d19/f91 [0,4194304] 0 2024-02-15T18:35:00.508 INFO:tasks.workunit.client.0.smithi038.stdout:4/827: creat d11/d1e/d43/d5e/f124 x:0 0 0 2024-02-15T18:35:00.508 INFO:tasks.workunit.client.0.smithi038.stdout:4/828: fdatasync d11/d1e/d20/d31/d8d/fb2 0 2024-02-15T18:35:00.509 INFO:tasks.workunit.client.0.smithi038.stdout:7/812: rename d0/db/d19/d7d/la7 to d0/d66/l124 0 2024-02-15T18:35:00.509 INFO:tasks.workunit.client.0.smithi038.stdout:7/813: fdatasync d0/d1b/d80/fa9 0 2024-02-15T18:35:00.509 INFO:tasks.workunit.client.0.smithi038.stdout:0/912: write d4/f88 [704102,39366] 0 2024-02-15T18:35:00.510 INFO:tasks.workunit.client.0.smithi038.stdout:4/829: symlink d11/d1e/d20/d31/d48/dac/dbe/daf/l125 0 2024-02-15T18:35:00.510 INFO:tasks.workunit.client.0.smithi038.stdout:9/923: dwrite d1/da/d20/d58/d76/fb5 [0,4194304] 0 2024-02-15T18:35:00.511 INFO:tasks.workunit.client.0.smithi038.stdout:7/814: mknod d0/db/d19/d7d/c125 0 2024-02-15T18:35:00.511 INFO:tasks.workunit.client.0.smithi038.stdout:7/815: write d0/d66/f104 [319116,115315] 0 2024-02-15T18:35:00.512 INFO:tasks.workunit.client.0.smithi038.stdout:6/805: dwrite d2/d4/d2c/d39/d76/dc2/fce [0,4194304] 0 2024-02-15T18:35:00.512 INFO:tasks.workunit.client.0.smithi038.stdout:6/806: truncate d2/d4/d2c/d39/fe0 315327 0 2024-02-15T18:35:00.513 INFO:tasks.workunit.client.0.smithi038.stdout:4/830: mknod d11/d1e/d20/d31/d48/dac/c126 0 2024-02-15T18:35:00.514 INFO:tasks.workunit.client.0.smithi038.stdout:4/831: write d11/f1b [1767245,96879] 0 2024-02-15T18:35:00.514 INFO:tasks.workunit.client.0.smithi038.stdout:6/807: write d2/f3 [3604925,33154] 0 2024-02-15T18:35:00.514 INFO:tasks.workunit.client.0.smithi038.stdout:6/808: fdatasync d2/d4/d19/d78/f9e 0 2024-02-15T18:35:00.514 INFO:tasks.workunit.client.0.smithi038.stdout:9/924: rename f0 to d1/da/d10/d13/d3d/d43/d4d/dfd/f14c 0 2024-02-15T18:35:00.515 INFO:tasks.workunit.client.0.smithi038.stdout:7/816: creat d0/d68/f126 x:0 0 0 2024-02-15T18:35:00.517 INFO:tasks.workunit.client.0.smithi038.stdout:3/732: dread d2/d25/fcc [0,4194304] 0 2024-02-15T18:35:00.522 INFO:tasks.workunit.client.0.smithi038.stdout:4/832: write d11/f100 [1636030,20368] 0 2024-02-15T18:35:00.522 INFO:tasks.workunit.client.0.smithi038.stdout:4/833: dread - d11/d1e/d20/d2b/d4d/d67/d73/f10f zero size 2024-02-15T18:35:00.523 INFO:tasks.workunit.client.0.smithi038.stdout:4/834: write d11/d1e/d20/d2b/d4d/d67/d73/fdb [46231,79311] 0 2024-02-15T18:35:00.523 INFO:tasks.workunit.client.0.smithi038.stdout:9/925: rename d1/da/d20/d58/d94/db9/dc6 to d1/dc/d4f/d14d 0 2024-02-15T18:35:00.527 INFO:tasks.workunit.client.0.smithi038.stdout:7/817: link d0/d66/lb2 d0/d68/dd9/l127 0 2024-02-15T18:35:00.528 INFO:tasks.workunit.client.0.smithi038.stdout:3/733: mkdir d2/db8/dce/de6 0 2024-02-15T18:35:00.528 INFO:tasks.workunit.client.0.smithi038.stdout:7/818: write d0/db/fb1 [1619836,48379] 0 2024-02-15T18:35:00.528 INFO:tasks.workunit.client.0.smithi038.stdout:7/819: write d0/d1b/d80/f6e [719051,60641] 0 2024-02-15T18:35:00.529 INFO:tasks.workunit.client.0.smithi038.stdout:4/835: creat d11/d1e/d20/d31/da2/d118/d7e/d105/f127 x:0 0 0 2024-02-15T18:35:00.530 INFO:tasks.workunit.client.0.smithi038.stdout:9/926: mkdir d1/da/d10/d13/d3d/d43/d4d/d105/d14e 0 2024-02-15T18:35:00.530 INFO:tasks.workunit.client.0.smithi038.stdout:9/927: readlink d1/da/d20/d58/d94/db9/de3/l115 0 2024-02-15T18:35:00.530 INFO:tasks.workunit.client.0.smithi038.stdout:9/928: dread - d1/da/d10/d13/d3d/fa3 zero size 2024-02-15T18:35:00.530 INFO:tasks.workunit.client.0.smithi038.stdout:3/734: mknod d2/d7f/dc1/de1/ce7 0 2024-02-15T18:35:00.532 INFO:tasks.workunit.client.0.smithi038.stdout:7/820: mkdir d0/db/d59/dc8/dd6/d61/d90/dc0/d128 0 2024-02-15T18:35:00.532 INFO:tasks.workunit.client.0.smithi038.stdout:7/821: stat d0/db/d19/l28 0 2024-02-15T18:35:00.532 INFO:tasks.workunit.client.0.smithi038.stdout:9/929: mkdir d1/da/d36/d93/d145/d14f 0 2024-02-15T18:35:00.533 INFO:tasks.workunit.client.0.smithi038.stdout:0/913: dwrite d4/d13/d53/fd5 [0,4194304] 0 2024-02-15T18:35:00.533 INFO:tasks.workunit.client.0.smithi038.stdout:6/809: dwrite d2/d10/fb8 [0,4194304] 0 2024-02-15T18:35:00.533 INFO:tasks.workunit.client.0.smithi038.stdout:7/822: creat d0/db/d19/d7d/daa/dd0/f129 x:0 0 0 2024-02-15T18:35:00.537 INFO:tasks.workunit.client.0.smithi038.stdout:4/836: dread d11/d1e/d20/f6b [0,4194304] 0 2024-02-15T18:35:00.541 INFO:tasks.workunit.client.0.smithi038.stdout:6/810: dread f1 [0,4194304] 0 2024-02-15T18:35:00.548 INFO:tasks.workunit.client.0.smithi038.stdout:9/930: mkdir d1/dc/d4f/d150 0 2024-02-15T18:35:00.549 INFO:tasks.workunit.client.0.smithi038.stdout:0/914: mknod d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/ddb/d120/c135 0 2024-02-15T18:35:00.552 INFO:tasks.workunit.client.0.smithi038.stdout:3/735: dwrite d2/d1b/d4f/d92/fde [0,4194304] 0 2024-02-15T18:35:00.552 INFO:tasks.workunit.client.0.smithi038.stdout:7/823: mkdir d0/db/d59/d15/d29/d7e/d12a 0 2024-02-15T18:35:00.552 INFO:tasks.workunit.client.0.smithi038.stdout:4/837: mkdir d11/d1e/d20/d31/d128 0 2024-02-15T18:35:00.553 INFO:tasks.workunit.client.0.smithi038.stdout:7/824: mkdir d0/d68/dfa/d12b 0 2024-02-15T18:35:00.554 INFO:tasks.workunit.client.0.smithi038.stdout:4/838: creat d11/d1e/d20/d31/d8d/f129 x:0 0 0 2024-02-15T18:35:00.557 INFO:tasks.workunit.client.0.smithi038.stdout:4/839: mknod d11/d1e/d43/c12a 0 2024-02-15T18:35:00.558 INFO:tasks.workunit.client.0.smithi038.stdout:7/825: fsync d0/db/f42 0 2024-02-15T18:35:00.560 INFO:tasks.workunit.client.0.smithi038.stdout:4/840: rename d11/d1e/d20/fd6 to d11/d1e/d20/d31/d48/d60/f12b 0 2024-02-15T18:35:00.562 INFO:tasks.workunit.client.0.smithi038.stdout:6/811: dwrite d2/d4/d2c/d39/f71 [0,4194304] 0 2024-02-15T18:35:00.562 INFO:tasks.workunit.client.0.smithi038.stdout:6/812: write d2/d4/d19/d78/fae [768476,29489] 0 2024-02-15T18:35:00.563 INFO:tasks.workunit.client.0.smithi038.stdout:6/813: write d2/d10/f4e [167617,125993] 0 2024-02-15T18:35:00.564 INFO:tasks.workunit.client.0.smithi038.stdout:7/826: mkdir d0/db/d59/dc8/dd6/d61/d12c 0 2024-02-15T18:35:00.565 INFO:tasks.workunit.client.0.smithi038.stdout:5/777: sync 2024-02-15T18:35:00.565 INFO:tasks.workunit.client.0.smithi038.stdout:1/767: sync 2024-02-15T18:35:00.565 INFO:tasks.workunit.client.0.smithi038.stdout:2/732: sync 2024-02-15T18:35:00.565 INFO:tasks.workunit.client.0.smithi038.stdout:1/768: chown d2/d10/d19/d22/d36/lc8 314771079 1 2024-02-15T18:35:00.568 INFO:tasks.workunit.client.0.smithi038.stdout:4/841: symlink d11/d1e/d43/d5e/l12c 0 2024-02-15T18:35:00.570 INFO:tasks.workunit.client.0.smithi038.stdout:6/814: creat d2/d4/d2c/ff7 x:0 0 0 2024-02-15T18:35:00.572 INFO:tasks.workunit.client.0.smithi038.stdout:7/827: symlink d0/db/d59/d15/d29/d7e/l12d 0 2024-02-15T18:35:00.572 INFO:tasks.workunit.client.0.smithi038.stdout:2/733: creat dd/fed x:0 0 0 2024-02-15T18:35:00.575 INFO:tasks.workunit.client.0.smithi038.stdout:4/842: rename d11/d1e/d20/d2b/d4d/d67/da1/ff7 to d11/d1e/d20/d2b/f12d 0 2024-02-15T18:35:00.575 INFO:tasks.workunit.client.0.smithi038.stdout:4/843: dread - d11/d1e/d20/d2b/d4d/d67/f8e zero size 2024-02-15T18:35:00.575 INFO:tasks.workunit.client.0.smithi038.stdout:4/844: chown d11/d1e/d20/d31/d8d/f129 1 1 2024-02-15T18:35:00.575 INFO:tasks.workunit.client.0.smithi038.stdout:2/734: rmdir dd/d12/d1d/d3b/d3c/d73 39 2024-02-15T18:35:00.579 INFO:tasks.workunit.client.0.smithi038.stdout:2/735: mkdir dd/d12/d59/dce/dea/dee 0 2024-02-15T18:35:00.579 INFO:tasks.workunit.client.0.smithi038.stdout:2/736: chown dd/d12/d3a/d67/fad 13 1 2024-02-15T18:35:00.582 INFO:tasks.workunit.client.0.smithi038.stdout:2/737: mkdir dd/d12/d59/def 0 2024-02-15T18:35:00.594 INFO:tasks.workunit.client.0.smithi038.stdout:0/915: dwrite d4/f63 [0,4194304] 0 2024-02-15T18:35:00.594 INFO:tasks.workunit.client.0.smithi038.stdout:0/916: stat d4/d13/d2a/cd2 0 2024-02-15T18:35:00.594 INFO:tasks.workunit.client.0.smithi038.stdout:0/917: write d4/d13/f39 [9271805,115141] 0 2024-02-15T18:35:00.596 INFO:tasks.workunit.client.0.smithi038.stdout:9/931: dwrite d1/dc/fe [4194304,4194304] 0 2024-02-15T18:35:00.596 INFO:tasks.workunit.client.0.smithi038.stdout:9/932: readlink d1/dc/l23 0 2024-02-15T18:35:00.596 INFO:tasks.workunit.client.0.smithi038.stdout:0/918: mkdir d4/d13/d2a/d35/d1e/d32/d48/d12e/d136 0 2024-02-15T18:35:00.598 INFO:tasks.workunit.client.0.smithi038.stdout:0/919: mkdir d4/d13/d19/d137 0 2024-02-15T18:35:00.598 INFO:tasks.workunit.client.0.smithi038.stdout:0/920: chown d4/d13/d53/c10e 7 1 2024-02-15T18:35:00.612 INFO:tasks.workunit.client.0.smithi038.stdout:3/736: dwrite d2/d24/fb2 [0,4194304] 0 2024-02-15T18:35:00.615 INFO:tasks.workunit.client.0.smithi038.stdout:3/737: rename d2/d1b/d3c/f31 to d2/db8/dce/fe8 0 2024-02-15T18:35:00.616 INFO:tasks.workunit.client.0.smithi038.stdout:3/738: symlink d2/d1b/d3c/d42/le9 0 2024-02-15T18:35:00.632 INFO:tasks.workunit.client.0.smithi038.stdout:1/769: dwrite d2/d10/d19/d22/d36/f44 [0,4194304] 0 2024-02-15T18:35:00.632 INFO:tasks.workunit.client.0.smithi038.stdout:5/778: dwrite d2/dcc/f84 [0,4194304] 0 2024-02-15T18:35:00.632 INFO:tasks.workunit.client.0.smithi038.stdout:6/815: dwrite d2/d4/d2c/f4f [0,4194304] 0 2024-02-15T18:35:00.633 INFO:tasks.workunit.client.0.smithi038.stdout:2/738: dwrite dd/d12/f47 [0,4194304] 0 2024-02-15T18:35:00.634 INFO:tasks.workunit.client.0.smithi038.stdout:9/933: dwrite d1/da/f17 [4194304,4194304] 0 2024-02-15T18:35:00.634 INFO:tasks.workunit.client.0.smithi038.stdout:7/828: dwrite d0/d95/f51 [0,4194304] 0 2024-02-15T18:35:00.634 INFO:tasks.workunit.client.0.smithi038.stdout:0/921: dwrite d4/ff [0,4194304] 0 2024-02-15T18:35:00.637 INFO:tasks.workunit.client.0.smithi038.stdout:3/739: dwrite d2/fa [0,4194304] 0 2024-02-15T18:35:00.639 INFO:tasks.workunit.client.0.smithi038.stdout:6/816: dread d2/f8e [0,4194304] 0 2024-02-15T18:35:00.639 INFO:tasks.workunit.client.0.smithi038.stdout:6/817: getdents d2/d10/dd0 0 2024-02-15T18:35:00.641 INFO:tasks.workunit.client.0.smithi038.stdout:5/779: creat d2/dcc/fea x:0 0 0 2024-02-15T18:35:00.642 INFO:tasks.workunit.client.0.smithi038.stdout:9/934: truncate d1/f124 710643 0 2024-02-15T18:35:00.647 INFO:tasks.workunit.client.0.smithi038.stdout:5/780: write d2/d3/d12/d89/fc9 [316799,3895] 0 2024-02-15T18:35:00.647 INFO:tasks.workunit.client.0.smithi038.stdout:5/781: fsync d2/d3/d12/f14 0 2024-02-15T18:35:00.647 INFO:tasks.workunit.client.0.smithi038.stdout:1/770: dread f1 [0,4194304] 0 2024-02-15T18:35:00.648 INFO:tasks.workunit.client.0.smithi038.stdout:0/922: creat d4/d13/d19/d137/f138 x:0 0 0 2024-02-15T18:35:00.649 INFO:tasks.workunit.client.0.smithi038.stdout:3/740: mknod d2/d1b/d69/cea 0 2024-02-15T18:35:00.649 INFO:tasks.workunit.client.0.smithi038.stdout:3/741: fsync d2/d24/f49 0 2024-02-15T18:35:00.650 INFO:tasks.workunit.client.0.smithi038.stdout:6/818: getdents d2/d4/d2c 0 2024-02-15T18:35:00.652 INFO:tasks.workunit.client.0.smithi038.stdout:9/935: truncate d1/da/f54 2979027 0 2024-02-15T18:35:00.652 INFO:tasks.workunit.client.0.smithi038.stdout:9/936: chown d1/dc/db2 28372327 1 2024-02-15T18:35:00.652 INFO:tasks.workunit.client.0.smithi038.stdout:9/937: write d1/da/d20/d58/d94/db9/de3/fe6 [390592,48633] 0 2024-02-15T18:35:00.653 INFO:tasks.workunit.client.0.smithi038.stdout:5/782: truncate d2/f6d 1549618 0 2024-02-15T18:35:00.653 INFO:tasks.workunit.client.0.smithi038.stdout:5/783: readlink d2/d3/d12/d5e/l91 0 2024-02-15T18:35:00.654 INFO:tasks.workunit.client.0.smithi038.stdout:5/784: dread - d2/d3/d12/d5e/fbb zero size 2024-02-15T18:35:00.654 INFO:tasks.workunit.client.0.smithi038.stdout:7/829: dwrite d0/d1b/d80/fa9 [0,4194304] 0 2024-02-15T18:35:00.654 INFO:tasks.workunit.client.0.smithi038.stdout:7/830: fdatasync d0/d1b/d80/f6e 0 2024-02-15T18:35:00.654 INFO:tasks.workunit.client.0.smithi038.stdout:0/923: truncate d4/d13/d2a/d35/d1e/f20 3988076 0 2024-02-15T18:35:00.654 INFO:tasks.workunit.client.0.smithi038.stdout:5/785: write d2/d30/f55 [4215642,103820] 0 2024-02-15T18:35:00.655 INFO:tasks.workunit.client.0.smithi038.stdout:3/742: mknod d2/d25/d60/ceb 0 2024-02-15T18:35:00.655 INFO:tasks.workunit.client.0.smithi038.stdout:6/819: dread d2/d4/d19/d55/fde [0,4194304] 0 2024-02-15T18:35:00.658 INFO:tasks.workunit.client.0.smithi038.stdout:7/831: link d0/db/d59/d15/d29/la5 d0/db/d59/d15/d29/d8d/l12e 0 2024-02-15T18:35:00.658 INFO:tasks.workunit.client.0.smithi038.stdout:7/832: write d0/db/d59/dc8/dd6/d61/ff1 [478139,47966] 0 2024-02-15T18:35:00.658 INFO:tasks.workunit.client.0.smithi038.stdout:7/833: truncate d0/db/d19/d7d/daa/dd0/f129 748200 0 2024-02-15T18:35:00.660 INFO:tasks.workunit.client.0.smithi038.stdout:5/786: link d2/d3/d12/cb d2/d3/da0/d33/ceb 0 2024-02-15T18:35:00.661 INFO:tasks.workunit.client.0.smithi038.stdout:3/743: creat d2/d7f/dc1/de1/fec x:0 0 0 2024-02-15T18:35:00.665 INFO:tasks.workunit.client.0.smithi038.stdout:6/820: mknod d2/d10/dd0/cf8 0 2024-02-15T18:35:00.665 INFO:tasks.workunit.client.0.smithi038.stdout:6/821: write d2/d10/f96 [1638003,65163] 0 2024-02-15T18:35:00.665 INFO:tasks.workunit.client.0.smithi038.stdout:6/822: fsync d2/d10/f49 0 2024-02-15T18:35:00.665 INFO:tasks.workunit.client.0.smithi038.stdout:6/823: write d2/d4/d6c/fcb [145786,65308] 0 2024-02-15T18:35:00.665 INFO:tasks.workunit.client.0.smithi038.stdout:6/824: fdatasync d2/d4/d2c/d39/db4/d5f/fa1 0 2024-02-15T18:35:00.671 INFO:tasks.workunit.client.0.smithi038.stdout:3/744: truncate d2/d24/f49 3037195 0 2024-02-15T18:35:00.672 INFO:tasks.workunit.client.0.smithi038.stdout:6/825: creat d2/d4/d19/d55/dc0/ff9 x:0 0 0 2024-02-15T18:35:00.673 INFO:tasks.workunit.client.0.smithi038.stdout:6/826: read d2/d4/d19/d78/dbf/fe5 [357515,118089] 0 2024-02-15T18:35:00.673 INFO:tasks.workunit.client.0.smithi038.stdout:6/827: chown d2/d4/d2c/d39/l3e 144 1 2024-02-15T18:35:00.673 INFO:tasks.workunit.client.0.smithi038.stdout:6/828: stat d2/d10/dd0/cf8 0 2024-02-15T18:35:00.674 INFO:tasks.workunit.client.0.smithi038.stdout:7/834: mkdir d0/d12f 0 2024-02-15T18:35:00.674 INFO:tasks.workunit.client.0.smithi038.stdout:0/924: write d4/d13/d2a/d35/f8b [778286,57303] 0 2024-02-15T18:35:00.674 INFO:tasks.workunit.client.0.smithi038.stdout:3/745: link d2/l1c d2/d1b/led 0 2024-02-15T18:35:00.678 INFO:tasks.workunit.client.0.smithi038.stdout:6/829: unlink d2/d4/d19/ce3 0 2024-02-15T18:35:00.678 INFO:tasks.workunit.client.0.smithi038.stdout:1/771: dwrite d2/d69/da9/dad/de2/fea [0,4194304] 0 2024-02-15T18:35:00.679 INFO:tasks.workunit.client.0.smithi038.stdout:7/835: creat d0/db/d19/d7d/daa/dd0/ddf/f130 x:0 0 0 2024-02-15T18:35:00.679 INFO:tasks.workunit.client.0.smithi038.stdout:7/836: dread - d0/db/d59/d97/f105 zero size 2024-02-15T18:35:00.679 INFO:tasks.workunit.client.0.smithi038.stdout:7/837: chown d0/d68/f126 426597 1 2024-02-15T18:35:00.680 INFO:tasks.workunit.client.0.smithi038.stdout:5/787: dwrite d2/d3/dd0/fda [0,4194304] 0 2024-02-15T18:35:00.681 INFO:tasks.workunit.client.0.smithi038.stdout:5/788: write d2/fc [3697250,53542] 0 2024-02-15T18:35:00.681 INFO:tasks.workunit.client.0.smithi038.stdout:5/789: fsync d2/d3/d12/fa 0 2024-02-15T18:35:00.681 INFO:tasks.workunit.client.0.smithi038.stdout:9/938: dwrite d1/da/d10/d13/f89 [0,4194304] 0 2024-02-15T18:35:00.683 INFO:tasks.workunit.client.0.smithi038.stdout:3/746: link d2/d25/d60/c83 d2/db8/dce/cee 0 2024-02-15T18:35:00.686 INFO:tasks.workunit.client.0.smithi038.stdout:5/790: symlink d2/d48/d83/lec 0 2024-02-15T18:35:00.687 INFO:tasks.workunit.client.0.smithi038.stdout:7/838: write d0/f54 [1910583,128135] 0 2024-02-15T18:35:00.687 INFO:tasks.workunit.client.0.smithi038.stdout:7/839: write d0/f48 [2079130,62235] 0 2024-02-15T18:35:00.691 INFO:tasks.workunit.client.0.smithi038.stdout:6/830: dread d2/f46 [4194304,4194304] 0 2024-02-15T18:35:00.693 INFO:tasks.workunit.client.0.smithi038.stdout:6/831: write d2/d4/d2c/d39/db4/f8d [8064815,130169] 0 2024-02-15T18:35:00.695 INFO:tasks.workunit.client.0.smithi038.stdout:1/772: write d2/d10/d19/d5f/fb9 [332973,90616] 0 2024-02-15T18:35:00.696 INFO:tasks.workunit.client.0.smithi038.stdout:9/939: mkdir d1/da/d10/d13/d3d/d43/d151 0 2024-02-15T18:35:00.696 INFO:tasks.workunit.client.0.smithi038.stdout:3/747: creat d2/d7f/dc1/fef x:0 0 0 2024-02-15T18:35:00.696 INFO:tasks.workunit.client.0.smithi038.stdout:3/748: write d2/d1b/d3c/d41/d4e/fa3 [674736,34550] 0 2024-02-15T18:35:00.696 INFO:tasks.workunit.client.0.smithi038.stdout:3/749: chown d2/d1b/d3c/cb7 31344388 1 2024-02-15T18:35:00.697 INFO:tasks.workunit.client.0.smithi038.stdout:9/940: write d1/da/d10/d13/f18 [1976985,42959] 0 2024-02-15T18:35:00.698 INFO:tasks.workunit.client.0.smithi038.stdout:0/925: dwrite d4/d13/d2a/d35/d6a/fde [0,4194304] 0 2024-02-15T18:35:00.701 INFO:tasks.workunit.client.0.smithi038.stdout:7/840: symlink d0/d66/l131 0 2024-02-15T18:35:00.701 INFO:tasks.workunit.client.0.smithi038.stdout:7/841: write d0/d1b/d80/f6e [1694221,72211] 0 2024-02-15T18:35:00.702 INFO:tasks.workunit.client.0.smithi038.stdout:6/832: creat d2/d10/ffa x:0 0 0 2024-02-15T18:35:00.703 INFO:tasks.workunit.client.0.smithi038.stdout:1/773: symlink d2/d69/da9/lfe 0 2024-02-15T18:35:00.703 INFO:tasks.workunit.client.0.smithi038.stdout:1/774: readlink d2/d10/d19/d22/l33 0 2024-02-15T18:35:00.704 INFO:tasks.workunit.client.0.smithi038.stdout:3/750: write d2/d24/f49 [2365640,75270] 0 2024-02-15T18:35:00.705 INFO:tasks.workunit.client.0.smithi038.stdout:5/791: dwrite d2/d3/d12/f14 [0,4194304] 0 2024-02-15T18:35:00.713 INFO:tasks.workunit.client.0.smithi038.stdout:1/775: dread d2/d10/d32/d53/f7b [0,4194304] 0 2024-02-15T18:35:00.713 INFO:tasks.workunit.client.0.smithi038.stdout:1/776: chown d2/d10/d19/d22/d54/dc3 4019 1 2024-02-15T18:35:00.715 INFO:tasks.workunit.client.0.smithi038.stdout:9/941: rmdir d1 39 2024-02-15T18:35:00.719 INFO:tasks.workunit.client.0.smithi038.stdout:5/792: dwrite d2/d3/da0/d33/fc8 [0,4194304] 0 2024-02-15T18:35:00.722 INFO:tasks.workunit.client.0.smithi038.stdout:7/842: truncate d0/d68/fae 2729404 0 2024-02-15T18:35:00.722 INFO:tasks.workunit.client.0.smithi038.stdout:6/833: rmdir d2/d4/d6c/d9d 39 2024-02-15T18:35:00.724 INFO:tasks.workunit.client.0.smithi038.stdout:3/751: mkdir d2/d7f/df0 0 2024-02-15T18:35:00.724 INFO:tasks.workunit.client.0.smithi038.stdout:3/752: stat d2/d1e/d7e/d7c 0 2024-02-15T18:35:00.724 INFO:tasks.workunit.client.0.smithi038.stdout:1/777: mknod d2/cff 0 2024-02-15T18:35:00.724 INFO:tasks.workunit.client.0.smithi038.stdout:1/778: dread - d2/d10/fa3 zero size 2024-02-15T18:35:00.725 INFO:tasks.workunit.client.0.smithi038.stdout:9/942: dread d1/da/d20/d58/d76/ff3 [0,4194304] 0 2024-02-15T18:35:00.728 INFO:tasks.workunit.client.0.smithi038.stdout:5/793: rename d2/f6d to d2/d3/d12/fed 0 2024-02-15T18:35:00.728 INFO:tasks.workunit.client.0.smithi038.stdout:0/926: mknod d4/c139 0 2024-02-15T18:35:00.729 INFO:tasks.workunit.client.0.smithi038.stdout:0/927: fdatasync d4/db3/fe2 0 2024-02-15T18:35:00.729 INFO:tasks.workunit.client.0.smithi038.stdout:0/928: chown d4/d13/d2a/d35/d1e/d32/d79/d7d 255 1 2024-02-15T18:35:00.729 INFO:tasks.workunit.client.0.smithi038.stdout:6/834: creat d2/d4/d6c/ffb x:0 0 0 2024-02-15T18:35:00.730 INFO:tasks.workunit.client.0.smithi038.stdout:3/753: creat d2/d1b/d3c/d41/dd5/ff1 x:0 0 0 2024-02-15T18:35:00.730 INFO:tasks.workunit.client.0.smithi038.stdout:3/754: fsync d2/d1b/d3c/d42/fbd 0 2024-02-15T18:35:00.731 INFO:tasks.workunit.client.0.smithi038.stdout:1/779: unlink d2/f73 0 2024-02-15T18:35:00.732 INFO:tasks.workunit.client.0.smithi038.stdout:1/780: write d2/d10/d19/d2f/fd6 [760622,123366] 0 2024-02-15T18:35:00.733 INFO:tasks.workunit.client.0.smithi038.stdout:9/943: symlink d1/dc/db2/d13a/l152 0 2024-02-15T18:35:00.733 INFO:tasks.workunit.client.0.smithi038.stdout:5/794: creat d2/d30/d57/fee x:0 0 0 2024-02-15T18:35:00.734 INFO:tasks.workunit.client.0.smithi038.stdout:4/845: sync 2024-02-15T18:35:00.734 INFO:tasks.workunit.client.0.smithi038.stdout:4/846: chown d11/d1e/d20/d31/d8d/d99/db5 4471 1 2024-02-15T18:35:00.734 INFO:tasks.workunit.client.0.smithi038.stdout:1/781: mknod d2/d69/da9/dbc/c100 0 2024-02-15T18:35:00.735 INFO:tasks.workunit.client.0.smithi038.stdout:9/944: truncate d1/dc/d69/d7d/f131 292198 0 2024-02-15T18:35:00.737 INFO:tasks.workunit.client.0.smithi038.stdout:5/795: creat d2/d3/d5f/fef x:0 0 0 2024-02-15T18:35:00.742 INFO:tasks.workunit.client.0.smithi038.stdout:1/782: creat d2/d10/d32/d53/d82/ded/f101 x:0 0 0 2024-02-15T18:35:00.743 INFO:tasks.workunit.client.0.smithi038.stdout:1/783: rmdir d2/d10/d19/d22/d54/d80 39 2024-02-15T18:35:00.744 INFO:tasks.workunit.client.0.smithi038.stdout:4/847: truncate d11/d1e/d20/d2b/d65/f6d 1949118 0 2024-02-15T18:35:00.744 INFO:tasks.workunit.client.0.smithi038.stdout:3/755: dread d2/db8/dce/fe8 [0,4194304] 0 2024-02-15T18:35:00.746 INFO:tasks.workunit.client.0.smithi038.stdout:1/784: mknod d2/d2e/c102 0 2024-02-15T18:35:00.747 INFO:tasks.workunit.client.0.smithi038.stdout:4/848: creat d11/d1e/d20/d31/d8d/f12e x:0 0 0 2024-02-15T18:35:00.747 INFO:tasks.workunit.client.0.smithi038.stdout:4/849: dread - d11/d1e/d43/d5e/f124 zero size 2024-02-15T18:35:00.747 INFO:tasks.workunit.client.0.smithi038.stdout:3/756: unlink d2/d1b/d3c/d41/laa 0 2024-02-15T18:35:00.747 INFO:tasks.workunit.client.0.smithi038.stdout:3/757: write d2/d1b/d3c/d3a/d55/da0/fd2 [786824,54924] 0 2024-02-15T18:35:00.748 INFO:tasks.workunit.client.0.smithi038.stdout:3/758: truncate d2/fb6 448773 0 2024-02-15T18:35:00.748 INFO:tasks.workunit.client.0.smithi038.stdout:1/785: symlink d2/d69/l103 0 2024-02-15T18:35:00.750 INFO:tasks.workunit.client.0.smithi038.stdout:4/850: creat d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/ddd/dba/f12f x:0 0 0 2024-02-15T18:35:00.750 INFO:tasks.workunit.client.0.smithi038.stdout:3/759: creat d2/d7f/dc1/de1/ff2 x:0 0 0 2024-02-15T18:35:00.751 INFO:tasks.workunit.client.0.smithi038.stdout:7/843: dwrite d0/d1b/d30/d50/f74 [0,4194304] 0 2024-02-15T18:35:00.751 INFO:tasks.workunit.client.0.smithi038.stdout:7/844: truncate d0/db/d59/dc8/dd6/d61/fe2 37116 0 2024-02-15T18:35:00.751 INFO:tasks.workunit.client.0.smithi038.stdout:1/786: rmdir d2/d10/d19/d22/d54/d80 39 2024-02-15T18:35:00.757 INFO:tasks.workunit.client.0.smithi038.stdout:4/851: rename d11/d1e/d20/d31/d48/d102/dca/le4 to d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/d11d/l130 0 2024-02-15T18:35:00.757 INFO:tasks.workunit.client.0.smithi038.stdout:4/852: fdatasync d11/d1e/d43/d4a/f5f 0 2024-02-15T18:35:00.757 INFO:tasks.workunit.client.0.smithi038.stdout:4/853: dread - d11/d1e/d20/d31/d8d/f12e zero size 2024-02-15T18:35:00.757 INFO:tasks.workunit.client.0.smithi038.stdout:0/929: dwrite d4/d13/d19/f5c [0,4194304] 0 2024-02-15T18:35:00.758 INFO:tasks.workunit.client.0.smithi038.stdout:3/760: mknod d2/d1b/d3c/d3a/d55/d65/cf3 0 2024-02-15T18:35:00.761 INFO:tasks.workunit.client.0.smithi038.stdout:4/854: rmdir d11/d1e/d43/d4a/dbd 39 2024-02-15T18:35:00.762 INFO:tasks.workunit.client.0.smithi038.stdout:0/930: mkdir d4/d13/d2a/d35/d1e/d32/d79/d7d/d13a 0 2024-02-15T18:35:00.764 INFO:tasks.workunit.client.0.smithi038.stdout:3/761: mkdir d2/d1b/d3c/d41/d4e/df4 0 2024-02-15T18:35:00.764 INFO:tasks.workunit.client.0.smithi038.stdout:3/762: chown d2/d1e/de3 2 1 2024-02-15T18:35:00.764 INFO:tasks.workunit.client.0.smithi038.stdout:3/763: chown d2/d24/db9/ccd 2045766 1 2024-02-15T18:35:00.764 INFO:tasks.workunit.client.0.smithi038.stdout:3/764: dread - d2/d1e/fd9 zero size 2024-02-15T18:35:00.765 INFO:tasks.workunit.client.0.smithi038.stdout:6/835: dwrite d2/d4/d19/d55/d5c/f70 [0,4194304] 0 2024-02-15T18:35:00.765 INFO:tasks.workunit.client.0.smithi038.stdout:6/836: fsync d2/f9 0 2024-02-15T18:35:00.765 INFO:tasks.workunit.client.0.smithi038.stdout:0/931: creat d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/d11a/f13b x:0 0 0 2024-02-15T18:35:00.767 INFO:tasks.workunit.client.0.smithi038.stdout:3/765: link d2/d1e/l47 d2/d1b/d3c/d41/d4e/df4/lf5 0 2024-02-15T18:35:00.770 INFO:tasks.workunit.client.0.smithi038.stdout:3/766: dread d2/d1b/f36 [0,4194304] 0 2024-02-15T18:35:00.770 INFO:tasks.workunit.client.0.smithi038.stdout:9/945: dwrite d1/da/d10/d82/fc5 [0,4194304] 0 2024-02-15T18:35:00.771 INFO:tasks.workunit.client.0.smithi038.stdout:3/767: readlink d2/d1e/l86 0 2024-02-15T18:35:00.771 INFO:tasks.workunit.client.0.smithi038.stdout:3/768: write d2/d1b/d3c/fd4 [76848,97265] 0 2024-02-15T18:35:00.771 INFO:tasks.workunit.client.0.smithi038.stdout:3/769: chown d2/d1b/d69/cba 14630 1 2024-02-15T18:35:00.771 INFO:tasks.workunit.client.0.smithi038.stdout:3/770: fdatasync d2/d1b/d3c/d41/d4e/f73 0 2024-02-15T18:35:00.771 INFO:tasks.workunit.client.0.smithi038.stdout:3/771: readlink d2/d1b/l21 0 2024-02-15T18:35:00.773 INFO:tasks.workunit.client.0.smithi038.stdout:4/855: write d11/d1e/d20/d31/d8d/fb2 [1296832,33058] 0 2024-02-15T18:35:00.774 INFO:tasks.workunit.client.0.smithi038.stdout:4/856: chown d11/d1e/d20/d31/d48/dac/dbe/cc2 503685369 1 2024-02-15T18:35:00.774 INFO:tasks.workunit.client.0.smithi038.stdout:7/845: dwrite d0/d95/f51 [0,4194304] 0 2024-02-15T18:35:00.774 INFO:tasks.workunit.client.0.smithi038.stdout:6/837: mkdir d2/d4/d2c/dfc 0 2024-02-15T18:35:00.775 INFO:tasks.workunit.client.0.smithi038.stdout:0/932: truncate d4/d13/d2a/d35/d1e/d32/d3b/daa/df8/fa0 1050845 0 2024-02-15T18:35:00.778 INFO:tasks.workunit.client.0.smithi038.stdout:1/787: dwrite d2/d10/d19/d22/d54/dc0/d6b/ff4 [0,4194304] 0 2024-02-15T18:35:00.778 INFO:tasks.workunit.client.0.smithi038.stdout:9/946: creat d1/dc/d4f/ddf/f153 x:0 0 0 2024-02-15T18:35:00.783 INFO:tasks.workunit.client.0.smithi038.stdout:4/857: rename d11/d1e/d20/d2b/d4d/d67/d73/fdb to d11/d1e/f131 0 2024-02-15T18:35:00.792 INFO:tasks.workunit.client.0.smithi038.stdout:2/739: sync 2024-02-15T18:35:00.795 INFO:tasks.workunit.client.0.smithi038.stdout:4/858: dread d11/d1e/d20/d2f/f3d [0,4194304] 0 2024-02-15T18:35:00.795 INFO:tasks.workunit.client.0.smithi038.stdout:4/859: read - d11/d1e/d20/d31/da2/d118/d7e/d105/f110 zero size 2024-02-15T18:35:00.796 INFO:tasks.workunit.client.0.smithi038.stdout:7/846: getdents d0/d68/dfa 0 2024-02-15T18:35:00.797 INFO:tasks.workunit.client.0.smithi038.stdout:6/838: creat d2/d4/d2c/d39/db4/d5f/ffd x:0 0 0 2024-02-15T18:35:00.799 INFO:tasks.workunit.client.0.smithi038.stdout:9/947: mkdir d1/dc/db2/d154 0 2024-02-15T18:35:00.800 INFO:tasks.workunit.client.0.smithi038.stdout:2/740: symlink dd/d12/d1d/d3b/lf0 0 2024-02-15T18:35:00.801 INFO:tasks.workunit.client.0.smithi038.stdout:2/741: chown dd/d12/d59/db2/fe3 11039940 1 2024-02-15T18:35:00.801 INFO:tasks.workunit.client.0.smithi038.stdout:0/933: dwrite d4/d13/d19/f82 [0,4194304] 0 2024-02-15T18:35:00.802 INFO:tasks.workunit.client.0.smithi038.stdout:4/860: mkdir d11/d1e/d20/d31/d48/dac/dbe/daf/d132 0 2024-02-15T18:35:00.802 INFO:tasks.workunit.client.0.smithi038.stdout:2/742: write dd/d12/d59/f21 [1880991,18596] 0 2024-02-15T18:35:00.802 INFO:tasks.workunit.client.0.smithi038.stdout:2/743: fsync dd/d12/d5d/d54/f8a 0 2024-02-15T18:35:00.803 INFO:tasks.workunit.client.0.smithi038.stdout:3/772: dwrite d2/d1b/d3c/d3a/d55/da0/fd2 [0,4194304] 0 2024-02-15T18:35:00.803 INFO:tasks.workunit.client.0.smithi038.stdout:3/773: chown d2/d1e/l3f 654215330 1 2024-02-15T18:35:00.804 INFO:tasks.workunit.client.0.smithi038.stdout:6/839: creat d2/d10/db6/ffe x:0 0 0 2024-02-15T18:35:00.805 INFO:tasks.workunit.client.0.smithi038.stdout:1/788: mkdir d2/d10/d19/d2f/d3e/d104 0 2024-02-15T18:35:00.807 INFO:tasks.workunit.client.0.smithi038.stdout:9/948: mkdir d1/dc/db2/d13a/d155 0 2024-02-15T18:35:00.807 INFO:tasks.workunit.client.0.smithi038.stdout:9/949: readlink d1/dc/d4f/d14d/lc9 0 2024-02-15T18:35:00.810 INFO:tasks.workunit.client.0.smithi038.stdout:0/934: unlink d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/df4/l11d 0 2024-02-15T18:35:00.810 INFO:tasks.workunit.client.0.smithi038.stdout:4/861: stat d11/d1e/d20/d31/d48/d60/la9 0 2024-02-15T18:35:00.812 INFO:tasks.workunit.client.0.smithi038.stdout:2/744: mkdir dd/d12/d3a/d5e/d65/dcd/df1 0 2024-02-15T18:35:00.812 INFO:tasks.workunit.client.0.smithi038.stdout:2/745: dread - dd/d12/d59/d2b/d7a/fe7 zero size 2024-02-15T18:35:00.813 INFO:tasks.workunit.client.0.smithi038.stdout:7/847: dwrite d0/db/d59/f7 [0,4194304] 0 2024-02-15T18:35:00.813 INFO:tasks.workunit.client.0.smithi038.stdout:7/848: write d0/f48 [2885815,25941] 0 2024-02-15T18:35:00.815 INFO:tasks.workunit.client.0.smithi038.stdout:3/774: rename d2/d1e/c52 to d2/d7f/dc1/de1/cf6 0 2024-02-15T18:35:00.815 INFO:tasks.workunit.client.0.smithi038.stdout:3/775: dread - d2/d1e/fd1 zero size 2024-02-15T18:35:00.816 INFO:tasks.workunit.client.0.smithi038.stdout:9/950: rmdir d1/da/d10/d13/d3d/d43/d4d/d8a 39 2024-02-15T18:35:00.816 INFO:tasks.workunit.client.0.smithi038.stdout:9/951: fsync d1/da/d20/d58/d66/f116 0 2024-02-15T18:35:00.816 INFO:tasks.workunit.client.0.smithi038.stdout:9/952: fsync d1/dc/d4f/d14d/d49/f4e 0 2024-02-15T18:35:00.816 INFO:tasks.workunit.client.0.smithi038.stdout:4/862: mkdir d11/d1e/d20/d2b/d65/d133 0 2024-02-15T18:35:00.817 INFO:tasks.workunit.client.0.smithi038.stdout:4/863: chown d11/d1e/d20/l58 3425 1 2024-02-15T18:35:00.818 INFO:tasks.workunit.client.0.smithi038.stdout:2/746: symlink dd/d12/d5d/d54/dcb/lf2 0 2024-02-15T18:35:00.823 INFO:tasks.workunit.client.0.smithi038.stdout:7/849: mkdir d0/d1b/dfd/d132 0 2024-02-15T18:35:00.823 INFO:tasks.workunit.client.0.smithi038.stdout:7/850: chown d0/db/d19/d7d/daa/dd0/ddf 2 1 2024-02-15T18:35:00.824 INFO:tasks.workunit.client.0.smithi038.stdout:1/789: rename d2/d69/da9/dbc/ddd to d2/d10/d19/d105 0 2024-02-15T18:35:00.825 INFO:tasks.workunit.client.0.smithi038.stdout:3/776: rmdir d2/d1b/d3c/d3a/d55/da0 39 2024-02-15T18:35:00.827 INFO:tasks.workunit.client.0.smithi038.stdout:7/851: creat d0/d68/f133 x:0 0 0 2024-02-15T18:35:00.831 INFO:tasks.workunit.client.0.smithi038.stdout:6/840: dwrite d2/f3 [0,4194304] 0 2024-02-15T18:35:00.833 INFO:tasks.workunit.client.0.smithi038.stdout:4/864: dwrite d11/f1d [0,4194304] 0 2024-02-15T18:35:00.834 INFO:tasks.workunit.client.0.smithi038.stdout:9/953: dread d1/fb [4194304,4194304] 0 2024-02-15T18:35:00.835 INFO:tasks.workunit.client.0.smithi038.stdout:2/747: dread dd/d12/d1d/d3b/d3c/f48 [0,4194304] 0 2024-02-15T18:35:00.835 INFO:tasks.workunit.client.0.smithi038.stdout:2/748: write dd/d12/d5d/d54/f97 [5522000,96015] 0 2024-02-15T18:35:00.839 INFO:tasks.workunit.client.0.smithi038.stdout:3/777: mknod d2/d1b/d69/ddb/cf7 0 2024-02-15T18:35:00.840 INFO:tasks.workunit.client.0.smithi038.stdout:1/790: rename d2/d10/d19/d22/d36/d94/dc1/cd1 to d2/d10/d62/c106 0 2024-02-15T18:35:00.840 INFO:tasks.workunit.client.0.smithi038.stdout:5/796: sync 2024-02-15T18:35:00.840 INFO:tasks.workunit.client.0.smithi038.stdout:5/797: chown d2/d30/d57/f8d 22364576 1 2024-02-15T18:35:00.840 INFO:tasks.workunit.client.0.smithi038.stdout:6/841: creat d2/d4/d2c/d39/db4/d5f/fff x:0 0 0 2024-02-15T18:35:00.840 INFO:tasks.workunit.client.0.smithi038.stdout:6/842: stat d2/d10/db6/fa5 0 2024-02-15T18:35:00.842 INFO:tasks.workunit.client.0.smithi038.stdout:5/798: creat d2/ff0 x:0 0 0 2024-02-15T18:35:00.843 INFO:tasks.workunit.client.0.smithi038.stdout:6/843: creat d2/d4/d19/d78/dbf/f100 x:0 0 0 2024-02-15T18:35:00.845 INFO:tasks.workunit.client.0.smithi038.stdout:5/799: creat d2/d3/ff1 x:0 0 0 2024-02-15T18:35:00.845 INFO:tasks.workunit.client.0.smithi038.stdout:5/800: chown d2/dcc/f70 683511 1 2024-02-15T18:35:00.845 INFO:tasks.workunit.client.0.smithi038.stdout:5/801: readlink d2/d64/lca 0 2024-02-15T18:35:00.846 INFO:tasks.workunit.client.0.smithi038.stdout:6/844: rmdir d2/d4/d19/d55/d5c/dc8 39 2024-02-15T18:35:00.846 INFO:tasks.workunit.client.0.smithi038.stdout:6/845: readlink d2/leb 0 2024-02-15T18:35:00.846 INFO:tasks.workunit.client.0.smithi038.stdout:7/852: dwrite d0/db/ff [0,4194304] 0 2024-02-15T18:35:00.846 INFO:tasks.workunit.client.0.smithi038.stdout:7/853: readlink d0/db/d59/d15/le0 0 2024-02-15T18:35:00.848 INFO:tasks.workunit.client.0.smithi038.stdout:6/846: creat d2/d4/d2c/d39/de9/f101 x:0 0 0 2024-02-15T18:35:00.848 INFO:tasks.workunit.client.0.smithi038.stdout:6/847: fdatasync d2/d4/d19/f74 0 2024-02-15T18:35:00.850 INFO:tasks.workunit.client.0.smithi038.stdout:7/854: rmdir d0/d1b/d80 39 2024-02-15T18:35:00.851 INFO:tasks.workunit.client.0.smithi038.stdout:5/802: write d2/d3/d12/d89/fc9 [561842,65229] 0 2024-02-15T18:35:00.851 INFO:tasks.workunit.client.0.smithi038.stdout:5/803: chown d2/d3/db9/dcf 0 1 2024-02-15T18:35:00.851 INFO:tasks.workunit.client.0.smithi038.stdout:6/848: creat d2/d4/d2c/d39/db4/d5f/f102 x:0 0 0 2024-02-15T18:35:00.852 INFO:tasks.workunit.client.0.smithi038.stdout:7/855: mkdir d0/db/d59/d15/d29/d8d/dcd/d134 0 2024-02-15T18:35:00.853 INFO:tasks.workunit.client.0.smithi038.stdout:5/804: creat d2/d3/d12/d5e/ff2 x:0 0 0 2024-02-15T18:35:00.857 INFO:tasks.workunit.client.0.smithi038.stdout:6/849: symlink d2/d4/d19/d78/dbf/l103 0 2024-02-15T18:35:00.857 INFO:tasks.workunit.client.0.smithi038.stdout:7/856: mknod d0/db/d19/c135 0 2024-02-15T18:35:00.857 INFO:tasks.workunit.client.0.smithi038.stdout:5/805: mknod d2/d3/db9/dcf/ddf/cf3 0 2024-02-15T18:35:00.859 INFO:tasks.workunit.client.0.smithi038.stdout:3/778: dread d2/d1b/f94 [0,4194304] 0 2024-02-15T18:35:00.859 INFO:tasks.workunit.client.0.smithi038.stdout:7/857: creat d0/db/d59/dc8/dd6/d61/d90/dc0/d128/f136 x:0 0 0 2024-02-15T18:35:00.860 INFO:tasks.workunit.client.0.smithi038.stdout:6/850: rename d2/d4/d19/d55/f6a to d2/d4/d19/d55/f104 0 2024-02-15T18:35:00.861 INFO:tasks.workunit.client.0.smithi038.stdout:5/806: rmdir d2/d3/da0/d33 39 2024-02-15T18:35:00.862 INFO:tasks.workunit.client.0.smithi038.stdout:7/858: mkdir d0/d1b/dfd/d137 0 2024-02-15T18:35:00.863 INFO:tasks.workunit.client.0.smithi038.stdout:6/851: truncate d2/d4/d2c/d39/db4/faf 588891 0 2024-02-15T18:35:00.863 INFO:tasks.workunit.client.0.smithi038.stdout:7/859: mknod d0/d66/c138 0 2024-02-15T18:35:00.865 INFO:tasks.workunit.client.0.smithi038.stdout:6/852: unlink d2/d4/d2c/d39/d76/dc2/cf4 0 2024-02-15T18:35:00.890 INFO:tasks.workunit.client.0.smithi038.stdout:7/860: write d0/d1b/d30/f43 [4046259,65776] 0 2024-02-15T18:35:00.898 INFO:tasks.workunit.client.0.smithi038.stdout:9/954: dwrite d1/da/d20/d58/d66/f116 [0,4194304] 0 2024-02-15T18:35:00.898 INFO:tasks.workunit.client.0.smithi038.stdout:2/749: dwrite dd/d12/d1d/d3b/d3c/d4a/f8c [0,4194304] 0 2024-02-15T18:35:00.898 INFO:tasks.workunit.client.0.smithi038.stdout:2/750: fsync dd/d12/d3a/f72 0 2024-02-15T18:35:00.901 INFO:tasks.workunit.client.0.smithi038.stdout:1/791: dwrite d2/d39/f7f [0,4194304] 0 2024-02-15T18:35:00.905 INFO:tasks.workunit.client.0.smithi038.stdout:9/955: dread d1/dc/d4f/d14d/fd3 [0,4194304] 0 2024-02-15T18:35:00.905 INFO:tasks.workunit.client.0.smithi038.stdout:2/751: dread dd/d12/d5d/fb1 [0,4194304] 0 2024-02-15T18:35:00.905 INFO:tasks.workunit.client.0.smithi038.stdout:1/792: creat d2/d10/d19/d22/d36/d94/dc1/f107 x:0 0 0 2024-02-15T18:35:00.905 INFO:tasks.workunit.client.0.smithi038.stdout:1/793: write d2/d10/d19/d5f/f97 [106423,42291] 0 2024-02-15T18:35:00.907 INFO:tasks.workunit.client.0.smithi038.stdout:4/865: dwrite d11/d1e/d20/d31/d48/fe0 [0,4194304] 0 2024-02-15T18:35:00.907 INFO:tasks.workunit.client.0.smithi038.stdout:3/779: dwrite d2/d25/d60/f76 [0,4194304] 0 2024-02-15T18:35:00.907 INFO:tasks.workunit.client.0.smithi038.stdout:1/794: creat d2/d10/d19/d22/f108 x:0 0 0 2024-02-15T18:35:00.908 INFO:tasks.workunit.client.0.smithi038.stdout:1/795: chown d2/d10/d19/d22/d54/d80/lce 1789374 1 2024-02-15T18:35:00.912 INFO:tasks.workunit.client.0.smithi038.stdout:6/853: dwrite d2/d4/d2c/d39/d76/fd2 [0,4194304] 0 2024-02-15T18:35:00.912 INFO:tasks.workunit.client.0.smithi038.stdout:5/807: dwrite d2/d3/d12/d5e/fbb [0,4194304] 0 2024-02-15T18:35:00.912 INFO:tasks.workunit.client.0.smithi038.stdout:7/861: dwrite d0/db/d59/dc8/dd6/d61/f75 [0,4194304] 0 2024-02-15T18:35:00.912 INFO:tasks.workunit.client.0.smithi038.stdout:5/808: read d2/d3/da0/d33/f93 [134208,53718] 0 2024-02-15T18:35:00.914 INFO:tasks.workunit.client.0.smithi038.stdout:6/854: write d2/f9 [6661364,99245] 0 2024-02-15T18:35:00.914 INFO:tasks.workunit.client.0.smithi038.stdout:6/855: fdatasync d2/d4/d28/f69 0 2024-02-15T18:35:00.925 INFO:tasks.workunit.client.0.smithi038.stdout:3/780: mkdir d2/d1b/d3c/d3a/d55/df8 0 2024-02-15T18:35:00.926 INFO:tasks.workunit.client.0.smithi038.stdout:3/781: truncate d2/d1b/d3c/d3a/fb0 548071 0 2024-02-15T18:35:00.926 INFO:tasks.workunit.client.0.smithi038.stdout:4/866: symlink d11/d1e/d20/d31/d8d/d99/db5/l134 0 2024-02-15T18:35:00.927 INFO:tasks.workunit.client.0.smithi038.stdout:1/796: mkdir d2/d10/d19/d5f/d109 0 2024-02-15T18:35:00.932 INFO:tasks.workunit.client.0.smithi038.stdout:5/809: unlink d2/d3/d12/d5e/f8f 0 2024-02-15T18:35:00.932 INFO:tasks.workunit.client.0.smithi038.stdout:5/810: fdatasync d2/d3/d12/d5e/ff2 0 2024-02-15T18:35:00.937 INFO:tasks.workunit.client.0.smithi038.stdout:2/752: dwrite dd/d12/d5d/d33/fd6 [4194304,4194304] 0 2024-02-15T18:35:00.940 INFO:tasks.workunit.client.0.smithi038.stdout:6/856: rmdir d2 39 2024-02-15T18:35:00.941 INFO:tasks.workunit.client.0.smithi038.stdout:4/867: mknod d11/d1e/d20/d2b/d4d/d67/dbf/d111/c135 0 2024-02-15T18:35:00.942 INFO:tasks.workunit.client.0.smithi038.stdout:7/862: dwrite d0/db/d59/dc8/dd6/d61/f101 [0,4194304] 0 2024-02-15T18:35:00.943 INFO:tasks.workunit.client.0.smithi038.stdout:9/956: dwrite d1/da/d20/d58/d94/d13c/d11c/ffc [0,4194304] 0 2024-02-15T18:35:00.943 INFO:tasks.workunit.client.0.smithi038.stdout:9/957: chown d1/dc/db2/d154 948846441 1 2024-02-15T18:35:00.943 INFO:tasks.workunit.client.0.smithi038.stdout:1/797: mkdir d2/d10/d19/d5f/d109/d10a 0 2024-02-15T18:35:00.944 INFO:tasks.workunit.client.0.smithi038.stdout:1/798: read d2/d2e/fb6 [101017,45246] 0 2024-02-15T18:35:00.944 INFO:tasks.workunit.client.0.smithi038.stdout:6/857: rmdir d2/d4/d19/d78/dc4 39 2024-02-15T18:35:00.946 INFO:tasks.workunit.client.0.smithi038.stdout:7/863: symlink d0/db/d59/d15/d29/l139 0 2024-02-15T18:35:00.951 INFO:tasks.workunit.client.0.smithi038.stdout:9/958: unlink d1/da/d10/d13/d3d/d43/c64 0 2024-02-15T18:35:00.951 INFO:tasks.workunit.client.0.smithi038.stdout:9/959: fdatasync d1/da/f54 0 2024-02-15T18:35:00.951 INFO:tasks.workunit.client.0.smithi038.stdout:6/858: mkdir d2/d4/d6c/d9d/dc7/d105 0 2024-02-15T18:35:00.952 INFO:tasks.workunit.client.0.smithi038.stdout:3/782: dwrite d2/d1b/d3c/d42/fc0 [0,4194304] 0 2024-02-15T18:35:00.961 INFO:tasks.workunit.client.0.smithi038.stdout:7/864: symlink d0/d12f/l13a 0 2024-02-15T18:35:00.961 INFO:tasks.workunit.client.0.smithi038.stdout:9/960: rmdir d1/da/d20/d58/d76 39 2024-02-15T18:35:00.961 INFO:tasks.workunit.client.0.smithi038.stdout:7/865: write d0/db/d59/d15/d29/f114 [1012800,54102] 0 2024-02-15T18:35:00.961 INFO:tasks.workunit.client.0.smithi038.stdout:0/935: sync 2024-02-15T18:35:00.963 INFO:tasks.workunit.client.0.smithi038.stdout:9/961: creat d1/dc/db2/d13a/f156 x:0 0 0 2024-02-15T18:35:00.964 INFO:tasks.workunit.client.0.smithi038.stdout:7/866: symlink d0/db/d59/dc8/dd6/d61/l13b 0 2024-02-15T18:35:00.965 INFO:tasks.workunit.client.0.smithi038.stdout:7/867: chown d0/d66/f119 24 1 2024-02-15T18:35:00.965 INFO:tasks.workunit.client.0.smithi038.stdout:7/868: chown d0/db/d59/dc8/dd6/d70/cf8 19558838 1 2024-02-15T18:35:00.965 INFO:tasks.workunit.client.0.smithi038.stdout:7/869: chown d0/d1b/dfd/d108/ff6 196630343 1 2024-02-15T18:35:00.965 INFO:tasks.workunit.client.0.smithi038.stdout:2/753: dwrite dd/d12/d6b/da5/dd8/fdc [0,4194304] 0 2024-02-15T18:35:00.967 INFO:tasks.workunit.client.0.smithi038.stdout:0/936: mkdir d4/d13/d2a/d35/d1e/d32/d79/d7d/da3/d118/d127/d13c 0 2024-02-15T18:35:00.975 INFO:tasks.workunit.client.0.smithi038.stdout:2/754: symlink dd/d12/d3a/d5e/d65/lf3 0 2024-02-15T18:35:00.978 INFO:tasks.workunit.client.0.smithi038.stdout:0/937: rmdir d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/df4 39 2024-02-15T18:35:00.980 INFO:tasks.workunit.client.0.smithi038.stdout:2/755: symlink dd/d12/d3a/d5e/lf4 0 2024-02-15T18:35:00.981 INFO:tasks.workunit.client.0.smithi038.stdout:2/756: creat dd/d12/d1d/ff5 x:0 0 0 2024-02-15T18:35:00.981 INFO:tasks.workunit.client.0.smithi038.stdout:2/757: chown dd/d12/d5d/d54/f8a 988458 1 2024-02-15T18:35:00.990 INFO:tasks.workunit.client.0.smithi038.stdout:5/811: dwrite d2/dcc/fd6 [0,4194304] 0 2024-02-15T18:35:00.990 INFO:tasks.workunit.client.0.smithi038.stdout:5/812: fsync d2/dcc/f8b 0 2024-02-15T18:35:00.990 INFO:tasks.workunit.client.0.smithi038.stdout:1/799: dwrite d2/d10/d19/d22/d54/dc0/d6b/fb7 [0,4194304] 0 2024-02-15T18:35:00.991 INFO:tasks.workunit.client.0.smithi038.stdout:5/813: unlink d2/d48/d83/d96/f85 0 2024-02-15T18:35:00.994 INFO:tasks.workunit.client.0.smithi038.stdout:1/800: unlink d2/d10/d19/f72 0 2024-02-15T18:35:00.994 INFO:tasks.workunit.client.0.smithi038.stdout:5/814: write d2/dcc/f7b [2731561,31976] 0 2024-02-15T18:35:00.995 INFO:tasks.workunit.client.0.smithi038.stdout:1/801: truncate d2/f9d 267796 0 2024-02-15T18:35:00.995 INFO:tasks.workunit.client.0.smithi038.stdout:1/802: truncate d2/d10/d19/d22/d36/d94/dc1/ff2 38975 0 2024-02-15T18:35:00.997 INFO:tasks.workunit.client.0.smithi038.stdout:5/815: creat d2/d3/da0/ff4 x:0 0 0 2024-02-15T18:35:00.997 INFO:tasks.workunit.client.0.smithi038.stdout:5/816: write d2/d3/db9/dcf/faa [713658,15492] 0 2024-02-15T18:35:00.997 INFO:tasks.workunit.client.0.smithi038.stdout:5/817: truncate d2/d3/da0/fae 957174 0 2024-02-15T18:35:00.997 INFO:tasks.workunit.client.0.smithi038.stdout:1/803: rmdir d2/d10/d19/d22/d36 39 2024-02-15T18:35:00.997 INFO:tasks.workunit.client.0.smithi038.stdout:5/818: symlink d2/d3/da0/lf5 0 2024-02-15T18:35:00.997 INFO:tasks.workunit.client.0.smithi038.stdout:5/819: fsync d2/d3/ff1 0 2024-02-15T18:35:00.997 INFO:tasks.workunit.client.0.smithi038.stdout:1/804: mkdir d2/d10/d10b 0 2024-02-15T18:35:00.999 INFO:tasks.workunit.client.0.smithi038.stdout:5/820: symlink d2/d48/d83/d96/lf6 0 2024-02-15T18:35:01.000 INFO:tasks.workunit.client.0.smithi038.stdout:5/821: unlink d2/d3/db9/de3/de4/lc0 0 2024-02-15T18:35:01.001 INFO:tasks.workunit.client.0.smithi038.stdout:4/868: dwrite d11/d1e/d20/d31/d48/d60/f12b [0,4194304] 0 2024-02-15T18:35:01.001 INFO:tasks.workunit.client.0.smithi038.stdout:4/869: write d11/d1e/fe8 [335264,15883] 0 2024-02-15T18:35:01.001 INFO:tasks.workunit.client.0.smithi038.stdout:4/870: dread - d11/d1e/d20/d31/d8d/f12e zero size 2024-02-15T18:35:01.001 INFO:tasks.workunit.client.0.smithi038.stdout:5/822: write d2/d64/d99/db6/fe8 [4298107,14565] 0 2024-02-15T18:35:01.001 INFO:tasks.workunit.client.0.smithi038.stdout:5/823: fdatasync d2/d3/da0/f50 0 2024-02-15T18:35:01.001 INFO:tasks.workunit.client.0.smithi038.stdout:5/824: stat d2/d3/db9/dcf/ca4 0 2024-02-15T18:35:01.004 INFO:tasks.workunit.client.0.smithi038.stdout:6/859: dwrite d2/d10/f49 [4194304,4194304] 0 2024-02-15T18:35:01.005 INFO:tasks.workunit.client.0.smithi038.stdout:3/783: dwrite d2/d1b/d4f/f8d [4194304,4194304] 0 2024-02-15T18:35:01.005 INFO:tasks.workunit.client.0.smithi038.stdout:7/870: dwrite d0/db/f42 [0,4194304] 0 2024-02-15T18:35:01.005 INFO:tasks.workunit.client.0.smithi038.stdout:7/871: readlink d0/d66/lb2 0 2024-02-15T18:35:01.005 INFO:tasks.workunit.client.0.smithi038.stdout:7/872: readlink d0/d66/l131 0 2024-02-15T18:35:01.005 INFO:tasks.workunit.client.0.smithi038.stdout:9/962: dwrite d1/dc/d4f/d14d/f53 [0,4194304] 0 2024-02-15T18:35:01.005 INFO:tasks.workunit.client.0.smithi038.stdout:2/758: dwrite dd/d12/d3a/d67/fc1 [0,4194304] 0 2024-02-15T18:35:01.006 INFO:tasks.workunit.client.0.smithi038.stdout:6/860: write d2/d4/d2c/d39/db4/f7c [2639146,89479] 0 2024-02-15T18:35:01.006 INFO:tasks.workunit.client.0.smithi038.stdout:6/861: truncate d2/d10/fa8 565389 0 2024-02-15T18:35:01.007 INFO:tasks.workunit.client.0.smithi038.stdout:0/938: dwrite d4/d13/d2a/d35/f83 [0,4194304] 0 2024-02-15T18:35:01.008 INFO:tasks.workunit.client.0.smithi038.stdout:4/871: link d11/d1e/f131 d11/d1e/d20/d31/d7a/dd8/f136 0 2024-02-15T18:35:01.014 INFO:tasks.workunit.client.0.smithi038.stdout:3/784: creat d2/d1b/d3c/d41/ff9 x:0 0 0 2024-02-15T18:35:01.014 INFO:tasks.workunit.client.0.smithi038.stdout:7/873: chown d0/db/d59/l58 394309017 1 2024-02-15T18:35:01.016 INFO:tasks.workunit.client.0.smithi038.stdout:6/862: link d2/d4/l3b d2/d4/d2c/d39/db4/l106 0 2024-02-15T18:35:01.020 INFO:tasks.workunit.client.0.smithi038.stdout:3/785: unlink d2/db8/dce/fe8 0 2024-02-15T18:35:01.023 INFO:tasks.workunit.client.0.smithi038.stdout:7/874: creat d0/db/d59/f13c x:0 0 0 2024-02-15T18:35:01.028 INFO:tasks.workunit.client.0.smithi038.stdout:6/863: creat d2/d10/db6/f107 x:0 0 0 2024-02-15T18:35:01.028 INFO:tasks.workunit.client.0.smithi038.stdout:6/864: fsync d2/d4/d19/d78/dbf/fe5 0 2024-02-15T18:35:01.029 INFO:tasks.workunit.client.0.smithi038.stdout:4/872: rename d11/d1e/d20/d2f/d7d/cb0 to d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/d11d/c137 0 2024-02-15T18:35:01.029 INFO:tasks.workunit.client.0.smithi038.stdout:7/875: symlink d0/d1b/dfd/d132/l13d 0 2024-02-15T18:35:01.036 INFO:tasks.workunit.client.0.smithi038.stdout:5/825: dwrite d2/d3/da0/d33/f68 [0,4194304] 0 2024-02-15T18:35:01.036 INFO:tasks.workunit.client.0.smithi038.stdout:5/826: fdatasync d2/d3/db9/dcf/fa9 0 2024-02-15T18:35:01.037 INFO:tasks.workunit.client.0.smithi038.stdout:6/865: symlink d2/d4/d19/d78/d7a/l108 0 2024-02-15T18:35:01.037 INFO:tasks.workunit.client.0.smithi038.stdout:6/866: chown d2/d4/d2c/d39/db4/f93 4 1 2024-02-15T18:35:01.038 INFO:tasks.workunit.client.0.smithi038.stdout:6/867: truncate d2/d4/d19/d55/fd9 843416 0 2024-02-15T18:35:01.038 INFO:tasks.workunit.client.0.smithi038.stdout:7/876: creat d0/db/d59/d15/d29/d8d/d102/f13e x:0 0 0 2024-02-15T18:35:01.038 INFO:tasks.workunit.client.0.smithi038.stdout:7/877: chown d0/d1b/d30/f43 12753756 1 2024-02-15T18:35:01.039 INFO:tasks.workunit.client.0.smithi038.stdout:5/827: mknod d2/d48/d83/d96/cf7 0 2024-02-15T18:35:01.044 INFO:tasks.workunit.client.0.smithi038.stdout:6/868: mkdir d2/d4/d19/d78/dc3/d109 0 2024-02-15T18:35:01.045 INFO:tasks.workunit.client.0.smithi038.stdout:7/878: creat d0/d1b/f13f x:0 0 0 2024-02-15T18:35:01.045 INFO:tasks.workunit.client.0.smithi038.stdout:3/786: dwrite d2/d1b/d3c/d41/d4e/d62/f87 [8388608,4194304] 0 2024-02-15T18:35:01.045 INFO:tasks.workunit.client.0.smithi038.stdout:5/828: mknod d2/d3/db9/dcf/cf8 0 2024-02-15T18:35:01.047 INFO:tasks.workunit.client.0.smithi038.stdout:3/787: mkdir d2/d7f/dc1/de1/dfa 0 2024-02-15T18:35:01.047 INFO:tasks.workunit.client.0.smithi038.stdout:5/829: read d2/d3/da0/d33/f68 [2927716,18174] 0 2024-02-15T18:35:01.049 INFO:tasks.workunit.client.0.smithi038.stdout:5/830: creat d2/d64/d99/ff9 x:0 0 0 2024-02-15T18:35:01.051 INFO:tasks.workunit.client.0.smithi038.stdout:2/759: dwrite dd/d12/f39 [0,4194304] 0 2024-02-15T18:35:01.051 INFO:tasks.workunit.client.0.smithi038.stdout:5/831: unlink d2/d3/d5f/f82 0 2024-02-15T18:35:01.053 INFO:tasks.workunit.client.0.smithi038.stdout:0/939: dwrite d4/f63 [0,4194304] 0 2024-02-15T18:35:01.053 INFO:tasks.workunit.client.0.smithi038.stdout:2/760: creat dd/d12/d5d/dc2/ff6 x:0 0 0 2024-02-15T18:35:01.053 INFO:tasks.workunit.client.0.smithi038.stdout:1/805: dwrite d2/d10/d19/d5f/f97 [0,4194304] 0 2024-02-15T18:35:01.053 INFO:tasks.workunit.client.0.smithi038.stdout:7/879: dread d0/db/d59/d15/f64 [0,4194304] 0 2024-02-15T18:35:01.055 INFO:tasks.workunit.client.0.smithi038.stdout:7/880: dread d0/db/d19/d7d/f111 [0,4194304] 0 2024-02-15T18:35:01.062 INFO:tasks.workunit.client.0.smithi038.stdout:5/832: creat d2/d3/dd0/ffa x:0 0 0 2024-02-15T18:35:01.062 INFO:tasks.workunit.client.0.smithi038.stdout:5/833: truncate d2/d48/d83/d96/fd3 645565 0 2024-02-15T18:35:01.062 INFO:tasks.workunit.client.0.smithi038.stdout:0/940: mknod d4/d13/d2a/d35/d6a/c13d 0 2024-02-15T18:35:01.063 INFO:tasks.workunit.client.0.smithi038.stdout:2/761: creat dd/d12/d5d/dc2/ff7 x:0 0 0 2024-02-15T18:35:01.064 INFO:tasks.workunit.client.0.smithi038.stdout:1/806: mkdir d2/d10/d32/d53/d10c 0 2024-02-15T18:35:01.064 INFO:tasks.workunit.client.0.smithi038.stdout:1/807: write d2/d10/d32/d53/d82/ded/f101 [988440,1657] 0 2024-02-15T18:35:01.064 INFO:tasks.workunit.client.0.smithi038.stdout:7/881: getdents d0/d1b/d80 0 2024-02-15T18:35:01.065 INFO:tasks.workunit.client.0.smithi038.stdout:3/788: dwrite d2/d1e/fd9 [0,4194304] 0 2024-02-15T18:35:01.066 INFO:tasks.workunit.client.0.smithi038.stdout:6/869: dwrite d2/d4/d2c/d39/db4/d5f/fff [0,4194304] 0 2024-02-15T18:35:01.066 INFO:tasks.workunit.client.0.smithi038.stdout:6/870: write d2/d4/d19/d78/fd8 [708501,116355] 0 2024-02-15T18:35:01.066 INFO:tasks.workunit.client.0.smithi038.stdout:5/834: unlink d2/d3/d5f/fef 0 2024-02-15T18:35:01.067 INFO:tasks.workunit.client.0.smithi038.stdout:2/762: symlink dd/d12/d3a/d5e/lf8 0 2024-02-15T18:35:01.068 INFO:tasks.workunit.client.0.smithi038.stdout:1/808: mknod d2/d69/d8b/c10d 0 2024-02-15T18:35:01.068 INFO:tasks.workunit.client.0.smithi038.stdout:5/835: read d2/d3/f2f [4842508,13958] 0 2024-02-15T18:35:01.070 INFO:tasks.workunit.client.0.smithi038.stdout:7/882: dread d0/db/fb1 [0,4194304] 0 2024-02-15T18:35:01.076 INFO:tasks.workunit.client.0.smithi038.stdout:6/871: link d2/d4/d19/d78/d7a/c80 d2/d4/d19/d78/dc4/df0/c10a 0 2024-02-15T18:35:01.076 INFO:tasks.workunit.client.0.smithi038.stdout:3/789: dread d2/d1b/d3c/d42/fc3 [0,4194304] 0 2024-02-15T18:35:01.077 INFO:tasks.workunit.client.0.smithi038.stdout:2/763: mkdir dd/d12/d6b/da5/dd8/df9 0 2024-02-15T18:35:01.077 INFO:tasks.workunit.client.0.smithi038.stdout:2/764: chown dd/d12/d3a/d67/d4f/d5a/d9d/fac 149456802 1 2024-02-15T18:35:01.085 INFO:tasks.workunit.client.0.smithi038.stdout:6/872: rmdir d2/d4/d6c/d9d 39 2024-02-15T18:35:01.087 INFO:tasks.workunit.client.0.smithi038.stdout:3/790: rename d2/f2f to d2/d1b/ffb 0 2024-02-15T18:35:01.087 INFO:tasks.workunit.client.0.smithi038.stdout:3/791: fsync d2/d1b/d4f/f5d 0 2024-02-15T18:35:01.087 INFO:tasks.workunit.client.0.smithi038.stdout:0/941: dwrite d4/f63 [0,4194304] 0 2024-02-15T18:35:01.088 INFO:tasks.workunit.client.0.smithi038.stdout:5/836: dwrite d2/d3/da0/fae [0,4194304] 0 2024-02-15T18:35:01.088 INFO:tasks.workunit.client.0.smithi038.stdout:5/837: chown d2/d3/da0/d33/f68 30 1 2024-02-15T18:35:01.088 INFO:tasks.workunit.client.0.smithi038.stdout:5/838: stat d2/d3/d5f/c9d 0 2024-02-15T18:35:01.088 INFO:tasks.workunit.client.0.smithi038.stdout:5/839: chown d2/d9a 101 1 2024-02-15T18:35:01.088 INFO:tasks.workunit.client.0.smithi038.stdout:2/765: mknod dd/d12/d5d/d93/cfa 0 2024-02-15T18:35:01.088 INFO:tasks.workunit.client.0.smithi038.stdout:7/883: mknod d0/db/d59/c140 0 2024-02-15T18:35:01.089 INFO:tasks.workunit.client.0.smithi038.stdout:6/873: mknod d2/d10/dd0/c10b 0 2024-02-15T18:35:01.093 INFO:tasks.workunit.client.0.smithi038.stdout:3/792: getdents d2/db8/dcf 0 2024-02-15T18:35:01.093 INFO:tasks.workunit.client.0.smithi038.stdout:3/793: write d2/d1e/d7e/fbc [42184,106097] 0 2024-02-15T18:35:01.096 INFO:tasks.workunit.client.0.smithi038.stdout:5/840: mknod d2/d48/cfb 0 2024-02-15T18:35:01.097 INFO:tasks.workunit.client.0.smithi038.stdout:2/766: rmdir dd/d12/d3a/d67 39 2024-02-15T18:35:01.097 INFO:tasks.workunit.client.0.smithi038.stdout:5/841: dread d2/d48/d83/d96/fd3 [0,4194304] 0 2024-02-15T18:35:01.098 INFO:tasks.workunit.client.0.smithi038.stdout:6/874: mknod d2/d4/d19/d78/dbf/c10c 0 2024-02-15T18:35:01.098 INFO:tasks.workunit.client.0.smithi038.stdout:6/875: truncate d2/d10/fa8 1120455 0 2024-02-15T18:35:01.099 INFO:tasks.workunit.client.0.smithi038.stdout:5/842: symlink d2/d3/db9/de3/de4/lfc 0 2024-02-15T18:35:01.100 INFO:tasks.workunit.client.0.smithi038.stdout:6/876: creat d2/d4/d2c/d39/db4/d5f/d99/f10d x:0 0 0 2024-02-15T18:35:01.102 INFO:tasks.workunit.client.0.smithi038.stdout:6/877: creat d2/d4/d2c/d39/f10e x:0 0 0 2024-02-15T18:35:01.105 INFO:tasks.workunit.client.0.smithi038.stdout:6/878: dread d2/f5a [0,4194304] 0 2024-02-15T18:35:01.105 INFO:tasks.workunit.client.0.smithi038.stdout:6/879: fsync d2/d4/d28/f3a 0 2024-02-15T18:35:01.105 INFO:tasks.workunit.client.0.smithi038.stdout:6/880: stat d2/d4/d19/d55/f8c 0 2024-02-15T18:35:01.108 INFO:tasks.workunit.client.0.smithi038.stdout:7/884: dwrite d0/db/d59/d15/d29/fe8 [0,4194304] 0 2024-02-15T18:35:01.109 INFO:tasks.workunit.client.0.smithi038.stdout:7/885: write d0/db/fc [2045625,98752] 0 2024-02-15T18:35:01.111 INFO:tasks.workunit.client.0.smithi038.stdout:6/881: rename d2/d4/d6f/la6 to d2/d4/d19/d55/d5c/dc8/l10f 0 2024-02-15T18:35:01.111 INFO:tasks.workunit.client.0.smithi038.stdout:6/882: fdatasync d2/d4/d19/d55/fde 0 2024-02-15T18:35:01.115 INFO:tasks.workunit.client.0.smithi038.stdout:5/843: dread d2/d30/f55 [0,4194304] 0 2024-02-15T18:35:01.115 INFO:tasks.workunit.client.0.smithi038.stdout:5/844: stat d2/d3/db9/dcf/ddf/cf3 0 2024-02-15T18:35:01.116 INFO:tasks.workunit.client.0.smithi038.stdout:2/767: dwrite dd/d12/d3a/d67/d4f/d5a/f6e [0,4194304] 0 2024-02-15T18:35:01.121 INFO:tasks.workunit.client.0.smithi038.stdout:5/845: mkdir d2/d3/d12/d5e/dfd 0 2024-02-15T18:35:01.123 INFO:tasks.workunit.client.0.smithi038.stdout:5/846: truncate d2/dcc/f84 116488 0 2024-02-15T18:35:01.123 INFO:tasks.workunit.client.0.smithi038.stdout:9/963: sync 2024-02-15T18:35:01.125 INFO:tasks.workunit.client.0.smithi038.stdout:5/847: truncate d2/d3/f1b 1980677 0 2024-02-15T18:35:01.129 INFO:tasks.workunit.client.0.smithi038.stdout:2/768: dread dd/d12/d3a/d67/f8f [4194304,4194304] 0 2024-02-15T18:35:01.129 INFO:tasks.workunit.client.0.smithi038.stdout:2/769: truncate dd/d12/d5d/d54/dcb/fec 198858 0 2024-02-15T18:35:01.129 INFO:tasks.workunit.client.0.smithi038.stdout:2/770: fdatasync dd/d12/d5d/d54/fa4 0 2024-02-15T18:35:01.130 INFO:tasks.workunit.client.0.smithi038.stdout:2/771: readlink dd/d12/d3a/d5e/lf4 0 2024-02-15T18:35:01.130 INFO:tasks.workunit.client.0.smithi038.stdout:2/772: chown dd/d12/d3a/d5e/d65/dcd/df1 107750569 1 2024-02-15T18:35:01.130 INFO:tasks.workunit.client.0.smithi038.stdout:7/886: dwrite d0/db/fb1 [0,4194304] 0 2024-02-15T18:35:01.130 INFO:tasks.workunit.client.0.smithi038.stdout:9/964: write d1/da/d36/d93/fa4 [2488596,59273] 0 2024-02-15T18:35:01.130 INFO:tasks.workunit.client.0.smithi038.stdout:9/965: fdatasync d1/da/d36/d3f/f75 0 2024-02-15T18:35:01.130 INFO:tasks.workunit.client.0.smithi038.stdout:2/773: mknod dd/d12/d3a/d5e/d65/dcd/cfb 0 2024-02-15T18:35:01.134 INFO:tasks.workunit.client.0.smithi038.stdout:9/966: unlink d1/da/d20/d58/d76/f8c 0 2024-02-15T18:35:01.134 INFO:tasks.workunit.client.0.smithi038.stdout:9/967: fdatasync d1/da/d10/d13/d3d/d43/d4d/dfd/f101 0 2024-02-15T18:35:01.137 INFO:tasks.workunit.client.0.smithi038.stdout:7/887: rename d0/db/d59/dc8/dd6/d61/f84 to d0/f141 0 2024-02-15T18:35:01.138 INFO:tasks.workunit.client.0.smithi038.stdout:9/968: mkdir d1/da/d10/d157 0 2024-02-15T18:35:01.139 INFO:tasks.workunit.client.0.smithi038.stdout:0/942: dwrite d4/d13/d2a/f2d [0,4194304] 0 2024-02-15T18:35:01.140 INFO:tasks.workunit.client.0.smithi038.stdout:7/888: truncate d0/db/d59/dc8/dd6/d61/fe2 5595 0 2024-02-15T18:35:01.143 INFO:tasks.workunit.client.0.smithi038.stdout:9/969: symlink d1/da/d10/d13/d3d/dfb/l158 0 2024-02-15T18:35:01.143 INFO:tasks.workunit.client.0.smithi038.stdout:0/943: symlink d4/d13/d53/d100/l13e 0 2024-02-15T18:35:01.146 INFO:tasks.workunit.client.0.smithi038.stdout:7/889: mkdir d0/db/d59/d15/d29/d8d/dcd/d134/d142 0 2024-02-15T18:35:01.146 INFO:tasks.workunit.client.0.smithi038.stdout:7/890: fdatasync d0/d68/fca 0 2024-02-15T18:35:01.147 INFO:tasks.workunit.client.0.smithi038.stdout:7/891: dread d0/db/d59/d15/fe3 [0,4194304] 0 2024-02-15T18:35:01.148 INFO:tasks.workunit.client.0.smithi038.stdout:6/883: dwrite d2/d4/d2c/d39/db4/fe1 [0,4194304] 0 2024-02-15T18:35:01.152 INFO:tasks.workunit.client.0.smithi038.stdout:6/884: rename d2/d4/d2c/d39/fe0 to d2/d4/d19/d78/dc4/df0/f110 0 2024-02-15T18:35:01.153 INFO:tasks.workunit.client.0.smithi038.stdout:4/873: sync 2024-02-15T18:35:01.156 INFO:tasks.workunit.client.0.smithi038.stdout:6/885: rmdir d2/d4/d6c/d9d/dbd 0 2024-02-15T18:35:01.156 INFO:tasks.workunit.client.0.smithi038.stdout:6/886: chown d2/d4/d2c/d39/db4/d5f/c85 458834717 1 2024-02-15T18:35:01.156 INFO:tasks.workunit.client.0.smithi038.stdout:5/848: dwrite d2/d3/d12/f8c [0,4194304] 0 2024-02-15T18:35:01.156 INFO:tasks.workunit.client.0.smithi038.stdout:6/887: stat d2/d4/d2c/d39/db4/d5f/ffd 0 2024-02-15T18:35:01.157 INFO:tasks.workunit.client.0.smithi038.stdout:6/888: dread d2/f58 [0,4194304] 0 2024-02-15T18:35:01.157 INFO:tasks.workunit.client.0.smithi038.stdout:5/849: rmdir d2/d64/d99/db6 39 2024-02-15T18:35:01.158 INFO:tasks.workunit.client.0.smithi038.stdout:6/889: getdents d2 0 2024-02-15T18:35:01.158 INFO:tasks.workunit.client.0.smithi038.stdout:6/890: fdatasync d2/d4/d6c/d9d/dd7/fe6 0 2024-02-15T18:35:01.158 INFO:tasks.workunit.client.0.smithi038.stdout:6/891: fsync d2/d4/d2c/d39/db4/f7c 0 2024-02-15T18:35:01.159 INFO:tasks.workunit.client.0.smithi038.stdout:2/774: dwrite dd/d12/d5d/d54/dcb/fec [0,4194304] 0 2024-02-15T18:35:01.159 INFO:tasks.workunit.client.0.smithi038.stdout:5/850: write d2/d3/da0/fb4 [3757985,92787] 0 2024-02-15T18:35:01.159 INFO:tasks.workunit.client.0.smithi038.stdout:5/851: chown d2/d3/d5f/lc1 24 1 2024-02-15T18:35:01.165 INFO:tasks.workunit.client.0.smithi038.stdout:2/775: creat dd/d12/d3a/d67/ffc x:0 0 0 2024-02-15T18:35:01.166 INFO:tasks.workunit.client.0.smithi038.stdout:5/852: creat d2/d3/dd0/ffe x:0 0 0 2024-02-15T18:35:01.168 INFO:tasks.workunit.client.0.smithi038.stdout:0/944: dwrite d4/d13/d19/d137/f138 [0,4194304] 0 2024-02-15T18:35:01.168 INFO:tasks.workunit.client.0.smithi038.stdout:2/776: unlink dd/d12/d59/d62/ldb 0 2024-02-15T18:35:01.168 INFO:tasks.workunit.client.0.smithi038.stdout:9/970: dwrite d1/da/d20/d58/d66/f116 [0,4194304] 0 2024-02-15T18:35:01.169 INFO:tasks.workunit.client.0.smithi038.stdout:2/777: write dd/d12/f47 [256005,3904] 0 2024-02-15T18:35:01.170 INFO:tasks.workunit.client.0.smithi038.stdout:2/778: write dd/d12/f47 [4936293,7535] 0 2024-02-15T18:35:01.170 INFO:tasks.workunit.client.0.smithi038.stdout:2/779: readlink dd/d12/d59/d62/lab 0 2024-02-15T18:35:01.173 INFO:tasks.workunit.client.0.smithi038.stdout:7/892: rmdir d0/db/d59/d15/d29 39 2024-02-15T18:35:01.178 INFO:tasks.workunit.client.0.smithi038.stdout:5/853: mkdir d2/d3/db9/de3/de4/dff 0 2024-02-15T18:35:01.185 INFO:tasks.workunit.client.0.smithi038.stdout:9/971: getdents d1/dc/d4f/d14d/d49/de4 0 2024-02-15T18:35:01.185 INFO:tasks.workunit.client.0.smithi038.stdout:9/972: dread - d1/da/d10/f9f zero size 2024-02-15T18:35:01.186 INFO:tasks.workunit.client.0.smithi038.stdout:2/780: mkdir dd/d12/d6b/dfd 0 2024-02-15T18:35:01.187 INFO:tasks.workunit.client.0.smithi038.stdout:5/854: creat d2/d3/db9/de3/de4/dff/f100 x:0 0 0 2024-02-15T18:35:01.187 INFO:tasks.workunit.client.0.smithi038.stdout:5/855: write d2/d3/d12/fe7 [887382,128874] 0 2024-02-15T18:35:01.187 INFO:tasks.workunit.client.0.smithi038.stdout:5/856: chown d2/d3/dd0/ffe 6348 1 2024-02-15T18:35:01.187 INFO:tasks.workunit.client.0.smithi038.stdout:5/857: fdatasync d2/d48/f59 0 2024-02-15T18:35:01.187 INFO:tasks.workunit.client.0.smithi038.stdout:2/781: rmdir dd/d12/d5d/d33 39 2024-02-15T18:35:01.187 INFO:tasks.workunit.client.0.smithi038.stdout:9/973: write d1/da/fed [1921102,25533] 0 2024-02-15T18:35:01.187 INFO:tasks.workunit.client.0.smithi038.stdout:9/974: stat d1/da/d10/d157 0 2024-02-15T18:35:01.188 INFO:tasks.workunit.client.0.smithi038.stdout:2/782: rmdir dd/d12/d6b/da5 39 2024-02-15T18:35:01.190 INFO:tasks.workunit.client.0.smithi038.stdout:9/975: symlink d1/da/d10/d13/d3d/d43/d4d/l159 0 2024-02-15T18:35:01.191 INFO:tasks.workunit.client.0.smithi038.stdout:7/893: dwrite d0/d66/f35 [4194304,4194304] 0 2024-02-15T18:35:01.191 INFO:tasks.workunit.client.0.smithi038.stdout:7/894: fsync d0/db/d59/d15/f71 0 2024-02-15T18:35:01.192 INFO:tasks.workunit.client.0.smithi038.stdout:9/976: dread d1/f14 [0,4194304] 0 2024-02-15T18:35:01.192 INFO:tasks.workunit.client.0.smithi038.stdout:9/977: readlink d1/dc/db2/d13a/l152 0 2024-02-15T18:35:01.195 INFO:tasks.workunit.client.0.smithi038.stdout:1/809: sync 2024-02-15T18:35:01.195 INFO:tasks.workunit.client.0.smithi038.stdout:1/810: chown d2/d10/d32/d53/d82/ffa 9 1 2024-02-15T18:35:01.195 INFO:tasks.workunit.client.0.smithi038.stdout:1/811: readlink d2/l8 0 2024-02-15T18:35:01.196 INFO:tasks.workunit.client.0.smithi038.stdout:7/895: creat d0/db/f143 x:0 0 0 2024-02-15T18:35:01.200 INFO:tasks.workunit.client.0.smithi038.stdout:7/896: rmdir d0/db/d59/d97/ddc 0 2024-02-15T18:35:01.200 INFO:tasks.workunit.client.0.smithi038.stdout:7/897: chown d0/db/d59/dc8/dd6/d61/d90/dc0/fce 11551207 1 2024-02-15T18:35:01.202 INFO:tasks.workunit.client.0.smithi038.stdout:3/794: sync 2024-02-15T18:35:01.204 INFO:tasks.workunit.client.0.smithi038.stdout:3/795: creat d2/d1b/d8c/ffc x:0 0 0 2024-02-15T18:35:01.205 INFO:tasks.workunit.client.0.smithi038.stdout:7/898: link d0/db/d59/dc8/dd6/d70/lbd d0/l144 0 2024-02-15T18:35:01.205 INFO:tasks.workunit.client.0.smithi038.stdout:3/796: truncate d2/d25/d60/f76 1592131 0 2024-02-15T18:35:01.206 INFO:tasks.workunit.client.0.smithi038.stdout:7/899: write d0/f21 [1974497,126781] 0 2024-02-15T18:35:01.206 INFO:tasks.workunit.client.0.smithi038.stdout:7/900: truncate d0/db/d19/d7d/db5/fcf 110074 0 2024-02-15T18:35:01.206 INFO:tasks.workunit.client.0.smithi038.stdout:7/901: dread - d0/d66/f89 zero size 2024-02-15T18:35:01.207 INFO:tasks.workunit.client.0.smithi038.stdout:3/797: dread d2/d1b/d4f/fb4 [0,4194304] 0 2024-02-15T18:35:01.210 INFO:tasks.workunit.client.0.smithi038.stdout:5/858: dwrite d2/d3/d12/d5e/ff2 [0,4194304] 0 2024-02-15T18:35:01.214 INFO:tasks.workunit.client.0.smithi038.stdout:7/902: rename d0/c67 to d0/d1b/dfd/d137/c145 0 2024-02-15T18:35:01.216 INFO:tasks.workunit.client.0.smithi038.stdout:5/859: mkdir d2/d3/d12/d5e/dfd/d101 0 2024-02-15T18:35:01.216 INFO:tasks.workunit.client.0.smithi038.stdout:5/860: readlink d2/d3/l9 0 2024-02-15T18:35:01.216 INFO:tasks.workunit.client.0.smithi038.stdout:5/861: chown d2/d3/db9/dcf/ddf/cf3 88150932 1 2024-02-15T18:35:01.216 INFO:tasks.workunit.client.0.smithi038.stdout:5/862: chown d2/d3/d5f/lc4 122321 1 2024-02-15T18:35:01.218 INFO:tasks.workunit.client.0.smithi038.stdout:5/863: creat d2/d3/db9/de3/f102 x:0 0 0 2024-02-15T18:35:01.226 INFO:tasks.workunit.client.0.smithi038.stdout:7/903: read d0/f54 [2833491,53166] 0 2024-02-15T18:35:01.227 INFO:tasks.workunit.client.0.smithi038.stdout:7/904: mkdir d0/d68/dfa/d146 0 2024-02-15T18:35:01.228 INFO:tasks.workunit.client.0.smithi038.stdout:2/783: dwrite dd/d12/d59/f21 [4194304,4194304] 0 2024-02-15T18:35:01.241 INFO:tasks.workunit.client.0.smithi038.stdout:9/978: dwrite d1/da/d20/d58/d94/d13c/d11c/d107/f55 [0,4194304] 0 2024-02-15T18:35:01.250 INFO:tasks.workunit.client.0.smithi038.stdout:1/812: dwrite d2/d69/f6c [0,4194304] 0 2024-02-15T18:35:01.251 INFO:tasks.workunit.client.0.smithi038.stdout:2/784: dread dd/d12/d3a/d67/fa0 [0,4194304] 0 2024-02-15T18:35:01.251 INFO:tasks.workunit.client.0.smithi038.stdout:2/785: readlink dd/d12/d1d/d3b/d3c/d4a/l76 0 2024-02-15T18:35:01.251 INFO:tasks.workunit.client.0.smithi038.stdout:2/786: readlink dd/d12/d59/d2b/d6a/l9a 0 2024-02-15T18:35:01.251 INFO:tasks.workunit.client.0.smithi038.stdout:5/864: dwrite d2/f80 [0,4194304] 0 2024-02-15T18:35:01.251 INFO:tasks.workunit.client.0.smithi038.stdout:5/865: dread - d2/d3/db9/de3/f102 zero size 2024-02-15T18:35:01.252 INFO:tasks.workunit.client.0.smithi038.stdout:1/813: symlink d2/d10/d19/d22/d54/dc0/l10e 0 2024-02-15T18:35:01.253 INFO:tasks.workunit.client.0.smithi038.stdout:2/787: symlink dd/d12/d6b/dfd/lfe 0 2024-02-15T18:35:01.253 INFO:tasks.workunit.client.0.smithi038.stdout:7/905: dwrite d0/db/d19/d7d/fe5 [0,4194304] 0 2024-02-15T18:35:01.253 INFO:tasks.workunit.client.0.smithi038.stdout:7/906: read - d0/d1b/f13f zero size 2024-02-15T18:35:01.255 INFO:tasks.workunit.client.0.smithi038.stdout:5/866: creat d2/d48/d83/d96/f103 x:0 0 0 2024-02-15T18:35:01.255 INFO:tasks.workunit.client.0.smithi038.stdout:1/814: creat d2/d2e/d8c/f10f x:0 0 0 2024-02-15T18:35:01.256 INFO:tasks.workunit.client.0.smithi038.stdout:2/788: mkdir dd/d12/d1d/d3b/d3c/d4a/dff 0 2024-02-15T18:35:01.259 INFO:tasks.workunit.client.0.smithi038.stdout:9/979: dwrite d1/fa1 [4194304,4194304] 0 2024-02-15T18:35:01.263 INFO:tasks.workunit.client.0.smithi038.stdout:7/907: link d0/d95/c4b d0/d66/c147 0 2024-02-15T18:35:01.263 INFO:tasks.workunit.client.0.smithi038.stdout:7/908: fdatasync d0/d1b/dfd/f116 0 2024-02-15T18:35:01.263 INFO:tasks.workunit.client.0.smithi038.stdout:1/815: symlink d2/d10/d19/d5f/l110 0 2024-02-15T18:35:01.263 INFO:tasks.workunit.client.0.smithi038.stdout:7/909: dread - d0/db/d59/d15/d29/d8d/d102/f13e zero size 2024-02-15T18:35:01.267 INFO:tasks.workunit.client.0.smithi038.stdout:6/892: sync 2024-02-15T18:35:01.268 INFO:tasks.workunit.client.0.smithi038.stdout:4/874: sync 2024-02-15T18:35:01.268 INFO:tasks.workunit.client.0.smithi038.stdout:0/945: sync 2024-02-15T18:35:01.268 INFO:tasks.workunit.client.0.smithi038.stdout:4/875: write d11/d1e/d20/d31/d48/d102/dca/fed [4850881,20292] 0 2024-02-15T18:35:01.268 INFO:tasks.workunit.client.0.smithi038.stdout:4/876: stat d11/d1e/d20/f76 0 2024-02-15T18:35:01.269 INFO:tasks.workunit.client.0.smithi038.stdout:9/980: rename d1/dc/db2/fd2 to d1/da/d36/d3f/f15a 0 2024-02-15T18:35:01.273 INFO:tasks.workunit.client.0.smithi038.stdout:1/816: unlink d2/d10/d19/d5f/fa8 0 2024-02-15T18:35:01.274 INFO:tasks.workunit.client.0.smithi038.stdout:7/910: mkdir d0/db/d59/d15/d29/d8d/dcd/d134/d148 0 2024-02-15T18:35:01.274 INFO:tasks.workunit.client.0.smithi038.stdout:7/911: dread - d0/d1b/ff3 zero size 2024-02-15T18:35:01.274 INFO:tasks.workunit.client.0.smithi038.stdout:7/912: chown d0/db/d59 19 1 2024-02-15T18:35:01.274 INFO:tasks.workunit.client.0.smithi038.stdout:7/913: write d0/db/d59/d15/fe3 [221859,113857] 0 2024-02-15T18:35:01.280 INFO:tasks.workunit.client.0.smithi038.stdout:6/893: unlink d2/d4/d2c/d39/db4/d5f/fff 0 2024-02-15T18:35:01.280 INFO:tasks.workunit.client.0.smithi038.stdout:6/894: chown d2/d4/d19/d55/cd4 2 1 2024-02-15T18:35:01.280 INFO:tasks.workunit.client.0.smithi038.stdout:0/946: creat d4/d13/d53/d10c/f13f x:0 0 0 2024-02-15T18:35:01.280 INFO:tasks.workunit.client.0.smithi038.stdout:9/981: creat d1/dc/d5a/f15b x:0 0 0 2024-02-15T18:35:01.280 INFO:tasks.workunit.client.0.smithi038.stdout:4/877: creat d11/d1e/d20/d2f/f138 x:0 0 0 2024-02-15T18:35:01.280 INFO:tasks.workunit.client.0.smithi038.stdout:4/878: write d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/ddd/feb [66113,38526] 0 2024-02-15T18:35:01.280 INFO:tasks.workunit.client.0.smithi038.stdout:7/914: creat d0/db/d59/d15/d4f/f149 x:0 0 0 2024-02-15T18:35:01.281 INFO:tasks.workunit.client.0.smithi038.stdout:7/915: chown d0/d66/c83 2 1 2024-02-15T18:35:01.281 INFO:tasks.workunit.client.0.smithi038.stdout:7/916: truncate d0/d1b/d30/f40 707224 0 2024-02-15T18:35:01.281 INFO:tasks.workunit.client.0.smithi038.stdout:6/895: symlink d2/d4/d19/d55/dc0/l111 0 2024-02-15T18:35:01.282 INFO:tasks.workunit.client.0.smithi038.stdout:6/896: readlink d2/d4/d6c/d9d/lba 0 2024-02-15T18:35:01.282 INFO:tasks.workunit.client.0.smithi038.stdout:9/982: rmdir d1/dc/d2b 39 2024-02-15T18:35:01.282 INFO:tasks.workunit.client.0.smithi038.stdout:7/917: write d0/db/d19/d7d/db5/fcf [43545,57038] 0 2024-02-15T18:35:01.282 INFO:tasks.workunit.client.0.smithi038.stdout:7/918: chown d0/d95/f51 13104008 1 2024-02-15T18:35:01.282 INFO:tasks.workunit.client.0.smithi038.stdout:7/919: dread - d0/db/d59/d97/f105 zero size 2024-02-15T18:35:01.282 INFO:tasks.workunit.client.0.smithi038.stdout:0/947: dread d4/d13/d2a/d35/d6a/f9e [0,4194304] 0 2024-02-15T18:35:01.282 INFO:tasks.workunit.client.0.smithi038.stdout:5/867: dwrite d2/dcc/fea [0,4194304] 0 2024-02-15T18:35:01.283 INFO:tasks.workunit.client.0.smithi038.stdout:5/868: fdatasync d2/d3/da0/d33/f93 0 2024-02-15T18:35:01.283 INFO:tasks.workunit.client.0.smithi038.stdout:5/869: chown d2/d3/db9/dcf/ddf/fe0 524070 1 2024-02-15T18:35:01.283 INFO:tasks.workunit.client.0.smithi038.stdout:5/870: dread - d2/d3/d12/d5e/fd9 zero size 2024-02-15T18:35:01.283 INFO:tasks.workunit.client.0.smithi038.stdout:2/789: dwrite dd/d12/d3a/d67/fc1 [0,4194304] 0 2024-02-15T18:35:01.284 INFO:tasks.workunit.client.0.smithi038.stdout:2/790: stat dd/d12/d3a/f45 0 2024-02-15T18:35:01.284 INFO:tasks.workunit.client.0.smithi038.stdout:4/879: rename d11/f13 to d11/d1e/d20/d2b/d4d/f139 0 2024-02-15T18:35:01.284 INFO:tasks.workunit.client.0.smithi038.stdout:4/880: chown d11/d1e/d20/d2b/d4d/d67/d73/f10f 8 1 2024-02-15T18:35:01.284 INFO:tasks.workunit.client.0.smithi038.stdout:2/791: read dd/d12/d3a/f45 [13129,106603] 0 2024-02-15T18:35:01.284 INFO:tasks.workunit.client.0.smithi038.stdout:6/897: mkdir d2/d4/d6c/d9d/d112 0 2024-02-15T18:35:01.284 INFO:tasks.workunit.client.0.smithi038.stdout:6/898: truncate d2/d4/d19/d78/fd6 208256 0 2024-02-15T18:35:01.286 INFO:tasks.workunit.client.0.smithi038.stdout:9/983: symlink d1/da/d20/d58/d94/d13c/l15c 0 2024-02-15T18:35:01.286 INFO:tasks.workunit.client.0.smithi038.stdout:9/984: truncate d1/da/d10/d13/d3d/d4b/daa/fab 842133 0 2024-02-15T18:35:01.287 INFO:tasks.workunit.client.0.smithi038.stdout:9/985: dread d1/dc/d4f/d14d/d49/f4e [0,4194304] 0 2024-02-15T18:35:01.287 INFO:tasks.workunit.client.0.smithi038.stdout:0/948: symlink d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/ddb/l140 0 2024-02-15T18:35:01.287 INFO:tasks.workunit.client.0.smithi038.stdout:0/949: chown d4/f12 0 1 2024-02-15T18:35:01.290 INFO:tasks.workunit.client.0.smithi038.stdout:2/792: dread dd/d12/f39 [0,4194304] 0 2024-02-15T18:35:01.292 INFO:tasks.workunit.client.0.smithi038.stdout:6/899: truncate d2/d4/d2c/d39/d76/dc2/fce 1820394 0 2024-02-15T18:35:01.292 INFO:tasks.workunit.client.0.smithi038.stdout:9/986: symlink d1/dc/d4f/d14d/d49/de4/l15d 0 2024-02-15T18:35:01.293 INFO:tasks.workunit.client.0.smithi038.stdout:6/900: readlink d2/d4/d2c/d39/l3e 0 2024-02-15T18:35:01.293 INFO:tasks.workunit.client.0.smithi038.stdout:0/950: unlink d4/d13/d2a/d35/d1e/d32/d48/c115 0 2024-02-15T18:35:01.294 INFO:tasks.workunit.client.0.smithi038.stdout:2/793: truncate dd/d12/f47 105364 0 2024-02-15T18:35:01.294 INFO:tasks.workunit.client.0.smithi038.stdout:2/794: chown dd/d12/d5d/d93/fc4 168828 1 2024-02-15T18:35:01.295 INFO:tasks.workunit.client.0.smithi038.stdout:9/987: mkdir d1/dc/d4f/d14d/d49/d15e 0 2024-02-15T18:35:01.296 INFO:tasks.workunit.client.0.smithi038.stdout:9/988: mknod d1/dc/db2/c15f 0 2024-02-15T18:35:01.304 INFO:tasks.workunit.client.0.smithi038.stdout:5/871: dwrite d2/d3/db9/dcf/ddf/fe0 [0,4194304] 0 2024-02-15T18:35:01.304 INFO:tasks.workunit.client.0.smithi038.stdout:1/817: dwrite d2/d10/d19/ff5 [0,4194304] 0 2024-02-15T18:35:01.304 INFO:tasks.workunit.client.0.smithi038.stdout:0/951: dread d4/d13/d2a/d35/d1e/d32/d3b/ff3 [0,4194304] 0 2024-02-15T18:35:01.304 INFO:tasks.workunit.client.0.smithi038.stdout:0/952: stat d4/d13/f64 0 2024-02-15T18:35:01.304 INFO:tasks.workunit.client.0.smithi038.stdout:0/953: chown d4/d13/d2a/d35/d6a/fde 1655 1 2024-02-15T18:35:01.304 INFO:tasks.workunit.client.0.smithi038.stdout:0/954: fsync d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/fa9 0 2024-02-15T18:35:01.304 INFO:tasks.workunit.client.0.smithi038.stdout:0/955: stat d4/d13/d2a/d35/d1e/d32/d79 0 2024-02-15T18:35:01.311 INFO:tasks.workunit.client.0.smithi038.stdout:7/920: rename d0/db/d59/d15/d29/d8d to d0/db/d19/d7d/daa/dd0/d14a 0 2024-02-15T18:35:01.326 INFO:tasks.workunit.client.0.smithi038.stdout:5/872: mkdir d2/d48/d83/d96/d104 0 2024-02-15T18:35:01.326 INFO:tasks.workunit.client.0.smithi038.stdout:0/956: truncate d4/d13/d2a/d35/db2/fdc 1329660 0 2024-02-15T18:35:01.326 INFO:tasks.workunit.client.0.smithi038.stdout:0/957: dread - d4/d13/d53/d10c/f13f zero size 2024-02-15T18:35:01.328 INFO:tasks.workunit.client.0.smithi038.stdout:4/881: rename d11/d1e/d43/d4a to d11/d1e/d20/d31/d48/d102/dca/d13a 0 2024-02-15T18:35:01.328 INFO:tasks.workunit.client.0.smithi038.stdout:4/882: chown d11/d1e/d20/d2b/d35/f89 1010615 1 2024-02-15T18:35:01.328 INFO:tasks.workunit.client.0.smithi038.stdout:7/921: creat d0/f14b x:0 0 0 2024-02-15T18:35:01.329 INFO:tasks.workunit.client.0.smithi038.stdout:7/922: truncate d0/d68/f126 245192 0 2024-02-15T18:35:01.332 INFO:tasks.workunit.client.0.smithi038.stdout:5/873: symlink d2/d3/d12/de6/l105 0 2024-02-15T18:35:01.332 INFO:tasks.workunit.client.0.smithi038.stdout:5/874: readlink d2/d3/d12/de6/l105 0 2024-02-15T18:35:01.333 INFO:tasks.workunit.client.0.smithi038.stdout:0/958: link d4/d13/l2f d4/d13/d2a/d35/d1e/d32/d79/d7d/da3/d118/l141 0 2024-02-15T18:35:01.333 INFO:tasks.workunit.client.0.smithi038.stdout:0/959: write d4/d13/d2a/d35/d1e/d32/d79/d7d/f96 [1572128,52089] 0 2024-02-15T18:35:01.334 INFO:tasks.workunit.client.0.smithi038.stdout:0/960: readlink d4/d13/d2a/d35/d1e/d32/l4a 0 2024-02-15T18:35:01.334 INFO:tasks.workunit.client.0.smithi038.stdout:4/883: creat d11/d1e/d20/d2b/d4d/d67/f13b x:0 0 0 2024-02-15T18:35:01.334 INFO:tasks.workunit.client.0.smithi038.stdout:4/884: chown d11/d1e/d20/d31/d48/dac/dbe/d40/d6f 416 1 2024-02-15T18:35:01.335 INFO:tasks.workunit.client.0.smithi038.stdout:9/989: dwrite d1/dc/d4f/d14d/f7c [4194304,4194304] 0 2024-02-15T18:35:01.335 INFO:tasks.workunit.client.0.smithi038.stdout:9/990: chown d1/dc/faf 4 1 2024-02-15T18:35:01.335 INFO:tasks.workunit.client.0.smithi038.stdout:7/923: creat d0/db/d19/d7d/daa/dd0/d14a/dcd/d134/f14c x:0 0 0 2024-02-15T18:35:01.336 INFO:tasks.workunit.client.0.smithi038.stdout:6/901: dwrite d2/d10/fa8 [0,4194304] 0 2024-02-15T18:35:01.336 INFO:tasks.workunit.client.0.smithi038.stdout:1/818: dwrite d2/d39/f8f [4194304,4194304] 0 2024-02-15T18:35:01.336 INFO:tasks.workunit.client.0.smithi038.stdout:1/819: chown d2/f11 19 1 2024-02-15T18:35:01.338 INFO:tasks.workunit.client.0.smithi038.stdout:5/875: write d2/dcc/f70 [893800,64718] 0 2024-02-15T18:35:01.338 INFO:tasks.workunit.client.0.smithi038.stdout:5/876: chown d2/d3/da0/d33/lb7 539052 1 2024-02-15T18:35:01.338 INFO:tasks.workunit.client.0.smithi038.stdout:2/795: dwrite dd/d12/d3a/d67/f29 [0,4194304] 0 2024-02-15T18:35:01.339 INFO:tasks.workunit.client.0.smithi038.stdout:0/961: symlink d4/db3/dc7/l142 0 2024-02-15T18:35:01.339 INFO:tasks.workunit.client.0.smithi038.stdout:4/885: rmdir d11/d1e/d20/d2b/d4d/d67/dbf 39 2024-02-15T18:35:01.345 INFO:tasks.workunit.client.0.smithi038.stdout:3/798: sync 2024-02-15T18:35:01.346 INFO:tasks.workunit.client.0.smithi038.stdout:5/877: dread d2/d3/da0/fae [0,4194304] 0 2024-02-15T18:35:01.347 INFO:tasks.workunit.client.0.smithi038.stdout:3/799: write d2/d1b/d3c/d3a/d55/fb1 [3846001,54893] 0 2024-02-15T18:35:01.347 INFO:tasks.workunit.client.0.smithi038.stdout:3/800: stat d2/d1b/d3c/d3a/f68 0 2024-02-15T18:35:01.348 INFO:tasks.workunit.client.0.smithi038.stdout:3/801: write d2/d1b/d8c/f99 [1178697,23797] 0 2024-02-15T18:35:01.349 INFO:tasks.workunit.client.0.smithi038.stdout:9/991: mkdir d1/da/d10/d13/d3d/d43/d4d/dfd/d160 0 2024-02-15T18:35:01.351 INFO:tasks.workunit.client.0.smithi038.stdout:7/924: dwrite d0/db/fc [0,4194304] 0 2024-02-15T18:35:01.353 INFO:tasks.workunit.client.0.smithi038.stdout:6/902: link d2/d4/d19/d55/d5c/f70 d2/d10/db6/f113 0 2024-02-15T18:35:01.356 INFO:tasks.workunit.client.0.smithi038.stdout:1/820: creat d2/d10/d32/d53/d10c/f111 x:0 0 0 2024-02-15T18:35:01.359 INFO:tasks.workunit.client.0.smithi038.stdout:2/796: creat dd/d12/d59/db2/f100 x:0 0 0 2024-02-15T18:35:01.361 INFO:tasks.workunit.client.0.smithi038.stdout:4/886: getdents d11/d1e/d20/d2b/d4d/d67/da1 0 2024-02-15T18:35:01.361 INFO:tasks.workunit.client.0.smithi038.stdout:4/887: chown d11/f1d 1 1 2024-02-15T18:35:01.363 INFO:tasks.workunit.client.0.smithi038.stdout:0/962: rename d4/d13/d53/fa7 to d4/d13/d2a/d35/d1e/d32/d79/d7d/d13a/f143 0 2024-02-15T18:35:01.363 INFO:tasks.workunit.client.0.smithi038.stdout:5/878: symlink d2/d3/l106 0 2024-02-15T18:35:01.363 INFO:tasks.workunit.client.0.smithi038.stdout:3/802: mknod d2/d1b/d3c/d3a/dbb/cfd 0 2024-02-15T18:35:01.365 INFO:tasks.workunit.client.0.smithi038.stdout:7/925: link d0/db/d19/c38 d0/db/d59/d15/d29/d7e/d12a/c14d 0 2024-02-15T18:35:01.368 INFO:tasks.workunit.client.0.smithi038.stdout:6/903: creat d2/d4/d19/f114 x:0 0 0 2024-02-15T18:35:01.371 INFO:tasks.workunit.client.0.smithi038.stdout:1/821: rmdir d2/d10/d19 39 2024-02-15T18:35:01.372 INFO:tasks.workunit.client.0.smithi038.stdout:4/888: dread d11/d1e/d20/d31/f91 [0,4194304] 0 2024-02-15T18:35:01.372 INFO:tasks.workunit.client.0.smithi038.stdout:2/797: truncate dd/d12/f26 1773799 0 2024-02-15T18:35:01.373 INFO:tasks.workunit.client.0.smithi038.stdout:9/992: dwrite d1/da/d20/d58/d94/d13c/d11c/db8/fbb [0,4194304] 0 2024-02-15T18:35:01.373 INFO:tasks.workunit.client.0.smithi038.stdout:2/798: dread dd/d12/d5d/d54/fa4 [0,4194304] 0 2024-02-15T18:35:01.374 INFO:tasks.workunit.client.0.smithi038.stdout:5/879: rmdir d2/d48/d83/d96/d104 0 2024-02-15T18:35:01.374 INFO:tasks.workunit.client.0.smithi038.stdout:9/993: write d1/da/d10/d13/d3d/d43/d4d/dfd/f14c [4710214,128072] 0 2024-02-15T18:35:01.377 INFO:tasks.workunit.client.0.smithi038.stdout:6/904: rmdir d2/d4/d19/d55/dc0 39 2024-02-15T18:35:01.379 INFO:tasks.workunit.client.0.smithi038.stdout:1/822: symlink d2/d2e/d8c/l112 0 2024-02-15T18:35:01.379 INFO:tasks.workunit.client.0.smithi038.stdout:4/889: creat d11/d1e/d20/d2f/f13c x:0 0 0 2024-02-15T18:35:01.379 INFO:tasks.workunit.client.0.smithi038.stdout:4/890: read - d11/d1e/d43/d5e/d62/f120 zero size 2024-02-15T18:35:01.379 INFO:tasks.workunit.client.0.smithi038.stdout:4/891: fsync d11/d1e/d20/d2b/d4d/d67/da1/f117 0 2024-02-15T18:35:01.380 INFO:tasks.workunit.client.0.smithi038.stdout:2/799: creat dd/d12/d6b/f101 x:0 0 0 2024-02-15T18:35:01.380 INFO:tasks.workunit.client.0.smithi038.stdout:5/880: rmdir d2/d48 39 2024-02-15T18:35:01.380 INFO:tasks.workunit.client.0.smithi038.stdout:5/881: write d2/d3/dd0/ffa [545655,119202] 0 2024-02-15T18:35:01.380 INFO:tasks.workunit.client.0.smithi038.stdout:5/882: read - d2/d3/d5f/fce zero size 2024-02-15T18:35:01.381 INFO:tasks.workunit.client.0.smithi038.stdout:9/994: creat d1/dc/d2b/f161 x:0 0 0 2024-02-15T18:35:01.382 INFO:tasks.workunit.client.0.smithi038.stdout:7/926: rename d0/db/d59/dc8/dd6/d61/d10e/f10d to d0/f14e 0 2024-02-15T18:35:01.383 INFO:tasks.workunit.client.0.smithi038.stdout:6/905: symlink d2/d4/d2c/d39/d76/l115 0 2024-02-15T18:35:01.385 INFO:tasks.workunit.client.0.smithi038.stdout:1/823: mkdir d2/d2e/d113 0 2024-02-15T18:35:01.385 INFO:tasks.workunit.client.0.smithi038.stdout:1/824: write d2/d10/d19/d5f/fdf [714999,43325] 0 2024-02-15T18:35:01.385 INFO:tasks.workunit.client.0.smithi038.stdout:1/825: readlink d2/d2e/l5b 0 2024-02-15T18:35:01.385 INFO:tasks.workunit.client.0.smithi038.stdout:4/892: mkdir d11/d1e/d20/d31/d48/d102/d13d 0 2024-02-15T18:35:01.385 INFO:tasks.workunit.client.0.smithi038.stdout:4/893: chown d11/d1e/d20/d31/d48/l5d 81773 1 2024-02-15T18:35:01.386 INFO:tasks.workunit.client.0.smithi038.stdout:4/894: dread - d11/d1e/d43/d5e/d62/f120 zero size 2024-02-15T18:35:01.386 INFO:tasks.workunit.client.0.smithi038.stdout:5/883: dread d2/d3/db9/dcf/faa [0,4194304] 0 2024-02-15T18:35:01.387 INFO:tasks.workunit.client.0.smithi038.stdout:2/800: symlink dd/d12/d59/dce/dea/dee/l102 0 2024-02-15T18:35:01.388 INFO:tasks.workunit.client.0.smithi038.stdout:1/826: rmdir d2/d69/da9 39 2024-02-15T18:35:01.388 INFO:tasks.workunit.client.0.smithi038.stdout:6/906: rmdir d2/d4/d19/d55/d5c 39 2024-02-15T18:35:01.389 INFO:tasks.workunit.client.0.smithi038.stdout:9/995: truncate d1/da/fed 3927339 0 2024-02-15T18:35:01.389 INFO:tasks.workunit.client.0.smithi038.stdout:0/963: dwrite d4/d13/d19/d137/f138 [0,4194304] 0 2024-02-15T18:35:01.390 INFO:tasks.workunit.client.0.smithi038.stdout:5/884: dread d2/fc [0,4194304] 0 2024-02-15T18:35:01.391 INFO:tasks.workunit.client.0.smithi038.stdout:3/803: dwrite d2/d1b/d3c/d3a/d55/d65/fd0 [0,4194304] 0 2024-02-15T18:35:01.396 INFO:tasks.workunit.client.0.smithi038.stdout:4/895: rename d11/d1e/d20/d31/d8d/fb2 to d11/d1e/d20/d31/da2/d118/d7e/f13e 0 2024-02-15T18:35:01.396 INFO:tasks.workunit.client.0.smithi038.stdout:9/996: creat d1/dc/d4f/d14d/d49/de4/f162 x:0 0 0 2024-02-15T18:35:01.397 INFO:tasks.workunit.client.0.smithi038.stdout:1/827: creat d2/d69/da9/dad/df3/f114 x:0 0 0 2024-02-15T18:35:01.398 INFO:tasks.workunit.client.0.smithi038.stdout:1/828: readlink d2/d69/da9/lfe 0 2024-02-15T18:35:01.403 INFO:tasks.workunit.client.0.smithi038.stdout:5/885: mknod d2/d64/d99/db6/c107 0 2024-02-15T18:35:01.403 INFO:tasks.workunit.client.0.smithi038.stdout:5/886: chown d2/d3/l9 3 1 2024-02-15T18:35:01.406 INFO:tasks.workunit.client.0.smithi038.stdout:3/804: symlink d2/db8/dce/de6/lfe 0 2024-02-15T18:35:01.406 INFO:tasks.workunit.client.0.smithi038.stdout:3/805: chown d2/d1b/ffb 77982 1 2024-02-15T18:35:01.406 INFO:tasks.workunit.client.0.smithi038.stdout:3/806: write d2/d1b/d3c/d3a/fe2 [1340679,105033] 0 2024-02-15T18:35:01.409 INFO:tasks.workunit.client.0.smithi038.stdout:0/964: dwrite d4/f134 [0,4194304] 0 2024-02-15T18:35:01.410 INFO:tasks.workunit.client.0.smithi038.stdout:2/801: dwrite dd/d12/d6b/f101 [0,4194304] 0 2024-02-15T18:35:01.410 INFO:tasks.workunit.client.0.smithi038.stdout:2/802: stat dd/d12/d59/db2/f100 0 2024-02-15T18:35:01.412 INFO:tasks.workunit.client.0.smithi038.stdout:6/907: rename d2/d4/d19/d78/dbf/c10c to d2/d4/d2c/d39/d76/dc2/c116 0 2024-02-15T18:35:01.412 INFO:tasks.workunit.client.0.smithi038.stdout:6/908: chown d2/d4/d6f/f88 28544403 1 2024-02-15T18:35:01.413 INFO:tasks.workunit.client.0.smithi038.stdout:9/997: mkdir d1/da/d20/d58/d94/d13c/d11c/d163 0 2024-02-15T18:35:01.414 INFO:tasks.workunit.client.0.smithi038.stdout:1/829: creat d2/d69/da9/dad/de2/f115 x:0 0 0 2024-02-15T18:35:01.414 INFO:tasks.workunit.client.0.smithi038.stdout:1/830: fsync d2/d10/d32/d53/d82/ffa 0 2024-02-15T18:35:01.414 INFO:tasks.workunit.client.0.smithi038.stdout:1/831: readlink d2/d10/d19/d22/d54/dc0/d3c/l6f 0 2024-02-15T18:35:01.418 INFO:tasks.workunit.client.0.smithi038.stdout:2/803: rename dd/d12/d59/d2b/d6a/ld5 to dd/d12/d1d/d3b/d3c/d73/l103 0 2024-02-15T18:35:01.418 INFO:tasks.workunit.client.0.smithi038.stdout:2/804: chown dd/d12/d3a/d5e/d65/dcd/df1 1 1 2024-02-15T18:35:01.419 INFO:tasks.workunit.client.0.smithi038.stdout:6/909: link d2/d4/d19/d78/fd8 d2/d4/d19/d55/f117 0 2024-02-15T18:35:01.419 INFO:tasks.workunit.client.0.smithi038.stdout:6/910: fdatasync d2/f58 0 2024-02-15T18:35:01.420 INFO:tasks.workunit.client.0.smithi038.stdout:1/832: symlink d2/d69/d8b/l116 0 2024-02-15T18:35:01.420 INFO:tasks.workunit.client.0.smithi038.stdout:2/805: write dd/d12/d59/f21 [2923873,75002] 0 2024-02-15T18:35:01.422 INFO:tasks.workunit.client.0.smithi038.stdout:6/911: read d2/f5b [2481964,86939] 0 2024-02-15T18:35:01.422 INFO:tasks.workunit.client.0.smithi038.stdout:9/998: rename d1/da/d10/d82/fc5 to d1/da/d10/d19/f164 0 2024-02-15T18:35:01.422 INFO:tasks.workunit.client.0.smithi038.stdout:0/965: dread d4/d13/d2a/f44 [0,4194304] 0 2024-02-15T18:35:01.422 INFO:tasks.workunit.client.0.smithi038.stdout:0/966: chown d4/d13/d2a/d35/d1e/d32/d48/d97/faf 241796 1 2024-02-15T18:35:01.422 INFO:tasks.workunit.client.0.smithi038.stdout:0/967: fsync d4/d13/d2a/d35/d1e/d32/d79/d7d/d7f/f108 0 2024-02-15T18:35:01.422 INFO:tasks.workunit.client.0.smithi038.stdout:4/896: dwrite d11/d1e/d20/d31/da2/d118/f82 [0,4194304] 0 2024-02-15T18:35:01.422 INFO:tasks.workunit.client.0.smithi038.stdout:5/887: dwrite d2/d3/db9/dcf/fc2 [4194304,4194304] 0 2024-02-15T18:35:01.423 INFO:tasks.workunit.client.0.smithi038.stdout:0/968: read d4/d13/d2a/d35/d1e/d32/d3b/f60 [3767223,78775] 0 2024-02-15T18:35:01.423 INFO:tasks.workunit.client.0.smithi038.stdout:0/969: fsync d4/d13/d19/d137/f138 0 2024-02-15T18:35:01.423 INFO:tasks.workunit.client.0.smithi038.stdout:2/806: mkdir dd/d12/d5d/d54/dcb/de1/d104 0 2024-02-15T18:35:01.423 INFO:tasks.workunit.client.0.smithi038.stdout:9/999: dread d1/f14 [0,4194304] 0 2024-02-15T18:35:01.428 INFO:tasks.workunit.client.0.smithi038.stdout:1/833: rename d2/f11 to d2/d10/d32/d53/d82/ded/f117 0 2024-02-15T18:35:01.433 INFO:tasks.workunit.client.0.smithi038.stdout:4/897: symlink d11/d1e/d20/d31/d48/d102/dca/l13f 0 2024-02-15T18:35:01.434 INFO:tasks.workunit.client.0.smithi038.stdout:5/888: mknod d2/d9a/c108 0 2024-02-15T18:35:01.435 INFO:tasks.workunit.client.0.smithi038.stdout:4/898: write d11/d1e/d20/d31/da2/d118/f82 [956565,117776] 0 2024-02-15T18:35:01.435 INFO:tasks.workunit.client.0.smithi038.stdout:4/899: readlink d11/d1e/la8 0 2024-02-15T18:35:01.441 INFO:tasks.workunit.client.0.smithi038.stdout:0/970: creat d4/d13/d2a/f144 x:0 0 0 2024-02-15T18:35:01.441 INFO:tasks.workunit.client.0.smithi038.stdout:0/971: chown d4/d13 26 1 2024-02-15T18:35:01.441 INFO:tasks.workunit.client.0.smithi038.stdout:2/807: rmdir dd/d12/d59/d2b 39 2024-02-15T18:35:01.442 INFO:tasks.workunit.client.0.smithi038.stdout:6/912: rename d2/d4/d2c/d39/db4/ddc to d2/d4/d19/d78/d118 0 2024-02-15T18:35:01.442 INFO:tasks.workunit.client.0.smithi038.stdout:1/834: rmdir d2/d10/d19/d22/d36/d94 39 2024-02-15T18:35:01.442 INFO:tasks.workunit.client.0.smithi038.stdout:1/835: read - d2/d10/d62/fa6 zero size 2024-02-15T18:35:01.445 INFO:tasks.workunit.client.0.smithi038.stdout:1/836: write d2/d10/d19/d22/f30 [515108,11526] 0 2024-02-15T18:35:01.445 INFO:tasks.workunit.client.0.smithi038.stdout:2/808: creat dd/d12/d3a/d5e/d65/f105 x:0 0 0 2024-02-15T18:35:01.445 INFO:tasks.workunit.client.0.smithi038.stdout:2/809: write dd/d12/d3a/d67/d4f/f71 [428626,26937] 0 2024-02-15T18:35:01.445 INFO:tasks.workunit.client.0.smithi038.stdout:2/810: truncate dd/d12/d6b/da5/dd8/fc6 487600 0 2024-02-15T18:35:01.445 INFO:tasks.workunit.client.0.smithi038.stdout:2/811: chown dd/d12/d6b/da5/dd8/df9 543476335 1 2024-02-15T18:35:01.446 INFO:tasks.workunit.client.0.smithi038.stdout:6/913: creat d2/d4/d19/d78/dbf/f119 x:0 0 0 2024-02-15T18:35:01.446 INFO:tasks.workunit.client.0.smithi038.stdout:6/914: write d2/d4/d19/d78/dbf/fe5 [1988398,66730] 0 2024-02-15T18:35:01.447 INFO:tasks.workunit.client.0.smithi038.stdout:5/889: creat d2/d64/d99/dbc/f109 x:0 0 0 2024-02-15T18:35:01.448 INFO:tasks.workunit.client.0.smithi038.stdout:0/972: link d4/d13/d2a/d35/d1e/d32/fb6 d4/f145 0 2024-02-15T18:35:01.448 INFO:tasks.workunit.client.0.smithi038.stdout:4/900: dwrite d11/d1e/d20/d2b/d4d/f95 [0,4194304] 0 2024-02-15T18:35:01.449 INFO:tasks.workunit.client.0.smithi038.stdout:5/890: read d2/d64/d99/db6/fe8 [1205089,87364] 0 2024-02-15T18:35:01.449 INFO:tasks.workunit.client.0.smithi038.stdout:1/837: rmdir d2/d10/d10b 0 2024-02-15T18:35:01.451 INFO:tasks.workunit.client.0.smithi038.stdout:2/812: rmdir dd/d12/d5d/d33/dd7 0 2024-02-15T18:35:01.451 INFO:tasks.workunit.client.0.smithi038.stdout:2/813: fsync dd/d12/d5d/d54/dcb/fec 0 2024-02-15T18:35:01.451 INFO:tasks.workunit.client.0.smithi038.stdout:0/973: truncate d4/d13/d2a/f4b 2380476 0 2024-02-15T18:35:01.454 INFO:tasks.workunit.client.0.smithi038.stdout:4/901: creat d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/d11d/f140 x:0 0 0 2024-02-15T18:35:01.454 INFO:tasks.workunit.client.0.smithi038.stdout:4/902: fsync d11/d1e/d43/f63 0 2024-02-15T18:35:01.456 INFO:tasks.workunit.client.0.smithi038.stdout:5/891: unlink d2/d30/lb3 0 2024-02-15T18:35:01.460 INFO:tasks.workunit.client.0.smithi038.stdout:5/892: dread d2/d30/d57/f90 [0,4194304] 0 2024-02-15T18:35:01.460 INFO:tasks.workunit.client.0.smithi038.stdout:1/838: dwrite d2/d69/da9/dad/de2/f115 [0,4194304] 0 2024-02-15T18:35:01.466 INFO:tasks.workunit.client.0.smithi038.stdout:0/974: creat d4/d13/d2a/d35/d1e/d32/d79/d7d/dda/f146 x:0 0 0 2024-02-15T18:35:01.467 INFO:tasks.workunit.client.0.smithi038.stdout:0/975: chown d4/d13/d2a/d35/d1e/d32/d3b/f117 9400 1 2024-02-15T18:35:01.467 INFO:tasks.workunit.client.0.smithi038.stdout:5/893: creat d2/d3/d12/de6/f10a x:0 0 0 2024-02-15T18:35:01.469 INFO:tasks.workunit.client.0.smithi038.stdout:1/839: mkdir d2/d10/d32/d53/d118 0 2024-02-15T18:35:01.470 INFO:tasks.workunit.client.0.smithi038.stdout:1/840: dread - d2/d10/d19/fcf zero size 2024-02-15T18:35:01.470 INFO:tasks.workunit.client.0.smithi038.stdout:4/903: dwrite d11/f1b [0,4194304] 0 2024-02-15T18:35:01.470 INFO:tasks.workunit.client.0.smithi038.stdout:4/904: write d11/d1e/d20/d31/d48/dac/dbe/d40/dde/f115 [254663,118864] 0 2024-02-15T18:35:01.470 INFO:tasks.workunit.client.0.smithi038.stdout:5/894: mknod d2/d3/db9/dcf/dd8/c10b 0 2024-02-15T18:35:01.473 INFO:tasks.workunit.client.0.smithi038.stdout:5/895: truncate d2/d3/f6f 502673 0 2024-02-15T18:35:01.474 INFO:tasks.workunit.client.0.smithi038.stdout:1/841: creat d2/d10/d19/d22/d36/d94/dc1/dd5/f119 x:0 0 0 2024-02-15T18:35:01.475 INFO:tasks.workunit.client.0.smithi038.stdout:0/976: write d4/d13/d2a/f44 [6046897,70793] 0 2024-02-15T18:35:01.482 INFO:tasks.workunit.client.0.smithi038.stdout:4/905: dread d11/d1e/d20/d31/d48/d60/f12b [0,4194304] 0 2024-02-15T18:35:01.483 INFO:tasks.workunit.client.0.smithi038.stdout:0/977: creat d4/d13/d2a/d35/d1e/d32/f147 x:0 0 0 2024-02-15T18:35:01.484 INFO:tasks.workunit.client.0.smithi038.stdout:4/906: mkdir d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/d141 0 2024-02-15T18:35:01.484 INFO:tasks.workunit.client.0.smithi038.stdout:4/907: dread - d11/d1e/d20/d31/da2/d118/d7e/d105/f127 zero size 2024-02-15T18:35:01.484 INFO:tasks.workunit.client.0.smithi038.stdout:4/908: chown d11/d1e/d20/d31/d48/d102/dca/d13a/dbd 48576 1 2024-02-15T18:35:01.486 INFO:tasks.workunit.client.0.smithi038.stdout:0/978: link d4/d13/d2a/d35/c5f d4/d13/d53/c148 0 2024-02-15T18:35:01.486 INFO:tasks.workunit.client.0.smithi038.stdout:2/814: dwrite dd/f41 [0,4194304] 0 2024-02-15T18:35:01.487 INFO:tasks.workunit.client.0.smithi038.stdout:2/815: write dd/d12/d59/db2/fbb [4167785,111328] 0 2024-02-15T18:35:01.488 INFO:tasks.workunit.client.0.smithi038.stdout:1/842: dwrite d2/d10/d19/d5f/d92/fb8 [0,4194304] 0 2024-02-15T18:35:01.489 INFO:tasks.workunit.client.0.smithi038.stdout:0/979: unlink d4/d13/d2a/d35/d1e/d32/d3b/daa/df8/fd1 0 2024-02-15T18:35:01.493 INFO:tasks.workunit.client.0.smithi038.stdout:5/896: rmdir d2 39 2024-02-15T18:35:01.497 INFO:tasks.workunit.client.0.smithi038.stdout:2/816: mknod dd/d12/d5d/d54/dcb/c106 0 2024-02-15T18:35:01.498 INFO:tasks.workunit.client.0.smithi038.stdout:1/843: truncate d2/d69/f83 349624 0 2024-02-15T18:35:01.500 INFO:tasks.workunit.client.0.smithi038.stdout:0/980: rename d4/d13/d53/c75 to d4/d13/d2a/d35/d1e/c149 0 2024-02-15T18:35:01.502 INFO:tasks.workunit.client.0.smithi038.stdout:5/897: creat d2/d3/da0/f10c x:0 0 0 2024-02-15T18:35:01.502 INFO:tasks.workunit.client.0.smithi038.stdout:5/898: dread - d2/d3/db9/de3/de4/dff/f100 zero size 2024-02-15T18:35:01.502 INFO:tasks.workunit.client.0.smithi038.stdout:5/899: chown d2/d3/db9/cbf 11846761 1 2024-02-15T18:35:01.503 INFO:tasks.workunit.client.0.smithi038.stdout:1/844: unlink d2/d10/d62/c106 0 2024-02-15T18:35:01.503 INFO:tasks.workunit.client.0.smithi038.stdout:4/909: dwrite d11/d1e/d20/d2f/fd3 [0,4194304] 0 2024-02-15T18:35:01.503 INFO:tasks.workunit.client.0.smithi038.stdout:4/910: chown d11/d1e/d20/d31/da2/d118/d7e/fd4 501 1 2024-02-15T18:35:01.504 INFO:tasks.workunit.client.0.smithi038.stdout:5/900: rename d2/dcc/l58 to d2/d3/d5f/l10d 0 2024-02-15T18:35:01.504 INFO:tasks.workunit.client.0.smithi038.stdout:1/845: rename d2/d10 to d2/d10/d19/d22/d36/d94/d11a 22 2024-02-15T18:35:01.504 INFO:tasks.workunit.client.0.smithi038.stdout:5/901: write d2/d3/db9/fcb [859967,82880] 0 2024-02-15T18:35:01.505 INFO:tasks.workunit.client.0.smithi038.stdout:2/817: write dd/d12/d1d/d3b/f4b [74448,46415] 0 2024-02-15T18:35:01.506 INFO:tasks.workunit.client.0.smithi038.stdout:2/818: truncate dd/d12/d3a/d67/d4f/d5a/f6e 8832159 0 2024-02-15T18:35:01.515 INFO:tasks.workunit.client.0.smithi038.stdout:4/911: creat d11/d1e/d20/d31/d48/dac/f142 x:0 0 0 2024-02-15T18:35:01.517 INFO:tasks.workunit.client.0.smithi038.stdout:1/846: creat d2/d69/d8b/f11b x:0 0 0 2024-02-15T18:35:01.518 INFO:tasks.workunit.client.0.smithi038.stdout:5/902: creat d2/d48/f10e x:0 0 0 2024-02-15T18:35:01.521 INFO:tasks.workunit.client.0.smithi038.stdout:4/912: mkdir d11/d1e/d20/d31/d7a/d143 0 2024-02-15T18:35:01.522 INFO:tasks.workunit.client.0.smithi038.stdout:1/847: mkdir d2/d11c 0 2024-02-15T18:35:01.522 INFO:tasks.workunit.client.0.smithi038.stdout:5/903: creat d2/d48/f10f x:0 0 0 2024-02-15T18:35:01.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:00.318711+0000 mgr.smithi038.khesbl (mgr.14162) 340 : cluster [DBG] pgmap v282: 65 pgs: 65 active+clean; 1.5 GiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 102 MiB/s rd, 192 MiB/s wr, 596 op/s 2024-02-15T18:35:01.523 INFO:tasks.workunit.client.0.smithi038.stdout:4/913: creat d11/d1e/d20/d2b/d65/f144 x:0 0 0 2024-02-15T18:35:01.524 INFO:tasks.workunit.client.0.smithi038.stdout:1/848: symlink d2/d10/d19/d22/d36/d94/l11d 0 2024-02-15T18:35:01.525 INFO:tasks.workunit.client.0.smithi038.stdout:4/914: rename d11/d1e/d43/d5e/dfe/ffb to d11/d1e/d20/d2b/d4d/d67/da1/d11c/f145 0 2024-02-15T18:35:01.525 INFO:tasks.workunit.client.0.smithi038.stdout:4/915: write d11/d1e/d20/d31/da2/d118/d7e/d105/f110 [59648,78706] 0 2024-02-15T18:35:01.526 INFO:tasks.workunit.client.0.smithi038.stdout:1/849: stat d2/d10/d19/d22/d54/c84 0 2024-02-15T18:35:01.527 INFO:tasks.workunit.client.0.smithi038.stdout:2/819: dwrite dd/d12/d1d/f2e [0,4194304] 0 2024-02-15T18:35:01.527 INFO:tasks.workunit.client.0.smithi038.stdout:2/820: chown dd/d12/d5d/d54/c99 219806 1 2024-02-15T18:35:01.527 INFO:tasks.workunit.client.0.smithi038.stdout:2/821: write dd/d12/d6b/da5/fda [504128,91992] 0 2024-02-15T18:35:01.527 INFO:tasks.workunit.client.0.smithi038.stdout:4/916: mkdir d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/d11d/d146 0 2024-02-15T18:35:01.528 INFO:tasks.workunit.client.0.smithi038.stdout:1/850: rmdir d2/d10/d19/d2f/d3e/d104 0 2024-02-15T18:35:01.529 INFO:tasks.workunit.client.0.smithi038.stdout:2/822: creat dd/d12/d3a/d5e/d65/dcd/f107 x:0 0 0 2024-02-15T18:35:01.531 INFO:tasks.workunit.client.0.smithi038.stdout:7/927: sync 2024-02-15T18:35:01.531 INFO:tasks.workunit.client.0.smithi038.stdout:7/928: fsync d0/db/d59/dc8/dd6/f82 0 2024-02-15T18:35:01.533 INFO:tasks.workunit.client.0.smithi038.stdout:4/917: link d11/d1e/d20/d2b/d65/f144 d11/d1e/d20/d31/f147 0 2024-02-15T18:35:01.534 INFO:tasks.workunit.client.0.smithi038.stdout:1/851: symlink d2/d10/d19/d22/d36/l11e 0 2024-02-15T18:35:01.535 INFO:tasks.workunit.client.0.smithi038.stdout:7/929: creat d0/db/d59/d15/d109/f14f x:0 0 0 2024-02-15T18:35:01.536 INFO:tasks.workunit.client.0.smithi038.stdout:4/918: creat d11/d1e/d20/d31/d48/d102/d13d/f148 x:0 0 0 2024-02-15T18:35:01.536 INFO:tasks.workunit.client.0.smithi038.stdout:1/852: creat d2/d69/d8b/f11f x:0 0 0 2024-02-15T18:35:01.536 INFO:tasks.workunit.client.0.smithi038.stdout:1/853: chown d2/d69/da9/dad/df3 20297557 1 2024-02-15T18:35:01.537 INFO:tasks.workunit.client.0.smithi038.stdout:1/854: chown d2/d10/d19/d22/c29 420944 1 2024-02-15T18:35:01.538 INFO:tasks.workunit.client.0.smithi038.stdout:2/823: link dd/d12/d59/d2b/d6a/l9a dd/l108 0 2024-02-15T18:35:01.539 INFO:tasks.workunit.client.0.smithi038.stdout:7/930: symlink d0/db/d59/dc8/dd6/d61/d90/dc0/d128/l150 0 2024-02-15T18:35:01.539 INFO:tasks.workunit.client.0.smithi038.stdout:7/931: fsync d0/db/d59/f1a 0 2024-02-15T18:35:01.539 INFO:tasks.workunit.client.0.smithi038.stdout:5/904: dwrite d2/d3/da0/d33/f93 [0,4194304] 0 2024-02-15T18:35:01.539 INFO:tasks.workunit.client.0.smithi038.stdout:4/919: creat d11/d1e/d20/d2b/d65/d133/f149 x:0 0 0 2024-02-15T18:35:01.540 INFO:tasks.workunit.client.0.smithi038.stdout:4/920: write d11/d1e/d20/d2b/d35/f89 [5035637,70598] 0 2024-02-15T18:35:01.540 INFO:tasks.workunit.client.0.smithi038.stdout:2/824: mknod dd/d12/d1d/d3b/c109 0 2024-02-15T18:35:01.540 INFO:tasks.workunit.client.0.smithi038.stdout:2/825: write dd/d12/d59/db2/fe3 [977752,107779] 0 2024-02-15T18:35:01.541 INFO:tasks.workunit.client.0.smithi038.stdout:7/932: mknod d0/d1b/dfd/c151 0 2024-02-15T18:35:01.541 INFO:tasks.workunit.client.0.smithi038.stdout:7/933: stat d0/db/d19/d7d/db5/fcf 0 2024-02-15T18:35:01.543 INFO:tasks.workunit.client.0.smithi038.stdout:5/905: link d2/d48/d83/d96/fd3 d2/d64/d99/dbc/f110 0 2024-02-15T18:35:01.544 INFO:tasks.workunit.client.0.smithi038.stdout:4/921: creat d11/d1e/d20/d2b/d4d/d67/f14a x:0 0 0 2024-02-15T18:35:01.544 INFO:tasks.workunit.client.0.smithi038.stdout:7/934: mknod d0/db/d59/dc8/c152 0 2024-02-15T18:35:01.545 INFO:tasks.workunit.client.0.smithi038.stdout:7/935: chown d0/db/d59/dc8/dd6/d61/fb6 227091959 1 2024-02-15T18:35:01.547 INFO:tasks.workunit.client.0.smithi038.stdout:4/922: getdents d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/ddd/dba 0 2024-02-15T18:35:01.548 INFO:tasks.workunit.client.0.smithi038.stdout:7/936: mkdir d0/db/d59/d15/d4f/d153 0 2024-02-15T18:35:01.549 INFO:tasks.workunit.client.0.smithi038.stdout:4/923: mkdir d11/d1e/d20/d31/d48/d11e/d14b 0 2024-02-15T18:35:01.551 INFO:tasks.workunit.client.0.smithi038.stdout:4/924: creat d11/d1e/d20/d31/da2/f14c x:0 0 0 2024-02-15T18:35:01.553 INFO:tasks.workunit.client.0.smithi038.stdout:4/925: mknod d11/d1e/d20/d31/da2/d118/c14d 0 2024-02-15T18:35:01.560 INFO:tasks.workunit.client.0.smithi038.stdout:4/926: read d11/d1e/d20/d2b/d4d/d67/dbf/f10e [2889647,17121] 0 2024-02-15T18:35:01.560 INFO:tasks.workunit.client.0.smithi038.stdout:4/927: chown d11/d1e/d20/d31/da2 148987 1 2024-02-15T18:35:01.565 INFO:tasks.workunit.client.0.smithi038.stdout:7/937: write d0/d66/f35 [6976502,60400] 0 2024-02-15T18:35:01.566 INFO:tasks.workunit.client.0.smithi038.stdout:7/938: mknod d0/db/d59/d15/d29/d7e/d12a/c154 0 2024-02-15T18:35:01.567 INFO:tasks.workunit.client.0.smithi038.stdout:7/939: creat d0/db/d19/d7d/f155 x:0 0 0 2024-02-15T18:35:01.567 INFO:tasks.workunit.client.0.smithi038.stdout:7/940: chown d0/db/d19/d7d/daa/dd0/d14a/dcd/d134/f14c 1 1 2024-02-15T18:35:01.568 INFO:tasks.workunit.client.0.smithi038.stdout:1/855: dwrite d2/d39/fdb [0,4194304] 0 2024-02-15T18:35:01.568 INFO:tasks.workunit.client.0.smithi038.stdout:1/856: chown d2/d2e/d8c/cfb 5062373 1 2024-02-15T18:35:01.568 INFO:tasks.workunit.client.0.smithi038.stdout:1/857: write d2/d69/f83 [1176175,1445] 0 2024-02-15T18:35:01.569 INFO:tasks.workunit.client.0.smithi038.stdout:2/826: dwrite dd/d12/d59/db2/f100 [0,4194304] 0 2024-02-15T18:35:01.569 INFO:tasks.workunit.client.0.smithi038.stdout:5/906: dwrite d2/d3/d5f/fce [0,4194304] 0 2024-02-15T18:35:01.573 INFO:tasks.workunit.client.0.smithi038.stdout:4/928: getdents d11/d1e/d20/d2b/d4d/d67/da1/d11c 0 2024-02-15T18:35:01.581 INFO:tasks.workunit.client.0.smithi038.stdout:1/858: dwrite d2/d10/d19/f1d [0,4194304] 0 2024-02-15T18:35:01.582 INFO:tasks.workunit.client.0.smithi038.stdout:7/941: getdents d0/db/d59/d15/d4f 0 2024-02-15T18:35:01.587 INFO:tasks.workunit.client.0.smithi038.stdout:2/827: fdatasync dd/fc5 0 2024-02-15T18:35:01.591 INFO:tasks.workunit.client.0.smithi038.stdout:5/907: rename d2/d3/c16 to d2/d30/c111 0 2024-02-15T18:35:01.591 INFO:tasks.workunit.client.0.smithi038.stdout:4/929: unlink d11/d1e/d20/ce1 0 2024-02-15T18:35:01.591 INFO:tasks.workunit.client.0.smithi038.stdout:4/930: write d11/d1e/d20/d31/da2/d118/d7e/f13e [5212500,111807] 0 2024-02-15T18:35:01.592 INFO:tasks.workunit.client.0.smithi038.stdout:1/859: mknod d2/d10/d19/d22/d54/dba/c120 0 2024-02-15T18:35:01.593 INFO:tasks.workunit.client.0.smithi038.stdout:2/828: creat dd/d12/d3a/d67/d4f/d5a/f10a x:0 0 0 2024-02-15T18:35:01.595 INFO:tasks.workunit.client.0.smithi038.stdout:5/908: creat d2/d3/db9/dcf/dd8/f112 x:0 0 0 2024-02-15T18:35:01.595 INFO:tasks.workunit.client.0.smithi038.stdout:5/909: readlink d2/d30/la2 0 2024-02-15T18:35:01.597 INFO:tasks.workunit.client.0.smithi038.stdout:2/829: symlink dd/d12/d59/dce/dea/dee/l10b 0 2024-02-15T18:35:01.598 INFO:tasks.workunit.client.0.smithi038.stdout:2/830: dread dd/d12/d5d/d54/fa4 [0,4194304] 0 2024-02-15T18:35:01.599 INFO:tasks.workunit.client.0.smithi038.stdout:2/831: chown dd/d12/d6b 13 1 2024-02-15T18:35:01.599 INFO:tasks.workunit.client.0.smithi038.stdout:5/910: rename d2/d3/d12/de6 to d2/d3/db9/d113 0 2024-02-15T18:35:01.601 INFO:tasks.workunit.client.0.smithi038.stdout:2/832: unlink dd/d12/d59/d2b/l83 0 2024-02-15T18:35:01.602 INFO:tasks.workunit.client.0.smithi038.stdout:7/942: dwrite d0/d95/fba [0,4194304] 0 2024-02-15T18:35:01.602 INFO:tasks.workunit.client.0.smithi038.stdout:5/911: symlink d2/d3/d12/d5e/dfd/d101/l114 0 2024-02-15T18:35:01.603 INFO:tasks.workunit.client.0.smithi038.stdout:2/833: rmdir dd/d12/d1d/d3b/d3c/d4a/dff 0 2024-02-15T18:35:01.605 INFO:tasks.workunit.client.0.smithi038.stdout:2/834: write dd/d12/f36 [7291802,110871] 0 2024-02-15T18:35:01.605 INFO:tasks.workunit.client.0.smithi038.stdout:2/835: readlink dd/d12/d3a/d5e/lf8 0 2024-02-15T18:35:01.606 INFO:tasks.workunit.client.0.smithi038.stdout:5/912: mkdir d2/d3/d115 0 2024-02-15T18:35:01.608 INFO:tasks.workunit.client.0.smithi038.stdout:2/836: getdents dd/d12/d3a/d67/d4f/d5a/dbc/de8/d90 0 2024-02-15T18:35:01.611 INFO:tasks.workunit.client.0.smithi038.stdout:2/837: dread dd/d12/d5d/fb1 [0,4194304] 0 2024-02-15T18:35:01.611 INFO:tasks.workunit.client.0.smithi038.stdout:7/943: symlink d0/db/d59/l156 0 2024-02-15T18:35:01.613 INFO:tasks.workunit.client.0.smithi038.stdout:2/838: dread dd/d12/d3a/d67/f8f [0,4194304] 0 2024-02-15T18:35:01.613 INFO:tasks.workunit.client.0.smithi038.stdout:2/839: chown dd/d12/d1d/d3b/d3c/d4a/fb3 0 1 2024-02-15T18:35:01.617 INFO:tasks.workunit.client.0.smithi038.stdout:2/840: rmdir dd/d12/d5d 39 2024-02-15T18:35:01.619 INFO:tasks.workunit.client.0.smithi038.stdout:4/931: dwrite d11/d1e/d20/d31/d8d/f12e [0,4194304] 0 2024-02-15T18:35:01.620 INFO:tasks.workunit.client.0.smithi038.stdout:1/860: dwrite d2/d10/fa3 [0,4194304] 0 2024-02-15T18:35:01.622 INFO:tasks.workunit.client.0.smithi038.stdout:4/932: mknod d11/d1e/d20/d31/d48/dac/dbe/d40/c14e 0 2024-02-15T18:35:01.622 INFO:tasks.workunit.client.0.smithi038.stdout:4/933: readlink d11/d1e/d43/d5e/d62/l80 0 2024-02-15T18:35:01.624 INFO:tasks.workunit.client.0.smithi038.stdout:1/861: dread d2/d2e/fb6 [0,4194304] 0 2024-02-15T18:35:01.630 INFO:tasks.workunit.client.0.smithi038.stdout:7/944: dwrite d0/d66/f117 [0,4194304] 0 2024-02-15T18:35:01.635 INFO:tasks.workunit.client.0.smithi038.stdout:2/841: dwrite dd/d12/d5d/d93/fc4 [4194304,4194304] 0 2024-02-15T18:35:01.637 INFO:tasks.workunit.client.0.smithi038.stdout:4/934: unlink d11/d1e/d20/d2b/d4d/l9e 0 2024-02-15T18:35:01.639 INFO:tasks.workunit.client.0.smithi038.stdout:3/807: sync 2024-02-15T18:35:01.639 INFO:tasks.workunit.client.0.smithi038.stdout:1/862: truncate d2/d10/d19/d22/d36/d94/dc1/f107 379935 0 2024-02-15T18:35:01.642 INFO:tasks.workunit.client.0.smithi038.stdout:7/945: getdents d0/d68 0 2024-02-15T18:35:01.643 INFO:tasks.workunit.client.0.smithi038.stdout:2/842: mknod dd/d12/d6b/da5/dd8/c10c 0 2024-02-15T18:35:01.644 INFO:tasks.workunit.client.0.smithi038.stdout:4/935: symlink d11/d1e/d20/d31/da2/d118/d7e/d105/l14f 0 2024-02-15T18:35:01.644 INFO:tasks.workunit.client.0.smithi038.stdout:3/808: dread d2/f50 [0,4194304] 0 2024-02-15T18:35:01.644 INFO:tasks.workunit.client.0.smithi038.stdout:4/936: stat d11/d1e/l64 0 2024-02-15T18:35:01.644 INFO:tasks.workunit.client.0.smithi038.stdout:4/937: chown d11/d1e/d20/d31/d7a 0 1 2024-02-15T18:35:01.644 INFO:tasks.workunit.client.0.smithi038.stdout:3/809: fsync d2/d1b/d3c/fd4 0 2024-02-15T18:35:01.647 INFO:tasks.workunit.client.0.smithi038.stdout:1/863: link d2/d10/d19/fcf d2/d69/da9/dad/df3/f121 0 2024-02-15T18:35:01.648 INFO:tasks.workunit.client.0.smithi038.stdout:7/946: symlink d0/d1b/dfd/l157 0 2024-02-15T18:35:01.649 INFO:tasks.workunit.client.0.smithi038.stdout:2/843: creat dd/d12/d5d/d54/dcb/de1/d104/f10d x:0 0 0 2024-02-15T18:35:01.650 INFO:tasks.workunit.client.0.smithi038.stdout:2/844: dread dd/d12/d3a/d67/d4f/d5a/d9d/fac [0,4194304] 0 2024-02-15T18:35:01.650 INFO:tasks.workunit.client.0.smithi038.stdout:4/938: creat d11/d1e/d20/d31/da2/d118/f150 x:0 0 0 2024-02-15T18:35:01.651 INFO:tasks.workunit.client.0.smithi038.stdout:3/810: symlink d2/d24/db9/lff 0 2024-02-15T18:35:01.652 INFO:tasks.workunit.client.0.smithi038.stdout:1/864: mknod d2/d10/d19/d5f/d92/c122 0 2024-02-15T18:35:01.652 INFO:tasks.workunit.client.0.smithi038.stdout:1/865: readlink d2/d69/d8b/l116 0 2024-02-15T18:35:01.658 INFO:tasks.workunit.client.0.smithi038.stdout:7/947: rmdir d0/db/d59/d15/d4f 39 2024-02-15T18:35:01.658 INFO:tasks.workunit.client.0.smithi038.stdout:4/939: getdents d11/d1e/d20/d31/d48/dac/dbe 0 2024-02-15T18:35:01.658 INFO:tasks.workunit.client.0.smithi038.stdout:2/845: rmdir dd/d12/d6b 39 2024-02-15T18:35:01.658 INFO:tasks.workunit.client.0.smithi038.stdout:4/940: write d11/d1e/d20/d31/da2/d118/d7e/fd4 [814639,9539] 0 2024-02-15T18:35:01.661 INFO:tasks.workunit.client.0.smithi038.stdout:1/866: rename d2/d39/fdb to d2/d10/d19/d22/d54/f123 0 2024-02-15T18:35:01.662 INFO:tasks.workunit.client.0.smithi038.stdout:7/948: mknod d0/d68/dd9/c158 0 2024-02-15T18:35:01.662 INFO:tasks.workunit.client.0.smithi038.stdout:1/867: write d2/d10/fa5 [1078142,79344] 0 2024-02-15T18:35:01.662 INFO:tasks.workunit.client.0.smithi038.stdout:2/846: mkdir dd/d12/d3a/d67/d10e 0 2024-02-15T18:35:01.663 INFO:tasks.workunit.client.0.smithi038.stdout:4/941: mknod d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/c151 0 2024-02-15T18:35:01.663 INFO:tasks.workunit.client.0.smithi038.stdout:4/942: fsync d11/d1e/d20/d31/f5b 0 2024-02-15T18:35:01.664 INFO:tasks.workunit.client.0.smithi038.stdout:7/949: creat d0/db/d19/d7d/daa/f159 x:0 0 0 2024-02-15T18:35:01.665 INFO:tasks.workunit.client.0.smithi038.stdout:1/868: link d2/d10/d19/d22/d54/dba/c120 d2/d10/d62/dfc/c124 0 2024-02-15T18:35:01.665 INFO:tasks.workunit.client.0.smithi038.stdout:2/847: mkdir dd/d12/d59/d2b/d6a/d10f 0 2024-02-15T18:35:01.666 INFO:tasks.workunit.client.0.smithi038.stdout:7/950: rename d0/db/d59/dc8/f103 to d0/db/d59/d15/d29/d7e/f15a 0 2024-02-15T18:35:01.667 INFO:tasks.workunit.client.0.smithi038.stdout:1/869: mknod d2/d69/d8b/c125 0 2024-02-15T18:35:01.668 INFO:tasks.workunit.client.0.smithi038.stdout:2/848: symlink dd/d12/d3a/l110 0 2024-02-15T18:35:01.669 INFO:tasks.workunit.client.0.smithi038.stdout:1/870: mknod d2/d10/d19/d22/d54/dc0/d3c/c126 0 2024-02-15T18:35:01.669 INFO:tasks.workunit.client.0.smithi038.stdout:1/871: dread - d2/d10/d19/d22/fe8 zero size 2024-02-15T18:35:01.669 INFO:tasks.workunit.client.0.smithi038.stdout:1/872: write d2/d10/d19/d22/fda [4900519,55428] 0 2024-02-15T18:35:01.669 INFO:tasks.workunit.client.0.smithi038.stdout:1/873: write d2/d69/d8b/f11f [190316,36482] 0 2024-02-15T18:35:01.669 INFO:tasks.workunit.client.0.smithi038.stdout:2/849: stat dd/d12/d3a/d67/d4f/c98 0 2024-02-15T18:35:01.669 INFO:tasks.workunit.client.0.smithi038.stdout:1/874: symlink d2/d69/l127 0 2024-02-15T18:35:01.670 INFO:tasks.workunit.client.0.smithi038.stdout:3/811: dwrite d2/f9 [0,4194304] 0 2024-02-15T18:35:01.670 INFO:tasks.workunit.client.0.smithi038.stdout:4/943: dread d11/d1e/d20/d31/f5b [0,4194304] 0 2024-02-15T18:35:01.673 INFO:tasks.workunit.client.0.smithi038.stdout:4/944: mkdir d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/d152 0 2024-02-15T18:35:01.675 INFO:tasks.workunit.client.0.smithi038.stdout:7/951: dread d0/db/d59/d15/d29/f114 [0,4194304] 0 2024-02-15T18:35:01.677 INFO:tasks.workunit.client.0.smithi038.stdout:7/952: write d0/d68/fae [659204,107904] 0 2024-02-15T18:35:01.686 INFO:tasks.workunit.client.0.smithi038.stdout:3/812: dread d2/d1b/d3c/d41/d4e/d62/f8f [0,4194304] 0 2024-02-15T18:35:01.688 INFO:tasks.workunit.client.0.smithi038.stdout:6/915: sync 2024-02-15T18:35:01.688 INFO:tasks.workunit.client.0.smithi038.stdout:0/981: sync 2024-02-15T18:35:01.688 INFO:tasks.workunit.client.0.smithi038.stdout:0/982: chown d4/d13/d2a/d35/d1e/d12b 471798627 1 2024-02-15T18:35:01.689 INFO:tasks.workunit.client.0.smithi038.stdout:6/916: write d2/d4/d19/d55/fef [4657122,78016] 0 2024-02-15T18:35:01.692 INFO:tasks.workunit.client.0.smithi038.stdout:3/813: rename d2/d1b/d3c/d3a/f68 to d2/d7f/dc1/de1/dfa/f100 0 2024-02-15T18:35:01.694 INFO:tasks.workunit.client.0.smithi038.stdout:3/814: dread d2/d1b/d4f/fb4 [0,4194304] 0 2024-02-15T18:35:01.694 INFO:tasks.workunit.client.0.smithi038.stdout:3/815: chown d2/d1b/d3c/d3a/d55/lc7 60 1 2024-02-15T18:35:01.694 INFO:tasks.workunit.client.0.smithi038.stdout:3/816: chown d2/d24/f49 15439557 1 2024-02-15T18:35:01.694 INFO:tasks.workunit.client.0.smithi038.stdout:3/817: truncate d2/d1b/d3c/d41/ff9 912041 0 2024-02-15T18:35:01.695 INFO:tasks.workunit.client.0.smithi038.stdout:3/818: write d2/d1b/d4f/f5d [1315776,8300] 0 2024-02-15T18:35:01.697 INFO:tasks.workunit.client.0.smithi038.stdout:3/819: truncate d2/d1b/d4f/fb4 135514 0 2024-02-15T18:35:01.699 INFO:tasks.workunit.client.0.smithi038.stdout:1/875: dread d2/d10/d19/d22/fda [0,4194304] 0 2024-02-15T18:35:01.700 INFO:tasks.workunit.client.0.smithi038.stdout:1/876: creat d2/d10/f128 x:0 0 0 2024-02-15T18:35:01.701 INFO:tasks.workunit.client.0.smithi038.stdout:2/850: dwrite dd/d12/d1d/f2e [0,4194304] 0 2024-02-15T18:35:01.702 INFO:tasks.workunit.client.0.smithi038.stdout:1/877: symlink d2/d10/d19/d22/d54/l129 0 2024-02-15T18:35:01.702 INFO:tasks.workunit.client.0.smithi038.stdout:4/945: dwrite d11/d1e/d20/d31/f91 [4194304,4194304] 0 2024-02-15T18:35:01.703 INFO:tasks.workunit.client.0.smithi038.stdout:1/878: link d2/d10/d19/d22/d36/l66 d2/d10/d32/d53/df0/l12a 0 2024-02-15T18:35:01.703 INFO:tasks.workunit.client.0.smithi038.stdout:1/879: stat d2/d2e/l86 0 2024-02-15T18:35:01.703 INFO:tasks.workunit.client.0.smithi038.stdout:1/880: chown d2/d10/d19/fcf 1525865 1 2024-02-15T18:35:01.703 INFO:tasks.workunit.client.0.smithi038.stdout:4/946: write d11/d1e/d20/d2b/d4d/f119 [2918552,4873] 0 2024-02-15T18:35:01.704 INFO:tasks.workunit.client.0.smithi038.stdout:4/947: creat d11/d1e/d20/d31/d48/d11e/d14b/f153 x:0 0 0 2024-02-15T18:35:01.704 INFO:tasks.workunit.client.0.smithi038.stdout:4/948: chown d11/d1e/d20/d31/d8d/d99/db5/l134 1739 1 2024-02-15T18:35:01.705 INFO:tasks.workunit.client.0.smithi038.stdout:4/949: symlink d11/d1e/d20/d31/d128/l154 0 2024-02-15T18:35:01.707 INFO:tasks.workunit.client.0.smithi038.stdout:1/881: dread d2/f2d [0,4194304] 0 2024-02-15T18:35:01.708 INFO:tasks.workunit.client.0.smithi038.stdout:7/953: dwrite d0/db/d59/dc8/dd6/d61/ff1 [0,4194304] 0 2024-02-15T18:35:01.708 INFO:tasks.workunit.client.0.smithi038.stdout:6/917: dwrite d2/d10/f8f [0,4194304] 0 2024-02-15T18:35:01.708 INFO:tasks.workunit.client.0.smithi038.stdout:6/918: read - d2/d4/d6c/d9d/dd7/fe6 zero size 2024-02-15T18:35:01.709 INFO:tasks.workunit.client.0.smithi038.stdout:6/919: dread - d2/d4/d6c/ffb zero size 2024-02-15T18:35:01.709 INFO:tasks.workunit.client.0.smithi038.stdout:6/920: readlink d2/d4/d19/d55/dc0/l111 0 2024-02-15T18:35:01.709 INFO:tasks.workunit.client.0.smithi038.stdout:2/851: write dd/d12/d1d/f50 [12493784,22133] 0 2024-02-15T18:35:01.709 INFO:tasks.workunit.client.0.smithi038.stdout:2/852: fdatasync dd/d12/f36 0 2024-02-15T18:35:01.710 INFO:tasks.workunit.client.0.smithi038.stdout:3/820: rename d2/d7f/dc1/de1 to d2/d7f/d101 0 2024-02-15T18:35:01.710 INFO:tasks.workunit.client.0.smithi038.stdout:3/821: chown d2/d1b/d3c/d3a/d55/fb1 0 1 2024-02-15T18:35:01.717 INFO:tasks.workunit.client.0.smithi038.stdout:7/954: mknod d0/db/d59/d15/d29/d7e/c15b 0 2024-02-15T18:35:01.719 INFO:tasks.workunit.client.0.smithi038.stdout:7/955: dread d0/d66/f104 [0,4194304] 0 2024-02-15T18:35:01.727 INFO:tasks.workunit.client.0.smithi038.stdout:6/921: mknod d2/d4/d2c/dfc/c11a 0 2024-02-15T18:35:01.727 INFO:tasks.workunit.client.0.smithi038.stdout:6/922: readlink d2/d4/d19/d55/d5c/dc8/l10f 0 2024-02-15T18:35:01.727 INFO:tasks.workunit.client.0.smithi038.stdout:6/923: write d2/d4/d19/d78/fd6 [308396,89465] 0 2024-02-15T18:35:01.728 INFO:tasks.workunit.client.0.smithi038.stdout:3/822: mkdir d2/db8/dcf/d102 0 2024-02-15T18:35:01.729 INFO:tasks.workunit.client.0.smithi038.stdout:1/882: rmdir d2/d39/dd4 0 2024-02-15T18:35:01.730 INFO:tasks.workunit.client.0.smithi038.stdout:6/924: mknod d2/d4/d19/d78/dc3/c11b 0 2024-02-15T18:35:01.730 INFO:tasks.workunit.client.0.smithi038.stdout:6/925: chown d2/d4/d2c/d39/d76/dc2 151 1 2024-02-15T18:35:01.732 INFO:tasks.workunit.client.0.smithi038.stdout:3/823: symlink d2/d1e/l103 0 2024-02-15T18:35:01.733 INFO:tasks.workunit.client.0.smithi038.stdout:1/883: link d2/d10/f128 d2/d10/d32/d53/f12b 0 2024-02-15T18:35:01.735 INFO:tasks.workunit.client.0.smithi038.stdout:3/824: rename d2/ca8 to d2/d7f/df0/c104 0 2024-02-15T18:35:01.735 INFO:tasks.workunit.client.0.smithi038.stdout:3/825: chown d2/d7f/df0 62603581 1 2024-02-15T18:35:01.746 INFO:tasks.workunit.client.0.smithi038.stdout:7/956: dwrite d0/db/d59/dc8/dd6/d61/d90/f9c [0,4194304] 0 2024-02-15T18:35:01.749 INFO:tasks.workunit.client.0.smithi038.stdout:5/913: sync 2024-02-15T18:35:01.751 INFO:tasks.workunit.client.0.smithi038.stdout:7/957: truncate d0/db/d59/dc8/dd6/f82 199162 0 2024-02-15T18:35:01.753 INFO:tasks.workunit.client.0.smithi038.stdout:7/958: unlink d0/d68/l9b 0 2024-02-15T18:35:01.754 INFO:tasks.workunit.client.0.smithi038.stdout:7/959: mkdir d0/db/d59/dc8/dd6/d70/d122/d15c 0 2024-02-15T18:35:01.754 INFO:tasks.workunit.client.0.smithi038.stdout:7/960: truncate d0/db/d59/d15/d29/f91 1373757 0 2024-02-15T18:35:01.754 INFO:tasks.workunit.client.0.smithi038.stdout:7/961: truncate d0/db/d19/d7d/daa/dd0/d14a/d102/d10a/f11f 25462 0 2024-02-15T18:35:01.755 INFO:tasks.workunit.client.0.smithi038.stdout:7/962: mknod d0/d68/c15d 0 2024-02-15T18:35:01.760 INFO:tasks.workunit.client.0.smithi038.stdout:1/884: dwrite d2/f4 [0,4194304] 0 2024-02-15T18:35:01.760 INFO:tasks.workunit.client.0.smithi038.stdout:3/826: dwrite d2/d1b/d3c/d3a/d55/d65/fd0 [4194304,4194304] 0 2024-02-15T18:35:01.761 INFO:tasks.workunit.client.0.smithi038.stdout:5/914: read d2/d30/d57/f8d [52381,97111] 0 2024-02-15T18:35:01.761 INFO:tasks.workunit.client.0.smithi038.stdout:5/915: dread - d2/d30/d57/fdb zero size 2024-02-15T18:35:01.761 INFO:tasks.workunit.client.0.smithi038.stdout:5/916: write d2/d30/d57/f78 [1613653,82318] 0 2024-02-15T18:35:01.761 INFO:tasks.workunit.client.0.smithi038.stdout:1/885: write d2/d10/d19/d22/f2c [3797835,112758] 0 2024-02-15T18:35:01.761 INFO:tasks.workunit.client.0.smithi038.stdout:1/886: truncate d2/d10/d62/fa6 794631 0 2024-02-15T18:35:01.765 INFO:tasks.workunit.client.0.smithi038.stdout:7/963: rename d0/db/d59/d15 to d0/db/d59/dc8/dd6/d70/d15e 0 2024-02-15T18:35:01.765 INFO:tasks.workunit.client.0.smithi038.stdout:7/964: truncate d0/db/d59/dc8/dd6/d70/d15e/d29/d7e/f15a 1467588 0 2024-02-15T18:35:01.768 INFO:tasks.workunit.client.0.smithi038.stdout:5/917: creat d2/d3/da0/d33/f116 x:0 0 0 2024-02-15T18:35:01.770 INFO:tasks.workunit.client.0.smithi038.stdout:3/827: dread d2/d24/f2e [0,4194304] 0 2024-02-15T18:35:01.770 INFO:tasks.workunit.client.0.smithi038.stdout:1/887: truncate d2/d10/d32/d53/f7b 1970395 0 2024-02-15T18:35:01.771 INFO:tasks.workunit.client.0.smithi038.stdout:7/965: creat d0/d68/f15f x:0 0 0 2024-02-15T18:35:01.771 INFO:tasks.workunit.client.0.smithi038.stdout:7/966: dread - d0/db/d59/dc8/dd6/d61/d90/dc0/d128/f136 zero size 2024-02-15T18:35:01.773 INFO:tasks.workunit.client.0.smithi038.stdout:1/888: dread d2/d10/d19/d2f/fd6 [0,4194304] 0 2024-02-15T18:35:01.773 INFO:tasks.workunit.client.0.smithi038.stdout:1/889: stat d2/f9d 0 2024-02-15T18:35:01.773 INFO:tasks.workunit.client.0.smithi038.stdout:1/890: chown d2/d10/d19/d22/d54/lc5 990044 1 2024-02-15T18:35:01.773 INFO:tasks.workunit.client.0.smithi038.stdout:1/891: chown d2/d10/d19/d22/d54/fd9 91910 1 2024-02-15T18:35:01.773 INFO:tasks.workunit.client.0.smithi038.stdout:1/892: chown d2/d10/d19/d5f/f70 3 1 2024-02-15T18:35:01.773 INFO:tasks.workunit.client.0.smithi038.stdout:1/893: fsync d2/d10/d19/d22/f30 0 2024-02-15T18:35:01.773 INFO:tasks.workunit.client.0.smithi038.stdout:3/828: creat d2/d1b/d69/dd3/f105 x:0 0 0 2024-02-15T18:35:01.779 INFO:tasks.workunit.client.0.smithi038.stdout:1/894: mkdir d2/d12c 0 2024-02-15T18:35:01.780 INFO:tasks.workunit.client.0.smithi038.stdout:1/895: stat d2/d10/d19/d22/d54/dc0/d6b/fbe 0 2024-02-15T18:35:01.780 INFO:tasks.workunit.client.0.smithi038.stdout:1/896: dread - d2/d10/d19/d22/d36/d94/dc1/dd5/f119 zero size 2024-02-15T18:35:01.780 INFO:tasks.workunit.client.0.smithi038.stdout:1/897: truncate d2/d10/d19/d22/d54/fd8 52559 0 2024-02-15T18:35:01.782 INFO:tasks.workunit.client.0.smithi038.stdout:1/898: rename d2/d10/d19/d22/d54/dc0/d3c/caa to d2/d10/d62/dfc/c12d 0 2024-02-15T18:35:01.797 INFO:tasks.workunit.client.0.smithi038.stdout:3/829: dwrite d2/d1b/d3c/d3a/d55/d65/fa6 [0,4194304] 0 2024-02-15T18:35:01.797 INFO:tasks.workunit.client.0.smithi038.stdout:0/983: sync 2024-02-15T18:35:01.799 INFO:tasks.workunit.client.0.smithi038.stdout:7/967: dwrite d0/db/d19/d2d/f52 [0,4194304] 0 2024-02-15T18:35:01.799 INFO:tasks.workunit.client.0.smithi038.stdout:3/830: creat d2/d1b/d3c/d41/d4e/d62/f106 x:0 0 0 2024-02-15T18:35:01.799 INFO:tasks.workunit.client.0.smithi038.stdout:3/831: fsync d2/d25/f64 0 2024-02-15T18:35:01.799 INFO:tasks.workunit.client.0.smithi038.stdout:3/832: chown d2/d1b/d8c/lc5 15312 1 2024-02-15T18:35:01.801 INFO:tasks.workunit.client.0.smithi038.stdout:3/833: mkdir d2/d24/d107 0 2024-02-15T18:35:01.801 INFO:tasks.workunit.client.0.smithi038.stdout:3/834: dread - d2/d1b/d4f/d92/f9c zero size 2024-02-15T18:35:01.813 INFO:tasks.workunit.client.0.smithi038.stdout:0/984: dwrite d4/d13/d2a/fd4 [0,4194304] 0 2024-02-15T18:35:01.813 INFO:tasks.workunit.client.0.smithi038.stdout:0/985: write d4/db3/fe2 [448094,92880] 0 2024-02-15T18:35:01.814 INFO:tasks.workunit.client.0.smithi038.stdout:0/986: readlink d4/db3/dc7/l142 0 2024-02-15T18:35:01.833 INFO:tasks.workunit.client.0.smithi038.stdout:7/968: dwrite d0/db/d59/dc8/dd6/d70/d15e/d29/f114 [4194304,4194304] 0 2024-02-15T18:35:01.834 INFO:tasks.workunit.client.0.smithi038.stdout:3/835: dwrite d2/d1b/d3c/d3a/fdf [0,4194304] 0 2024-02-15T18:35:01.834 INFO:tasks.workunit.client.0.smithi038.stdout:0/987: dwrite d4/d13/d2a/d35/d1e/d32/d3b/f117 [0,4194304] 0 2024-02-15T18:35:01.840 INFO:tasks.workunit.client.0.smithi038.stdout:0/988: rename d4/f11 to d4/d13/d19/f14a 0 2024-02-15T18:35:01.840 INFO:tasks.workunit.client.0.smithi038.stdout:0/989: chown d4/db3/dc7 77842262 1 2024-02-15T18:35:01.842 INFO:tasks.workunit.client.0.smithi038.stdout:0/990: dread d4/d13/d19/f91 [0,4194304] 0 2024-02-15T18:35:01.842 INFO:tasks.workunit.client.0.smithi038.stdout:3/836: dread d2/d1e/f89 [0,4194304] 0 2024-02-15T18:35:01.843 INFO:tasks.workunit.client.0.smithi038.stdout:3/837: symlink d2/d7f/d101/dfa/l108 0 2024-02-15T18:35:01.845 INFO:tasks.workunit.client.0.smithi038.stdout:3/838: getdents d2/d1b/d3c/d3a/d55/d65 0 2024-02-15T18:35:01.847 INFO:tasks.workunit.client.0.smithi038.stdout:3/839: mknod d2/d1b/d69/dd3/c109 0 2024-02-15T18:35:01.849 INFO:tasks.workunit.client.0.smithi038.stdout:3/840: rename d2/f1a to d2/db8/dcf/f10a 0 2024-02-15T18:35:01.851 INFO:tasks.workunit.client.0.smithi038.stdout:3/841: rename d2/f12 to d2/d1b/d3c/d3a/d55/df8/f10b 0 2024-02-15T18:35:01.853 INFO:tasks.workunit.client.0.smithi038.stdout:7/969: dwrite d0/d1b/ff3 [0,4194304] 0 2024-02-15T18:35:01.853 INFO:tasks.workunit.client.0.smithi038.stdout:7/970: chown d0/db/d19/l11a 158 1 2024-02-15T18:35:01.853 INFO:tasks.workunit.client.0.smithi038.stdout:7/971: chown d0 4146971 1 2024-02-15T18:35:01.853 INFO:tasks.workunit.client.0.smithi038.stdout:3/842: rmdir d2/db8/dce 39 2024-02-15T18:35:01.853 INFO:tasks.workunit.client.0.smithi038.stdout:3/843: readlink d2/d1b/d4f/la1 0 2024-02-15T18:35:01.857 INFO:tasks.workunit.client.0.smithi038.stdout:7/972: mknod d0/db/d59/dc8/c160 0 2024-02-15T18:35:01.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:00.318711+0000 mgr.smithi038.khesbl (mgr.14162) 340 : cluster [DBG] pgmap v282: 65 pgs: 65 active+clean; 1.5 GiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 102 MiB/s rd, 192 MiB/s wr, 596 op/s 2024-02-15T18:35:01.858 INFO:tasks.workunit.client.0.smithi038.stdout:0/991: dwrite d4/d13/d2a/d35/d1e/d32/d79/d7d/f86 [0,4194304] 0 2024-02-15T18:35:01.858 INFO:tasks.workunit.client.0.smithi038.stdout:7/973: mknod d0/db/d59/d97/c161 0 2024-02-15T18:35:01.858 INFO:tasks.workunit.client.0.smithi038.stdout:7/974: truncate d0/d66/f119 954829 0 2024-02-15T18:35:01.858 INFO:tasks.workunit.client.0.smithi038.stdout:3/844: dread d2/d1b/d3c/f6b [0,4194304] 0 2024-02-15T18:35:01.859 INFO:tasks.workunit.client.0.smithi038.stdout:7/975: write d0/db/d59/dc8/dd6/d61/d90/fab [154165,81351] 0 2024-02-15T18:35:01.860 INFO:tasks.workunit.client.0.smithi038.stdout:0/992: creat d4/d13/d2a/d35/d1e/d32/d79/d7d/dc6/ddb/f14b x:0 0 0 2024-02-15T18:35:01.861 INFO:tasks.workunit.client.0.smithi038.stdout:7/976: write d0/d1b/d30/d50/f74 [1494705,9798] 0 2024-02-15T18:35:01.861 INFO:tasks.workunit.client.0.smithi038.stdout:3/845: mkdir d2/d1b/d69/d10c 0 2024-02-15T18:35:01.861 INFO:tasks.workunit.client.0.smithi038.stdout:3/846: read - d2/d25/d60/fae zero size 2024-02-15T18:35:01.861 INFO:tasks.workunit.client.0.smithi038.stdout:3/847: fsync d2/d1b/d3c/d3a/d55/da0/fc6 0 2024-02-15T18:35:01.862 INFO:tasks.workunit.client.0.smithi038.stdout:0/993: rmdir d4/d13/d2a/d35/d1e/d32/d3b/d11b 0 2024-02-15T18:35:01.865 INFO:tasks.workunit.client.0.smithi038.stdout:7/977: symlink d0/db/d59/dc8/dd6/d70/d15e/l162 0 2024-02-15T18:35:01.865 INFO:tasks.workunit.client.0.smithi038.stdout:7/978: write d0/d66/f104 [1151801,61071] 0 2024-02-15T18:35:01.865 INFO:tasks.workunit.client.0.smithi038.stdout:7/979: readlink d0/db/d59/dc8/dd6/d61/l13b 0 2024-02-15T18:35:01.865 INFO:tasks.workunit.client.0.smithi038.stdout:7/980: readlink d0/d1b/dfd/l10f 0 2024-02-15T18:35:01.865 INFO:tasks.workunit.client.0.smithi038.stdout:7/981: read - d0/db/d19/d7d/daa/dd0/d14a/d102/f13e zero size 2024-02-15T18:35:01.865 INFO:tasks.workunit.client.0.smithi038.stdout:2/853: sync 2024-02-15T18:35:01.866 INFO:tasks.workunit.client.0.smithi038.stdout:6/926: sync 2024-02-15T18:35:01.866 INFO:tasks.workunit.client.0.smithi038.stdout:4/950: sync 2024-02-15T18:35:01.866 INFO:tasks.workunit.client.0.smithi038.stdout:4/951: write d11/d1e/d20/d31/d7a/dd8/f11a [360155,59528] 0 2024-02-15T18:35:01.867 INFO:tasks.workunit.client.0.smithi038.stdout:2/854: dread dd/d12/f47 [0,4194304] 0 2024-02-15T18:35:01.867 INFO:tasks.workunit.client.0.smithi038.stdout:3/848: mkdir d2/d7f/df0/d10d 0 2024-02-15T18:35:01.869 INFO:tasks.workunit.client.0.smithi038.stdout:0/994: creat d4/d13/d2a/d35/d1e/d32/d48/f14c x:0 0 0 2024-02-15T18:35:01.869 INFO:tasks.workunit.client.0.smithi038.stdout:7/982: rmdir d0/db/d19/d7d/daa 39 2024-02-15T18:35:01.869 INFO:tasks.workunit.client.0.smithi038.stdout:7/983: rename d0/db to d0/db/d19/d7d/dc9/d163 22 2024-02-15T18:35:01.869 INFO:tasks.workunit.client.0.smithi038.stdout:7/984: chown d0/d66/c79 31168044 1 2024-02-15T18:35:01.871 INFO:tasks.workunit.client.0.smithi038.stdout:0/995: write d4/d13/d2a/d35/d1e/d32/d79/d7d/f86 [2980557,46422] 0 2024-02-15T18:35:01.871 INFO:tasks.workunit.client.0.smithi038.stdout:4/952: unlink d11/d1e/d20/d97/f9f 0 2024-02-15T18:35:01.872 INFO:tasks.workunit.client.0.smithi038.stdout:3/849: rmdir d2/d1b/d69/dd3 39 2024-02-15T18:35:01.872 INFO:tasks.workunit.client.0.smithi038.stdout:3/850: chown d2/d1b/d69 129034 1 2024-02-15T18:35:01.878 INFO:tasks.workunit.client.0.smithi038.stdout:2/855: dread dd/d12/d3a/d67/f29 [0,4194304] 0 2024-02-15T18:35:01.880 INFO:tasks.workunit.client.0.smithi038.stdout:2/856: dread dd/d12/d5d/fb1 [0,4194304] 0 2024-02-15T18:35:01.881 INFO:tasks.workunit.client.0.smithi038.stdout:6/927: dwrite d2/d4/d19/d78/fae [0,4194304] 0 2024-02-15T18:35:01.882 INFO:tasks.workunit.client.0.smithi038.stdout:0/996: dread d4/d13/d2a/fd4 [0,4194304] 0 2024-02-15T18:35:01.882 INFO:tasks.workunit.client.0.smithi038.stdout:0/997: fsync d4/d13/d2a/d35/d1e/d32/d48/d9a/fc3 0 2024-02-15T18:35:01.882 INFO:tasks.workunit.client.0.smithi038.stdout:4/953: write d11/d1e/d20/d31/d7a/dd8/f11a [23455,70983] 0 2024-02-15T18:35:01.883 INFO:tasks.workunit.client.0.smithi038.stdout:4/954: write d11/d1e/d20/d2b/d4d/d67/da1/d11c/f145 [932602,74488] 0 2024-02-15T18:35:01.884 INFO:tasks.workunit.client.0.smithi038.stdout:2/857: mknod dd/d12/d5d/dc2/c111 0 2024-02-15T18:35:01.886 INFO:tasks.workunit.client.0.smithi038.stdout:6/928: creat d2/d4/d28/f11c x:0 0 0 2024-02-15T18:35:01.888 INFO:tasks.workunit.client.0.smithi038.stdout:0/998: mknod d4/d13/d19/dfc/c14d 0 2024-02-15T18:35:01.889 INFO:tasks.workunit.client.0.smithi038.stdout:3/851: dwrite d2/d1b/d8c/f99 [0,4194304] 0 2024-02-15T18:35:01.891 INFO:tasks.workunit.client.0.smithi038.stdout:4/955: dread d11/d1e/d20/d31/da2/d118/f90 [0,4194304] 0 2024-02-15T18:35:01.891 INFO:tasks.workunit.client.0.smithi038.stdout:4/956: chown d11/d1e/d20/d31/ffd 1 1 2024-02-15T18:35:01.891 INFO:tasks.workunit.client.0.smithi038.stdout:2/858: creat dd/d12/d3a/d67/d4f/d5a/dbc/f112 x:0 0 0 2024-02-15T18:35:01.891 INFO:tasks.workunit.client.0.smithi038.stdout:2/859: dread - dd/d12/d1d/d3b/fb8 zero size 2024-02-15T18:35:01.894 INFO:tasks.workunit.client.0.smithi038.stdout:4/957: link d11/d1e/d20/d31/d48/d60/lb7 d11/d1e/d20/d31/da2/d118/d7e/l155 0 2024-02-15T18:35:01.894 INFO:tasks.workunit.client.0.smithi038.stdout:2/860: stat dd/d12/d1d/d3b/d3c/d73/l103 0 2024-02-15T18:35:01.894 INFO:tasks.workunit.client.0.smithi038.stdout:2/861: stat dd/d12/d5d/d54/dcb/c106 0 2024-02-15T18:35:01.895 INFO:tasks.workunit.client.0.smithi038.stdout:4/958: write d11/d1e/d20/d31/d8d/de7/f10a [138352,12196] 0 2024-02-15T18:35:01.896 INFO:tasks.workunit.client.0.smithi038.stdout:2/862: creat dd/d12/d6b/f113 x:0 0 0 2024-02-15T18:35:01.896 INFO:tasks.workunit.client.0.smithi038.stdout:2/863: chown dd/f41 1097 1 2024-02-15T18:35:01.897 INFO:tasks.workunit.client.0.smithi038.stdout:2/864: read dd/d12/d3a/d67/f8f [103939,28673] 0 2024-02-15T18:35:01.898 INFO:tasks.workunit.client.0.smithi038.stdout:4/959: symlink d11/d1e/d20/d2b/d4d/d67/da1/d11c/l156 0 2024-02-15T18:35:01.898 INFO:tasks.workunit.client.0.smithi038.stdout:4/960: truncate d11/d1e/d20/d31/da2/d118/d7e/d105/f110 545665 0 2024-02-15T18:35:01.900 INFO:tasks.workunit.client.0.smithi038.stdout:2/865: mkdir dd/d12/d59/d114 0 2024-02-15T18:35:01.900 INFO:tasks.workunit.client.0.smithi038.stdout:2/866: chown dd/d12/f36 4208 1 2024-02-15T18:35:01.900 INFO:tasks.workunit.client.0.smithi038.stdout:2/867: getdents dd/d12/d6b/dde 0 2024-02-15T18:35:01.904 INFO:tasks.workunit.client.0.smithi038.stdout:4/961: symlink d11/d1e/d20/d31/d8d/l157 0 2024-02-15T18:35:01.906 INFO:tasks.workunit.client.0.smithi038.stdout:2/868: truncate dd/d12/d5d/fb1 807355 0 2024-02-15T18:35:01.916 INFO:tasks.workunit.client.0.smithi038.stdout:6/929: dwrite d2/d4/d6c/d9d/dd7/fe6 [0,4194304] 0 2024-02-15T18:35:01.929 INFO:tasks.workunit.client.0.smithi038.stdout:0/999: dwrite d4/d13/f39 [4194304,4194304] 0 2024-02-15T18:35:01.930 INFO:tasks.workunit.client.0.smithi038.stdout:3/852: dwrite d2/d1b/f36 [0,4194304] 0 2024-02-15T18:35:01.930 INFO:tasks.workunit.client.0.smithi038.stdout:2/869: dwrite dd/d12/d3a/d67/d4f/d5a/f6e [8388608,4194304] 0 2024-02-15T18:35:01.930 INFO:tasks.workunit.client.0.smithi038.stdout:2/870: read - dd/d12/d5d/dc2/ff7 zero size 2024-02-15T18:35:01.930 INFO:tasks.workunit.client.0.smithi038.stdout:2/871: stat dd/d12/d3a/d67/d10e 0 2024-02-15T18:35:01.930 INFO:tasks.workunit.client.0.smithi038.stdout:4/962: dwrite d11/d1e/d20/d31/f91 [0,4194304] 0 2024-02-15T18:35:01.930 INFO:tasks.workunit.client.0.smithi038.stdout:4/963: fsync d11/d1e/f37 0 2024-02-15T18:35:01.931 INFO:tasks.workunit.client.0.smithi038.stdout:2/872: read dd/d12/f80 [6383204,79765] 0 2024-02-15T18:35:01.931 INFO:tasks.workunit.client.0.smithi038.stdout:2/873: stat dd/d12/d3a/d67/d4f/d5a/f7f 0 2024-02-15T18:35:01.933 INFO:tasks.workunit.client.0.smithi038.stdout:3/853: creat d2/d24/db9/f10e x:0 0 0 2024-02-15T18:35:01.939 INFO:tasks.workunit.client.0.smithi038.stdout:4/964: rename d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/d152 to d11/d1e/d20/d31/d48/d102/d13d/d158 0 2024-02-15T18:35:01.942 INFO:tasks.workunit.client.0.smithi038.stdout:5/918: sync 2024-02-15T18:35:01.942 INFO:tasks.workunit.client.0.smithi038.stdout:1/899: sync 2024-02-15T18:35:01.942 INFO:tasks.workunit.client.0.smithi038.stdout:1/900: fsync d2/d10/f37 0 2024-02-15T18:35:01.946 INFO:tasks.workunit.client.0.smithi038.stdout:2/874: rmdir dd/d12 39 2024-02-15T18:35:01.947 INFO:tasks.workunit.client.0.smithi038.stdout:3/854: link d2/d25/c6d d2/d7f/df0/c10f 0 2024-02-15T18:35:01.949 INFO:tasks.workunit.client.0.smithi038.stdout:4/965: creat d11/d1e/d20/d31/f159 x:0 0 0 2024-02-15T18:35:01.949 INFO:tasks.workunit.client.0.smithi038.stdout:4/966: write d11/d1e/d43/f45 [921022,117509] 0 2024-02-15T18:35:01.949 INFO:tasks.workunit.client.0.smithi038.stdout:4/967: write d11/d1e/d3e/f54 [1359444,92330] 0 2024-02-15T18:35:01.949 INFO:tasks.workunit.client.0.smithi038.stdout:5/919: creat d2/d3/db9/de3/de4/dff/f117 x:0 0 0 2024-02-15T18:35:01.951 INFO:tasks.workunit.client.0.smithi038.stdout:1/901: rename d2/d69/d8b/l116 to d2/d69/l12e 0 2024-02-15T18:35:01.952 INFO:tasks.workunit.client.0.smithi038.stdout:4/968: creat d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/d11d/d146/f15a x:0 0 0 2024-02-15T18:35:01.952 INFO:tasks.workunit.client.0.smithi038.stdout:4/969: chown d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/ddd/ff2 11059988 1 2024-02-15T18:35:01.952 INFO:tasks.workunit.client.0.smithi038.stdout:5/920: getdents d2/d3 0 2024-02-15T18:35:01.957 INFO:tasks.workunit.client.0.smithi038.stdout:2/875: dread dd/d12/d1d/d3b/d3c/d4a/fb3 [0,4194304] 0 2024-02-15T18:35:01.977 INFO:tasks.workunit.client.0.smithi038.stdout:2/876: dwrite dd/d12/d6b/da5/dd8/fc6 [0,4194304] 0 2024-02-15T18:35:01.977 INFO:tasks.workunit.client.0.smithi038.stdout:2/877: chown dd/d12/d59/db2/fe3 89545819 1 2024-02-15T18:35:01.977 INFO:tasks.workunit.client.0.smithi038.stdout:2/878: read - dd/d12/d3a/d67/d4f/d5a/f10a zero size 2024-02-15T18:35:01.977 INFO:tasks.workunit.client.0.smithi038.stdout:2/879: chown dd/d12/d5d/d54/dcb/de1/d104/f10d 117 1 2024-02-15T18:35:01.995 INFO:tasks.workunit.client.0.smithi038.stdout:1/902: unlink d2/d69/da9/dad/de2/fea 0 2024-02-15T18:35:02.039 INFO:tasks.workunit.client.0.smithi038.stdout:7/985: sync 2024-02-15T18:35:02.043 INFO:tasks.workunit.client.0.smithi038.stdout:4/970: getdents d11/d1e/d20/d31/d48/dac/dbe/d40 0 2024-02-15T18:35:02.043 INFO:tasks.workunit.client.0.smithi038.stdout:4/971: chown d11/d1e/d20/d2f/d71/ffc 244468766 1 2024-02-15T18:35:02.043 INFO:tasks.workunit.client.0.smithi038.stdout:4/972: read d11/d1e/d20/d31/d48/d60/f12b [1416191,70262] 0 2024-02-15T18:35:02.045 INFO:tasks.workunit.client.0.smithi038.stdout:5/921: creat d2/f118 x:0 0 0 2024-02-15T18:35:02.111 INFO:tasks.workunit.client.0.smithi038.stdout:3/855: rename d2/d25/l35 to d2/d1b/d4f/d92/l110 0 2024-02-15T18:35:02.111 INFO:tasks.workunit.client.0.smithi038.stdout:3/856: truncate d2/d1b/d4f/d92/f9c 936621 0 2024-02-15T18:35:02.115 INFO:tasks.workunit.client.0.smithi038.stdout:3/857: dread d2/d24/f49 [0,4194304] 0 2024-02-15T18:35:02.116 INFO:tasks.workunit.client.0.smithi038.stdout:3/858: write d2/d1b/d8c/f99 [1081934,108386] 0 2024-02-15T18:35:02.118 INFO:tasks.workunit.client.0.smithi038.stdout:3/859: write d2/d1b/d3c/d42/fc0 [809825,118111] 0 2024-02-15T18:35:02.129 INFO:tasks.workunit.client.0.smithi038.stdout:2/880: getdents dd/d12/d3a/d67 0 2024-02-15T18:35:02.131 INFO:tasks.workunit.client.0.smithi038.stdout:1/903: rmdir d2/d10/d19/d22/deb 39 2024-02-15T18:35:02.132 INFO:tasks.workunit.client.0.smithi038.stdout:7/986: rmdir d0/db/d19/d7d/daa/dd0 39 2024-02-15T18:35:02.136 INFO:tasks.workunit.client.0.smithi038.stdout:6/930: sync 2024-02-15T18:35:02.136 INFO:tasks.workunit.client.0.smithi038.stdout:6/931: dread - d2/d4/d19/d78/dc3/fea zero size 2024-02-15T18:35:02.136 INFO:tasks.workunit.client.0.smithi038.stdout:6/932: write d2/d10/fd1 [1264038,61188] 0 2024-02-15T18:35:02.140 INFO:tasks.workunit.client.0.smithi038.stdout:4/973: creat d11/d1e/d20/f15b x:0 0 0 2024-02-15T18:35:02.141 INFO:tasks.workunit.client.0.smithi038.stdout:5/922: rename d2/fc to d2/d48/f119 0 2024-02-15T18:35:02.142 INFO:tasks.workunit.client.0.smithi038.stdout:3/860: mknod d2/d1b/d3c/d41/d4e/df4/c111 0 2024-02-15T18:35:02.143 INFO:tasks.workunit.client.0.smithi038.stdout:2/881: creat dd/d12/d6b/da5/f115 x:0 0 0 2024-02-15T18:35:02.145 INFO:tasks.workunit.client.0.smithi038.stdout:1/904: symlink d2/d10/d62/dfc/l12f 0 2024-02-15T18:35:02.145 INFO:tasks.workunit.client.0.smithi038.stdout:1/905: stat d2/d10/d19/d22/d36/d4f/cc7 0 2024-02-15T18:35:02.145 INFO:tasks.workunit.client.0.smithi038.stdout:1/906: truncate d2/d10/d19/d5f/fb9 4905777 0 2024-02-15T18:35:02.145 INFO:tasks.workunit.client.0.smithi038.stdout:1/907: chown d2/d2e/d8c 3 1 2024-02-15T18:35:02.146 INFO:tasks.workunit.client.0.smithi038.stdout:7/987: mknod d0/d12f/c164 0 2024-02-15T18:35:02.146 INFO:tasks.workunit.client.0.smithi038.stdout:7/988: chown d0/db/d59/dc8/dd6/d70/d15e/d29/d7e/c15b 16 1 2024-02-15T18:35:02.148 INFO:tasks.workunit.client.0.smithi038.stdout:6/933: creat d2/d4/d19/d55/d5c/f11d x:0 0 0 2024-02-15T18:35:02.148 INFO:tasks.workunit.client.0.smithi038.stdout:6/934: dread - d2/d4/d19/d55/d5c/f11d zero size 2024-02-15T18:35:02.148 INFO:tasks.workunit.client.0.smithi038.stdout:5/923: mkdir d2/d3/d12/d5e/dfd/d101/d11a 0 2024-02-15T18:35:02.148 INFO:tasks.workunit.client.0.smithi038.stdout:5/924: truncate d2/d30/d57/fdb 88229 0 2024-02-15T18:35:02.149 INFO:tasks.workunit.client.0.smithi038.stdout:3/861: truncate d2/d1b/d3c/d3a/d55/fb1 3359746 0 2024-02-15T18:35:02.150 INFO:tasks.workunit.client.0.smithi038.stdout:1/908: mkdir d2/d2e/d130 0 2024-02-15T18:35:02.150 INFO:tasks.workunit.client.0.smithi038.stdout:1/909: write d2/d69/d8b/f11f [383997,108710] 0 2024-02-15T18:35:02.152 INFO:tasks.workunit.client.0.smithi038.stdout:6/935: write d2/d4/d6c/fac [3969602,34369] 0 2024-02-15T18:35:02.153 INFO:tasks.workunit.client.0.smithi038.stdout:5/925: symlink d2/d3/da0/l11b 0 2024-02-15T18:35:02.154 INFO:tasks.workunit.client.0.smithi038.stdout:3/862: rename d2/d1e/f89 to d2/d1b/d3c/d3a/d55/d65/f112 0 2024-02-15T18:35:02.158 INFO:tasks.workunit.client.0.smithi038.stdout:1/910: unlink f1 0 2024-02-15T18:35:02.160 INFO:tasks.workunit.client.0.smithi038.stdout:5/926: write d2/d30/f5d [2331167,34803] 0 2024-02-15T18:35:02.160 INFO:tasks.workunit.client.0.smithi038.stdout:5/927: chown d2/dcc/lb1 58602705 1 2024-02-15T18:35:02.160 INFO:tasks.workunit.client.0.smithi038.stdout:5/928: read d2/d3/da0/d33/f66 [2702895,105242] 0 2024-02-15T18:35:02.163 INFO:tasks.workunit.client.0.smithi038.stdout:5/929: rename d2/d3/da0/d33/ceb to d2/dcc/c11c 0 2024-02-15T18:35:02.166 INFO:tasks.workunit.client.0.smithi038.stdout:4/974: dwrite d11/d1e/d20/d31/d48/dac/f142 [0,4194304] 0 2024-02-15T18:35:02.173 INFO:tasks.workunit.client.0.smithi038.stdout:7/989: dwrite d0/db/d19/d7d/fd3 [0,4194304] 0 2024-02-15T18:35:02.175 INFO:tasks.workunit.client.0.smithi038.stdout:6/936: dwrite d2/d4/d19/d55/fd9 [0,4194304] 0 2024-02-15T18:35:02.175 INFO:tasks.workunit.client.0.smithi038.stdout:4/975: mknod d11/d1e/d20/d31/d48/d11e/c15c 0 2024-02-15T18:35:02.183 INFO:tasks.workunit.client.0.smithi038.stdout:6/937: mknod d2/c11e 0 2024-02-15T18:35:02.184 INFO:tasks.workunit.client.0.smithi038.stdout:4/976: link d11/d1e/d20/d31/f147 d11/d1e/d20/d31/d48/d102/dca/ddc/f15d 0 2024-02-15T18:35:02.187 INFO:tasks.workunit.client.0.smithi038.stdout:6/938: link d2/d4/d6c/fed d2/d4/d2c/d39/db4/dcd/f11f 0 2024-02-15T18:35:02.187 INFO:tasks.workunit.client.0.smithi038.stdout:6/939: readlink d2/d4/d2c/d39/lcc 0 2024-02-15T18:35:02.187 INFO:tasks.workunit.client.0.smithi038.stdout:6/940: read - d2/d4/d19/d78/dbf/f119 zero size 2024-02-15T18:35:02.187 INFO:tasks.workunit.client.0.smithi038.stdout:6/941: truncate d2/d4/d19/d55/fde 795962 0 2024-02-15T18:35:02.189 INFO:tasks.workunit.client.0.smithi038.stdout:6/942: rename d2/d4/d19/d78/dc3 to d2/d4/d6c/d9d/dc7/d105/d120 0 2024-02-15T18:35:02.189 INFO:tasks.workunit.client.0.smithi038.stdout:6/943: chown d2/d4/d19/d78/dc4/df0/lf6 97916 1 2024-02-15T18:35:02.191 INFO:tasks.workunit.client.0.smithi038.stdout:7/990: dwrite d0/d66/f104 [0,4194304] 0 2024-02-15T18:35:02.191 INFO:tasks.workunit.client.0.smithi038.stdout:7/991: readlink d0/d1b/d30/d50/l6b 0 2024-02-15T18:35:02.193 INFO:tasks.workunit.client.0.smithi038.stdout:7/992: getdents d0/d68/dd9 0 2024-02-15T18:35:02.194 INFO:tasks.workunit.client.0.smithi038.stdout:7/993: mknod d0/db/d19/d7d/daa/dd0/d14a/dcd/d134/c165 0 2024-02-15T18:35:02.196 INFO:tasks.workunit.client.0.smithi038.stdout:7/994: rename d0/db/d19/d7d/daa/dd0/d14a/d102/d10a/f11f to d0/d1b/dfd/f166 0 2024-02-15T18:35:02.197 INFO:tasks.workunit.client.0.smithi038.stdout:7/995: rmdir d0/db/d59/dc8/dd6/d70/d15e/d29/d7e/d12a 39 2024-02-15T18:35:02.197 INFO:tasks.workunit.client.0.smithi038.stdout:7/996: chown d0/d1b/d80/dbc 203926 1 2024-02-15T18:35:02.199 INFO:tasks.workunit.client.0.smithi038.stdout:7/997: read d0/db/d59/dc8/dd6/d61/ff1 [758394,112439] 0 2024-02-15T18:35:02.200 INFO:tasks.workunit.client.0.smithi038.stdout:7/998: creat d0/d95/f167 x:0 0 0 2024-02-15T18:35:02.200 INFO:tasks.workunit.client.0.smithi038.stdout:7/999: write d0/d66/f89 [641861,17971] 0 2024-02-15T18:35:02.201 INFO:tasks.workunit.client.0.smithi038.stdout:6/944: dread d2/d4/f1e [0,4194304] 0 2024-02-15T18:35:02.201 INFO:tasks.workunit.client.0.smithi038.stdout:4/977: dwrite d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/faa [0,4194304] 0 2024-02-15T18:35:02.201 INFO:tasks.workunit.client.0.smithi038.stdout:6/945: stat d2/d4/d19/d78/dc4/df0 0 2024-02-15T18:35:02.202 INFO:tasks.workunit.client.0.smithi038.stdout:3/863: dwrite d2/d1e/fd9 [0,4194304] 0 2024-02-15T18:35:02.205 INFO:tasks.workunit.client.0.smithi038.stdout:6/946: symlink d2/d4/d19/l121 0 2024-02-15T18:35:02.207 INFO:tasks.workunit.client.0.smithi038.stdout:4/978: symlink d11/d1e/d20/d31/da2/d118/l15e 0 2024-02-15T18:35:02.210 INFO:tasks.workunit.client.0.smithi038.stdout:3/864: unlink d2/d1b/d3c/d42/fc0 0 2024-02-15T18:35:02.213 INFO:tasks.workunit.client.0.smithi038.stdout:3/865: creat d2/d1b/d8c/f113 x:0 0 0 2024-02-15T18:35:02.214 INFO:tasks.workunit.client.0.smithi038.stdout:3/866: symlink d2/d1b/d3c/d3a/d55/da0/l114 0 2024-02-15T18:35:02.214 INFO:tasks.workunit.client.0.smithi038.stdout:3/867: fdatasync d2/d1b/d69/f96 0 2024-02-15T18:35:02.214 INFO:tasks.workunit.client.0.smithi038.stdout:3/868: stat d2/d1b/lbe 0 2024-02-15T18:35:02.214 INFO:tasks.workunit.client.0.smithi038.stdout:3/869: fsync d2/d1b/d69/f6a 0 2024-02-15T18:35:02.215 INFO:tasks.workunit.client.0.smithi038.stdout:3/870: creat d2/db8/dce/de6/f115 x:0 0 0 2024-02-15T18:35:02.220 INFO:tasks.workunit.client.0.smithi038.stdout:3/871: dread d2/d1b/f94 [0,4194304] 0 2024-02-15T18:35:02.220 INFO:tasks.workunit.client.0.smithi038.stdout:3/872: write d2/f51 [3327558,17325] 0 2024-02-15T18:35:02.233 INFO:tasks.workunit.client.0.smithi038.stdout:3/873: dread d2/d25/f64 [0,4194304] 0 2024-02-15T18:35:02.233 INFO:tasks.workunit.client.0.smithi038.stdout:3/874: creat d2/d24/db9/f116 x:0 0 0 2024-02-15T18:35:02.234 INFO:tasks.workunit.client.0.smithi038.stdout:4/979: dread d11/d1e/d20/d2b/d65/f6d [0,4194304] 0 2024-02-15T18:35:02.235 INFO:tasks.workunit.client.0.smithi038.stdout:3/875: creat d2/d7f/dc1/f117 x:0 0 0 2024-02-15T18:35:02.235 INFO:tasks.workunit.client.0.smithi038.stdout:3/876: fdatasync d2/d1b/d3c/d3a/d55/d65/fd0 0 2024-02-15T18:35:02.236 INFO:tasks.workunit.client.0.smithi038.stdout:4/980: unlink ld 0 2024-02-15T18:35:02.236 INFO:tasks.workunit.client.0.smithi038.stdout:4/981: chown d11/d1e/d20/d31/ffd 18450138 1 2024-02-15T18:35:02.237 INFO:tasks.workunit.client.0.smithi038.stdout:3/877: link d2/d1b/d3c/d3a/d55/fb1 d2/d1b/d3c/d41/d4e/d62/f118 0 2024-02-15T18:35:02.240 INFO:tasks.workunit.client.0.smithi038.stdout:4/982: symlink d11/d1e/d20/d31/d128/l15f 0 2024-02-15T18:35:02.240 INFO:tasks.workunit.client.0.smithi038.stdout:4/983: truncate d11/d1e/d20/f15b 719518 0 2024-02-15T18:35:02.240 INFO:tasks.workunit.client.0.smithi038.stdout:4/984: fsync d11/d1e/d20/d97/f103 0 2024-02-15T18:35:02.240 INFO:tasks.workunit.client.0.smithi038.stdout:4/985: truncate d11/d1e/d20/d31/d48/d102/fad 242335 0 2024-02-15T18:35:02.240 INFO:tasks.workunit.client.0.smithi038.stdout:4/986: write d11/d1e/d20/d2f/f13c [821792,46832] 0 2024-02-15T18:35:02.242 INFO:tasks.workunit.client.0.smithi038.stdout:3/878: creat d2/d1b/d3c/d3a/d55/f119 x:0 0 0 2024-02-15T18:35:02.243 INFO:tasks.workunit.client.0.smithi038.stdout:4/987: getdents d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/d11d 0 2024-02-15T18:35:02.244 INFO:tasks.workunit.client.0.smithi038.stdout:4/988: creat d11/d1e/d20/d31/d48/d102/f160 x:0 0 0 2024-02-15T18:35:02.246 INFO:tasks.workunit.client.0.smithi038.stdout:4/989: creat d11/d1e/d20/f161 x:0 0 0 2024-02-15T18:35:02.246 INFO:tasks.workunit.client.0.smithi038.stdout:4/990: fdatasync d11/d1e/d20/d31/d48/d60/f12b 0 2024-02-15T18:35:02.254 INFO:tasks.workunit.client.0.smithi038.stdout:4/991: dread d11/f17 [0,4194304] 0 2024-02-15T18:35:02.256 INFO:tasks.workunit.client.0.smithi038.stdout:3/879: dwrite d2/d25/d60/f76 [0,4194304] 0 2024-02-15T18:35:02.257 INFO:tasks.workunit.client.0.smithi038.stdout:3/880: dread d2/d1b/d3c/f6b [0,4194304] 0 2024-02-15T18:35:02.258 INFO:tasks.workunit.client.0.smithi038.stdout:3/881: symlink d2/d7f/l11a 0 2024-02-15T18:35:02.258 INFO:tasks.workunit.client.0.smithi038.stdout:3/882: chown d2/d7f/dab 1174639641 1 2024-02-15T18:35:02.260 INFO:tasks.workunit.client.0.smithi038.stdout:3/883: rename d2/d24/db9/f10e to d2/d7f/dab/f11b 0 2024-02-15T18:35:02.261 INFO:tasks.workunit.client.0.smithi038.stdout:3/884: symlink d2/d1e/d7e/d7c/l11c 0 2024-02-15T18:35:02.261 INFO:tasks.workunit.client.0.smithi038.stdout:3/885: stat d2/d1b/d3c/d3a/d55/da0/fc6 0 2024-02-15T18:35:02.262 INFO:tasks.workunit.client.0.smithi038.stdout:3/886: rename d2/fb6 to d2/d1b/d3c/d3a/d55/d65/f11d 0 2024-02-15T18:35:02.264 INFO:tasks.workunit.client.0.smithi038.stdout:3/887: rename d2/d1b/d4f/c54 to d2/d7f/dc1/c11e 0 2024-02-15T18:35:02.264 INFO:tasks.workunit.client.0.smithi038.stdout:4/992: dread d11/d1e/d20/d2b/d4d/d67/d73/fe3 [0,4194304] 0 2024-02-15T18:35:02.264 INFO:tasks.workunit.client.0.smithi038.stdout:3/888: write d2/d1b/d4f/d92/f9c [899615,100917] 0 2024-02-15T18:35:02.266 INFO:tasks.workunit.client.0.smithi038.stdout:3/889: truncate d2/d25/f64 1013765 0 2024-02-15T18:35:02.268 INFO:tasks.workunit.client.0.smithi038.stdout:3/890: creat d2/d1e/de3/f11f x:0 0 0 2024-02-15T18:35:02.275 INFO:tasks.workunit.client.0.smithi038.stdout:3/891: dread d2/d1e/fd9 [0,4194304] 0 2024-02-15T18:35:02.275 INFO:tasks.workunit.client.0.smithi038.stdout:3/892: dread - d2/d1b/d8c/f113 zero size 2024-02-15T18:35:02.277 INFO:tasks.workunit.client.0.smithi038.stdout:3/893: rename d2/d1b/d3c/d3a/c6c to d2/d1b/d3c/d3a/d55/c120 0 2024-02-15T18:35:02.277 INFO:tasks.workunit.client.0.smithi038.stdout:3/894: chown d2/d1e/d7e 146774855 1 2024-02-15T18:35:02.277 INFO:tasks.workunit.client.0.smithi038.stdout:3/895: write d2/d1e/fd1 [295743,83776] 0 2024-02-15T18:35:02.278 INFO:tasks.workunit.client.0.smithi038.stdout:3/896: rmdir d2/d1b/d3c 39 2024-02-15T18:35:02.280 INFO:tasks.workunit.client.0.smithi038.stdout:3/897: dread d2/d1b/d3c/d3a/d55/d65/f112 [0,4194304] 0 2024-02-15T18:35:02.280 INFO:tasks.workunit.client.0.smithi038.stdout:4/993: dwrite d11/d1e/d20/d31/d8d/d99/db5/fcb [0,4194304] 0 2024-02-15T18:35:02.280 INFO:tasks.workunit.client.0.smithi038.stdout:4/994: fdatasync d11/d1e/d20/d31/d48/dac/dbe/d40/d6f/faa 0 2024-02-15T18:35:02.280 INFO:tasks.workunit.client.0.smithi038.stdout:3/898: symlink d2/d7f/df0/l121 0 2024-02-15T18:35:02.281 INFO:tasks.workunit.client.0.smithi038.stdout:3/899: read d2/db8/dcf/fd7 [41102,4076] 0 2024-02-15T18:35:02.284 INFO:tasks.workunit.client.0.smithi038.stdout:4/995: mkdir d11/d1e/d20/d31/d48/dac/dbe/d40/d162 0 2024-02-15T18:35:02.285 INFO:tasks.workunit.client.0.smithi038.stdout:4/996: write d11/d1e/d20/d2b/d4d/d67/da1/d11c/f145 [3183976,21629] 0 2024-02-15T18:35:02.285 INFO:tasks.workunit.client.0.smithi038.stdout:4/997: chown d11/d1e/d20/d31/l50 52355 1 2024-02-15T18:35:02.290 INFO:tasks.workunit.client.0.smithi038.stdout:4/998: creat d11/d1e/d20/d31/d8d/de7/f163 x:0 0 0 2024-02-15T18:35:02.292 INFO:tasks.workunit.client.0.smithi038.stdout:4/999: unlink d11/d1e/d20/d31/d48/d102/dca/d13a/dbd/fd7 0 2024-02-15T18:35:02.295 INFO:tasks.workunit.client.0.smithi038.stdout:3/900: dwrite d2/d1b/d3c/d3a/d55/f81 [0,4194304] 0 2024-02-15T18:35:02.296 INFO:tasks.workunit.client.0.smithi038.stdout:2/882: sync 2024-02-15T18:35:02.306 INFO:tasks.workunit.client.0.smithi038.stdout:1/911: sync 2024-02-15T18:35:02.306 INFO:tasks.workunit.client.0.smithi038.stdout:5/930: sync 2024-02-15T18:35:02.306 INFO:tasks.workunit.client.0.smithi038.stdout:6/947: sync 2024-02-15T18:35:02.306 INFO:tasks.workunit.client.0.smithi038.stdout:6/948: truncate d2/d4/d2c/d39/db4/f7c 4683481 0 2024-02-15T18:35:02.306 INFO:tasks.workunit.client.0.smithi038.stdout:5/931: truncate d2/d3/d12/fa 3060104 0 2024-02-15T18:35:02.306 INFO:tasks.workunit.client.0.smithi038.stdout:6/949: readlink d2/d4/d28/l34 0 2024-02-15T18:35:02.306 INFO:tasks.workunit.client.0.smithi038.stdout:5/932: chown d2/d30/d57/fee 62 1 2024-02-15T18:35:02.306 INFO:tasks.workunit.client.0.smithi038.stdout:5/933: stat d2/d48 0 2024-02-15T18:35:02.306 INFO:tasks.workunit.client.0.smithi038.stdout:5/934: chown d2/d3/d115 0 1 2024-02-15T18:35:02.307 INFO:tasks.workunit.client.0.smithi038.stdout:1/912: truncate d2/d69/d8b/f11f 513806 0 2024-02-15T18:35:02.307 INFO:tasks.workunit.client.0.smithi038.stdout:6/950: creat d2/d4/d19/d55/dc0/f122 x:0 0 0 2024-02-15T18:35:02.309 INFO:tasks.workunit.client.0.smithi038.stdout:1/913: write d2/d10/d62/f93 [3395847,25350] 0 2024-02-15T18:35:02.309 INFO:tasks.workunit.client.0.smithi038.stdout:1/914: rename d2/d10/d19 to d2/d10/d19/d22/d54/dc0/d131 22 2024-02-15T18:35:02.314 INFO:tasks.workunit.client.0.smithi038.stdout:2/883: dwrite dd/fed [0,4194304] 0 2024-02-15T18:35:02.314 INFO:tasks.workunit.client.0.smithi038.stdout:5/935: dread d2/d3/d12/fe7 [0,4194304] 0 2024-02-15T18:35:02.318 INFO:tasks.workunit.client.0.smithi038.stdout:1/915: dread d2/d10/d19/d22/d36/d94/dc1/fe6 [0,4194304] 0 2024-02-15T18:35:02.318 INFO:tasks.workunit.client.0.smithi038.stdout:5/936: unlink d2/d3/da0/d33/l67 0 2024-02-15T18:35:02.320 INFO:tasks.workunit.client.0.smithi038.stdout:5/937: getdents d2/d3/db9/de3/de4/dff 0 2024-02-15T18:35:02.322 INFO:tasks.workunit.client.0.smithi038.stdout:1/916: rmdir d2/d11c 0 2024-02-15T18:35:02.323 INFO:tasks.workunit.client.0.smithi038.stdout:5/938: dread d2/d30/d57/f78 [0,4194304] 0 2024-02-15T18:35:02.325 INFO:tasks.workunit.client.0.smithi038.stdout:1/917: truncate d2/f4 2196493 0 2024-02-15T18:35:02.325 INFO:tasks.workunit.client.0.smithi038.stdout:5/939: write d2/f39 [3632148,111054] 0 2024-02-15T18:35:02.327 INFO:tasks.workunit.client.0.smithi038.stdout:1/918: unlink d2/d10/c6d 0 2024-02-15T18:35:02.328 INFO:tasks.workunit.client.0.smithi038.stdout:5/940: getdents d2/d3/d12/d5e/dfd/d101 0 2024-02-15T18:35:02.328 INFO:tasks.workunit.client.0.smithi038.stdout:1/919: readlink d2/d2e/d8c/lef 0 2024-02-15T18:35:02.339 INFO:tasks.workunit.client.0.smithi038.stdout:5/941: write d2/dcc/fd6 [3458041,87475] 0 2024-02-15T18:35:02.340 INFO:tasks.workunit.client.0.smithi038.stdout:2/884: dwrite dd/d12/d1d/f35 [0,4194304] 0 2024-02-15T18:35:02.340 INFO:tasks.workunit.client.0.smithi038.stdout:2/885: chown dd/d12/d59/d2b/d7a/c70 22 1 2024-02-15T18:35:02.343 INFO:tasks.workunit.client.0.smithi038.stdout:2/886: symlink dd/d12/d1d/d3b/l116 0 2024-02-15T18:35:02.343 INFO:tasks.workunit.client.0.smithi038.stdout:6/951: dwrite d2/d4/d2c/d39/db4/f9c [0,4194304] 0 2024-02-15T18:35:02.344 INFO:tasks.workunit.client.0.smithi038.stdout:2/887: write dd/d12/d3a/d67/f30 [1313358,102959] 0 2024-02-15T18:35:02.345 INFO:tasks.workunit.client.0.smithi038.stdout:6/952: creat d2/d4/d19/d78/dbf/f123 x:0 0 0 2024-02-15T18:35:02.347 INFO:tasks.workunit.client.0.smithi038.stdout:2/888: rmdir dd/d12/d6b/da5 39 2024-02-15T18:35:02.347 INFO:tasks.workunit.client.0.smithi038.stdout:2/889: read - dd/d12/d1d/d3b/d3c/d4a/fb7 zero size 2024-02-15T18:35:02.347 INFO:tasks.workunit.client.0.smithi038.stdout:2/890: chown dd/f41 917 1 2024-02-15T18:35:02.349 INFO:tasks.workunit.client.0.smithi038.stdout:2/891: symlink dd/d12/d6b/l117 0 2024-02-15T18:35:02.350 INFO:tasks.workunit.client.0.smithi038.stdout:2/892: symlink dd/d12/d59/d62/l118 0 2024-02-15T18:35:02.350 INFO:tasks.workunit.client.0.smithi038.stdout:2/893: chown dd/d12/d5d/d93 152788 1 2024-02-15T18:35:02.350 INFO:tasks.workunit.client.0.smithi038.stdout:2/894: write dd/d12/d3a/d67/d4f/fa7 [718119,40985] 0 2024-02-15T18:35:02.362 INFO:tasks.workunit.client.0.smithi038.stdout:1/920: dwrite d2/f2d [0,4194304] 0 2024-02-15T18:35:02.362 INFO:tasks.workunit.client.0.smithi038.stdout:1/921: fsync d2/d69/f83 0 2024-02-15T18:35:02.362 INFO:tasks.workunit.client.0.smithi038.stdout:5/942: dwrite d2/d48/d83/d96/f103 [0,4194304] 0 2024-02-15T18:35:02.362 INFO:tasks.workunit.client.0.smithi038.stdout:5/943: write d2/d48/d83/fe2 [504440,74666] 0 2024-02-15T18:35:02.363 INFO:tasks.workunit.client.0.smithi038.stdout:5/944: dread - d2/d3/db9/de3/f102 zero size 2024-02-15T18:35:02.364 INFO:tasks.workunit.client.0.smithi038.stdout:2/895: dread dd/d12/d3a/d67/f30 [0,4194304] 0 2024-02-15T18:35:02.365 INFO:tasks.workunit.client.0.smithi038.stdout:6/953: dwrite d2/d4/d19/d78/fd8 [0,4194304] 0 2024-02-15T18:35:02.366 INFO:tasks.workunit.client.0.smithi038.stdout:6/954: dread d2/d4/d6c/fcb [0,4194304] 0 2024-02-15T18:35:02.370 INFO:tasks.workunit.client.0.smithi038.stdout:1/922: creat d2/d10/d19/d5f/d109/d10a/f132 x:0 0 0 2024-02-15T18:35:02.372 INFO:tasks.workunit.client.0.smithi038.stdout:5/945: creat d2/d3/db9/dcf/dd8/f11d x:0 0 0 2024-02-15T18:35:02.376 INFO:tasks.workunit.client.0.smithi038.stdout:1/923: dread d2/d10/fa3 [0,4194304] 0 2024-02-15T18:35:02.376 INFO:tasks.workunit.client.0.smithi038.stdout:2/896: mknod dd/d12/d3a/d5e/d65/dcd/c119 0 2024-02-15T18:35:02.376 INFO:tasks.workunit.client.0.smithi038.stdout:1/924: chown d2/d10/d32/d53/df0/l12a 126561888 1 2024-02-15T18:35:02.376 INFO:tasks.workunit.client.0.smithi038.stdout:3/901: sync 2024-02-15T18:35:02.376 INFO:tasks.workunit.client.0.smithi038.stdout:3/902: fsync d2/d1b/d3c/d41/d4e/d62/f75 0 2024-02-15T18:35:02.377 INFO:tasks.workunit.client.0.smithi038.stdout:6/955: truncate d2/d4/d28/f69 3263960 0 2024-02-15T18:35:02.379 INFO:tasks.workunit.client.0.smithi038.stdout:5/946: symlink d2/d3/d12/d5e/dfd/l11e 0 2024-02-15T18:35:02.381 INFO:tasks.workunit.client.0.smithi038.stdout:2/897: creat dd/d12/d5d/d54/dcb/f11a x:0 0 0 2024-02-15T18:35:02.386 INFO:tasks.workunit.client.0.smithi038.stdout:3/903: creat d2/d1b/d69/ddb/f122 x:0 0 0 2024-02-15T18:35:02.386 INFO:tasks.workunit.client.0.smithi038.stdout:5/947: mknod d2/d3/db9/de3/c11f 0 2024-02-15T18:35:02.386 INFO:tasks.workunit.client.0.smithi038.stdout:2/898: creat dd/d12/d3a/d5e/d65/dcd/df1/f11b x:0 0 0 2024-02-15T18:35:02.386 INFO:tasks.workunit.client.0.smithi038.stdout:1/925: creat d2/d10/d32/d53/d10c/f133 x:0 0 0 2024-02-15T18:35:02.386 INFO:tasks.workunit.client.0.smithi038.stdout:5/948: write d2/dcc/fad [104290,52766] 0 2024-02-15T18:35:02.387 INFO:tasks.workunit.client.0.smithi038.stdout:5/949: write d2/dcc/fea [432066,120720] 0 2024-02-15T18:35:02.388 INFO:tasks.workunit.client.0.smithi038.stdout:3/904: rename d2/d1b/d69/cea to d2/d1b/d3c/d41/dd5/c123 0 2024-02-15T18:35:02.388 INFO:tasks.workunit.client.0.smithi038.stdout:3/905: write d2/d7f/dab/f11b [779625,87792] 0 2024-02-15T18:35:02.388 INFO:tasks.workunit.client.0.smithi038.stdout:3/906: write d2/d1b/d3c/d42/fc3 [9027923,94703] 0 2024-02-15T18:35:02.389 INFO:tasks.workunit.client.0.smithi038.stdout:2/899: creat dd/d12/d3a/d5e/d65/f11c x:0 0 0 2024-02-15T18:35:02.393 INFO:tasks.workunit.client.0.smithi038.stdout:3/907: mknod d2/d7f/df0/d10d/c124 0 2024-02-15T18:35:02.393 INFO:tasks.workunit.client.0.smithi038.stdout:3/908: write d2/d1b/d3c/d3a/d55/f119 [237179,68391] 0 2024-02-15T18:35:02.393 INFO:tasks.workunit.client.0.smithi038.stdout:3/909: readlink d2/d1b/d8c/lc5 0 2024-02-15T18:35:02.393 INFO:tasks.workunit.client.0.smithi038.stdout:3/910: fsync d2/d7f/dc1/fdd 0 2024-02-15T18:35:02.393 INFO:tasks.workunit.client.0.smithi038.stdout:2/900: truncate dd/f46 1276889 0 2024-02-15T18:35:02.393 INFO:tasks.workunit.client.0.smithi038.stdout:2/901: chown dd/d12/d3a/d67/d4f/f64 37150 1 2024-02-15T18:35:02.393 INFO:tasks.workunit.client.0.smithi038.stdout:2/902: chown dd/d12/d3a/d5e/lf4 7068325 1 2024-02-15T18:35:02.393 INFO:tasks.workunit.client.0.smithi038.stdout:6/956: dwrite d2/d4/f1f [0,4194304] 0 2024-02-15T18:35:02.393 INFO:tasks.workunit.client.0.smithi038.stdout:2/903: write dd/d12/d3a/d5e/d65/dcd/df1/f11b [427478,76255] 0 2024-02-15T18:35:02.394 INFO:tasks.workunit.client.0.smithi038.stdout:5/950: read d2/d3/d5f/fce [838793,87642] 0 2024-02-15T18:35:02.395 INFO:tasks.workunit.client.0.smithi038.stdout:5/951: write d2/dcc/fea [1560618,25316] 0 2024-02-15T18:35:02.396 INFO:tasks.workunit.client.0.smithi038.stdout:1/926: rmdir d2/d10 39 2024-02-15T18:35:02.397 INFO:tasks.workunit.client.0.smithi038.stdout:6/957: dread d2/d4/d19/d55/f104 [0,4194304] 0 2024-02-15T18:35:02.397 INFO:tasks.workunit.client.0.smithi038.stdout:6/958: chown d2/d4/d28/l34 352381413 1 2024-02-15T18:35:02.399 INFO:tasks.workunit.client.0.smithi038.stdout:6/959: write d2/f5b [3455955,6253] 0 2024-02-15T18:35:02.399 INFO:tasks.workunit.client.0.smithi038.stdout:6/960: readlink d2/l61 0 2024-02-15T18:35:02.400 INFO:tasks.workunit.client.0.smithi038.stdout:3/911: rmdir d2/d7f/dc1 39 2024-02-15T18:35:02.400 INFO:tasks.workunit.client.0.smithi038.stdout:3/912: readlink d2/l16 0 2024-02-15T18:35:02.401 INFO:tasks.workunit.client.0.smithi038.stdout:5/952: unlink d2/ff0 0 2024-02-15T18:35:02.401 INFO:tasks.workunit.client.0.smithi038.stdout:3/913: write d2/d1b/d69/ddb/f122 [798020,8583] 0 2024-02-15T18:35:02.401 INFO:tasks.workunit.client.0.smithi038.stdout:1/927: rename d2/d10/d19/d105/cf9 to d2/d10/d32/d53/df0/c134 0 2024-02-15T18:35:02.401 INFO:tasks.workunit.client.0.smithi038.stdout:5/953: read d2/d48/d83/fe2 [537463,60102] 0 2024-02-15T18:35:02.401 INFO:tasks.workunit.client.0.smithi038.stdout:3/914: chown d2/d7f/d101/ce7 208 1 2024-02-15T18:35:02.401 INFO:tasks.workunit.client.0.smithi038.stdout:3/915: readlink d2/d7f/df0/l121 0 2024-02-15T18:35:02.402 INFO:tasks.workunit.client.0.smithi038.stdout:5/954: mknod d2/d64/c120 0 2024-02-15T18:35:02.402 INFO:tasks.workunit.client.0.smithi038.stdout:6/961: symlink d2/d4/d2c/d39/d76/l124 0 2024-02-15T18:35:02.404 INFO:tasks.workunit.client.0.smithi038.stdout:1/928: mkdir d2/d2e/d8c/d135 0 2024-02-15T18:35:02.407 INFO:tasks.workunit.client.0.smithi038.stdout:5/955: symlink d2/d3/da0/d33/l121 0 2024-02-15T18:35:02.409 INFO:tasks.workunit.client.0.smithi038.stdout:2/904: dwrite dd/d12/d1d/d3b/fb8 [0,4194304] 0 2024-02-15T18:35:02.413 INFO:tasks.workunit.client.0.smithi038.stdout:5/956: creat d2/d3/d12/d5e/f122 x:0 0 0 2024-02-15T18:35:02.413 INFO:tasks.workunit.client.0.smithi038.stdout:5/957: chown d2/d3/da0/d33/fc8 18 1 2024-02-15T18:35:02.415 INFO:tasks.workunit.client.0.smithi038.stdout:5/958: dread d2/d3/f6f [0,4194304] 0 2024-02-15T18:35:02.415 INFO:tasks.workunit.client.0.smithi038.stdout:5/959: dread - d2/d3/db9/de3/de4/dff/f100 zero size 2024-02-15T18:35:02.415 INFO:tasks.workunit.client.0.smithi038.stdout:5/960: write d2/d48/d83/d96/fd3 [1108516,81764] 0 2024-02-15T18:35:02.416 INFO:tasks.workunit.client.0.smithi038.stdout:5/961: readlink d2/d3/da0/d33/lab 0 2024-02-15T18:35:02.416 INFO:tasks.workunit.client.0.smithi038.stdout:2/905: symlink dd/d12/d3a/d67/d4f/d5a/dbc/de8/l11d 0 2024-02-15T18:35:02.418 INFO:tasks.workunit.client.0.smithi038.stdout:5/962: rmdir d2/d30 39 2024-02-15T18:35:02.419 INFO:tasks.workunit.client.0.smithi038.stdout:5/963: write d2/d3/da0/d33/f116 [804286,91534] 0 2024-02-15T18:35:02.420 INFO:tasks.workunit.client.0.smithi038.stdout:2/906: unlink dd/d12/d59/d2b/d6a/ce0 0 2024-02-15T18:35:02.421 INFO:tasks.workunit.client.0.smithi038.stdout:5/964: creat d2/d3/db9/dcf/f123 x:0 0 0 2024-02-15T18:35:02.422 INFO:tasks.workunit.client.0.smithi038.stdout:2/907: mkdir dd/d12/d1d/d3b/d3c/d11e 0 2024-02-15T18:35:02.422 INFO:tasks.workunit.client.0.smithi038.stdout:2/908: dread - dd/d12/d6b/f113 zero size 2024-02-15T18:35:02.423 INFO:tasks.workunit.client.0.smithi038.stdout:5/965: link d2/d64/f8a d2/d3/db9/d113/f124 0 2024-02-15T18:35:02.424 INFO:tasks.workunit.client.0.smithi038.stdout:5/966: mknod d2/d3/d12/c125 0 2024-02-15T18:35:02.425 INFO:tasks.workunit.client.0.smithi038.stdout:5/967: creat d2/dcc/f126 x:0 0 0 2024-02-15T18:35:02.425 INFO:tasks.workunit.client.0.smithi038.stdout:5/968: chown d2/d48/d83/cb5 1 1 2024-02-15T18:35:02.426 INFO:tasks.workunit.client.0.smithi038.stdout:5/969: mkdir d2/d3/db9/de3/d127 0 2024-02-15T18:35:02.426 INFO:tasks.workunit.client.0.smithi038.stdout:5/970: fdatasync d2/d3/da0/d33/f68 0 2024-02-15T18:35:02.427 INFO:tasks.workunit.client.0.smithi038.stdout:5/971: creat d2/d48/d83/d96/f128 x:0 0 0 2024-02-15T18:35:02.427 INFO:tasks.workunit.client.0.smithi038.stdout:5/972: readlink d2/d48/d83/ld7 0 2024-02-15T18:35:02.427 INFO:tasks.workunit.client.0.smithi038.stdout:5/973: dread - d2/d3/d12/d5e/fd9 zero size 2024-02-15T18:35:02.428 INFO:tasks.workunit.client.0.smithi038.stdout:5/974: creat d2/d3/db9/dcf/dd8/f129 x:0 0 0 2024-02-15T18:35:02.429 INFO:tasks.workunit.client.0.smithi038.stdout:5/975: unlink d2/d48/d83/fe2 0 2024-02-15T18:35:02.429 INFO:tasks.workunit.client.0.smithi038.stdout:5/976: write d2/d3/f9e [1252882,95855] 0 2024-02-15T18:35:02.430 INFO:tasks.workunit.client.0.smithi038.stdout:6/962: dwrite d2/d4/d2c/d39/db4/d5f/d99/fb0 [0,4194304] 0 2024-02-15T18:35:02.431 INFO:tasks.workunit.client.0.smithi038.stdout:1/929: dwrite d2/f1e [0,4194304] 0 2024-02-15T18:35:02.431 INFO:tasks.workunit.client.0.smithi038.stdout:2/909: dread dd/d12/d59/db2/fbb [0,4194304] 0 2024-02-15T18:35:02.436 INFO:tasks.workunit.client.0.smithi038.stdout:6/963: dread d2/d4/d2c/d39/db4/f8d [4194304,4194304] 0 2024-02-15T18:35:02.440 INFO:tasks.workunit.client.0.smithi038.stdout:1/930: creat d2/d10/d19/d22/d54/dc3/f136 x:0 0 0 2024-02-15T18:35:02.441 INFO:tasks.workunit.client.0.smithi038.stdout:1/931: write d2/d10/d19/d2f/d3e/fc2 [4442432,116584] 0 2024-02-15T18:35:02.441 INFO:tasks.workunit.client.0.smithi038.stdout:1/932: fsync d2/d2e/d8c/fe9 0 2024-02-15T18:35:02.441 INFO:tasks.workunit.client.0.smithi038.stdout:2/910: mknod dd/d12/d6b/da5/dd8/df9/c11f 0 2024-02-15T18:35:02.441 INFO:tasks.workunit.client.0.smithi038.stdout:5/977: getdents d2/d3/db9/de3 0 2024-02-15T18:35:02.443 INFO:tasks.workunit.client.0.smithi038.stdout:6/964: rmdir d2/d4/d19/d55 39 2024-02-15T18:35:02.444 INFO:tasks.workunit.client.0.smithi038.stdout:1/933: symlink d2/d2e/d130/l137 0 2024-02-15T18:35:02.445 INFO:tasks.workunit.client.0.smithi038.stdout:2/911: symlink dd/d12/d59/db2/l120 0 2024-02-15T18:35:02.445 INFO:tasks.workunit.client.0.smithi038.stdout:2/912: dread - dd/d12/d6b/da5/f115 zero size 2024-02-15T18:35:02.445 INFO:tasks.workunit.client.0.smithi038.stdout:6/965: stat d2/d4/d19/d78/dc4/lca 0 2024-02-15T18:35:02.446 INFO:tasks.workunit.client.0.smithi038.stdout:5/978: dread d2/d3/f1b [0,4194304] 0 2024-02-15T18:35:02.446 INFO:tasks.workunit.client.0.smithi038.stdout:5/979: readlink d2/d3/d12/d89/ld2 0 2024-02-15T18:35:02.446 INFO:tasks.workunit.client.0.smithi038.stdout:2/913: creat dd/d12/d59/dce/f121 x:0 0 0 2024-02-15T18:35:02.446 INFO:tasks.workunit.client.0.smithi038.stdout:5/980: chown d2/d3/d12/d5e/l91 640717 1 2024-02-15T18:35:02.447 INFO:tasks.workunit.client.0.smithi038.stdout:1/934: write d2/d2e/d8c/fae [2029065,40893] 0 2024-02-15T18:35:02.447 INFO:tasks.workunit.client.0.smithi038.stdout:5/981: write d2/d48/d83/d96/fd3 [386443,64080] 0 2024-02-15T18:35:02.447 INFO:tasks.workunit.client.0.smithi038.stdout:1/935: chown d2/d10/d19/d22/f30 15976 1 2024-02-15T18:35:02.447 INFO:tasks.workunit.client.0.smithi038.stdout:1/936: write d2/d10/d62/f93 [4907885,119314] 0 2024-02-15T18:35:02.448 INFO:tasks.workunit.client.0.smithi038.stdout:6/966: mkdir d2/d10/dd0/d125 0 2024-02-15T18:35:02.448 INFO:tasks.workunit.client.0.smithi038.stdout:1/937: read d2/d69/d8b/f11f [132848,119748] 0 2024-02-15T18:35:02.448 INFO:tasks.workunit.client.0.smithi038.stdout:6/967: symlink d2/d4/d2c/d39/db4/d5f/l126 0 2024-02-15T18:35:02.448 INFO:tasks.workunit.client.0.smithi038.stdout:6/968: read - d2/d4/d19/d55/dc0/ff9 zero size 2024-02-15T18:35:02.450 INFO:tasks.workunit.client.0.smithi038.stdout:6/969: rename d2/d4/d2c/cb7 to d2/d4/d6c/d9d/dc7/d105/d120/d109/c127 0 2024-02-15T18:35:02.455 INFO:tasks.workunit.client.0.smithi038.stdout:6/970: rename d2/d4/d6f/c89 to d2/c128 0 2024-02-15T18:35:02.455 INFO:tasks.workunit.client.0.smithi038.stdout:2/914: dread dd/d12/d3a/d5e/f87 [0,4194304] 0 2024-02-15T18:35:02.459 INFO:tasks.workunit.client.0.smithi038.stdout:6/971: write d2/d10/f96 [274743,79289] 0 2024-02-15T18:35:02.460 INFO:tasks.workunit.client.0.smithi038.stdout:2/915: dread dd/d12/d3a/d5e/d65/dcd/df1/f11b [0,4194304] 0 2024-02-15T18:35:02.461 INFO:tasks.workunit.client.0.smithi038.stdout:6/972: rename d2/d10/dd0/c10b to d2/d10/db6/c129 0 2024-02-15T18:35:02.461 INFO:tasks.workunit.client.0.smithi038.stdout:6/973: chown d2/d4/d19/d55/d5c/faa 2149548 1 2024-02-15T18:35:02.462 INFO:tasks.workunit.client.0.smithi038.stdout:2/916: creat dd/d12/d3a/d5e/d65/dcd/df1/f122 x:0 0 0 2024-02-15T18:35:02.463 INFO:tasks.workunit.client.0.smithi038.stdout:6/974: creat d2/f12a x:0 0 0 2024-02-15T18:35:02.463 INFO:tasks.workunit.client.0.smithi038.stdout:6/975: fdatasync d2/d4/d2c/d39/db4/d5f/ffd 0 2024-02-15T18:35:02.464 INFO:tasks.workunit.client.0.smithi038.stdout:2/917: rename dd/d12/d59/d2b/c42 to dd/d12/d59/dce/dea/c123 0 2024-02-15T18:35:02.464 INFO:tasks.workunit.client.0.smithi038.stdout:2/918: readlink dd/d12/d1d/d3b/l116 0 2024-02-15T18:35:02.465 INFO:tasks.workunit.client.0.smithi038.stdout:6/976: creat d2/d10/f12b x:0 0 0 2024-02-15T18:35:02.465 INFO:tasks.workunit.client.0.smithi038.stdout:2/919: creat dd/d12/d6b/f124 x:0 0 0 2024-02-15T18:35:02.466 INFO:tasks.workunit.client.0.smithi038.stdout:2/920: read dd/d12/d3a/d67/f29 [3224794,44419] 0 2024-02-15T18:35:02.466 INFO:tasks.workunit.client.0.smithi038.stdout:6/977: creat d2/d4/f12c x:0 0 0 2024-02-15T18:35:02.466 INFO:tasks.workunit.client.0.smithi038.stdout:6/978: dread - d2/d4/d28/f11c zero size 2024-02-15T18:35:02.467 INFO:tasks.workunit.client.0.smithi038.stdout:6/979: truncate d2/f8e 1072975 0 2024-02-15T18:35:02.467 INFO:tasks.workunit.client.0.smithi038.stdout:6/980: chown d2/d10/dd0/d125 15315277 1 2024-02-15T18:35:02.468 INFO:tasks.workunit.client.0.smithi038.stdout:2/921: dread dd/d12/d3a/d67/f30 [0,4194304] 0 2024-02-15T18:35:02.468 INFO:tasks.workunit.client.0.smithi038.stdout:2/922: chown dd/d12/d59/db2/f100 26131 1 2024-02-15T18:35:02.468 INFO:tasks.workunit.client.0.smithi038.stdout:2/923: chown dd/d12/d3a/d67/d4f/d5a/dbc/de8/d90 52 1 2024-02-15T18:35:02.468 INFO:tasks.workunit.client.0.smithi038.stdout:2/924: chown dd/d12/d3a/d5e/f87 41261 1 2024-02-15T18:35:02.469 INFO:tasks.workunit.client.0.smithi038.stdout:6/981: mknod d2/d4/d2c/d39/c12d 0 2024-02-15T18:35:02.469 INFO:tasks.workunit.client.0.smithi038.stdout:1/938: dwrite d2/d10/d19/d22/fe8 [0,4194304] 0 2024-02-15T18:35:02.470 INFO:tasks.workunit.client.0.smithi038.stdout:5/982: dwrite d2/d3/db9/dcf/dd8/f112 [0,4194304] 0 2024-02-15T18:35:02.470 INFO:tasks.workunit.client.0.smithi038.stdout:2/925: creat dd/d12/d6b/da5/dd8/f125 x:0 0 0 2024-02-15T18:35:02.470 INFO:tasks.workunit.client.0.smithi038.stdout:2/926: write dd/d12/d1d/ff5 [838110,75962] 0 2024-02-15T18:35:02.471 INFO:tasks.workunit.client.0.smithi038.stdout:1/939: getdents d2/d10/d19/d22/deb 0 2024-02-15T18:35:02.472 INFO:tasks.workunit.client.0.smithi038.stdout:5/983: write d2/d3/d12/f19 [3704339,7506] 0 2024-02-15T18:35:02.472 INFO:tasks.workunit.client.0.smithi038.stdout:5/984: truncate d2/d48/d83/d96/f128 695867 0 2024-02-15T18:35:02.473 INFO:tasks.workunit.client.0.smithi038.stdout:6/982: dread d2/d4/d2c/d39/d76/dc2/fce [0,4194304] 0 2024-02-15T18:35:02.474 INFO:tasks.workunit.client.0.smithi038.stdout:2/927: unlink dd/d12/d6b/da5/dd8/cb9 0 2024-02-15T18:35:02.474 INFO:tasks.workunit.client.0.smithi038.stdout:2/928: read dd/d12/d1d/d3b/d3c/f48 [496370,121065] 0 2024-02-15T18:35:02.475 INFO:tasks.workunit.client.0.smithi038.stdout:1/940: rmdir d2/d69/da9 39 2024-02-15T18:35:02.486 INFO:tasks.workunit.client.0.smithi038.stdout:5/985: rmdir d2/d3/d12 39 2024-02-15T18:35:02.488 INFO:tasks.workunit.client.0.smithi038.stdout:6/983: getdents d2/d4/d2c/d39/d76/dc2 0 2024-02-15T18:35:02.489 INFO:tasks.workunit.client.0.smithi038.stdout:6/984: write d2/d4/d2c/d39/db4/d5f/f102 [625835,4999] 0 2024-02-15T18:35:02.489 INFO:tasks.workunit.client.0.smithi038.stdout:6/985: readlink d2/l61 0 2024-02-15T18:35:02.489 INFO:tasks.workunit.client.0.smithi038.stdout:5/986: write d2/d64/d99/db6/fe8 [1955919,60500] 0 2024-02-15T18:35:02.489 INFO:tasks.workunit.client.0.smithi038.stdout:1/941: dwrite d2/d10/d19/d22/d36/d94/dc1/f107 [0,4194304] 0 2024-02-15T18:35:02.491 INFO:tasks.workunit.client.0.smithi038.stdout:2/929: creat dd/d12/d5d/d54/dcb/de1/d104/f126 x:0 0 0 2024-02-15T18:35:02.491 INFO:tasks.workunit.client.0.smithi038.stdout:2/930: chown dd/d12/d3a/d67/d4f/d5a/dbc/de8/l11d 1319740 1 2024-02-15T18:35:02.491 INFO:tasks.workunit.client.0.smithi038.stdout:5/987: write d2/d64/d99/db6/fe8 [2769308,46988] 0 2024-02-15T18:35:02.491 INFO:tasks.workunit.client.0.smithi038.stdout:5/988: write d2/d3/f6f [931512,105214] 0 2024-02-15T18:35:02.491 INFO:tasks.workunit.client.0.smithi038.stdout:5/989: readlink d2/d30/lc5 0 2024-02-15T18:35:02.491 INFO:tasks.workunit.client.0.smithi038.stdout:6/986: rmdir d2/d4/d6c/d9d/dc7/d105 39 2024-02-15T18:35:02.497 INFO:tasks.workunit.client.0.smithi038.stdout:5/990: dread d2/d3/da0/d33/fc8 [0,4194304] 0 2024-02-15T18:35:02.502 INFO:tasks.workunit.client.0.smithi038.stdout:5/991: mkdir d2/d3/db9/de3/d12a 0 2024-02-15T18:35:02.502 INFO:tasks.workunit.client.0.smithi038.stdout:2/931: truncate dd/fed 2674029 0 2024-02-15T18:35:02.502 INFO:tasks.workunit.client.0.smithi038.stdout:2/932: stat dd/d12/d3a/d67/d4f/d5a/dbc/de8/fa8 0 2024-02-15T18:35:02.503 INFO:tasks.workunit.client.0.smithi038.stdout:5/992: mknod d2/d3/db9/d113/c12b 0 2024-02-15T18:35:02.504 INFO:tasks.workunit.client.0.smithi038.stdout:2/933: link dd/d12/d59/db2/fbb dd/d12/d5d/d93/f127 0 2024-02-15T18:35:02.507 INFO:tasks.workunit.client.0.smithi038.stdout:5/993: rmdir d2/d48 39 2024-02-15T18:35:02.510 INFO:tasks.workunit.client.0.smithi038.stdout:1/942: dwrite d2/d10/d19/d5f/fb9 [0,4194304] 0 2024-02-15T18:35:02.510 INFO:tasks.workunit.client.0.smithi038.stdout:1/943: fsync d2/d10/d19/d22/d54/fd9 0 2024-02-15T18:35:02.513 INFO:tasks.workunit.client.0.smithi038.stdout:6/987: dread d2/d10/f96 [0,4194304] 0 2024-02-15T18:35:02.514 INFO:tasks.workunit.client.0.smithi038.stdout:3/916: sync 2024-02-15T18:35:02.514 INFO:tasks.workunit.client.0.smithi038.stdout:3/917: fdatasync d2/d1b/d4f/fac 0 2024-02-15T18:35:02.515 INFO:tasks.workunit.client.0.smithi038.stdout:3/918: read d2/d7f/d101/dfa/f100 [588369,47705] 0 2024-02-15T18:35:02.520 INFO:tasks.workunit.client.0.smithi038.stdout:6/988: truncate d2/d10/f6b 2535195 0 2024-02-15T18:35:02.523 INFO:tasks.workunit.client.0.smithi038.stdout:1/944: rename d2/d10/d19/d22/d54/dc0/d6b/cac to d2/d10/d19/d22/d54/dba/c138 0 2024-02-15T18:35:02.527 INFO:tasks.workunit.client.0.smithi038.stdout:1/945: creat d2/d10/d19/d22/d36/d94/dc1/dd5/f139 x:0 0 0 2024-02-15T18:35:02.527 INFO:tasks.workunit.client.0.smithi038.stdout:5/994: dwrite d2/d3/d5f/fde [0,4194304] 0 2024-02-15T18:35:02.533 INFO:tasks.workunit.client.0.smithi038.stdout:3/919: dwrite d2/d1b/d3c/f6b [0,4194304] 0 2024-02-15T18:35:02.533 INFO:tasks.workunit.client.0.smithi038.stdout:3/920: write d2/d1b/d69/fd8 [1042162,84217] 0 2024-02-15T18:35:02.534 INFO:tasks.workunit.client.0.smithi038.stdout:6/989: rename d2/d10 to d2/d4/d2c/d39/db4/dcd/d12e 0 2024-02-15T18:35:02.534 INFO:tasks.workunit.client.0.smithi038.stdout:6/990: readlink d2/d4/d19/d55/d5c/dc8/l10f 0 2024-02-15T18:35:02.534 INFO:tasks.workunit.client.0.smithi038.stdout:6/991: readlink d2/d4/d6c/d9d/lba 0 2024-02-15T18:35:02.536 INFO:tasks.workunit.client.0.smithi038.stdout:3/921: write d2/d25/f26 [1815139,120092] 0 2024-02-15T18:35:02.539 INFO:tasks.workunit.client.0.smithi038.stdout:6/992: mkdir d2/d4/d2c/d39/db4/dcd/d12f 0 2024-02-15T18:35:02.540 INFO:tasks.workunit.client.0.smithi038.stdout:3/922: mkdir d2/d7f/d125 0 2024-02-15T18:35:02.540 INFO:tasks.workunit.client.0.smithi038.stdout:6/993: symlink d2/d4/d2c/d39/db4/dcd/d12e/dd0/d125/l130 0 2024-02-15T18:35:02.543 INFO:tasks.workunit.client.0.smithi038.stdout:3/923: write d2/d1b/d4f/d92/fde [1569866,81291] 0 2024-02-15T18:35:02.544 INFO:tasks.workunit.client.0.smithi038.stdout:3/924: mknod d2/c126 0 2024-02-15T18:35:02.545 INFO:tasks.workunit.client.0.smithi038.stdout:3/925: symlink d2/d1b/d3c/d3a/d55/df8/l127 0 2024-02-15T18:35:02.545 INFO:tasks.workunit.client.0.smithi038.stdout:3/926: stat d2/f15 0 2024-02-15T18:35:02.545 INFO:tasks.workunit.client.0.smithi038.stdout:3/927: chown d2/d1b/f4a 12756 1 2024-02-15T18:35:02.551 INFO:tasks.workunit.client.0.smithi038.stdout:5/995: dwrite d2/d64/d99/dbc/f110 [0,4194304] 0 2024-02-15T18:35:02.551 INFO:tasks.workunit.client.0.smithi038.stdout:1/946: dwrite d2/d10/d19/d22/d54/fd9 [0,4194304] 0 2024-02-15T18:35:02.553 INFO:tasks.workunit.client.0.smithi038.stdout:5/996: write d2/d3/db9/dcf/fa9 [1795413,30455] 0 2024-02-15T18:35:02.553 INFO:tasks.workunit.client.0.smithi038.stdout:5/997: truncate d2/d3/db9/fcb 1903466 0 2024-02-15T18:35:02.553 INFO:tasks.workunit.client.0.smithi038.stdout:5/998: stat d2/d48/f10e 0 2024-02-15T18:35:02.553 INFO:tasks.workunit.client.0.smithi038.stdout:5/999: chown d2/d3/db9/dcf/fa9 699668 1 2024-02-15T18:35:02.557 INFO:tasks.workunit.client.0.smithi038.stdout:1/947: creat d2/d10/d19/d5f/d92/f13a x:0 0 0 2024-02-15T18:35:02.560 INFO:tasks.workunit.client.0.smithi038.stdout:3/928: dwrite d2/d1b/f8e [0,4194304] 0 2024-02-15T18:35:02.575 INFO:tasks.workunit.client.0.smithi038.stdout:1/948: dwrite d2/d10/d19/d22/d36/d94/dc1/f107 [0,4194304] 0 2024-02-15T18:35:02.577 INFO:tasks.workunit.client.0.smithi038.stdout:1/949: getdents d2/d69 0 2024-02-15T18:35:02.577 INFO:tasks.workunit.client.0.smithi038.stdout:1/950: read d2/d2e/f7d [111359,56663] 0 2024-02-15T18:35:02.577 INFO:tasks.workunit.client.0.smithi038.stdout:1/951: chown d2/d10/d19/d2f/d3e/fc2 295345414 1 2024-02-15T18:35:02.578 INFO:tasks.workunit.client.0.smithi038.stdout:1/952: creat d2/d10/d19/d22/d54/f13b x:0 0 0 2024-02-15T18:35:02.579 INFO:tasks.workunit.client.0.smithi038.stdout:1/953: creat d2/d69/da9/f13c x:0 0 0 2024-02-15T18:35:02.580 INFO:tasks.workunit.client.0.smithi038.stdout:3/929: dwrite d2/d1b/d3c/d3a/d55/f80 [0,4194304] 0 2024-02-15T18:35:02.580 INFO:tasks.workunit.client.0.smithi038.stdout:1/954: rename d2/d10/d19/d22/d36/d94/dc1/dd5/f139 to d2/d10/d19/d105/f13d 0 2024-02-15T18:35:02.581 INFO:tasks.workunit.client.0.smithi038.stdout:3/930: read d2/d1b/f36 [1407561,30303] 0 2024-02-15T18:35:02.581 INFO:tasks.workunit.client.0.smithi038.stdout:3/931: chown d2/d1b/d3c/l8a 76 1 2024-02-15T18:35:02.582 INFO:tasks.workunit.client.0.smithi038.stdout:3/932: rmdir d2/d7f/dab 39 2024-02-15T18:35:02.583 INFO:tasks.workunit.client.0.smithi038.stdout:3/933: mkdir d2/d7f/d101/d128 0 2024-02-15T18:35:02.585 INFO:tasks.workunit.client.0.smithi038.stdout:3/934: creat d2/d1b/d3c/d3a/dbb/f129 x:0 0 0 2024-02-15T18:35:02.586 INFO:tasks.workunit.client.0.smithi038.stdout:3/935: write d2/d1b/d3c/d3a/d55/da0/fc6 [130225,92474] 0 2024-02-15T18:35:02.587 INFO:tasks.workunit.client.0.smithi038.stdout:3/936: link d2/d25/d60/f76 d2/d1e/d7e/f12a 0 2024-02-15T18:35:02.588 INFO:tasks.workunit.client.0.smithi038.stdout:3/937: write d2/d25/f26 [5115727,48789] 0 2024-02-15T18:35:02.589 INFO:tasks.workunit.client.0.smithi038.stdout:3/938: creat d2/d7f/df0/f12b x:0 0 0 2024-02-15T18:35:02.598 INFO:tasks.workunit.client.0.smithi038.stdout:1/955: dwrite d2/d69/fb4 [0,4194304] 0 2024-02-15T18:35:02.600 INFO:tasks.workunit.client.0.smithi038.stdout:1/956: symlink d2/d10/d19/d22/l13e 0 2024-02-15T18:35:02.602 INFO:tasks.workunit.client.0.smithi038.stdout:1/957: creat d2/d12c/f13f x:0 0 0 2024-02-15T18:35:02.602 INFO:tasks.workunit.client.0.smithi038.stdout:1/958: readlink d2/d69/l127 0 2024-02-15T18:35:02.603 INFO:tasks.workunit.client.0.smithi038.stdout:1/959: rmdir d2/d10/d19/d22/d54/dc0/d6b 39 2024-02-15T18:35:02.608 INFO:tasks.workunit.client.0.smithi038.stdout:3/939: dwrite d2/d7f/df0/f12b [0,4194304] 0 2024-02-15T18:35:02.614 INFO:tasks.workunit.client.0.smithi038.stdout:1/960: dread d2/d2e/f38 [0,4194304] 0 2024-02-15T18:35:02.617 INFO:tasks.workunit.client.0.smithi038.stdout:1/961: symlink d2/d69/da9/dad/l140 0 2024-02-15T18:35:02.618 INFO:tasks.workunit.client.0.smithi038.stdout:1/962: creat d2/d10/d19/d22/d36/f141 x:0 0 0 2024-02-15T18:35:02.619 INFO:tasks.workunit.client.0.smithi038.stdout:1/963: truncate d2/d10/d19/d22/f2c 1510525 0 2024-02-15T18:35:02.620 INFO:tasks.workunit.client.0.smithi038.stdout:1/964: mkdir d2/d69/da9/dad/df3/d142 0 2024-02-15T18:35:02.624 INFO:tasks.workunit.client.0.smithi038.stdout:1/965: dread d2/d10/d19/d22/d36/d94/f9c [0,4194304] 0 2024-02-15T18:35:02.626 INFO:tasks.workunit.client.0.smithi038.stdout:3/940: dwrite d2/d1b/d3c/d41/f56 [0,4194304] 0 2024-02-15T18:35:02.630 INFO:tasks.workunit.client.0.smithi038.stdout:3/941: rmdir d2/d7f/dc1 39 2024-02-15T18:35:02.632 INFO:tasks.workunit.client.0.smithi038.stdout:2/934: sync 2024-02-15T18:35:02.633 INFO:tasks.workunit.client.0.smithi038.stdout:2/935: dread - dd/d12/d5d/dc2/ff6 zero size 2024-02-15T18:35:02.634 INFO:tasks.workunit.client.0.smithi038.stdout:2/936: mknod dd/d12/d5d/d54/dcb/c128 0 2024-02-15T18:35:02.637 INFO:tasks.workunit.client.0.smithi038.stdout:2/937: symlink dd/d12/d1d/l129 0 2024-02-15T18:35:02.641 INFO:tasks.workunit.client.0.smithi038.stdout:1/966: dwrite d2/d39/f7f [0,4194304] 0 2024-02-15T18:35:02.643 INFO:tasks.workunit.client.0.smithi038.stdout:3/942: dwrite d2/d1b/d3c/d41/d4e/fa3 [0,4194304] 0 2024-02-15T18:35:02.643 INFO:tasks.workunit.client.0.smithi038.stdout:1/967: rename d2/d10/d19/d2f/d3e/ccc to d2/d10/d62/c143 0 2024-02-15T18:35:02.645 INFO:tasks.workunit.client.0.smithi038.stdout:3/943: mkdir d2/d1e/d7e/d7c/d12c 0 2024-02-15T18:35:02.645 INFO:tasks.workunit.client.0.smithi038.stdout:3/944: read d2/d7f/dab/f11b [763635,69467] 0 2024-02-15T18:35:02.646 INFO:tasks.workunit.client.0.smithi038.stdout:1/968: creat d2/d10/d19/d22/d36/d4f/f144 x:0 0 0 2024-02-15T18:35:02.646 INFO:tasks.workunit.client.0.smithi038.stdout:3/945: symlink d2/d7f/dc1/l12d 0 2024-02-15T18:35:02.648 INFO:tasks.workunit.client.0.smithi038.stdout:3/946: symlink d2/d7f/dc1/l12e 0 2024-02-15T18:35:02.649 INFO:tasks.workunit.client.0.smithi038.stdout:3/947: write d2/d1b/d8c/f99 [25114,14500] 0 2024-02-15T18:35:02.649 INFO:tasks.workunit.client.0.smithi038.stdout:3/948: dread - d2/d1b/d8c/f113 zero size 2024-02-15T18:35:02.649 INFO:tasks.workunit.client.0.smithi038.stdout:3/949: write d2/f15 [4718675,89563] 0 2024-02-15T18:35:02.652 INFO:tasks.workunit.client.0.smithi038.stdout:1/969: dread d2/d10/d19/d22/f30 [0,4194304] 0 2024-02-15T18:35:02.652 INFO:tasks.workunit.client.0.smithi038.stdout:1/970: chown d2/d10/d19/d2f/d3e/ld3 1 1 2024-02-15T18:35:02.664 INFO:tasks.workunit.client.0.smithi038.stdout:2/938: dwrite dd/d12/d5d/d54/dcb/de1/d104/f126 [0,4194304] 0 2024-02-15T18:35:02.666 INFO:tasks.workunit.client.0.smithi038.stdout:2/939: mknod dd/d12/d3a/d67/d4f/d5a/dbc/c12a 0 2024-02-15T18:35:02.666 INFO:tasks.workunit.client.0.smithi038.stdout:2/940: rmdir dd/d12/d3a/d67/d4f/d5a/d9d 39 2024-02-15T18:35:02.667 INFO:tasks.workunit.client.0.smithi038.stdout:2/941: mkdir dd/d12/d6b/da5/d12b 0 2024-02-15T18:35:02.669 INFO:tasks.workunit.client.0.smithi038.stdout:2/942: mkdir dd/d12c 0 2024-02-15T18:35:02.669 INFO:tasks.workunit.client.0.smithi038.stdout:2/943: truncate dd/d12/d3a/d67/d4f/d5a/d9d/fac 1434867 0 2024-02-15T18:35:02.669 INFO:tasks.workunit.client.0.smithi038.stdout:2/944: chown dd/d12/d5d/dc2/c111 1 1 2024-02-15T18:35:02.670 INFO:tasks.workunit.client.0.smithi038.stdout:2/945: mkdir dd/d12/d3a/d5e/d12d 0 2024-02-15T18:35:02.670 INFO:tasks.workunit.client.0.smithi038.stdout:2/946: creat dd/d12c/f12e x:0 0 0 2024-02-15T18:35:02.671 INFO:tasks.workunit.client.0.smithi038.stdout:2/947: chown dd/d12/d59/d2b/d7a/fe7 44925776 1 2024-02-15T18:35:02.672 INFO:tasks.workunit.client.0.smithi038.stdout:3/950: dwrite d2/d1b/d3c/d3a/d55/da0/fe4 [0,4194304] 0 2024-02-15T18:35:02.672 INFO:tasks.workunit.client.0.smithi038.stdout:3/951: chown d2/d24/c5b 128 1 2024-02-15T18:35:02.674 INFO:tasks.workunit.client.0.smithi038.stdout:1/971: dwrite d2/d2e/d8c/fa0 [8388608,4194304] 0 2024-02-15T18:35:02.674 INFO:tasks.workunit.client.0.smithi038.stdout:1/972: write d2/d69/d8b/f11f [940336,98678] 0 2024-02-15T18:35:02.674 INFO:tasks.workunit.client.0.smithi038.stdout:1/973: write d2/d10/d19/d22/d36/d94/dc1/dd5/f119 [154641,2663] 0 2024-02-15T18:35:02.674 INFO:tasks.workunit.client.0.smithi038.stdout:1/974: chown d2/d69/da9/dad/df3/f121 10 1 2024-02-15T18:35:02.674 INFO:tasks.workunit.client.0.smithi038.stdout:3/952: write d2/f9 [1574228,86085] 0 2024-02-15T18:35:02.684 INFO:tasks.workunit.client.0.smithi038.stdout:3/953: rename d2/d24/f2e to d2/db8/dcf/f12f 0 2024-02-15T18:35:02.686 INFO:tasks.workunit.client.0.smithi038.stdout:3/954: creat d2/d1b/f130 x:0 0 0 2024-02-15T18:35:02.686 INFO:tasks.workunit.client.0.smithi038.stdout:3/955: stat d2/d1b/f8e 0 2024-02-15T18:35:02.686 INFO:tasks.workunit.client.0.smithi038.stdout:2/948: dread dd/d12/d1d/d3b/d3c/d4a/fdd [0,4194304] 0 2024-02-15T18:35:02.692 INFO:tasks.workunit.client.0.smithi038.stdout:6/994: sync 2024-02-15T18:35:02.693 INFO:tasks.workunit.client.0.smithi038.stdout:1/975: dwrite d2/d10/d19/d22/d54/dc3/f136 [0,4194304] 0 2024-02-15T18:35:02.694 INFO:tasks.workunit.client.0.smithi038.stdout:6/995: mknod d2/d4/d19/d78/dbf/c131 0 2024-02-15T18:35:02.697 INFO:tasks.workunit.client.0.smithi038.stdout:6/996: truncate d2/d4/d19/f74 4000475 0 2024-02-15T18:35:02.697 INFO:tasks.workunit.client.0.smithi038.stdout:6/997: chown d2/d4/d2c/d39/db4/dcd/d12e/ffa 28693 1 2024-02-15T18:35:02.699 INFO:tasks.workunit.client.0.smithi038.stdout:6/998: unlink d2/d4/d19/d55/f117 0 2024-02-15T18:35:02.699 INFO:tasks.workunit.client.0.smithi038.stdout:6/999: chown d2/d4/d19/d55/f104 29 1 2024-02-15T18:35:02.703 INFO:tasks.workunit.client.0.smithi038.stdout:2/949: dwrite dd/d12/d5d/f28 [0,4194304] 0 2024-02-15T18:35:02.703 INFO:tasks.workunit.client.0.smithi038.stdout:2/950: getdents dd/d12/d3a/d5e 0 2024-02-15T18:35:02.705 INFO:tasks.workunit.client.0.smithi038.stdout:2/951: mkdir dd/d12/d5d/d54/dcb/de1/d104/d12f 0 2024-02-15T18:35:02.705 INFO:tasks.workunit.client.0.smithi038.stdout:2/952: stat dd/d12/d3a/d5e/c9f 0 2024-02-15T18:35:02.712 INFO:tasks.workunit.client.0.smithi038.stdout:1/976: dwrite d2/d69/da9/f13c [0,4194304] 0 2024-02-15T18:35:02.714 INFO:tasks.workunit.client.0.smithi038.stdout:2/953: write dd/d12/d6b/fa3 [1445718,4164] 0 2024-02-15T18:35:02.714 INFO:tasks.workunit.client.0.smithi038.stdout:2/954: fdatasync dd/d12/f36 0 2024-02-15T18:35:02.715 INFO:tasks.workunit.client.0.smithi038.stdout:2/955: fdatasync dd/d12/d5d/f28 0 2024-02-15T18:35:02.715 INFO:tasks.workunit.client.0.smithi038.stdout:1/977: rename d2/d10/d32/d53/f12b to d2/d10/d19/d22/f145 0 2024-02-15T18:35:02.717 INFO:tasks.workunit.client.0.smithi038.stdout:2/956: creat dd/d12c/f130 x:0 0 0 2024-02-15T18:35:02.726 INFO:tasks.workunit.client.0.smithi038.stdout:2/957: dread dd/f77 [0,4194304] 0 2024-02-15T18:35:02.726 INFO:tasks.workunit.client.0.smithi038.stdout:1/978: dwrite d2/d10/d19/d2f/fd6 [0,4194304] 0 2024-02-15T18:35:02.733 INFO:tasks.workunit.client.0.smithi038.stdout:1/979: dread d2/d10/d19/d22/d36/d94/dc1/dd5/fe7 [0,4194304] 0 2024-02-15T18:35:02.735 INFO:tasks.workunit.client.0.smithi038.stdout:1/980: unlink d2/d10/d32/d53/d10c/f111 0 2024-02-15T18:35:02.737 INFO:tasks.workunit.client.0.smithi038.stdout:1/981: rename d2/d69/da9/dad/df3 to d2/d10/d32/d53/df0/d146 0 2024-02-15T18:35:02.738 INFO:tasks.workunit.client.0.smithi038.stdout:1/982: read d2/d10/d19/d5f/fdf [186426,80810] 0 2024-02-15T18:35:02.738 INFO:tasks.workunit.client.0.smithi038.stdout:1/983: read - d2/d10/d19/d22/f145 zero size 2024-02-15T18:35:02.738 INFO:tasks.workunit.client.0.smithi038.stdout:1/984: truncate d2/d10/d19/d22/d54/dc0/d6b/ff1 986078 0 2024-02-15T18:35:02.738 INFO:tasks.workunit.client.0.smithi038.stdout:1/985: fsync d2/d10/d19/d22/f2c 0 2024-02-15T18:35:02.738 INFO:tasks.workunit.client.0.smithi038.stdout:1/986: fsync d2/d39/f7f 0 2024-02-15T18:35:02.739 INFO:tasks.workunit.client.0.smithi038.stdout:1/987: chown d2/d10/d19/d22/d36/d4f/l76 95856 1 2024-02-15T18:35:02.740 INFO:tasks.workunit.client.0.smithi038.stdout:1/988: rename d2/d10/d19/d22/d36/d94/dc1/fe6 to d2/d10/d19/d2f/d3e/f147 0 2024-02-15T18:35:02.741 INFO:tasks.workunit.client.0.smithi038.stdout:1/989: truncate d2/d39/f7f 567331 0 2024-02-15T18:35:02.742 INFO:tasks.workunit.client.0.smithi038.stdout:1/990: write d2/d10/d62/fd2 [1941593,58499] 0 2024-02-15T18:35:02.743 INFO:tasks.workunit.client.0.smithi038.stdout:1/991: rename d2/d10/d19/d22/c29 to d2/d2e/d8c/d135/c148 0 2024-02-15T18:35:02.745 INFO:tasks.workunit.client.0.smithi038.stdout:2/958: dwrite dd/fc5 [0,4194304] 0 2024-02-15T18:35:02.748 INFO:tasks.workunit.client.0.smithi038.stdout:2/959: mknod dd/d12/d59/dce/c131 0 2024-02-15T18:35:02.749 INFO:tasks.workunit.client.0.smithi038.stdout:2/960: creat dd/d12/d1d/d3b/d3c/d4a/f132 x:0 0 0 2024-02-15T18:35:02.751 INFO:tasks.workunit.client.0.smithi038.stdout:2/961: creat dd/d12/d3a/d67/d10e/f133 x:0 0 0 2024-02-15T18:35:02.751 INFO:tasks.workunit.client.0.smithi038.stdout:2/962: dread dd/d12/d3a/d67/d4f/f64 [0,4194304] 0 2024-02-15T18:35:02.753 INFO:tasks.workunit.client.0.smithi038.stdout:2/963: creat dd/d12/d3a/d5e/d65/dcd/f134 x:0 0 0 2024-02-15T18:35:02.754 INFO:tasks.workunit.client.0.smithi038.stdout:2/964: unlink dd/d12/d3a/d67/fa0 0 2024-02-15T18:35:02.756 INFO:tasks.workunit.client.0.smithi038.stdout:2/965: mkdir dd/d12/d5d/d54/dcb/d135 0 2024-02-15T18:35:02.756 INFO:tasks.workunit.client.0.smithi038.stdout:2/966: fsync dd/d12c/f12e 0 2024-02-15T18:35:02.757 INFO:tasks.workunit.client.0.smithi038.stdout:2/967: read dd/d12/d5d/d33/fd6 [30556,113277] 0 2024-02-15T18:35:02.758 INFO:tasks.workunit.client.0.smithi038.stdout:1/992: dwrite d2/d12c/f13f [0,4194304] 0 2024-02-15T18:35:02.758 INFO:tasks.workunit.client.0.smithi038.stdout:1/993: stat d2/d10/d19/d22/d54/dc0/d6b 0 2024-02-15T18:35:02.758 INFO:tasks.workunit.client.0.smithi038.stdout:2/968: rename dd/d12/d5d/d54/dcb/de1/d104/d12f to dd/d12/d5d/d54/dcb/de1/d136 0 2024-02-15T18:35:02.758 INFO:tasks.workunit.client.0.smithi038.stdout:2/969: write dd/d12/d3a/d67/d4f/f71 [1372123,78965] 0 2024-02-15T18:35:02.760 INFO:tasks.workunit.client.0.smithi038.stdout:1/994: creat d2/d69/da9/dad/f149 x:0 0 0 2024-02-15T18:35:02.760 INFO:tasks.workunit.client.0.smithi038.stdout:1/995: dread - d2/d10/d19/fcf zero size 2024-02-15T18:35:02.762 INFO:tasks.workunit.client.0.smithi038.stdout:2/970: rename dd/d12/d59/d62/ce2 to dd/d12/d59/dce/c137 0 2024-02-15T18:35:02.765 INFO:tasks.workunit.client.0.smithi038.stdout:1/996: dread d2/d10/d19/d22/d36/d94/dc1/ff2 [0,4194304] 0 2024-02-15T18:35:02.765 INFO:tasks.workunit.client.0.smithi038.stdout:2/971: link dd/d12/d6b/da5/lbe dd/d12/d59/def/l138 0 2024-02-15T18:35:02.765 INFO:tasks.workunit.client.0.smithi038.stdout:1/997: dread - d2/d69/da9/dad/f149 zero size 2024-02-15T18:35:02.765 INFO:tasks.workunit.client.0.smithi038.stdout:1/998: getdents d2/d10/d32/d53/df0 0 2024-02-15T18:35:02.766 INFO:tasks.workunit.client.0.smithi038.stdout:1/999: readlink d2/d10/d19/d22/d36/l11e 0 2024-02-15T18:35:02.766 INFO:tasks.workunit.client.0.smithi038.stdout:2/972: write dd/fed [560632,84016] 0 2024-02-15T18:35:02.769 INFO:tasks.workunit.client.0.smithi038.stdout:2/973: creat dd/d12/d59/d62/f139 x:0 0 0 2024-02-15T18:35:02.769 INFO:tasks.workunit.client.0.smithi038.stdout:2/974: chown dd/d12/d5d/d54/dcb/fec 330 1 2024-02-15T18:35:02.770 INFO:tasks.workunit.client.0.smithi038.stdout:2/975: mknod dd/d12/d5d/d54/c13a 0 2024-02-15T18:35:02.773 INFO:tasks.workunit.client.0.smithi038.stdout:3/956: sync 2024-02-15T18:35:02.777 INFO:tasks.workunit.client.0.smithi038.stdout:2/976: dread dd/d12/d1d/d3b/fb8 [0,4194304] 0 2024-02-15T18:35:02.800 INFO:tasks.workunit.client.0.smithi038.stdout:3/957: dwrite d2/d1b/ffb [0,4194304] 0 2024-02-15T18:35:02.801 INFO:tasks.workunit.client.0.smithi038.stdout:2/977: dwrite dd/d12/d5d/d93/f127 [0,4194304] 0 2024-02-15T18:35:02.801 INFO:tasks.workunit.client.0.smithi038.stdout:3/958: getdents d2/d25/d60 0 2024-02-15T18:35:02.803 INFO:tasks.workunit.client.0.smithi038.stdout:2/978: getdents dd/d12/d3a/d67/d4f/d5a/d9d 0 2024-02-15T18:35:02.805 INFO:tasks.workunit.client.0.smithi038.stdout:3/959: rename d2/d1b/d4f/ca7 to d2/d1b/d3c/d3a/d55/d65/c131 0 2024-02-15T18:35:02.806 INFO:tasks.workunit.client.0.smithi038.stdout:3/960: write d2/d1b/d3c/d3a/d55/fb1 [2383189,121832] 0 2024-02-15T18:35:02.806 INFO:tasks.workunit.client.0.smithi038.stdout:2/979: write dd/d12/d3a/d67/d4f/d5a/f60 [2676028,16240] 0 2024-02-15T18:35:02.807 INFO:tasks.workunit.client.0.smithi038.stdout:3/961: mknod d2/d1b/d3c/d3a/d55/df8/c132 0 2024-02-15T18:35:02.807 INFO:tasks.workunit.client.0.smithi038.stdout:2/980: getdents dd/d12/d59/def 0 2024-02-15T18:35:02.808 INFO:tasks.workunit.client.0.smithi038.stdout:3/962: rmdir d2/d7f/d101 39 2024-02-15T18:35:02.808 INFO:tasks.workunit.client.0.smithi038.stdout:2/981: read dd/d12/d1d/f31 [1688996,79045] 0 2024-02-15T18:35:02.809 INFO:tasks.workunit.client.0.smithi038.stdout:2/982: mknod dd/d12/d5d/de4/c13b 0 2024-02-15T18:35:02.810 INFO:tasks.workunit.client.0.smithi038.stdout:3/963: rename d2/d7f/d101 to d2/d7f/dc1/d133 0 2024-02-15T18:35:02.810 INFO:tasks.workunit.client.0.smithi038.stdout:3/964: chown d2/d1b/d3c/d41/dd5 49839694 1 2024-02-15T18:35:02.810 INFO:tasks.workunit.client.0.smithi038.stdout:2/983: mkdir dd/d12/d59/d2b/d13c 0 2024-02-15T18:35:02.812 INFO:tasks.workunit.client.0.smithi038.stdout:3/965: rename d2/d25/d60 to d2/db8/dcf/d134 0 2024-02-15T18:35:02.813 INFO:tasks.workunit.client.0.smithi038.stdout:2/984: dread dd/d12/d1d/ff5 [0,4194304] 0 2024-02-15T18:35:02.813 INFO:tasks.workunit.client.0.smithi038.stdout:3/966: link d2/d1b/d3c/d42/le9 d2/d1b/d4f/d92/l135 0 2024-02-15T18:35:02.813 INFO:tasks.workunit.client.0.smithi038.stdout:3/967: write d2/d24/db9/f116 [669073,62230] 0 2024-02-15T18:35:02.814 INFO:tasks.workunit.client.0.smithi038.stdout:2/985: rename dd/d12/d59/d2b/l81 to dd/d12/d3a/d67/d10e/l13d 0 2024-02-15T18:35:02.815 INFO:tasks.workunit.client.0.smithi038.stdout:3/968: rmdir d2/d24/d107 0 2024-02-15T18:35:02.815 INFO:tasks.workunit.client.0.smithi038.stdout:3/969: fdatasync d2/d1b/d3c/d41/d4e/d62/f75 0 2024-02-15T18:35:02.816 INFO:tasks.workunit.client.0.smithi038.stdout:2/986: rmdir dd/d12/d59/dce/dea/dee 39 2024-02-15T18:35:02.816 INFO:tasks.workunit.client.0.smithi038.stdout:2/987: readlink dd/d12/d3a/l110 0 2024-02-15T18:35:02.816 INFO:tasks.workunit.client.0.smithi038.stdout:2/988: fdatasync dd/d12/f47 0 2024-02-15T18:35:02.817 INFO:tasks.workunit.client.0.smithi038.stdout:2/989: link dd/d12/d3a/c85 dd/d12/d1d/d3b/d3c/d11e/c13e 0 2024-02-15T18:35:02.817 INFO:tasks.workunit.client.0.smithi038.stdout:2/990: write dd/d12/d3a/d67/d4f/d5a/f10a [362334,82622] 0 2024-02-15T18:35:02.817 INFO:tasks.workunit.client.0.smithi038.stdout:2/991: dread - dd/d12/d59/d62/f139 zero size 2024-02-15T18:35:02.817 INFO:tasks.workunit.client.0.smithi038.stdout:2/992: chown dd/d12/f80 129119446 1 2024-02-15T18:35:02.818 INFO:tasks.workunit.client.0.smithi038.stdout:2/993: rename dd/d12/d59/dce/dea/dee to dd/d12/d6b/d13f 0 2024-02-15T18:35:02.819 INFO:tasks.workunit.client.0.smithi038.stdout:2/994: write dd/d12/d3a/d67/d4f/d5a/d9d/fac [2246287,65104] 0 2024-02-15T18:35:02.819 INFO:tasks.workunit.client.0.smithi038.stdout:2/995: getdents dd/d12/d6b/dde 0 2024-02-15T18:35:02.819 INFO:tasks.workunit.client.0.smithi038.stdout:2/996: read dd/d12/d1d/f50 [1847453,1248] 0 2024-02-15T18:35:02.820 INFO:tasks.workunit.client.0.smithi038.stdout:2/997: mknod dd/d12/d5d/d54/dcb/d135/c140 0 2024-02-15T18:35:02.822 INFO:tasks.workunit.client.0.smithi038.stdout:2/998: creat dd/d12/d3a/f141 x:0 0 0 2024-02-15T18:35:02.822 INFO:tasks.workunit.client.0.smithi038.stdout:2/999: readlink dd/d12/d3a/d67/d4f/d5a/la2 0 2024-02-15T18:35:02.825 INFO:tasks.workunit.client.0.smithi038.stdout:3/970: dread d2/d1b/d3c/d3a/d55/df8/f10b [0,4194304] 0 2024-02-15T18:35:02.825 INFO:tasks.workunit.client.0.smithi038.stdout:3/971: read - d2/d7f/dc1/f117 zero size 2024-02-15T18:35:02.825 INFO:tasks.workunit.client.0.smithi038.stdout:3/972: chown d2/d1b/d4f/d92/l135 28651910 1 2024-02-15T18:35:02.831 INFO:tasks.workunit.client.0.smithi038.stdout:3/973: dread d2/d7f/dc1/d133/dfa/f100 [0,4194304] 0 2024-02-15T18:35:02.832 INFO:tasks.workunit.client.0.smithi038.stdout:3/974: dread d2/d1b/d3c/fd4 [0,4194304] 0 2024-02-15T18:35:02.834 INFO:tasks.workunit.client.0.smithi038.stdout:3/975: dread d2/d1b/d3c/d3a/fe2 [0,4194304] 0 2024-02-15T18:35:02.836 INFO:tasks.workunit.client.0.smithi038.stdout:3/976: mkdir d2/d7f/dc1/d133/d128/d136 0 2024-02-15T18:35:02.836 INFO:tasks.workunit.client.0.smithi038.stdout:3/977: chown d2/d1b/d4f/d92/lb5 25669689 1 2024-02-15T18:35:02.841 INFO:tasks.workunit.client.0.smithi038.stdout:3/978: dread d2/d1b/d3c/d3a/d55/f80 [0,4194304] 0 2024-02-15T18:35:02.858 INFO:tasks.workunit.client.0.smithi038.stdout:3/979: dwrite d2/d1b/d3c/d3a/d55/d65/fd0 [4194304,4194304] 0 2024-02-15T18:35:02.859 INFO:tasks.workunit.client.0.smithi038.stdout:3/980: getdents d2/db8/dcf 0 2024-02-15T18:35:02.874 INFO:tasks.workunit.client.0.smithi038.stdout:3/981: dwrite d2/d1b/d8c/ffc [0,4194304] 0 2024-02-15T18:35:02.889 INFO:tasks.workunit.client.0.smithi038.stdout:3/982: dwrite d2/d1b/d3c/d41/d4e/fa3 [0,4194304] 0 2024-02-15T18:35:02.902 INFO:tasks.workunit.client.0.smithi038.stdout:3/983: dwrite d2/d1b/d3c/f6b [4194304,4194304] 0 2024-02-15T18:35:02.919 INFO:tasks.workunit.client.0.smithi038.stdout:3/984: dwrite d2/d1e/f44 [0,4194304] 0 2024-02-15T18:35:02.920 INFO:tasks.workunit.client.0.smithi038.stdout:3/985: getdents d2/d25 0 2024-02-15T18:35:02.921 INFO:tasks.workunit.client.0.smithi038.stdout:3/986: stat d2/d1b/led 0 2024-02-15T18:35:02.921 INFO:tasks.workunit.client.0.smithi038.stdout:3/987: mknod d2/d25/c137 0 2024-02-15T18:35:02.922 INFO:tasks.workunit.client.0.smithi038.stdout:3/988: dread - d2/d1b/d3c/d41/d4e/d62/f75 zero size 2024-02-15T18:35:02.923 INFO:tasks.workunit.client.0.smithi038.stdout:3/989: rmdir d2/d1b/d4f 39 2024-02-15T18:35:02.923 INFO:tasks.workunit.client.0.smithi038.stdout:3/990: getdents d2/d1e/d7e/d7c/d12c 0 2024-02-15T18:35:02.924 INFO:tasks.workunit.client.0.smithi038.stdout:3/991: symlink d2/d1b/d3c/d41/d4e/d62/l138 0 2024-02-15T18:35:02.924 INFO:tasks.workunit.client.0.smithi038.stdout:3/992: fdatasync d2/d1b/d3c/d3a/d55/da0/fe4 0 2024-02-15T18:35:02.925 INFO:tasks.workunit.client.0.smithi038.stdout:3/993: creat d2/d1b/d3c/d3a/dbb/f139 x:0 0 0 2024-02-15T18:35:02.926 INFO:tasks.workunit.client.0.smithi038.stdout:3/994: creat d2/d1b/d69/d10c/f13a x:0 0 0 2024-02-15T18:35:02.927 INFO:tasks.workunit.client.0.smithi038.stdout:3/995: mkdir d2/d1b/d4f/d13b 0 2024-02-15T18:35:02.927 INFO:tasks.workunit.client.0.smithi038.stdout:3/996: write d2/d1b/d3c/d3a/d55/f119 [580382,47725] 0 2024-02-15T18:35:02.943 INFO:tasks.workunit.client.0.smithi038.stdout:3/997: dwrite d2/d1b/d3c/d3a/dbb/f139 [0,4194304] 0 2024-02-15T18:35:02.944 INFO:tasks.workunit.client.0.smithi038.stdout:3/998: write d2/d1b/d69/dd3/f105 [696749,13218] 0 2024-02-15T18:35:02.960 INFO:tasks.workunit.client.0.smithi038.stdout:3/999: dwrite d2/d1b/d3c/d3a/d55/d65/f112 [0,4194304] 0 2024-02-15T18:35:02.963 INFO:tasks.workunit.client.0.smithi038.stderr:+ rm -rf -- ./tmp.aNO9Ecf3kJ 2024-02-15T18:35:03.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:02.319153+0000 mgr.smithi038.khesbl (mgr.14162) 341 : cluster [DBG] pgmap v283: 65 pgs: 65 active+clean; 1.8 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 140 MiB/s rd, 257 MiB/s wr, 581 op/s 2024-02-15T18:35:03.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:02.319153+0000 mgr.smithi038.khesbl (mgr.14162) 341 : cluster [DBG] pgmap v283: 65 pgs: 65 active+clean; 1.8 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 140 MiB/s rd, 257 MiB/s wr, 581 op/s 2024-02-15T18:35:05.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:04.319621+0000 mgr.smithi038.khesbl (mgr.14162) 342 : cluster [DBG] pgmap v284: 65 pgs: 65 active+clean; 2.1 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 162 MiB/s rd, 308 MiB/s wr, 464 op/s 2024-02-15T18:35:05.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:04.319621+0000 mgr.smithi038.khesbl (mgr.14162) 342 : cluster [DBG] pgmap v284: 65 pgs: 65 active+clean; 2.1 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 162 MiB/s rd, 308 MiB/s wr, 464 op/s 2024-02-15T18:35:07.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:06.320309+0000 mgr.smithi038.khesbl (mgr.14162) 343 : cluster [DBG] pgmap v285: 65 pgs: 65 active+clean; 2.0 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 159 MiB/s rd, 310 MiB/s wr, 506 op/s 2024-02-15T18:35:07.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:06.320309+0000 mgr.smithi038.khesbl (mgr.14162) 343 : cluster [DBG] pgmap v285: 65 pgs: 65 active+clean; 2.0 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 159 MiB/s rd, 310 MiB/s wr, 506 op/s 2024-02-15T18:35:08.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:07.494900+0000 mon.smithi038 (mon.0) 572 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:active} 2 up:standby 2024-02-15T18:35:08.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:07.494900+0000 mon.smithi038 (mon.0) 572 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:active} 2 up:standby 2024-02-15T18:35:09.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:08.320967+0000 mgr.smithi038.khesbl (mgr.14162) 344 : cluster [DBG] pgmap v286: 65 pgs: 65 active+clean; 1.8 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 118 MiB/s rd, 231 MiB/s wr, 398 op/s 2024-02-15T18:35:09.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:08.320967+0000 mgr.smithi038.khesbl (mgr.14162) 344 : cluster [DBG] pgmap v286: 65 pgs: 65 active+clean; 1.8 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 118 MiB/s rd, 231 MiB/s wr, 398 op/s 2024-02-15T18:35:11.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:10.321359+0000 mgr.smithi038.khesbl (mgr.14162) 345 : cluster [DBG] pgmap v287: 65 pgs: 65 active+clean; 1.5 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 98 MiB/s rd, 192 MiB/s wr, 347 op/s 2024-02-15T18:35:11.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:10.321359+0000 mgr.smithi038.khesbl (mgr.14162) 345 : cluster [DBG] pgmap v287: 65 pgs: 65 active+clean; 1.5 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 98 MiB/s rd, 192 MiB/s wr, 347 op/s 2024-02-15T18:35:13.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:12.321832+0000 mgr.smithi038.khesbl (mgr.14162) 346 : cluster [DBG] pgmap v288: 65 pgs: 65 active+clean; 1.3 GiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 71 MiB/s rd, 152 MiB/s wr, 313 op/s 2024-02-15T18:35:13.812 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-15T18:35:13.812 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-15T18:35:13.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:12.321832+0000 mgr.smithi038.khesbl (mgr.14162) 346 : cluster [DBG] pgmap v288: 65 pgs: 65 active+clean; 1.3 GiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 71 MiB/s rd, 152 MiB/s wr, 313 op/s 2024-02-15T18:35:15.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.563+0000 7f64158ea700 1 -- 172.21.15.38:0/154433867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64100f66d0 msgr2=0x7f64100f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:15.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.563+0000 7f64158ea700 1 --2- 172.21.15.38:0/154433867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64100f66d0 0x7f64100f6b30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f63fc004660 tx=0x7f63fc009b30).stop 2024-02-15T18:35:15.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.563+0000 7f64158ea700 1 -- 172.21.15.38:0/154433867 shutdown_connections 2024-02-15T18:35:15.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.563+0000 7f64158ea700 1 --2- 172.21.15.38:0/154433867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64100f66d0 0x7f64100f6b30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:15.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.563+0000 7f64158ea700 1 --2- 172.21.15.38:0/154433867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64100f5430 0x7f64100f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:15.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.563+0000 7f64158ea700 1 -- 172.21.15.38:0/154433867 >> 172.21.15.38:0/154433867 conn(0x7f64100f0b90 msgr2=0x7f64100f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:15.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.563+0000 7f64158ea700 1 -- 172.21.15.38:0/154433867 shutdown_connections 2024-02-15T18:35:15.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.563+0000 7f64158ea700 1 -- 172.21.15.38:0/154433867 wait complete. 2024-02-15T18:35:15.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.564+0000 7f64158ea700 1 Processor -- start 2024-02-15T18:35:15.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.564+0000 7f64158ea700 1 -- start start 2024-02-15T18:35:15.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.564+0000 7f64158ea700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64100f5430 0x7f6410109ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:15.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.564+0000 7f64158ea700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64100f66d0 0x7f641010bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:15.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.564+0000 7f64158ea700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64100603c0 con 0x7f64100f66d0 2024-02-15T18:35:15.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.564+0000 7f64158ea700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6410060500 con 0x7f64100f5430 2024-02-15T18:35:15.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.564+0000 7f640ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64100f66d0 0x7f641010bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:15.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.564+0000 7f64148e8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64100f5430 0x7f6410109ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:15.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.564+0000 7f640ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64100f66d0 0x7f641010bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37426/0 (socket says 172.21.15.38:37426) 2024-02-15T18:35:15.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.564+0000 7f640ffff700 1 -- 172.21.15.38:0/243208930 learned_addr learned my addr 172.21.15.38:0/243208930 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:35:15.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.565+0000 7f640ffff700 1 -- 172.21.15.38:0/243208930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64100f5430 msgr2=0x7f6410109ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:15.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.565+0000 7f640ffff700 1 --2- 172.21.15.38:0/243208930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64100f5430 0x7f6410109ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:15.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.565+0000 7f640ffff700 1 -- 172.21.15.38:0/243208930 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63fc005040 con 0x7f64100f66d0 2024-02-15T18:35:15.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.565+0000 7f640ffff700 1 --2- 172.21.15.38:0/243208930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64100f66d0 0x7f641010bfd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f63fc000c00 tx=0x7f63fc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:15.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.565+0000 7f640d7fa700 1 -- 172.21.15.38:0/243208930 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63fc019070 con 0x7f64100f66d0 2024-02-15T18:35:15.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.565+0000 7f640d7fa700 1 -- 172.21.15.38:0/243208930 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f63fc007e80 con 0x7f64100f66d0 2024-02-15T18:35:15.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.565+0000 7f64158ea700 1 -- 172.21.15.38:0/243208930 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f641010c520 con 0x7f64100f66d0 2024-02-15T18:35:15.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.565+0000 7f640d7fa700 1 -- 172.21.15.38:0/243208930 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63fc014db0 con 0x7f64100f66d0 2024-02-15T18:35:15.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.565+0000 7f64158ea700 1 -- 172.21.15.38:0/243208930 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f641010ca50 con 0x7f64100f66d0 2024-02-15T18:35:15.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.566+0000 7f64158ea700 1 -- 172.21.15.38:0/243208930 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6410059100 con 0x7f64100f66d0 2024-02-15T18:35:15.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.566+0000 7f640d7fa700 1 -- 172.21.15.38:0/243208930 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 74675+0+0 (secure 0 0 0) 0x7f63fc025070 con 0x7f64100f66d0 2024-02-15T18:35:15.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.567+0000 7f640d7fa700 1 --2- 172.21.15.38:0/243208930 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f640005e8e0 0x7f6400060da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:15.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.567+0000 7f640d7fa700 1 -- 172.21.15.38:0/243208930 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f63fc079e40 con 0x7f64100f66d0 2024-02-15T18:35:15.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.567+0000 7f64148e8700 1 --2- 172.21.15.38:0/243208930 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f640005e8e0 0x7f6400060da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:15.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.567+0000 7f64148e8700 1 --2- 172.21.15.38:0/243208930 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f640005e8e0 0x7f6400060da0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f6410060040 tx=0x7f640400b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:15.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.571+0000 7f640d7fa700 1 -- 172.21.15.38:0/243208930 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f63fc04cab0 con 0x7f64100f66d0 2024-02-15T18:35:15.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.775+0000 7f64158ea700 1 -- 172.21.15.38:0/243208930 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f64100fde80 con 0x7f640005e8e0 2024-02-15T18:35:15.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.776+0000 7f640d7fa700 1 -- 172.21.15.38:0/243208930 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+248 (secure 0 0 0) 0x7f64100fde80 con 0x7f640005e8e0 2024-02-15T18:35:15.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.777+0000 7f64158ea700 1 -- 172.21.15.38:0/243208930 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f640005e8e0 msgr2=0x7f6400060da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:15.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.777+0000 7f64158ea700 1 --2- 172.21.15.38:0/243208930 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f640005e8e0 0x7f6400060da0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f6410060040 tx=0x7f640400b040).stop 2024-02-15T18:35:15.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.777+0000 7f64158ea700 1 -- 172.21.15.38:0/243208930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64100f66d0 msgr2=0x7f641010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:15.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.777+0000 7f64158ea700 1 --2- 172.21.15.38:0/243208930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64100f66d0 0x7f641010bfd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f63fc000c00 tx=0x7f63fc00b040).stop 2024-02-15T18:35:15.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.778+0000 7f64158ea700 1 -- 172.21.15.38:0/243208930 shutdown_connections 2024-02-15T18:35:15.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.778+0000 7f64158ea700 1 --2- 172.21.15.38:0/243208930 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f640005e8e0 0x7f6400060da0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:15.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.778+0000 7f64158ea700 1 --2- 172.21.15.38:0/243208930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64100f5430 0x7f6410109ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:15.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.778+0000 7f64158ea700 1 --2- 172.21.15.38:0/243208930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64100f66d0 0x7f641010bfd0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:15.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.778+0000 7f64158ea700 1 -- 172.21.15.38:0/243208930 >> 172.21.15.38:0/243208930 conn(0x7f64100f0b90 msgr2=0x7f64100fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:15.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.778+0000 7f64158ea700 1 -- 172.21.15.38:0/243208930 shutdown_connections 2024-02-15T18:35:15.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.778+0000 7f64158ea700 1 -- 172.21.15.38:0/243208930 wait complete. 2024-02-15T18:35:15.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:14.322587+0000 mgr.smithi038.khesbl (mgr.14162) 347 : cluster [DBG] pgmap v289: 65 pgs: 65 active+clean; 1010 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 32 MiB/s rd, 84 MiB/s wr, 218 op/s 2024-02-15T18:35:15.778 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-02-15T18:35:15.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:14.322587+0000 mgr.smithi038.khesbl (mgr.14162) 347 : cluster [DBG] pgmap v289: 65 pgs: 65 active+clean; 1010 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 32 MiB/s rd, 84 MiB/s wr, 218 op/s 2024-02-15T18:35:15.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.880+0000 7f1a36900700 1 -- 172.21.15.38:0/1312360860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a300f55c0 msgr2=0x7f1a300f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:15.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.880+0000 7f1a36900700 1 --2- 172.21.15.38:0/1312360860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a300f55c0 0x7f1a300f59e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f1a20004660 tx=0x7f1a20009b30).stop 2024-02-15T18:35:15.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.881+0000 7f1a36900700 1 -- 172.21.15.38:0/1312360860 shutdown_connections 2024-02-15T18:35:15.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.881+0000 7f1a36900700 1 --2- 172.21.15.38:0/1312360860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a300f6860 0x7f1a300f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:15.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.881+0000 7f1a36900700 1 --2- 172.21.15.38:0/1312360860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a300f55c0 0x7f1a300f59e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:15.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.881+0000 7f1a36900700 1 -- 172.21.15.38:0/1312360860 >> 172.21.15.38:0/1312360860 conn(0x7f1a300f0d40 msgr2=0x7f1a300f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:15.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.881+0000 7f1a36900700 1 -- 172.21.15.38:0/1312360860 shutdown_connections 2024-02-15T18:35:15.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.881+0000 7f1a36900700 1 -- 172.21.15.38:0/1312360860 wait complete. 2024-02-15T18:35:15.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.881+0000 7f1a36900700 1 Processor -- start 2024-02-15T18:35:15.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.881+0000 7f1a36900700 1 -- start start 2024-02-15T18:35:15.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.881+0000 7f1a36900700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a300f55c0 0x7f1a301091f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:15.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.882+0000 7f1a36900700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a300f6860 0x7f1a301096e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:15.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.882+0000 7f1a36900700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a30109c90 con 0x7f1a300f6860 2024-02-15T18:35:15.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.882+0000 7f1a36900700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a30109dd0 con 0x7f1a300f55c0 2024-02-15T18:35:15.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.882+0000 7f1a2f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a300f6860 0x7f1a301096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:15.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.882+0000 7f1a2f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a300f6860 0x7f1a301096e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37452/0 (socket says 172.21.15.38:37452) 2024-02-15T18:35:15.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.882+0000 7f1a2f7fe700 1 -- 172.21.15.38:0/1933589286 learned_addr learned my addr 172.21.15.38:0/1933589286 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:35:15.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.882+0000 7f1a2ffff700 1 --2- 172.21.15.38:0/1933589286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a300f55c0 0x7f1a301091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:15.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.882+0000 7f1a2f7fe700 1 -- 172.21.15.38:0/1933589286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a300f55c0 msgr2=0x7f1a301091f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:15.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.882+0000 7f1a2f7fe700 1 --2- 172.21.15.38:0/1933589286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a300f55c0 0x7f1a301091f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:15.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.882+0000 7f1a2f7fe700 1 -- 172.21.15.38:0/1933589286 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a20005040 con 0x7f1a300f6860 2024-02-15T18:35:15.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.882+0000 7f1a2f7fe700 1 --2- 172.21.15.38:0/1933589286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a300f6860 0x7f1a301096e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f1a1800e7f0 tx=0x7f1a1800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:15.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.882+0000 7f1a2cff9700 1 -- 172.21.15.38:0/1933589286 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a180092e0 con 0x7f1a300f6860 2024-02-15T18:35:15.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.882+0000 7f1a2cff9700 1 -- 172.21.15.38:0/1933589286 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f1a1800f040 con 0x7f1a300f6860 2024-02-15T18:35:15.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.882+0000 7f1a36900700 1 -- 172.21.15.38:0/1933589286 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3010a770 con 0x7f1a300f6860 2024-02-15T18:35:15.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.882+0000 7f1a2cff9700 1 -- 172.21.15.38:0/1933589286 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a18014850 con 0x7f1a300f6860 2024-02-15T18:35:15.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.882+0000 7f1a36900700 1 -- 172.21.15.38:0/1933589286 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3010ad30 con 0x7f1a300f6860 2024-02-15T18:35:15.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.883+0000 7f1a36900700 1 -- 172.21.15.38:0/1933589286 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a30045ad0 con 0x7f1a300f6860 2024-02-15T18:35:15.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.884+0000 7f1a2cff9700 1 -- 172.21.15.38:0/1933589286 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 74675+0+0 (secure 0 0 0) 0x7f1a1801b030 con 0x7f1a300f6860 2024-02-15T18:35:15.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.884+0000 7f1a2cff9700 1 --2- 172.21.15.38:0/1933589286 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1a1c05e8f0 0x7f1a1c060db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:15.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.884+0000 7f1a2cff9700 1 -- 172.21.15.38:0/1933589286 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f1a18078bf0 con 0x7f1a300f6860 2024-02-15T18:35:15.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.884+0000 7f1a2ffff700 1 --2- 172.21.15.38:0/1933589286 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1a1c05e8f0 0x7f1a1c060db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:15.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.884+0000 7f1a2ffff700 1 --2- 172.21.15.38:0/1933589286 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1a1c05e8f0 0x7f1a1c060db0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f1a20009720 tx=0x7f1a20016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:15.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:15.888+0000 7f1a2cff9700 1 -- 172.21.15.38:0/1933589286 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f1a180482b0 con 0x7f1a300f6860 2024-02-15T18:35:16.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.112+0000 7f1a36900700 1 -- 172.21.15.38:0/1933589286 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a300faaa0 con 0x7f1a1c05e8f0 2024-02-15T18:35:16.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.114+0000 7f1a2cff9700 1 -- 172.21.15.38:0/1933589286 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+248 (secure 0 0 0) 0x7f1a300faaa0 con 0x7f1a1c05e8f0 2024-02-15T18:35:16.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.117+0000 7f1a36900700 1 -- 172.21.15.38:0/1933589286 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1a1c05e8f0 msgr2=0x7f1a1c060db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:16.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.117+0000 7f1a36900700 1 --2- 172.21.15.38:0/1933589286 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1a1c05e8f0 0x7f1a1c060db0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f1a20009720 tx=0x7f1a20016040).stop 2024-02-15T18:35:16.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.117+0000 7f1a36900700 1 -- 172.21.15.38:0/1933589286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a300f6860 msgr2=0x7f1a301096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:16.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.117+0000 7f1a36900700 1 --2- 172.21.15.38:0/1933589286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a300f6860 0x7f1a301096e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f1a1800e7f0 tx=0x7f1a1800b040).stop 2024-02-15T18:35:16.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.119+0000 7f1a36900700 1 -- 172.21.15.38:0/1933589286 shutdown_connections 2024-02-15T18:35:16.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.119+0000 7f1a36900700 1 --2- 172.21.15.38:0/1933589286 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f1a1c05e8f0 0x7f1a1c060db0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:16.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.119+0000 7f1a36900700 1 --2- 172.21.15.38:0/1933589286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a300f55c0 0x7f1a301091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:16.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.119+0000 7f1a36900700 1 --2- 172.21.15.38:0/1933589286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a300f6860 0x7f1a301096e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:16.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.119+0000 7f1a36900700 1 -- 172.21.15.38:0/1933589286 >> 172.21.15.38:0/1933589286 conn(0x7f1a300f0d40 msgr2=0x7f1a3005fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:16.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.119+0000 7f1a36900700 1 -- 172.21.15.38:0/1933589286 shutdown_connections 2024-02-15T18:35:16.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.119+0000 7f1a36900700 1 -- 172.21.15.38:0/1933589286 wait complete. 2024-02-15T18:35:16.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.246+0000 7f9a48fd1700 1 -- 172.21.15.38:0/1322663950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a440f66e0 msgr2=0x7f9a440f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:16.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.246+0000 7f9a48fd1700 1 --2- 172.21.15.38:0/1322663950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a440f66e0 0x7f9a440f6b00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f9a34004660 tx=0x7f9a34009b30).stop 2024-02-15T18:35:16.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.246+0000 7f9a48fd1700 1 -- 172.21.15.38:0/1322663950 shutdown_connections 2024-02-15T18:35:16.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.246+0000 7f9a48fd1700 1 --2- 172.21.15.38:0/1322663950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a440f7980 0x7f9a440f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:16.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.246+0000 7f9a48fd1700 1 --2- 172.21.15.38:0/1322663950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a440f66e0 0x7f9a440f6b00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:16.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.246+0000 7f9a48fd1700 1 -- 172.21.15.38:0/1322663950 >> 172.21.15.38:0/1322663950 conn(0x7f9a440f1e40 msgr2=0x7f9a440f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:16.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.246+0000 7f9a48fd1700 1 -- 172.21.15.38:0/1322663950 shutdown_connections 2024-02-15T18:35:16.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.246+0000 7f9a48fd1700 1 -- 172.21.15.38:0/1322663950 wait complete. 2024-02-15T18:35:16.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.247+0000 7f9a48fd1700 1 Processor -- start 2024-02-15T18:35:16.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.247+0000 7f9a48fd1700 1 -- start start 2024-02-15T18:35:16.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.247+0000 7f9a48fd1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a440f66e0 0x7f9a44107030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:16.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.247+0000 7f9a48fd1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a440f7980 0x7f9a44107520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:16.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.247+0000 7f9a48fd1700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a44107ad0 con 0x7f9a440f7980 2024-02-15T18:35:16.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.247+0000 7f9a48fd1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a44107c10 con 0x7f9a440f66e0 2024-02-15T18:35:16.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.247+0000 7f9a41d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a440f7980 0x7f9a44107520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:16.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.247+0000 7f9a41d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a440f7980 0x7f9a44107520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37462/0 (socket says 172.21.15.38:37462) 2024-02-15T18:35:16.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.247+0000 7f9a41d9b700 1 -- 172.21.15.38:0/2732965294 learned_addr learned my addr 172.21.15.38:0/2732965294 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:35:16.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.247+0000 7f9a4259c700 1 --2- 172.21.15.38:0/2732965294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a440f66e0 0x7f9a44107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:16.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.247+0000 7f9a41d9b700 1 -- 172.21.15.38:0/2732965294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a440f66e0 msgr2=0x7f9a44107030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:16.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.247+0000 7f9a41d9b700 1 --2- 172.21.15.38:0/2732965294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a440f66e0 0x7f9a44107030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:16.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.247+0000 7f9a41d9b700 1 -- 172.21.15.38:0/2732965294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a34005040 con 0x7f9a440f7980 2024-02-15T18:35:16.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.247+0000 7f9a41d9b700 1 --2- 172.21.15.38:0/2732965294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a440f7980 0x7f9a44107520 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f9a2c00e7f0 tx=0x7f9a2c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:16.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.247+0000 7f9a3affd700 1 -- 172.21.15.38:0/2732965294 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a2c0092e0 con 0x7f9a440f7980 2024-02-15T18:35:16.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.248+0000 7f9a3affd700 1 -- 172.21.15.38:0/2732965294 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f9a2c00f040 con 0x7f9a440f7980 2024-02-15T18:35:16.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.248+0000 7f9a48fd1700 1 -- 172.21.15.38:0/2732965294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a441085b0 con 0x7f9a440f7980 2024-02-15T18:35:16.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.248+0000 7f9a3affd700 1 -- 172.21.15.38:0/2732965294 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a2c014850 con 0x7f9a440f7980 2024-02-15T18:35:16.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.248+0000 7f9a48fd1700 1 -- 172.21.15.38:0/2732965294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a44108b70 con 0x7f9a440f7980 2024-02-15T18:35:16.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.249+0000 7f9a48fd1700 1 -- 172.21.15.38:0/2732965294 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a44045ad0 con 0x7f9a440f7980 2024-02-15T18:35:16.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.249+0000 7f9a3affd700 1 -- 172.21.15.38:0/2732965294 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 74675+0+0 (secure 0 0 0) 0x7f9a2c01b030 con 0x7f9a440f7980 2024-02-15T18:35:16.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.249+0000 7f9a3affd700 1 --2- 172.21.15.38:0/2732965294 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9a3005e8f0 0x7f9a30060db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:16.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.249+0000 7f9a3affd700 1 -- 172.21.15.38:0/2732965294 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f9a2c078bf0 con 0x7f9a440f7980 2024-02-15T18:35:16.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.249+0000 7f9a4259c700 1 --2- 172.21.15.38:0/2732965294 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9a3005e8f0 0x7f9a30060db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:16.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.250+0000 7f9a4259c700 1 --2- 172.21.15.38:0/2732965294 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9a3005e8f0 0x7f9a30060db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f9a34009720 tx=0x7f9a34016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:16.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.253+0000 7f9a3affd700 1 -- 172.21.15.38:0/2732965294 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9a2c04b900 con 0x7f9a440f7980 2024-02-15T18:35:16.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.452+0000 7f9a48fd1700 1 -- 172.21.15.38:0/2732965294 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a44108de0 con 0x7f9a3005e8f0 2024-02-15T18:35:16.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.460+0000 7f9a3affd700 1 -- 172.21.15.38:0/2732965294 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2706 (secure 0 0 0) 0x7f9a44108de0 con 0x7f9a3005e8f0 2024-02-15T18:35:16.451 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE VERSION IMAGE ID CONTAINER ID 2024-02-15T18:35:16.451 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (6m) 117s ago 8m 0.20.0 0881eb8f169f c2034c318e7c 2024-02-15T18:35:16.451 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (8m) 117s ago 8m 16.2.4 8d91d370c2b8 0fd3f945d009 2024-02-15T18:35:16.451 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi121 smithi121 running (6m) 2m ago 6m 16.2.4 8d91d370c2b8 affc02db354e 2024-02-15T18:35:16.451 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (5m) 117s ago 7m 6.7.4 557c83e11646 2dcccfbfbd2a 2024-02-15T18:35:16.451 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.hxoggl smithi038 running (2m) 117s ago 2m 16.2.4 8d91d370c2b8 be2533754b9c 2024-02-15T18:35:16.451 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.qgkzgn smithi038 running (2m) 117s ago 2m 16.2.4 8d91d370c2b8 dd3ffafba9bd 2024-02-15T18:35:16.451 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.mvumlh smithi121 running (2m) 2m ago 2m 16.2.4 8d91d370c2b8 88b7d876a39b 2024-02-15T18:35:16.451 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.ovuzki smithi121 running (2m) 2m ago 2m 16.2.4 8d91d370c2b8 21cacf0012f9 2024-02-15T18:35:16.451 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.khesbl smithi038 *:9283 running (9m) 117s ago 9m 16.2.4 8d91d370c2b8 b0d099c255f4 2024-02-15T18:35:16.451 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi121.ckiryi smithi121 *:8443,9283 starting - - 2024-02-15T18:35:16.451 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (9m) 117s ago 9m 16.2.4 8d91d370c2b8 01b30eaebcfe 2024-02-15T18:35:16.451 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi121 smithi121 running (6m) 2m ago 6m 16.2.4 8d91d370c2b8 8f588ecacb71 2024-02-15T18:35:16.451 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (7m) 117s ago 7m 0.18.1 e5a616e4b9cf 037f60e38158 2024-02-15T18:35:16.451 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi121 smithi121 *:9100 running (6m) 2m ago 6m 0.18.1 e5a616e4b9cf a867bff85e18 2024-02-15T18:35:16.452 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (6m) 117s ago 6m 16.2.4 8d91d370c2b8 51512003a2ef 2024-02-15T18:35:16.452 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (5m) 117s ago 5m 16.2.4 8d91d370c2b8 16dbdf20b170 2024-02-15T18:35:16.452 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (5m) 117s ago 5m 16.2.4 8d91d370c2b8 6d6ee813e5be 2024-02-15T18:35:16.452 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi121 running (4m) 2m ago 4m 16.2.4 8d91d370c2b8 cba2d9ed4730 2024-02-15T18:35:16.452 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi121 running (4m) 2m ago 4m 16.2.4 8d91d370c2b8 ac17ad7dc628 2024-02-15T18:35:16.452 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi121 running (4m) 2m ago 4m 16.2.4 8d91d370c2b8 cdc8c669c4db 2024-02-15T18:35:16.452 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (5m) 117s ago 7m 2.18.1 de242295e225 574a5d75991e 2024-02-15T18:35:16.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.461+0000 7f9a48fd1700 1 -- 172.21.15.38:0/2732965294 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9a3005e8f0 msgr2=0x7f9a30060db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:16.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.461+0000 7f9a48fd1700 1 --2- 172.21.15.38:0/2732965294 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9a3005e8f0 0x7f9a30060db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f9a34009720 tx=0x7f9a34016040).stop 2024-02-15T18:35:16.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.461+0000 7f9a48fd1700 1 -- 172.21.15.38:0/2732965294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a440f7980 msgr2=0x7f9a44107520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:16.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.461+0000 7f9a48fd1700 1 --2- 172.21.15.38:0/2732965294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a440f7980 0x7f9a44107520 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f9a2c00e7f0 tx=0x7f9a2c00b040).stop 2024-02-15T18:35:16.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.461+0000 7f9a48fd1700 1 -- 172.21.15.38:0/2732965294 shutdown_connections 2024-02-15T18:35:16.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.461+0000 7f9a48fd1700 1 --2- 172.21.15.38:0/2732965294 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f9a3005e8f0 0x7f9a30060db0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:16.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.461+0000 7f9a48fd1700 1 --2- 172.21.15.38:0/2732965294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a440f66e0 0x7f9a44107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:16.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.461+0000 7f9a48fd1700 1 --2- 172.21.15.38:0/2732965294 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9a440f7980 0x7f9a44107520 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:16.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.461+0000 7f9a48fd1700 1 -- 172.21.15.38:0/2732965294 >> 172.21.15.38:0/2732965294 conn(0x7f9a440f1e40 msgr2=0x7f9a440fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:16.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.462+0000 7f9a48fd1700 1 -- 172.21.15.38:0/2732965294 shutdown_connections 2024-02-15T18:35:16.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.462+0000 7f9a48fd1700 1 -- 172.21.15.38:0/2732965294 wait complete. 2024-02-15T18:35:16.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.564+0000 7f7d4c44a700 1 -- 172.21.15.38:0/1817747884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d440f6860 msgr2=0x7f7d440f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.564+0000 7f7d4c44a700 1 --2- 172.21.15.38:0/1817747884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d440f6860 0x7f7d440f6cc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f7d34004660 tx=0x7f7d34009b30).stop 2024-02-15T18:35:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.564+0000 7f7d4c44a700 1 -- 172.21.15.38:0/1817747884 shutdown_connections 2024-02-15T18:35:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.564+0000 7f7d4c44a700 1 --2- 172.21.15.38:0/1817747884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d440f6860 0x7f7d440f6cc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.564+0000 7f7d4c44a700 1 --2- 172.21.15.38:0/1817747884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d440f55c0 0x7f7d440f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.564+0000 7f7d4c44a700 1 -- 172.21.15.38:0/1817747884 >> 172.21.15.38:0/1817747884 conn(0x7f7d440f0d40 msgr2=0x7f7d440f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.564+0000 7f7d4c44a700 1 -- 172.21.15.38:0/1817747884 shutdown_connections 2024-02-15T18:35:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.564+0000 7f7d4c44a700 1 -- 172.21.15.38:0/1817747884 wait complete. 2024-02-15T18:35:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.565+0000 7f7d4c44a700 1 Processor -- start 2024-02-15T18:35:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.565+0000 7f7d4c44a700 1 -- start start 2024-02-15T18:35:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.565+0000 7f7d4c44a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d440f55c0 0x7f7d44109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.565+0000 7f7d4c44a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d440f6860 0x7f7d4410c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.565+0000 7f7d4c44a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d440603c0 con 0x7f7d440f55c0 2024-02-15T18:35:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.565+0000 7f7d4c44a700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d44060500 con 0x7f7d440f6860 2024-02-15T18:35:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.565+0000 7f7d4a1e6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d440f55c0 0x7f7d44109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.565+0000 7f7d4a1e6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d440f55c0 0x7f7d44109cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37478/0 (socket says 172.21.15.38:37478) 2024-02-15T18:35:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.565+0000 7f7d499e5700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d440f6860 0x7f7d4410c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.565+0000 7f7d4a1e6700 1 -- 172.21.15.38:0/2454195564 learned_addr learned my addr 172.21.15.38:0/2454195564 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:35:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.566+0000 7f7d4a1e6700 1 -- 172.21.15.38:0/2454195564 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d440f6860 msgr2=0x7f7d4410c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.566+0000 7f7d4a1e6700 1 --2- 172.21.15.38:0/2454195564 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d440f6860 0x7f7d4410c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.566+0000 7f7d4a1e6700 1 -- 172.21.15.38:0/2454195564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d34005040 con 0x7f7d440f55c0 2024-02-15T18:35:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.566+0000 7f7d4a1e6700 1 --2- 172.21.15.38:0/2454195564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d440f55c0 0x7f7d44109cd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f7d4000e7c0 tx=0x7f7d4000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.566+0000 7f7d3affd700 1 -- 172.21.15.38:0/2454195564 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d400092e0 con 0x7f7d440f55c0 2024-02-15T18:35:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.566+0000 7f7d3affd700 1 -- 172.21.15.38:0/2454195564 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f7d4000f040 con 0x7f7d440f55c0 2024-02-15T18:35:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.566+0000 7f7d4c44a700 1 -- 172.21.15.38:0/2454195564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4410c780 con 0x7f7d440f55c0 2024-02-15T18:35:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.566+0000 7f7d3affd700 1 -- 172.21.15.38:0/2454195564 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d40014920 con 0x7f7d440f55c0 2024-02-15T18:35:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.566+0000 7f7d4c44a700 1 -- 172.21.15.38:0/2454195564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4410cd10 con 0x7f7d440f55c0 2024-02-15T18:35:16.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.567+0000 7f7d4c44a700 1 -- 172.21.15.38:0/2454195564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d44045ad0 con 0x7f7d440f55c0 2024-02-15T18:35:16.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.568+0000 7f7d3affd700 1 -- 172.21.15.38:0/2454195564 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 74675+0+0 (secure 0 0 0) 0x7f7d4001b030 con 0x7f7d440f55c0 2024-02-15T18:35:16.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.568+0000 7f7d3affd700 1 --2- 172.21.15.38:0/2454195564 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f7d3005e8f0 0x7f7d30060db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:16.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.568+0000 7f7d3affd700 1 -- 172.21.15.38:0/2454195564 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f7d40079340 con 0x7f7d440f55c0 2024-02-15T18:35:16.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.568+0000 7f7d499e5700 1 --2- 172.21.15.38:0/2454195564 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f7d3005e8f0 0x7f7d30060db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:16.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.568+0000 7f7d499e5700 1 --2- 172.21.15.38:0/2454195564 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f7d3005e8f0 0x7f7d30060db0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f7d34009720 tx=0x7f7d34016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:16.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.572+0000 7f7d3affd700 1 -- 172.21.15.38:0/2454195564 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f7d4004bfa0 con 0x7f7d440f55c0 2024-02-15T18:35:16.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:15.776171+0000 mgr.smithi038.khesbl (mgr.14162) 348 : audit [DBG] from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:35:16.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:15.776171+0000 mgr.smithi038.khesbl (mgr.14162) 348 : audit [DBG] from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:35:16.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.800+0000 7f7d4c44a700 1 -- 172.21.15.38:0/2454195564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7d440fde90 con 0x7f7d440f55c0 2024-02-15T18:35:16.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.801+0000 7f7d3affd700 1 -- 172.21.15.38:0/2454195564 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f7d40019330 con 0x7f7d440f55c0 2024-02-15T18:35:16.791 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:35:16.792 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-02-15T18:35:16.792 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2024-02-15T18:35:16.792 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:35:16.792 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-02-15T18:35:16.792 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 1, 2024-02-15T18:35:16.792 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 1 2024-02-15T18:35:16.792 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:35:16.792 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-02-15T18:35:16.792 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2024-02-15T18:35:16.792 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:35:16.792 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-02-15T18:35:16.792 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:35:16.792 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:35:16.793 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-02-15T18:35:16.793 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 1, 2024-02-15T18:35:16.793 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 13 2024-02-15T18:35:16.793 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-02-15T18:35:16.793 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:35:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.802+0000 7f7d4c44a700 1 -- 172.21.15.38:0/2454195564 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f7d3005e8f0 msgr2=0x7f7d30060db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.802+0000 7f7d4c44a700 1 --2- 172.21.15.38:0/2454195564 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f7d3005e8f0 0x7f7d30060db0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f7d34009720 tx=0x7f7d34016040).stop 2024-02-15T18:35:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.802+0000 7f7d4c44a700 1 -- 172.21.15.38:0/2454195564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d440f55c0 msgr2=0x7f7d44109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.802+0000 7f7d4c44a700 1 --2- 172.21.15.38:0/2454195564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d440f55c0 0x7f7d44109cd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f7d4000e7c0 tx=0x7f7d4000b040).stop 2024-02-15T18:35:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.802+0000 7f7d4c44a700 1 -- 172.21.15.38:0/2454195564 shutdown_connections 2024-02-15T18:35:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.802+0000 7f7d4c44a700 1 --2- 172.21.15.38:0/2454195564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d440f55c0 0x7f7d44109cd0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.802+0000 7f7d4c44a700 1 --2- 172.21.15.38:0/2454195564 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f7d3005e8f0 0x7f7d30060db0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.802+0000 7f7d4c44a700 1 --2- 172.21.15.38:0/2454195564 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d440f6860 0x7f7d4410c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.802+0000 7f7d4c44a700 1 -- 172.21.15.38:0/2454195564 >> 172.21.15.38:0/2454195564 conn(0x7f7d440f0d40 msgr2=0x7f7d440fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.803+0000 7f7d4c44a700 1 -- 172.21.15.38:0/2454195564 shutdown_connections 2024-02-15T18:35:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.803+0000 7f7d4c44a700 1 -- 172.21.15.38:0/2454195564 wait complete. 2024-02-15T18:35:16.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.904+0000 7f931bc50700 1 -- 172.21.15.38:0/4281523307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93140f6860 msgr2=0x7f93140f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:16.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.904+0000 7f931bc50700 1 --2- 172.21.15.38:0/4281523307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93140f6860 0x7f93140f6cc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f9310004660 tx=0x7f9310009b30).stop 2024-02-15T18:35:16.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.904+0000 7f931bc50700 1 -- 172.21.15.38:0/4281523307 shutdown_connections 2024-02-15T18:35:16.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.904+0000 7f931bc50700 1 --2- 172.21.15.38:0/4281523307 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93140f6860 0x7f93140f6cc0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:16.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.904+0000 7f931bc50700 1 --2- 172.21.15.38:0/4281523307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93140f55c0 0x7f93140f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:16.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.904+0000 7f931bc50700 1 -- 172.21.15.38:0/4281523307 >> 172.21.15.38:0/4281523307 conn(0x7f93140f0d40 msgr2=0x7f93140f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:16.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.904+0000 7f931bc50700 1 -- 172.21.15.38:0/4281523307 shutdown_connections 2024-02-15T18:35:16.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.904+0000 7f931bc50700 1 -- 172.21.15.38:0/4281523307 wait complete. 2024-02-15T18:35:16.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.905+0000 7f931bc50700 1 Processor -- start 2024-02-15T18:35:16.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.905+0000 7f931bc50700 1 -- start start 2024-02-15T18:35:16.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.905+0000 7f931bc50700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93140f55c0 0x7f9314109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:16.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.905+0000 7f931bc50700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93140f6860 0x7f931410c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:16.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.905+0000 7f931bc50700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93140603c0 con 0x7f93140f55c0 2024-02-15T18:35:16.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.905+0000 7f931bc50700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9314060500 con 0x7f93140f6860 2024-02-15T18:35:16.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.905+0000 7f93199ec700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93140f55c0 0x7f9314109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:16.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.905+0000 7f93199ec700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93140f55c0 0x7f9314109cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37494/0 (socket says 172.21.15.38:37494) 2024-02-15T18:35:16.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.905+0000 7f93199ec700 1 -- 172.21.15.38:0/3349259780 learned_addr learned my addr 172.21.15.38:0/3349259780 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:35:16.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.905+0000 7f93191eb700 1 --2- 172.21.15.38:0/3349259780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93140f6860 0x7f931410c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:16.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.906+0000 7f93199ec700 1 -- 172.21.15.38:0/3349259780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93140f6860 msgr2=0x7f931410c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:16.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.906+0000 7f93199ec700 1 --2- 172.21.15.38:0/3349259780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93140f6860 0x7f931410c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:16.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.906+0000 7f93199ec700 1 -- 172.21.15.38:0/3349259780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9310005040 con 0x7f93140f55c0 2024-02-15T18:35:16.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.906+0000 7f93191eb700 1 --2- 172.21.15.38:0/3349259780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93140f6860 0x7f931410c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:35:16.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.906+0000 7f93199ec700 1 --2- 172.21.15.38:0/3349259780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93140f55c0 0x7f9314109cd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f930800e7c0 tx=0x7f930800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:16.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.906+0000 7f93067fc700 1 -- 172.21.15.38:0/3349259780 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93080092e0 con 0x7f93140f55c0 2024-02-15T18:35:16.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.906+0000 7f93067fc700 1 -- 172.21.15.38:0/3349259780 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f930800f040 con 0x7f93140f55c0 2024-02-15T18:35:16.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.906+0000 7f93067fc700 1 -- 172.21.15.38:0/3349259780 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9308014920 con 0x7f93140f55c0 2024-02-15T18:35:16.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.906+0000 7f931bc50700 1 -- 172.21.15.38:0/3349259780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931410c780 con 0x7f93140f55c0 2024-02-15T18:35:16.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.906+0000 7f931bc50700 1 -- 172.21.15.38:0/3349259780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931410ccc0 con 0x7f93140f55c0 2024-02-15T18:35:16.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.907+0000 7f931bc50700 1 -- 172.21.15.38:0/3349259780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9314045ad0 con 0x7f93140f55c0 2024-02-15T18:35:16.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.907+0000 7f93067fc700 1 -- 172.21.15.38:0/3349259780 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 74675+0+0 (secure 0 0 0) 0x7f930801b030 con 0x7f93140f55c0 2024-02-15T18:35:16.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.908+0000 7f93067fc700 1 --2- 172.21.15.38:0/3349259780 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f930005e8f0 0x7f9300060db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:16.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.908+0000 7f93067fc700 1 -- 172.21.15.38:0/3349259780 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f9308078e60 con 0x7f93140f55c0 2024-02-15T18:35:16.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.908+0000 7f93191eb700 1 --2- 172.21.15.38:0/3349259780 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f930005e8f0 0x7f9300060db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:16.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.908+0000 7f93191eb700 1 --2- 172.21.15.38:0/3349259780 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f930005e8f0 0x7f9300060db0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f9310005010 tx=0x7f9310016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:16.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:16.912+0000 7f93067fc700 1 -- 172.21.15.38:0/3349259780 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f930804bac0 con 0x7f93140f55c0 2024-02-15T18:35:17.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.135+0000 7f931bc50700 1 -- 172.21.15.38:0/3349259780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9314027a00 con 0x7f93140f55c0 2024-02-15T18:35:17.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.136+0000 7f93067fc700 1 -- 172.21.15.38:0/3349259780 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1660 (secure 0 0 0) 0x7f9308019030 con 0x7f93140f55c0 2024-02-15T18:35:17.127 INFO:teuthology.orchestra.run.smithi038.stdout:e13 2024-02-15T18:35:17.127 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:35:17.127 INFO:teuthology.orchestra.run.smithi038.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:35:17.127 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-02-15T18:35:17.127 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:35:17.127 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:35:17.127 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-02-15T18:35:17.127 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 13 2024-02-15T18:35:17.128 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 2024-02-15T18:35:17.128 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:35:17.128 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-02-15T18:35:07.485909+0000 2024-02-15T18:35:17.128 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-02-15T18:35:17.128 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-02-15T18:35:17.128 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-02-15T18:35:17.128 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-02-15T18:35:17.128 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-02-15T18:35:17.128 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-02-15T18:35:17.128 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-02-15T18:35:17.128 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-02-15T18:35:17.129 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:35:17.129 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-02-15T18:35:17.129 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-02-15T18:35:17.129 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=24261,1=24275} 2024-02-15T18:35:17.129 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-02-15T18:35:17.129 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-02-15T18:35:17.129 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-02-15T18:35:17.129 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-02-15T18:35:17.129 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-02-15T18:35:17.129 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-02-15T18:35:17.129 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-02-15T18:35:17.129 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-02-15T18:35:17.129 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.mvumlh{0:24261} state up:active seq 3 export targets 1 join_fscid=1 addr [v2:172.21.15.121:6824/1422707571,v1:172.21.15.121:6825/1422707571]] 2024-02-15T18:35:17.130 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.ovuzki{1:24275} state up:active seq 2 export targets 0 join_fscid=1 addr [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845]] 2024-02-15T18:35:17.130 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:35:17.130 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:35:17.130 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-02-15T18:35:17.130 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:35:17.130 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.hxoggl{-1:14446} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.38:6826/3822880235,v1:172.21.15.38:6827/3822880235]] 2024-02-15T18:35:17.130 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.qgkzgn{-1:14456} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.38:6828/703030231,v1:172.21.15.38:6829/703030231]] 2024-02-15T18:35:17.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.138+0000 7f931bc50700 1 -- 172.21.15.38:0/3349259780 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f930005e8f0 msgr2=0x7f9300060db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:17.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.138+0000 7f931bc50700 1 --2- 172.21.15.38:0/3349259780 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f930005e8f0 0x7f9300060db0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f9310005010 tx=0x7f9310016040).stop 2024-02-15T18:35:17.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.138+0000 7f931bc50700 1 -- 172.21.15.38:0/3349259780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93140f55c0 msgr2=0x7f9314109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:17.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.138+0000 7f931bc50700 1 --2- 172.21.15.38:0/3349259780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93140f55c0 0x7f9314109cd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f930800e7c0 tx=0x7f930800b040).stop 2024-02-15T18:35:17.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.138+0000 7f931bc50700 1 -- 172.21.15.38:0/3349259780 shutdown_connections 2024-02-15T18:35:17.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.138+0000 7f931bc50700 1 --2- 172.21.15.38:0/3349259780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93140f55c0 0x7f9314109cd0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:17.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.138+0000 7f931bc50700 1 --2- 172.21.15.38:0/3349259780 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7f930005e8f0 0x7f9300060db0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:17.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.138+0000 7f931bc50700 1 --2- 172.21.15.38:0/3349259780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93140f6860 0x7f931410c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:17.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.138+0000 7f931bc50700 1 -- 172.21.15.38:0/3349259780 >> 172.21.15.38:0/3349259780 conn(0x7f93140f0d40 msgr2=0x7f93140fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:17.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.138+0000 7f931bc50700 1 -- 172.21.15.38:0/3349259780 shutdown_connections 2024-02-15T18:35:17.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.138+0000 7f931bc50700 1 -- 172.21.15.38:0/3349259780 wait complete. 2024-02-15T18:35:17.132 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 13 2024-02-15T18:35:17.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.258+0000 7fb2ccbe6700 1 -- 172.21.15.38:0/1937238792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2c80f6860 msgr2=0x7fb2c80f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:17.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.258+0000 7fb2ccbe6700 1 --2- 172.21.15.38:0/1937238792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2c80f6860 0x7fb2c80f6cc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fb2b0004660 tx=0x7fb2b0009b30).stop 2024-02-15T18:35:17.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.258+0000 7fb2ccbe6700 1 -- 172.21.15.38:0/1937238792 shutdown_connections 2024-02-15T18:35:17.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.258+0000 7fb2ccbe6700 1 --2- 172.21.15.38:0/1937238792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2c80f6860 0x7fb2c80f6cc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:17.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.258+0000 7fb2ccbe6700 1 --2- 172.21.15.38:0/1937238792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2c80f55c0 0x7fb2c80f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:17.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.258+0000 7fb2ccbe6700 1 -- 172.21.15.38:0/1937238792 >> 172.21.15.38:0/1937238792 conn(0x7fb2c80f0d40 msgr2=0x7fb2c80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:17.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.258+0000 7fb2ccbe6700 1 -- 172.21.15.38:0/1937238792 shutdown_connections 2024-02-15T18:35:17.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.258+0000 7fb2ccbe6700 1 -- 172.21.15.38:0/1937238792 wait complete. 2024-02-15T18:35:17.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.258+0000 7fb2ccbe6700 1 Processor -- start 2024-02-15T18:35:17.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.258+0000 7fb2ccbe6700 1 -- start start 2024-02-15T18:35:17.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.259+0000 7fb2ccbe6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2c80f55c0 0x7fb2c80f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.259+0000 7fb2ccbe6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2c80f6860 0x7fb2c80f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.259+0000 7fb2ccbe6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c80f8100 con 0x7fb2c80f55c0 2024-02-15T18:35:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.259+0000 7fb2ccbe6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c80f8240 con 0x7fb2c80f6860 2024-02-15T18:35:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.259+0000 7fb2c659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2c80f55c0 0x7fb2c80f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.259+0000 7fb2c659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2c80f55c0 0x7fb2c80f9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37508/0 (socket says 172.21.15.38:37508) 2024-02-15T18:35:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.259+0000 7fb2c659c700 1 -- 172.21.15.38:0/2858431719 learned_addr learned my addr 172.21.15.38:0/2858431719 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:35:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.259+0000 7fb2c5d9b700 1 --2- 172.21.15.38:0/2858431719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2c80f6860 0x7fb2c80f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.259+0000 7fb2c659c700 1 -- 172.21.15.38:0/2858431719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2c80f6860 msgr2=0x7fb2c80f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.259+0000 7fb2c659c700 1 --2- 172.21.15.38:0/2858431719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2c80f6860 0x7fb2c80f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.259+0000 7fb2c659c700 1 -- 172.21.15.38:0/2858431719 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b0005040 con 0x7fb2c80f55c0 2024-02-15T18:35:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.259+0000 7fb2c659c700 1 --2- 172.21.15.38:0/2858431719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2c80f55c0 0x7fb2c80f9720 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fb2b800e7c0 tx=0x7fb2b800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:17.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.259+0000 7fb2beffd700 1 -- 172.21.15.38:0/2858431719 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2b80092e0 con 0x7fb2c80f55c0 2024-02-15T18:35:17.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.259+0000 7fb2beffd700 1 -- 172.21.15.38:0/2858431719 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fb2b800f040 con 0x7fb2c80f55c0 2024-02-15T18:35:17.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.259+0000 7fb2beffd700 1 -- 172.21.15.38:0/2858431719 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2b8014920 con 0x7fb2c80f55c0 2024-02-15T18:35:17.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.259+0000 7fb2ccbe6700 1 -- 172.21.15.38:0/2858431719 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c80f8520 con 0x7fb2c80f55c0 2024-02-15T18:35:17.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.259+0000 7fb2ccbe6700 1 -- 172.21.15.38:0/2858431719 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c810eff0 con 0x7fb2c80f55c0 2024-02-15T18:35:17.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.260+0000 7fb2ccbe6700 1 -- 172.21.15.38:0/2858431719 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c8045ad0 con 0x7fb2c80f55c0 2024-02-15T18:35:17.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.261+0000 7fb2beffd700 1 -- 172.21.15.38:0/2858431719 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) v1 ==== 74675+0+0 (secure 0 0 0) 0x7fb2b801b030 con 0x7fb2c80f55c0 2024-02-15T18:35:17.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.261+0000 7fb2beffd700 1 --2- 172.21.15.38:0/2858431719 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb2b405e8f0 0x7fb2b4060db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:17.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.261+0000 7fb2beffd700 1 -- 172.21.15.38:0/2858431719 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fb2b8078db0 con 0x7fb2c80f55c0 2024-02-15T18:35:17.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.261+0000 7fb2c5d9b700 1 --2- 172.21.15.38:0/2858431719 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb2b405e8f0 0x7fb2b4060db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:17.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.262+0000 7fb2c5d9b700 1 --2- 172.21.15.38:0/2858431719 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb2b405e8f0 0x7fb2b4060db0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fb2b0009720 tx=0x7fb2b0016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:17.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.265+0000 7fb2beffd700 1 -- 172.21.15.38:0/2858431719 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb2b804ba10 con 0x7fb2c80f55c0 2024-02-15T18:35:17.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.464+0000 7fb2ccbe6700 1 -- 172.21.15.38:0/2858431719 --> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2c80faca0 con 0x7fb2b405e8f0 2024-02-15T18:35:17.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.466+0000 7fb2beffd700 1 -- 172.21.15.38:0/2858431719 <== mgr.14162 v2:172.21.15.38:6800/2900497610 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+274 (secure 0 0 0) 0x7fb2c80faca0 con 0x7fb2b405e8f0 2024-02-15T18:35:17.456 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:35:17.456 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:35:17.456 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-02-15T18:35:17.457 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [], 2024-02-15T18:35:17.457 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "1/21 ceph daemons upgraded", 2024-02-15T18:35:17.457 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading mgr daemons" 2024-02-15T18:35:17.457 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:35:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.467+0000 7fb2ccbe6700 1 -- 172.21.15.38:0/2858431719 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb2b405e8f0 msgr2=0x7fb2b4060db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.467+0000 7fb2ccbe6700 1 --2- 172.21.15.38:0/2858431719 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb2b405e8f0 0x7fb2b4060db0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fb2b0009720 tx=0x7fb2b0016040).stop 2024-02-15T18:35:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.467+0000 7fb2ccbe6700 1 -- 172.21.15.38:0/2858431719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2c80f55c0 msgr2=0x7fb2c80f9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.467+0000 7fb2ccbe6700 1 --2- 172.21.15.38:0/2858431719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2c80f55c0 0x7fb2c80f9720 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fb2b800e7c0 tx=0x7fb2b800b040).stop 2024-02-15T18:35:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.467+0000 7fb2ccbe6700 1 -- 172.21.15.38:0/2858431719 shutdown_connections 2024-02-15T18:35:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.467+0000 7fb2ccbe6700 1 --2- 172.21.15.38:0/2858431719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2c80f55c0 0x7fb2c80f9720 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.467+0000 7fb2ccbe6700 1 --2- 172.21.15.38:0/2858431719 >> [v2:172.21.15.38:6800/2900497610,v1:172.21.15.38:6801/2900497610] conn(0x7fb2b405e8f0 0x7fb2b4060db0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.467+0000 7fb2ccbe6700 1 --2- 172.21.15.38:0/2858431719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2c80f6860 0x7fb2c80f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.467+0000 7fb2ccbe6700 1 -- 172.21.15.38:0/2858431719 >> 172.21.15.38:0/2858431719 conn(0x7fb2c80f0d40 msgr2=0x7fb2c805fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.468+0000 7fb2ccbe6700 1 -- 172.21.15.38:0/2858431719 shutdown_connections 2024-02-15T18:35:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:17.468+0000 7fb2ccbe6700 1 -- 172.21.15.38:0/2858431719 wait complete. 2024-02-15T18:35:17.719 INFO:tasks.workunit.client.1.smithi121.stdout:7/87: sync 2024-02-15T18:35:17.719 INFO:tasks.workunit.client.1.smithi121.stdout:9/17: sync 2024-02-15T18:35:17.719 INFO:tasks.workunit.client.1.smithi121.stdout:2/3: sync 2024-02-15T18:35:17.720 INFO:tasks.workunit.client.1.smithi121.stdout:1/29: sync 2024-02-15T18:35:17.720 INFO:tasks.workunit.client.1.smithi121.stdout:5/0: sync 2024-02-15T18:35:17.720 INFO:tasks.workunit.client.1.smithi121.stdout:3/23: sync 2024-02-15T18:35:17.720 INFO:tasks.workunit.client.1.smithi121.stdout:8/112: sync 2024-02-15T18:35:17.720 INFO:tasks.workunit.client.1.smithi121.stdout:0/22: sync 2024-02-15T18:35:17.720 INFO:tasks.workunit.client.1.smithi121.stdout:6/38: sync 2024-02-15T18:35:17.720 INFO:tasks.workunit.client.1.smithi121.stdout:4/1: sync 2024-02-15T18:35:17.720 INFO:tasks.workunit.client.1.smithi121.stdout:9/18: write f5 [347938,23254] 0 2024-02-15T18:35:17.720 INFO:tasks.workunit.client.1.smithi121.stdout:9/19: chown c4 851784 1 2024-02-15T18:35:17.721 INFO:tasks.workunit.client.1.smithi121.stdout:8/113: write d0/d7/de/f18 [4048363,16986] 0 2024-02-15T18:35:17.721 INFO:tasks.workunit.client.1.smithi121.stdout:9/20: write f1 [2545823,233] 0 2024-02-15T18:35:17.722 INFO:tasks.workunit.client.1.smithi121.stdout:1/30: mknod d0/d6/ca 0 2024-02-15T18:35:17.722 INFO:tasks.workunit.client.1.smithi121.stdout:1/31: truncate - no filename 2024-02-15T18:35:17.722 INFO:tasks.workunit.client.1.smithi121.stdout:1/32: dread - no filename 2024-02-15T18:35:17.723 INFO:tasks.workunit.client.1.smithi121.stdout:2/4: mknod c2 0 2024-02-15T18:35:17.723 INFO:tasks.workunit.client.1.smithi121.stdout:2/5: write - no filename 2024-02-15T18:35:17.724 INFO:tasks.workunit.client.1.smithi121.stdout:3/24: creat d2/f9 x:0 0 0 2024-02-15T18:35:17.726 INFO:tasks.workunit.client.1.smithi121.stdout:9/21: fsync f1 0 2024-02-15T18:35:17.726 INFO:tasks.workunit.client.1.smithi121.stdout:7/88: dread d0/d5/f9 [0,4194304] 0 2024-02-15T18:35:17.727 INFO:tasks.workunit.client.1.smithi121.stdout:5/1: mkdir d0 0 2024-02-15T18:35:17.728 INFO:tasks.workunit.client.1.smithi121.stdout:6/39: creat d7/f11 x:0 0 0 2024-02-15T18:35:17.728 INFO:tasks.workunit.client.1.smithi121.stdout:6/40: chown d7/dc/lf 845 1 2024-02-15T18:35:17.730 INFO:tasks.workunit.client.1.smithi121.stdout:4/2: symlink l0 0 2024-02-15T18:35:17.730 INFO:tasks.workunit.client.1.smithi121.stdout:4/3: dread - no filename 2024-02-15T18:35:17.730 INFO:tasks.workunit.client.1.smithi121.stdout:4/4: write - no filename 2024-02-15T18:35:17.730 INFO:tasks.workunit.client.1.smithi121.stdout:4/5: stat l0 0 2024-02-15T18:35:17.730 INFO:tasks.workunit.client.1.smithi121.stdout:8/114: mknod d0/d7/de/c2b 0 2024-02-15T18:35:17.733 INFO:tasks.workunit.client.1.smithi121.stdout:1/33: creat d0/fb x:0 0 0 2024-02-15T18:35:17.735 INFO:tasks.workunit.client.1.smithi121.stdout:0/23: dwrite f3 [0,4194304] 0 2024-02-15T18:35:17.736 INFO:tasks.workunit.client.1.smithi121.stdout:2/6: creat f3 x:0 0 0 2024-02-15T18:35:17.739 INFO:tasks.workunit.client.1.smithi121.stdout:9/22: mkdir d7/d8 0 2024-02-15T18:35:17.741 INFO:tasks.workunit.client.1.smithi121.stdout:7/89: rename d0/d5/f1e to d0/d5/d11/f20 0 2024-02-15T18:35:17.741 INFO:tasks.workunit.client.1.smithi121.stdout:7/90: readlink d0/d5/l1b 0 2024-02-15T18:35:17.743 INFO:tasks.workunit.client.1.smithi121.stdout:5/2: mknod d0/c1 0 2024-02-15T18:35:17.743 INFO:tasks.workunit.client.1.smithi121.stdout:5/3: dwrite - no filename 2024-02-15T18:35:17.743 INFO:tasks.workunit.client.1.smithi121.stdout:5/4: read - no filename 2024-02-15T18:35:17.745 INFO:tasks.workunit.client.1.smithi121.stdout:3/25: dwrite d2/f7 [0,4194304] 0 2024-02-15T18:35:17.745 INFO:tasks.workunit.client.1.smithi121.stdout:3/26: fdatasync d2/f7 0 2024-02-15T18:35:17.745 INFO:tasks.workunit.client.1.smithi121.stdout:4/6: unlink l0 0 2024-02-15T18:35:17.745 INFO:tasks.workunit.client.1.smithi121.stdout:4/7: dwrite - no filename 2024-02-15T18:35:17.745 INFO:tasks.workunit.client.1.smithi121.stdout:4/8: unlink - no file 2024-02-15T18:35:17.746 INFO:tasks.workunit.client.1.smithi121.stdout:4/9: dwrite - no filename 2024-02-15T18:35:17.748 INFO:tasks.workunit.client.1.smithi121.stdout:8/115: truncate d0/f2 2183182 0 2024-02-15T18:35:17.749 INFO:tasks.workunit.client.1.smithi121.stdout:1/34: mknod d0/d6/d7/cc 0 2024-02-15T18:35:17.754 INFO:tasks.workunit.client.1.smithi121.stdout:2/7: creat f4 x:0 0 0 2024-02-15T18:35:17.755 INFO:tasks.workunit.client.1.smithi121.stdout:9/23: mknod d7/c9 0 2024-02-15T18:35:17.760 INFO:tasks.workunit.client.1.smithi121.stdout:5/5: mkdir d0/d2 0 2024-02-15T18:35:17.760 INFO:tasks.workunit.client.1.smithi121.stdout:5/6: write - no filename 2024-02-15T18:35:17.761 INFO:tasks.workunit.client.1.smithi121.stdout:5/7: chown d0/c1 668380 1 2024-02-15T18:35:17.761 INFO:tasks.workunit.client.1.smithi121.stdout:5/8: truncate - no filename 2024-02-15T18:35:17.761 INFO:tasks.workunit.client.1.smithi121.stdout:5/9: getdents d0/d2 0 2024-02-15T18:35:17.761 INFO:tasks.workunit.client.1.smithi121.stdout:5/10: write - no filename 2024-02-15T18:35:17.764 INFO:tasks.workunit.client.1.smithi121.stdout:0/24: dwrite f2 [0,4194304] 0 2024-02-15T18:35:17.764 INFO:tasks.workunit.client.1.smithi121.stdout:0/25: dread - f4 zero size 2024-02-15T18:35:17.764 INFO:tasks.workunit.client.1.smithi121.stdout:3/27: symlink d2/la 0 2024-02-15T18:35:17.765 INFO:tasks.workunit.client.1.smithi121.stdout:6/41: dwrite d7/f9 [4194304,4194304] 0 2024-02-15T18:35:17.765 INFO:tasks.workunit.client.1.smithi121.stdout:6/42: write d7/dc/fd [141354,40819] 0 2024-02-15T18:35:17.765 INFO:tasks.workunit.client.1.smithi121.stdout:4/10: creat f1 x:0 0 0 2024-02-15T18:35:17.770 INFO:tasks.workunit.client.1.smithi121.stdout:6/43: dread d7/f8 [0,4194304] 0 2024-02-15T18:35:17.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:16.114248+0000 mgr.smithi038.khesbl (mgr.14162) 349 : audit [DBG] from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:35:17.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:16.322965+0000 mgr.smithi038.khesbl (mgr.14162) 350 : cluster [DBG] pgmap v290: 65 pgs: 65 active+clean; 489 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 7.9 MiB/s rd, 30 MiB/s wr, 202 op/s 2024-02-15T18:35:17.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:16.453476+0000 mgr.smithi038.khesbl (mgr.14162) 351 : audit [DBG] from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:35:17.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:16.801470+0000 mon.smithi038 (mon.0) 573 : audit [DBG] from='client.? 172.21.15.38:0/2454195564' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:35:17.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:17.136613+0000 mon.smithi038 (mon.0) 574 : audit [DBG] from='client.? 172.21.15.38:0/3349259780' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:35:17.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:17.297919+0000 mon.smithi038 (mon.0) 575 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:35:17.785 INFO:tasks.workunit.client.1.smithi121.stdout:1/35: mknod d0/d6/cd 0 2024-02-15T18:35:17.786 INFO:tasks.workunit.client.1.smithi121.stdout:2/8: mkdir d5 0 2024-02-15T18:35:17.786 INFO:tasks.workunit.client.1.smithi121.stdout:2/9: fdatasync f4 0 2024-02-15T18:35:17.787 INFO:tasks.workunit.client.1.smithi121.stdout:7/91: rename d0/f10 to d0/d5/d11/d1c/f21 0 2024-02-15T18:35:17.789 INFO:tasks.workunit.client.1.smithi121.stdout:0/26: creat d5/f9 x:0 0 0 2024-02-15T18:35:17.795 INFO:tasks.workunit.client.1.smithi121.stdout:3/28: unlink d2/la 0 2024-02-15T18:35:17.800 INFO:tasks.workunit.client.1.smithi121.stdout:8/116: rename d0/d7/f1b to d0/d7/de/d27/d28/f2c 0 2024-02-15T18:35:17.800 INFO:tasks.workunit.client.1.smithi121.stdout:8/117: fdatasync d0/d7/de/d27/d28/f2c 0 2024-02-15T18:35:17.800 INFO:tasks.workunit.client.1.smithi121.stdout:8/118: chown d0/d3/c9 602692376 1 2024-02-15T18:35:17.801 INFO:tasks.workunit.client.1.smithi121.stdout:7/92: creat d0/d5/f22 x:0 0 0 2024-02-15T18:35:17.802 INFO:tasks.workunit.client.1.smithi121.stdout:7/93: fdatasync d0/d5/d11/d1c/f21 0 2024-02-15T18:35:17.803 INFO:tasks.workunit.client.1.smithi121.stdout:3/29: rename d2/c3 to d2/cb 0 2024-02-15T18:35:17.806 INFO:tasks.workunit.client.1.smithi121.stdout:0/27: rename d5/d6/c8 to d5/ca 0 2024-02-15T18:35:17.806 INFO:tasks.workunit.client.1.smithi121.stdout:3/30: rename d2 to d2/dc 22 2024-02-15T18:35:17.807 INFO:tasks.workunit.client.1.smithi121.stdout:6/44: dwrite f0 [0,4194304] 0 2024-02-15T18:35:17.807 INFO:tasks.workunit.client.1.smithi121.stdout:6/45: truncate d7/f11 801189 0 2024-02-15T18:35:17.808 INFO:tasks.workunit.client.1.smithi121.stdout:4/11: dwrite f1 [0,4194304] 0 2024-02-15T18:35:17.808 INFO:tasks.workunit.client.1.smithi121.stdout:4/12: fdatasync f1 0 2024-02-15T18:35:17.808 INFO:tasks.workunit.client.1.smithi121.stdout:2/10: dwrite f3 [0,4194304] 0 2024-02-15T18:35:17.808 INFO:tasks.workunit.client.1.smithi121.stdout:1/36: dwrite d0/fb [0,4194304] 0 2024-02-15T18:35:17.808 INFO:tasks.workunit.client.1.smithi121.stdout:8/119: creat d0/d7/de/f2d x:0 0 0 2024-02-15T18:35:17.810 INFO:tasks.workunit.client.1.smithi121.stdout:8/120: dread d0/f5 [0,4194304] 0 2024-02-15T18:35:17.811 INFO:tasks.workunit.client.1.smithi121.stdout:8/121: write d0/f5 [247195,57587] 0 2024-02-15T18:35:17.811 INFO:tasks.workunit.client.1.smithi121.stdout:8/122: write d0/f5 [1388326,16009] 0 2024-02-15T18:35:17.811 INFO:tasks.workunit.client.1.smithi121.stdout:8/123: chown d0/d3/c17 713777 1 2024-02-15T18:35:17.815 INFO:tasks.workunit.client.1.smithi121.stdout:7/94: creat d0/d5/d11/d1c/f23 x:0 0 0 2024-02-15T18:35:17.815 INFO:tasks.workunit.client.1.smithi121.stdout:7/95: chown d0/d5/l1b 31 1 2024-02-15T18:35:17.817 INFO:tasks.workunit.client.1.smithi121.stdout:3/31: rename d2/f4 to d2/fd 0 2024-02-15T18:35:17.817 INFO:tasks.workunit.client.1.smithi121.stdout:3/32: fdatasync d2/fd 0 2024-02-15T18:35:17.820 INFO:tasks.workunit.client.1.smithi121.stdout:4/13: mknod c2 0 2024-02-15T18:35:17.828 INFO:tasks.workunit.client.1.smithi121.stdout:4/14: dread f1 [0,4194304] 0 2024-02-15T18:35:17.828 INFO:tasks.workunit.client.1.smithi121.stdout:4/15: write f1 [4212961,105604] 0 2024-02-15T18:35:17.831 INFO:tasks.workunit.client.1.smithi121.stdout:2/11: creat d5/f6 x:0 0 0 2024-02-15T18:35:17.834 INFO:tasks.workunit.client.1.smithi121.stdout:1/37: symlink d0/le 0 2024-02-15T18:35:17.838 INFO:tasks.workunit.client.1.smithi121.stdout:7/96: symlink d0/d5/d11/l24 0 2024-02-15T18:35:17.839 INFO:tasks.workunit.client.1.smithi121.stdout:3/33: rmdir d2 39 2024-02-15T18:35:17.843 INFO:tasks.workunit.client.1.smithi121.stdout:2/12: rename c0 to d5/c7 0 2024-02-15T18:35:17.843 INFO:tasks.workunit.client.1.smithi121.stdout:2/13: write f4 [558367,127025] 0 2024-02-15T18:35:17.843 INFO:tasks.workunit.client.1.smithi121.stdout:1/38: mknod d0/cf 0 2024-02-15T18:35:17.849 INFO:tasks.workunit.client.1.smithi121.stdout:3/34: truncate d2/f7 1550265 0 2024-02-15T18:35:17.852 INFO:tasks.workunit.client.1.smithi121.stdout:7/97: link d0/c2 d0/d5/d11/c25 0 2024-02-15T18:35:17.852 INFO:tasks.workunit.client.1.smithi121.stdout:3/35: creat d2/fe x:0 0 0 2024-02-15T18:35:17.852 INFO:tasks.workunit.client.1.smithi121.stdout:3/36: fsync d2/f9 0 2024-02-15T18:35:17.855 INFO:tasks.workunit.client.1.smithi121.stdout:4/16: truncate f1 1299930 0 2024-02-15T18:35:17.856 INFO:tasks.workunit.client.1.smithi121.stdout:3/37: link d2/cb d2/cf 0 2024-02-15T18:35:17.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:16.114248+0000 mgr.smithi038.khesbl (mgr.14162) 349 : audit [DBG] from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:35:17.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:16.322965+0000 mgr.smithi038.khesbl (mgr.14162) 350 : cluster [DBG] pgmap v290: 65 pgs: 65 active+clean; 489 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 7.9 MiB/s rd, 30 MiB/s wr, 202 op/s 2024-02-15T18:35:17.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:16.453476+0000 mgr.smithi038.khesbl (mgr.14162) 351 : audit [DBG] from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:35:17.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:16.801470+0000 mon.smithi038 (mon.0) 573 : audit [DBG] from='client.? 172.21.15.38:0/2454195564' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:35:17.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:17.136613+0000 mon.smithi038 (mon.0) 574 : audit [DBG] from='client.? 172.21.15.38:0/3349259780' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:35:17.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:17.297919+0000 mon.smithi038 (mon.0) 575 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:35:17.858 INFO:tasks.workunit.client.1.smithi121.stdout:3/38: mknod d2/c10 0 2024-02-15T18:35:17.859 INFO:tasks.workunit.client.1.smithi121.stdout:2/14: dwrite f3 [0,4194304] 0 2024-02-15T18:35:17.860 INFO:tasks.workunit.client.1.smithi121.stdout:3/39: creat d2/f11 x:0 0 0 2024-02-15T18:35:17.866 INFO:tasks.workunit.client.1.smithi121.stdout:2/15: rename f3 to d5/f8 0 2024-02-15T18:35:17.867 INFO:tasks.workunit.client.1.smithi121.stdout:3/40: mknod d2/c12 0 2024-02-15T18:35:17.870 INFO:tasks.workunit.client.1.smithi121.stdout:4/17: read f1 [1116262,86812] 0 2024-02-15T18:35:17.871 INFO:tasks.workunit.client.1.smithi121.stdout:7/98: dwrite d0/f18 [0,4194304] 0 2024-02-15T18:35:17.871 INFO:tasks.workunit.client.1.smithi121.stdout:7/99: fsync d0/d5/d11/d1c/f1f 0 2024-02-15T18:35:17.871 INFO:tasks.workunit.client.1.smithi121.stdout:7/100: fsync d0/f18 0 2024-02-15T18:35:17.871 INFO:tasks.workunit.client.1.smithi121.stdout:2/16: mknod d5/c9 0 2024-02-15T18:35:17.872 INFO:tasks.workunit.client.1.smithi121.stdout:3/41: symlink d2/l13 0 2024-02-15T18:35:17.873 INFO:tasks.workunit.client.1.smithi121.stdout:3/42: chown d2/cf 67123 1 2024-02-15T18:35:17.877 INFO:tasks.workunit.client.1.smithi121.stdout:7/101: truncate d0/fc 514418 0 2024-02-15T18:35:17.894 INFO:tasks.workunit.client.1.smithi121.stdout:3/43: dwrite d2/f7 [0,4194304] 0 2024-02-15T18:35:17.894 INFO:tasks.workunit.client.1.smithi121.stdout:3/44: readlink l0 0 2024-02-15T18:35:17.894 INFO:tasks.workunit.client.1.smithi121.stdout:3/45: readlink d2/l13 0 2024-02-15T18:35:17.894 INFO:tasks.workunit.client.1.smithi121.stdout:2/17: dwrite d5/f8 [0,4194304] 0 2024-02-15T18:35:17.894 INFO:tasks.workunit.client.1.smithi121.stdout:2/18: chown d5/c7 13178650 1 2024-02-15T18:35:17.896 INFO:tasks.workunit.client.1.smithi121.stdout:3/46: dread d2/fd [0,4194304] 0 2024-02-15T18:35:17.896 INFO:tasks.workunit.client.1.smithi121.stdout:2/19: dread f4 [0,4194304] 0 2024-02-15T18:35:17.899 INFO:tasks.workunit.client.1.smithi121.stdout:3/47: mknod d2/c14 0 2024-02-15T18:35:17.900 INFO:tasks.workunit.client.1.smithi121.stdout:2/20: getdents d5 0 2024-02-15T18:35:17.903 INFO:tasks.workunit.client.1.smithi121.stdout:3/48: unlink d2/c8 0 2024-02-15T18:35:17.905 INFO:tasks.workunit.client.1.smithi121.stdout:4/18: dwrite f1 [0,4194304] 0 2024-02-15T18:35:17.905 INFO:tasks.workunit.client.1.smithi121.stdout:4/19: fdatasync f1 0 2024-02-15T18:35:17.905 INFO:tasks.workunit.client.1.smithi121.stdout:4/20: stat c2 0 2024-02-15T18:35:17.905 INFO:tasks.workunit.client.1.smithi121.stdout:3/49: mkdir d2/d15 0 2024-02-15T18:35:17.908 INFO:tasks.workunit.client.1.smithi121.stdout:4/21: mknod c3 0 2024-02-15T18:35:17.909 INFO:tasks.workunit.client.1.smithi121.stdout:4/22: fsync f1 0 2024-02-15T18:35:17.910 INFO:tasks.workunit.client.1.smithi121.stdout:3/50: creat d2/f16 x:0 0 0 2024-02-15T18:35:17.920 INFO:tasks.workunit.client.1.smithi121.stdout:3/51: write d2/fd [77184,41956] 0 2024-02-15T18:35:17.921 INFO:tasks.workunit.client.1.smithi121.stdout:3/52: mknod d2/d15/c17 0 2024-02-15T18:35:17.922 INFO:tasks.workunit.client.1.smithi121.stdout:3/53: symlink d2/d15/l18 0 2024-02-15T18:35:17.922 INFO:tasks.workunit.client.1.smithi121.stdout:3/54: chown d2/l13 119 1 2024-02-15T18:35:17.924 INFO:tasks.workunit.client.1.smithi121.stdout:3/55: rename d2/f16 to d2/d15/f19 0 2024-02-15T18:35:17.926 INFO:tasks.workunit.client.1.smithi121.stdout:3/56: mkdir d2/d1a 0 2024-02-15T18:35:17.928 INFO:tasks.workunit.client.1.smithi121.stdout:3/57: creat d2/d15/f1b x:0 0 0 2024-02-15T18:35:17.928 INFO:tasks.workunit.client.1.smithi121.stdout:3/58: truncate d2/d15/f1b 161161 0 2024-02-15T18:35:17.933 INFO:tasks.workunit.client.1.smithi121.stdout:4/23: dwrite f1 [0,4194304] 0 2024-02-15T18:35:17.963 INFO:tasks.workunit.client.1.smithi121.stdout:3/59: dwrite d2/f7 [4194304,4194304] 0 2024-02-15T18:35:17.964 INFO:tasks.workunit.client.1.smithi121.stdout:3/60: dread d2/d15/f1b [0,4194304] 0 2024-02-15T18:35:17.965 INFO:tasks.workunit.client.1.smithi121.stdout:4/24: dwrite f1 [0,4194304] 0 2024-02-15T18:35:17.967 INFO:tasks.workunit.client.1.smithi121.stdout:3/61: link d2/fe d2/d1a/f1c 0 2024-02-15T18:35:17.967 INFO:tasks.workunit.client.1.smithi121.stdout:3/62: dread - d2/d15/f19 zero size 2024-02-15T18:35:17.976 INFO:tasks.workunit.client.1.smithi121.stdout:4/25: dread f1 [0,4194304] 0 2024-02-15T18:35:17.979 INFO:tasks.workunit.client.1.smithi121.stdout:3/63: dwrite d2/f11 [0,4194304] 0 2024-02-15T18:35:17.984 INFO:tasks.workunit.client.1.smithi121.stdout:3/64: symlink d2/d1a/l1d 0 2024-02-15T18:35:17.984 INFO:tasks.workunit.client.1.smithi121.stdout:3/65: chown d2/f11 609834 1 2024-02-15T18:35:17.992 INFO:tasks.workunit.client.1.smithi121.stdout:3/66: dread d2/f11 [0,4194304] 0 2024-02-15T18:35:17.992 INFO:tasks.workunit.client.1.smithi121.stdout:3/67: chown d2/d15/l18 13257712 1 2024-02-15T18:35:17.992 INFO:tasks.workunit.client.1.smithi121.stdout:3/68: truncate d2/f9 508581 0 2024-02-15T18:35:17.995 INFO:tasks.workunit.client.1.smithi121.stdout:3/69: symlink d2/d1a/l1e 0 2024-02-15T18:35:18.004 INFO:tasks.workunit.client.1.smithi121.stdout:4/26: dwrite f1 [0,4194304] 0 2024-02-15T18:35:18.006 INFO:tasks.workunit.client.1.smithi121.stdout:4/27: write f1 [314087,73211] 0 2024-02-15T18:35:18.016 INFO:tasks.workunit.client.1.smithi121.stdout:3/70: dwrite d2/f9 [0,4194304] 0 2024-02-15T18:35:18.020 INFO:tasks.workunit.client.1.smithi121.stdout:3/71: creat d2/d1a/f1f x:0 0 0 2024-02-15T18:35:18.031 INFO:tasks.workunit.client.1.smithi121.stdout:3/72: unlink d2/l13 0 2024-02-15T18:35:18.031 INFO:tasks.workunit.client.1.smithi121.stdout:3/73: readlink d2/d1a/l1e 0 2024-02-15T18:35:18.035 INFO:tasks.workunit.client.1.smithi121.stdout:3/74: unlink d2/c10 0 2024-02-15T18:35:18.036 INFO:tasks.workunit.client.1.smithi121.stdout:3/75: symlink d2/l20 0 2024-02-15T18:35:18.036 INFO:tasks.workunit.client.1.smithi121.stdout:3/76: write d2/fe [688919,40649] 0 2024-02-15T18:35:18.038 INFO:tasks.workunit.client.1.smithi121.stdout:4/28: dwrite f1 [0,4194304] 0 2024-02-15T18:35:18.043 INFO:tasks.workunit.client.1.smithi121.stdout:3/77: getdents d2/d1a 0 2024-02-15T18:35:18.045 INFO:tasks.workunit.client.1.smithi121.stdout:3/78: symlink d2/d1a/l21 0 2024-02-15T18:35:18.046 INFO:tasks.workunit.client.1.smithi121.stdout:3/79: mkdir d2/d1a/d22 0 2024-02-15T18:35:18.049 INFO:tasks.workunit.client.1.smithi121.stdout:3/80: write d2/f7 [6081268,78792] 0 2024-02-15T18:35:18.064 INFO:tasks.workunit.client.1.smithi121.stdout:4/29: dwrite f1 [0,4194304] 0 2024-02-15T18:35:18.065 INFO:tasks.workunit.client.1.smithi121.stdout:3/81: dread d2/fd [0,4194304] 0 2024-02-15T18:35:18.065 INFO:tasks.workunit.client.1.smithi121.stdout:3/82: chown l0 2303806 1 2024-02-15T18:35:18.070 INFO:tasks.workunit.client.1.smithi121.stdout:4/30: mknod c4 0 2024-02-15T18:35:18.070 INFO:tasks.workunit.client.1.smithi121.stdout:3/83: mknod d2/d1a/d22/c23 0 2024-02-15T18:35:18.082 INFO:tasks.workunit.client.1.smithi121.stdout:4/31: dread f1 [0,4194304] 0 2024-02-15T18:35:18.083 INFO:tasks.workunit.client.1.smithi121.stdout:4/32: mkdir d5 0 2024-02-15T18:35:18.084 INFO:tasks.workunit.client.1.smithi121.stdout:4/33: chown c3 3052 1 2024-02-15T18:35:18.086 INFO:tasks.workunit.client.1.smithi121.stdout:4/34: creat d5/f6 x:0 0 0 2024-02-15T18:35:18.086 INFO:tasks.workunit.client.1.smithi121.stdout:4/35: write f1 [3611334,56491] 0 2024-02-15T18:35:18.089 INFO:tasks.workunit.client.1.smithi121.stdout:4/36: mkdir d5/d7 0 2024-02-15T18:35:18.089 INFO:tasks.workunit.client.1.smithi121.stdout:4/37: read - d5/f6 zero size 2024-02-15T18:35:18.090 INFO:tasks.workunit.client.1.smithi121.stdout:3/84: dwrite d2/d15/f1b [0,4194304] 0 2024-02-15T18:35:18.090 INFO:tasks.workunit.client.1.smithi121.stdout:3/85: write d2/d15/f19 [586199,106158] 0 2024-02-15T18:35:18.090 INFO:tasks.workunit.client.1.smithi121.stdout:3/86: fsync d2/fd 0 2024-02-15T18:35:18.091 INFO:tasks.workunit.client.1.smithi121.stdout:3/87: write d2/fe [148806,94539] 0 2024-02-15T18:35:18.094 INFO:tasks.workunit.client.1.smithi121.stdout:4/38: creat d5/d7/f8 x:0 0 0 2024-02-15T18:35:18.099 INFO:tasks.workunit.client.1.smithi121.stdout:4/39: mkdir d5/d9 0 2024-02-15T18:35:18.099 INFO:tasks.workunit.client.1.smithi121.stdout:4/40: truncate d5/d7/f8 215067 0 2024-02-15T18:35:18.102 INFO:tasks.workunit.client.1.smithi121.stdout:4/41: creat d5/fa x:0 0 0 2024-02-15T18:35:18.102 INFO:tasks.workunit.client.1.smithi121.stdout:4/42: write d5/fa [395461,18337] 0 2024-02-15T18:35:18.105 INFO:tasks.workunit.client.1.smithi121.stdout:3/88: dread d2/f7 [0,4194304] 0 2024-02-15T18:35:18.110 INFO:tasks.workunit.client.1.smithi121.stdout:3/89: symlink d2/d1a/l24 0 2024-02-15T18:35:18.115 INFO:tasks.workunit.client.1.smithi121.stdout:4/43: dread f1 [0,4194304] 0 2024-02-15T18:35:18.128 INFO:tasks.workunit.client.1.smithi121.stdout:3/90: mkdir d2/d15/d25 0 2024-02-15T18:35:18.147 INFO:tasks.workunit.client.1.smithi121.stdout:4/44: creat d5/d7/fb x:0 0 0 2024-02-15T18:35:18.147 INFO:tasks.workunit.client.1.smithi121.stdout:4/45: fdatasync f1 0 2024-02-15T18:35:18.147 INFO:tasks.workunit.client.1.smithi121.stdout:3/91: write d2/d1a/f1f [600229,45552] 0 2024-02-15T18:35:18.147 INFO:tasks.workunit.client.1.smithi121.stdout:3/92: fdatasync d2/f11 0 2024-02-15T18:35:18.147 INFO:tasks.workunit.client.1.smithi121.stdout:3/93: read d2/f7 [3132963,68598] 0 2024-02-15T18:35:18.149 INFO:tasks.workunit.client.1.smithi121.stdout:4/46: mknod d5/cc 0 2024-02-15T18:35:18.150 INFO:tasks.workunit.client.1.smithi121.stdout:4/47: read f1 [2539720,53114] 0 2024-02-15T18:35:18.150 INFO:tasks.workunit.client.1.smithi121.stdout:4/48: dread d5/d7/f8 [0,4194304] 0 2024-02-15T18:35:18.150 INFO:tasks.workunit.client.1.smithi121.stdout:4/49: chown d5/f6 3 1 2024-02-15T18:35:18.154 INFO:tasks.workunit.client.1.smithi121.stdout:4/50: symlink d5/d9/ld 0 2024-02-15T18:35:18.154 INFO:tasks.workunit.client.1.smithi121.stdout:4/51: chown d5/f6 43541706 1 2024-02-15T18:35:18.155 INFO:tasks.workunit.client.1.smithi121.stdout:4/52: stat d5/d9 0 2024-02-15T18:35:18.155 INFO:tasks.workunit.client.1.smithi121.stdout:4/53: dread - d5/d7/fb zero size 2024-02-15T18:35:18.157 INFO:tasks.workunit.client.1.smithi121.stdout:4/54: mkdir d5/d9/de 0 2024-02-15T18:35:18.157 INFO:tasks.workunit.client.1.smithi121.stdout:3/94: fsync d2/d15/f19 0 2024-02-15T18:35:18.161 INFO:tasks.workunit.client.1.smithi121.stdout:3/95: getdents d2 0 2024-02-15T18:35:18.182 INFO:tasks.workunit.client.1.smithi121.stdout:4/55: dwrite d5/d7/fb [0,4194304] 0 2024-02-15T18:35:18.182 INFO:tasks.workunit.client.1.smithi121.stdout:4/56: write d5/fa [918839,49919] 0 2024-02-15T18:35:18.185 INFO:tasks.workunit.client.1.smithi121.stdout:3/96: dwrite d2/d15/f19 [0,4194304] 0 2024-02-15T18:35:18.193 INFO:tasks.workunit.client.1.smithi121.stdout:3/97: rename d2/d15 to d2/d1a/d22/d26 0 2024-02-15T18:35:18.196 INFO:tasks.workunit.client.1.smithi121.stdout:3/98: mkdir d2/d1a/d27 0 2024-02-15T18:35:18.201 INFO:tasks.workunit.client.1.smithi121.stdout:3/99: mknod d2/d1a/d22/c28 0 2024-02-15T18:35:18.202 INFO:tasks.workunit.client.1.smithi121.stdout:3/100: creat d2/d1a/d27/f29 x:0 0 0 2024-02-15T18:35:18.203 INFO:tasks.workunit.client.1.smithi121.stdout:3/101: creat d2/f2a x:0 0 0 2024-02-15T18:35:18.205 INFO:tasks.workunit.client.1.smithi121.stdout:4/57: dwrite d5/fa [0,4194304] 0 2024-02-15T18:35:18.205 INFO:tasks.workunit.client.1.smithi121.stdout:3/102: creat d2/d1a/d22/f2b x:0 0 0 2024-02-15T18:35:18.218 INFO:tasks.workunit.client.1.smithi121.stdout:3/103: dread d2/f9 [0,4194304] 0 2024-02-15T18:35:18.220 INFO:tasks.workunit.client.1.smithi121.stdout:3/104: truncate d2/f11 3057963 0 2024-02-15T18:35:18.220 INFO:tasks.workunit.client.1.smithi121.stdout:3/105: chown d2/d1a/d22/d26/f1b 0 1 2024-02-15T18:35:18.221 INFO:tasks.workunit.client.1.smithi121.stdout:3/106: getdents d2/d1a 0 2024-02-15T18:35:18.224 INFO:tasks.workunit.client.1.smithi121.stdout:4/58: dwrite d5/fa [0,4194304] 0 2024-02-15T18:35:18.228 INFO:tasks.workunit.client.1.smithi121.stdout:4/59: creat d5/ff x:0 0 0 2024-02-15T18:35:18.228 INFO:tasks.workunit.client.1.smithi121.stdout:4/60: readlink d5/d9/ld 0 2024-02-15T18:35:18.237 INFO:tasks.workunit.client.1.smithi121.stdout:4/61: dread d5/d7/fb [0,4194304] 0 2024-02-15T18:35:18.238 INFO:tasks.workunit.client.1.smithi121.stdout:4/62: creat d5/f10 x:0 0 0 2024-02-15T18:35:18.239 INFO:tasks.workunit.client.1.smithi121.stdout:3/107: dwrite d2/d1a/d27/f29 [0,4194304] 0 2024-02-15T18:35:18.239 INFO:tasks.workunit.client.1.smithi121.stdout:4/63: dread d5/d7/f8 [0,4194304] 0 2024-02-15T18:35:18.239 INFO:tasks.workunit.client.1.smithi121.stdout:4/64: write d5/f10 [900430,91210] 0 2024-02-15T18:35:18.244 INFO:tasks.workunit.client.1.smithi121.stdout:3/108: rename d2/d1a/d22/d26 to d2/d1a/d27/d2c 0 2024-02-15T18:35:18.245 INFO:tasks.workunit.client.1.smithi121.stdout:3/109: dread - d2/f2a zero size 2024-02-15T18:35:18.245 INFO:tasks.workunit.client.1.smithi121.stdout:3/110: chown d2/d1a/f1c 254643 1 2024-02-15T18:35:18.248 INFO:tasks.workunit.client.1.smithi121.stdout:3/111: creat d2/d1a/d27/f2d x:0 0 0 2024-02-15T18:35:18.248 INFO:tasks.workunit.client.1.smithi121.stdout:3/112: chown d2/d1a/d27/f29 4 1 2024-02-15T18:35:18.248 INFO:tasks.workunit.client.1.smithi121.stdout:3/113: write d2/d1a/d27/f2d [951870,109814] 0 2024-02-15T18:35:18.249 INFO:tasks.workunit.client.1.smithi121.stdout:3/114: symlink d2/d1a/d27/d2c/l2e 0 2024-02-15T18:35:18.270 INFO:tasks.workunit.client.1.smithi121.stdout:4/65: dwrite f1 [0,4194304] 0 2024-02-15T18:35:18.271 INFO:tasks.workunit.client.1.smithi121.stdout:3/115: dwrite d2/d1a/d27/d2c/f1b [4194304,4194304] 0 2024-02-15T18:35:18.271 INFO:tasks.workunit.client.1.smithi121.stdout:4/66: creat d5/f11 x:0 0 0 2024-02-15T18:35:18.280 INFO:tasks.workunit.client.1.smithi121.stdout:3/116: getdents d2/d1a/d27/d2c 0 2024-02-15T18:35:18.288 INFO:tasks.workunit.client.1.smithi121.stdout:4/67: dwrite d5/f6 [0,4194304] 0 2024-02-15T18:35:18.293 INFO:tasks.workunit.client.1.smithi121.stdout:3/117: dread d2/d1a/d27/d2c/f19 [0,4194304] 0 2024-02-15T18:35:18.293 INFO:tasks.workunit.client.1.smithi121.stdout:3/118: write d2/f2a [659964,121212] 0 2024-02-15T18:35:18.294 INFO:tasks.workunit.client.1.smithi121.stdout:3/119: write d2/f7 [404264,19166] 0 2024-02-15T18:35:18.294 INFO:tasks.workunit.client.1.smithi121.stdout:3/120: truncate d2/d1a/f1f 916162 0 2024-02-15T18:35:18.297 INFO:tasks.workunit.client.1.smithi121.stdout:3/121: creat d2/d1a/d27/d2c/f2f x:0 0 0 2024-02-15T18:35:18.304 INFO:tasks.workunit.client.1.smithi121.stdout:4/68: dwrite d5/f11 [0,4194304] 0 2024-02-15T18:35:18.308 INFO:tasks.workunit.client.1.smithi121.stdout:4/69: getdents d5/d9 0 2024-02-15T18:35:18.309 INFO:tasks.workunit.client.1.smithi121.stdout:4/70: fdatasync d5/f11 0 2024-02-15T18:35:18.313 INFO:tasks.workunit.client.1.smithi121.stdout:4/71: mknod d5/c12 0 2024-02-15T18:35:18.313 INFO:tasks.workunit.client.1.smithi121.stdout:4/72: write d5/d7/fb [1654702,2279] 0 2024-02-15T18:35:18.321 INFO:tasks.workunit.client.1.smithi121.stdout:3/122: dwrite d2/f2a [0,4194304] 0 2024-02-15T18:35:18.325 INFO:tasks.workunit.client.1.smithi121.stdout:3/123: mkdir d2/d1a/d27/d2c/d25/d30 0 2024-02-15T18:35:18.334 INFO:tasks.workunit.client.1.smithi121.stdout:3/124: mkdir d2/d1a/d27/d2c/d25/d31 0 2024-02-15T18:35:18.334 INFO:tasks.workunit.client.1.smithi121.stdout:3/125: stat d2/d1a/d22/c28 0 2024-02-15T18:35:18.825 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:17.466056+0000 mgr.smithi038.khesbl (mgr.14162) 352 : audit [DBG] from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:35:18.826 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:17.582336+0000 mon.smithi038 (mon.0) 576 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:35:18.826 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:17.585651+0000 mon.smithi038 (mon.0) 577 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:35:18.826 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:35:17.586617+0000 mgr.smithi038.khesbl (mgr.14162) 353 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi038.khesbl) 2024-02-15T18:35:18.826 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:35:17.588768+0000 mgr.smithi038.khesbl (mgr.14162) 354 : cephadm [INF] Failing over to other MGR 2024-02-15T18:35:18.826 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:17.588996+0000 mon.smithi038 (mon.0) 578 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mgr fail", "who": "smithi038.khesbl"}]: dispatch 2024-02-15T18:35:18.826 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:17.592296+0000 mon.smithi038 (mon.0) 579 : cluster [DBG] osdmap e44: 6 total, 6 up, 6 in 2024-02-15T18:35:18.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:17.466056+0000 mgr.smithi038.khesbl (mgr.14162) 352 : audit [DBG] from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:35:18.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:17.582336+0000 mon.smithi038 (mon.0) 576 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' 2024-02-15T18:35:18.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:17.585651+0000 mon.smithi038 (mon.0) 577 : audit [DBG] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:35:18.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:35:17.586617+0000 mgr.smithi038.khesbl (mgr.14162) 353 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi038.khesbl) 2024-02-15T18:35:18.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:35:17.588768+0000 mgr.smithi038.khesbl (mgr.14162) 354 : cephadm [INF] Failing over to other MGR 2024-02-15T18:35:18.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:17.588996+0000 mon.smithi038 (mon.0) 578 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mgr fail", "who": "smithi038.khesbl"}]: dispatch 2024-02-15T18:35:18.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:17.592296+0000 mon.smithi038 (mon.0) 579 : cluster [DBG] osdmap e44: 6 total, 6 up, 6 in 2024-02-15T18:35:18.957 INFO:tasks.workunit.client.1.smithi121.stdout:9/24: sync 2024-02-15T18:35:18.957 INFO:tasks.workunit.client.1.smithi121.stdout:0/28: sync 2024-02-15T18:35:18.957 INFO:tasks.workunit.client.1.smithi121.stdout:5/11: sync 2024-02-15T18:35:18.959 INFO:tasks.workunit.client.1.smithi121.stdout:0/29: symlink d5/d6/lb 0 2024-02-15T18:35:18.960 INFO:tasks.workunit.client.1.smithi121.stdout:9/25: unlink c4 0 2024-02-15T18:35:18.961 INFO:tasks.workunit.client.1.smithi121.stdout:5/12: link d0/c1 d0/c3 0 2024-02-15T18:35:18.964 INFO:tasks.workunit.client.1.smithi121.stdout:0/30: rename d5/c7 to d5/d6/cc 0 2024-02-15T18:35:18.964 INFO:tasks.workunit.client.1.smithi121.stdout:0/31: write d5/f9 [508581,87773] 0 2024-02-15T18:35:18.964 INFO:tasks.workunit.client.1.smithi121.stdout:0/32: fdatasync f3 0 2024-02-15T18:35:18.964 INFO:tasks.workunit.client.1.smithi121.stdout:9/26: rmdir d7 39 2024-02-15T18:35:18.965 INFO:tasks.workunit.client.1.smithi121.stdout:5/13: symlink d0/d2/l4 0 2024-02-15T18:35:18.965 INFO:tasks.workunit.client.1.smithi121.stdout:5/14: write - no filename 2024-02-15T18:35:18.966 INFO:tasks.workunit.client.1.smithi121.stdout:9/27: dread f5 [0,4194304] 0 2024-02-15T18:35:18.967 INFO:tasks.workunit.client.1.smithi121.stdout:0/33: mkdir d5/d6/dd 0 2024-02-15T18:35:18.969 INFO:tasks.workunit.client.1.smithi121.stdout:5/15: creat d0/d2/f5 x:0 0 0 2024-02-15T18:35:18.970 INFO:tasks.workunit.client.1.smithi121.stdout:0/34: truncate f3 1040840 0 2024-02-15T18:35:18.991 INFO:tasks.workunit.client.1.smithi121.stdout:9/28: dwrite f1 [0,4194304] 0 2024-02-15T18:35:19.019 INFO:tasks.workunit.client.1.smithi121.stdout:9/29: getdents d7/d8 0 2024-02-15T18:35:19.019 INFO:tasks.workunit.client.1.smithi121.stdout:9/30: readlink l2 0 2024-02-15T18:35:19.019 INFO:tasks.workunit.client.1.smithi121.stdout:9/31: chown f1 14050378 1 2024-02-15T18:35:19.019 INFO:tasks.workunit.client.1.smithi121.stdout:9/32: readlink l2 0 2024-02-15T18:35:19.020 INFO:tasks.workunit.client.1.smithi121.stdout:9/33: stat d7/c9 0 2024-02-15T18:35:19.020 INFO:tasks.workunit.client.1.smithi121.stdout:5/16: dwrite d0/d2/f5 [0,4194304] 0 2024-02-15T18:35:19.021 INFO:tasks.workunit.client.1.smithi121.stdout:9/34: mknod d7/d8/ca 0 2024-02-15T18:35:19.021 INFO:tasks.workunit.client.1.smithi121.stdout:5/17: mknod d0/c6 0 2024-02-15T18:35:19.021 INFO:tasks.workunit.client.1.smithi121.stdout:5/18: fsync d0/d2/f5 0 2024-02-15T18:35:19.021 INFO:tasks.workunit.client.1.smithi121.stdout:5/19: truncate d0/d2/f5 5059294 0 2024-02-15T18:35:19.021 INFO:tasks.workunit.client.1.smithi121.stdout:5/20: stat d0/d2/f5 0 2024-02-15T18:35:19.021 INFO:tasks.workunit.client.1.smithi121.stdout:5/21: chown d0/c3 317 1 2024-02-15T18:35:19.021 INFO:tasks.workunit.client.1.smithi121.stdout:9/35: rename f1 to d7/fb 0 2024-02-15T18:35:19.021 INFO:tasks.workunit.client.1.smithi121.stdout:0/35: write f3 [730645,111795] 0 2024-02-15T18:35:19.021 INFO:tasks.workunit.client.1.smithi121.stdout:9/36: dread f5 [0,4194304] 0 2024-02-15T18:35:19.021 INFO:tasks.workunit.client.1.smithi121.stdout:0/36: truncate f2 4164636 0 2024-02-15T18:35:19.022 INFO:tasks.workunit.client.1.smithi121.stdout:0/37: rmdir d5/d6 39 2024-02-15T18:35:19.022 INFO:tasks.workunit.client.1.smithi121.stdout:5/22: dread d0/d2/f5 [0,4194304] 0 2024-02-15T18:35:19.022 INFO:tasks.workunit.client.1.smithi121.stdout:0/38: symlink d5/d6/le 0 2024-02-15T18:35:19.025 INFO:tasks.workunit.client.1.smithi121.stdout:5/23: getdents d0/d2 0 2024-02-15T18:35:19.027 INFO:tasks.workunit.client.1.smithi121.stdout:0/39: rename d5/ca to d5/d6/dd/cf 0 2024-02-15T18:35:19.027 INFO:tasks.workunit.client.1.smithi121.stdout:0/40: stat f2 0 2024-02-15T18:35:19.029 INFO:tasks.workunit.client.1.smithi121.stdout:5/24: mknod d0/d2/c7 0 2024-02-15T18:35:19.030 INFO:tasks.workunit.client.1.smithi121.stdout:0/41: creat d5/f10 x:0 0 0 2024-02-15T18:35:19.031 INFO:tasks.workunit.client.1.smithi121.stdout:5/25: link d0/c1 d0/c8 0 2024-02-15T18:35:19.033 INFO:tasks.workunit.client.1.smithi121.stdout:0/42: rename d5/f9 to d5/d6/f11 0 2024-02-15T18:35:19.033 INFO:tasks.workunit.client.1.smithi121.stdout:0/43: write d5/f10 [310236,104762] 0 2024-02-15T18:35:19.033 INFO:tasks.workunit.client.1.smithi121.stdout:0/44: chown d5/d6/lb 1 1 2024-02-15T18:35:19.033 INFO:tasks.workunit.client.1.smithi121.stdout:0/45: chown d5/f10 1243668 1 2024-02-15T18:35:19.038 INFO:tasks.workunit.client.1.smithi121.stdout:0/46: mknod d5/d6/c12 0 2024-02-15T18:35:19.038 INFO:tasks.workunit.client.1.smithi121.stdout:0/47: stat d5/d6/c12 0 2024-02-15T18:35:19.038 INFO:tasks.workunit.client.1.smithi121.stdout:9/37: dwrite d7/fb [0,4194304] 0 2024-02-15T18:35:19.039 INFO:tasks.workunit.client.1.smithi121.stdout:0/48: rmdir d5/d6 39 2024-02-15T18:35:19.041 INFO:tasks.workunit.client.1.smithi121.stdout:9/38: mknod d7/cc 0 2024-02-15T18:35:19.043 INFO:tasks.workunit.client.1.smithi121.stdout:0/49: creat d5/d6/f13 x:0 0 0 2024-02-15T18:35:19.043 INFO:tasks.workunit.client.1.smithi121.stdout:9/39: symlink d7/ld 0 2024-02-15T18:35:19.051 INFO:tasks.workunit.client.1.smithi121.stdout:5/26: truncate d0/d2/f5 2621771 0 2024-02-15T18:35:19.053 INFO:tasks.workunit.client.1.smithi121.stdout:0/50: unlink f4 0 2024-02-15T18:35:19.053 INFO:tasks.workunit.client.1.smithi121.stdout:9/40: creat d7/d8/fe x:0 0 0 2024-02-15T18:35:19.053 INFO:tasks.workunit.client.1.smithi121.stdout:9/41: write f5 [1213665,48749] 0 2024-02-15T18:35:19.063 INFO:tasks.workunit.client.1.smithi121.stdout:0/51: dread f2 [0,4194304] 0 2024-02-15T18:35:19.070 INFO:tasks.workunit.client.1.smithi121.stdout:9/42: symlink d7/d8/lf 0 2024-02-15T18:35:19.071 INFO:tasks.workunit.client.1.smithi121.stdout:0/52: mknod d5/d6/dd/c14 0 2024-02-15T18:35:19.072 INFO:tasks.workunit.client.1.smithi121.stdout:0/53: write d5/f10 [665775,34584] 0 2024-02-15T18:35:19.072 INFO:tasks.workunit.client.1.smithi121.stdout:0/54: write f2 [4907120,52738] 0 2024-02-15T18:35:19.076 INFO:tasks.workunit.client.1.smithi121.stdout:9/43: rename d7/fb to d7/d8/f10 0 2024-02-15T18:35:19.077 INFO:tasks.workunit.client.1.smithi121.stdout:9/44: chown d7/d8/ca 69787 1 2024-02-15T18:35:19.081 INFO:tasks.workunit.client.1.smithi121.stdout:9/45: write d7/d8/f10 [3697387,80081] 0 2024-02-15T18:35:19.081 INFO:tasks.workunit.client.1.smithi121.stdout:9/46: truncate d7/d8/fe 425770 0 2024-02-15T18:35:19.081 INFO:tasks.workunit.client.1.smithi121.stdout:5/27: dwrite d0/d2/f5 [0,4194304] 0 2024-02-15T18:35:19.084 INFO:tasks.workunit.client.1.smithi121.stdout:0/55: symlink d5/d6/l15 0 2024-02-15T18:35:19.093 INFO:tasks.workunit.client.1.smithi121.stdout:9/47: mknod d7/d8/c11 0 2024-02-15T18:35:19.094 INFO:tasks.workunit.client.1.smithi121.stdout:5/28: getdents d0/d2 0 2024-02-15T18:35:19.094 INFO:tasks.workunit.client.1.smithi121.stdout:5/29: rename d0 to d0/d2/d9 22 2024-02-15T18:35:19.106 INFO:tasks.workunit.client.1.smithi121.stdout:5/30: dread d0/d2/f5 [0,4194304] 0 2024-02-15T18:35:19.108 INFO:tasks.workunit.client.1.smithi121.stdout:5/31: mknod d0/ca 0 2024-02-15T18:35:19.112 INFO:tasks.workunit.client.1.smithi121.stdout:0/56: dwrite f3 [0,4194304] 0 2024-02-15T18:35:19.112 INFO:tasks.workunit.client.1.smithi121.stdout:0/57: fdatasync d5/d6/f13 0 2024-02-15T18:35:19.112 INFO:tasks.workunit.client.1.smithi121.stdout:0/58: chown d5/d6/f11 0 1 2024-02-15T18:35:19.124 INFO:tasks.workunit.client.1.smithi121.stdout:9/48: dread d7/d8/f10 [0,4194304] 0 2024-02-15T18:35:19.124 INFO:tasks.workunit.client.1.smithi121.stdout:9/49: chown d7/cc 160 1 2024-02-15T18:35:19.135 INFO:tasks.workunit.client.1.smithi121.stdout:5/32: rmdir d0/d2 39 2024-02-15T18:35:19.135 INFO:tasks.workunit.client.1.smithi121.stdout:5/33: write d0/d2/f5 [1509388,16705] 0 2024-02-15T18:35:19.139 INFO:tasks.workunit.client.1.smithi121.stdout:9/50: mknod d7/c12 0 2024-02-15T18:35:19.140 INFO:tasks.workunit.client.1.smithi121.stdout:5/34: rename d0/d2/f5 to d0/fb 0 2024-02-15T18:35:19.142 INFO:tasks.workunit.client.1.smithi121.stdout:9/51: creat d7/f13 x:0 0 0 2024-02-15T18:35:19.142 INFO:tasks.workunit.client.1.smithi121.stdout:9/52: stat d7/d8/c11 0 2024-02-15T18:35:19.143 INFO:tasks.workunit.client.1.smithi121.stdout:9/53: truncate d7/f13 897333 0 2024-02-15T18:35:19.143 INFO:tasks.workunit.client.1.smithi121.stdout:9/54: stat d7/d8 0 2024-02-15T18:35:19.144 INFO:tasks.workunit.client.1.smithi121.stdout:9/55: symlink d7/l14 0 2024-02-15T18:35:19.159 INFO:tasks.workunit.client.1.smithi121.stdout:5/35: link d0/fb d0/d2/fc 0 2024-02-15T18:35:19.160 INFO:tasks.workunit.client.1.smithi121.stdout:0/59: truncate f3 1204352 0 2024-02-15T18:35:19.160 INFO:tasks.workunit.client.1.smithi121.stdout:0/60: fdatasync d5/d6/f13 0 2024-02-15T18:35:19.160 INFO:tasks.workunit.client.1.smithi121.stdout:5/36: write d0/fb [778820,103418] 0 2024-02-15T18:35:19.162 INFO:tasks.workunit.client.1.smithi121.stdout:5/37: write d0/d2/fc [2956128,7012] 0 2024-02-15T18:35:19.162 INFO:tasks.workunit.client.1.smithi121.stdout:9/56: rmdir d7 39 2024-02-15T18:35:19.180 INFO:tasks.workunit.client.1.smithi121.stdout:0/61: unlink f3 0 2024-02-15T18:35:19.182 INFO:tasks.workunit.client.1.smithi121.stdout:0/62: creat d5/d6/dd/f16 x:0 0 0 2024-02-15T18:35:19.184 INFO:tasks.workunit.client.1.smithi121.stdout:5/38: dwrite d0/d2/fc [4194304,4194304] 0 2024-02-15T18:35:19.192 INFO:tasks.workunit.client.1.smithi121.stdout:0/63: dread d5/d6/f11 [0,4194304] 0 2024-02-15T18:35:19.196 INFO:tasks.workunit.client.1.smithi121.stdout:5/39: truncate d0/d2/fc 3519883 0 2024-02-15T18:35:19.200 INFO:tasks.workunit.client.1.smithi121.stdout:0/64: dread f2 [0,4194304] 0 2024-02-15T18:35:19.203 INFO:tasks.workunit.client.1.smithi121.stdout:0/65: creat d5/d6/dd/f17 x:0 0 0 2024-02-15T18:35:19.203 INFO:tasks.workunit.client.1.smithi121.stdout:0/66: fsync f2 0 2024-02-15T18:35:19.208 INFO:tasks.workunit.client.1.smithi121.stdout:0/67: creat d5/d6/dd/f18 x:0 0 0 2024-02-15T18:35:19.230 INFO:tasks.workunit.client.1.smithi121.stdout:0/68: dwrite d5/d6/dd/f16 [0,4194304] 0 2024-02-15T18:35:19.244 INFO:tasks.workunit.client.1.smithi121.stdout:0/69: dread d5/d6/dd/f16 [0,4194304] 0 2024-02-15T18:35:19.245 INFO:tasks.workunit.client.1.smithi121.stdout:0/70: write d5/d6/f11 [800358,65357] 0 2024-02-15T18:35:19.245 INFO:tasks.workunit.client.1.smithi121.stdout:0/71: chown d5/d6/cc 75102 1 2024-02-15T18:35:19.245 INFO:tasks.workunit.client.1.smithi121.stdout:0/72: stat d5/d6/lb 0 2024-02-15T18:35:19.245 INFO:tasks.workunit.client.1.smithi121.stdout:0/73: dread - d5/d6/dd/f17 zero size 2024-02-15T18:35:19.245 INFO:tasks.workunit.client.1.smithi121.stdout:0/74: mknod d5/d6/dd/c19 0 2024-02-15T18:35:19.246 INFO:tasks.workunit.client.1.smithi121.stdout:0/75: read - d5/d6/f13 zero size 2024-02-15T18:35:19.246 INFO:tasks.workunit.client.1.smithi121.stdout:0/76: readlink d5/d6/le 0 2024-02-15T18:35:19.247 INFO:tasks.workunit.client.1.smithi121.stdout:0/77: rename d5/d6/f13 to d5/d6/f1a 0 2024-02-15T18:35:19.249 INFO:tasks.workunit.client.1.smithi121.stdout:0/78: mknod d5/d6/dd/c1b 0 2024-02-15T18:35:19.249 INFO:tasks.workunit.client.1.smithi121.stdout:0/79: write d5/d6/f1a [586460,83141] 0 2024-02-15T18:35:19.249 INFO:tasks.workunit.client.1.smithi121.stdout:0/80: chown d5/d6/le 5996505 1 2024-02-15T18:35:19.249 INFO:tasks.workunit.client.1.smithi121.stdout:0/81: stat d5/d6/c12 0 2024-02-15T18:35:19.250 INFO:tasks.workunit.client.1.smithi121.stdout:0/82: write d5/d6/f11 [153685,117002] 0 2024-02-15T18:35:19.251 INFO:tasks.workunit.client.1.smithi121.stdout:0/83: creat d5/d6/f1c x:0 0 0 2024-02-15T18:35:19.252 INFO:tasks.workunit.client.1.smithi121.stdout:0/84: rename d5/d6/dd/c19 to d5/d6/c1d 0 2024-02-15T18:35:19.254 INFO:tasks.workunit.client.1.smithi121.stdout:0/85: mknod d5/c1e 0 2024-02-15T18:35:19.256 INFO:tasks.workunit.client.1.smithi121.stdout:0/86: dread d5/f10 [0,4194304] 0 2024-02-15T18:35:19.257 INFO:tasks.workunit.client.1.smithi121.stdout:0/87: creat d5/d6/f1f x:0 0 0 2024-02-15T18:35:19.257 INFO:tasks.workunit.client.1.smithi121.stdout:0/88: fdatasync d5/d6/f1f 0 2024-02-15T18:35:19.257 INFO:tasks.workunit.client.1.smithi121.stdout:0/89: creat d5/f20 x:0 0 0 2024-02-15T18:35:19.258 INFO:tasks.workunit.client.1.smithi121.stdout:0/90: symlink d5/l21 0 2024-02-15T18:35:19.259 INFO:tasks.workunit.client.1.smithi121.stdout:0/91: creat d5/d6/dd/f22 x:0 0 0 2024-02-15T18:35:19.259 INFO:tasks.workunit.client.1.smithi121.stdout:0/92: symlink d5/l23 0 2024-02-15T18:35:19.259 INFO:tasks.workunit.client.1.smithi121.stdout:0/93: fdatasync d5/d6/dd/f17 0 2024-02-15T18:35:19.267 INFO:tasks.workunit.client.1.smithi121.stdout:0/94: rename d5/d6/f1a to d5/d6/f24 0 2024-02-15T18:35:19.274 INFO:tasks.workunit.client.1.smithi121.stdout:0/95: link d5/d6/f11 d5/d6/dd/f25 0 2024-02-15T18:35:19.275 INFO:tasks.workunit.client.1.smithi121.stdout:0/96: symlink d5/d6/dd/l26 0 2024-02-15T18:35:19.284 INFO:tasks.workunit.client.1.smithi121.stdout:0/97: rmdir d5 39 2024-02-15T18:35:19.284 INFO:tasks.workunit.client.1.smithi121.stdout:0/98: chown d5/l21 3 1 2024-02-15T18:35:19.284 INFO:tasks.workunit.client.1.smithi121.stdout:0/99: write d5/d6/dd/f25 [1531796,45894] 0 2024-02-15T18:35:19.287 INFO:tasks.workunit.client.1.smithi121.stdout:0/100: creat d5/f27 x:0 0 0 2024-02-15T18:35:19.287 INFO:tasks.workunit.client.1.smithi121.stdout:0/101: readlink d5/l21 0 2024-02-15T18:35:19.310 INFO:tasks.workunit.client.1.smithi121.stdout:0/102: dwrite d5/d6/f24 [0,4194304] 0 2024-02-15T18:35:19.312 INFO:tasks.workunit.client.1.smithi121.stdout:0/103: symlink d5/d6/dd/l28 0 2024-02-15T18:35:19.327 INFO:tasks.workunit.client.1.smithi121.stdout:0/104: dwrite d5/d6/f1c [0,4194304] 0 2024-02-15T18:35:19.333 INFO:tasks.workunit.client.1.smithi121.stdout:0/105: mknod d5/d6/dd/c29 0 2024-02-15T18:35:19.334 INFO:tasks.workunit.client.1.smithi121.stdout:0/106: mkdir d5/d6/d2a 0 2024-02-15T18:35:19.336 INFO:tasks.workunit.client.1.smithi121.stdout:0/107: link d5/f20 d5/d6/d2a/f2b 0 2024-02-15T18:35:19.336 INFO:tasks.workunit.client.1.smithi121.stdout:0/108: write d5/f10 [90324,30822] 0 2024-02-15T18:35:19.338 INFO:tasks.workunit.client.1.smithi121.stdout:0/109: link d5/d6/f1f d5/d6/dd/f2c 0 2024-02-15T18:35:19.339 INFO:tasks.workunit.client.1.smithi121.stdout:0/110: chown d5/c1e 57848617 1 2024-02-15T18:35:19.340 INFO:tasks.workunit.client.1.smithi121.stdout:0/111: creat d5/d6/dd/f2d x:0 0 0 2024-02-15T18:35:19.340 INFO:tasks.workunit.client.1.smithi121.stdout:0/112: symlink d5/d6/d2a/l2e 0 2024-02-15T18:35:19.341 INFO:tasks.workunit.client.1.smithi121.stdout:0/113: creat d5/d6/d2a/f2f x:0 0 0 2024-02-15T18:35:19.411 INFO:tasks.workunit.client.1.smithi121.stdout:6/46: sync 2024-02-15T18:35:19.411 INFO:tasks.workunit.client.1.smithi121.stdout:1/39: sync 2024-02-15T18:35:19.411 INFO:tasks.workunit.client.1.smithi121.stdout:8/124: sync 2024-02-15T18:35:19.411 INFO:tasks.workunit.client.1.smithi121.stdout:8/125: chown d0/d7/de/d27/d28/f2c 9095 1 2024-02-15T18:35:19.414 INFO:tasks.workunit.client.1.smithi121.stdout:1/40: stat d0/c5 0 2024-02-15T18:35:19.414 INFO:tasks.workunit.client.1.smithi121.stdout:1/41: rename d0/d6 to d0/d6/d10 22 2024-02-15T18:35:19.414 INFO:tasks.workunit.client.1.smithi121.stdout:6/47: symlink d7/dc/d10/l12 0 2024-02-15T18:35:19.416 INFO:tasks.workunit.client.1.smithi121.stdout:8/126: getdents d0 0 2024-02-15T18:35:19.416 INFO:tasks.workunit.client.1.smithi121.stdout:8/127: fdatasync d0/d7/de/f18 0 2024-02-15T18:35:19.416 INFO:tasks.workunit.client.1.smithi121.stdout:1/42: getdents d0/d6/d7 0 2024-02-15T18:35:19.418 INFO:tasks.workunit.client.1.smithi121.stdout:6/48: truncate d7/f8 725993 0 2024-02-15T18:35:19.418 INFO:tasks.workunit.client.1.smithi121.stdout:6/49: stat d7/dc/d10/l12 0 2024-02-15T18:35:19.418 INFO:tasks.workunit.client.1.smithi121.stdout:1/43: write d0/fb [593202,63879] 0 2024-02-15T18:35:19.420 INFO:tasks.workunit.client.1.smithi121.stdout:8/128: mknod d0/d7/d25/c2e 0 2024-02-15T18:35:19.422 INFO:tasks.workunit.client.1.smithi121.stdout:6/50: creat d7/dc/d10/f13 x:0 0 0 2024-02-15T18:35:19.426 INFO:tasks.workunit.client.1.smithi121.stdout:1/44: creat d0/d6/d7/f11 x:0 0 0 2024-02-15T18:35:19.429 INFO:tasks.workunit.client.1.smithi121.stdout:8/129: read d0/f2 [1203036,108544] 0 2024-02-15T18:35:19.430 INFO:tasks.workunit.client.1.smithi121.stdout:1/45: creat d0/d6/f12 x:0 0 0 2024-02-15T18:35:19.437 INFO:tasks.workunit.client.1.smithi121.stdout:8/130: symlink d0/d7/de/d27/l2f 0 2024-02-15T18:35:19.442 INFO:tasks.workunit.client.1.smithi121.stdout:6/51: dwrite d7/f9 [0,4194304] 0 2024-02-15T18:35:19.442 INFO:tasks.workunit.client.1.smithi121.stdout:6/52: fdatasync d7/f8 0 2024-02-15T18:35:19.442 INFO:tasks.workunit.client.1.smithi121.stdout:6/53: write f2 [672536,106631] 0 2024-02-15T18:35:19.446 INFO:tasks.workunit.client.1.smithi121.stdout:1/46: fsync d0/d6/f12 0 2024-02-15T18:35:19.447 INFO:tasks.workunit.client.1.smithi121.stdout:1/47: truncate d0/d6/d7/f11 92240 0 2024-02-15T18:35:19.447 INFO:tasks.workunit.client.1.smithi121.stdout:1/48: write d0/d6/d7/f11 [160162,25796] 0 2024-02-15T18:35:19.447 INFO:tasks.workunit.client.1.smithi121.stdout:8/131: dread d0/d7/fa [0,4194304] 0 2024-02-15T18:35:19.450 INFO:tasks.workunit.client.1.smithi121.stdout:1/49: creat d0/d6/d7/f13 x:0 0 0 2024-02-15T18:35:19.451 INFO:tasks.workunit.client.1.smithi121.stdout:8/132: symlink d0/d7/de/d27/l30 0 2024-02-15T18:35:19.458 INFO:tasks.workunit.client.1.smithi121.stdout:8/133: dread d0/d3/f26 [0,4194304] 0 2024-02-15T18:35:19.461 INFO:tasks.workunit.client.1.smithi121.stdout:8/134: rename d0/d7/cd to d0/d2a/c31 0 2024-02-15T18:35:19.481 INFO:tasks.workunit.client.1.smithi121.stdout:1/50: dwrite d0/fb [0,4194304] 0 2024-02-15T18:35:19.482 INFO:tasks.workunit.client.1.smithi121.stdout:1/51: stat d0/le 0 2024-02-15T18:35:19.483 INFO:tasks.workunit.client.1.smithi121.stdout:8/135: dwrite d0/d7/de/f1c [0,4194304] 0 2024-02-15T18:35:19.483 INFO:tasks.workunit.client.1.smithi121.stdout:8/136: stat d0/d7 0 2024-02-15T18:35:19.483 INFO:tasks.workunit.client.1.smithi121.stdout:8/137: chown d0/l14 4067 1 2024-02-15T18:35:19.483 INFO:tasks.workunit.client.1.smithi121.stdout:1/52: mkdir d0/d6/d7/d9/d14 0 2024-02-15T18:35:19.483 INFO:tasks.workunit.client.1.smithi121.stdout:1/53: readlink d0/l4 0 2024-02-15T18:35:19.483 INFO:tasks.workunit.client.1.smithi121.stdout:1/54: dread - d0/d6/d7/f13 zero size 2024-02-15T18:35:19.483 INFO:tasks.workunit.client.1.smithi121.stdout:1/55: chown d0/fb 16771474 1 2024-02-15T18:35:19.489 INFO:tasks.workunit.client.1.smithi121.stdout:8/138: mknod d0/d7/d25/c32 0 2024-02-15T18:35:19.489 INFO:tasks.workunit.client.1.smithi121.stdout:8/139: fsync d0/d7/fa 0 2024-02-15T18:35:19.490 INFO:tasks.workunit.client.1.smithi121.stdout:1/56: mknod d0/d6/c15 0 2024-02-15T18:35:19.494 INFO:tasks.workunit.client.1.smithi121.stdout:1/57: rename d0/l8 to d0/d6/l16 0 2024-02-15T18:35:19.496 INFO:tasks.workunit.client.1.smithi121.stdout:1/58: write d0/fb [3502834,102276] 0 2024-02-15T18:35:19.496 INFO:tasks.workunit.client.1.smithi121.stdout:1/59: dread - d0/d6/f12 zero size 2024-02-15T18:35:19.503 INFO:tasks.workunit.client.1.smithi121.stdout:8/140: dwrite d0/d7/de/f1a [0,4194304] 0 2024-02-15T18:35:19.506 INFO:tasks.workunit.client.1.smithi121.stdout:8/141: creat d0/d3/d20/f33 x:0 0 0 2024-02-15T18:35:19.509 INFO:tasks.workunit.client.1.smithi121.stdout:1/60: rename d0/d6/l16 to d0/l17 0 2024-02-15T18:35:19.509 INFO:tasks.workunit.client.1.smithi121.stdout:1/61: chown d0/fb 51 1 2024-02-15T18:35:19.513 INFO:tasks.workunit.client.1.smithi121.stdout:8/142: write d0/f8 [1765277,125145] 0 2024-02-15T18:35:19.516 INFO:tasks.workunit.client.1.smithi121.stdout:8/143: mknod d0/c34 0 2024-02-15T18:35:19.517 INFO:tasks.workunit.client.1.smithi121.stdout:1/62: rmdir d0 39 2024-02-15T18:35:19.517 INFO:tasks.workunit.client.1.smithi121.stdout:1/63: truncate d0/d6/d7/f11 810196 0 2024-02-15T18:35:19.524 INFO:tasks.workunit.client.1.smithi121.stdout:1/64: dread d0/d6/d7/f11 [0,4194304] 0 2024-02-15T18:35:19.526 INFO:tasks.workunit.client.1.smithi121.stdout:1/65: creat d0/f18 x:0 0 0 2024-02-15T18:35:19.527 INFO:tasks.workunit.client.1.smithi121.stdout:1/66: creat d0/d6/f19 x:0 0 0 2024-02-15T18:35:19.527 INFO:tasks.workunit.client.1.smithi121.stdout:1/67: fdatasync d0/f18 0 2024-02-15T18:35:19.528 INFO:tasks.workunit.client.1.smithi121.stdout:1/68: creat d0/d6/f1a x:0 0 0 2024-02-15T18:35:19.532 INFO:tasks.workunit.client.1.smithi121.stdout:8/144: dwrite d0/d7/de/f1a [0,4194304] 0 2024-02-15T18:35:19.538 INFO:tasks.workunit.client.1.smithi121.stdout:1/69: rmdir d0 39 2024-02-15T18:35:19.629 INFO:tasks.workunit.client.1.smithi121.stdout:2/21: sync 2024-02-15T18:35:19.629 INFO:tasks.workunit.client.1.smithi121.stdout:9/57: sync 2024-02-15T18:35:19.629 INFO:tasks.workunit.client.1.smithi121.stdout:3/126: sync 2024-02-15T18:35:19.629 INFO:tasks.workunit.client.1.smithi121.stdout:7/102: sync 2024-02-15T18:35:19.629 INFO:tasks.workunit.client.1.smithi121.stdout:4/73: sync 2024-02-15T18:35:19.629 INFO:tasks.workunit.client.1.smithi121.stdout:7/103: read - d0/d5/d11/f20 zero size 2024-02-15T18:35:19.629 INFO:tasks.workunit.client.1.smithi121.stdout:5/40: sync 2024-02-15T18:35:19.631 INFO:tasks.workunit.client.1.smithi121.stdout:5/41: creat d0/d2/fd x:0 0 0 2024-02-15T18:35:19.631 INFO:tasks.workunit.client.1.smithi121.stdout:4/74: write d5/d7/fb [3622352,23080] 0 2024-02-15T18:35:19.631 INFO:tasks.workunit.client.1.smithi121.stdout:2/22: write d5/f8 [3065773,62266] 0 2024-02-15T18:35:19.632 INFO:tasks.workunit.client.1.smithi121.stdout:4/75: rmdir d5/d9 39 2024-02-15T18:35:19.635 INFO:tasks.workunit.client.1.smithi121.stdout:2/23: creat d5/fa x:0 0 0 2024-02-15T18:35:19.635 INFO:tasks.workunit.client.1.smithi121.stdout:5/42: dread d0/fb [0,4194304] 0 2024-02-15T18:35:19.637 INFO:tasks.workunit.client.1.smithi121.stdout:4/76: rename c2 to d5/d9/de/c13 0 2024-02-15T18:35:19.638 INFO:tasks.workunit.client.1.smithi121.stdout:4/77: fsync f1 0 2024-02-15T18:35:19.638 INFO:tasks.workunit.client.1.smithi121.stdout:4/78: chown d5/d7/f8 34368 1 2024-02-15T18:35:19.638 INFO:tasks.workunit.client.1.smithi121.stdout:9/58: dread d7/d8/f10 [0,4194304] 0 2024-02-15T18:35:19.641 INFO:tasks.workunit.client.1.smithi121.stdout:4/79: read d5/f11 [2200323,56787] 0 2024-02-15T18:35:19.646 INFO:tasks.workunit.client.1.smithi121.stdout:3/127: dwrite d2/d1a/f1c [0,4194304] 0 2024-02-15T18:35:19.646 INFO:tasks.workunit.client.1.smithi121.stdout:3/128: stat d2/d1a/d22/c23 0 2024-02-15T18:35:19.647 INFO:tasks.workunit.client.1.smithi121.stdout:4/80: mkdir d5/d7/d14 0 2024-02-15T18:35:19.651 INFO:tasks.workunit.client.1.smithi121.stdout:4/81: write d5/f6 [3108675,35320] 0 2024-02-15T18:35:19.651 INFO:tasks.workunit.client.1.smithi121.stdout:3/129: symlink d2/d1a/d27/d2c/l32 0 2024-02-15T18:35:19.653 INFO:tasks.workunit.client.1.smithi121.stdout:4/82: mknod d5/c15 0 2024-02-15T18:35:19.656 INFO:tasks.workunit.client.1.smithi121.stdout:3/130: rename d2/d1a/l1d to d2/d1a/d27/d2c/d25/d31/l33 0 2024-02-15T18:35:19.657 INFO:tasks.workunit.client.1.smithi121.stdout:2/24: truncate d5/f8 204875 0 2024-02-15T18:35:19.658 INFO:tasks.workunit.client.1.smithi121.stdout:2/25: chown d5/f6 2113140372 1 2024-02-15T18:35:19.659 INFO:tasks.workunit.client.1.smithi121.stdout:4/83: link d5/d9/ld d5/d7/l16 0 2024-02-15T18:35:19.660 INFO:tasks.workunit.client.1.smithi121.stdout:9/59: dwrite d7/d8/f10 [0,4194304] 0 2024-02-15T18:35:19.663 INFO:tasks.workunit.client.1.smithi121.stdout:3/131: dread d2/fe [0,4194304] 0 2024-02-15T18:35:19.666 INFO:tasks.workunit.client.1.smithi121.stdout:2/26: unlink d5/c9 0 2024-02-15T18:35:19.671 INFO:tasks.workunit.client.1.smithi121.stdout:9/60: creat d7/d8/f15 x:0 0 0 2024-02-15T18:35:19.672 INFO:tasks.workunit.client.1.smithi121.stdout:2/27: mkdir d5/db 0 2024-02-15T18:35:19.677 INFO:tasks.workunit.client.1.smithi121.stdout:4/84: truncate d5/d7/fb 2408693 0 2024-02-15T18:35:19.678 INFO:tasks.workunit.client.1.smithi121.stdout:4/85: write d5/f6 [984105,82360] 0 2024-02-15T18:35:19.678 INFO:tasks.workunit.client.1.smithi121.stdout:2/28: creat d5/fc x:0 0 0 2024-02-15T18:35:19.682 INFO:tasks.workunit.client.1.smithi121.stdout:4/86: unlink d5/cc 0 2024-02-15T18:35:19.682 INFO:tasks.workunit.client.1.smithi121.stdout:4/87: chown d5/d7 43395711 1 2024-02-15T18:35:19.683 INFO:tasks.workunit.client.1.smithi121.stdout:2/29: creat d5/fd x:0 0 0 2024-02-15T18:35:19.684 INFO:tasks.workunit.client.1.smithi121.stdout:2/30: read d5/f8 [80463,100460] 0 2024-02-15T18:35:19.684 INFO:tasks.workunit.client.1.smithi121.stdout:2/31: write d5/fa [230819,50539] 0 2024-02-15T18:35:19.684 INFO:tasks.workunit.client.1.smithi121.stdout:2/32: fsync d5/f6 0 2024-02-15T18:35:19.684 INFO:tasks.workunit.client.1.smithi121.stdout:4/88: creat d5/d7/d14/f17 x:0 0 0 2024-02-15T18:35:19.688 INFO:tasks.workunit.client.1.smithi121.stdout:2/33: unlink d5/f6 0 2024-02-15T18:35:19.691 INFO:tasks.workunit.client.1.smithi121.stdout:4/89: mknod d5/d7/c18 0 2024-02-15T18:35:19.692 INFO:tasks.workunit.client.1.smithi121.stdout:2/34: mknod d5/db/ce 0 2024-02-15T18:35:19.692 INFO:tasks.workunit.client.1.smithi121.stdout:2/35: stat f4 0 2024-02-15T18:35:19.694 INFO:tasks.workunit.client.1.smithi121.stdout:4/90: mkdir d5/d9/d19 0 2024-02-15T18:35:19.695 INFO:tasks.workunit.client.1.smithi121.stdout:4/91: mkdir d5/d7/d1a 0 2024-02-15T18:35:19.697 INFO:tasks.workunit.client.1.smithi121.stdout:4/92: rename d5/d9/ld to d5/d9/l1b 0 2024-02-15T18:35:19.700 INFO:tasks.workunit.client.1.smithi121.stdout:4/93: dread d5/d7/f8 [0,4194304] 0 2024-02-15T18:35:19.700 INFO:tasks.workunit.client.1.smithi121.stdout:4/94: stat d5/fa 0 2024-02-15T18:35:19.702 INFO:tasks.workunit.client.1.smithi121.stdout:4/95: symlink d5/d7/l1c 0 2024-02-15T18:35:19.704 INFO:tasks.workunit.client.1.smithi121.stdout:4/96: mkdir d5/d9/d1d 0 2024-02-15T18:35:19.705 INFO:tasks.workunit.client.1.smithi121.stdout:4/97: mkdir d5/d7/d1e 0 2024-02-15T18:35:19.706 INFO:tasks.workunit.client.1.smithi121.stdout:4/98: mknod d5/d7/d14/c1f 0 2024-02-15T18:35:19.710 INFO:tasks.workunit.client.1.smithi121.stdout:4/99: creat d5/d9/f20 x:0 0 0 2024-02-15T18:35:19.715 INFO:tasks.workunit.client.1.smithi121.stdout:4/100: creat d5/d9/de/f21 x:0 0 0 2024-02-15T18:35:19.718 INFO:tasks.workunit.client.1.smithi121.stdout:2/36: dwrite f4 [0,4194304] 0 2024-02-15T18:35:19.722 INFO:tasks.workunit.client.1.smithi121.stdout:2/37: mkdir d5/df 0 2024-02-15T18:35:19.729 INFO:tasks.workunit.client.1.smithi121.stdout:4/101: dwrite d5/fa [4194304,4194304] 0 2024-02-15T18:35:19.761 INFO:tasks.workunit.client.1.smithi121.stdout:0/114: sync 2024-02-15T18:35:19.761 INFO:tasks.workunit.client.1.smithi121.stdout:0/115: fsync d5/f27 0 2024-02-15T18:35:19.764 INFO:tasks.workunit.client.1.smithi121.stdout:0/116: truncate d5/d6/f11 1229969 0 2024-02-15T18:35:19.764 INFO:tasks.workunit.client.1.smithi121.stdout:0/117: readlink d5/l23 0 2024-02-15T18:35:19.764 INFO:tasks.workunit.client.1.smithi121.stdout:0/118: readlink d5/d6/dd/l28 0 2024-02-15T18:35:19.766 INFO:tasks.workunit.client.1.smithi121.stdout:0/119: mkdir d5/d30 0 2024-02-15T18:35:19.766 INFO:tasks.workunit.client.1.smithi121.stdout:0/120: chown d5/f27 86577 1 2024-02-15T18:35:19.771 INFO:tasks.workunit.client.1.smithi121.stdout:0/121: getdents d5/d6 0 2024-02-15T18:35:19.771 INFO:tasks.workunit.client.1.smithi121.stdout:0/122: dread - d5/d6/dd/f18 zero size 2024-02-15T18:35:19.771 INFO:tasks.workunit.client.1.smithi121.stdout:0/123: dread - d5/f27 zero size 2024-02-15T18:35:19.771 INFO:tasks.workunit.client.1.smithi121.stdout:0/124: truncate d5/d6/f11 1648970 0 2024-02-15T18:35:19.772 INFO:tasks.workunit.client.1.smithi121.stdout:0/125: creat d5/d6/dd/f31 x:0 0 0 2024-02-15T18:35:19.796 INFO:tasks.workunit.client.1.smithi121.stdout:0/126: dwrite d5/d6/dd/f25 [0,4194304] 0 2024-02-15T18:35:19.796 INFO:tasks.workunit.client.1.smithi121.stdout:0/127: write d5/d6/dd/f22 [812864,92752] 0 2024-02-15T18:35:19.798 INFO:tasks.workunit.client.1.smithi121.stdout:0/128: creat d5/d6/dd/f32 x:0 0 0 2024-02-15T18:35:19.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.585789+0000 mon.smithi038 (mon.0) 580 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "mgr fail", "who": "smithi038.khesbl"}]': finished 2024-02-15T18:35:19.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:18.585836+0000 mon.smithi038 (mon.0) 581 : cluster [DBG] mgrmap e20: smithi121.ckiryi(active, starting, since 0.996155s) 2024-02-15T18:35:19.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.586867+0000 mon.smithi121 (mon.1) 24 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-02-15T18:35:19.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.587049+0000 mon.smithi121 (mon.1) 25 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:35:19.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.587176+0000 mon.smithi121 (mon.1) 26 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.hxoggl"}]: dispatch 2024-02-15T18:35:19.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.587268+0000 mon.smithi121 (mon.1) 27 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.qgkzgn"}]: dispatch 2024-02-15T18:35:19.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.587347+0000 mon.smithi121 (mon.1) 28 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.mvumlh"}]: dispatch 2024-02-15T18:35:19.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.587431+0000 mon.smithi121 (mon.1) 29 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.ovuzki"}]: dispatch 2024-02-15T18:35:19.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.587752+0000 mon.smithi121 (mon.1) 30 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mgr metadata", "who": "smithi121.ckiryi", "id": "smithi121.ckiryi"}]: dispatch 2024-02-15T18:35:19.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.587992+0000 mon.smithi121 (mon.1) 31 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:35:19.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.588116+0000 mon.smithi121 (mon.1) 32 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:35:19.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.588247+0000 mon.smithi121 (mon.1) 33 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:35:19.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.588374+0000 mon.smithi121 (mon.1) 34 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:35:19.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.588497+0000 mon.smithi121 (mon.1) 35 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:35:19.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.588615+0000 mon.smithi121 (mon.1) 36 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:35:19.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.590510+0000 mon.smithi121 (mon.1) 37 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-15T18:35:19.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.590647+0000 mon.smithi121 (mon.1) 38 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-15T18:35:19.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.590755+0000 mon.smithi121 (mon.1) 39 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-15T18:35:19.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:18.611948+0000 mon.smithi038 (mon.0) 582 : cluster [INF] Manager daemon smithi121.ckiryi is now available 2024-02-15T18:35:19.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.718873+0000 mon.smithi121 (mon.1) 40 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:35:19.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.720211+0000 mon.smithi121 (mon.1) 41 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:35:19.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.740664+0000 mon.smithi038 (mon.0) 583 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:19.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.762163+0000 mon.smithi121 (mon.1) 42 : audit [INF] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi121.ckiryi/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:35:19.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.762732+0000 mon.smithi038 (mon.0) 584 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi121.ckiryi/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:35:19.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.770529+0000 mon.smithi121 (mon.1) 43 : audit [INF] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi121.ckiryi/trash_purge_schedule"}]: dispatch 2024-02-15T18:35:19.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:18.772653+0000 mon.smithi038 (mon.0) 585 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi121.ckiryi/trash_purge_schedule"}]: dispatch 2024-02-15T18:35:19.956 INFO:tasks.workunit.client.1.smithi121.stdout:6/54: sync 2024-02-15T18:35:19.957 INFO:tasks.workunit.client.1.smithi121.stdout:6/55: write d7/f8 [724789,6264] 0 2024-02-15T18:35:19.962 INFO:tasks.workunit.client.1.smithi121.stdout:6/56: symlink d7/dc/l14 0 2024-02-15T18:35:19.964 INFO:tasks.workunit.client.1.smithi121.stdout:6/57: link d7/f9 d7/dc/f15 0 2024-02-15T18:35:19.965 INFO:tasks.workunit.client.1.smithi121.stdout:6/58: creat d7/f16 x:0 0 0 2024-02-15T18:35:19.967 INFO:tasks.workunit.client.1.smithi121.stdout:6/59: rename d7/dc/fd to d7/dc/f17 0 2024-02-15T18:35:19.967 INFO:tasks.workunit.client.1.smithi121.stdout:6/60: chown d7/dc/f15 15290 1 2024-02-15T18:35:19.968 INFO:tasks.workunit.client.1.smithi121.stdout:6/61: rename d7/f16 to d7/f18 0 2024-02-15T18:35:19.970 INFO:tasks.workunit.client.1.smithi121.stdout:6/62: mknod d7/dc/c19 0 2024-02-15T18:35:19.970 INFO:tasks.workunit.client.1.smithi121.stdout:6/63: fdatasync d7/f18 0 2024-02-15T18:35:19.972 INFO:tasks.workunit.client.1.smithi121.stdout:6/64: creat d7/dc/f1a x:0 0 0 2024-02-15T18:35:19.979 INFO:tasks.workunit.client.1.smithi121.stdout:6/65: unlink d7/dc/f17 0 2024-02-15T18:35:19.983 INFO:tasks.workunit.client.1.smithi121.stdout:6/66: getdents d7/dc/d10 0 2024-02-15T18:35:20.006 INFO:tasks.workunit.client.1.smithi121.stdout:6/67: dwrite d7/f8 [0,4194304] 0 2024-02-15T18:35:20.008 INFO:tasks.workunit.client.1.smithi121.stdout:6/68: mknod d7/dc/c1b 0 2024-02-15T18:35:20.008 INFO:tasks.workunit.client.1.smithi121.stdout:6/69: write d7/dc/f15 [8729840,57198] 0 2024-02-15T18:35:20.008 INFO:tasks.workunit.client.1.smithi121.stdout:6/70: write d7/f11 [1579807,69770] 0 2024-02-15T18:35:20.008 INFO:tasks.workunit.client.1.smithi121.stdout:6/71: chown f0 348526 1 2024-02-15T18:35:20.008 INFO:tasks.workunit.client.1.smithi121.stdout:6/72: chown d7/cb 4541925 1 2024-02-15T18:35:20.010 INFO:tasks.workunit.client.1.smithi121.stdout:6/73: rename f2 to d7/dc/f1c 0 2024-02-15T18:35:20.014 INFO:tasks.workunit.client.1.smithi121.stdout:6/74: unlink d7/dc/f1a 0 2024-02-15T18:35:20.014 INFO:tasks.workunit.client.1.smithi121.stdout:6/75: chown d7/dc/d10/l12 121020 1 2024-02-15T18:35:20.014 INFO:tasks.workunit.client.1.smithi121.stdout:6/76: chown d7/dc/f1c 28410 1 2024-02-15T18:35:20.014 INFO:tasks.workunit.client.1.smithi121.stdout:6/77: chown d7 5 1 2024-02-15T18:35:20.022 INFO:tasks.workunit.client.1.smithi121.stdout:6/78: getdents d7/dc 0 2024-02-15T18:35:20.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.585789+0000 mon.smithi038 (mon.0) 580 : audit [INF] from='mgr.14162 172.21.15.38:0/3102451189' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "mgr fail", "who": "smithi038.khesbl"}]': finished 2024-02-15T18:35:20.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:18.585836+0000 mon.smithi038 (mon.0) 581 : cluster [DBG] mgrmap e20: smithi121.ckiryi(active, starting, since 0.996155s) 2024-02-15T18:35:20.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.586867+0000 mon.smithi121 (mon.1) 24 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-02-15T18:35:20.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.587049+0000 mon.smithi121 (mon.1) 25 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:35:20.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.587176+0000 mon.smithi121 (mon.1) 26 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.hxoggl"}]: dispatch 2024-02-15T18:35:20.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.587268+0000 mon.smithi121 (mon.1) 27 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.qgkzgn"}]: dispatch 2024-02-15T18:35:20.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.587347+0000 mon.smithi121 (mon.1) 28 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.mvumlh"}]: dispatch 2024-02-15T18:35:20.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.587431+0000 mon.smithi121 (mon.1) 29 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.ovuzki"}]: dispatch 2024-02-15T18:35:20.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.587752+0000 mon.smithi121 (mon.1) 30 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mgr metadata", "who": "smithi121.ckiryi", "id": "smithi121.ckiryi"}]: dispatch 2024-02-15T18:35:20.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.587992+0000 mon.smithi121 (mon.1) 31 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:35:20.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.588116+0000 mon.smithi121 (mon.1) 32 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:35:20.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.588247+0000 mon.smithi121 (mon.1) 33 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:35:20.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.588374+0000 mon.smithi121 (mon.1) 34 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:35:20.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.588497+0000 mon.smithi121 (mon.1) 35 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:35:20.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.588615+0000 mon.smithi121 (mon.1) 36 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:35:20.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.590510+0000 mon.smithi121 (mon.1) 37 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-15T18:35:20.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.590647+0000 mon.smithi121 (mon.1) 38 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-15T18:35:20.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.590755+0000 mon.smithi121 (mon.1) 39 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-15T18:35:20.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:18.611948+0000 mon.smithi038 (mon.0) 582 : cluster [INF] Manager daemon smithi121.ckiryi is now available 2024-02-15T18:35:20.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.718873+0000 mon.smithi121 (mon.1) 40 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:35:20.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.720211+0000 mon.smithi121 (mon.1) 41 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:35:20.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.740664+0000 mon.smithi038 (mon.0) 583 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:20.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.762163+0000 mon.smithi121 (mon.1) 42 : audit [INF] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi121.ckiryi/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:35:20.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.762732+0000 mon.smithi038 (mon.0) 584 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi121.ckiryi/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:35:20.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.770529+0000 mon.smithi121 (mon.1) 43 : audit [INF] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi121.ckiryi/trash_purge_schedule"}]: dispatch 2024-02-15T18:35:20.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:18.772653+0000 mon.smithi038 (mon.0) 585 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi121.ckiryi/trash_purge_schedule"}]: dispatch 2024-02-15T18:35:20.026 INFO:tasks.workunit.client.1.smithi121.stdout:6/79: mknod d7/dc/d10/c1d 0 2024-02-15T18:35:20.026 INFO:tasks.workunit.client.1.smithi121.stdout:6/80: write d7/dc/f15 [7843818,32267] 0 2024-02-15T18:35:20.026 INFO:tasks.workunit.client.1.smithi121.stdout:6/81: readlink l6 0 2024-02-15T18:35:20.027 INFO:tasks.workunit.client.1.smithi121.stdout:6/82: write d7/dc/f15 [3426608,59609] 0 2024-02-15T18:35:20.030 INFO:tasks.workunit.client.1.smithi121.stdout:6/83: dread d7/dc/f1c [0,4194304] 0 2024-02-15T18:35:20.031 INFO:tasks.workunit.client.1.smithi121.stdout:6/84: symlink d7/l1e 0 2024-02-15T18:35:20.033 INFO:tasks.workunit.client.1.smithi121.stdout:6/85: creat d7/dc/d10/f1f x:0 0 0 2024-02-15T18:35:20.033 INFO:tasks.workunit.client.1.smithi121.stdout:6/86: readlink d7/l1e 0 2024-02-15T18:35:20.033 INFO:tasks.workunit.client.1.smithi121.stdout:6/87: dread - d7/dc/d10/f13 zero size 2024-02-15T18:35:20.033 INFO:tasks.workunit.client.1.smithi121.stdout:6/88: stat d7/dc/f1c 0 2024-02-15T18:35:20.049 INFO:tasks.workunit.client.1.smithi121.stdout:6/89: dwrite f0 [0,4194304] 0 2024-02-15T18:35:20.049 INFO:tasks.workunit.client.1.smithi121.stdout:6/90: truncate d7/dc/f1c 1005884 0 2024-02-15T18:35:20.050 INFO:tasks.workunit.client.1.smithi121.stdout:6/91: write d7/dc/f1c [1777431,12091] 0 2024-02-15T18:35:20.050 INFO:tasks.workunit.client.1.smithi121.stdout:6/92: dread - d7/dc/d10/f13 zero size 2024-02-15T18:35:20.050 INFO:tasks.workunit.client.1.smithi121.stdout:6/93: creat d7/dc/f20 x:0 0 0 2024-02-15T18:35:20.056 INFO:tasks.workunit.client.1.smithi121.stdout:6/94: truncate d7/f8 3407283 0 2024-02-15T18:35:20.064 INFO:tasks.workunit.client.1.smithi121.stdout:6/95: truncate d7/dc/f1c 865054 0 2024-02-15T18:35:20.067 INFO:tasks.workunit.client.1.smithi121.stdout:6/96: creat d7/dc/f21 x:0 0 0 2024-02-15T18:35:20.067 INFO:tasks.workunit.client.1.smithi121.stdout:6/97: readlink d7/l1e 0 2024-02-15T18:35:20.067 INFO:tasks.workunit.client.1.smithi121.stdout:6/98: read f5 [159071,90492] 0 2024-02-15T18:35:20.068 INFO:tasks.workunit.client.1.smithi121.stdout:6/99: chown d7/f11 19935770 1 2024-02-15T18:35:20.068 INFO:tasks.workunit.client.1.smithi121.stdout:6/100: fdatasync f0 0 2024-02-15T18:35:20.068 INFO:tasks.workunit.client.1.smithi121.stdout:6/101: rmdir d7/dc/d10 39 2024-02-15T18:35:20.086 INFO:tasks.workunit.client.1.smithi121.stdout:6/102: dwrite f0 [0,4194304] 0 2024-02-15T18:35:20.086 INFO:tasks.workunit.client.1.smithi121.stdout:6/103: truncate d7/dc/d10/f13 71080 0 2024-02-15T18:35:20.091 INFO:tasks.workunit.client.1.smithi121.stdout:6/104: getdents d7/dc 0 2024-02-15T18:35:20.091 INFO:tasks.workunit.client.1.smithi121.stdout:6/105: stat d7/dc/f20 0 2024-02-15T18:35:20.092 INFO:tasks.workunit.client.1.smithi121.stdout:6/106: fsync f5 0 2024-02-15T18:35:20.092 INFO:tasks.workunit.client.1.smithi121.stdout:6/107: write d7/dc/f21 [173198,88022] 0 2024-02-15T18:35:20.101 INFO:tasks.workunit.client.1.smithi121.stdout:8/145: sync 2024-02-15T18:35:20.116 INFO:tasks.workunit.client.1.smithi121.stdout:8/146: link d0/d7/de/f2d d0/d7/f35 0 2024-02-15T18:35:20.120 INFO:tasks.workunit.client.1.smithi121.stdout:8/147: mkdir d0/d7/d36 0 2024-02-15T18:35:20.128 INFO:tasks.workunit.client.1.smithi121.stdout:8/148: dread d0/d7/de/f18 [0,4194304] 0 2024-02-15T18:35:20.128 INFO:tasks.workunit.client.1.smithi121.stdout:8/149: chown d0/d3/d20/f33 21 1 2024-02-15T18:35:20.132 INFO:tasks.workunit.client.1.smithi121.stdout:8/150: write d0/d7/de/f1a [2910903,64969] 0 2024-02-15T18:35:20.134 INFO:tasks.workunit.client.1.smithi121.stdout:8/151: link d0/f2 d0/f37 0 2024-02-15T18:35:20.134 INFO:tasks.workunit.client.1.smithi121.stdout:8/152: chown d0/d3/c17 3 1 2024-02-15T18:35:20.134 INFO:tasks.workunit.client.1.smithi121.stdout:8/153: truncate d0/d7/f35 686908 0 2024-02-15T18:35:20.135 INFO:tasks.workunit.client.1.smithi121.stdout:8/154: creat d0/d7/de/d27/f38 x:0 0 0 2024-02-15T18:35:20.135 INFO:tasks.workunit.client.1.smithi121.stdout:8/155: write d0/f13 [712456,60337] 0 2024-02-15T18:35:20.135 INFO:tasks.workunit.client.1.smithi121.stdout:8/156: truncate d0/d3/d20/f33 391942 0 2024-02-15T18:35:20.137 INFO:tasks.workunit.client.1.smithi121.stdout:8/157: mkdir d0/d3/d20/d39 0 2024-02-15T18:35:20.149 INFO:tasks.workunit.client.1.smithi121.stdout:8/158: getdents d0/d3/d20 0 2024-02-15T18:35:20.154 INFO:tasks.workunit.client.1.smithi121.stdout:8/159: fsync d0/d7/de/f1a 0 2024-02-15T18:35:20.159 INFO:tasks.workunit.client.1.smithi121.stdout:8/160: rename d0/d7/de/f18 to d0/d7/de/d27/d28/f3a 0 2024-02-15T18:35:20.161 INFO:tasks.workunit.client.1.smithi121.stdout:8/161: creat d0/d3/d20/d39/f3b x:0 0 0 2024-02-15T18:35:20.161 INFO:tasks.workunit.client.1.smithi121.stdout:8/162: write d0/d7/de/d27/d28/f3a [4243441,27698] 0 2024-02-15T18:35:20.163 INFO:tasks.workunit.client.1.smithi121.stdout:8/163: creat d0/d2a/f3c x:0 0 0 2024-02-15T18:35:20.168 INFO:tasks.workunit.client.1.smithi121.stdout:8/164: dread d0/f8 [0,4194304] 0 2024-02-15T18:35:20.194 INFO:tasks.workunit.client.1.smithi121.stdout:8/165: dwrite d0/f8 [0,4194304] 0 2024-02-15T18:35:20.194 INFO:tasks.workunit.client.1.smithi121.stdout:8/166: dread - d0/d2a/f3c zero size 2024-02-15T18:35:20.242 INFO:tasks.workunit.client.1.smithi121.stdout:1/70: sync 2024-02-15T18:35:20.390 INFO:tasks.workunit.client.1.smithi121.stdout:1/71: rename d0/f18 to d0/d6/d7/f1b 0 2024-02-15T18:35:20.391 INFO:tasks.workunit.client.1.smithi121.stdout:1/72: dread - d0/d6/f19 zero size 2024-02-15T18:35:20.393 INFO:tasks.workunit.client.1.smithi121.stdout:1/73: mknod d0/d6/d7/c1c 0 2024-02-15T18:35:20.393 INFO:tasks.workunit.client.1.smithi121.stdout:1/74: read - d0/d6/d7/f1b zero size 2024-02-15T18:35:20.393 INFO:tasks.workunit.client.1.smithi121.stdout:1/75: chown d0/l17 501020 1 2024-02-15T18:35:20.394 INFO:tasks.workunit.client.1.smithi121.stdout:1/76: rename d0/d6/d7/f11 to d0/f1d 0 2024-02-15T18:35:20.394 INFO:tasks.workunit.client.1.smithi121.stdout:1/77: readlink d0/l4 0 2024-02-15T18:35:20.396 INFO:tasks.workunit.client.1.smithi121.stdout:1/78: creat d0/d6/d7/d9/f1e x:0 0 0 2024-02-15T18:35:20.398 INFO:tasks.workunit.client.1.smithi121.stdout:1/79: creat d0/f1f x:0 0 0 2024-02-15T18:35:20.398 INFO:tasks.workunit.client.1.smithi121.stdout:1/80: chown d0/cf 146577587 1 2024-02-15T18:35:20.399 INFO:tasks.workunit.client.1.smithi121.stdout:1/81: dread d0/f1d [0,4194304] 0 2024-02-15T18:35:20.400 INFO:tasks.workunit.client.1.smithi121.stdout:1/82: readlink d0/l4 0 2024-02-15T18:35:20.400 INFO:tasks.workunit.client.1.smithi121.stdout:1/83: truncate d0/fb 30738 0 2024-02-15T18:35:20.415 INFO:tasks.workunit.client.1.smithi121.stdout:9/61: sync 2024-02-15T18:35:20.415 INFO:tasks.workunit.client.1.smithi121.stdout:7/104: sync 2024-02-15T18:35:20.419 INFO:tasks.workunit.client.1.smithi121.stdout:7/105: creat d0/d5/f26 x:0 0 0 2024-02-15T18:35:20.421 INFO:tasks.workunit.client.1.smithi121.stdout:7/106: rename d0/d5/f22 to d0/f27 0 2024-02-15T18:35:20.422 INFO:tasks.workunit.client.1.smithi121.stdout:7/107: creat d0/d5/d11/d1c/f28 x:0 0 0 2024-02-15T18:35:20.423 INFO:tasks.workunit.client.1.smithi121.stdout:7/108: creat d0/f29 x:0 0 0 2024-02-15T18:35:20.423 INFO:tasks.workunit.client.1.smithi121.stdout:7/109: write d0/d5/d11/d1c/f1f [21561,56833] 0 2024-02-15T18:35:20.426 INFO:tasks.workunit.client.1.smithi121.stdout:7/110: symlink d0/l2a 0 2024-02-15T18:35:20.427 INFO:tasks.workunit.client.1.smithi121.stdout:7/111: creat d0/d5/f2b x:0 0 0 2024-02-15T18:35:20.427 INFO:tasks.workunit.client.1.smithi121.stdout:7/112: write d0/fd [218828,101863] 0 2024-02-15T18:35:20.436 INFO:tasks.workunit.client.1.smithi121.stdout:9/62: dwrite d7/d8/f10 [0,4194304] 0 2024-02-15T18:35:20.436 INFO:tasks.workunit.client.1.smithi121.stdout:9/63: creat d7/d8/f16 x:0 0 0 2024-02-15T18:35:20.436 INFO:tasks.workunit.client.1.smithi121.stdout:9/64: creat d7/d8/f17 x:0 0 0 2024-02-15T18:35:20.436 INFO:tasks.workunit.client.1.smithi121.stdout:9/65: stat d7/ld 0 2024-02-15T18:35:20.436 INFO:tasks.workunit.client.1.smithi121.stdout:7/113: dread d0/fd [0,4194304] 0 2024-02-15T18:35:20.437 INFO:tasks.workunit.client.1.smithi121.stdout:9/66: mkdir d7/d8/d18 0 2024-02-15T18:35:20.448 INFO:tasks.workunit.client.1.smithi121.stdout:7/114: rmdir d0 39 2024-02-15T18:35:20.451 INFO:tasks.workunit.client.1.smithi121.stdout:9/67: dwrite d7/d8/fe [0,4194304] 0 2024-02-15T18:35:20.455 INFO:tasks.workunit.client.1.smithi121.stdout:9/68: creat d7/d8/d18/f19 x:0 0 0 2024-02-15T18:35:20.464 INFO:tasks.workunit.client.1.smithi121.stdout:7/115: dwrite d0/f27 [0,4194304] 0 2024-02-15T18:35:20.465 INFO:tasks.workunit.client.1.smithi121.stdout:7/116: rename d0/d5 to d0/d5/d11/d1c/d2c 22 2024-02-15T18:35:20.465 INFO:tasks.workunit.client.1.smithi121.stdout:7/117: stat d0/f29 0 2024-02-15T18:35:20.465 INFO:tasks.workunit.client.1.smithi121.stdout:7/118: write d0/d5/f9 [1898155,7244] 0 2024-02-15T18:35:20.465 INFO:tasks.workunit.client.1.smithi121.stdout:7/119: stat d0/d5/d11/c13 0 2024-02-15T18:35:20.465 INFO:tasks.workunit.client.1.smithi121.stdout:9/69: truncate d7/d8/f10 3765811 0 2024-02-15T18:35:20.465 INFO:tasks.workunit.client.1.smithi121.stdout:9/70: write d7/d8/f17 [614160,84401] 0 2024-02-15T18:35:20.482 INFO:tasks.workunit.client.1.smithi121.stdout:9/71: dread d7/d8/f10 [0,4194304] 0 2024-02-15T18:35:20.483 INFO:tasks.workunit.client.1.smithi121.stdout:9/72: link d7/c9 d7/d8/c1a 0 2024-02-15T18:35:20.484 INFO:tasks.workunit.client.1.smithi121.stdout:9/73: creat d7/f1b x:0 0 0 2024-02-15T18:35:20.487 INFO:tasks.workunit.client.1.smithi121.stdout:9/74: unlink d7/d8/fe 0 2024-02-15T18:35:20.487 INFO:tasks.workunit.client.1.smithi121.stdout:9/75: write d7/d8/f16 [33864,113528] 0 2024-02-15T18:35:20.526 INFO:tasks.workunit.client.1.smithi121.stdout:3/132: sync 2024-02-15T18:35:20.527 INFO:tasks.workunit.client.1.smithi121.stdout:5/43: sync 2024-02-15T18:35:20.528 INFO:tasks.workunit.client.1.smithi121.stdout:3/133: unlink d2/f11 0 2024-02-15T18:35:20.529 INFO:tasks.workunit.client.1.smithi121.stdout:5/44: dread d0/fb [0,4194304] 0 2024-02-15T18:35:20.529 INFO:tasks.workunit.client.1.smithi121.stdout:5/45: chown d0/d2/fc 6889 1 2024-02-15T18:35:20.529 INFO:tasks.workunit.client.1.smithi121.stdout:5/46: write d0/d2/fc [976296,47565] 0 2024-02-15T18:35:20.530 INFO:tasks.workunit.client.1.smithi121.stdout:3/134: unlink d2/f2a 0 2024-02-15T18:35:20.531 INFO:tasks.workunit.client.1.smithi121.stdout:3/135: write d2/d1a/f1c [2126543,15425] 0 2024-02-15T18:35:20.531 INFO:tasks.workunit.client.1.smithi121.stdout:3/136: read - d2/d1a/d22/f2b zero size 2024-02-15T18:35:20.531 INFO:tasks.workunit.client.1.smithi121.stdout:3/137: fdatasync d2/d1a/d22/f2b 0 2024-02-15T18:35:20.532 INFO:tasks.workunit.client.1.smithi121.stdout:3/138: read d2/fd [41508,71167] 0 2024-02-15T18:35:20.532 INFO:tasks.workunit.client.1.smithi121.stdout:5/47: rename d0/d2/c7 to d0/ce 0 2024-02-15T18:35:20.533 INFO:tasks.workunit.client.1.smithi121.stdout:5/48: read d0/fb [50178,115423] 0 2024-02-15T18:35:20.533 INFO:tasks.workunit.client.1.smithi121.stdout:3/139: getdents d2/d1a/d27/d2c/d25 0 2024-02-15T18:35:20.534 INFO:tasks.workunit.client.1.smithi121.stdout:3/140: write d2/d1a/d27/d2c/f19 [1088793,112436] 0 2024-02-15T18:35:20.534 INFO:tasks.workunit.client.1.smithi121.stdout:3/141: fdatasync d2/d1a/d27/d2c/f2f 0 2024-02-15T18:35:20.536 INFO:tasks.workunit.client.1.smithi121.stdout:3/142: unlink d2/d1a/d27/d2c/l32 0 2024-02-15T18:35:20.536 INFO:tasks.workunit.client.1.smithi121.stdout:5/49: fsync d0/d2/fc 0 2024-02-15T18:35:20.537 INFO:tasks.workunit.client.1.smithi121.stdout:3/143: mkdir d2/d1a/d27/d2c/d25/d31/d34 0 2024-02-15T18:35:20.538 INFO:tasks.workunit.client.1.smithi121.stdout:3/144: write d2/d1a/f1c [124359,55286] 0 2024-02-15T18:35:20.540 INFO:tasks.workunit.client.1.smithi121.stdout:5/50: dread d0/fb [0,4194304] 0 2024-02-15T18:35:20.543 INFO:tasks.workunit.client.1.smithi121.stdout:5/51: dread d0/fb [0,4194304] 0 2024-02-15T18:35:20.544 INFO:tasks.workunit.client.1.smithi121.stdout:5/52: dread - d0/d2/fd zero size 2024-02-15T18:35:20.544 INFO:tasks.workunit.client.1.smithi121.stdout:5/53: mknod d0/cf 0 2024-02-15T18:35:20.545 INFO:tasks.workunit.client.1.smithi121.stdout:5/54: chown d0/ca 6039630 1 2024-02-15T18:35:20.545 INFO:tasks.workunit.client.1.smithi121.stdout:5/55: read d0/d2/fc [994649,6681] 0 2024-02-15T18:35:20.547 INFO:tasks.workunit.client.1.smithi121.stdout:2/38: sync 2024-02-15T18:35:20.552 INFO:tasks.workunit.client.1.smithi121.stdout:3/145: truncate d2/d1a/d27/d2c/f19 3736748 0 2024-02-15T18:35:20.554 INFO:tasks.workunit.client.1.smithi121.stdout:3/146: creat d2/d1a/d22/f35 x:0 0 0 2024-02-15T18:35:20.562 INFO:tasks.workunit.client.1.smithi121.stdout:2/39: dread f4 [0,4194304] 0 2024-02-15T18:35:20.564 INFO:tasks.workunit.client.1.smithi121.stdout:2/40: mkdir d5/df/d10 0 2024-02-15T18:35:20.570 INFO:tasks.workunit.client.1.smithi121.stdout:5/56: dwrite d0/d2/fd [0,4194304] 0 2024-02-15T18:35:20.571 INFO:tasks.workunit.client.1.smithi121.stdout:5/57: write d0/d2/fc [4518089,17616] 0 2024-02-15T18:35:20.576 INFO:tasks.workunit.client.1.smithi121.stdout:2/41: rmdir d5 39 2024-02-15T18:35:20.576 INFO:tasks.workunit.client.1.smithi121.stdout:2/42: chown f4 2241 1 2024-02-15T18:35:20.577 INFO:tasks.workunit.client.1.smithi121.stdout:3/147: dwrite d2/f9 [0,4194304] 0 2024-02-15T18:35:20.579 INFO:tasks.workunit.client.1.smithi121.stdout:2/43: creat d5/df/d10/f11 x:0 0 0 2024-02-15T18:35:20.580 INFO:tasks.workunit.client.1.smithi121.stdout:2/44: chown d5/df 376 1 2024-02-15T18:35:20.585 INFO:tasks.workunit.client.1.smithi121.stdout:2/45: dread f4 [0,4194304] 0 2024-02-15T18:35:20.587 INFO:tasks.workunit.client.1.smithi121.stdout:2/46: getdents d5 0 2024-02-15T18:35:20.590 INFO:tasks.workunit.client.1.smithi121.stdout:5/58: dwrite d0/d2/fc [4194304,4194304] 0 2024-02-15T18:35:20.590 INFO:tasks.workunit.client.1.smithi121.stdout:5/59: stat d0 0 2024-02-15T18:35:20.591 INFO:tasks.workunit.client.1.smithi121.stdout:5/60: getdents d0/d2 0 2024-02-15T18:35:20.595 INFO:tasks.workunit.client.1.smithi121.stdout:5/61: unlink d0/d2/l4 0 2024-02-15T18:35:20.602 INFO:tasks.workunit.client.1.smithi121.stdout:5/62: link d0/ca d0/d2/c10 0 2024-02-15T18:35:20.612 INFO:tasks.workunit.client.1.smithi121.stdout:5/63: dread d0/d2/fd [0,4194304] 0 2024-02-15T18:35:20.612 INFO:tasks.workunit.client.1.smithi121.stdout:5/64: read d0/d2/fc [3310103,57629] 0 2024-02-15T18:35:20.619 INFO:tasks.workunit.client.1.smithi121.stdout:5/65: dread d0/d2/fc [4194304,4194304] 0 2024-02-15T18:35:20.624 INFO:tasks.workunit.client.1.smithi121.stdout:5/66: symlink d0/d2/l11 0 2024-02-15T18:35:20.624 INFO:tasks.workunit.client.1.smithi121.stdout:5/67: chown d0/c1 27 1 2024-02-15T18:35:20.625 INFO:tasks.workunit.client.1.smithi121.stdout:5/68: symlink d0/l12 0 2024-02-15T18:35:20.625 INFO:tasks.workunit.client.1.smithi121.stdout:5/69: fsync d0/d2/fc 0 2024-02-15T18:35:20.748 INFO:tasks.workunit.client.1.smithi121.stdout:0/129: sync 2024-02-15T18:35:20.748 INFO:tasks.workunit.client.1.smithi121.stdout:6/108: sync 2024-02-15T18:35:20.748 INFO:tasks.workunit.client.1.smithi121.stdout:4/102: sync 2024-02-15T18:35:20.749 INFO:tasks.workunit.client.1.smithi121.stdout:6/109: dread d7/dc/d10/f13 [0,4194304] 0 2024-02-15T18:35:20.750 INFO:tasks.workunit.client.1.smithi121.stdout:4/103: truncate d5/f10 385845 0 2024-02-15T18:35:20.750 INFO:tasks.workunit.client.1.smithi121.stdout:4/104: readlink d5/d7/l1c 0 2024-02-15T18:35:20.751 INFO:tasks.workunit.client.1.smithi121.stdout:6/110: mknod d7/dc/c22 0 2024-02-15T18:35:20.754 INFO:tasks.workunit.client.1.smithi121.stdout:4/105: rename d5/ff to d5/d9/d1d/f22 0 2024-02-15T18:35:20.755 INFO:tasks.workunit.client.1.smithi121.stdout:6/111: mknod d7/dc/c23 0 2024-02-15T18:35:20.764 INFO:tasks.workunit.client.1.smithi121.stdout:0/130: dwrite d5/f20 [0,4194304] 0 2024-02-15T18:35:20.776 INFO:tasks.workunit.client.1.smithi121.stdout:4/106: dwrite d5/d9/d1d/f22 [0,4194304] 0 2024-02-15T18:35:20.777 INFO:tasks.workunit.client.1.smithi121.stdout:4/107: fsync d5/d7/d14/f17 0 2024-02-15T18:35:20.779 INFO:tasks.workunit.client.1.smithi121.stdout:6/112: rename f0 to d7/f24 0 2024-02-15T18:35:20.779 INFO:tasks.workunit.client.1.smithi121.stdout:6/113: readlink l6 0 2024-02-15T18:35:20.783 INFO:tasks.workunit.client.1.smithi121.stdout:4/108: rmdir d5/d9/d1d 39 2024-02-15T18:35:20.785 INFO:tasks.workunit.client.1.smithi121.stdout:0/131: dwrite d5/d6/dd/f22 [0,4194304] 0 2024-02-15T18:35:20.786 INFO:tasks.workunit.client.1.smithi121.stdout:6/114: creat d7/dc/f25 x:0 0 0 2024-02-15T18:35:20.791 INFO:tasks.workunit.client.1.smithi121.stdout:0/132: link d5/d6/dd/f16 d5/d30/f33 0 2024-02-15T18:35:20.811 INFO:tasks.workunit.client.1.smithi121.stdout:4/109: dwrite d5/d7/fb [0,4194304] 0 2024-02-15T18:35:20.813 INFO:tasks.workunit.client.1.smithi121.stdout:4/110: creat d5/d7/d1a/f23 x:0 0 0 2024-02-15T18:35:20.816 INFO:tasks.workunit.client.1.smithi121.stdout:6/115: dwrite d7/f11 [0,4194304] 0 2024-02-15T18:35:20.817 INFO:tasks.workunit.client.1.smithi121.stdout:0/133: dwrite d5/f20 [0,4194304] 0 2024-02-15T18:35:20.817 INFO:tasks.workunit.client.1.smithi121.stdout:4/111: unlink d5/d7/f8 0 2024-02-15T18:35:20.819 INFO:tasks.workunit.client.1.smithi121.stdout:4/112: write d5/d9/d1d/f22 [4065616,127932] 0 2024-02-15T18:35:20.822 INFO:tasks.workunit.client.1.smithi121.stdout:0/134: creat d5/d6/dd/f34 x:0 0 0 2024-02-15T18:35:20.833 INFO:tasks.workunit.client.1.smithi121.stdout:4/113: creat d5/d9/d1d/f24 x:0 0 0 2024-02-15T18:35:20.833 INFO:tasks.workunit.client.1.smithi121.stdout:4/114: chown d5/d9/de 6 1 2024-02-15T18:35:20.841 INFO:tasks.workunit.client.1.smithi121.stdout:6/116: dwrite d7/dc/f1c [0,4194304] 0 2024-02-15T18:35:20.844 INFO:tasks.workunit.client.1.smithi121.stdout:6/117: creat d7/dc/d10/f26 x:0 0 0 2024-02-15T18:35:20.847 INFO:tasks.workunit.client.1.smithi121.stdout:0/135: dwrite d5/f27 [0,4194304] 0 2024-02-15T18:35:20.853 INFO:tasks.workunit.client.1.smithi121.stdout:0/136: creat d5/f35 x:0 0 0 2024-02-15T18:35:20.854 INFO:tasks.workunit.client.1.smithi121.stdout:0/137: fdatasync d5/d6/dd/f34 0 2024-02-15T18:35:20.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:20 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:19.603256+0000 mon.smithi038 (mon.0) 586 : cluster [DBG] mgrmap e21: smithi121.ckiryi(active, since 2s) 2024-02-15T18:35:20.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:20 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:19.617437+0000 mgr.smithi121.ckiryi (mgr.24385) 1 : cluster [DBG] pgmap v3: 65 pgs: 65 active+clean; 445 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:35:20.866 INFO:tasks.workunit.client.1.smithi121.stdout:6/118: dwrite d7/dc/f15 [0,4194304] 0 2024-02-15T18:35:20.866 INFO:tasks.workunit.client.1.smithi121.stdout:6/119: chown d7/dc/le 40 1 2024-02-15T18:35:20.875 INFO:tasks.workunit.client.1.smithi121.stdout:0/138: dwrite d5/d6/f1f [0,4194304] 0 2024-02-15T18:35:20.875 INFO:tasks.workunit.client.1.smithi121.stdout:0/139: write d5/d6/d2a/f2b [5180812,53133] 0 2024-02-15T18:35:20.875 INFO:tasks.workunit.client.1.smithi121.stdout:0/140: dread - d5/d6/d2a/f2f zero size 2024-02-15T18:35:20.875 INFO:tasks.workunit.client.1.smithi121.stdout:0/141: truncate d5/d6/dd/f2c 4843746 0 2024-02-15T18:35:20.880 INFO:tasks.workunit.client.1.smithi121.stdout:0/142: mkdir d5/d6/d2a/d36 0 2024-02-15T18:35:20.880 INFO:tasks.workunit.client.1.smithi121.stdout:6/120: dwrite d7/dc/d10/f13 [0,4194304] 0 2024-02-15T18:35:20.880 INFO:tasks.workunit.client.1.smithi121.stdout:6/121: chown d7/dc/d10/f13 13 1 2024-02-15T18:35:20.881 INFO:tasks.workunit.client.1.smithi121.stdout:0/143: read d5/d6/d2a/f2b [2163020,22101] 0 2024-02-15T18:35:20.883 INFO:tasks.workunit.client.1.smithi121.stdout:6/122: symlink d7/l27 0 2024-02-15T18:35:20.891 INFO:tasks.workunit.client.1.smithi121.stdout:0/144: creat d5/f37 x:0 0 0 2024-02-15T18:35:20.897 INFO:tasks.workunit.client.1.smithi121.stdout:6/123: dwrite d7/dc/d10/f26 [0,4194304] 0 2024-02-15T18:35:20.902 INFO:tasks.workunit.client.1.smithi121.stdout:0/145: rmdir d5 39 2024-02-15T18:35:20.902 INFO:tasks.workunit.client.1.smithi121.stdout:0/146: stat f2 0 2024-02-15T18:35:20.902 INFO:tasks.workunit.client.1.smithi121.stdout:0/147: stat d5/d6/dd/f16 0 2024-02-15T18:35:20.903 INFO:tasks.workunit.client.1.smithi121.stdout:6/124: creat d7/f28 x:0 0 0 2024-02-15T18:35:20.903 INFO:tasks.workunit.client.1.smithi121.stdout:6/125: stat d7/dc/c22 0 2024-02-15T18:35:20.907 INFO:tasks.workunit.client.1.smithi121.stdout:6/126: mknod d7/c29 0 2024-02-15T18:35:20.908 INFO:tasks.workunit.client.1.smithi121.stdout:6/127: symlink d7/dc/l2a 0 2024-02-15T18:35:20.909 INFO:tasks.workunit.client.1.smithi121.stdout:6/128: rename d7/dc/l14 to d7/l2b 0 2024-02-15T18:35:20.917 INFO:tasks.workunit.client.1.smithi121.stdout:0/148: dwrite d5/d6/dd/f34 [0,4194304] 0 2024-02-15T18:35:20.919 INFO:tasks.workunit.client.1.smithi121.stdout:0/149: write d5/d6/f1f [648230,99754] 0 2024-02-15T18:35:20.920 INFO:tasks.workunit.client.1.smithi121.stdout:0/150: mknod d5/c38 0 2024-02-15T18:35:20.921 INFO:tasks.workunit.client.1.smithi121.stdout:0/151: mknod d5/d6/d2a/d36/c39 0 2024-02-15T18:35:20.921 INFO:tasks.workunit.client.1.smithi121.stdout:0/152: truncate d5/d6/dd/f17 1014735 0 2024-02-15T18:35:20.921 INFO:tasks.workunit.client.1.smithi121.stdout:0/153: stat d5/d6/lb 0 2024-02-15T18:35:20.922 INFO:tasks.workunit.client.1.smithi121.stdout:6/129: getdents d7/dc 0 2024-02-15T18:35:20.926 INFO:tasks.workunit.client.1.smithi121.stdout:0/154: creat d5/d6/f3a x:0 0 0 2024-02-15T18:35:20.928 INFO:tasks.workunit.client.1.smithi121.stdout:0/155: creat d5/f3b x:0 0 0 2024-02-15T18:35:20.929 INFO:tasks.workunit.client.1.smithi121.stdout:6/130: dread d7/f24 [0,4194304] 0 2024-02-15T18:35:20.930 INFO:tasks.workunit.client.1.smithi121.stdout:0/156: mkdir d5/d30/d3c 0 2024-02-15T18:35:20.931 INFO:tasks.workunit.client.1.smithi121.stdout:6/131: mkdir d7/dc/d2c 0 2024-02-15T18:35:20.932 INFO:tasks.workunit.client.1.smithi121.stdout:0/157: read d5/d6/f11 [1082442,111503] 0 2024-02-15T18:35:20.933 INFO:tasks.workunit.client.1.smithi121.stdout:6/132: creat d7/dc/f2d x:0 0 0 2024-02-15T18:35:20.936 INFO:tasks.workunit.client.1.smithi121.stdout:8/167: sync 2024-02-15T18:35:20.936 INFO:tasks.workunit.client.1.smithi121.stdout:1/84: sync 2024-02-15T18:35:20.936 INFO:tasks.workunit.client.1.smithi121.stdout:7/120: sync 2024-02-15T18:35:20.936 INFO:tasks.workunit.client.1.smithi121.stdout:7/121: readlink d0/l2a 0 2024-02-15T18:35:20.936 INFO:tasks.workunit.client.1.smithi121.stdout:1/85: fdatasync d0/d6/d7/f1b 0 2024-02-15T18:35:20.937 INFO:tasks.workunit.client.1.smithi121.stdout:1/86: readlink d0/l4 0 2024-02-15T18:35:20.937 INFO:tasks.workunit.client.1.smithi121.stdout:1/87: truncate d0/d6/d7/f13 132154 0 2024-02-15T18:35:20.937 INFO:tasks.workunit.client.1.smithi121.stdout:0/158: truncate d5/d6/dd/f22 2828904 0 2024-02-15T18:35:20.937 INFO:tasks.workunit.client.1.smithi121.stdout:8/168: mknod d0/d3/d20/c3d 0 2024-02-15T18:35:20.940 INFO:tasks.workunit.client.1.smithi121.stdout:1/88: rename d0/f1f to d0/d6/d7/d9/f20 0 2024-02-15T18:35:20.940 INFO:tasks.workunit.client.1.smithi121.stdout:1/89: fsync d0/d6/f1a 0 2024-02-15T18:35:20.940 INFO:tasks.workunit.client.1.smithi121.stdout:1/90: fdatasync d0/d6/f19 0 2024-02-15T18:35:20.942 INFO:tasks.workunit.client.1.smithi121.stdout:6/133: dread d7/dc/d10/f26 [0,4194304] 0 2024-02-15T18:35:20.942 INFO:tasks.workunit.client.1.smithi121.stdout:0/159: creat d5/d6/d2a/d36/f3d x:0 0 0 2024-02-15T18:35:20.944 INFO:tasks.workunit.client.1.smithi121.stdout:8/169: creat d0/d7/de/d27/d28/f3e x:0 0 0 2024-02-15T18:35:20.945 INFO:tasks.workunit.client.1.smithi121.stdout:1/91: creat d0/f21 x:0 0 0 2024-02-15T18:35:20.948 INFO:tasks.workunit.client.1.smithi121.stdout:0/160: link d5/f37 d5/d30/f3e 0 2024-02-15T18:35:20.949 INFO:tasks.workunit.client.1.smithi121.stdout:6/134: mknod d7/dc/c2e 0 2024-02-15T18:35:20.949 INFO:tasks.workunit.client.1.smithi121.stdout:6/135: fdatasync d7/dc/f25 0 2024-02-15T18:35:20.952 INFO:tasks.workunit.client.1.smithi121.stdout:8/170: mkdir d0/d7/d3f 0 2024-02-15T18:35:20.953 INFO:tasks.workunit.client.1.smithi121.stdout:0/161: unlink d5/d6/c1d 0 2024-02-15T18:35:20.955 INFO:tasks.workunit.client.1.smithi121.stdout:8/171: truncate d0/f2 1013622 0 2024-02-15T18:35:20.957 INFO:tasks.workunit.client.1.smithi121.stdout:0/162: symlink d5/d30/d3c/l3f 0 2024-02-15T18:35:20.958 INFO:tasks.workunit.client.1.smithi121.stdout:8/172: rmdir d0/d3/d20 39 2024-02-15T18:35:20.967 INFO:tasks.workunit.client.1.smithi121.stdout:7/122: dwrite d0/d5/d11/d1c/f21 [0,4194304] 0 2024-02-15T18:35:20.967 INFO:tasks.workunit.client.1.smithi121.stdout:6/136: dread d7/dc/f1c [0,4194304] 0 2024-02-15T18:35:20.967 INFO:tasks.workunit.client.1.smithi121.stdout:6/137: chown d7/dc/d10/f13 981 1 2024-02-15T18:35:20.968 INFO:tasks.workunit.client.1.smithi121.stdout:7/123: getdents d0/d5/d11 0 2024-02-15T18:35:20.970 INFO:tasks.workunit.client.1.smithi121.stdout:6/138: mknod d7/dc/d10/c2f 0 2024-02-15T18:35:20.971 INFO:tasks.workunit.client.1.smithi121.stdout:7/124: mkdir d0/d2d 0 2024-02-15T18:35:20.971 INFO:tasks.workunit.client.1.smithi121.stdout:8/173: dread d0/f13 [0,4194304] 0 2024-02-15T18:35:20.972 INFO:tasks.workunit.client.1.smithi121.stdout:1/92: dwrite d0/d6/f1a [0,4194304] 0 2024-02-15T18:35:20.974 INFO:tasks.workunit.client.1.smithi121.stdout:6/139: write d7/f8 [483987,68848] 0 2024-02-15T18:35:20.974 INFO:tasks.workunit.client.1.smithi121.stdout:0/163: dwrite d5/d6/f24 [4194304,4194304] 0 2024-02-15T18:35:20.976 INFO:tasks.workunit.client.1.smithi121.stdout:8/174: unlink d0/d7/de/c22 0 2024-02-15T18:35:20.977 INFO:tasks.workunit.client.1.smithi121.stdout:0/164: dread f2 [4194304,4194304] 0 2024-02-15T18:35:20.990 INFO:tasks.workunit.client.1.smithi121.stdout:7/125: write d0/d5/f15 [214571,66354] 0 2024-02-15T18:35:20.990 INFO:tasks.workunit.client.1.smithi121.stdout:7/126: chown d0/d5/d11/c25 3955 1 2024-02-15T18:35:20.993 INFO:tasks.workunit.client.1.smithi121.stdout:6/140: rename d7/dc/d10/c1d to d7/c30 0 2024-02-15T18:35:20.995 INFO:tasks.workunit.client.1.smithi121.stdout:0/165: truncate d5/d6/dd/f34 1240385 0 2024-02-15T18:35:20.995 INFO:tasks.workunit.client.1.smithi121.stdout:0/166: chown d5/f10 207 1 2024-02-15T18:35:20.998 INFO:tasks.workunit.client.1.smithi121.stdout:8/175: dwrite d0/d7/de/d27/d28/f2c [0,4194304] 0 2024-02-15T18:35:20.998 INFO:tasks.workunit.client.1.smithi121.stdout:6/141: creat d7/f31 x:0 0 0 2024-02-15T18:35:20.998 INFO:tasks.workunit.client.1.smithi121.stdout:0/167: unlink d5/d6/dd/f31 0 2024-02-15T18:35:20.999 INFO:tasks.workunit.client.1.smithi121.stdout:1/93: dwrite d0/f21 [0,4194304] 0 2024-02-15T18:35:20.999 INFO:tasks.workunit.client.1.smithi121.stdout:1/94: write d0/d6/f19 [644430,58939] 0 2024-02-15T18:35:20.999 INFO:tasks.workunit.client.1.smithi121.stdout:1/95: fdatasync d0/fb 0 2024-02-15T18:35:20.999 INFO:tasks.workunit.client.1.smithi121.stdout:1/96: chown d0/cf 310 1 2024-02-15T18:35:20.999 INFO:tasks.workunit.client.1.smithi121.stdout:8/176: symlink d0/d7/d3f/l40 0 2024-02-15T18:35:20.999 INFO:tasks.workunit.client.1.smithi121.stdout:8/177: write d0/f5 [514337,46878] 0 2024-02-15T18:35:21.001 INFO:tasks.workunit.client.1.smithi121.stdout:0/168: getdents d5/d6 0 2024-02-15T18:35:21.001 INFO:tasks.workunit.client.1.smithi121.stdout:0/169: stat d5/d6/dd/f34 0 2024-02-15T18:35:21.001 INFO:tasks.workunit.client.1.smithi121.stdout:0/170: truncate d5/d6/f3a 932590 0 2024-02-15T18:35:21.003 INFO:tasks.workunit.client.1.smithi121.stdout:1/97: mknod d0/d6/d7/d9/d14/c22 0 2024-02-15T18:35:21.004 INFO:tasks.workunit.client.1.smithi121.stdout:0/171: rename d5/f35 to d5/f40 0 2024-02-15T18:35:21.004 INFO:tasks.workunit.client.1.smithi121.stdout:1/98: dread d0/fb [0,4194304] 0 2024-02-15T18:35:21.004 INFO:tasks.workunit.client.1.smithi121.stdout:1/99: stat d0/d6/d7/d9/f20 0 2024-02-15T18:35:21.005 INFO:tasks.workunit.client.1.smithi121.stdout:0/172: rmdir d5/d30 39 2024-02-15T18:35:21.006 INFO:tasks.workunit.client.1.smithi121.stdout:8/178: write d0/f2 [978050,33869] 0 2024-02-15T18:35:21.006 INFO:tasks.workunit.client.1.smithi121.stdout:6/142: write d7/f11 [3650678,96154] 0 2024-02-15T18:35:21.006 INFO:tasks.workunit.client.1.smithi121.stdout:0/173: symlink d5/d30/l41 0 2024-02-15T18:35:21.006 INFO:tasks.workunit.client.1.smithi121.stdout:0/174: stat d5/l23 0 2024-02-15T18:35:21.007 INFO:tasks.workunit.client.1.smithi121.stdout:8/179: mknod d0/d7/de/d27/c41 0 2024-02-15T18:35:21.008 INFO:tasks.workunit.client.1.smithi121.stdout:7/127: dwrite d0/f29 [0,4194304] 0 2024-02-15T18:35:21.013 INFO:tasks.workunit.client.1.smithi121.stdout:6/143: creat d7/dc/d2c/f32 x:0 0 0 2024-02-15T18:35:21.015 INFO:tasks.workunit.client.1.smithi121.stdout:8/180: link d0/l24 d0/l42 0 2024-02-15T18:35:21.015 INFO:tasks.workunit.client.1.smithi121.stdout:8/181: write d0/d3/d20/d39/f3b [200188,96007] 0 2024-02-15T18:35:21.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:20 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:19.603256+0000 mon.smithi038 (mon.0) 586 : cluster [DBG] mgrmap e21: smithi121.ckiryi(active, since 2s) 2024-02-15T18:35:21.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:20 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:19.617437+0000 mgr.smithi121.ckiryi (mgr.24385) 1 : cluster [DBG] pgmap v3: 65 pgs: 65 active+clean; 445 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:35:21.025 INFO:tasks.workunit.client.1.smithi121.stdout:0/175: dwrite d5/d6/d2a/f2f [0,4194304] 0 2024-02-15T18:35:21.028 INFO:tasks.workunit.client.1.smithi121.stdout:7/128: rmdir d0/d5 39 2024-02-15T18:35:21.030 INFO:tasks.workunit.client.1.smithi121.stdout:6/144: mknod d7/dc/d10/c33 0 2024-02-15T18:35:21.034 INFO:tasks.workunit.client.1.smithi121.stdout:0/176: mknod d5/d6/d2a/d36/c42 0 2024-02-15T18:35:21.034 INFO:tasks.workunit.client.1.smithi121.stdout:0/177: stat d5/d30 0 2024-02-15T18:35:21.034 INFO:tasks.workunit.client.1.smithi121.stdout:0/178: write d5/f3b [602594,37887] 0 2024-02-15T18:35:21.034 INFO:tasks.workunit.client.1.smithi121.stdout:0/179: chown d5/d6/f24 25978 1 2024-02-15T18:35:21.034 INFO:tasks.workunit.client.1.smithi121.stdout:0/180: fsync d5/f37 0 2024-02-15T18:35:21.035 INFO:tasks.workunit.client.1.smithi121.stdout:0/181: creat d5/d6/f43 x:0 0 0 2024-02-15T18:35:21.036 INFO:tasks.workunit.client.1.smithi121.stdout:0/182: dread - d5/f40 zero size 2024-02-15T18:35:21.037 INFO:tasks.workunit.client.1.smithi121.stdout:6/145: write d7/dc/f15 [4099399,38001] 0 2024-02-15T18:35:21.037 INFO:tasks.workunit.client.1.smithi121.stdout:6/146: dread - d7/f31 zero size 2024-02-15T18:35:21.037 INFO:tasks.workunit.client.1.smithi121.stdout:0/183: mkdir d5/d6/d2a/d36/d44 0 2024-02-15T18:35:21.040 INFO:tasks.workunit.client.1.smithi121.stdout:0/184: link d5/d30/d3c/l3f d5/d6/dd/l45 0 2024-02-15T18:35:21.046 INFO:tasks.workunit.client.1.smithi121.stdout:0/185: dread d5/d6/dd/f16 [0,4194304] 0 2024-02-15T18:35:21.048 INFO:tasks.workunit.client.1.smithi121.stdout:1/100: dwrite d0/d6/f12 [0,4194304] 0 2024-02-15T18:35:21.049 INFO:tasks.workunit.client.1.smithi121.stdout:8/182: dwrite d0/d7/f35 [0,4194304] 0 2024-02-15T18:35:21.049 INFO:tasks.workunit.client.1.smithi121.stdout:8/183: read - d0/d7/de/d27/d28/f3e zero size 2024-02-15T18:35:21.049 INFO:tasks.workunit.client.1.smithi121.stdout:7/129: dwrite d0/d5/f2b [0,4194304] 0 2024-02-15T18:35:21.059 INFO:tasks.workunit.client.1.smithi121.stdout:6/147: rename d7/dc/d10 to d7/dc/d34 0 2024-02-15T18:35:21.059 INFO:tasks.workunit.client.1.smithi121.stdout:6/148: write f5 [1212871,42739] 0 2024-02-15T18:35:21.061 INFO:tasks.workunit.client.1.smithi121.stdout:3/148: sync 2024-02-15T18:35:21.062 INFO:tasks.workunit.client.1.smithi121.stdout:5/70: sync 2024-02-15T18:35:21.062 INFO:tasks.workunit.client.1.smithi121.stdout:9/76: sync 2024-02-15T18:35:21.062 INFO:tasks.workunit.client.1.smithi121.stdout:2/47: sync 2024-02-15T18:35:21.062 INFO:tasks.workunit.client.1.smithi121.stdout:2/48: chown c2 5057 1 2024-02-15T18:35:21.062 INFO:tasks.workunit.client.1.smithi121.stdout:3/149: dread - d2/d1a/d22/f2b zero size 2024-02-15T18:35:21.062 INFO:tasks.workunit.client.1.smithi121.stdout:2/49: write d5/df/d10/f11 [827091,81241] 0 2024-02-15T18:35:21.062 INFO:tasks.workunit.client.1.smithi121.stdout:2/50: chown d5/fc 7069409 1 2024-02-15T18:35:21.069 INFO:tasks.workunit.client.1.smithi121.stdout:1/101: rename d0/d6/d7/f13 to d0/d6/d7/d9/f23 0 2024-02-15T18:35:21.072 INFO:tasks.workunit.client.1.smithi121.stdout:7/130: dwrite d0/d5/f9 [0,4194304] 0 2024-02-15T18:35:21.073 INFO:tasks.workunit.client.1.smithi121.stdout:0/186: dwrite d5/f3b [0,4194304] 0 2024-02-15T18:35:21.073 INFO:tasks.workunit.client.1.smithi121.stdout:6/149: rmdir d7/dc/d2c 39 2024-02-15T18:35:21.074 INFO:tasks.workunit.client.1.smithi121.stdout:5/71: rename d0/c6 to d0/c13 0 2024-02-15T18:35:21.075 INFO:tasks.workunit.client.1.smithi121.stdout:5/72: read d0/d2/fc [5940568,21891] 0 2024-02-15T18:35:21.075 INFO:tasks.workunit.client.1.smithi121.stdout:5/73: fsync d0/d2/fc 0 2024-02-15T18:35:21.075 INFO:tasks.workunit.client.1.smithi121.stdout:5/74: write d0/d2/fc [5102315,13959] 0 2024-02-15T18:35:21.077 INFO:tasks.workunit.client.1.smithi121.stdout:9/77: creat d7/f1c x:0 0 0 2024-02-15T18:35:21.080 INFO:tasks.workunit.client.1.smithi121.stdout:2/51: symlink d5/df/l12 0 2024-02-15T18:35:21.080 INFO:tasks.workunit.client.1.smithi121.stdout:0/187: read d5/d6/dd/f22 [1793555,6971] 0 2024-02-15T18:35:21.081 INFO:tasks.workunit.client.1.smithi121.stdout:2/52: read f4 [421541,128018] 0 2024-02-15T18:35:21.081 INFO:tasks.workunit.client.1.smithi121.stdout:5/75: dread d0/d2/fc [0,4194304] 0 2024-02-15T18:35:21.091 INFO:tasks.workunit.client.1.smithi121.stdout:8/184: rename d0/d7/d25 to d0/d3/d20/d39/d43 0 2024-02-15T18:35:21.091 INFO:tasks.workunit.client.1.smithi121.stdout:8/185: truncate d0/f5 2025748 0 2024-02-15T18:35:21.091 INFO:tasks.workunit.client.1.smithi121.stdout:8/186: write d0/f5 [2671859,34663] 0 2024-02-15T18:35:21.091 INFO:tasks.workunit.client.1.smithi121.stdout:8/187: read d0/d3/d20/d39/f3b [146857,16146] 0 2024-02-15T18:35:21.091 INFO:tasks.workunit.client.1.smithi121.stdout:8/188: readlink d0/l4 0 2024-02-15T18:35:21.091 INFO:tasks.workunit.client.1.smithi121.stdout:1/102: getdents d0/d6 0 2024-02-15T18:35:21.091 INFO:tasks.workunit.client.1.smithi121.stdout:8/189: chown d0/d7/de/d27 154553 1 2024-02-15T18:35:21.091 INFO:tasks.workunit.client.1.smithi121.stdout:1/103: chown d0/d6/d7/d9 13659 1 2024-02-15T18:35:21.096 INFO:tasks.workunit.client.1.smithi121.stdout:7/131: dwrite d0/f27 [4194304,4194304] 0 2024-02-15T18:35:21.096 INFO:tasks.workunit.client.1.smithi121.stdout:9/78: creat d7/d8/f1d x:0 0 0 2024-02-15T18:35:21.097 INFO:tasks.workunit.client.1.smithi121.stdout:2/53: dwrite d5/fd [0,4194304] 0 2024-02-15T18:35:21.109 INFO:tasks.workunit.client.1.smithi121.stdout:0/188: creat d5/d30/d3c/f46 x:0 0 0 2024-02-15T18:35:21.110 INFO:tasks.workunit.client.1.smithi121.stdout:0/189: write d5/d30/d3c/f46 [227484,86706] 0 2024-02-15T18:35:21.110 INFO:tasks.workunit.client.1.smithi121.stdout:5/76: creat d0/d2/f14 x:0 0 0 2024-02-15T18:35:21.113 INFO:tasks.workunit.client.1.smithi121.stdout:3/150: rename d2/d1a/f1c to d2/f36 0 2024-02-15T18:35:21.115 INFO:tasks.workunit.client.1.smithi121.stdout:8/190: symlink d0/d3/d20/d39/l44 0 2024-02-15T18:35:21.119 INFO:tasks.workunit.client.1.smithi121.stdout:9/79: dwrite d7/d8/f10 [0,4194304] 0 2024-02-15T18:35:21.120 INFO:tasks.workunit.client.1.smithi121.stdout:1/104: mknod d0/d6/d7/d9/d14/c24 0 2024-02-15T18:35:21.120 INFO:tasks.workunit.client.1.smithi121.stdout:1/105: write d0/d6/d7/f1b [286497,56089] 0 2024-02-15T18:35:21.121 INFO:tasks.workunit.client.1.smithi121.stdout:7/132: unlink d0/fc 0 2024-02-15T18:35:21.123 INFO:tasks.workunit.client.1.smithi121.stdout:3/151: write d2/d1a/d27/f29 [2443093,19261] 0 2024-02-15T18:35:21.123 INFO:tasks.workunit.client.1.smithi121.stdout:3/152: readlink d2/d1a/d27/d2c/l18 0 2024-02-15T18:35:21.123 INFO:tasks.workunit.client.1.smithi121.stdout:3/153: readlink d2/d1a/l21 0 2024-02-15T18:35:21.123 INFO:tasks.workunit.client.1.smithi121.stdout:0/190: dread d5/f3b [0,4194304] 0 2024-02-15T18:35:21.124 INFO:tasks.workunit.client.1.smithi121.stdout:2/54: mkdir d5/df/d13 0 2024-02-15T18:35:21.124 INFO:tasks.workunit.client.1.smithi121.stdout:3/154: dread d2/fd [0,4194304] 0 2024-02-15T18:35:21.124 INFO:tasks.workunit.client.1.smithi121.stdout:3/155: fsync d2/d1a/d27/d2c/f2f 0 2024-02-15T18:35:21.125 INFO:tasks.workunit.client.1.smithi121.stdout:3/156: truncate d2/d1a/f1f 1243263 0 2024-02-15T18:35:21.125 INFO:tasks.workunit.client.1.smithi121.stdout:3/157: read d2/fd [218769,57688] 0 2024-02-15T18:35:21.125 INFO:tasks.workunit.client.1.smithi121.stdout:5/77: unlink d0/c8 0 2024-02-15T18:35:21.125 INFO:tasks.workunit.client.1.smithi121.stdout:3/158: chown d2/d1a/d22/f2b 6316201 1 2024-02-15T18:35:21.125 INFO:tasks.workunit.client.1.smithi121.stdout:5/78: stat d0/d2/f14 0 2024-02-15T18:35:21.126 INFO:tasks.workunit.client.1.smithi121.stdout:5/79: write d0/fb [4497820,95832] 0 2024-02-15T18:35:21.127 INFO:tasks.workunit.client.1.smithi121.stdout:6/150: rename c4 to d7/dc/d2c/c35 0 2024-02-15T18:35:21.130 INFO:tasks.workunit.client.1.smithi121.stdout:9/80: link d7/l14 d7/l1e 0 2024-02-15T18:35:21.132 INFO:tasks.workunit.client.1.smithi121.stdout:1/106: creat d0/d6/d7/d9/f25 x:0 0 0 2024-02-15T18:35:21.133 INFO:tasks.workunit.client.1.smithi121.stdout:4/115: sync 2024-02-15T18:35:21.134 INFO:tasks.workunit.client.1.smithi121.stdout:8/191: dwrite d0/f13 [0,4194304] 0 2024-02-15T18:35:21.135 INFO:tasks.workunit.client.1.smithi121.stdout:1/107: write d0/d6/f12 [1487423,65358] 0 2024-02-15T18:35:21.135 INFO:tasks.workunit.client.1.smithi121.stdout:1/108: chown d0/d6/d7/cc 1901086804 1 2024-02-15T18:35:21.159 INFO:tasks.workunit.client.1.smithi121.stdout:9/81: dwrite d7/d8/f10 [0,4194304] 0 2024-02-15T18:35:21.162 INFO:tasks.workunit.client.1.smithi121.stdout:7/133: symlink d0/d5/d11/d1c/l2e 0 2024-02-15T18:35:21.162 INFO:tasks.workunit.client.1.smithi121.stdout:0/191: truncate d5/f27 3320036 0 2024-02-15T18:35:21.163 INFO:tasks.workunit.client.1.smithi121.stdout:3/159: creat d2/d1a/f37 x:0 0 0 2024-02-15T18:35:21.166 INFO:tasks.workunit.client.1.smithi121.stdout:5/80: creat d0/f15 x:0 0 0 2024-02-15T18:35:21.166 INFO:tasks.workunit.client.1.smithi121.stdout:4/116: mkdir d5/d7/d1a/d25 0 2024-02-15T18:35:21.171 INFO:tasks.workunit.client.1.smithi121.stdout:8/192: unlink d0/d3/d20/c3d 0 2024-02-15T18:35:21.171 INFO:tasks.workunit.client.1.smithi121.stdout:6/151: mkdir d7/dc/d2c/d36 0 2024-02-15T18:35:21.172 INFO:tasks.workunit.client.1.smithi121.stdout:1/109: link d0/f1d d0/d6/d7/f26 0 2024-02-15T18:35:21.172 INFO:tasks.workunit.client.1.smithi121.stdout:1/110: truncate d0/fb 414334 0 2024-02-15T18:35:21.173 INFO:tasks.workunit.client.1.smithi121.stdout:5/81: mkdir d0/d16 0 2024-02-15T18:35:21.173 INFO:tasks.workunit.client.1.smithi121.stdout:5/82: write d0/d2/fc [8996766,65902] 0 2024-02-15T18:35:21.177 INFO:tasks.workunit.client.1.smithi121.stdout:0/192: dread d5/f20 [0,4194304] 0 2024-02-15T18:35:21.177 INFO:tasks.workunit.client.1.smithi121.stdout:1/111: creat d0/d6/d7/f27 x:0 0 0 2024-02-15T18:35:21.177 INFO:tasks.workunit.client.1.smithi121.stdout:1/112: fdatasync d0/d6/d7/d9/f20 0 2024-02-15T18:35:21.177 INFO:tasks.workunit.client.1.smithi121.stdout:1/113: fdatasync d0/d6/d7/f27 0 2024-02-15T18:35:21.177 INFO:tasks.workunit.client.1.smithi121.stdout:7/134: symlink d0/l2f 0 2024-02-15T18:35:21.177 INFO:tasks.workunit.client.1.smithi121.stdout:6/152: rmdir d7/dc/d34 39 2024-02-15T18:35:21.178 INFO:tasks.workunit.client.1.smithi121.stdout:2/55: link d5/fd d5/f14 0 2024-02-15T18:35:21.179 INFO:tasks.workunit.client.1.smithi121.stdout:5/83: rename d0/d2/fc to d0/f17 0 2024-02-15T18:35:21.182 INFO:tasks.workunit.client.1.smithi121.stdout:2/56: dread d5/fa [0,4194304] 0 2024-02-15T18:35:21.187 INFO:tasks.workunit.client.1.smithi121.stdout:0/193: unlink d5/f37 0 2024-02-15T18:35:21.189 INFO:tasks.workunit.client.1.smithi121.stdout:9/82: dwrite d7/d8/f10 [0,4194304] 0 2024-02-15T18:35:21.190 INFO:tasks.workunit.client.1.smithi121.stdout:0/194: read d5/d6/dd/f16 [551287,92955] 0 2024-02-15T18:35:21.190 INFO:tasks.workunit.client.1.smithi121.stdout:0/195: readlink d5/d6/dd/l26 0 2024-02-15T18:35:21.190 INFO:tasks.workunit.client.1.smithi121.stdout:1/114: link d0/d6/d7/cc d0/c28 0 2024-02-15T18:35:21.190 INFO:tasks.workunit.client.1.smithi121.stdout:7/135: symlink d0/d5/d11/l30 0 2024-02-15T18:35:21.190 INFO:tasks.workunit.client.1.smithi121.stdout:7/136: chown d0/d5/d11/f20 50137 1 2024-02-15T18:35:21.192 INFO:tasks.workunit.client.1.smithi121.stdout:2/57: rmdir d5/df/d13 0 2024-02-15T18:35:21.198 INFO:tasks.workunit.client.1.smithi121.stdout:2/58: write d5/f14 [1159371,41319] 0 2024-02-15T18:35:21.201 INFO:tasks.workunit.client.1.smithi121.stdout:1/115: mkdir d0/d6/d7/d29 0 2024-02-15T18:35:21.201 INFO:tasks.workunit.client.1.smithi121.stdout:9/83: creat d7/f1f x:0 0 0 2024-02-15T18:35:21.202 INFO:tasks.workunit.client.1.smithi121.stdout:6/153: dread d7/dc/f15 [4194304,4194304] 0 2024-02-15T18:35:21.202 INFO:tasks.workunit.client.1.smithi121.stdout:2/59: mkdir d5/df/d10/d15 0 2024-02-15T18:35:21.207 INFO:tasks.workunit.client.1.smithi121.stdout:0/196: dread d5/d6/dd/f34 [0,4194304] 0 2024-02-15T18:35:21.208 INFO:tasks.workunit.client.1.smithi121.stdout:7/137: dread d0/f18 [0,4194304] 0 2024-02-15T18:35:21.208 INFO:tasks.workunit.client.1.smithi121.stdout:3/160: dwrite d2/f7 [0,4194304] 0 2024-02-15T18:35:21.208 INFO:tasks.workunit.client.1.smithi121.stdout:3/161: truncate d2/d1a/f1f 1643192 0 2024-02-15T18:35:21.208 INFO:tasks.workunit.client.1.smithi121.stdout:1/116: write d0/d6/f12 [2757027,129541] 0 2024-02-15T18:35:21.208 INFO:tasks.workunit.client.1.smithi121.stdout:5/84: dread d0/f17 [4194304,4194304] 0 2024-02-15T18:35:21.208 INFO:tasks.workunit.client.1.smithi121.stdout:5/85: dread - d0/d2/f14 zero size 2024-02-15T18:35:21.208 INFO:tasks.workunit.client.1.smithi121.stdout:5/86: chown d0/c13 126430 1 2024-02-15T18:35:21.208 INFO:tasks.workunit.client.1.smithi121.stdout:9/84: symlink d7/l20 0 2024-02-15T18:35:21.215 INFO:tasks.workunit.client.1.smithi121.stdout:6/154: creat d7/dc/f37 x:0 0 0 2024-02-15T18:35:21.216 INFO:tasks.workunit.client.1.smithi121.stdout:2/60: creat d5/df/f16 x:0 0 0 2024-02-15T18:35:21.217 INFO:tasks.workunit.client.1.smithi121.stdout:0/197: creat d5/d6/dd/f47 x:0 0 0 2024-02-15T18:35:21.217 INFO:tasks.workunit.client.1.smithi121.stdout:7/138: truncate d0/d5/d11/d1c/f21 3278039 0 2024-02-15T18:35:21.217 INFO:tasks.workunit.client.1.smithi121.stdout:3/162: dread d2/f9 [0,4194304] 0 2024-02-15T18:35:21.221 INFO:tasks.workunit.client.1.smithi121.stdout:5/87: unlink d0/ca 0 2024-02-15T18:35:21.221 INFO:tasks.workunit.client.1.smithi121.stdout:1/117: rename d0/d6/d7/c1c to d0/d6/d7/c2a 0 2024-02-15T18:35:21.221 INFO:tasks.workunit.client.1.smithi121.stdout:5/88: fdatasync d0/f17 0 2024-02-15T18:35:21.221 INFO:tasks.workunit.client.1.smithi121.stdout:2/61: write f4 [2321522,41907] 0 2024-02-15T18:35:21.221 INFO:tasks.workunit.client.1.smithi121.stdout:2/62: chown d5/fd 22262 1 2024-02-15T18:35:21.221 INFO:tasks.workunit.client.1.smithi121.stdout:2/63: write f4 [4697787,113895] 0 2024-02-15T18:35:21.221 INFO:tasks.workunit.client.1.smithi121.stdout:2/64: write d5/fa [776454,124858] 0 2024-02-15T18:35:21.222 INFO:tasks.workunit.client.1.smithi121.stdout:9/85: rmdir d7/d8 39 2024-02-15T18:35:21.225 INFO:tasks.workunit.client.1.smithi121.stdout:6/155: creat d7/dc/d2c/d36/f38 x:0 0 0 2024-02-15T18:35:21.227 INFO:tasks.workunit.client.1.smithi121.stdout:7/139: rmdir d0/d5 39 2024-02-15T18:35:21.227 INFO:tasks.workunit.client.1.smithi121.stdout:7/140: write d0/d5/d11/d1c/f28 [445751,67695] 0 2024-02-15T18:35:21.233 INFO:tasks.workunit.client.1.smithi121.stdout:0/198: symlink d5/d6/d2a/d36/d44/l48 0 2024-02-15T18:35:21.233 INFO:tasks.workunit.client.1.smithi121.stdout:3/163: creat d2/d1a/d27/d2c/d25/d30/f38 x:0 0 0 2024-02-15T18:35:21.233 INFO:tasks.workunit.client.1.smithi121.stdout:0/199: fdatasync d5/d6/dd/f16 0 2024-02-15T18:35:21.233 INFO:tasks.workunit.client.1.smithi121.stdout:0/200: write d5/d6/f3a [1741938,52645] 0 2024-02-15T18:35:21.235 INFO:tasks.workunit.client.1.smithi121.stdout:6/156: unlink d7/l27 0 2024-02-15T18:35:21.235 INFO:tasks.workunit.client.1.smithi121.stdout:6/157: fdatasync d7/f24 0 2024-02-15T18:35:21.235 INFO:tasks.workunit.client.1.smithi121.stdout:7/141: mknod d0/d5/d11/c31 0 2024-02-15T18:35:21.238 INFO:tasks.workunit.client.1.smithi121.stdout:2/65: getdents d5/df/d10 0 2024-02-15T18:35:21.238 INFO:tasks.workunit.client.1.smithi121.stdout:2/66: fdatasync d5/df/f16 0 2024-02-15T18:35:21.238 INFO:tasks.workunit.client.1.smithi121.stdout:2/67: dread - d5/fc zero size 2024-02-15T18:35:21.241 INFO:tasks.workunit.client.1.smithi121.stdout:3/164: creat d2/d1a/d27/d2c/d25/d30/f39 x:0 0 0 2024-02-15T18:35:21.241 INFO:tasks.workunit.client.1.smithi121.stdout:3/165: write d2/d1a/d27/d2c/d25/d30/f38 [372855,32281] 0 2024-02-15T18:35:21.242 INFO:tasks.workunit.client.1.smithi121.stdout:5/89: mkdir d0/d2/d18 0 2024-02-15T18:35:21.242 INFO:tasks.workunit.client.1.smithi121.stdout:5/90: dread - d0/f15 zero size 2024-02-15T18:35:21.244 INFO:tasks.workunit.client.1.smithi121.stdout:9/86: dwrite d7/f1f [0,4194304] 0 2024-02-15T18:35:21.245 INFO:tasks.workunit.client.1.smithi121.stdout:9/87: truncate d7/d8/d18/f19 1038898 0 2024-02-15T18:35:21.245 INFO:tasks.workunit.client.1.smithi121.stdout:6/158: mknod d7/dc/c39 0 2024-02-15T18:35:21.245 INFO:tasks.workunit.client.1.smithi121.stdout:6/159: fdatasync d7/f18 0 2024-02-15T18:35:21.247 INFO:tasks.workunit.client.1.smithi121.stdout:8/193: dwrite d0/d7/f10 [0,4194304] 0 2024-02-15T18:35:21.249 INFO:tasks.workunit.client.1.smithi121.stdout:7/142: rmdir d0/d2d 0 2024-02-15T18:35:21.249 INFO:tasks.workunit.client.1.smithi121.stdout:3/166: creat d2/d1a/d27/d2c/f3a x:0 0 0 2024-02-15T18:35:21.249 INFO:tasks.workunit.client.1.smithi121.stdout:2/68: rename d5/db to d5/df/d17 0 2024-02-15T18:35:21.250 INFO:tasks.workunit.client.1.smithi121.stdout:1/118: dread d0/d6/f12 [0,4194304] 0 2024-02-15T18:35:21.250 INFO:tasks.workunit.client.1.smithi121.stdout:0/201: truncate d5/d30/f33 2224724 0 2024-02-15T18:35:21.250 INFO:tasks.workunit.client.1.smithi121.stdout:5/91: symlink d0/d16/l19 0 2024-02-15T18:35:21.254 INFO:tasks.workunit.client.1.smithi121.stdout:6/160: symlink d7/l3a 0 2024-02-15T18:35:21.254 INFO:tasks.workunit.client.1.smithi121.stdout:6/161: write d7/dc/f25 [546505,114494] 0 2024-02-15T18:35:21.255 INFO:tasks.workunit.client.1.smithi121.stdout:5/92: dread d0/f17 [4194304,4194304] 0 2024-02-15T18:35:21.256 INFO:tasks.workunit.client.1.smithi121.stdout:3/167: write d2/fe [1196991,74631] 0 2024-02-15T18:35:21.256 INFO:tasks.workunit.client.1.smithi121.stdout:3/168: chown d2/d1a/d22/c23 14117 1 2024-02-15T18:35:21.260 INFO:tasks.workunit.client.1.smithi121.stdout:1/119: truncate d0/f21 1274927 0 2024-02-15T18:35:21.261 INFO:tasks.workunit.client.1.smithi121.stdout:1/120: chown d0/d6/d7/d9/d14/c22 1357 1 2024-02-15T18:35:21.261 INFO:tasks.workunit.client.1.smithi121.stdout:1/121: fsync d0/d6/d7/d9/f23 0 2024-02-15T18:35:21.261 INFO:tasks.workunit.client.1.smithi121.stdout:1/122: chown d0/d6/d7/d29 203 1 2024-02-15T18:35:21.262 INFO:tasks.workunit.client.1.smithi121.stdout:0/202: mknod d5/d6/d2a/d36/c49 0 2024-02-15T18:35:21.262 INFO:tasks.workunit.client.1.smithi121.stdout:0/203: chown d5/d6/dd/f2c 0 1 2024-02-15T18:35:21.266 INFO:tasks.workunit.client.1.smithi121.stdout:2/69: write d5/df/d10/f11 [598667,39735] 0 2024-02-15T18:35:21.266 INFO:tasks.workunit.client.1.smithi121.stdout:2/70: fdatasync d5/f8 0 2024-02-15T18:35:21.269 INFO:tasks.workunit.client.1.smithi121.stdout:6/162: mknod d7/c3b 0 2024-02-15T18:35:21.271 INFO:tasks.workunit.client.1.smithi121.stdout:3/169: mknod d2/d1a/d22/c3b 0 2024-02-15T18:35:21.275 INFO:tasks.workunit.client.1.smithi121.stdout:9/88: dwrite d7/d8/d18/f19 [0,4194304] 0 2024-02-15T18:35:21.278 INFO:tasks.workunit.client.1.smithi121.stdout:1/123: symlink d0/d6/d7/d9/l2b 0 2024-02-15T18:35:21.280 INFO:tasks.workunit.client.1.smithi121.stdout:1/124: write d0/f1d [102848,21552] 0 2024-02-15T18:35:21.283 INFO:tasks.workunit.client.1.smithi121.stdout:0/204: symlink d5/d6/dd/l4a 0 2024-02-15T18:35:21.285 INFO:tasks.workunit.client.1.smithi121.stdout:2/71: dread d5/df/d10/f11 [0,4194304] 0 2024-02-15T18:35:21.285 INFO:tasks.workunit.client.1.smithi121.stdout:2/72: fsync d5/df/f16 0 2024-02-15T18:35:21.285 INFO:tasks.workunit.client.1.smithi121.stdout:2/73: dread - d5/df/f16 zero size 2024-02-15T18:35:21.285 INFO:tasks.workunit.client.1.smithi121.stdout:2/74: chown d5/df/d10/f11 3379418 1 2024-02-15T18:35:21.286 INFO:tasks.workunit.client.1.smithi121.stdout:6/163: link d7/dc/d34/f1f d7/dc/d2c/d36/f3c 0 2024-02-15T18:35:21.286 INFO:tasks.workunit.client.1.smithi121.stdout:6/164: chown d7/c3b 77 1 2024-02-15T18:35:21.286 INFO:tasks.workunit.client.1.smithi121.stdout:3/170: truncate d2/d1a/d27/d2c/f19 3272305 0 2024-02-15T18:35:21.288 INFO:tasks.workunit.client.1.smithi121.stdout:9/89: getdents d7/d8 0 2024-02-15T18:35:21.288 INFO:tasks.workunit.client.1.smithi121.stdout:9/90: fdatasync d7/d8/d18/f19 0 2024-02-15T18:35:21.294 INFO:tasks.workunit.client.1.smithi121.stdout:7/143: truncate d0/f27 6560386 0 2024-02-15T18:35:21.296 INFO:tasks.workunit.client.1.smithi121.stdout:1/125: mknod d0/d6/d7/d9/c2c 0 2024-02-15T18:35:21.296 INFO:tasks.workunit.client.1.smithi121.stdout:1/126: chown d0/d6/f19 472 1 2024-02-15T18:35:21.296 INFO:tasks.workunit.client.1.smithi121.stdout:1/127: readlink d0/d6/d7/d9/l2b 0 2024-02-15T18:35:21.296 INFO:tasks.workunit.client.1.smithi121.stdout:1/128: write d0/d6/f19 [1184213,128676] 0 2024-02-15T18:35:21.296 INFO:tasks.workunit.client.1.smithi121.stdout:7/144: write d0/d5/d11/d1c/f21 [1940464,109708] 0 2024-02-15T18:35:21.297 INFO:tasks.workunit.client.1.smithi121.stdout:7/145: write d0/d5/d11/d1c/f23 [67034,124828] 0 2024-02-15T18:35:21.323 INFO:tasks.workunit.client.1.smithi121.stdout:2/75: mkdir d5/df/d10/d15/d18 0 2024-02-15T18:35:21.323 INFO:tasks.workunit.client.1.smithi121.stdout:2/76: write d5/df/f16 [519220,53767] 0 2024-02-15T18:35:21.323 INFO:tasks.workunit.client.1.smithi121.stdout:6/165: creat d7/dc/d34/f3d x:0 0 0 2024-02-15T18:35:21.323 INFO:tasks.workunit.client.1.smithi121.stdout:2/77: write d5/df/f16 [836635,80258] 0 2024-02-15T18:35:21.324 INFO:tasks.workunit.client.1.smithi121.stdout:2/78: fdatasync f4 0 2024-02-15T18:35:21.324 INFO:tasks.workunit.client.1.smithi121.stdout:6/166: fdatasync d7/dc/f15 0 2024-02-15T18:35:21.324 INFO:tasks.workunit.client.1.smithi121.stdout:3/171: link d2/d1a/d27/d2c/d25/d31/l33 d2/d1a/d22/l3c 0 2024-02-15T18:35:21.324 INFO:tasks.workunit.client.1.smithi121.stdout:3/172: stat d2/d1a/f1f 0 2024-02-15T18:35:21.325 INFO:tasks.workunit.client.1.smithi121.stdout:9/91: rename d7/l1e to d7/d8/d18/l21 0 2024-02-15T18:35:21.325 INFO:tasks.workunit.client.1.smithi121.stdout:9/92: readlink l2 0 2024-02-15T18:35:21.325 INFO:tasks.workunit.client.1.smithi121.stdout:9/93: write f5 [304620,55181] 0 2024-02-15T18:35:21.327 INFO:tasks.workunit.client.1.smithi121.stdout:2/79: write d5/fd [2987602,64073] 0 2024-02-15T18:35:21.328 INFO:tasks.workunit.client.1.smithi121.stdout:1/129: dwrite d0/f1d [0,4194304] 0 2024-02-15T18:35:21.330 INFO:tasks.workunit.client.1.smithi121.stdout:0/205: dwrite d5/d30/f33 [0,4194304] 0 2024-02-15T18:35:21.330 INFO:tasks.workunit.client.1.smithi121.stdout:0/206: fsync d5/d6/dd/f25 0 2024-02-15T18:35:21.335 INFO:tasks.workunit.client.1.smithi121.stdout:5/93: symlink d0/d2/l1a 0 2024-02-15T18:35:21.341 INFO:tasks.workunit.client.1.smithi121.stdout:0/207: dread d5/d6/f11 [0,4194304] 0 2024-02-15T18:35:21.344 INFO:tasks.workunit.client.1.smithi121.stdout:6/167: truncate d7/dc/d34/f13 2161510 0 2024-02-15T18:35:21.345 INFO:tasks.workunit.client.1.smithi121.stdout:3/173: mknod d2/d1a/d27/d2c/d25/d31/c3d 0 2024-02-15T18:35:21.345 INFO:tasks.workunit.client.1.smithi121.stdout:3/174: dread - d2/d1a/f37 zero size 2024-02-15T18:35:21.345 INFO:tasks.workunit.client.1.smithi121.stdout:3/175: write d2/d1a/d27/d2c/d25/d30/f39 [249473,111237] 0 2024-02-15T18:35:21.346 INFO:tasks.workunit.client.1.smithi121.stdout:1/130: dwrite d0/d6/d7/d9/f25 [0,4194304] 0 2024-02-15T18:35:21.348 INFO:tasks.workunit.client.1.smithi121.stdout:9/94: link d7/ld d7/l22 0 2024-02-15T18:35:21.349 INFO:tasks.workunit.client.1.smithi121.stdout:2/80: mkdir d5/df/d10/d15/d18/d19 0 2024-02-15T18:35:21.355 INFO:tasks.workunit.client.1.smithi121.stdout:1/131: dread d0/d6/f1a [0,4194304] 0 2024-02-15T18:35:21.371 INFO:tasks.workunit.client.1.smithi121.stdout:0/208: unlink d5/d6/dd/f18 0 2024-02-15T18:35:21.373 INFO:tasks.workunit.client.1.smithi121.stdout:7/146: mknod d0/c32 0 2024-02-15T18:35:21.373 INFO:tasks.workunit.client.1.smithi121.stdout:6/168: creat d7/dc/d2c/d36/f3e x:0 0 0 2024-02-15T18:35:21.374 INFO:tasks.workunit.client.1.smithi121.stdout:3/176: getdents d2/d1a/d27/d2c/d25/d30 0 2024-02-15T18:35:21.376 INFO:tasks.workunit.client.1.smithi121.stdout:9/95: link d7/d8/d18/f19 d7/f23 0 2024-02-15T18:35:21.377 INFO:tasks.workunit.client.1.smithi121.stdout:3/177: write d2/d1a/d27/f29 [1254667,36748] 0 2024-02-15T18:35:21.377 INFO:tasks.workunit.client.1.smithi121.stdout:2/81: creat d5/df/d10/f1a x:0 0 0 2024-02-15T18:35:21.378 INFO:tasks.workunit.client.1.smithi121.stdout:5/94: mknod d0/d2/d18/c1b 0 2024-02-15T18:35:21.378 INFO:tasks.workunit.client.1.smithi121.stdout:5/95: fsync d0/f17 0 2024-02-15T18:35:21.380 INFO:tasks.workunit.client.1.smithi121.stdout:9/96: rmdir d7/d8/d18 39 2024-02-15T18:35:21.381 INFO:tasks.workunit.client.1.smithi121.stdout:3/178: mknod d2/d1a/d27/d2c/d25/c3e 0 2024-02-15T18:35:21.381 INFO:tasks.workunit.client.1.smithi121.stdout:3/179: write d2/d1a/d27/d2c/f2f [548017,91707] 0 2024-02-15T18:35:21.381 INFO:tasks.workunit.client.1.smithi121.stdout:1/132: dwrite d0/f1d [4194304,4194304] 0 2024-02-15T18:35:21.381 INFO:tasks.workunit.client.1.smithi121.stdout:1/133: chown d0/d6/d7/cc 74 1 2024-02-15T18:35:21.382 INFO:tasks.workunit.client.1.smithi121.stdout:9/97: symlink d7/d8/d18/l24 0 2024-02-15T18:35:21.383 INFO:tasks.workunit.client.1.smithi121.stdout:7/147: dread d0/d5/f9 [0,4194304] 0 2024-02-15T18:35:21.383 INFO:tasks.workunit.client.1.smithi121.stdout:5/96: read d0/d2/fd [2322948,104632] 0 2024-02-15T18:35:21.383 INFO:tasks.workunit.client.1.smithi121.stdout:5/97: readlink d0/d16/l19 0 2024-02-15T18:35:21.383 INFO:tasks.workunit.client.1.smithi121.stdout:3/180: mkdir d2/d1a/d3f 0 2024-02-15T18:35:21.384 INFO:tasks.workunit.client.1.smithi121.stdout:0/209: dread d5/d6/d2a/f2f [0,4194304] 0 2024-02-15T18:35:21.386 INFO:tasks.workunit.client.1.smithi121.stdout:1/134: link d0/d6/d7/d9/f23 d0/d6/d7/d9/f2d 0 2024-02-15T18:35:21.387 INFO:tasks.workunit.client.1.smithi121.stdout:5/98: mknod d0/c1c 0 2024-02-15T18:35:21.390 INFO:tasks.workunit.client.1.smithi121.stdout:3/181: rmdir d2/d1a/d3f 0 2024-02-15T18:35:21.390 INFO:tasks.workunit.client.1.smithi121.stdout:3/182: write d2/d1a/d27/d2c/d25/d30/f38 [559247,21259] 0 2024-02-15T18:35:21.394 INFO:tasks.workunit.client.1.smithi121.stdout:1/135: rename d0/d6/cd to d0/d6/d7/d29/c2e 0 2024-02-15T18:35:21.397 INFO:tasks.workunit.client.1.smithi121.stdout:3/183: link d2/d1a/f1f d2/f40 0 2024-02-15T18:35:21.397 INFO:tasks.workunit.client.1.smithi121.stdout:3/184: getdents d2/d1a/d27/d2c/d25/d31/d34 0 2024-02-15T18:35:21.401 INFO:tasks.workunit.client.1.smithi121.stdout:3/185: write d2/f36 [614979,77118] 0 2024-02-15T18:35:21.404 INFO:tasks.workunit.client.1.smithi121.stdout:6/169: rmdir d7/dc/d2c/d36 39 2024-02-15T18:35:21.409 INFO:tasks.workunit.client.1.smithi121.stdout:3/186: creat d2/d1a/d27/d2c/d25/d30/f41 x:0 0 0 2024-02-15T18:35:21.410 INFO:tasks.workunit.client.1.smithi121.stdout:6/170: getdents d7 0 2024-02-15T18:35:21.410 INFO:tasks.workunit.client.1.smithi121.stdout:6/171: fsync d7/dc/f15 0 2024-02-15T18:35:21.418 INFO:tasks.workunit.client.1.smithi121.stdout:3/187: creat d2/d1a/d22/f42 x:0 0 0 2024-02-15T18:35:21.418 INFO:tasks.workunit.client.1.smithi121.stdout:3/188: truncate d2/fd 297691 0 2024-02-15T18:35:21.420 INFO:tasks.workunit.client.1.smithi121.stdout:9/98: dwrite d7/d8/f17 [0,4194304] 0 2024-02-15T18:35:21.421 INFO:tasks.workunit.client.1.smithi121.stdout:2/82: dwrite d5/df/f16 [0,4194304] 0 2024-02-15T18:35:21.421 INFO:tasks.workunit.client.1.smithi121.stdout:3/189: dread d2/d1a/f1f [0,4194304] 0 2024-02-15T18:35:21.422 INFO:tasks.workunit.client.1.smithi121.stdout:5/99: dwrite d0/fb [0,4194304] 0 2024-02-15T18:35:21.423 INFO:tasks.workunit.client.1.smithi121.stdout:9/99: read d7/d8/f10 [757323,71740] 0 2024-02-15T18:35:21.428 INFO:tasks.workunit.client.1.smithi121.stdout:5/100: dread d0/d2/fd [0,4194304] 0 2024-02-15T18:35:21.435 INFO:tasks.workunit.client.1.smithi121.stdout:2/83: creat d5/df/f1b x:0 0 0 2024-02-15T18:35:21.435 INFO:tasks.workunit.client.1.smithi121.stdout:3/190: rmdir d2/d1a/d27/d2c/d25/d31 39 2024-02-15T18:35:21.437 INFO:tasks.workunit.client.1.smithi121.stdout:9/100: mkdir d7/d8/d25 0 2024-02-15T18:35:21.438 INFO:tasks.workunit.client.1.smithi121.stdout:5/101: dread - d0/d2/f14 zero size 2024-02-15T18:35:21.440 INFO:tasks.workunit.client.1.smithi121.stdout:1/136: dwrite d0/d6/d7/f26 [0,4194304] 0 2024-02-15T18:35:21.440 INFO:tasks.workunit.client.1.smithi121.stdout:6/172: dwrite d7/f31 [0,4194304] 0 2024-02-15T18:35:21.441 INFO:tasks.workunit.client.1.smithi121.stdout:3/191: creat d2/d1a/d27/d2c/d25/d31/f43 x:0 0 0 2024-02-15T18:35:21.441 INFO:tasks.workunit.client.1.smithi121.stdout:3/192: dread - d2/d1a/d27/d2c/f3a zero size 2024-02-15T18:35:21.441 INFO:tasks.workunit.client.1.smithi121.stdout:3/193: fdatasync d2/d1a/d27/f29 0 2024-02-15T18:35:21.441 INFO:tasks.workunit.client.1.smithi121.stdout:3/194: rename d2/d1a/d27/d2c to d2/d1a/d27/d2c/d25/d44 22 2024-02-15T18:35:21.443 INFO:tasks.workunit.client.1.smithi121.stdout:0/210: dwrite d5/d6/dd/f25 [0,4194304] 0 2024-02-15T18:35:21.444 INFO:tasks.workunit.client.1.smithi121.stdout:9/101: mkdir d7/d26 0 2024-02-15T18:35:21.445 INFO:tasks.workunit.client.1.smithi121.stdout:5/102: truncate d0/d2/fd 499797 0 2024-02-15T18:35:21.448 INFO:tasks.workunit.client.1.smithi121.stdout:9/102: write d7/f1f [364639,72309] 0 2024-02-15T18:35:21.452 INFO:tasks.workunit.client.1.smithi121.stdout:6/173: symlink d7/dc/d2c/d36/l3f 0 2024-02-15T18:35:21.452 INFO:tasks.workunit.client.1.smithi121.stdout:6/174: dread - d7/dc/d34/f1f zero size 2024-02-15T18:35:21.453 INFO:tasks.workunit.client.1.smithi121.stdout:0/211: dread d5/d6/f24 [4194304,4194304] 0 2024-02-15T18:35:21.453 INFO:tasks.workunit.client.1.smithi121.stdout:0/212: chown d5/d6/d2a/d36/d44/l48 1023 1 2024-02-15T18:35:21.454 INFO:tasks.workunit.client.1.smithi121.stdout:0/213: truncate d5/d6/dd/f2d 193307 0 2024-02-15T18:35:21.454 INFO:tasks.workunit.client.1.smithi121.stdout:6/175: read d7/f11 [951831,71336] 0 2024-02-15T18:35:21.457 INFO:tasks.workunit.client.1.smithi121.stdout:0/214: dread d5/f20 [0,4194304] 0 2024-02-15T18:35:21.461 INFO:tasks.workunit.client.1.smithi121.stdout:6/176: getdents d7/dc/d2c 0 2024-02-15T18:35:21.462 INFO:tasks.workunit.client.1.smithi121.stdout:0/215: creat d5/d6/dd/f4b x:0 0 0 2024-02-15T18:35:21.464 INFO:tasks.workunit.client.1.smithi121.stdout:6/177: rmdir d7/dc/d34 39 2024-02-15T18:35:21.466 INFO:tasks.workunit.client.1.smithi121.stdout:0/216: mkdir d5/d6/d4c 0 2024-02-15T18:35:21.468 INFO:tasks.workunit.client.1.smithi121.stdout:6/178: symlink d7/dc/d2c/d36/l40 0 2024-02-15T18:35:21.470 INFO:tasks.workunit.client.1.smithi121.stdout:6/179: write d7/dc/f15 [7939735,83989] 0 2024-02-15T18:35:21.470 INFO:tasks.workunit.client.1.smithi121.stdout:6/180: chown d7/dc 1701815 1 2024-02-15T18:35:21.473 INFO:tasks.workunit.client.1.smithi121.stdout:0/217: rename d5/f10 to d5/d30/f4d 0 2024-02-15T18:35:21.475 INFO:tasks.workunit.client.1.smithi121.stdout:0/218: rmdir d5/d6/d2a/d36 39 2024-02-15T18:35:21.482 INFO:tasks.workunit.client.1.smithi121.stdout:0/219: dread d5/d6/d2a/f2b [4194304,4194304] 0 2024-02-15T18:35:21.482 INFO:tasks.workunit.client.1.smithi121.stdout:0/220: stat d5/d6/dd/f34 0 2024-02-15T18:35:21.483 INFO:tasks.workunit.client.1.smithi121.stdout:0/221: mkdir d5/d6/d2a/d4e 0 2024-02-15T18:35:21.483 INFO:tasks.workunit.client.1.smithi121.stdout:0/222: write d5/d6/dd/f32 [99108,64467] 0 2024-02-15T18:35:21.483 INFO:tasks.workunit.client.1.smithi121.stdout:0/223: write d5/f40 [123306,32654] 0 2024-02-15T18:35:21.484 INFO:tasks.workunit.client.1.smithi121.stdout:0/224: creat d5/d30/f4f x:0 0 0 2024-02-15T18:35:21.484 INFO:tasks.workunit.client.1.smithi121.stdout:0/225: stat d5/f3b 0 2024-02-15T18:35:21.487 INFO:tasks.workunit.client.1.smithi121.stdout:1/137: dwrite d0/d6/f12 [0,4194304] 0 2024-02-15T18:35:21.489 INFO:tasks.workunit.client.1.smithi121.stdout:1/138: write d0/d6/f12 [2418447,113000] 0 2024-02-15T18:35:21.490 INFO:tasks.workunit.client.1.smithi121.stdout:1/139: creat d0/d6/d7/d9/d14/f2f x:0 0 0 2024-02-15T18:35:21.490 INFO:tasks.workunit.client.1.smithi121.stdout:1/140: dread - d0/d6/d7/d9/d14/f2f zero size 2024-02-15T18:35:21.495 INFO:tasks.workunit.client.1.smithi121.stdout:1/141: dread d0/d6/f19 [0,4194304] 0 2024-02-15T18:35:21.495 INFO:tasks.workunit.client.1.smithi121.stdout:5/103: dwrite d0/d2/fd [0,4194304] 0 2024-02-15T18:35:21.496 INFO:tasks.workunit.client.1.smithi121.stdout:5/104: write d0/fb [5430707,119244] 0 2024-02-15T18:35:21.496 INFO:tasks.workunit.client.1.smithi121.stdout:6/181: dwrite d7/dc/d2c/d36/f38 [0,4194304] 0 2024-02-15T18:35:21.497 INFO:tasks.workunit.client.1.smithi121.stdout:9/103: dwrite d7/f1b [0,4194304] 0 2024-02-15T18:35:21.497 INFO:tasks.workunit.client.1.smithi121.stdout:1/142: rename d0/d6/ca to d0/c30 0 2024-02-15T18:35:21.505 INFO:tasks.workunit.client.1.smithi121.stdout:0/226: dwrite d5/d6/dd/f2c [0,4194304] 0 2024-02-15T18:35:21.518 INFO:tasks.workunit.client.1.smithi121.stdout:9/104: symlink d7/l27 0 2024-02-15T18:35:21.519 INFO:tasks.workunit.client.1.smithi121.stdout:6/182: rename d7/dc/l2a to d7/l41 0 2024-02-15T18:35:21.520 INFO:tasks.workunit.client.1.smithi121.stdout:6/183: read d7/dc/d34/f13 [1643771,21988] 0 2024-02-15T18:35:21.520 INFO:tasks.workunit.client.1.smithi121.stdout:1/143: creat d0/d6/f31 x:0 0 0 2024-02-15T18:35:21.521 INFO:tasks.workunit.client.1.smithi121.stdout:5/105: mknod d0/d2/c1d 0 2024-02-15T18:35:21.521 INFO:tasks.workunit.client.1.smithi121.stdout:1/144: dread d0/d6/d7/d9/f23 [0,4194304] 0 2024-02-15T18:35:21.522 INFO:tasks.workunit.client.1.smithi121.stdout:9/105: mknod d7/d26/c28 0 2024-02-15T18:35:21.523 INFO:tasks.workunit.client.1.smithi121.stdout:6/184: mkdir d7/dc/d34/d42 0 2024-02-15T18:35:21.523 INFO:tasks.workunit.client.1.smithi121.stdout:6/185: fsync d7/dc/f20 0 2024-02-15T18:35:21.523 INFO:tasks.workunit.client.1.smithi121.stdout:6/186: chown d7/dc/d2c/d36/l3f 371 1 2024-02-15T18:35:21.523 INFO:tasks.workunit.client.1.smithi121.stdout:6/187: chown d7/c30 1842412 1 2024-02-15T18:35:21.524 INFO:tasks.workunit.client.1.smithi121.stdout:5/106: truncate d0/d2/fd 1422080 0 2024-02-15T18:35:21.524 INFO:tasks.workunit.client.1.smithi121.stdout:0/227: dwrite d5/d6/d2a/f2f [0,4194304] 0 2024-02-15T18:35:21.524 INFO:tasks.workunit.client.1.smithi121.stdout:1/145: mknod d0/c32 0 2024-02-15T18:35:21.525 INFO:tasks.workunit.client.1.smithi121.stdout:9/106: symlink d7/l29 0 2024-02-15T18:35:21.527 INFO:tasks.workunit.client.1.smithi121.stdout:4/117: sync 2024-02-15T18:35:21.528 INFO:tasks.workunit.client.1.smithi121.stdout:6/188: creat d7/dc/d34/f43 x:0 0 0 2024-02-15T18:35:21.534 INFO:tasks.workunit.client.1.smithi121.stdout:0/228: getdents d5/d6/dd 0 2024-02-15T18:35:21.536 INFO:tasks.workunit.client.1.smithi121.stdout:1/146: symlink d0/l33 0 2024-02-15T18:35:21.538 INFO:tasks.workunit.client.1.smithi121.stdout:9/107: mknod d7/c2a 0 2024-02-15T18:35:21.543 INFO:tasks.workunit.client.1.smithi121.stdout:6/189: rename d7/dc/d2c/f32 to d7/dc/d2c/d36/f44 0 2024-02-15T18:35:21.543 INFO:tasks.workunit.client.1.smithi121.stdout:5/107: mkdir d0/d2/d1e 0 2024-02-15T18:35:21.544 INFO:tasks.workunit.client.1.smithi121.stdout:4/118: mkdir d5/d9/d1d/d26 0 2024-02-15T18:35:21.544 INFO:tasks.workunit.client.1.smithi121.stdout:0/229: link d5/c38 d5/d6/d4c/c50 0 2024-02-15T18:35:21.546 INFO:tasks.workunit.client.1.smithi121.stdout:6/190: creat d7/dc/d34/f45 x:0 0 0 2024-02-15T18:35:21.548 INFO:tasks.workunit.client.1.smithi121.stdout:5/108: symlink d0/d2/d1e/l1f 0 2024-02-15T18:35:21.548 INFO:tasks.workunit.client.1.smithi121.stdout:5/109: dread - d0/d2/f14 zero size 2024-02-15T18:35:21.549 INFO:tasks.workunit.client.1.smithi121.stdout:4/119: read d5/f11 [2104359,41295] 0 2024-02-15T18:35:21.551 INFO:tasks.workunit.client.1.smithi121.stdout:6/191: unlink f5 0 2024-02-15T18:35:21.551 INFO:tasks.workunit.client.1.smithi121.stdout:6/192: dread - d7/f18 zero size 2024-02-15T18:35:21.551 INFO:tasks.workunit.client.1.smithi121.stdout:6/193: fdatasync d7/f28 0 2024-02-15T18:35:21.553 INFO:tasks.workunit.client.1.smithi121.stdout:5/110: rename d0/f15 to d0/d2/d18/f20 0 2024-02-15T18:35:21.553 INFO:tasks.workunit.client.1.smithi121.stdout:5/111: chown d0/d2/d18/c1b 324584 1 2024-02-15T18:35:21.554 INFO:tasks.workunit.client.1.smithi121.stdout:4/120: symlink d5/l27 0 2024-02-15T18:35:21.555 INFO:tasks.workunit.client.1.smithi121.stdout:6/194: mknod d7/dc/d34/c46 0 2024-02-15T18:35:21.555 INFO:tasks.workunit.client.1.smithi121.stdout:6/195: stat d7/dc/c1b 0 2024-02-15T18:35:21.555 INFO:tasks.workunit.client.1.smithi121.stdout:6/196: stat d7/dc/f20 0 2024-02-15T18:35:21.556 INFO:tasks.workunit.client.1.smithi121.stdout:1/147: dwrite d0/d6/d7/d9/f23 [0,4194304] 0 2024-02-15T18:35:21.556 INFO:tasks.workunit.client.1.smithi121.stdout:1/148: dread - d0/d6/d7/d9/f20 zero size 2024-02-15T18:35:21.557 INFO:tasks.workunit.client.1.smithi121.stdout:5/112: mkdir d0/d2/d1e/d21 0 2024-02-15T18:35:21.559 INFO:tasks.workunit.client.1.smithi121.stdout:0/230: rmdir d5/d6 39 2024-02-15T18:35:21.561 INFO:tasks.workunit.client.1.smithi121.stdout:4/121: truncate d5/d7/fb 3888384 0 2024-02-15T18:35:21.561 INFO:tasks.workunit.client.1.smithi121.stdout:0/231: read d5/f27 [2102946,32616] 0 2024-02-15T18:35:21.562 INFO:tasks.workunit.client.1.smithi121.stdout:6/197: creat d7/f47 x:0 0 0 2024-02-15T18:35:21.563 INFO:tasks.workunit.client.1.smithi121.stdout:0/232: write d5/d6/f1f [2267855,80558] 0 2024-02-15T18:35:21.565 INFO:tasks.workunit.client.1.smithi121.stdout:4/122: unlink d5/l27 0 2024-02-15T18:35:21.565 INFO:tasks.workunit.client.1.smithi121.stdout:4/123: fsync f1 0 2024-02-15T18:35:21.566 INFO:tasks.workunit.client.1.smithi121.stdout:6/198: creat d7/dc/f48 x:0 0 0 2024-02-15T18:35:21.566 INFO:tasks.workunit.client.1.smithi121.stdout:6/199: chown d7/dc/d34/f43 884 1 2024-02-15T18:35:21.566 INFO:tasks.workunit.client.1.smithi121.stdout:0/233: creat d5/d6/d2a/d36/f51 x:0 0 0 2024-02-15T18:35:21.568 INFO:tasks.workunit.client.1.smithi121.stdout:4/124: rename d5/d9/de to d5/d9/d19/d28 0 2024-02-15T18:35:21.570 INFO:tasks.workunit.client.1.smithi121.stdout:6/200: creat d7/dc/d2c/d36/f49 x:0 0 0 2024-02-15T18:35:21.573 INFO:tasks.workunit.client.1.smithi121.stdout:4/125: getdents d5/d9/d19/d28 0 2024-02-15T18:35:21.575 INFO:tasks.workunit.client.1.smithi121.stdout:1/149: dwrite d0/d6/d7/d9/d14/f2f [0,4194304] 0 2024-02-15T18:35:21.576 INFO:tasks.workunit.client.1.smithi121.stdout:6/201: creat d7/dc/d34/f4a x:0 0 0 2024-02-15T18:35:21.577 INFO:tasks.workunit.client.1.smithi121.stdout:6/202: write d7/f47 [336674,60593] 0 2024-02-15T18:35:21.577 INFO:tasks.workunit.client.1.smithi121.stdout:1/150: dread d0/fb [0,4194304] 0 2024-02-15T18:35:21.580 INFO:tasks.workunit.client.1.smithi121.stdout:4/126: rename d5/f11 to d5/d7/d14/f29 0 2024-02-15T18:35:21.584 INFO:tasks.workunit.client.1.smithi121.stdout:0/234: dwrite d5/d6/d2a/f2b [0,4194304] 0 2024-02-15T18:35:21.584 INFO:tasks.workunit.client.1.smithi121.stdout:0/235: chown d5/d6/d2a/d36/c49 20817 1 2024-02-15T18:35:21.589 INFO:tasks.workunit.client.1.smithi121.stdout:0/236: symlink d5/d6/d2a/l52 0 2024-02-15T18:35:21.601 INFO:tasks.workunit.client.1.smithi121.stdout:6/203: dwrite d7/dc/d34/f43 [0,4194304] 0 2024-02-15T18:35:21.601 INFO:tasks.workunit.client.1.smithi121.stdout:4/127: dwrite d5/d9/f20 [0,4194304] 0 2024-02-15T18:35:21.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:20.590814+0000 mgr.smithi121.ckiryi (mgr.24385) 2 : cluster [DBG] pgmap v4: 65 pgs: 65 active+clean; 445 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:35:21.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:20.604716+0000 mon.smithi038 (mon.0) 587 : cluster [DBG] mgrmap e22: smithi121.ckiryi(active, since 3s) 2024-02-15T18:35:21.606 INFO:tasks.workunit.client.1.smithi121.stdout:0/237: dread d5/d6/f1c [0,4194304] 0 2024-02-15T18:35:21.610 INFO:tasks.workunit.client.1.smithi121.stdout:0/238: unlink d5/f20 0 2024-02-15T18:35:21.610 INFO:tasks.workunit.client.1.smithi121.stdout:0/239: truncate d5/d6/dd/f47 566630 0 2024-02-15T18:35:21.610 INFO:tasks.workunit.client.1.smithi121.stdout:0/240: symlink d5/l53 0 2024-02-15T18:35:21.610 INFO:tasks.workunit.client.1.smithi121.stdout:0/241: stat d5/d6/dd/f2d 0 2024-02-15T18:35:21.613 INFO:tasks.workunit.client.1.smithi121.stdout:6/204: dread d7/f8 [0,4194304] 0 2024-02-15T18:35:21.613 INFO:tasks.workunit.client.1.smithi121.stdout:0/242: creat d5/d6/d2a/d36/f54 x:0 0 0 2024-02-15T18:35:21.613 INFO:tasks.workunit.client.1.smithi121.stdout:6/205: mkdir d7/d4b 0 2024-02-15T18:35:21.613 INFO:tasks.workunit.client.1.smithi121.stdout:6/206: dread - d7/dc/d2c/d36/f3c zero size 2024-02-15T18:35:21.614 INFO:tasks.workunit.client.1.smithi121.stdout:0/243: truncate d5/d6/f1c 3113290 0 2024-02-15T18:35:21.616 INFO:tasks.workunit.client.1.smithi121.stdout:1/151: dwrite d0/d6/d7/d9/d14/f2f [4194304,4194304] 0 2024-02-15T18:35:21.617 INFO:tasks.workunit.client.1.smithi121.stdout:6/207: dread d7/dc/d34/f26 [0,4194304] 0 2024-02-15T18:35:21.618 INFO:tasks.workunit.client.1.smithi121.stdout:1/152: symlink d0/d6/d7/d29/l34 0 2024-02-15T18:35:21.619 INFO:tasks.workunit.client.1.smithi121.stdout:4/128: dwrite f1 [4194304,4194304] 0 2024-02-15T18:35:21.620 INFO:tasks.workunit.client.1.smithi121.stdout:4/129: read d5/f10 [385806,75242] 0 2024-02-15T18:35:21.620 INFO:tasks.workunit.client.1.smithi121.stdout:4/130: write d5/d9/d19/d28/f21 [749982,83742] 0 2024-02-15T18:35:21.634 INFO:tasks.workunit.client.1.smithi121.stdout:0/244: dwrite d5/f27 [0,4194304] 0 2024-02-15T18:35:21.647 INFO:tasks.workunit.client.1.smithi121.stdout:0/245: dwrite d5/f40 [0,4194304] 0 2024-02-15T18:35:21.668 INFO:tasks.workunit.client.1.smithi121.stdout:6/208: creat d7/dc/f4c x:0 0 0 2024-02-15T18:35:21.669 INFO:tasks.workunit.client.1.smithi121.stdout:1/153: creat d0/d6/d7/d9/f35 x:0 0 0 2024-02-15T18:35:21.669 INFO:tasks.workunit.client.1.smithi121.stdout:1/154: write d0/d6/d7/f27 [215057,14253] 0 2024-02-15T18:35:21.700 INFO:tasks.workunit.client.1.smithi121.stdout:6/209: dwrite d7/dc/d34/f43 [0,4194304] 0 2024-02-15T18:35:21.702 INFO:tasks.workunit.client.1.smithi121.stdout:0/246: creat d5/d6/d2a/d36/d44/f55 x:0 0 0 2024-02-15T18:35:21.712 INFO:tasks.workunit.client.1.smithi121.stdout:1/155: getdents d0/d6/d7/d9/d14 0 2024-02-15T18:35:21.715 INFO:tasks.workunit.client.1.smithi121.stdout:8/194: sync 2024-02-15T18:35:21.715 INFO:tasks.workunit.client.1.smithi121.stdout:8/195: readlink d0/l19 0 2024-02-15T18:35:21.718 INFO:tasks.workunit.client.1.smithi121.stdout:6/210: unlink l6 0 2024-02-15T18:35:21.722 INFO:tasks.workunit.client.1.smithi121.stdout:0/247: getdents d5/d6/d4c 0 2024-02-15T18:35:21.726 INFO:tasks.workunit.client.1.smithi121.stdout:1/156: symlink d0/l36 0 2024-02-15T18:35:21.726 INFO:tasks.workunit.client.1.smithi121.stdout:8/196: getdents d0/d3/d20/d39/d43 0 2024-02-15T18:35:21.731 INFO:tasks.workunit.client.1.smithi121.stdout:0/248: truncate d5/d6/d2a/d36/d44/f55 552435 0 2024-02-15T18:35:21.732 INFO:tasks.workunit.client.1.smithi121.stdout:6/211: dwrite d7/dc/d2c/d36/f38 [0,4194304] 0 2024-02-15T18:35:21.733 INFO:tasks.workunit.client.1.smithi121.stdout:8/197: rename d0/d7/de/d27/d28 to d0/d7/d3f/d45 0 2024-02-15T18:35:21.735 INFO:tasks.workunit.client.1.smithi121.stdout:0/249: rename d5/d6/f1c to d5/d6/f56 0 2024-02-15T18:35:21.736 INFO:tasks.workunit.client.1.smithi121.stdout:6/212: unlink d7/dc/d2c/d36/f44 0 2024-02-15T18:35:21.738 INFO:tasks.workunit.client.1.smithi121.stdout:8/198: rmdir d0/d2a 39 2024-02-15T18:35:21.738 INFO:tasks.workunit.client.1.smithi121.stdout:8/199: chown d0/d3/c9 382486 1 2024-02-15T18:35:21.740 INFO:tasks.workunit.client.1.smithi121.stdout:1/157: dwrite d0/d6/d7/d9/f20 [0,4194304] 0 2024-02-15T18:35:21.741 INFO:tasks.workunit.client.1.smithi121.stdout:0/250: rename d5/d6/dd/l45 to d5/d6/d2a/d36/l57 0 2024-02-15T18:35:21.742 INFO:tasks.workunit.client.1.smithi121.stdout:8/200: link d0/d3/c16 d0/d7/d3f/d45/c46 0 2024-02-15T18:35:21.742 INFO:tasks.workunit.client.1.smithi121.stdout:8/201: getdents d0/d7/d36 0 2024-02-15T18:35:21.745 INFO:tasks.workunit.client.1.smithi121.stdout:8/202: dread d0/f5 [0,4194304] 0 2024-02-15T18:35:21.749 INFO:tasks.workunit.client.1.smithi121.stdout:1/158: truncate d0/f1d 1942547 0 2024-02-15T18:35:21.752 INFO:tasks.workunit.client.1.smithi121.stdout:6/213: rename d7/l1e to d7/d4b/l4d 0 2024-02-15T18:35:21.755 INFO:tasks.workunit.client.1.smithi121.stdout:1/159: fsync d0/f1d 0 2024-02-15T18:35:21.758 INFO:tasks.workunit.client.1.smithi121.stdout:0/251: link d5/d6/le d5/d6/l58 0 2024-02-15T18:35:21.758 INFO:tasks.workunit.client.1.smithi121.stdout:8/203: link d0/c34 d0/d2a/c47 0 2024-02-15T18:35:21.760 INFO:tasks.workunit.client.1.smithi121.stdout:8/204: read d0/f37 [445583,126517] 0 2024-02-15T18:35:21.761 INFO:tasks.workunit.client.1.smithi121.stdout:6/214: link d7/dc/d34/c33 d7/dc/d34/d42/c4e 0 2024-02-15T18:35:21.761 INFO:tasks.workunit.client.1.smithi121.stdout:6/215: write d7/dc/f25 [465072,85615] 0 2024-02-15T18:35:21.762 INFO:tasks.workunit.client.1.smithi121.stdout:8/205: mkdir d0/d48 0 2024-02-15T18:35:21.765 INFO:tasks.workunit.client.1.smithi121.stdout:6/216: symlink d7/dc/d2c/d36/l4f 0 2024-02-15T18:35:21.765 INFO:tasks.workunit.client.1.smithi121.stdout:6/217: truncate d7/f18 663694 0 2024-02-15T18:35:21.765 INFO:tasks.workunit.client.1.smithi121.stdout:6/218: fdatasync d7/f8 0 2024-02-15T18:35:21.765 INFO:tasks.workunit.client.1.smithi121.stdout:8/206: creat d0/d7/d3f/f49 x:0 0 0 2024-02-15T18:35:21.767 INFO:tasks.workunit.client.1.smithi121.stdout:1/160: dread d0/d6/d7/d9/d14/f2f [0,4194304] 0 2024-02-15T18:35:21.767 INFO:tasks.workunit.client.1.smithi121.stdout:8/207: read d0/d7/f10 [3681355,108878] 0 2024-02-15T18:35:21.769 INFO:tasks.workunit.client.1.smithi121.stdout:1/161: unlink d0/d6/d7/d9/f25 0 2024-02-15T18:35:21.771 INFO:tasks.workunit.client.1.smithi121.stdout:8/208: rename d0/d3/d20/d39/d43 to d0/d4a 0 2024-02-15T18:35:21.773 INFO:tasks.workunit.client.1.smithi121.stdout:8/209: creat d0/d2a/f4b x:0 0 0 2024-02-15T18:35:21.777 INFO:tasks.workunit.client.1.smithi121.stdout:8/210: creat d0/d7/de/f4c x:0 0 0 2024-02-15T18:35:21.777 INFO:tasks.workunit.client.1.smithi121.stdout:8/211: chown d0/d7/de/d27/c41 114960 1 2024-02-15T18:35:21.784 INFO:tasks.workunit.client.1.smithi121.stdout:0/252: dwrite d5/d6/d2a/d36/f54 [0,4194304] 0 2024-02-15T18:35:21.784 INFO:tasks.workunit.client.1.smithi121.stdout:0/253: fsync d5/d6/d2a/d36/f54 0 2024-02-15T18:35:21.784 INFO:tasks.workunit.client.1.smithi121.stdout:0/254: readlink d5/l23 0 2024-02-15T18:35:21.787 INFO:tasks.workunit.client.1.smithi121.stdout:6/219: dwrite d7/dc/d34/f26 [4194304,4194304] 0 2024-02-15T18:35:21.787 INFO:tasks.workunit.client.1.smithi121.stdout:6/220: fdatasync d7/dc/d34/f26 0 2024-02-15T18:35:21.790 INFO:tasks.workunit.client.1.smithi121.stdout:0/255: mknod d5/d6/dd/c59 0 2024-02-15T18:35:21.791 INFO:tasks.workunit.client.1.smithi121.stdout:1/162: dwrite d0/d6/d7/f27 [0,4194304] 0 2024-02-15T18:35:21.791 INFO:tasks.workunit.client.1.smithi121.stdout:1/163: readlink d0/l36 0 2024-02-15T18:35:21.791 INFO:tasks.workunit.client.1.smithi121.stdout:1/164: write d0/d6/d7/f26 [2458033,34824] 0 2024-02-15T18:35:21.793 INFO:tasks.workunit.client.1.smithi121.stdout:8/212: fsync d0/d2a/f4b 0 2024-02-15T18:35:21.793 INFO:tasks.workunit.client.1.smithi121.stdout:8/213: write d0/f2 [520574,34386] 0 2024-02-15T18:35:21.793 INFO:tasks.workunit.client.1.smithi121.stdout:8/214: fsync d0/d3/d20/d39/f3b 0 2024-02-15T18:35:21.798 INFO:tasks.workunit.client.1.smithi121.stdout:6/221: creat d7/f50 x:0 0 0 2024-02-15T18:35:21.798 INFO:tasks.workunit.client.1.smithi121.stdout:6/222: readlink d7/dc/d2c/d36/l4f 0 2024-02-15T18:35:21.803 INFO:tasks.workunit.client.1.smithi121.stdout:1/165: rename d0/l33 to d0/d6/d7/l37 0 2024-02-15T18:35:21.804 INFO:tasks.workunit.client.1.smithi121.stdout:0/256: creat d5/d6/d2a/d4e/f5a x:0 0 0 2024-02-15T18:35:21.804 INFO:tasks.workunit.client.1.smithi121.stdout:0/257: stat d5/d6/d2a/d4e 0 2024-02-15T18:35:21.806 INFO:tasks.workunit.client.1.smithi121.stdout:6/223: rename d7/dc/d34/c46 to d7/c51 0 2024-02-15T18:35:21.806 INFO:tasks.workunit.client.1.smithi121.stdout:1/166: creat d0/d6/f38 x:0 0 0 2024-02-15T18:35:21.808 INFO:tasks.workunit.client.1.smithi121.stdout:0/258: unlink d5/d6/d2a/d36/d44/f55 0 2024-02-15T18:35:21.811 INFO:tasks.workunit.client.1.smithi121.stdout:8/215: dwrite d0/d7/de/f1c [0,4194304] 0 2024-02-15T18:35:21.811 INFO:tasks.workunit.client.1.smithi121.stdout:6/224: creat d7/dc/f52 x:0 0 0 2024-02-15T18:35:21.811 INFO:tasks.workunit.client.1.smithi121.stdout:6/225: dread - d7/dc/d34/f45 zero size 2024-02-15T18:35:21.813 INFO:tasks.workunit.client.1.smithi121.stdout:1/167: mkdir d0/d39 0 2024-02-15T18:35:21.831 INFO:tasks.workunit.client.1.smithi121.stdout:8/216: mknod d0/d7/d36/c4d 0 2024-02-15T18:35:21.833 INFO:tasks.workunit.client.1.smithi121.stdout:1/168: symlink d0/d39/l3a 0 2024-02-15T18:35:21.833 INFO:tasks.workunit.client.1.smithi121.stdout:1/169: dread - d0/d6/d7/d9/f35 zero size 2024-02-15T18:35:21.833 INFO:tasks.workunit.client.1.smithi121.stdout:1/170: write d0/d6/d7/d9/f35 [428945,66373] 0 2024-02-15T18:35:21.835 INFO:tasks.workunit.client.1.smithi121.stdout:0/259: rename d5/d30/f3e to d5/d6/d2a/d4e/f5b 0 2024-02-15T18:35:21.837 INFO:tasks.workunit.client.1.smithi121.stdout:8/217: symlink d0/d7/de/d27/l4e 0 2024-02-15T18:35:21.839 INFO:tasks.workunit.client.1.smithi121.stdout:6/226: unlink d7/dc/f52 0 2024-02-15T18:35:21.839 INFO:tasks.workunit.client.1.smithi121.stdout:0/260: mknod d5/d6/dd/c5c 0 2024-02-15T18:35:21.840 INFO:tasks.workunit.client.1.smithi121.stdout:8/218: creat d0/d7/d3f/f4f x:0 0 0 2024-02-15T18:35:21.841 INFO:tasks.workunit.client.1.smithi121.stdout:6/227: getdents d7/dc/d34/d42 0 2024-02-15T18:35:21.843 INFO:tasks.workunit.client.1.smithi121.stdout:6/228: creat d7/d4b/f53 x:0 0 0 2024-02-15T18:35:21.845 INFO:tasks.workunit.client.1.smithi121.stdout:6/229: link d7/dc/f20 d7/dc/d34/d42/f54 0 2024-02-15T18:35:21.846 INFO:tasks.workunit.client.1.smithi121.stdout:1/171: dwrite d0/d6/d7/f1b [0,4194304] 0 2024-02-15T18:35:21.847 INFO:tasks.workunit.client.1.smithi121.stdout:6/230: unlink d7/dc/c2e 0 2024-02-15T18:35:21.847 INFO:tasks.workunit.client.1.smithi121.stdout:6/231: write d7/dc/f2d [976111,30256] 0 2024-02-15T18:35:21.847 INFO:tasks.workunit.client.1.smithi121.stdout:1/172: write d0/d6/f12 [1386600,100194] 0 2024-02-15T18:35:21.855 INFO:tasks.workunit.client.1.smithi121.stdout:1/173: creat d0/d6/d7/d9/d14/f3b x:0 0 0 2024-02-15T18:35:21.857 INFO:tasks.workunit.client.1.smithi121.stdout:1/174: mknod d0/d6/d7/c3c 0 2024-02-15T18:35:21.858 INFO:tasks.workunit.client.1.smithi121.stdout:1/175: fsync d0/d6/f12 0 2024-02-15T18:35:21.858 INFO:tasks.workunit.client.1.smithi121.stdout:1/176: write d0/d6/d7/d9/f1e [984520,47452] 0 2024-02-15T18:35:21.858 INFO:tasks.workunit.client.1.smithi121.stdout:0/261: dwrite d5/d6/f56 [0,4194304] 0 2024-02-15T18:35:21.860 INFO:tasks.workunit.client.1.smithi121.stdout:8/219: rmdir d0/d7/de 39 2024-02-15T18:35:21.861 INFO:tasks.workunit.client.1.smithi121.stdout:8/220: write d0/d3/f26 [3455418,84542] 0 2024-02-15T18:35:21.866 INFO:tasks.workunit.client.1.smithi121.stdout:1/177: creat d0/f3d x:0 0 0 2024-02-15T18:35:21.866 INFO:tasks.workunit.client.1.smithi121.stdout:1/178: write d0/f21 [2209262,99093] 0 2024-02-15T18:35:21.873 INFO:tasks.workunit.client.1.smithi121.stdout:1/179: mkdir d0/d6/d7/d9/d14/d3e 0 2024-02-15T18:35:21.875 INFO:tasks.workunit.client.1.smithi121.stdout:1/180: rename d0/d6/d7/d9/l2b to d0/d6/d7/d9/l3f 0 2024-02-15T18:35:21.877 INFO:tasks.workunit.client.1.smithi121.stdout:0/262: dwrite d5/d6/dd/f25 [4194304,4194304] 0 2024-02-15T18:35:21.878 INFO:tasks.workunit.client.1.smithi121.stdout:8/221: dwrite d0/d7/d3f/d45/f2c [4194304,4194304] 0 2024-02-15T18:35:21.879 INFO:tasks.workunit.client.1.smithi121.stdout:8/222: fdatasync d0/d7/d3f/d45/f2c 0 2024-02-15T18:35:21.883 INFO:tasks.workunit.client.1.smithi121.stdout:0/263: rename d5/d6/d4c to d5/d30/d5d 0 2024-02-15T18:35:21.890 INFO:tasks.workunit.client.1.smithi121.stdout:8/223: rename d0/l1f to d0/d2a/l50 0 2024-02-15T18:35:21.891 INFO:tasks.workunit.client.1.smithi121.stdout:8/224: chown d0/d7/d3f/d45/c46 0 1 2024-02-15T18:35:21.893 INFO:tasks.workunit.client.1.smithi121.stdout:1/181: dwrite d0/fb [0,4194304] 0 2024-02-15T18:35:21.893 INFO:tasks.workunit.client.1.smithi121.stdout:1/182: write d0/d6/f31 [961894,9652] 0 2024-02-15T18:35:21.893 INFO:tasks.workunit.client.1.smithi121.stdout:1/183: write d0/d6/f31 [1097694,61086] 0 2024-02-15T18:35:21.893 INFO:tasks.workunit.client.1.smithi121.stdout:8/225: rename d0/d7/de/d27 to d0/d2a/d51 0 2024-02-15T18:35:21.901 INFO:tasks.workunit.client.1.smithi121.stdout:8/226: dread d0/d7/f1e [0,4194304] 0 2024-02-15T18:35:21.903 INFO:tasks.workunit.client.1.smithi121.stdout:8/227: mkdir d0/d7/d36/d52 0 2024-02-15T18:35:21.903 INFO:tasks.workunit.client.1.smithi121.stdout:8/228: chown d0/d4a 1 1 2024-02-15T18:35:21.905 INFO:tasks.workunit.client.1.smithi121.stdout:0/264: dwrite d5/d30/f33 [0,4194304] 0 2024-02-15T18:35:21.907 INFO:tasks.workunit.client.1.smithi121.stdout:0/265: write d5/d6/dd/f16 [320460,106621] 0 2024-02-15T18:35:21.910 INFO:tasks.workunit.client.1.smithi121.stdout:0/266: dread d5/d30/d3c/f46 [0,4194304] 0 2024-02-15T18:35:21.913 INFO:tasks.workunit.client.1.smithi121.stdout:0/267: write d5/d6/dd/f25 [4016183,102873] 0 2024-02-15T18:35:21.914 INFO:tasks.workunit.client.1.smithi121.stdout:0/268: truncate d5/d6/dd/f25 9427585 0 2024-02-15T18:35:21.914 INFO:tasks.workunit.client.1.smithi121.stdout:0/269: stat d5/d6/dd/l4a 0 2024-02-15T18:35:21.914 INFO:tasks.workunit.client.1.smithi121.stdout:0/270: fsync d5/d30/f4d 0 2024-02-15T18:35:21.914 INFO:tasks.workunit.client.1.smithi121.stdout:0/271: stat d5/d6/d2a/f2f 0 2024-02-15T18:35:21.919 INFO:tasks.workunit.client.1.smithi121.stdout:0/272: read d5/d6/f11 [7472844,57496] 0 2024-02-15T18:35:21.920 INFO:tasks.workunit.client.1.smithi121.stdout:1/184: dwrite d0/d6/f19 [0,4194304] 0 2024-02-15T18:35:21.920 INFO:tasks.workunit.client.1.smithi121.stdout:1/185: fdatasync d0/d6/d7/d9/d14/f3b 0 2024-02-15T18:35:21.920 INFO:tasks.workunit.client.1.smithi121.stdout:1/186: readlink d0/l4 0 2024-02-15T18:35:21.927 INFO:tasks.workunit.client.1.smithi121.stdout:0/273: rename d5/d30/f4d to d5/d30/d5d/f5e 0 2024-02-15T18:35:21.927 INFO:tasks.workunit.client.1.smithi121.stdout:0/274: readlink d5/d6/dd/l28 0 2024-02-15T18:35:21.927 INFO:tasks.workunit.client.1.smithi121.stdout:8/229: dwrite d0/d3/d20/d39/f3b [0,4194304] 0 2024-02-15T18:35:21.927 INFO:tasks.workunit.client.1.smithi121.stdout:8/230: chown d0/d7/d36/d52 4639 1 2024-02-15T18:35:21.927 INFO:tasks.workunit.client.1.smithi121.stdout:8/231: stat d0/d7/de/f2d 0 2024-02-15T18:35:21.937 INFO:tasks.workunit.client.1.smithi121.stdout:0/275: rmdir d5/d6/dd 39 2024-02-15T18:35:21.937 INFO:tasks.workunit.client.1.smithi121.stdout:0/276: write d5/d6/dd/f32 [716756,63727] 0 2024-02-15T18:35:21.939 INFO:tasks.workunit.client.1.smithi121.stdout:1/187: dwrite d0/d6/d7/d9/d14/f2f [0,4194304] 0 2024-02-15T18:35:21.941 INFO:tasks.workunit.client.1.smithi121.stdout:1/188: write d0/f21 [803138,66167] 0 2024-02-15T18:35:21.941 INFO:tasks.workunit.client.1.smithi121.stdout:8/232: mkdir d0/d7/d3f/d45/d53 0 2024-02-15T18:35:21.943 INFO:tasks.workunit.client.1.smithi121.stdout:1/189: symlink d0/d6/l40 0 2024-02-15T18:35:21.945 INFO:tasks.workunit.client.1.smithi121.stdout:1/190: write d0/d6/f19 [1198002,123658] 0 2024-02-15T18:35:21.948 INFO:tasks.workunit.client.1.smithi121.stdout:0/277: dread d5/d6/f56 [0,4194304] 0 2024-02-15T18:35:21.948 INFO:tasks.workunit.client.1.smithi121.stdout:0/278: chown d5/l21 236025 1 2024-02-15T18:35:21.950 INFO:tasks.workunit.client.1.smithi121.stdout:1/191: creat d0/d6/d7/d29/f41 x:0 0 0 2024-02-15T18:35:21.951 INFO:tasks.workunit.client.1.smithi121.stdout:0/279: symlink d5/d6/d2a/d4e/l5f 0 2024-02-15T18:35:21.952 INFO:tasks.workunit.client.1.smithi121.stdout:0/280: dread d5/d6/dd/f2d [0,4194304] 0 2024-02-15T18:35:21.952 INFO:tasks.workunit.client.1.smithi121.stdout:1/192: mknod d0/d39/c42 0 2024-02-15T18:35:21.952 INFO:tasks.workunit.client.1.smithi121.stdout:1/193: read d0/d6/d7/d9/f1e [72514,84445] 0 2024-02-15T18:35:21.953 INFO:tasks.workunit.client.1.smithi121.stdout:8/233: rmdir d0/d7/d3f 39 2024-02-15T18:35:21.953 INFO:tasks.workunit.client.1.smithi121.stdout:1/194: read d0/d6/d7/f26 [2136367,36036] 0 2024-02-15T18:35:21.955 INFO:tasks.workunit.client.1.smithi121.stdout:0/281: write d5/f27 [1201745,62617] 0 2024-02-15T18:35:21.956 INFO:tasks.workunit.client.1.smithi121.stdout:1/195: creat d0/d6/d7/d9/d14/d3e/f43 x:0 0 0 2024-02-15T18:35:21.956 INFO:tasks.workunit.client.1.smithi121.stdout:8/234: write d0/f15 [3939877,29048] 0 2024-02-15T18:35:21.960 INFO:tasks.workunit.client.1.smithi121.stdout:0/282: rename d5/d6/d2a/d36/c39 to d5/d30/d3c/c60 0 2024-02-15T18:35:21.960 INFO:tasks.workunit.client.1.smithi121.stdout:0/283: dread - d5/d6/d2a/d36/f51 zero size 2024-02-15T18:35:21.962 INFO:tasks.workunit.client.1.smithi121.stdout:8/235: getdents d0/d3/d20 0 2024-02-15T18:35:21.963 INFO:tasks.workunit.client.1.smithi121.stdout:0/284: mknod d5/d6/d2a/d36/d44/c61 0 2024-02-15T18:35:21.965 INFO:tasks.workunit.client.1.smithi121.stdout:8/236: creat d0/d7/de/f54 x:0 0 0 2024-02-15T18:35:21.968 INFO:tasks.workunit.client.1.smithi121.stdout:0/285: mknod d5/d30/d5d/c62 0 2024-02-15T18:35:21.968 INFO:tasks.workunit.client.1.smithi121.stdout:0/286: chown d5/d6/d2a/d4e 4 1 2024-02-15T18:35:21.968 INFO:tasks.workunit.client.1.smithi121.stdout:0/287: readlink d5/l53 0 2024-02-15T18:35:21.970 INFO:tasks.workunit.client.1.smithi121.stdout:0/288: rename d5/d6/d2a/d36/c49 to d5/d6/dd/c63 0 2024-02-15T18:35:21.971 INFO:tasks.workunit.client.1.smithi121.stdout:0/289: creat d5/d6/d2a/f64 x:0 0 0 2024-02-15T18:35:21.971 INFO:tasks.workunit.client.1.smithi121.stdout:0/290: write d5/d30/f4f [486905,115809] 0 2024-02-15T18:35:21.972 INFO:tasks.workunit.client.1.smithi121.stdout:8/237: dread d0/d7/d3f/d45/f2c [0,4194304] 0 2024-02-15T18:35:21.974 INFO:tasks.workunit.client.1.smithi121.stdout:0/291: write d5/d6/d2a/f2f [2753608,119330] 0 2024-02-15T18:35:21.974 INFO:tasks.workunit.client.1.smithi121.stdout:1/196: dwrite d0/d6/d7/d29/f41 [0,4194304] 0 2024-02-15T18:35:21.974 INFO:tasks.workunit.client.1.smithi121.stdout:1/197: write d0/d6/d7/f26 [2772820,100675] 0 2024-02-15T18:35:21.974 INFO:tasks.workunit.client.1.smithi121.stdout:1/198: stat d0/d6/f31 0 2024-02-15T18:35:21.974 INFO:tasks.workunit.client.1.smithi121.stdout:1/199: readlink d0/d6/d7/d9/l3f 0 2024-02-15T18:35:21.974 INFO:tasks.workunit.client.1.smithi121.stdout:1/200: stat d0/d6/d7/d9/d14 0 2024-02-15T18:35:21.975 INFO:tasks.workunit.client.1.smithi121.stdout:8/238: symlink d0/d4a/l55 0 2024-02-15T18:35:21.983 INFO:tasks.workunit.client.1.smithi121.stdout:7/148: sync 2024-02-15T18:35:21.984 INFO:tasks.workunit.client.1.smithi121.stdout:7/149: mkdir d0/d33 0 2024-02-15T18:35:21.987 INFO:tasks.workunit.client.1.smithi121.stdout:7/150: rename d0/fd to d0/d33/f34 0 2024-02-15T18:35:21.989 INFO:tasks.workunit.client.1.smithi121.stdout:7/151: write d0/d33/f34 [63585,61187] 0 2024-02-15T18:35:22.003 INFO:tasks.workunit.client.1.smithi121.stdout:0/292: dwrite d5/d6/d2a/d4e/f5a [0,4194304] 0 2024-02-15T18:35:22.006 INFO:tasks.workunit.client.1.smithi121.stdout:8/239: dwrite d0/d3/f26 [0,4194304] 0 2024-02-15T18:35:22.006 INFO:tasks.workunit.client.1.smithi121.stdout:1/201: dwrite d0/d6/f12 [0,4194304] 0 2024-02-15T18:35:22.008 INFO:tasks.workunit.client.1.smithi121.stdout:8/240: rmdir d0 39 2024-02-15T18:35:22.008 INFO:tasks.workunit.client.1.smithi121.stdout:8/241: readlink d0/d2a/d51/l2f 0 2024-02-15T18:35:22.008 INFO:tasks.workunit.client.1.smithi121.stdout:8/242: chown d0/d3/c17 139474837 1 2024-02-15T18:35:22.008 INFO:tasks.workunit.client.1.smithi121.stdout:1/202: read d0/d6/d7/d29/f41 [2506944,32928] 0 2024-02-15T18:35:22.009 INFO:tasks.workunit.client.1.smithi121.stdout:0/293: write d5/d6/d2a/f2b [1745676,38117] 0 2024-02-15T18:35:22.016 INFO:tasks.workunit.client.1.smithi121.stdout:0/294: dread d5/d6/d2a/f2f [0,4194304] 0 2024-02-15T18:35:22.020 INFO:tasks.workunit.client.1.smithi121.stdout:8/243: rmdir d0/d7 39 2024-02-15T18:35:22.020 INFO:tasks.workunit.client.1.smithi121.stdout:8/244: write d0/d7/de/f4c [570957,80947] 0 2024-02-15T18:35:22.021 INFO:tasks.workunit.client.1.smithi121.stdout:1/203: link d0/d6/d7/l37 d0/d6/d7/l44 0 2024-02-15T18:35:22.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:20.590814+0000 mgr.smithi121.ckiryi (mgr.24385) 2 : cluster [DBG] pgmap v4: 65 pgs: 65 active+clean; 445 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:35:22.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:20.604716+0000 mon.smithi038 (mon.0) 587 : cluster [DBG] mgrmap e22: smithi121.ckiryi(active, since 3s) 2024-02-15T18:35:22.023 INFO:tasks.workunit.client.1.smithi121.stdout:0/295: dread d5/f27 [0,4194304] 0 2024-02-15T18:35:22.024 INFO:tasks.workunit.client.1.smithi121.stdout:0/296: write d5/d6/d2a/f64 [41348,48634] 0 2024-02-15T18:35:22.024 INFO:tasks.workunit.client.1.smithi121.stdout:0/297: fdatasync d5/d6/dd/f16 0 2024-02-15T18:35:22.026 INFO:tasks.workunit.client.1.smithi121.stdout:8/245: creat d0/d3/f56 x:0 0 0 2024-02-15T18:35:22.029 INFO:tasks.workunit.client.1.smithi121.stdout:1/204: mknod d0/d39/c45 0 2024-02-15T18:35:22.029 INFO:tasks.workunit.client.1.smithi121.stdout:1/205: readlink d0/l4 0 2024-02-15T18:35:22.029 INFO:tasks.workunit.client.1.smithi121.stdout:1/206: read - d0/d6/f38 zero size 2024-02-15T18:35:22.031 INFO:tasks.workunit.client.1.smithi121.stdout:8/246: rename d0/d3/f26 to d0/d2a/d51/f57 0 2024-02-15T18:35:22.031 INFO:tasks.workunit.client.1.smithi121.stdout:8/247: truncate d0/d3/f56 1038470 0 2024-02-15T18:35:22.031 INFO:tasks.workunit.client.1.smithi121.stdout:8/248: stat d0 0 2024-02-15T18:35:22.032 INFO:tasks.workunit.client.1.smithi121.stdout:8/249: rename d0/d2a/l50 to d0/d7/d36/l58 0 2024-02-15T18:35:22.034 INFO:tasks.workunit.client.1.smithi121.stdout:0/298: dread d5/d6/d2a/f2b [0,4194304] 0 2024-02-15T18:35:22.034 INFO:tasks.workunit.client.1.smithi121.stdout:8/250: symlink d0/d3/d20/l59 0 2024-02-15T18:35:22.034 INFO:tasks.workunit.client.1.smithi121.stdout:0/299: read d5/d6/f11 [5833422,126535] 0 2024-02-15T18:35:22.034 INFO:tasks.workunit.client.1.smithi121.stdout:0/300: chown d5/l21 1204716 1 2024-02-15T18:35:22.037 INFO:tasks.workunit.client.1.smithi121.stdout:8/251: unlink d0/d4a/c32 0 2024-02-15T18:35:22.038 INFO:tasks.workunit.client.1.smithi121.stdout:1/207: dread d0/d6/d7/d9/d14/f2f [0,4194304] 0 2024-02-15T18:35:22.038 INFO:tasks.workunit.client.1.smithi121.stdout:0/301: symlink d5/d6/d2a/d36/d44/l65 0 2024-02-15T18:35:22.038 INFO:tasks.workunit.client.1.smithi121.stdout:0/302: write d5/d6/dd/f2d [744978,61564] 0 2024-02-15T18:35:22.041 INFO:tasks.workunit.client.1.smithi121.stdout:8/252: unlink d0/d7/de/c2b 0 2024-02-15T18:35:22.044 INFO:tasks.workunit.client.1.smithi121.stdout:1/208: symlink d0/d39/l46 0 2024-02-15T18:35:22.044 INFO:tasks.workunit.client.1.smithi121.stdout:8/253: symlink d0/d7/d3f/d45/l5a 0 2024-02-15T18:35:22.046 INFO:tasks.workunit.client.1.smithi121.stdout:1/209: creat d0/f47 x:0 0 0 2024-02-15T18:35:22.047 INFO:tasks.workunit.client.1.smithi121.stdout:8/254: creat d0/d7/d36/d52/f5b x:0 0 0 2024-02-15T18:35:22.049 INFO:tasks.workunit.client.1.smithi121.stdout:8/255: creat d0/d7/d36/f5c x:0 0 0 2024-02-15T18:35:22.052 INFO:tasks.workunit.client.1.smithi121.stdout:8/256: dread d0/f2 [0,4194304] 0 2024-02-15T18:35:22.052 INFO:tasks.workunit.client.1.smithi121.stdout:0/303: dwrite d5/d6/d2a/d4e/f5b [0,4194304] 0 2024-02-15T18:35:22.055 INFO:tasks.workunit.client.1.smithi121.stdout:8/257: mknod d0/d7/d3f/d45/d53/c5d 0 2024-02-15T18:35:22.055 INFO:tasks.workunit.client.1.smithi121.stdout:8/258: chown d0/d7/d3f/d45/d53/c5d 11 1 2024-02-15T18:35:22.058 INFO:tasks.workunit.client.1.smithi121.stdout:8/259: rename d0/d7/fa to d0/d7/f5e 0 2024-02-15T18:35:22.058 INFO:tasks.workunit.client.1.smithi121.stdout:8/260: chown d0/d7/de/f1c 50048 1 2024-02-15T18:35:22.058 INFO:tasks.workunit.client.1.smithi121.stdout:0/304: dread d5/d6/dd/f2d [0,4194304] 0 2024-02-15T18:35:22.063 INFO:tasks.workunit.client.1.smithi121.stdout:8/261: creat d0/d4a/f5f x:0 0 0 2024-02-15T18:35:22.065 INFO:tasks.workunit.client.1.smithi121.stdout:1/210: dwrite d0/d6/d7/d9/f2d [4194304,4194304] 0 2024-02-15T18:35:22.068 INFO:tasks.workunit.client.1.smithi121.stdout:0/305: truncate d5/d30/d5d/f5e 443820 0 2024-02-15T18:35:22.075 INFO:tasks.workunit.client.1.smithi121.stdout:2/84: sync 2024-02-15T18:35:22.075 INFO:tasks.workunit.client.1.smithi121.stdout:3/195: sync 2024-02-15T18:35:22.075 INFO:tasks.workunit.client.1.smithi121.stdout:3/196: write d2/f36 [4926865,91344] 0 2024-02-15T18:35:22.075 INFO:tasks.workunit.client.1.smithi121.stdout:1/211: mkdir d0/d6/d7/d9/d14/d48 0 2024-02-15T18:35:22.077 INFO:tasks.workunit.client.1.smithi121.stdout:3/197: read d2/f7 [1945420,88493] 0 2024-02-15T18:35:22.077 INFO:tasks.workunit.client.1.smithi121.stdout:3/198: fdatasync d2/d1a/f37 0 2024-02-15T18:35:22.078 INFO:tasks.workunit.client.1.smithi121.stdout:3/199: write d2/d1a/d22/f2b [1048420,98822] 0 2024-02-15T18:35:22.078 INFO:tasks.workunit.client.1.smithi121.stdout:3/200: write d2/d1a/d27/f29 [4800153,59728] 0 2024-02-15T18:35:22.079 INFO:tasks.workunit.client.1.smithi121.stdout:0/306: unlink d5/d30/d3c/f46 0 2024-02-15T18:35:22.080 INFO:tasks.workunit.client.1.smithi121.stdout:1/212: dread d0/d6/f1a [0,4194304] 0 2024-02-15T18:35:22.083 INFO:tasks.workunit.client.1.smithi121.stdout:2/85: creat d5/df/d10/d15/f1c x:0 0 0 2024-02-15T18:35:22.083 INFO:tasks.workunit.client.1.smithi121.stdout:2/86: write d5/f8 [1248597,17910] 0 2024-02-15T18:35:22.083 INFO:tasks.workunit.client.1.smithi121.stdout:2/87: write d5/df/d10/f1a [673482,62074] 0 2024-02-15T18:35:22.083 INFO:tasks.workunit.client.1.smithi121.stdout:2/88: write d5/df/d10/d15/f1c [247997,102094] 0 2024-02-15T18:35:22.086 INFO:tasks.workunit.client.1.smithi121.stdout:0/307: mknod d5/d6/dd/c66 0 2024-02-15T18:35:22.087 INFO:tasks.workunit.client.1.smithi121.stdout:1/213: symlink d0/d6/d7/d9/d14/l49 0 2024-02-15T18:35:22.088 INFO:tasks.workunit.client.1.smithi121.stdout:8/262: dwrite d0/d4a/f5f [0,4194304] 0 2024-02-15T18:35:22.107 INFO:tasks.workunit.client.1.smithi121.stdout:2/89: dwrite d5/f8 [0,4194304] 0 2024-02-15T18:35:22.107 INFO:tasks.workunit.client.1.smithi121.stdout:2/90: fsync d5/df/f1b 0 2024-02-15T18:35:22.107 INFO:tasks.workunit.client.1.smithi121.stdout:2/91: stat d5/df/d10/d15/f1c 0 2024-02-15T18:35:22.111 INFO:tasks.workunit.client.1.smithi121.stdout:0/308: truncate d5/d6/f11 8199706 0 2024-02-15T18:35:22.111 INFO:tasks.workunit.client.1.smithi121.stdout:0/309: chown d5/l53 1012 1 2024-02-15T18:35:22.115 INFO:tasks.workunit.client.1.smithi121.stdout:8/263: link d0/l42 d0/d7/d36/l60 0 2024-02-15T18:35:22.116 INFO:tasks.workunit.client.1.smithi121.stdout:1/214: dwrite d0/d6/d7/d9/f1e [0,4194304] 0 2024-02-15T18:35:22.116 INFO:tasks.workunit.client.1.smithi121.stdout:3/201: truncate d2/d1a/d27/f29 4463118 0 2024-02-15T18:35:22.116 INFO:tasks.workunit.client.1.smithi121.stdout:1/215: write d0/d6/d7/f26 [3637867,83528] 0 2024-02-15T18:35:22.131 INFO:tasks.workunit.client.1.smithi121.stdout:2/92: rmdir d5/df/d10/d15/d18/d19 0 2024-02-15T18:35:22.134 INFO:tasks.workunit.client.1.smithi121.stdout:0/310: mknod d5/d6/d2a/d36/d44/c67 0 2024-02-15T18:35:22.134 INFO:tasks.workunit.client.1.smithi121.stdout:0/311: write d5/d6/dd/f34 [1916340,53570] 0 2024-02-15T18:35:22.134 INFO:tasks.workunit.client.1.smithi121.stdout:0/312: fdatasync d5/f27 0 2024-02-15T18:35:22.135 INFO:tasks.workunit.client.1.smithi121.stdout:2/93: write d5/fd [384239,99604] 0 2024-02-15T18:35:22.136 INFO:tasks.workunit.client.1.smithi121.stdout:2/94: chown f4 62 1 2024-02-15T18:35:22.141 INFO:tasks.workunit.client.1.smithi121.stdout:8/264: stat d0/l24 0 2024-02-15T18:35:22.143 INFO:tasks.workunit.client.1.smithi121.stdout:3/202: mknod d2/d1a/d27/d2c/c45 0 2024-02-15T18:35:22.145 INFO:tasks.workunit.client.1.smithi121.stdout:1/216: creat d0/d6/d7/d9/d14/d3e/f4a x:0 0 0 2024-02-15T18:35:22.146 INFO:tasks.workunit.client.1.smithi121.stdout:0/313: creat d5/d6/dd/f68 x:0 0 0 2024-02-15T18:35:22.149 INFO:tasks.workunit.client.1.smithi121.stdout:2/95: symlink d5/df/d10/l1d 0 2024-02-15T18:35:22.149 INFO:tasks.workunit.client.1.smithi121.stdout:2/96: fdatasync d5/f8 0 2024-02-15T18:35:22.164 INFO:tasks.workunit.client.1.smithi121.stdout:8/265: mkdir d0/d61 0 2024-02-15T18:35:22.164 INFO:tasks.workunit.client.1.smithi121.stdout:8/266: chown d0/d3/d20/d39/l44 9 1 2024-02-15T18:35:22.165 INFO:tasks.workunit.client.1.smithi121.stdout:8/267: dread - d0/d7/d36/d52/f5b zero size 2024-02-15T18:35:22.166 INFO:tasks.workunit.client.1.smithi121.stdout:3/203: link d2/d1a/d27/f2d d2/d1a/d27/d2c/d25/d30/f46 0 2024-02-15T18:35:22.166 INFO:tasks.workunit.client.1.smithi121.stdout:3/204: fsync d2/d1a/d27/d2c/f1b 0 2024-02-15T18:35:22.167 INFO:tasks.workunit.client.1.smithi121.stdout:1/217: mkdir d0/d39/d4b 0 2024-02-15T18:35:22.170 INFO:tasks.workunit.client.1.smithi121.stdout:1/218: write d0/d6/d7/f27 [3645851,49191] 0 2024-02-15T18:35:22.170 INFO:tasks.workunit.client.1.smithi121.stdout:1/219: write d0/d6/d7/d9/f20 [4842442,89715] 0 2024-02-15T18:35:22.175 INFO:tasks.workunit.client.1.smithi121.stdout:0/314: creat d5/d6/dd/f69 x:0 0 0 2024-02-15T18:35:22.175 INFO:tasks.workunit.client.1.smithi121.stdout:3/205: creat d2/d1a/d22/f47 x:0 0 0 2024-02-15T18:35:22.175 INFO:tasks.workunit.client.1.smithi121.stdout:2/97: mknod d5/df/d10/d15/c1e 0 2024-02-15T18:35:22.175 INFO:tasks.workunit.client.1.smithi121.stdout:8/268: mknod d0/d2a/c62 0 2024-02-15T18:35:22.176 INFO:tasks.workunit.client.1.smithi121.stdout:8/269: write d0/f37 [1207975,24797] 0 2024-02-15T18:35:22.177 INFO:tasks.workunit.client.1.smithi121.stdout:1/220: creat d0/d6/d7/f4c x:0 0 0 2024-02-15T18:35:22.177 INFO:tasks.workunit.client.1.smithi121.stdout:1/221: write d0/d6/d7/f27 [4215268,119605] 0 2024-02-15T18:35:22.177 INFO:tasks.workunit.client.1.smithi121.stdout:0/315: creat d5/f6a x:0 0 0 2024-02-15T18:35:22.179 INFO:tasks.workunit.client.1.smithi121.stdout:3/206: mknod d2/d1a/c48 0 2024-02-15T18:35:22.180 INFO:tasks.workunit.client.1.smithi121.stdout:3/207: truncate d2/d1a/d27/d2c/d25/d31/f43 121984 0 2024-02-15T18:35:22.180 INFO:tasks.workunit.client.1.smithi121.stdout:3/208: fsync d2/d1a/d27/d2c/f1b 0 2024-02-15T18:35:22.180 INFO:tasks.workunit.client.1.smithi121.stdout:2/98: mknod d5/df/d17/c1f 0 2024-02-15T18:35:22.180 INFO:tasks.workunit.client.1.smithi121.stdout:2/99: chown d5/df/d10/l1d 6138474 1 2024-02-15T18:35:22.183 INFO:tasks.workunit.client.1.smithi121.stdout:8/270: rename d0/d3/c23 to d0/d2a/d51/c63 0 2024-02-15T18:35:22.184 INFO:tasks.workunit.client.1.smithi121.stdout:2/100: symlink d5/df/l20 0 2024-02-15T18:35:22.184 INFO:tasks.workunit.client.1.smithi121.stdout:2/101: write d5/df/f1b [144219,23965] 0 2024-02-15T18:35:22.184 INFO:tasks.workunit.client.1.smithi121.stdout:8/271: creat d0/d3/f64 x:0 0 0 2024-02-15T18:35:22.188 INFO:tasks.workunit.client.1.smithi121.stdout:3/209: dread d2/d1a/d27/d2c/f19 [0,4194304] 0 2024-02-15T18:35:22.189 INFO:tasks.workunit.client.1.smithi121.stdout:2/102: creat d5/df/d17/f21 x:0 0 0 2024-02-15T18:35:22.197 INFO:tasks.workunit.client.1.smithi121.stdout:2/103: dread d5/df/f1b [0,4194304] 0 2024-02-15T18:35:22.200 INFO:tasks.workunit.client.1.smithi121.stdout:2/104: write d5/df/f16 [228820,47507] 0 2024-02-15T18:35:22.205 INFO:tasks.workunit.client.1.smithi121.stdout:2/105: mkdir d5/df/d10/d15/d22 0 2024-02-15T18:35:22.207 INFO:tasks.workunit.client.1.smithi121.stdout:2/106: dread d5/df/d10/f11 [0,4194304] 0 2024-02-15T18:35:22.208 INFO:tasks.workunit.client.1.smithi121.stdout:2/107: write d5/fd [2833048,62140] 0 2024-02-15T18:35:22.208 INFO:tasks.workunit.client.1.smithi121.stdout:2/108: fdatasync d5/fa 0 2024-02-15T18:35:22.208 INFO:tasks.workunit.client.1.smithi121.stdout:2/109: truncate d5/df/d10/f1a 1367758 0 2024-02-15T18:35:22.209 INFO:tasks.workunit.client.1.smithi121.stdout:2/110: write d5/fa [605150,37521] 0 2024-02-15T18:35:22.209 INFO:tasks.workunit.client.1.smithi121.stdout:2/111: fdatasync d5/df/d17/f21 0 2024-02-15T18:35:22.211 INFO:tasks.workunit.client.1.smithi121.stdout:2/112: unlink d5/fa 0 2024-02-15T18:35:22.212 INFO:tasks.workunit.client.1.smithi121.stdout:2/113: mknod d5/c23 0 2024-02-15T18:35:22.212 INFO:tasks.workunit.client.1.smithi121.stdout:2/114: truncate d5/fc 525909 0 2024-02-15T18:35:22.257 INFO:tasks.workunit.client.1.smithi121.stdout:8/272: dwrite d0/d2a/d51/f38 [0,4194304] 0 2024-02-15T18:35:22.257 INFO:tasks.workunit.client.1.smithi121.stdout:8/273: chown d0/d4a 27 1 2024-02-15T18:35:22.257 INFO:tasks.workunit.client.1.smithi121.stdout:8/274: readlink d0/d2a/d51/l30 0 2024-02-15T18:35:22.264 INFO:tasks.workunit.client.1.smithi121.stdout:3/210: dwrite d2/d1a/d27/d2c/d25/d31/f43 [0,4194304] 0 2024-02-15T18:35:22.267 INFO:tasks.workunit.client.1.smithi121.stdout:8/275: dread d0/d7/f10 [0,4194304] 0 2024-02-15T18:35:22.267 INFO:tasks.workunit.client.1.smithi121.stdout:8/276: fsync d0/f8 0 2024-02-15T18:35:22.269 INFO:tasks.workunit.client.1.smithi121.stdout:2/115: getdents d5 0 2024-02-15T18:35:22.281 INFO:tasks.workunit.client.1.smithi121.stdout:3/211: link d2/d1a/d22/c28 d2/d1a/d27/d2c/d25/d31/d34/c49 0 2024-02-15T18:35:22.283 INFO:tasks.workunit.client.1.smithi121.stdout:3/212: write d2/f40 [959794,89906] 0 2024-02-15T18:35:22.286 INFO:tasks.workunit.client.1.smithi121.stdout:8/277: creat d0/d7/de/f65 x:0 0 0 2024-02-15T18:35:22.289 INFO:tasks.workunit.client.1.smithi121.stdout:8/278: unlink d0/l42 0 2024-02-15T18:35:22.294 INFO:tasks.workunit.client.1.smithi121.stdout:8/279: rename d0/d3/d20/d39 to d0/d61/d66 0 2024-02-15T18:35:22.297 INFO:tasks.workunit.client.1.smithi121.stdout:3/213: rmdir d2/d1a/d22 39 2024-02-15T18:35:22.301 INFO:tasks.workunit.client.1.smithi121.stdout:3/214: mkdir d2/d1a/d27/d2c/d25/d31/d34/d4a 0 2024-02-15T18:35:22.303 INFO:tasks.workunit.client.1.smithi121.stdout:3/215: dread d2/d1a/d27/d2c/d25/d30/f46 [0,4194304] 0 2024-02-15T18:35:22.303 INFO:tasks.workunit.client.1.smithi121.stdout:3/216: dread - d2/d1a/d22/f35 zero size 2024-02-15T18:35:22.312 INFO:tasks.workunit.client.1.smithi121.stdout:8/280: dwrite d0/d7/de/f1a [0,4194304] 0 2024-02-15T18:35:22.312 INFO:tasks.workunit.client.1.smithi121.stdout:8/281: write d0/d7/f1e [4458149,38004] 0 2024-02-15T18:35:22.313 INFO:tasks.workunit.client.1.smithi121.stdout:9/108: sync 2024-02-15T18:35:22.313 INFO:tasks.workunit.client.1.smithi121.stdout:9/109: chown d7/d8 3601 1 2024-02-15T18:35:22.313 INFO:tasks.workunit.client.1.smithi121.stdout:9/110: chown f5 63159 1 2024-02-15T18:35:22.313 INFO:tasks.workunit.client.1.smithi121.stdout:3/217: dread d2/d1a/d27/f29 [0,4194304] 0 2024-02-15T18:35:22.314 INFO:tasks.workunit.client.1.smithi121.stdout:9/111: rmdir d7 39 2024-02-15T18:35:22.315 INFO:tasks.workunit.client.1.smithi121.stdout:3/218: creat d2/d1a/d27/f4b x:0 0 0 2024-02-15T18:35:22.317 INFO:tasks.workunit.client.1.smithi121.stdout:9/112: mkdir d7/d2b 0 2024-02-15T18:35:22.318 INFO:tasks.workunit.client.1.smithi121.stdout:3/219: symlink d2/d1a/d27/d2c/l4c 0 2024-02-15T18:35:22.318 INFO:tasks.workunit.client.1.smithi121.stdout:3/220: truncate d2/d1a/f1f 2581691 0 2024-02-15T18:35:22.318 INFO:tasks.workunit.client.1.smithi121.stdout:3/221: stat d2/d1a/d22/c23 0 2024-02-15T18:35:22.319 INFO:tasks.workunit.client.1.smithi121.stdout:9/113: symlink d7/d8/d18/l2c 0 2024-02-15T18:35:22.319 INFO:tasks.workunit.client.1.smithi121.stdout:9/114: readlink d7/l27 0 2024-02-15T18:35:22.319 INFO:tasks.workunit.client.1.smithi121.stdout:8/282: dread d0/d7/f10 [0,4194304] 0 2024-02-15T18:35:22.321 INFO:tasks.workunit.client.1.smithi121.stdout:3/222: mknod d2/c4d 0 2024-02-15T18:35:22.321 INFO:tasks.workunit.client.1.smithi121.stdout:3/223: dread - d2/d1a/d22/f42 zero size 2024-02-15T18:35:22.322 INFO:tasks.workunit.client.1.smithi121.stdout:8/283: mkdir d0/d7/de/d67 0 2024-02-15T18:35:22.325 INFO:tasks.workunit.client.1.smithi121.stdout:8/284: write d0/d7/de/f1a [1765374,109831] 0 2024-02-15T18:35:22.326 INFO:tasks.workunit.client.1.smithi121.stdout:3/224: creat d2/d1a/d27/d2c/d25/d31/f4e x:0 0 0 2024-02-15T18:35:22.328 INFO:tasks.workunit.client.1.smithi121.stdout:9/115: dread d7/d8/f17 [0,4194304] 0 2024-02-15T18:35:22.328 INFO:tasks.workunit.client.1.smithi121.stdout:8/285: symlink d0/d7/d36/l68 0 2024-02-15T18:35:22.330 INFO:tasks.workunit.client.1.smithi121.stdout:8/286: link d0/d3/c17 d0/d7/c69 0 2024-02-15T18:35:22.331 INFO:tasks.workunit.client.1.smithi121.stdout:8/287: creat d0/d48/f6a x:0 0 0 2024-02-15T18:35:22.337 INFO:tasks.workunit.client.1.smithi121.stdout:9/116: dread d7/d8/d18/f19 [0,4194304] 0 2024-02-15T18:35:22.338 INFO:tasks.workunit.client.1.smithi121.stdout:9/117: rename d7/l22 to d7/l2d 0 2024-02-15T18:35:22.339 INFO:tasks.workunit.client.1.smithi121.stdout:9/118: fdatasync d7/d8/f15 0 2024-02-15T18:35:22.339 INFO:tasks.workunit.client.1.smithi121.stdout:9/119: read - d7/d8/f1d zero size 2024-02-15T18:35:22.339 INFO:tasks.workunit.client.1.smithi121.stdout:8/288: dread d0/d7/d3f/d45/f3a [0,4194304] 0 2024-02-15T18:35:22.342 INFO:tasks.workunit.client.1.smithi121.stdout:8/289: rename d0/d7/f35 to d0/d2a/d51/f6b 0 2024-02-15T18:35:22.342 INFO:tasks.workunit.client.1.smithi121.stdout:8/290: fsync d0/d7/d3f/d45/f3e 0 2024-02-15T18:35:22.344 INFO:tasks.workunit.client.1.smithi121.stdout:3/225: dwrite d2/d1a/d27/d2c/d25/d30/f39 [0,4194304] 0 2024-02-15T18:35:22.344 INFO:tasks.workunit.client.1.smithi121.stdout:3/226: read - d2/d1a/d22/f47 zero size 2024-02-15T18:35:22.344 INFO:tasks.workunit.client.1.smithi121.stdout:3/227: dread - d2/d1a/d27/d2c/d25/d30/f41 zero size 2024-02-15T18:35:22.344 INFO:tasks.workunit.client.1.smithi121.stdout:3/228: write d2/d1a/d22/f47 [521496,105583] 0 2024-02-15T18:35:22.344 INFO:tasks.workunit.client.1.smithi121.stdout:3/229: chown d2/l20 1615 1 2024-02-15T18:35:22.345 INFO:tasks.workunit.client.1.smithi121.stdout:3/230: stat d2/d1a/l21 0 2024-02-15T18:35:22.345 INFO:tasks.workunit.client.1.smithi121.stdout:8/291: unlink d0/d2a/d51/l2f 0 2024-02-15T18:35:22.347 INFO:tasks.workunit.client.1.smithi121.stdout:9/120: dread d7/d8/f10 [0,4194304] 0 2024-02-15T18:35:22.348 INFO:tasks.workunit.client.1.smithi121.stdout:3/231: symlink d2/d1a/d27/d2c/d25/d31/l4f 0 2024-02-15T18:35:22.349 INFO:tasks.workunit.client.1.smithi121.stdout:8/292: symlink d0/d7/d3f/d45/l6c 0 2024-02-15T18:35:22.349 INFO:tasks.workunit.client.1.smithi121.stdout:9/121: write d7/f1b [3590054,108750] 0 2024-02-15T18:35:22.349 INFO:tasks.workunit.client.1.smithi121.stdout:9/122: readlink d7/ld 0 2024-02-15T18:35:22.349 INFO:tasks.workunit.client.1.smithi121.stdout:9/123: chown d7/d8/d18/l21 30674 1 2024-02-15T18:35:22.349 INFO:tasks.workunit.client.1.smithi121.stdout:9/124: truncate d7/d8/f1d 157678 0 2024-02-15T18:35:22.351 INFO:tasks.workunit.client.1.smithi121.stdout:3/232: dread d2/d1a/d27/d2c/f19 [0,4194304] 0 2024-02-15T18:35:22.359 INFO:tasks.workunit.client.1.smithi121.stdout:8/293: link d0/d7/d36/d52/f5b d0/d61/d66/f6d 0 2024-02-15T18:35:22.360 INFO:tasks.workunit.client.1.smithi121.stdout:3/233: creat d2/d1a/d27/d2c/d25/d31/f50 x:0 0 0 2024-02-15T18:35:22.366 INFO:tasks.workunit.client.1.smithi121.stdout:8/294: dread d0/d7/d3f/d45/f2c [4194304,4194304] 0 2024-02-15T18:35:22.369 INFO:tasks.workunit.client.1.smithi121.stdout:3/234: truncate d2/d1a/d27/d2c/f1b 4669853 0 2024-02-15T18:35:22.369 INFO:tasks.workunit.client.1.smithi121.stdout:3/235: chown d2/cb 154 1 2024-02-15T18:35:22.370 INFO:tasks.workunit.client.1.smithi121.stdout:8/295: creat d0/d2a/f6e x:0 0 0 2024-02-15T18:35:22.371 INFO:tasks.workunit.client.1.smithi121.stdout:3/236: creat d2/d1a/d22/f51 x:0 0 0 2024-02-15T18:35:22.373 INFO:tasks.workunit.client.1.smithi121.stdout:8/296: creat d0/d2a/d51/f6f x:0 0 0 2024-02-15T18:35:22.374 INFO:tasks.workunit.client.1.smithi121.stdout:8/297: symlink d0/d61/l70 0 2024-02-15T18:35:22.393 INFO:tasks.workunit.client.1.smithi121.stdout:3/237: dwrite d2/f9 [0,4194304] 0 2024-02-15T18:35:22.397 INFO:tasks.workunit.client.1.smithi121.stdout:8/298: dwrite d0/d61/d66/f3b [4194304,4194304] 0 2024-02-15T18:35:22.400 INFO:tasks.workunit.client.1.smithi121.stdout:5/113: sync 2024-02-15T18:35:22.400 INFO:tasks.workunit.client.1.smithi121.stdout:6/232: sync 2024-02-15T18:35:22.402 INFO:tasks.workunit.client.1.smithi121.stdout:7/152: sync 2024-02-15T18:35:22.402 INFO:tasks.workunit.client.1.smithi121.stdout:4/131: sync 2024-02-15T18:35:22.403 INFO:tasks.workunit.client.1.smithi121.stdout:7/153: stat d0/d33/f34 0 2024-02-15T18:35:22.403 INFO:tasks.workunit.client.1.smithi121.stdout:7/154: chown d0/d5/d11/c25 188466 1 2024-02-15T18:35:22.404 INFO:tasks.workunit.client.1.smithi121.stdout:8/299: rename d0/d2a/d51/c63 to d0/d2a/d51/c71 0 2024-02-15T18:35:22.413 INFO:tasks.workunit.client.1.smithi121.stdout:3/238: dwrite d2/f9 [0,4194304] 0 2024-02-15T18:35:22.429 INFO:tasks.workunit.client.1.smithi121.stdout:7/155: dwrite d0/f18 [0,4194304] 0 2024-02-15T18:35:22.429 INFO:tasks.workunit.client.1.smithi121.stdout:3/239: dwrite d2/d1a/d27/f2d [0,4194304] 0 2024-02-15T18:35:22.429 INFO:tasks.workunit.client.1.smithi121.stdout:3/240: chown d2/f7 24 1 2024-02-15T18:35:22.450 INFO:tasks.workunit.client.1.smithi121.stdout:7/156: dwrite d0/f27 [4194304,4194304] 0 2024-02-15T18:35:22.512 INFO:tasks.workunit.client.1.smithi121.stdout:6/233: mkdir d7/dc/d55 0 2024-02-15T18:35:22.513 INFO:tasks.workunit.client.1.smithi121.stdout:6/234: chown d7/c29 50001 1 2024-02-15T18:35:22.515 INFO:tasks.workunit.client.1.smithi121.stdout:5/114: creat d0/f22 x:0 0 0 2024-02-15T18:35:22.516 INFO:tasks.workunit.client.1.smithi121.stdout:4/132: mkdir d5/d9/d19/d2a 0 2024-02-15T18:35:22.516 INFO:tasks.workunit.client.1.smithi121.stdout:4/133: readlink d5/d7/l16 0 2024-02-15T18:35:22.517 INFO:tasks.workunit.client.1.smithi121.stdout:4/134: read d5/d9/d1d/f22 [3751776,54628] 0 2024-02-15T18:35:22.517 INFO:tasks.workunit.client.1.smithi121.stdout:4/135: fsync d5/f6 0 2024-02-15T18:35:22.517 INFO:tasks.workunit.client.1.smithi121.stdout:8/300: symlink d0/d7/d3f/l72 0 2024-02-15T18:35:22.517 INFO:tasks.workunit.client.1.smithi121.stdout:8/301: fdatasync d0/d7/f5e 0 2024-02-15T18:35:22.518 INFO:tasks.workunit.client.1.smithi121.stdout:8/302: fdatasync d0/d7/d3f/d45/f3a 0 2024-02-15T18:35:22.525 INFO:tasks.workunit.client.1.smithi121.stdout:3/241: creat d2/d1a/d27/d2c/d25/f52 x:0 0 0 2024-02-15T18:35:22.526 INFO:tasks.workunit.client.1.smithi121.stdout:3/242: dread d2/d1a/d27/d2c/d25/d30/f38 [0,4194304] 0 2024-02-15T18:35:22.529 INFO:tasks.workunit.client.1.smithi121.stdout:7/157: unlink d0/d5/d11/f20 0 2024-02-15T18:35:22.534 INFO:tasks.workunit.client.1.smithi121.stdout:6/235: unlink d7/dc/le 0 2024-02-15T18:35:22.535 INFO:tasks.workunit.client.1.smithi121.stdout:5/115: rmdir d0/d2 39 2024-02-15T18:35:22.535 INFO:tasks.workunit.client.1.smithi121.stdout:4/136: dwrite d5/d9/d1d/f22 [0,4194304] 0 2024-02-15T18:35:22.535 INFO:tasks.workunit.client.1.smithi121.stdout:5/116: write d0/f22 [518309,101775] 0 2024-02-15T18:35:22.536 INFO:tasks.workunit.client.1.smithi121.stdout:8/303: mkdir d0/d73 0 2024-02-15T18:35:22.543 INFO:tasks.workunit.client.1.smithi121.stdout:3/243: creat d2/d1a/d22/f53 x:0 0 0 2024-02-15T18:35:22.545 INFO:tasks.workunit.client.1.smithi121.stdout:7/158: creat d0/d5/d11/d1c/f35 x:0 0 0 2024-02-15T18:35:22.546 INFO:tasks.workunit.client.1.smithi121.stdout:4/137: mknod d5/d9/d19/d2a/c2b 0 2024-02-15T18:35:22.546 INFO:tasks.workunit.client.1.smithi121.stdout:4/138: chown d5/d7/l16 753 1 2024-02-15T18:35:22.546 INFO:tasks.workunit.client.1.smithi121.stdout:4/139: readlink d5/d7/l1c 0 2024-02-15T18:35:22.548 INFO:tasks.workunit.client.1.smithi121.stdout:8/304: rename d0/d7/f1e to d0/d7/d3f/d45/f74 0 2024-02-15T18:35:22.548 INFO:tasks.workunit.client.1.smithi121.stdout:8/305: dread - d0/d2a/f6e zero size 2024-02-15T18:35:22.549 INFO:tasks.workunit.client.1.smithi121.stdout:3/244: mkdir d2/d1a/d54 0 2024-02-15T18:35:22.551 INFO:tasks.workunit.client.1.smithi121.stdout:7/159: link d0/l2a d0/d33/l36 0 2024-02-15T18:35:22.551 INFO:tasks.workunit.client.1.smithi121.stdout:7/160: dread - d0/d5/f26 zero size 2024-02-15T18:35:22.555 INFO:tasks.workunit.client.1.smithi121.stdout:4/140: creat d5/d9/d19/d2a/f2c x:0 0 0 2024-02-15T18:35:22.558 INFO:tasks.workunit.client.1.smithi121.stdout:5/117: dwrite d0/f17 [4194304,4194304] 0 2024-02-15T18:35:22.570 INFO:tasks.workunit.client.1.smithi121.stdout:4/141: dread d5/fa [4194304,4194304] 0 2024-02-15T18:35:22.570 INFO:tasks.workunit.client.1.smithi121.stdout:4/142: dread - d5/d9/d1d/f24 zero size 2024-02-15T18:35:22.571 INFO:tasks.workunit.client.1.smithi121.stdout:4/143: write d5/d9/d19/d2a/f2c [860576,28389] 0 2024-02-15T18:35:22.571 INFO:tasks.workunit.client.1.smithi121.stdout:4/144: fdatasync d5/d7/fb 0 2024-02-15T18:35:22.589 INFO:tasks.workunit.client.1.smithi121.stdout:8/306: dwrite d0/d7/de/f1a [0,4194304] 0 2024-02-15T18:35:22.589 INFO:tasks.workunit.client.1.smithi121.stdout:8/307: readlink d0/l24 0 2024-02-15T18:35:22.589 INFO:tasks.workunit.client.1.smithi121.stdout:8/308: chown d0/d2a/f3c 2477414 1 2024-02-15T18:35:22.589 INFO:tasks.workunit.client.1.smithi121.stdout:6/236: dwrite d7/dc/f15 [4194304,4194304] 0 2024-02-15T18:35:22.589 INFO:tasks.workunit.client.1.smithi121.stdout:6/237: fsync d7/dc/f2d 0 2024-02-15T18:35:22.589 INFO:tasks.workunit.client.1.smithi121.stdout:6/238: write d7/dc/f20 [763849,61197] 0 2024-02-15T18:35:22.589 INFO:tasks.workunit.client.1.smithi121.stdout:3/245: dwrite d2/d1a/d27/d2c/f2f [0,4194304] 0 2024-02-15T18:35:22.589 INFO:tasks.workunit.client.1.smithi121.stdout:3/246: readlink d2/d1a/l24 0 2024-02-15T18:35:22.589 INFO:tasks.workunit.client.1.smithi121.stdout:3/247: write d2/d1a/d22/f47 [749054,30828] 0 2024-02-15T18:35:22.589 INFO:tasks.workunit.client.1.smithi121.stdout:7/161: dwrite d0/d5/d11/d1c/f21 [0,4194304] 0 2024-02-15T18:35:22.589 INFO:tasks.workunit.client.1.smithi121.stdout:7/162: read - d0/d5/d11/d1c/f35 zero size 2024-02-15T18:35:22.604 INFO:tasks.workunit.client.1.smithi121.stdout:8/309: mknod d0/d2a/c75 0 2024-02-15T18:35:22.605 INFO:tasks.workunit.client.1.smithi121.stdout:6/239: creat d7/dc/d2c/d36/f56 x:0 0 0 2024-02-15T18:35:22.609 INFO:tasks.workunit.client.1.smithi121.stdout:3/248: symlink d2/d1a/l55 0 2024-02-15T18:35:22.611 INFO:tasks.workunit.client.1.smithi121.stdout:6/240: rename d7/dc/d2c/c35 to d7/dc/c57 0 2024-02-15T18:35:22.611 INFO:tasks.workunit.client.1.smithi121.stdout:6/241: read d7/dc/f25 [528415,93780] 0 2024-02-15T18:35:22.612 INFO:tasks.workunit.client.1.smithi121.stdout:7/163: symlink d0/l37 0 2024-02-15T18:35:22.619 INFO:tasks.workunit.client.1.smithi121.stdout:7/164: mkdir d0/d38 0 2024-02-15T18:35:22.619 INFO:tasks.workunit.client.1.smithi121.stdout:7/165: fdatasync d0/d5/f2b 0 2024-02-15T18:35:22.619 INFO:tasks.workunit.client.1.smithi121.stdout:8/310: dwrite d0/f37 [0,4194304] 0 2024-02-15T18:35:22.622 INFO:tasks.workunit.client.1.smithi121.stdout:7/166: rename d0/c1d to d0/d38/c39 0 2024-02-15T18:35:22.624 INFO:tasks.workunit.client.1.smithi121.stdout:8/311: creat d0/d61/d66/f76 x:0 0 0 2024-02-15T18:35:22.626 INFO:tasks.workunit.client.1.smithi121.stdout:7/167: symlink d0/d5/d11/l3a 0 2024-02-15T18:35:22.626 INFO:tasks.workunit.client.1.smithi121.stdout:7/168: write d0/d5/d11/d1c/f35 [608556,53081] 0 2024-02-15T18:35:22.634 INFO:tasks.workunit.client.1.smithi121.stdout:3/249: dwrite d2/f40 [0,4194304] 0 2024-02-15T18:35:22.638 INFO:tasks.workunit.client.1.smithi121.stdout:6/242: dwrite d7/d4b/f53 [0,4194304] 0 2024-02-15T18:35:22.642 INFO:tasks.workunit.client.1.smithi121.stdout:7/169: dwrite d0/d5/f26 [0,4194304] 0 2024-02-15T18:35:22.642 INFO:tasks.workunit.client.1.smithi121.stdout:7/170: chown d0/c32 458 1 2024-02-15T18:35:22.706 INFO:tasks.workunit.client.1.smithi121.stdout:3/250: creat d2/d1a/d27/d2c/f56 x:0 0 0 2024-02-15T18:35:22.707 INFO:tasks.workunit.client.1.smithi121.stdout:3/251: write d2/f36 [1829107,21899] 0 2024-02-15T18:35:22.707 INFO:tasks.workunit.client.1.smithi121.stdout:3/252: read - d2/d1a/d22/f35 zero size 2024-02-15T18:35:22.709 INFO:tasks.workunit.client.1.smithi121.stdout:6/243: unlink d7/dc/f20 0 2024-02-15T18:35:22.709 INFO:tasks.workunit.client.1.smithi121.stdout:3/253: dread d2/d1a/d22/f47 [0,4194304] 0 2024-02-15T18:35:22.710 INFO:tasks.workunit.client.1.smithi121.stdout:3/254: fdatasync d2/d1a/d27/d2c/f56 0 2024-02-15T18:35:22.710 INFO:tasks.workunit.client.1.smithi121.stdout:3/255: fsync d2/d1a/d27/d2c/f2f 0 2024-02-15T18:35:22.717 INFO:tasks.workunit.client.1.smithi121.stdout:7/171: creat d0/d33/f3b x:0 0 0 2024-02-15T18:35:22.717 INFO:tasks.workunit.client.1.smithi121.stdout:7/172: readlink d0/l14 0 2024-02-15T18:35:22.722 INFO:tasks.workunit.client.1.smithi121.stdout:3/256: truncate d2/f7 6307167 0 2024-02-15T18:35:22.731 INFO:tasks.workunit.client.1.smithi121.stdout:6/244: link d7/d4b/f53 d7/dc/d2c/d36/f58 0 2024-02-15T18:35:22.731 INFO:tasks.workunit.client.1.smithi121.stdout:6/245: dread - d7/dc/d2c/d36/f3e zero size 2024-02-15T18:35:22.731 INFO:tasks.workunit.client.1.smithi121.stdout:6/246: truncate d7/dc/d2c/d36/f49 152130 0 2024-02-15T18:35:22.731 INFO:tasks.workunit.client.1.smithi121.stdout:7/173: mkdir d0/d38/d3c 0 2024-02-15T18:35:22.731 INFO:tasks.workunit.client.1.smithi121.stdout:7/174: stat d0/d5/d11 0 2024-02-15T18:35:22.737 INFO:tasks.workunit.client.1.smithi121.stdout:6/247: dread d7/dc/f21 [0,4194304] 0 2024-02-15T18:35:22.737 INFO:tasks.workunit.client.1.smithi121.stdout:6/248: stat d7/f47 0 2024-02-15T18:35:22.739 INFO:tasks.workunit.client.1.smithi121.stdout:6/249: unlink d7/f50 0 2024-02-15T18:35:22.739 INFO:tasks.workunit.client.1.smithi121.stdout:6/250: write d7/f47 [285327,75108] 0 2024-02-15T18:35:22.740 INFO:tasks.workunit.client.1.smithi121.stdout:7/175: dread d0/d5/f15 [0,4194304] 0 2024-02-15T18:35:22.740 INFO:tasks.workunit.client.1.smithi121.stdout:3/257: dwrite d2/d1a/d27/d2c/f56 [0,4194304] 0 2024-02-15T18:35:22.740 INFO:tasks.workunit.client.1.smithi121.stdout:3/258: chown d2/d1a/d27/d2c/d25/d31/l33 784 1 2024-02-15T18:35:22.741 INFO:tasks.workunit.client.1.smithi121.stdout:6/251: mkdir d7/d4b/d59 0 2024-02-15T18:35:22.741 INFO:tasks.workunit.client.1.smithi121.stdout:6/252: chown d7/dc/f4c 7587223 1 2024-02-15T18:35:22.741 INFO:tasks.workunit.client.1.smithi121.stdout:7/176: dread d0/d33/f34 [0,4194304] 0 2024-02-15T18:35:22.741 INFO:tasks.workunit.client.1.smithi121.stdout:7/177: stat d0/d38/d3c 0 2024-02-15T18:35:22.748 INFO:tasks.workunit.client.1.smithi121.stdout:2/116: sync 2024-02-15T18:35:22.748 INFO:tasks.workunit.client.1.smithi121.stdout:0/316: sync 2024-02-15T18:35:22.748 INFO:tasks.workunit.client.1.smithi121.stdout:0/317: readlink d5/l23 0 2024-02-15T18:35:22.748 INFO:tasks.workunit.client.1.smithi121.stdout:0/318: chown d5/d6/dd/f22 253975419 1 2024-02-15T18:35:22.748 INFO:tasks.workunit.client.1.smithi121.stdout:0/319: fdatasync d5/d6/d2a/d36/f51 0 2024-02-15T18:35:22.748 INFO:tasks.workunit.client.1.smithi121.stdout:1/222: sync 2024-02-15T18:35:22.750 INFO:tasks.workunit.client.1.smithi121.stdout:3/259: rename d2/d1a/d22/f51 to d2/d1a/d27/d2c/d25/d31/f57 0 2024-02-15T18:35:22.754 INFO:tasks.workunit.client.1.smithi121.stdout:7/178: rmdir d0/d33 39 2024-02-15T18:35:22.756 INFO:tasks.workunit.client.1.smithi121.stdout:2/117: rename d5/df/f16 to d5/df/f24 0 2024-02-15T18:35:22.757 INFO:tasks.workunit.client.1.smithi121.stdout:0/320: mkdir d5/d6b 0 2024-02-15T18:35:22.758 INFO:tasks.workunit.client.1.smithi121.stdout:3/260: mkdir d2/d1a/d27/d2c/d25/d31/d58 0 2024-02-15T18:35:22.758 INFO:tasks.workunit.client.1.smithi121.stdout:3/261: write d2/d1a/d27/d2c/d25/d31/f57 [939459,27495] 0 2024-02-15T18:35:22.759 INFO:tasks.workunit.client.1.smithi121.stdout:6/253: dwrite d7/f24 [0,4194304] 0 2024-02-15T18:35:22.759 INFO:tasks.workunit.client.1.smithi121.stdout:6/254: write d7/dc/d2c/d36/f49 [1172539,129460] 0 2024-02-15T18:35:22.760 INFO:tasks.workunit.client.1.smithi121.stdout:0/321: unlink d5/d6/dd/l26 0 2024-02-15T18:35:22.760 INFO:tasks.workunit.client.1.smithi121.stdout:0/322: chown d5/l23 5216709 1 2024-02-15T18:35:22.761 INFO:tasks.workunit.client.1.smithi121.stdout:2/118: write d5/f14 [1210136,9118] 0 2024-02-15T18:35:22.762 INFO:tasks.workunit.client.1.smithi121.stdout:3/262: creat d2/d1a/d54/f59 x:0 0 0 2024-02-15T18:35:22.763 INFO:tasks.workunit.client.1.smithi121.stdout:6/255: symlink d7/dc/d2c/l5a 0 2024-02-15T18:35:22.763 INFO:tasks.workunit.client.1.smithi121.stdout:6/256: fdatasync d7/dc/f48 0 2024-02-15T18:35:22.763 INFO:tasks.workunit.client.1.smithi121.stdout:6/257: write d7/dc/d2c/d36/f3c [549827,16797] 0 2024-02-15T18:35:22.764 INFO:tasks.workunit.client.1.smithi121.stdout:0/323: symlink d5/d6/d2a/d36/d44/l6c 0 2024-02-15T18:35:22.764 INFO:tasks.workunit.client.1.smithi121.stdout:1/223: dwrite d0/d6/f1a [0,4194304] 0 2024-02-15T18:35:22.766 INFO:tasks.workunit.client.1.smithi121.stdout:3/263: write d2/d1a/d27/d2c/d25/d30/f46 [3389138,122965] 0 2024-02-15T18:35:22.767 INFO:tasks.workunit.client.1.smithi121.stdout:7/179: dread d0/d5/f26 [0,4194304] 0 2024-02-15T18:35:22.769 INFO:tasks.workunit.client.1.smithi121.stdout:7/180: dread d0/d33/f34 [0,4194304] 0 2024-02-15T18:35:22.775 INFO:tasks.workunit.client.1.smithi121.stdout:6/258: rename d7/cb to d7/d4b/d59/c5b 0 2024-02-15T18:35:22.775 INFO:tasks.workunit.client.1.smithi121.stdout:1/224: rename d0/d6 to d0/d6/d7/d9/d14/d4d 22 2024-02-15T18:35:22.775 INFO:tasks.workunit.client.1.smithi121.stdout:2/119: unlink d5/df/d10/f1a 0 2024-02-15T18:35:22.777 INFO:tasks.workunit.client.1.smithi121.stdout:0/324: symlink d5/d30/d5d/l6d 0 2024-02-15T18:35:22.778 INFO:tasks.workunit.client.1.smithi121.stdout:0/325: write d5/d6/d2a/f2f [3766137,46125] 0 2024-02-15T18:35:22.783 INFO:tasks.workunit.client.1.smithi121.stdout:6/259: rename d7/dc/f1c to d7/dc/d2c/f5c 0 2024-02-15T18:35:22.783 INFO:tasks.workunit.client.1.smithi121.stdout:6/260: chown d7/dc/c19 1615489 1 2024-02-15T18:35:22.783 INFO:tasks.workunit.client.1.smithi121.stdout:6/261: readlink d7/dc/d2c/d36/l4f 0 2024-02-15T18:35:22.783 INFO:tasks.workunit.client.1.smithi121.stdout:6/262: truncate d7/f9 9782897 0 2024-02-15T18:35:22.784 INFO:tasks.workunit.client.1.smithi121.stdout:2/120: dread d5/df/f24 [0,4194304] 0 2024-02-15T18:35:22.784 INFO:tasks.workunit.client.1.smithi121.stdout:1/225: mkdir d0/d6/d7/d4e 0 2024-02-15T18:35:22.785 INFO:tasks.workunit.client.1.smithi121.stdout:1/226: dread - d0/d6/d7/d9/d14/d3e/f43 zero size 2024-02-15T18:35:22.785 INFO:tasks.workunit.client.1.smithi121.stdout:0/326: rmdir d5/d6/dd 39 2024-02-15T18:35:22.787 INFO:tasks.workunit.client.1.smithi121.stdout:0/327: read d5/d6/dd/f47 [119780,6294] 0 2024-02-15T18:35:22.788 INFO:tasks.workunit.client.1.smithi121.stdout:0/328: chown d5/f40 103362738 1 2024-02-15T18:35:22.789 INFO:tasks.workunit.client.1.smithi121.stdout:6/263: rename d7/dc/d34/f45 to d7/f5d 0 2024-02-15T18:35:22.789 INFO:tasks.workunit.client.1.smithi121.stdout:6/264: truncate d7/f47 837891 0 2024-02-15T18:35:22.790 INFO:tasks.workunit.client.1.smithi121.stdout:2/121: truncate d5/fc 178345 0 2024-02-15T18:35:22.792 INFO:tasks.workunit.client.1.smithi121.stdout:6/265: creat d7/dc/f5e x:0 0 0 2024-02-15T18:35:22.796 INFO:tasks.workunit.client.1.smithi121.stdout:6/266: rename d7/f9 to d7/dc/d55/f5f 0 2024-02-15T18:35:22.796 INFO:tasks.workunit.client.1.smithi121.stdout:6/267: fsync d7/dc/f21 0 2024-02-15T18:35:22.800 INFO:tasks.workunit.client.1.smithi121.stdout:6/268: mkdir d7/d4b/d60 0 2024-02-15T18:35:22.800 INFO:tasks.workunit.client.1.smithi121.stdout:6/269: readlink d7/dc/d2c/d36/l3f 0 2024-02-15T18:35:22.808 INFO:tasks.workunit.client.1.smithi121.stdout:6/270: symlink d7/dc/d34/d42/l61 0 2024-02-15T18:35:22.817 INFO:tasks.workunit.client.1.smithi121.stdout:1/227: dwrite d0/d6/d7/d9/d14/d3e/f4a [0,4194304] 0 2024-02-15T18:35:22.818 INFO:tasks.workunit.client.1.smithi121.stdout:2/122: dwrite d5/f14 [0,4194304] 0 2024-02-15T18:35:22.818 INFO:tasks.workunit.client.1.smithi121.stdout:2/123: stat d5/fc 0 2024-02-15T18:35:22.820 INFO:tasks.workunit.client.1.smithi121.stdout:0/329: dwrite d5/d6/dd/f47 [0,4194304] 0 2024-02-15T18:35:22.820 INFO:tasks.workunit.client.1.smithi121.stdout:0/330: chown d5/d6/dd/f32 2522 1 2024-02-15T18:35:22.821 INFO:tasks.workunit.client.1.smithi121.stdout:1/228: symlink d0/d6/d7/d29/l4f 0 2024-02-15T18:35:22.821 INFO:tasks.workunit.client.1.smithi121.stdout:1/229: dread - d0/d6/d7/d9/d14/f3b zero size 2024-02-15T18:35:22.822 INFO:tasks.workunit.client.1.smithi121.stdout:1/230: write d0/d6/d7/d9/d14/f2f [5960208,11396] 0 2024-02-15T18:35:22.829 INFO:tasks.workunit.client.1.smithi121.stdout:2/124: truncate d5/df/d10/d15/f1c 215970 0 2024-02-15T18:35:22.834 INFO:tasks.workunit.client.1.smithi121.stdout:1/231: dread d0/d6/f1a [0,4194304] 0 2024-02-15T18:35:22.834 INFO:tasks.workunit.client.1.smithi121.stdout:1/232: read - d0/d6/d7/d9/d14/f3b zero size 2024-02-15T18:35:22.834 INFO:tasks.workunit.client.1.smithi121.stdout:1/233: dread - d0/f47 zero size 2024-02-15T18:35:22.844 INFO:tasks.workunit.client.1.smithi121.stdout:9/125: sync 2024-02-15T18:35:22.845 INFO:tasks.workunit.client.1.smithi121.stdout:5/118: sync 2024-02-15T18:35:22.845 INFO:tasks.workunit.client.1.smithi121.stdout:4/145: sync 2024-02-15T18:35:22.845 INFO:tasks.workunit.client.1.smithi121.stdout:0/331: mknod d5/d6b/c6e 0 2024-02-15T18:35:22.851 INFO:tasks.workunit.client.1.smithi121.stdout:1/234: rmdir d0/d6/d7/d9/d14 39 2024-02-15T18:35:22.853 INFO:tasks.workunit.client.1.smithi121.stdout:0/332: dread d5/d6/d2a/d4e/f5a [0,4194304] 0 2024-02-15T18:35:22.855 INFO:tasks.workunit.client.1.smithi121.stdout:9/126: symlink d7/l2e 0 2024-02-15T18:35:22.857 INFO:tasks.workunit.client.1.smithi121.stdout:5/119: symlink d0/l23 0 2024-02-15T18:35:22.857 INFO:tasks.workunit.client.1.smithi121.stdout:5/120: dread - d0/d2/d18/f20 zero size 2024-02-15T18:35:22.860 INFO:tasks.workunit.client.1.smithi121.stdout:1/235: dread d0/d6/f12 [0,4194304] 0 2024-02-15T18:35:22.860 INFO:tasks.workunit.client.1.smithi121.stdout:4/146: link d5/d7/l1c d5/d9/d19/d2a/l2d 0 2024-02-15T18:35:22.860 INFO:tasks.workunit.client.1.smithi121.stdout:4/147: truncate d5/d7/d14/f17 648587 0 2024-02-15T18:35:22.860 INFO:tasks.workunit.client.1.smithi121.stdout:4/148: write d5/d9/d1d/f24 [788396,3918] 0 2024-02-15T18:35:22.861 INFO:tasks.workunit.client.1.smithi121.stdout:2/125: truncate d5/f14 2821058 0 2024-02-15T18:35:22.863 INFO:tasks.workunit.client.1.smithi121.stdout:4/149: write d5/d9/d19/d28/f21 [713983,9757] 0 2024-02-15T18:35:22.863 INFO:tasks.workunit.client.1.smithi121.stdout:0/333: rmdir d5/d6/dd 39 2024-02-15T18:35:22.864 INFO:tasks.workunit.client.1.smithi121.stdout:0/334: stat d5/d30/d5d/l6d 0 2024-02-15T18:35:22.864 INFO:tasks.workunit.client.1.smithi121.stdout:0/335: dread - d5/d6/dd/f4b zero size 2024-02-15T18:35:22.870 INFO:tasks.workunit.client.1.smithi121.stdout:9/127: creat d7/d2b/f2f x:0 0 0 2024-02-15T18:35:22.870 INFO:tasks.workunit.client.1.smithi121.stdout:2/126: symlink d5/df/d10/d15/l25 0 2024-02-15T18:35:22.870 INFO:tasks.workunit.client.1.smithi121.stdout:2/127: chown d5/df/d10/l1d 806893682 1 2024-02-15T18:35:22.871 INFO:tasks.workunit.client.1.smithi121.stdout:9/128: mkdir d7/d8/d25/d30 0 2024-02-15T18:35:22.883 INFO:tasks.workunit.client.1.smithi121.stdout:5/121: dwrite d0/fb [4194304,4194304] 0 2024-02-15T18:35:22.889 INFO:tasks.workunit.client.1.smithi121.stdout:1/236: dwrite d0/d6/d7/d9/f20 [0,4194304] 0 2024-02-15T18:35:22.889 INFO:tasks.workunit.client.1.smithi121.stdout:1/237: truncate d0/f3d 221029 0 2024-02-15T18:35:22.889 INFO:tasks.workunit.client.1.smithi121.stdout:1/238: fdatasync d0/f1d 0 2024-02-15T18:35:22.891 INFO:tasks.workunit.client.1.smithi121.stdout:9/129: dread d7/f1f [0,4194304] 0 2024-02-15T18:35:22.892 INFO:tasks.workunit.client.1.smithi121.stdout:2/128: fsync d5/fd 0 2024-02-15T18:35:22.892 INFO:tasks.workunit.client.1.smithi121.stdout:2/129: fsync f4 0 2024-02-15T18:35:22.893 INFO:tasks.workunit.client.1.smithi121.stdout:1/239: mkdir d0/d6/d7/d50 0 2024-02-15T18:35:22.899 INFO:tasks.workunit.client.1.smithi121.stdout:9/130: mknod d7/d26/c31 0 2024-02-15T18:35:22.901 INFO:tasks.workunit.client.1.smithi121.stdout:2/130: mkdir d5/df/d17/d26 0 2024-02-15T18:35:22.901 INFO:tasks.workunit.client.1.smithi121.stdout:2/131: fsync d5/fd 0 2024-02-15T18:35:22.902 INFO:tasks.workunit.client.1.smithi121.stdout:1/240: mkdir d0/d6/d7/d9/d51 0 2024-02-15T18:35:22.902 INFO:tasks.workunit.client.1.smithi121.stdout:1/241: readlink d0/l36 0 2024-02-15T18:35:22.903 INFO:tasks.workunit.client.1.smithi121.stdout:2/132: dread d5/df/d10/f11 [0,4194304] 0 2024-02-15T18:35:22.903 INFO:tasks.workunit.client.1.smithi121.stdout:2/133: chown c2 16 1 2024-02-15T18:35:22.903 INFO:tasks.workunit.client.1.smithi121.stdout:9/131: creat d7/d8/d25/d30/f32 x:0 0 0 2024-02-15T18:35:22.904 INFO:tasks.workunit.client.1.smithi121.stdout:1/242: creat d0/d6/d7/d9/f52 x:0 0 0 2024-02-15T18:35:22.906 INFO:tasks.workunit.client.1.smithi121.stdout:9/132: mkdir d7/d8/d25/d30/d33 0 2024-02-15T18:35:22.907 INFO:tasks.workunit.client.1.smithi121.stdout:5/122: dwrite d0/d2/d18/f20 [0,4194304] 0 2024-02-15T18:35:22.909 INFO:tasks.workunit.client.1.smithi121.stdout:9/133: unlink d7/l2d 0 2024-02-15T18:35:22.909 INFO:tasks.workunit.client.1.smithi121.stdout:5/123: write d0/d2/d18/f20 [2613024,130650] 0 2024-02-15T18:35:22.911 INFO:tasks.workunit.client.1.smithi121.stdout:2/134: rename d5/df/d17 to d5/df/d27 0 2024-02-15T18:35:22.914 INFO:tasks.workunit.client.1.smithi121.stdout:5/124: dread d0/d2/fd [0,4194304] 0 2024-02-15T18:35:22.914 INFO:tasks.workunit.client.1.smithi121.stdout:1/243: rmdir d0/d6/d7 39 2024-02-15T18:35:22.914 INFO:tasks.workunit.client.1.smithi121.stdout:2/135: dread d5/df/d10/d15/f1c [0,4194304] 0 2024-02-15T18:35:22.914 INFO:tasks.workunit.client.1.smithi121.stdout:2/136: truncate d5/f14 3326312 0 2024-02-15T18:35:22.914 INFO:tasks.workunit.client.1.smithi121.stdout:2/137: getdents d5 0 2024-02-15T18:35:22.917 INFO:tasks.workunit.client.1.smithi121.stdout:9/134: mknod d7/d8/c34 0 2024-02-15T18:35:22.921 INFO:tasks.workunit.client.1.smithi121.stdout:1/244: creat d0/f53 x:0 0 0 2024-02-15T18:35:22.922 INFO:tasks.workunit.client.1.smithi121.stdout:2/138: link d5/df/l20 d5/df/d27/d26/l28 0 2024-02-15T18:35:22.923 INFO:tasks.workunit.client.1.smithi121.stdout:9/135: rmdir d7/d8/d18 39 2024-02-15T18:35:22.923 INFO:tasks.workunit.client.1.smithi121.stdout:1/245: write d0/d6/f19 [4105337,30890] 0 2024-02-15T18:35:22.923 INFO:tasks.workunit.client.1.smithi121.stdout:1/246: fdatasync d0/d6/d7/f4c 0 2024-02-15T18:35:22.923 INFO:tasks.workunit.client.1.smithi121.stdout:1/247: dread - d0/d6/d7/d9/d14/f3b zero size 2024-02-15T18:35:22.924 INFO:tasks.workunit.client.1.smithi121.stdout:1/248: chown d0/d39/l46 10244070 1 2024-02-15T18:35:22.924 INFO:tasks.workunit.client.1.smithi121.stdout:1/249: write d0/d6/d7/d9/d14/d3e/f43 [712547,102028] 0 2024-02-15T18:35:22.926 INFO:tasks.workunit.client.1.smithi121.stdout:2/139: link d5/df/d10/f11 d5/df/d27/d26/f29 0 2024-02-15T18:35:22.926 INFO:tasks.workunit.client.1.smithi121.stdout:9/136: mknod d7/d8/d25/d30/d33/c35 0 2024-02-15T18:35:22.928 INFO:tasks.workunit.client.1.smithi121.stdout:2/140: unlink d5/f14 0 2024-02-15T18:35:22.930 INFO:tasks.workunit.client.1.smithi121.stdout:2/141: creat d5/df/f2a x:0 0 0 2024-02-15T18:35:22.932 INFO:tasks.workunit.client.1.smithi121.stdout:5/125: dwrite d0/d2/f14 [0,4194304] 0 2024-02-15T18:35:22.932 INFO:tasks.workunit.client.1.smithi121.stdout:2/142: write d5/f8 [2224960,35650] 0 2024-02-15T18:35:22.936 INFO:tasks.workunit.client.1.smithi121.stdout:5/126: symlink d0/d2/d1e/d21/l24 0 2024-02-15T18:35:22.936 INFO:tasks.workunit.client.1.smithi121.stdout:5/127: fdatasync d0/fb 0 2024-02-15T18:35:22.936 INFO:tasks.workunit.client.1.smithi121.stdout:5/128: truncate d0/f22 1205666 0 2024-02-15T18:35:22.939 INFO:tasks.workunit.client.1.smithi121.stdout:1/250: dread d0/d6/d7/f27 [0,4194304] 0 2024-02-15T18:35:22.949 INFO:tasks.workunit.client.1.smithi121.stdout:1/251: rename d0/d6/d7/f27 to d0/d6/d7/d29/f54 0 2024-02-15T18:35:22.950 INFO:tasks.workunit.client.1.smithi121.stdout:9/137: dwrite d7/d8/d18/f19 [0,4194304] 0 2024-02-15T18:35:22.950 INFO:tasks.workunit.client.1.smithi121.stdout:9/138: write d7/d8/f15 [529224,67766] 0 2024-02-15T18:35:22.953 INFO:tasks.workunit.client.1.smithi121.stdout:1/252: link d0/d6/f38 d0/d39/d4b/f55 0 2024-02-15T18:35:22.954 INFO:tasks.workunit.client.1.smithi121.stdout:9/139: write d7/f1b [3145003,5121] 0 2024-02-15T18:35:22.955 INFO:tasks.workunit.client.1.smithi121.stdout:1/253: mknod d0/c56 0 2024-02-15T18:35:22.958 INFO:tasks.workunit.client.1.smithi121.stdout:9/140: rename d7/d2b to d7/d8/d36 0 2024-02-15T18:35:22.958 INFO:tasks.workunit.client.1.smithi121.stdout:9/141: stat d7/d8/f16 0 2024-02-15T18:35:22.958 INFO:tasks.workunit.client.1.smithi121.stdout:5/129: dwrite d0/d2/f14 [0,4194304] 0 2024-02-15T18:35:22.962 INFO:tasks.workunit.client.1.smithi121.stdout:1/254: mknod d0/d6/d7/d9/c57 0 2024-02-15T18:35:22.964 INFO:tasks.workunit.client.1.smithi121.stdout:9/142: symlink d7/d8/d36/l37 0 2024-02-15T18:35:22.964 INFO:tasks.workunit.client.1.smithi121.stdout:9/143: readlink d7/l14 0 2024-02-15T18:35:22.969 INFO:tasks.workunit.client.1.smithi121.stdout:1/255: creat d0/d6/d7/d9/d14/f58 x:0 0 0 2024-02-15T18:35:22.974 INFO:tasks.workunit.client.1.smithi121.stdout:5/130: dwrite d0/f22 [0,4194304] 0 2024-02-15T18:35:22.977 INFO:tasks.workunit.client.1.smithi121.stdout:5/131: creat d0/d2/f25 x:0 0 0 2024-02-15T18:35:22.978 INFO:tasks.workunit.client.1.smithi121.stdout:9/144: dread d7/f1b [0,4194304] 0 2024-02-15T18:35:22.983 INFO:tasks.workunit.client.1.smithi121.stdout:9/145: creat d7/d8/f38 x:0 0 0 2024-02-15T18:35:22.984 INFO:tasks.workunit.client.1.smithi121.stdout:9/146: rename d7/d8/c34 to d7/d8/d25/d30/d33/c39 0 2024-02-15T18:35:22.984 INFO:tasks.workunit.client.1.smithi121.stdout:9/147: chown d7/d8/f15 2456 1 2024-02-15T18:35:22.984 INFO:tasks.workunit.client.1.smithi121.stdout:9/148: read d7/d8/f15 [570941,2476] 0 2024-02-15T18:35:23.006 INFO:tasks.workunit.client.1.smithi121.stdout:5/132: dwrite d0/d2/d18/f20 [0,4194304] 0 2024-02-15T18:35:23.006 INFO:tasks.workunit.client.1.smithi121.stdout:9/149: dwrite d7/d8/f15 [0,4194304] 0 2024-02-15T18:35:23.011 INFO:tasks.workunit.client.1.smithi121.stdout:9/150: mkdir d7/d3a 0 2024-02-15T18:35:23.012 INFO:tasks.workunit.client.1.smithi121.stdout:9/151: creat d7/d8/f3b x:0 0 0 2024-02-15T18:35:23.012 INFO:tasks.workunit.client.1.smithi121.stdout:9/152: fdatasync f5 0 2024-02-15T18:35:23.027 INFO:tasks.workunit.client.1.smithi121.stdout:5/133: dwrite d0/f22 [0,4194304] 0 2024-02-15T18:35:23.031 INFO:tasks.workunit.client.1.smithi121.stdout:9/153: dwrite f5 [0,4194304] 0 2024-02-15T18:35:23.038 INFO:tasks.workunit.client.1.smithi121.stdout:5/134: truncate d0/fb 8855329 0 2024-02-15T18:35:23.038 INFO:tasks.workunit.client.1.smithi121.stdout:5/135: write d0/d2/f25 [333837,9568] 0 2024-02-15T18:35:23.038 INFO:tasks.workunit.client.1.smithi121.stdout:8/312: sync 2024-02-15T18:35:23.039 INFO:tasks.workunit.client.1.smithi121.stdout:9/154: mkdir d7/d8/d25/d30/d33/d3c 0 2024-02-15T18:35:23.040 INFO:tasks.workunit.client.1.smithi121.stdout:5/136: symlink d0/l26 0 2024-02-15T18:35:23.041 INFO:tasks.workunit.client.1.smithi121.stdout:5/137: read d0/d2/f25 [208601,13441] 0 2024-02-15T18:35:23.042 INFO:tasks.workunit.client.1.smithi121.stdout:9/155: mkdir d7/d3d 0 2024-02-15T18:35:23.044 INFO:tasks.workunit.client.1.smithi121.stdout:5/138: creat d0/d2/d18/f27 x:0 0 0 2024-02-15T18:35:23.046 INFO:tasks.workunit.client.1.smithi121.stdout:5/139: rename d0/d2/l1a to d0/l28 0 2024-02-15T18:35:23.046 INFO:tasks.workunit.client.1.smithi121.stdout:5/140: fdatasync d0/d2/f14 0 2024-02-15T18:35:23.050 INFO:tasks.workunit.client.1.smithi121.stdout:5/141: creat d0/d2/d18/f29 x:0 0 0 2024-02-15T18:35:23.050 INFO:tasks.workunit.client.1.smithi121.stdout:5/142: truncate d0/d2/f25 728706 0 2024-02-15T18:35:23.051 INFO:tasks.workunit.client.1.smithi121.stdout:5/143: symlink d0/d2/d1e/d21/l2a 0 2024-02-15T18:35:23.056 INFO:tasks.workunit.client.1.smithi121.stdout:5/144: getdents d0 0 2024-02-15T18:35:23.060 INFO:tasks.workunit.client.1.smithi121.stdout:5/145: creat d0/d16/f2b x:0 0 0 2024-02-15T18:35:23.062 INFO:tasks.workunit.client.1.smithi121.stdout:5/146: link d0/f22 d0/d2/d1e/d21/f2c 0 2024-02-15T18:35:23.069 INFO:tasks.workunit.client.1.smithi121.stdout:5/147: dread d0/fb [4194304,4194304] 0 2024-02-15T18:35:23.069 INFO:tasks.workunit.client.1.smithi121.stdout:5/148: truncate d0/d2/f25 1028605 0 2024-02-15T18:35:23.070 INFO:tasks.workunit.client.1.smithi121.stdout:5/149: link d0/d2/d1e/d21/l2a d0/d16/l2d 0 2024-02-15T18:35:23.072 INFO:tasks.workunit.client.1.smithi121.stdout:5/150: rename d0/d16/l19 to d0/d2/l2e 0 2024-02-15T18:35:23.073 INFO:tasks.workunit.client.1.smithi121.stdout:5/151: link d0/ce d0/d2/d1e/d21/c2f 0 2024-02-15T18:35:23.082 INFO:tasks.workunit.client.1.smithi121.stdout:5/152: rename d0/c3 to d0/c30 0 2024-02-15T18:35:23.083 INFO:tasks.workunit.client.1.smithi121.stdout:5/153: creat d0/d2/d18/f31 x:0 0 0 2024-02-15T18:35:23.084 INFO:tasks.workunit.client.1.smithi121.stdout:5/154: symlink d0/d16/l32 0 2024-02-15T18:35:23.084 INFO:tasks.workunit.client.1.smithi121.stdout:5/155: read - d0/d2/d18/f29 zero size 2024-02-15T18:35:23.085 INFO:tasks.workunit.client.1.smithi121.stdout:5/156: mkdir d0/d2/d1e/d21/d33 0 2024-02-15T18:35:23.087 INFO:tasks.workunit.client.1.smithi121.stdout:5/157: mkdir d0/d16/d34 0 2024-02-15T18:35:23.089 INFO:tasks.workunit.client.1.smithi121.stdout:5/158: symlink d0/d2/l35 0 2024-02-15T18:35:23.091 INFO:tasks.workunit.client.1.smithi121.stdout:5/159: write d0/d2/f14 [3118478,37546] 0 2024-02-15T18:35:23.092 INFO:tasks.workunit.client.1.smithi121.stdout:5/160: link d0/c1c d0/d2/d18/c36 0 2024-02-15T18:35:23.093 INFO:tasks.workunit.client.1.smithi121.stdout:5/161: truncate d0/d2/fd 1171219 0 2024-02-15T18:35:23.096 INFO:tasks.workunit.client.1.smithi121.stdout:5/162: mkdir d0/d16/d37 0 2024-02-15T18:35:23.096 INFO:tasks.workunit.client.1.smithi121.stdout:5/163: fdatasync d0/d2/f25 0 2024-02-15T18:35:23.096 INFO:tasks.workunit.client.1.smithi121.stdout:5/164: chown d0/d2 14102217 1 2024-02-15T18:35:23.116 INFO:tasks.workunit.client.1.smithi121.stdout:5/165: truncate d0/d2/f25 521463 0 2024-02-15T18:35:23.117 INFO:tasks.workunit.client.1.smithi121.stdout:5/166: truncate d0/f17 5578324 0 2024-02-15T18:35:23.131 INFO:tasks.workunit.client.1.smithi121.stdout:5/167: read d0/f17 [3973046,102866] 0 2024-02-15T18:35:23.151 INFO:tasks.workunit.client.1.smithi121.stdout:5/168: dwrite d0/fb [0,4194304] 0 2024-02-15T18:35:23.153 INFO:tasks.workunit.client.1.smithi121.stdout:5/169: mknod d0/d2/d18/c38 0 2024-02-15T18:35:23.156 INFO:tasks.workunit.client.1.smithi121.stdout:5/170: mkdir d0/d2/d1e/d21/d39 0 2024-02-15T18:35:23.182 INFO:tasks.workunit.client.1.smithi121.stdout:5/171: creat d0/d16/d34/f3a x:0 0 0 2024-02-15T18:35:23.192 INFO:tasks.workunit.client.1.smithi121.stdout:3/264: sync 2024-02-15T18:35:23.193 INFO:tasks.workunit.client.1.smithi121.stdout:3/265: mkdir d2/d5a 0 2024-02-15T18:35:23.194 INFO:tasks.workunit.client.1.smithi121.stdout:3/266: creat d2/d1a/d27/d2c/d25/d30/f5b x:0 0 0 2024-02-15T18:35:23.194 INFO:tasks.workunit.client.1.smithi121.stdout:3/267: dread - d2/d1a/d54/f59 zero size 2024-02-15T18:35:23.196 INFO:tasks.workunit.client.1.smithi121.stdout:3/268: creat d2/d1a/d27/d2c/d25/d31/d34/d4a/f5c x:0 0 0 2024-02-15T18:35:23.200 INFO:tasks.workunit.client.1.smithi121.stdout:5/172: dwrite d0/d2/f25 [0,4194304] 0 2024-02-15T18:35:23.203 INFO:tasks.workunit.client.1.smithi121.stdout:5/173: symlink d0/d2/d1e/d21/d39/l3b 0 2024-02-15T18:35:23.203 INFO:tasks.workunit.client.1.smithi121.stdout:5/174: fdatasync d0/d2/d18/f20 0 2024-02-15T18:35:23.203 INFO:tasks.workunit.client.1.smithi121.stdout:5/175: stat d0/fb 0 2024-02-15T18:35:23.205 INFO:tasks.workunit.client.1.smithi121.stdout:5/176: mknod d0/d2/d1e/d21/d39/c3c 0 2024-02-15T18:35:23.206 INFO:tasks.workunit.client.1.smithi121.stdout:5/177: write d0/d2/f25 [2591877,78737] 0 2024-02-15T18:35:23.214 INFO:tasks.workunit.client.1.smithi121.stdout:3/269: dwrite d2/d1a/d22/f47 [0,4194304] 0 2024-02-15T18:35:23.217 INFO:tasks.workunit.client.1.smithi121.stdout:5/178: dread d0/fb [0,4194304] 0 2024-02-15T18:35:23.217 INFO:tasks.workunit.client.1.smithi121.stdout:5/179: fsync d0/d2/d18/f29 0 2024-02-15T18:35:23.217 INFO:tasks.workunit.client.1.smithi121.stdout:5/180: chown d0/d2/d1e/l1f 888768142 1 2024-02-15T18:35:23.218 INFO:tasks.workunit.client.1.smithi121.stdout:3/270: getdents d2/d1a/d27/d2c/d25/d31 0 2024-02-15T18:35:23.218 INFO:tasks.workunit.client.1.smithi121.stdout:5/181: write d0/d2/f14 [3558455,32530] 0 2024-02-15T18:35:23.222 INFO:tasks.workunit.client.1.smithi121.stdout:3/271: symlink d2/d1a/l5d 0 2024-02-15T18:35:23.223 INFO:tasks.workunit.client.1.smithi121.stdout:3/272: write d2/d1a/d27/d2c/d25/d30/f46 [1432292,84689] 0 2024-02-15T18:35:23.224 INFO:tasks.workunit.client.1.smithi121.stdout:5/182: symlink d0/d2/l3d 0 2024-02-15T18:35:23.246 INFO:tasks.workunit.client.1.smithi121.stdout:3/273: dwrite d2/d1a/d27/d2c/d25/d30/f38 [0,4194304] 0 2024-02-15T18:35:23.246 INFO:tasks.workunit.client.1.smithi121.stdout:3/274: readlink d2/d1a/l21 0 2024-02-15T18:35:23.249 INFO:tasks.workunit.client.1.smithi121.stdout:3/275: creat d2/d1a/d27/f5e x:0 0 0 2024-02-15T18:35:23.252 INFO:tasks.workunit.client.1.smithi121.stdout:3/276: link d2/d1a/d27/d2c/d25/d31/f50 d2/d5a/f5f 0 2024-02-15T18:35:23.252 INFO:tasks.workunit.client.1.smithi121.stdout:3/277: fsync d2/d1a/d22/f35 0 2024-02-15T18:35:23.252 INFO:tasks.workunit.client.1.smithi121.stdout:3/278: stat d2/d1a/d54/f59 0 2024-02-15T18:35:23.253 INFO:tasks.workunit.client.1.smithi121.stdout:5/183: dwrite d0/d2/fd [0,4194304] 0 2024-02-15T18:35:23.255 INFO:tasks.workunit.client.1.smithi121.stdout:3/279: truncate d2/fd 166493 0 2024-02-15T18:35:23.257 INFO:tasks.workunit.client.1.smithi121.stdout:5/184: unlink d0/d2/c1d 0 2024-02-15T18:35:23.257 INFO:tasks.workunit.client.1.smithi121.stdout:5/185: chown d0/d2/d1e/d21/l2a 8060 1 2024-02-15T18:35:23.262 INFO:tasks.workunit.client.1.smithi121.stdout:3/280: mknod d2/d1a/d22/c60 0 2024-02-15T18:35:23.262 INFO:tasks.workunit.client.1.smithi121.stdout:3/281: truncate d2/d1a/d27/d2c/f2f 4225765 0 2024-02-15T18:35:23.262 INFO:tasks.workunit.client.1.smithi121.stdout:3/282: write d2/f36 [4043033,49566] 0 2024-02-15T18:35:23.264 INFO:tasks.workunit.client.1.smithi121.stdout:5/186: link d0/d2/d1e/d21/c2f d0/c3e 0 2024-02-15T18:35:23.269 INFO:tasks.workunit.client.1.smithi121.stdout:5/187: symlink d0/d2/l3f 0 2024-02-15T18:35:23.269 INFO:tasks.workunit.client.1.smithi121.stdout:5/188: dread - d0/d2/d18/f31 zero size 2024-02-15T18:35:23.270 INFO:tasks.workunit.client.1.smithi121.stdout:5/189: readlink d0/l23 0 2024-02-15T18:35:23.270 INFO:tasks.workunit.client.1.smithi121.stdout:5/190: rmdir d0/d2/d1e 39 2024-02-15T18:35:23.272 INFO:tasks.workunit.client.1.smithi121.stdout:5/191: mkdir d0/d2/d1e/d21/d40 0 2024-02-15T18:35:23.273 INFO:tasks.workunit.client.1.smithi121.stdout:5/192: getdents d0/d2/d1e/d21/d39 0 2024-02-15T18:35:23.273 INFO:tasks.workunit.client.1.smithi121.stdout:5/193: fdatasync d0/d16/f2b 0 2024-02-15T18:35:23.282 INFO:tasks.workunit.client.1.smithi121.stdout:3/283: dwrite d2/d1a/d27/d2c/d25/d30/f41 [0,4194304] 0 2024-02-15T18:35:23.299 INFO:tasks.workunit.client.1.smithi121.stdout:3/284: dread d2/d1a/d27/d2c/f56 [0,4194304] 0 2024-02-15T18:35:23.299 INFO:tasks.workunit.client.1.smithi121.stdout:5/194: dwrite d0/d2/d1e/d21/f2c [0,4194304] 0 2024-02-15T18:35:23.299 INFO:tasks.workunit.client.1.smithi121.stdout:3/285: stat d2/d1a/d27/d2c/d25/d31/f50 0 2024-02-15T18:35:23.300 INFO:tasks.workunit.client.1.smithi121.stdout:5/195: symlink d0/d2/d1e/d21/d40/l41 0 2024-02-15T18:35:23.300 INFO:tasks.workunit.client.1.smithi121.stdout:5/196: chown d0/d2/l3f 362439205 1 2024-02-15T18:35:23.301 INFO:tasks.workunit.client.1.smithi121.stdout:3/286: creat d2/d1a/d27/d2c/d25/d31/d58/f61 x:0 0 0 2024-02-15T18:35:23.303 INFO:tasks.workunit.client.1.smithi121.stdout:7/181: sync 2024-02-15T18:35:23.304 INFO:tasks.workunit.client.1.smithi121.stdout:3/287: creat d2/d1a/d27/d2c/d25/d30/f62 x:0 0 0 2024-02-15T18:35:23.307 INFO:tasks.workunit.client.1.smithi121.stdout:3/288: mkdir d2/d1a/d27/d63 0 2024-02-15T18:35:23.308 INFO:tasks.workunit.client.1.smithi121.stdout:3/289: creat d2/d1a/d22/f64 x:0 0 0 2024-02-15T18:35:23.316 INFO:tasks.workunit.client.1.smithi121.stdout:7/182: dwrite d0/d5/f26 [4194304,4194304] 0 2024-02-15T18:35:23.319 INFO:tasks.workunit.client.1.smithi121.stdout:3/290: rmdir d2/d1a/d27/d2c 39 2024-02-15T18:35:23.320 INFO:tasks.workunit.client.1.smithi121.stdout:7/183: mknod d0/d5/d11/d1c/c3d 0 2024-02-15T18:35:23.320 INFO:tasks.workunit.client.1.smithi121.stdout:7/184: readlink d0/l37 0 2024-02-15T18:35:23.323 INFO:tasks.workunit.client.1.smithi121.stdout:3/291: rename d2/l20 to d2/d1a/d27/d2c/d25/d30/l65 0 2024-02-15T18:35:23.323 INFO:tasks.workunit.client.1.smithi121.stdout:3/292: fsync d2/d1a/d22/f64 0 2024-02-15T18:35:23.328 INFO:tasks.workunit.client.1.smithi121.stdout:3/293: rename d2/c12 to d2/c66 0 2024-02-15T18:35:23.328 INFO:tasks.workunit.client.1.smithi121.stdout:3/294: dread - d2/d1a/d27/d2c/d25/d30/f62 zero size 2024-02-15T18:35:23.328 INFO:tasks.workunit.client.1.smithi121.stdout:3/295: stat d2/d1a/d27/d2c 0 2024-02-15T18:35:23.337 INFO:tasks.workunit.client.1.smithi121.stdout:7/185: dwrite d0/d5/d11/d1c/f21 [0,4194304] 0 2024-02-15T18:35:23.338 INFO:tasks.workunit.client.1.smithi121.stdout:7/186: creat d0/d5/d11/d1c/f3e x:0 0 0 2024-02-15T18:35:23.344 INFO:tasks.workunit.client.1.smithi121.stdout:3/296: dread d2/d1a/d27/d2c/d25/d30/f41 [0,4194304] 0 2024-02-15T18:35:23.346 INFO:tasks.workunit.client.1.smithi121.stdout:3/297: dread d2/fd [0,4194304] 0 2024-02-15T18:35:23.346 INFO:tasks.workunit.client.1.smithi121.stdout:3/298: creat d2/d5a/f67 x:0 0 0 2024-02-15T18:35:23.348 INFO:tasks.workunit.client.1.smithi121.stdout:3/299: mknod d2/d1a/d27/c68 0 2024-02-15T18:35:23.348 INFO:tasks.workunit.client.1.smithi121.stdout:3/300: write d2/d1a/d27/d2c/d25/f52 [452686,74390] 0 2024-02-15T18:35:23.348 INFO:tasks.workunit.client.1.smithi121.stdout:3/301: read - d2/d1a/d27/d2c/d25/d30/f5b zero size 2024-02-15T18:35:23.353 INFO:tasks.workunit.client.1.smithi121.stdout:7/187: dwrite d0/d5/d11/d1c/f23 [0,4194304] 0 2024-02-15T18:35:23.353 INFO:tasks.workunit.client.1.smithi121.stdout:7/188: read d0/d5/f15 [211365,112218] 0 2024-02-15T18:35:23.356 INFO:tasks.workunit.client.1.smithi121.stdout:7/189: mkdir d0/d38/d3f 0 2024-02-15T18:35:23.357 INFO:tasks.workunit.client.1.smithi121.stdout:7/190: getdents d0 0 2024-02-15T18:35:23.359 INFO:tasks.workunit.client.1.smithi121.stdout:3/302: dread d2/f9 [0,4194304] 0 2024-02-15T18:35:23.363 INFO:tasks.workunit.client.1.smithi121.stdout:7/191: creat d0/d5/d11/f40 x:0 0 0 2024-02-15T18:35:23.363 INFO:tasks.workunit.client.1.smithi121.stdout:7/192: chown d0/fb 15960630 1 2024-02-15T18:35:23.364 INFO:tasks.workunit.client.1.smithi121.stdout:3/303: creat d2/d1a/d54/f69 x:0 0 0 2024-02-15T18:35:23.365 INFO:tasks.workunit.client.1.smithi121.stdout:7/193: symlink d0/d5/l41 0 2024-02-15T18:35:23.370 INFO:tasks.workunit.client.1.smithi121.stdout:3/304: creat d2/d1a/d22/f6a x:0 0 0 2024-02-15T18:35:23.377 INFO:tasks.workunit.client.1.smithi121.stdout:7/194: rmdir d0/d5/d11 39 2024-02-15T18:35:23.378 INFO:tasks.workunit.client.1.smithi121.stdout:7/195: dread d0/fb [0,4194304] 0 2024-02-15T18:35:23.381 INFO:tasks.workunit.client.1.smithi121.stdout:7/196: creat d0/f42 x:0 0 0 2024-02-15T18:35:23.382 INFO:tasks.workunit.client.1.smithi121.stdout:4/150: sync 2024-02-15T18:35:23.382 INFO:tasks.workunit.client.1.smithi121.stdout:8/313: sync 2024-02-15T18:35:23.383 INFO:tasks.workunit.client.1.smithi121.stdout:1/256: sync 2024-02-15T18:35:23.383 INFO:tasks.workunit.client.1.smithi121.stdout:9/156: sync 2024-02-15T18:35:23.383 INFO:tasks.workunit.client.1.smithi121.stdout:6/271: sync 2024-02-15T18:35:23.383 INFO:tasks.workunit.client.1.smithi121.stdout:2/143: sync 2024-02-15T18:35:23.383 INFO:tasks.workunit.client.1.smithi121.stdout:1/257: fsync d0/d6/f31 0 2024-02-15T18:35:23.383 INFO:tasks.workunit.client.1.smithi121.stdout:6/272: chown d7/dc/d34/f43 0 1 2024-02-15T18:35:23.383 INFO:tasks.workunit.client.1.smithi121.stdout:2/144: truncate d5/df/f1b 754856 0 2024-02-15T18:35:23.383 INFO:tasks.workunit.client.1.smithi121.stdout:1/258: readlink d0/d6/d7/l37 0 2024-02-15T18:35:23.383 INFO:tasks.workunit.client.1.smithi121.stdout:1/259: truncate d0/d39/d4b/f55 334768 0 2024-02-15T18:35:23.383 INFO:tasks.workunit.client.1.smithi121.stdout:9/157: write d7/d8/f1d [511671,88350] 0 2024-02-15T18:35:23.383 INFO:tasks.workunit.client.1.smithi121.stdout:0/336: sync 2024-02-15T18:35:23.383 INFO:tasks.workunit.client.1.smithi121.stdout:9/158: read d7/f13 [537127,64336] 0 2024-02-15T18:35:23.383 INFO:tasks.workunit.client.1.smithi121.stdout:9/159: chown d7/d8/d25/d30/d33/c35 3731 1 2024-02-15T18:35:23.383 INFO:tasks.workunit.client.1.smithi121.stdout:9/160: chown c6 2 1 2024-02-15T18:35:23.384 INFO:tasks.workunit.client.1.smithi121.stdout:9/161: read d7/d8/f10 [1719325,69630] 0 2024-02-15T18:35:23.384 INFO:tasks.workunit.client.1.smithi121.stdout:7/197: mkdir d0/d38/d3f/d43 0 2024-02-15T18:35:23.384 INFO:tasks.workunit.client.1.smithi121.stdout:7/198: read - d0/d33/f3b zero size 2024-02-15T18:35:23.386 INFO:tasks.workunit.client.1.smithi121.stdout:4/151: creat d5/d9/f2e x:0 0 0 2024-02-15T18:35:23.387 INFO:tasks.workunit.client.1.smithi121.stdout:6/273: truncate d7/dc/f2d 486942 0 2024-02-15T18:35:23.388 INFO:tasks.workunit.client.1.smithi121.stdout:2/145: creat d5/df/f2b x:0 0 0 2024-02-15T18:35:23.388 INFO:tasks.workunit.client.1.smithi121.stdout:2/146: dread - d5/df/f2b zero size 2024-02-15T18:35:23.388 INFO:tasks.workunit.client.1.smithi121.stdout:2/147: chown l1 3906731 1 2024-02-15T18:35:23.390 INFO:tasks.workunit.client.1.smithi121.stdout:1/260: mknod d0/d6/d7/c59 0 2024-02-15T18:35:23.390 INFO:tasks.workunit.client.1.smithi121.stdout:0/337: creat d5/d30/f6f x:0 0 0 2024-02-15T18:35:23.393 INFO:tasks.workunit.client.1.smithi121.stdout:7/199: unlink d0/c32 0 2024-02-15T18:35:23.394 INFO:tasks.workunit.client.1.smithi121.stdout:4/152: creat d5/d9/d1d/f2f x:0 0 0 2024-02-15T18:35:23.394 INFO:tasks.workunit.client.1.smithi121.stdout:4/153: chown d5/f6 1375 1 2024-02-15T18:35:23.394 INFO:tasks.workunit.client.1.smithi121.stdout:4/154: getdents d5/d9/d1d/d26 0 2024-02-15T18:35:23.398 INFO:tasks.workunit.client.1.smithi121.stdout:2/148: mknod d5/df/d10/c2c 0 2024-02-15T18:35:23.398 INFO:tasks.workunit.client.1.smithi121.stdout:2/149: dread - d5/df/f2b zero size 2024-02-15T18:35:23.400 INFO:tasks.workunit.client.1.smithi121.stdout:2/150: write d5/df/d10/f11 [478388,65375] 0 2024-02-15T18:35:23.401 INFO:tasks.workunit.client.1.smithi121.stdout:4/155: write f1 [5505475,116872] 0 2024-02-15T18:35:23.402 INFO:tasks.workunit.client.1.smithi121.stdout:4/156: dread d5/f10 [0,4194304] 0 2024-02-15T18:35:23.403 INFO:tasks.workunit.client.1.smithi121.stdout:1/261: unlink d0/d6/d7/d9/f1e 0 2024-02-15T18:35:23.403 INFO:tasks.workunit.client.1.smithi121.stdout:0/338: chown d5/d6/d2a/l52 180874 1 2024-02-15T18:35:23.405 INFO:tasks.workunit.client.1.smithi121.stdout:6/274: mkdir d7/dc/d34/d42/d62 0 2024-02-15T18:35:23.405 INFO:tasks.workunit.client.1.smithi121.stdout:6/275: chown d7/f47 17132757 1 2024-02-15T18:35:23.406 INFO:tasks.workunit.client.1.smithi121.stdout:2/151: truncate f4 1193440 0 2024-02-15T18:35:23.407 INFO:tasks.workunit.client.1.smithi121.stdout:4/157: rmdir d5/d7 39 2024-02-15T18:35:23.409 INFO:tasks.workunit.client.1.smithi121.stdout:1/262: rename d0/d6/d7/d9/c57 to d0/d6/d7/d9/d14/d48/c5a 0 2024-02-15T18:35:23.410 INFO:tasks.workunit.client.1.smithi121.stdout:0/339: mknod d5/d6/d2a/d4e/c70 0 2024-02-15T18:35:23.411 INFO:tasks.workunit.client.1.smithi121.stdout:8/314: dwrite d0/f2 [0,4194304] 0 2024-02-15T18:35:23.411 INFO:tasks.workunit.client.1.smithi121.stdout:8/315: chown d0/d3/c9 141903 1 2024-02-15T18:35:23.411 INFO:tasks.workunit.client.1.smithi121.stdout:8/316: dread - d0/d7/d3f/d45/f3e zero size 2024-02-15T18:35:23.411 INFO:tasks.workunit.client.1.smithi121.stdout:2/152: creat d5/df/d10/d15/f2d x:0 0 0 2024-02-15T18:35:23.415 INFO:tasks.workunit.client.1.smithi121.stdout:8/317: dread d0/d7/d3f/d45/f3a [0,4194304] 0 2024-02-15T18:35:23.415 INFO:tasks.workunit.client.1.smithi121.stdout:1/263: creat d0/d6/d7/d4e/f5b x:0 0 0 2024-02-15T18:35:23.415 INFO:tasks.workunit.client.1.smithi121.stdout:7/200: dwrite d0/d5/d11/f40 [0,4194304] 0 2024-02-15T18:35:23.415 INFO:tasks.workunit.client.1.smithi121.stdout:7/201: readlink d0/d5/l1b 0 2024-02-15T18:35:23.415 INFO:tasks.workunit.client.1.smithi121.stdout:9/162: dwrite d7/d8/d18/f19 [0,4194304] 0 2024-02-15T18:35:23.415 INFO:tasks.workunit.client.1.smithi121.stdout:9/163: write d7/d8/f16 [322993,4336] 0 2024-02-15T18:35:23.415 INFO:tasks.workunit.client.1.smithi121.stdout:9/164: chown d7/d8/d36 2572 1 2024-02-15T18:35:23.416 INFO:tasks.workunit.client.1.smithi121.stdout:0/340: mkdir d5/d6/d2a/d36/d44/d71 0 2024-02-15T18:35:23.417 INFO:tasks.workunit.client.1.smithi121.stdout:4/158: write d5/f6 [874954,67855] 0 2024-02-15T18:35:23.417 INFO:tasks.workunit.client.1.smithi121.stdout:6/276: write d7/dc/d2c/d36/f38 [601675,5265] 0 2024-02-15T18:35:23.428 INFO:tasks.workunit.client.1.smithi121.stdout:2/153: truncate d5/df/f24 615124 0 2024-02-15T18:35:23.429 INFO:tasks.workunit.client.1.smithi121.stdout:2/154: fsync d5/fd 0 2024-02-15T18:35:23.429 INFO:tasks.workunit.client.1.smithi121.stdout:8/318: creat d0/d4a/f77 x:0 0 0 2024-02-15T18:35:23.433 INFO:tasks.workunit.client.1.smithi121.stdout:1/264: unlink d0/d6/f31 0 2024-02-15T18:35:23.434 INFO:tasks.workunit.client.1.smithi121.stdout:7/202: creat d0/d5/d11/d1c/f44 x:0 0 0 2024-02-15T18:35:23.435 INFO:tasks.workunit.client.1.smithi121.stdout:9/165: mkdir d7/d8/d25/d30/d33/d3e 0 2024-02-15T18:35:23.435 INFO:tasks.workunit.client.1.smithi121.stdout:9/166: chown d7/d8/d25/d30/d33 44 1 2024-02-15T18:35:23.436 INFO:tasks.workunit.client.1.smithi121.stdout:9/167: write d7/f1c [591477,44859] 0 2024-02-15T18:35:23.437 INFO:tasks.workunit.client.1.smithi121.stdout:6/277: mkdir d7/dc/d34/d42/d62/d63 0 2024-02-15T18:35:23.437 INFO:tasks.workunit.client.1.smithi121.stdout:6/278: chown d7/dc/f25 31767 1 2024-02-15T18:35:23.437 INFO:tasks.workunit.client.1.smithi121.stdout:6/279: write d7/dc/d34/f4a [123021,27262] 0 2024-02-15T18:35:23.438 INFO:tasks.workunit.client.1.smithi121.stdout:0/341: dwrite d5/d6/dd/f16 [0,4194304] 0 2024-02-15T18:35:23.440 INFO:tasks.workunit.client.1.smithi121.stdout:0/342: dread d5/d6/dd/f32 [0,4194304] 0 2024-02-15T18:35:23.440 INFO:tasks.workunit.client.1.smithi121.stdout:0/343: dread - d5/d6/d2a/d36/f51 zero size 2024-02-15T18:35:23.440 INFO:tasks.workunit.client.1.smithi121.stdout:0/344: fdatasync d5/d6/d2a/d36/f3d 0 2024-02-15T18:35:23.449 INFO:tasks.workunit.client.1.smithi121.stdout:8/319: creat d0/d3/d20/f78 x:0 0 0 2024-02-15T18:35:23.455 INFO:tasks.workunit.client.1.smithi121.stdout:9/168: mkdir d7/d8/d25/d30/d33/d3c/d3f 0 2024-02-15T18:35:23.455 INFO:tasks.workunit.client.1.smithi121.stdout:9/169: chown d7/d8/d25/d30/d33/d3e 366 1 2024-02-15T18:35:23.455 INFO:tasks.workunit.client.1.smithi121.stdout:9/170: fdatasync d7/d8/f17 0 2024-02-15T18:35:23.456 INFO:tasks.workunit.client.1.smithi121.stdout:6/280: truncate d7/f11 4103956 0 2024-02-15T18:35:23.457 INFO:tasks.workunit.client.1.smithi121.stdout:6/281: write d7/dc/d34/f43 [1561627,30242] 0 2024-02-15T18:35:23.458 INFO:tasks.workunit.client.1.smithi121.stdout:2/155: unlink d5/df/f24 0 2024-02-15T18:35:23.460 INFO:tasks.workunit.client.1.smithi121.stdout:8/320: rmdir d0 39 2024-02-15T18:35:23.460 INFO:tasks.workunit.client.1.smithi121.stdout:9/171: mkdir d7/d8/d40 0 2024-02-15T18:35:23.462 INFO:tasks.workunit.client.1.smithi121.stdout:8/321: rmdir d0/d61/d66 39 2024-02-15T18:35:23.462 INFO:tasks.workunit.client.1.smithi121.stdout:8/322: fsync d0/d7/d3f/d45/f3e 0 2024-02-15T18:35:23.465 INFO:tasks.workunit.client.1.smithi121.stdout:9/172: rmdir d7/d3d 0 2024-02-15T18:35:23.465 INFO:tasks.workunit.client.1.smithi121.stdout:9/173: dread - d7/d8/f38 zero size 2024-02-15T18:35:23.468 INFO:tasks.workunit.client.1.smithi121.stdout:6/282: read d7/dc/d34/f13 [749488,130045] 0 2024-02-15T18:35:23.468 INFO:tasks.workunit.client.1.smithi121.stdout:6/283: getdents d7/dc/d34/d42/d62/d63 0 2024-02-15T18:35:23.468 INFO:tasks.workunit.client.1.smithi121.stdout:6/284: write d7/dc/d2c/d36/f3e [17120,17074] 0 2024-02-15T18:35:23.468 INFO:tasks.workunit.client.1.smithi121.stdout:9/174: rename d7/d8/f15 to d7/d8/d25/d30/d33/d3e/f41 0 2024-02-15T18:35:23.470 INFO:tasks.workunit.client.1.smithi121.stdout:8/323: write d0/f15 [3041097,49597] 0 2024-02-15T18:35:23.470 INFO:tasks.workunit.client.1.smithi121.stdout:8/324: truncate d0/d7/de/f65 524208 0 2024-02-15T18:35:23.470 INFO:tasks.workunit.client.1.smithi121.stdout:0/345: dwrite d5/f27 [0,4194304] 0 2024-02-15T18:35:23.471 INFO:tasks.workunit.client.1.smithi121.stdout:7/203: dwrite d0/d5/d11/f40 [4194304,4194304] 0 2024-02-15T18:35:23.471 INFO:tasks.workunit.client.1.smithi121.stdout:8/325: write d0/d7/de/f1c [4209757,18508] 0 2024-02-15T18:35:23.471 INFO:tasks.workunit.client.1.smithi121.stdout:8/326: fdatasync d0/d61/d66/f3b 0 2024-02-15T18:35:23.471 INFO:tasks.workunit.client.1.smithi121.stdout:6/285: mknod d7/dc/d34/d42/c64 0 2024-02-15T18:35:23.475 INFO:tasks.workunit.client.1.smithi121.stdout:2/156: dread d5/f8 [0,4194304] 0 2024-02-15T18:35:23.475 INFO:tasks.workunit.client.1.smithi121.stdout:2/157: dread - d5/df/f2b zero size 2024-02-15T18:35:23.476 INFO:tasks.workunit.client.1.smithi121.stdout:0/346: rename d5/d30/d5d/l6d to d5/d6/d2a/d36/d44/d71/l72 0 2024-02-15T18:35:23.476 INFO:tasks.workunit.client.1.smithi121.stdout:0/347: stat d5/d6/f56 0 2024-02-15T18:35:23.479 INFO:tasks.workunit.client.1.smithi121.stdout:7/204: link d0/d5/d11/d1c/f1f d0/d38/d3c/f45 0 2024-02-15T18:35:23.479 INFO:tasks.workunit.client.1.smithi121.stdout:7/205: write d0/d5/d11/d1c/f23 [5072766,82473] 0 2024-02-15T18:35:23.482 INFO:tasks.workunit.client.1.smithi121.stdout:7/206: dread d0/d5/d11/d1c/f35 [0,4194304] 0 2024-02-15T18:35:23.482 INFO:tasks.workunit.client.1.smithi121.stdout:7/207: stat d0/l2f 0 2024-02-15T18:35:23.483 INFO:tasks.workunit.client.1.smithi121.stdout:6/286: truncate d7/dc/d2c/d36/f38 2748196 0 2024-02-15T18:35:23.485 INFO:tasks.workunit.client.1.smithi121.stdout:2/158: unlink d5/df/d27/d26/f29 0 2024-02-15T18:35:23.486 INFO:tasks.workunit.client.1.smithi121.stdout:6/287: mknod d7/dc/d2c/c65 0 2024-02-15T18:35:23.486 INFO:tasks.workunit.client.1.smithi121.stdout:6/288: chown d7/dc/d34 8412487 1 2024-02-15T18:35:23.486 INFO:tasks.workunit.client.1.smithi121.stdout:6/289: chown d7/dc/f2d 18431 1 2024-02-15T18:35:23.486 INFO:tasks.workunit.client.1.smithi121.stdout:6/290: chown d7/dc/d55/f5f 94257 1 2024-02-15T18:35:23.486 INFO:tasks.workunit.client.1.smithi121.stdout:2/159: link d5/df/d10/l1d d5/df/d10/d15/l2e 0 2024-02-15T18:35:23.495 INFO:tasks.workunit.client.1.smithi121.stdout:6/291: mkdir d7/dc/d34/d42/d62/d63/d66 0 2024-02-15T18:35:23.497 INFO:tasks.workunit.client.1.smithi121.stdout:2/160: creat d5/df/d10/d15/d22/f2f x:0 0 0 2024-02-15T18:35:23.499 INFO:tasks.workunit.client.1.smithi121.stdout:6/292: rmdir d7/dc/d34/d42 39 2024-02-15T18:35:23.499 INFO:tasks.workunit.client.1.smithi121.stdout:6/293: read - d7/dc/f37 zero size 2024-02-15T18:35:23.503 INFO:tasks.workunit.client.1.smithi121.stdout:6/294: symlink d7/dc/d34/d42/d62/d63/d66/l67 0 2024-02-15T18:35:23.503 INFO:tasks.workunit.client.1.smithi121.stdout:0/348: dwrite d5/d6/d2a/d36/f54 [0,4194304] 0 2024-02-15T18:35:23.503 INFO:tasks.workunit.client.1.smithi121.stdout:7/208: dread d0/f18 [0,4194304] 0 2024-02-15T18:35:23.504 INFO:tasks.workunit.client.1.smithi121.stdout:2/161: unlink d5/df/d10/d15/d22/f2f 0 2024-02-15T18:35:23.506 INFO:tasks.workunit.client.1.smithi121.stdout:6/295: symlink d7/dc/d55/l68 0 2024-02-15T18:35:23.506 INFO:tasks.workunit.client.1.smithi121.stdout:8/327: dwrite d0/d61/d66/f76 [0,4194304] 0 2024-02-15T18:35:23.506 INFO:tasks.workunit.client.1.smithi121.stdout:8/328: write d0/d7/d36/d52/f5b [402926,41840] 0 2024-02-15T18:35:23.507 INFO:tasks.workunit.client.1.smithi121.stdout:9/175: dwrite d7/f1c [0,4194304] 0 2024-02-15T18:35:23.507 INFO:tasks.workunit.client.1.smithi121.stdout:9/176: fdatasync d7/f1f 0 2024-02-15T18:35:23.507 INFO:tasks.workunit.client.1.smithi121.stdout:9/177: readlink d7/d8/d18/l21 0 2024-02-15T18:35:23.507 INFO:tasks.workunit.client.1.smithi121.stdout:9/178: readlink d7/ld 0 2024-02-15T18:35:23.508 INFO:tasks.workunit.client.1.smithi121.stdout:7/209: mkdir d0/d38/d46 0 2024-02-15T18:35:23.511 INFO:tasks.workunit.client.1.smithi121.stdout:6/296: mknod d7/dc/d34/c69 0 2024-02-15T18:35:23.511 INFO:tasks.workunit.client.1.smithi121.stdout:6/297: write d7/dc/f21 [1191190,6195] 0 2024-02-15T18:35:23.511 INFO:tasks.workunit.client.1.smithi121.stdout:6/298: write d7/dc/d2c/d36/f3e [1035112,51766] 0 2024-02-15T18:35:23.513 INFO:tasks.workunit.client.1.smithi121.stdout:9/179: write d7/f1c [2696939,65018] 0 2024-02-15T18:35:23.513 INFO:tasks.workunit.client.1.smithi121.stdout:9/180: stat d7/d8/d18/l24 0 2024-02-15T18:35:23.523 INFO:tasks.workunit.client.1.smithi121.stdout:0/349: mkdir d5/d6/d2a/d73 0 2024-02-15T18:35:23.524 INFO:tasks.workunit.client.1.smithi121.stdout:7/210: creat d0/d5/f47 x:0 0 0 2024-02-15T18:35:23.525 INFO:tasks.workunit.client.1.smithi121.stdout:6/299: creat d7/dc/d2c/d36/f6a x:0 0 0 2024-02-15T18:35:23.525 INFO:tasks.workunit.client.1.smithi121.stdout:9/181: mkdir d7/d3a/d42 0 2024-02-15T18:35:23.525 INFO:tasks.workunit.client.1.smithi121.stdout:9/182: rename d7 to d7/d8/d25/d30/d33/d3c/d43 22 2024-02-15T18:35:23.526 INFO:tasks.workunit.client.1.smithi121.stdout:9/183: fdatasync d7/d8/d25/d30/d33/d3e/f41 0 2024-02-15T18:35:23.527 INFO:tasks.workunit.client.1.smithi121.stdout:0/350: mkdir d5/d6/d74 0 2024-02-15T18:35:23.527 INFO:tasks.workunit.client.1.smithi121.stdout:0/351: stat d5/d6/dd/c29 0 2024-02-15T18:35:23.528 INFO:tasks.workunit.client.1.smithi121.stdout:0/352: chown d5/d6/d2a/d36/f51 250 1 2024-02-15T18:35:23.528 INFO:tasks.workunit.client.1.smithi121.stdout:7/211: creat d0/d5/d11/f48 x:0 0 0 2024-02-15T18:35:23.529 INFO:tasks.workunit.client.1.smithi121.stdout:6/300: write d7/dc/d34/d42/f54 [1793183,124646] 0 2024-02-15T18:35:23.530 INFO:tasks.workunit.client.1.smithi121.stdout:9/184: mknod d7/c44 0 2024-02-15T18:35:23.530 INFO:tasks.workunit.client.1.smithi121.stdout:8/329: dwrite d0/d3/f56 [0,4194304] 0 2024-02-15T18:35:23.530 INFO:tasks.workunit.client.1.smithi121.stdout:8/330: dread - d0/d4a/f77 zero size 2024-02-15T18:35:23.530 INFO:tasks.workunit.client.1.smithi121.stdout:0/353: write d5/d6/f1f [3574691,71087] 0 2024-02-15T18:35:23.530 INFO:tasks.workunit.client.1.smithi121.stdout:2/162: dwrite d5/df/d10/d15/f1c [0,4194304] 0 2024-02-15T18:35:23.530 INFO:tasks.workunit.client.1.smithi121.stdout:2/163: truncate d5/df/d27/f21 249630 0 2024-02-15T18:35:23.531 INFO:tasks.workunit.client.1.smithi121.stdout:7/212: link d0/d5/d11/l3a d0/l49 0 2024-02-15T18:35:23.532 INFO:tasks.workunit.client.1.smithi121.stdout:6/301: creat d7/dc/d34/d42/d62/f6b x:0 0 0 2024-02-15T18:35:23.534 INFO:tasks.workunit.client.1.smithi121.stdout:9/185: creat d7/d8/d18/f45 x:0 0 0 2024-02-15T18:35:23.539 INFO:tasks.workunit.client.1.smithi121.stdout:8/331: creat d0/d7/d3f/d45/f79 x:0 0 0 2024-02-15T18:35:23.540 INFO:tasks.workunit.client.1.smithi121.stdout:0/354: mknod d5/d6/d74/c75 0 2024-02-15T18:35:23.540 INFO:tasks.workunit.client.1.smithi121.stdout:0/355: stat d5/d6/dd/cf 0 2024-02-15T18:35:23.551 INFO:tasks.workunit.client.1.smithi121.stdout:2/164: symlink d5/df/d10/l30 0 2024-02-15T18:35:23.551 INFO:tasks.workunit.client.1.smithi121.stdout:2/165: truncate d5/df/d10/d15/f2d 565872 0 2024-02-15T18:35:23.551 INFO:tasks.workunit.client.1.smithi121.stdout:6/302: dwrite d7/f24 [4194304,4194304] 0 2024-02-15T18:35:23.552 INFO:tasks.workunit.client.1.smithi121.stdout:7/213: mkdir d0/d4a 0 2024-02-15T18:35:23.552 INFO:tasks.workunit.client.1.smithi121.stdout:9/186: mknod d7/d8/d25/d30/d33/d3e/c46 0 2024-02-15T18:35:23.554 INFO:tasks.workunit.client.1.smithi121.stdout:0/356: dread d5/f40 [0,4194304] 0 2024-02-15T18:35:23.554 INFO:tasks.workunit.client.1.smithi121.stdout:2/166: truncate d5/df/d10/f11 1209029 0 2024-02-15T18:35:23.555 INFO:tasks.workunit.client.1.smithi121.stdout:9/187: read d7/f1c [3542143,90856] 0 2024-02-15T18:35:23.561 INFO:tasks.workunit.client.1.smithi121.stdout:8/332: dwrite d0/d2a/d51/f38 [4194304,4194304] 0 2024-02-15T18:35:23.561 INFO:tasks.workunit.client.1.smithi121.stdout:8/333: fdatasync d0/d7/de/f1a 0 2024-02-15T18:35:23.562 INFO:tasks.workunit.client.1.smithi121.stdout:8/334: write d0/f5 [2639177,107610] 0 2024-02-15T18:35:23.580 INFO:tasks.workunit.client.1.smithi121.stdout:6/303: mkdir d7/d4b/d60/d6c 0 2024-02-15T18:35:23.580 INFO:tasks.workunit.client.1.smithi121.stdout:6/304: write d7/dc/d2c/d36/f3e [1663706,93062] 0 2024-02-15T18:35:23.582 INFO:tasks.workunit.client.1.smithi121.stdout:7/214: creat d0/d38/d3c/f4b x:0 0 0 2024-02-15T18:35:23.590 INFO:tasks.workunit.client.1.smithi121.stdout:0/357: mknod d5/d30/d5d/c76 0 2024-02-15T18:35:23.590 INFO:tasks.workunit.client.1.smithi121.stdout:2/167: mknod d5/df/d27/d26/c31 0 2024-02-15T18:35:23.590 INFO:tasks.workunit.client.1.smithi121.stdout:2/168: dread - d5/df/f2a zero size 2024-02-15T18:35:23.592 INFO:tasks.workunit.client.1.smithi121.stdout:9/188: creat d7/d8/f47 x:0 0 0 2024-02-15T18:35:23.592 INFO:tasks.workunit.client.1.smithi121.stdout:9/189: dread - d7/d8/d25/d30/f32 zero size 2024-02-15T18:35:23.592 INFO:tasks.workunit.client.1.smithi121.stdout:2/169: write d5/df/d10/d15/f1c [2301962,96952] 0 2024-02-15T18:35:23.595 INFO:tasks.workunit.client.1.smithi121.stdout:8/335: mkdir d0/d7/d36/d52/d7a 0 2024-02-15T18:35:23.597 INFO:tasks.workunit.client.1.smithi121.stdout:6/305: mknod d7/dc/d34/d42/d62/c6d 0 2024-02-15T18:35:23.597 INFO:tasks.workunit.client.1.smithi121.stdout:6/306: fsync d7/f18 0 2024-02-15T18:35:23.598 INFO:tasks.workunit.client.1.smithi121.stdout:7/215: dwrite d0/d38/d3c/f4b [0,4194304] 0 2024-02-15T18:35:23.603 INFO:tasks.workunit.client.1.smithi121.stdout:8/336: dread d0/d7/de/f2d [0,4194304] 0 2024-02-15T18:35:23.603 INFO:tasks.workunit.client.1.smithi121.stdout:8/337: truncate d0/d4a/f77 822288 0 2024-02-15T18:35:23.605 INFO:tasks.workunit.client.1.smithi121.stdout:5/197: sync 2024-02-15T18:35:23.616 INFO:tasks.workunit.client.1.smithi121.stdout:6/307: dwrite d7/dc/d2c/d36/f58 [0,4194304] 0 2024-02-15T18:35:23.616 INFO:tasks.workunit.client.1.smithi121.stdout:5/198: dread d0/d2/f25 [0,4194304] 0 2024-02-15T18:35:23.616 INFO:tasks.workunit.client.1.smithi121.stdout:5/199: truncate d0/d16/d34/f3a 910671 0 2024-02-15T18:35:23.616 INFO:tasks.workunit.client.1.smithi121.stdout:5/200: fdatasync d0/d2/d1e/d21/f2c 0 2024-02-15T18:35:23.616 INFO:tasks.workunit.client.1.smithi121.stdout:5/201: truncate d0/d2/d18/f27 456818 0 2024-02-15T18:35:23.616 INFO:tasks.workunit.client.1.smithi121.stdout:5/202: readlink d0/d2/l35 0 2024-02-15T18:35:23.616 INFO:tasks.workunit.client.1.smithi121.stdout:5/203: write d0/d2/d18/f31 [115166,17806] 0 2024-02-15T18:35:23.633 INFO:tasks.workunit.client.1.smithi121.stdout:6/308: dwrite d7/dc/f48 [0,4194304] 0 2024-02-15T18:35:23.634 INFO:tasks.workunit.client.1.smithi121.stdout:6/309: write d7/dc/d2c/d36/f3e [768365,19021] 0 2024-02-15T18:35:23.634 INFO:tasks.workunit.client.1.smithi121.stdout:6/310: chown d7/f47 2516 1 2024-02-15T18:35:23.634 INFO:tasks.workunit.client.1.smithi121.stdout:6/311: write d7/dc/d2c/d36/f3e [1796706,17282] 0 2024-02-15T18:35:23.634 INFO:tasks.workunit.client.1.smithi121.stdout:6/312: chown d7/l2b 1798 1 2024-02-15T18:35:23.643 INFO:tasks.workunit.client.1.smithi121.stdout:0/358: symlink d5/d6/d2a/d73/l77 0 2024-02-15T18:35:23.643 INFO:tasks.workunit.client.1.smithi121.stdout:0/359: fdatasync d5/d30/f4f 0 2024-02-15T18:35:23.643 INFO:tasks.workunit.client.1.smithi121.stdout:0/360: fsync d5/d6/d2a/f64 0 2024-02-15T18:35:23.643 INFO:tasks.workunit.client.1.smithi121.stdout:9/190: mkdir d7/d3a/d48 0 2024-02-15T18:35:23.648 INFO:tasks.workunit.client.1.smithi121.stdout:0/361: dread f2 [0,4194304] 0 2024-02-15T18:35:23.699 INFO:tasks.workunit.client.1.smithi121.stdout:7/216: truncate d0/f18 2141535 0 2024-02-15T18:35:23.700 INFO:tasks.workunit.client.1.smithi121.stdout:7/217: write d0/d5/f2b [1970226,117141] 0 2024-02-15T18:35:23.702 INFO:tasks.workunit.client.1.smithi121.stdout:8/338: mkdir d0/d61/d7b 0 2024-02-15T18:35:23.702 INFO:tasks.workunit.client.1.smithi121.stdout:8/339: readlink d0/l24 0 2024-02-15T18:35:23.706 INFO:tasks.workunit.client.1.smithi121.stdout:5/204: creat d0/d2/d1e/f42 x:0 0 0 2024-02-15T18:35:23.709 INFO:tasks.workunit.client.1.smithi121.stdout:8/340: dread d0/d2a/d51/f57 [0,4194304] 0 2024-02-15T18:35:23.714 INFO:tasks.workunit.client.1.smithi121.stdout:9/191: symlink d7/d8/d25/d30/d33/l49 0 2024-02-15T18:35:23.714 INFO:tasks.workunit.client.1.smithi121.stdout:9/192: write d7/d8/d36/f2f [134166,105143] 0 2024-02-15T18:35:23.715 INFO:tasks.workunit.client.1.smithi121.stdout:9/193: read d7/d8/f17 [1424612,61347] 0 2024-02-15T18:35:23.715 INFO:tasks.workunit.client.1.smithi121.stdout:9/194: chown d7/d26/c28 0 1 2024-02-15T18:35:23.716 INFO:tasks.workunit.client.1.smithi121.stdout:0/362: rmdir d5/d6/dd 39 2024-02-15T18:35:23.716 INFO:tasks.workunit.client.1.smithi121.stdout:0/363: dread - d5/d30/f6f zero size 2024-02-15T18:35:23.724 INFO:tasks.workunit.client.1.smithi121.stdout:7/218: mkdir d0/d5/d4c 0 2024-02-15T18:35:23.724 INFO:tasks.workunit.client.1.smithi121.stdout:7/219: chown d0/d5/d11/l3a 0 1 2024-02-15T18:35:23.725 INFO:tasks.workunit.client.1.smithi121.stdout:7/220: write d0/d33/f3b [404565,27501] 0 2024-02-15T18:35:23.725 INFO:tasks.workunit.client.1.smithi121.stdout:5/205: creat d0/d16/d34/f43 x:0 0 0 2024-02-15T18:35:23.726 INFO:tasks.workunit.client.1.smithi121.stdout:8/341: mkdir d0/d7/de/d7c 0 2024-02-15T18:35:23.727 INFO:tasks.workunit.client.1.smithi121.stdout:6/313: mkdir d7/dc/d34/d42/d6e 0 2024-02-15T18:35:23.729 INFO:tasks.workunit.client.1.smithi121.stdout:9/195: mknod d7/d26/c4a 0 2024-02-15T18:35:23.729 INFO:tasks.workunit.client.1.smithi121.stdout:0/364: mknod d5/d6/d2a/d36/d44/c78 0 2024-02-15T18:35:23.731 INFO:tasks.workunit.client.1.smithi121.stdout:8/342: link d0/d61/d66/f76 d0/d2a/f7d 0 2024-02-15T18:35:23.736 INFO:tasks.workunit.client.1.smithi121.stdout:9/196: mkdir d7/d8/d40/d4b 0 2024-02-15T18:35:23.736 INFO:tasks.workunit.client.1.smithi121.stdout:9/197: truncate d7/d8/f3b 404661 0 2024-02-15T18:35:23.737 INFO:tasks.workunit.client.1.smithi121.stdout:8/343: symlink d0/d7/d3f/d45/d53/l7e 0 2024-02-15T18:35:23.740 INFO:tasks.workunit.client.1.smithi121.stdout:9/198: creat d7/d8/d25/d30/f4c x:0 0 0 2024-02-15T18:35:23.740 INFO:tasks.workunit.client.1.smithi121.stdout:9/199: readlink d7/l29 0 2024-02-15T18:35:23.740 INFO:tasks.workunit.client.1.smithi121.stdout:9/200: fsync d7/d8/d18/f19 0 2024-02-15T18:35:23.741 INFO:tasks.workunit.client.1.smithi121.stdout:8/344: rename d0/d2a/f3c to d0/d61/d7b/f7f 0 2024-02-15T18:35:23.743 INFO:tasks.workunit.client.1.smithi121.stdout:8/345: creat d0/d61/d66/f80 x:0 0 0 2024-02-15T18:35:23.751 INFO:tasks.workunit.client.1.smithi121.stdout:5/206: dwrite d0/d2/d1e/f42 [0,4194304] 0 2024-02-15T18:35:23.752 INFO:tasks.workunit.client.1.smithi121.stdout:7/221: dwrite d0/d33/f3b [0,4194304] 0 2024-02-15T18:35:23.758 INFO:tasks.workunit.client.1.smithi121.stdout:5/207: dread d0/f17 [0,4194304] 0 2024-02-15T18:35:23.758 INFO:tasks.workunit.client.1.smithi121.stdout:5/208: fsync d0/d16/d34/f43 0 2024-02-15T18:35:23.758 INFO:tasks.workunit.client.1.smithi121.stdout:5/209: fdatasync d0/fb 0 2024-02-15T18:35:23.758 INFO:tasks.workunit.client.1.smithi121.stdout:7/222: stat d0/l49 0 2024-02-15T18:35:23.761 INFO:tasks.workunit.client.1.smithi121.stdout:7/223: rename d0/d38/d3c/f45 to d0/d38/d3c/f4d 0 2024-02-15T18:35:23.761 INFO:tasks.workunit.client.1.smithi121.stdout:7/224: fsync d0/d38/d3c/f4b 0 2024-02-15T18:35:23.762 INFO:tasks.workunit.client.1.smithi121.stdout:9/201: dwrite d7/f23 [0,4194304] 0 2024-02-15T18:35:23.762 INFO:tasks.workunit.client.1.smithi121.stdout:7/225: getdents d0/d33 0 2024-02-15T18:35:23.763 INFO:tasks.workunit.client.1.smithi121.stdout:5/210: write d0/d2/f14 [1227646,104393] 0 2024-02-15T18:35:23.768 INFO:tasks.workunit.client.1.smithi121.stdout:8/346: dwrite d0/d3/f56 [0,4194304] 0 2024-02-15T18:35:23.770 INFO:tasks.workunit.client.1.smithi121.stdout:9/202: rename d7/d8/c11 to d7/d8/d25/d30/d33/d3e/c4d 0 2024-02-15T18:35:23.770 INFO:tasks.workunit.client.1.smithi121.stdout:9/203: write d7/d8/f47 [965479,18347] 0 2024-02-15T18:35:23.771 INFO:tasks.workunit.client.1.smithi121.stdout:7/226: creat d0/d38/d3f/d43/f4e x:0 0 0 2024-02-15T18:35:23.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:22.591140+0000 mgr.smithi121.ckiryi (mgr.24385) 3 : cluster [DBG] pgmap v5: 65 pgs: 65 active+clean; 445 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:35:23.773 INFO:tasks.workunit.client.1.smithi121.stdout:5/211: dread d0/d2/d1e/f42 [0,4194304] 0 2024-02-15T18:35:23.776 INFO:tasks.workunit.client.1.smithi121.stdout:5/212: creat d0/d16/d34/f44 x:0 0 0 2024-02-15T18:35:23.778 INFO:tasks.workunit.client.1.smithi121.stdout:8/347: dread d0/d61/d66/f76 [0,4194304] 0 2024-02-15T18:35:23.778 INFO:tasks.workunit.client.1.smithi121.stdout:8/348: read - d0/d7/d36/f5c zero size 2024-02-15T18:35:23.779 INFO:tasks.workunit.client.1.smithi121.stdout:8/349: chown d0/d7/d3f/d45/d53/c5d 1598561 1 2024-02-15T18:35:23.779 INFO:tasks.workunit.client.1.smithi121.stdout:9/204: dread d7/f1c [0,4194304] 0 2024-02-15T18:35:23.782 INFO:tasks.workunit.client.1.smithi121.stdout:5/213: creat d0/d16/d34/f45 x:0 0 0 2024-02-15T18:35:23.782 INFO:tasks.workunit.client.1.smithi121.stdout:9/205: dread d7/f1f [0,4194304] 0 2024-02-15T18:35:23.784 INFO:tasks.workunit.client.1.smithi121.stdout:8/350: creat d0/d7/d36/f81 x:0 0 0 2024-02-15T18:35:23.787 INFO:tasks.workunit.client.1.smithi121.stdout:8/351: creat d0/d7/de/d7c/f82 x:0 0 0 2024-02-15T18:35:23.789 INFO:tasks.workunit.client.1.smithi121.stdout:9/206: dread f5 [0,4194304] 0 2024-02-15T18:35:23.789 INFO:tasks.workunit.client.1.smithi121.stdout:9/207: chown d7/d3a/d42 658 1 2024-02-15T18:35:23.789 INFO:tasks.workunit.client.1.smithi121.stdout:9/208: write d7/d8/f3b [4085,122665] 0 2024-02-15T18:35:23.790 INFO:tasks.workunit.client.1.smithi121.stdout:5/214: mknod d0/d2/c46 0 2024-02-15T18:35:23.792 INFO:tasks.workunit.client.1.smithi121.stdout:9/209: getdents d7/d8/d36 0 2024-02-15T18:35:23.792 INFO:tasks.workunit.client.1.smithi121.stdout:9/210: getdents d7/d8/d25/d30/d33/d3c/d3f 0 2024-02-15T18:35:23.794 INFO:tasks.workunit.client.1.smithi121.stdout:9/211: read d7/f1c [4065673,33887] 0 2024-02-15T18:35:23.794 INFO:tasks.workunit.client.1.smithi121.stdout:8/352: dread d0/f13 [0,4194304] 0 2024-02-15T18:35:23.794 INFO:tasks.workunit.client.1.smithi121.stdout:5/215: truncate d0/f22 1054503 0 2024-02-15T18:35:23.796 INFO:tasks.workunit.client.1.smithi121.stdout:8/353: getdents d0/d7/de 0 2024-02-15T18:35:23.796 INFO:tasks.workunit.client.1.smithi121.stdout:8/354: truncate d0/f15 5088767 0 2024-02-15T18:35:23.797 INFO:tasks.workunit.client.1.smithi121.stdout:8/355: write d0/d3/d20/f33 [212792,75749] 0 2024-02-15T18:35:23.797 INFO:tasks.workunit.client.1.smithi121.stdout:8/356: chown d0 3458593 1 2024-02-15T18:35:23.799 INFO:tasks.workunit.client.1.smithi121.stdout:9/212: dread d7/d8/f3b [0,4194304] 0 2024-02-15T18:35:23.801 INFO:tasks.workunit.client.1.smithi121.stdout:5/216: mkdir d0/d2/d1e/d21/d47 0 2024-02-15T18:35:23.803 INFO:tasks.workunit.client.1.smithi121.stdout:8/357: rename d0/d7/d3f/d45/d53/c5d to d0/d7/de/d7c/c83 0 2024-02-15T18:35:23.807 INFO:tasks.workunit.client.1.smithi121.stdout:9/213: getdents d7/d8/d40 0 2024-02-15T18:35:23.807 INFO:tasks.workunit.client.1.smithi121.stdout:9/214: chown d7/d8/d18/f19 121460738 1 2024-02-15T18:35:23.807 INFO:tasks.workunit.client.1.smithi121.stdout:9/215: truncate d7/d8/f1d 1177295 0 2024-02-15T18:35:23.811 INFO:tasks.workunit.client.1.smithi121.stdout:9/216: creat d7/f4e x:0 0 0 2024-02-15T18:35:23.811 INFO:tasks.workunit.client.1.smithi121.stdout:9/217: chown d7/d8/d25/d30/d33/d3c/d3f 1 1 2024-02-15T18:35:23.815 INFO:tasks.workunit.client.1.smithi121.stdout:8/358: rename d0/d7/de/d7c/c83 to d0/d2a/c84 0 2024-02-15T18:35:23.817 INFO:tasks.workunit.client.1.smithi121.stdout:5/217: truncate d0/d2/d1e/d21/f2c 679295 0 2024-02-15T18:35:23.818 INFO:tasks.workunit.client.1.smithi121.stdout:9/218: getdents d7/d8 0 2024-02-15T18:35:23.819 INFO:tasks.workunit.client.1.smithi121.stdout:5/218: mknod d0/d2/d1e/d21/d47/c48 0 2024-02-15T18:35:23.823 INFO:tasks.workunit.client.1.smithi121.stdout:5/219: symlink d0/d16/d34/l49 0 2024-02-15T18:35:23.824 INFO:tasks.workunit.client.1.smithi121.stdout:8/359: dread d0/d61/d66/f3b [0,4194304] 0 2024-02-15T18:35:23.826 INFO:tasks.workunit.client.1.smithi121.stdout:5/220: symlink d0/d16/d37/l4a 0 2024-02-15T18:35:23.826 INFO:tasks.workunit.client.1.smithi121.stdout:5/221: stat d0/d2/d18/c38 0 2024-02-15T18:35:23.828 INFO:tasks.workunit.client.1.smithi121.stdout:8/360: link d0/d48/f6a d0/d73/f85 0 2024-02-15T18:35:23.830 INFO:tasks.workunit.client.1.smithi121.stdout:9/219: truncate d7/d8/d18/f19 1806224 0 2024-02-15T18:35:23.832 INFO:tasks.workunit.client.1.smithi121.stdout:5/222: creat d0/f4b x:0 0 0 2024-02-15T18:35:23.832 INFO:tasks.workunit.client.1.smithi121.stdout:5/223: stat d0/d2/d1e/d21/d47/c48 0 2024-02-15T18:35:23.836 INFO:tasks.workunit.client.1.smithi121.stdout:8/361: dread d0/f2 [0,4194304] 0 2024-02-15T18:35:23.836 INFO:tasks.workunit.client.1.smithi121.stdout:8/362: write d0/d7/d3f/d45/f79 [618888,104944] 0 2024-02-15T18:35:23.836 INFO:tasks.workunit.client.1.smithi121.stdout:9/220: mkdir d7/d26/d4f 0 2024-02-15T18:35:23.837 INFO:tasks.workunit.client.1.smithi121.stdout:8/363: mkdir d0/d48/d86 0 2024-02-15T18:35:23.837 INFO:tasks.workunit.client.1.smithi121.stdout:9/221: rmdir d7/d8/d25 39 2024-02-15T18:35:23.839 INFO:tasks.workunit.client.1.smithi121.stdout:8/364: symlink d0/d48/d86/l87 0 2024-02-15T18:35:23.840 INFO:tasks.workunit.client.1.smithi121.stdout:9/222: symlink d7/d8/d25/l50 0 2024-02-15T18:35:23.840 INFO:tasks.workunit.client.1.smithi121.stdout:9/223: stat d7/d8/d25/d30 0 2024-02-15T18:35:23.841 INFO:tasks.workunit.client.1.smithi121.stdout:5/224: dread d0/d2/fd [0,4194304] 0 2024-02-15T18:35:23.850 INFO:tasks.workunit.client.1.smithi121.stdout:8/365: mknod d0/d4a/c88 0 2024-02-15T18:35:23.851 INFO:tasks.workunit.client.1.smithi121.stdout:9/224: truncate d7/f1f 17363 0 2024-02-15T18:35:23.852 INFO:tasks.workunit.client.1.smithi121.stdout:9/225: fsync d7/f1c 0 2024-02-15T18:35:23.854 INFO:tasks.workunit.client.1.smithi121.stdout:5/225: rmdir d0/d2/d1e 39 2024-02-15T18:35:23.858 INFO:tasks.workunit.client.1.smithi121.stdout:8/366: creat d0/d7/d3f/d45/f89 x:0 0 0 2024-02-15T18:35:23.859 INFO:tasks.workunit.client.1.smithi121.stdout:9/226: symlink d7/d8/d25/d30/d33/d3c/d3f/l51 0 2024-02-15T18:35:23.859 INFO:tasks.workunit.client.1.smithi121.stdout:9/227: truncate d7/d8/d25/d30/f32 862485 0 2024-02-15T18:35:23.860 INFO:tasks.workunit.client.1.smithi121.stdout:5/226: truncate d0/d2/f14 521519 0 2024-02-15T18:35:23.861 INFO:tasks.workunit.client.1.smithi121.stdout:8/367: creat d0/d7/d36/d52/d7a/f8a x:0 0 0 2024-02-15T18:35:23.864 INFO:tasks.workunit.client.1.smithi121.stdout:9/228: link d7/d8/d25/d30/f32 d7/d8/d40/f52 0 2024-02-15T18:35:23.864 INFO:tasks.workunit.client.1.smithi121.stdout:9/229: write d7/f1c [5072569,49538] 0 2024-02-15T18:35:23.864 INFO:tasks.workunit.client.1.smithi121.stdout:9/230: chown d7/d8/d40 3317 1 2024-02-15T18:35:23.865 INFO:tasks.workunit.client.1.smithi121.stdout:5/227: mkdir d0/d2/d1e/d21/d33/d4c 0 2024-02-15T18:35:23.868 INFO:tasks.workunit.client.1.smithi121.stdout:8/368: creat d0/d3/d20/f8b x:0 0 0 2024-02-15T18:35:23.873 INFO:tasks.workunit.client.1.smithi121.stdout:9/231: unlink d7/d26/c31 0 2024-02-15T18:35:23.873 INFO:tasks.workunit.client.1.smithi121.stdout:9/232: dread - d7/f4e zero size 2024-02-15T18:35:23.873 INFO:tasks.workunit.client.1.smithi121.stdout:9/233: write d7/d8/f1d [1997771,26275] 0 2024-02-15T18:35:23.873 INFO:tasks.workunit.client.1.smithi121.stdout:9/234: chown d7/d8/f1d 403 1 2024-02-15T18:35:23.873 INFO:tasks.workunit.client.1.smithi121.stdout:9/235: write d7/d8/d25/d30/d33/d3e/f41 [4343199,7373] 0 2024-02-15T18:35:23.878 INFO:tasks.workunit.client.1.smithi121.stdout:9/236: dread d7/d8/f17 [0,4194304] 0 2024-02-15T18:35:23.895 INFO:tasks.workunit.client.1.smithi121.stdout:5/228: dwrite d0/fb [0,4194304] 0 2024-02-15T18:35:23.895 INFO:tasks.workunit.client.1.smithi121.stdout:5/229: chown d0/d2/d1e/d21 2707247 1 2024-02-15T18:35:23.895 INFO:tasks.workunit.client.1.smithi121.stdout:5/230: chown d0/d2/d1e/d21/d39/l3b 326 1 2024-02-15T18:35:23.896 INFO:tasks.workunit.client.1.smithi121.stdout:9/237: dwrite d7/d8/f47 [0,4194304] 0 2024-02-15T18:35:23.897 INFO:tasks.workunit.client.1.smithi121.stdout:9/238: dread d7/d8/d25/d30/f32 [0,4194304] 0 2024-02-15T18:35:23.901 INFO:tasks.workunit.client.1.smithi121.stdout:5/231: getdents d0/d2 0 2024-02-15T18:35:23.911 INFO:tasks.workunit.client.1.smithi121.stdout:9/239: creat d7/d8/d18/f53 x:0 0 0 2024-02-15T18:35:23.911 INFO:tasks.workunit.client.1.smithi121.stdout:9/240: chown d7/d8/d18/f45 2546469 1 2024-02-15T18:35:23.911 INFO:tasks.workunit.client.1.smithi121.stdout:9/241: chown d7/d8/f10 13387 1 2024-02-15T18:35:23.911 INFO:tasks.workunit.client.1.smithi121.stdout:5/232: link d0/d2/l3d d0/d2/d1e/d21/d33/l4d 0 2024-02-15T18:35:23.911 INFO:tasks.workunit.client.1.smithi121.stdout:5/233: fdatasync d0/f22 0 2024-02-15T18:35:23.913 INFO:tasks.workunit.client.1.smithi121.stdout:9/242: creat d7/d8/d25/d30/d33/d3c/f54 x:0 0 0 2024-02-15T18:35:23.914 INFO:tasks.workunit.client.1.smithi121.stdout:9/243: symlink d7/d8/d40/l55 0 2024-02-15T18:35:23.914 INFO:tasks.workunit.client.1.smithi121.stdout:9/244: chown d7/d8/f38 604656629 1 2024-02-15T18:35:23.915 INFO:tasks.workunit.client.1.smithi121.stdout:9/245: write d7/d8/f1d [2984324,54894] 0 2024-02-15T18:35:23.915 INFO:tasks.workunit.client.1.smithi121.stdout:9/246: mkdir d7/d3a/d42/d56 0 2024-02-15T18:35:23.916 INFO:tasks.workunit.client.1.smithi121.stdout:9/247: creat d7/f57 x:0 0 0 2024-02-15T18:35:23.917 INFO:tasks.workunit.client.1.smithi121.stdout:5/234: dread d0/d2/f25 [0,4194304] 0 2024-02-15T18:35:23.917 INFO:tasks.workunit.client.1.smithi121.stdout:5/235: chown d0/d16/d34/f3a 3389 1 2024-02-15T18:35:23.918 INFO:tasks.workunit.client.1.smithi121.stdout:5/236: mkdir d0/d2/d1e/d4e 0 2024-02-15T18:35:23.918 INFO:tasks.workunit.client.1.smithi121.stdout:9/248: dread d7/d8/d36/f2f [0,4194304] 0 2024-02-15T18:35:23.918 INFO:tasks.workunit.client.1.smithi121.stdout:5/237: symlink d0/d2/l4f 0 2024-02-15T18:35:23.923 INFO:tasks.workunit.client.1.smithi121.stdout:5/238: dread d0/d2/d1e/f42 [0,4194304] 0 2024-02-15T18:35:23.925 INFO:tasks.workunit.client.1.smithi121.stdout:5/239: rename d0/d2/l35 to d0/d2/d1e/d21/l50 0 2024-02-15T18:35:23.926 INFO:tasks.workunit.client.1.smithi121.stdout:5/240: unlink d0/f17 0 2024-02-15T18:35:23.926 INFO:tasks.workunit.client.1.smithi121.stdout:5/241: chown d0/d2/l3f 111463 1 2024-02-15T18:35:23.927 INFO:tasks.workunit.client.1.smithi121.stdout:5/242: creat d0/d2/d1e/d21/d39/f51 x:0 0 0 2024-02-15T18:35:23.927 INFO:tasks.workunit.client.1.smithi121.stdout:5/243: truncate d0/d2/d1e/d21/f2c 263119 0 2024-02-15T18:35:23.927 INFO:tasks.workunit.client.1.smithi121.stdout:5/244: chown d0/d2/d18/f31 975 1 2024-02-15T18:35:23.928 INFO:tasks.workunit.client.1.smithi121.stdout:5/245: mknod d0/d2/d1e/d21/c52 0 2024-02-15T18:35:23.928 INFO:tasks.workunit.client.1.smithi121.stdout:5/246: write d0/d2/d18/f29 [810166,100011] 0 2024-02-15T18:35:23.945 INFO:tasks.workunit.client.1.smithi121.stdout:5/247: dwrite d0/d16/d34/f44 [0,4194304] 0 2024-02-15T18:35:23.945 INFO:tasks.workunit.client.1.smithi121.stdout:5/248: dread - d0/d16/d34/f45 zero size 2024-02-15T18:35:23.947 INFO:tasks.workunit.client.1.smithi121.stdout:5/249: creat d0/d2/d1e/d21/d39/f53 x:0 0 0 2024-02-15T18:35:23.948 INFO:tasks.workunit.client.1.smithi121.stdout:5/250: creat d0/d2/d1e/d21/d33/d4c/f54 x:0 0 0 2024-02-15T18:35:23.948 INFO:tasks.workunit.client.1.smithi121.stdout:5/251: creat d0/d2/d1e/d4e/f55 x:0 0 0 2024-02-15T18:35:23.949 INFO:tasks.workunit.client.1.smithi121.stdout:5/252: truncate d0/d2/d18/f20 4047588 0 2024-02-15T18:35:23.949 INFO:tasks.workunit.client.1.smithi121.stdout:5/253: dread - d0/d16/d34/f43 zero size 2024-02-15T18:35:23.950 INFO:tasks.workunit.client.1.smithi121.stdout:5/254: rmdir d0/d16 39 2024-02-15T18:35:23.951 INFO:tasks.workunit.client.1.smithi121.stdout:5/255: rmdir d0/d2 39 2024-02-15T18:35:23.951 INFO:tasks.workunit.client.1.smithi121.stdout:5/256: stat d0/d2/d1e/d21/l24 0 2024-02-15T18:35:23.952 INFO:tasks.workunit.client.1.smithi121.stdout:5/257: creat d0/d2/d1e/d21/d33/f56 x:0 0 0 2024-02-15T18:35:23.953 INFO:tasks.workunit.client.1.smithi121.stdout:5/258: rename d0/d2/d1e/d21/d40/l41 to d0/d2/d18/l57 0 2024-02-15T18:35:23.954 INFO:tasks.workunit.client.1.smithi121.stdout:3/305: sync 2024-02-15T18:35:23.954 INFO:tasks.workunit.client.1.smithi121.stdout:4/159: sync 2024-02-15T18:35:23.954 INFO:tasks.workunit.client.1.smithi121.stdout:5/259: creat d0/d16/f58 x:0 0 0 2024-02-15T18:35:23.957 INFO:tasks.workunit.client.1.smithi121.stdout:3/306: read d2/fe [4003772,127639] 0 2024-02-15T18:35:23.957 INFO:tasks.workunit.client.1.smithi121.stdout:3/307: truncate d2/d1a/d27/d2c/d25/d31/f50 322431 0 2024-02-15T18:35:23.967 INFO:tasks.workunit.client.1.smithi121.stdout:9/249: getdents d7/d3a/d42 0 2024-02-15T18:35:23.983 INFO:tasks.workunit.client.1.smithi121.stdout:3/308: dwrite d2/d1a/d22/f42 [0,4194304] 0 2024-02-15T18:35:23.983 INFO:tasks.workunit.client.1.smithi121.stdout:3/309: dread - d2/d1a/d22/f35 zero size 2024-02-15T18:35:23.985 INFO:tasks.workunit.client.1.smithi121.stdout:5/260: dwrite d0/d2/d1e/d21/d39/f51 [0,4194304] 0 2024-02-15T18:35:23.985 INFO:tasks.workunit.client.1.smithi121.stdout:4/160: dwrite d5/fa [4194304,4194304] 0 2024-02-15T18:35:23.987 INFO:tasks.workunit.client.1.smithi121.stdout:3/310: rmdir d2/d1a/d27/d2c/d25/d31/d58 39 2024-02-15T18:35:23.992 INFO:tasks.workunit.client.1.smithi121.stdout:5/261: creat d0/d2/d1e/d21/d33/f59 x:0 0 0 2024-02-15T18:35:23.992 INFO:tasks.workunit.client.1.smithi121.stdout:5/262: truncate d0/f22 898987 0 2024-02-15T18:35:23.992 INFO:tasks.workunit.client.1.smithi121.stdout:5/263: write d0/d2/d18/f20 [4978301,35649] 0 2024-02-15T18:35:23.992 INFO:tasks.workunit.client.1.smithi121.stdout:4/161: creat d5/f30 x:0 0 0 2024-02-15T18:35:23.992 INFO:tasks.workunit.client.1.smithi121.stdout:4/162: write d5/d9/d1d/f22 [4650396,57598] 0 2024-02-15T18:35:23.993 INFO:tasks.workunit.client.1.smithi121.stdout:3/311: mkdir d2/d1a/d6b 0 2024-02-15T18:35:23.993 INFO:tasks.workunit.client.1.smithi121.stdout:3/312: dread - d2/d1a/d22/f53 zero size 2024-02-15T18:35:23.994 INFO:tasks.workunit.client.1.smithi121.stdout:4/163: write d5/d9/d19/d2a/f2c [431526,5989] 0 2024-02-15T18:35:23.994 INFO:tasks.workunit.client.1.smithi121.stdout:3/313: creat d2/d1a/d27/f6c x:0 0 0 2024-02-15T18:35:23.996 INFO:tasks.workunit.client.1.smithi121.stdout:3/314: link d2/d1a/d27/f6c d2/d1a/d27/d2c/d25/d31/d34/d4a/f6d 0 2024-02-15T18:35:24.006 INFO:tasks.workunit.client.1.smithi121.stdout:9/250: dwrite d7/d8/f1d [0,4194304] 0 2024-02-15T18:35:24.013 INFO:tasks.workunit.client.1.smithi121.stdout:3/315: dwrite d2/d1a/d27/d2c/f1b [0,4194304] 0 2024-02-15T18:35:24.015 INFO:tasks.workunit.client.1.smithi121.stdout:3/316: unlink d2/d1a/d27/d2c/d25/d30/f62 0 2024-02-15T18:35:24.015 INFO:tasks.workunit.client.1.smithi121.stdout:3/317: write d2/d1a/d27/d2c/d25/d31/f57 [1218986,112612] 0 2024-02-15T18:35:24.022 INFO:tasks.workunit.client.1.smithi121.stdout:9/251: dwrite d7/d8/f1d [0,4194304] 0 2024-02-15T18:35:24.036 INFO:tasks.workunit.client.1.smithi121.stdout:9/252: dwrite d7/d8/d18/f53 [0,4194304] 0 2024-02-15T18:35:24.039 INFO:tasks.workunit.client.1.smithi121.stdout:9/253: creat d7/d8/d25/d30/d33/f58 x:0 0 0 2024-02-15T18:35:24.039 INFO:tasks.workunit.client.1.smithi121.stdout:9/254: read - d7/f4e zero size 2024-02-15T18:35:24.040 INFO:tasks.workunit.client.1.smithi121.stdout:9/255: fsync d7/d8/f38 0 2024-02-15T18:35:24.040 INFO:tasks.workunit.client.1.smithi121.stdout:9/256: write d7/f57 [161862,26554] 0 2024-02-15T18:35:24.040 INFO:tasks.workunit.client.1.smithi121.stdout:9/257: truncate d7/f1f 45848 0 2024-02-15T18:35:24.040 INFO:tasks.workunit.client.1.smithi121.stdout:9/258: write d7/f57 [1188875,63589] 0 2024-02-15T18:35:24.043 INFO:tasks.workunit.client.1.smithi121.stdout:9/259: creat d7/f59 x:0 0 0 2024-02-15T18:35:24.044 INFO:tasks.workunit.client.1.smithi121.stdout:9/260: mknod d7/d8/d25/d30/d33/c5a 0 2024-02-15T18:35:24.044 INFO:tasks.workunit.client.1.smithi121.stdout:9/261: chown d7/f1f 14574831 1 2024-02-15T18:35:24.045 INFO:tasks.workunit.client.1.smithi121.stdout:9/262: rmdir d7/d8/d40 39 2024-02-15T18:35:24.048 INFO:tasks.workunit.client.1.smithi121.stdout:9/263: rename d7/d3a/d48 to d7/d26/d4f/d5b 0 2024-02-15T18:35:24.049 INFO:tasks.workunit.client.1.smithi121.stdout:9/264: write d7/d8/f47 [2408354,76928] 0 2024-02-15T18:35:24.049 INFO:tasks.workunit.client.1.smithi121.stdout:9/265: read d7/f57 [1115982,76603] 0 2024-02-15T18:35:24.074 INFO:tasks.workunit.client.1.smithi121.stdout:1/265: sync 2024-02-15T18:35:24.074 INFO:tasks.workunit.client.1.smithi121.stdout:2/170: sync 2024-02-15T18:35:24.074 INFO:tasks.workunit.client.1.smithi121.stdout:6/314: sync 2024-02-15T18:35:24.074 INFO:tasks.workunit.client.1.smithi121.stdout:6/315: stat d7/f8 0 2024-02-15T18:35:24.075 INFO:tasks.workunit.client.1.smithi121.stdout:1/266: truncate d0/d6/d7/d9/f35 360756 0 2024-02-15T18:35:24.075 INFO:tasks.workunit.client.1.smithi121.stdout:1/267: stat d0/d6/f12 0 2024-02-15T18:35:24.075 INFO:tasks.workunit.client.1.smithi121.stdout:2/171: dread d5/df/d10/d15/f2d [0,4194304] 0 2024-02-15T18:35:24.075 INFO:tasks.workunit.client.1.smithi121.stdout:2/172: readlink d5/df/l12 0 2024-02-15T18:35:24.078 INFO:tasks.workunit.client.1.smithi121.stdout:1/268: rename d0/d6/d7/d9/f52 to d0/d39/f5c 0 2024-02-15T18:35:24.078 INFO:tasks.workunit.client.1.smithi121.stdout:1/269: truncate d0/d6/f38 864845 0 2024-02-15T18:35:24.078 INFO:tasks.workunit.client.1.smithi121.stdout:1/270: dread - d0/f47 zero size 2024-02-15T18:35:24.079 INFO:tasks.workunit.client.1.smithi121.stdout:2/173: dread f4 [0,4194304] 0 2024-02-15T18:35:24.079 INFO:tasks.workunit.client.1.smithi121.stdout:2/174: chown d5/df/d10/c2c 116556 1 2024-02-15T18:35:24.079 INFO:tasks.workunit.client.1.smithi121.stdout:2/175: rename d5/df/d10/d15 to d5/df/d10/d15/d22/d32 22 2024-02-15T18:35:24.081 INFO:tasks.workunit.client.1.smithi121.stdout:1/271: write d0/d6/d7/f26 [2439121,3992] 0 2024-02-15T18:35:24.084 INFO:tasks.workunit.client.1.smithi121.stdout:1/272: creat d0/d6/d7/d29/f5d x:0 0 0 2024-02-15T18:35:24.085 INFO:tasks.workunit.client.1.smithi121.stdout:1/273: creat d0/d6/d7/f5e x:0 0 0 2024-02-15T18:35:24.106 INFO:tasks.workunit.client.1.smithi121.stdout:6/316: dwrite d7/dc/f5e [0,4194304] 0 2024-02-15T18:35:24.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:22.591140+0000 mgr.smithi121.ckiryi (mgr.24385) 3 : cluster [DBG] pgmap v5: 65 pgs: 65 active+clean; 445 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:35:24.107 INFO:tasks.workunit.client.1.smithi121.stdout:2/176: dwrite d5/f8 [0,4194304] 0 2024-02-15T18:35:24.107 INFO:tasks.workunit.client.1.smithi121.stdout:2/177: chown f4 3 1 2024-02-15T18:35:24.109 INFO:tasks.workunit.client.1.smithi121.stdout:1/274: dwrite d0/d6/d7/d9/f35 [0,4194304] 0 2024-02-15T18:35:24.109 INFO:tasks.workunit.client.1.smithi121.stdout:2/178: dread f4 [0,4194304] 0 2024-02-15T18:35:24.120 INFO:tasks.workunit.client.1.smithi121.stdout:1/275: mkdir d0/d6/d7/d9/d51/d5f 0 2024-02-15T18:35:24.133 INFO:tasks.workunit.client.1.smithi121.stdout:2/179: mkdir d5/df/d27/d26/d33 0 2024-02-15T18:35:24.146 INFO:tasks.workunit.client.1.smithi121.stdout:1/276: dwrite d0/d6/d7/d9/d14/f2f [4194304,4194304] 0 2024-02-15T18:35:24.146 INFO:tasks.workunit.client.1.smithi121.stdout:1/277: stat d0/d6/f19 0 2024-02-15T18:35:24.146 INFO:tasks.workunit.client.1.smithi121.stdout:1/278: dread - d0/f53 zero size 2024-02-15T18:35:24.149 INFO:tasks.workunit.client.1.smithi121.stdout:1/279: unlink d0/c28 0 2024-02-15T18:35:24.153 INFO:tasks.workunit.client.1.smithi121.stdout:2/180: dwrite d5/df/d10/f11 [0,4194304] 0 2024-02-15T18:35:24.158 INFO:tasks.workunit.client.1.smithi121.stdout:1/280: dread d0/d6/d7/f26 [0,4194304] 0 2024-02-15T18:35:24.160 INFO:tasks.workunit.client.1.smithi121.stdout:1/281: unlink d0/d6/d7/d9/d14/f2f 0 2024-02-15T18:35:24.160 INFO:tasks.workunit.client.1.smithi121.stdout:1/282: readlink d0/d6/d7/d29/l4f 0 2024-02-15T18:35:24.179 INFO:tasks.workunit.client.1.smithi121.stdout:1/283: creat d0/d6/d7/d9/d14/d48/f60 x:0 0 0 2024-02-15T18:35:24.181 INFO:tasks.workunit.client.1.smithi121.stdout:1/284: creat d0/d39/d4b/f61 x:0 0 0 2024-02-15T18:35:24.182 INFO:tasks.workunit.client.1.smithi121.stdout:1/285: truncate d0/d6/f12 2302932 0 2024-02-15T18:35:24.182 INFO:tasks.workunit.client.1.smithi121.stdout:1/286: chown d0/d6/d7/d9/d14/c24 12726 1 2024-02-15T18:35:24.184 INFO:tasks.workunit.client.1.smithi121.stdout:1/287: creat d0/d6/d7/d9/d51/d5f/f62 x:0 0 0 2024-02-15T18:35:24.184 INFO:tasks.workunit.client.1.smithi121.stdout:1/288: truncate d0/d6/d7/f4c 234746 0 2024-02-15T18:35:24.186 INFO:tasks.workunit.client.1.smithi121.stdout:1/289: dread d0/d6/f38 [0,4194304] 0 2024-02-15T18:35:24.186 INFO:tasks.workunit.client.1.smithi121.stdout:1/290: write d0/f53 [586818,19730] 0 2024-02-15T18:35:24.186 INFO:tasks.workunit.client.1.smithi121.stdout:1/291: fsync d0/d6/d7/d29/f54 0 2024-02-15T18:35:24.186 INFO:tasks.workunit.client.1.smithi121.stdout:1/292: chown d0/c56 4744 1 2024-02-15T18:35:24.186 INFO:tasks.workunit.client.1.smithi121.stdout:1/293: truncate d0/d6/d7/d9/d51/d5f/f62 129769 0 2024-02-15T18:35:24.186 INFO:tasks.workunit.client.1.smithi121.stdout:1/294: chown d0/d39/c45 1596 1 2024-02-15T18:35:24.187 INFO:tasks.workunit.client.1.smithi121.stdout:2/181: dwrite f4 [0,4194304] 0 2024-02-15T18:35:24.193 INFO:tasks.workunit.client.1.smithi121.stdout:2/182: symlink d5/df/d10/d15/l34 0 2024-02-15T18:35:24.195 INFO:tasks.workunit.client.1.smithi121.stdout:2/183: mkdir d5/df/d27/d26/d35 0 2024-02-15T18:35:24.282 INFO:tasks.workunit.client.1.smithi121.stdout:2/184: truncate d5/f8 2466170 0 2024-02-15T18:35:24.285 INFO:tasks.workunit.client.1.smithi121.stdout:2/185: dread d5/fc [0,4194304] 0 2024-02-15T18:35:24.286 INFO:tasks.workunit.client.1.smithi121.stdout:1/295: dwrite d0/f47 [0,4194304] 0 2024-02-15T18:35:24.286 INFO:tasks.workunit.client.1.smithi121.stdout:1/296: write d0/d39/f5c [165645,78967] 0 2024-02-15T18:35:24.287 INFO:tasks.workunit.client.1.smithi121.stdout:2/186: creat d5/df/d10/d15/d18/f36 x:0 0 0 2024-02-15T18:35:24.290 INFO:tasks.workunit.client.1.smithi121.stdout:1/297: symlink d0/d6/d7/d9/l63 0 2024-02-15T18:35:24.292 INFO:tasks.workunit.client.1.smithi121.stdout:2/187: creat d5/df/d10/d15/d22/f37 x:0 0 0 2024-02-15T18:35:24.296 INFO:tasks.workunit.client.1.smithi121.stdout:1/298: unlink d0/d6/f38 0 2024-02-15T18:35:24.297 INFO:tasks.workunit.client.1.smithi121.stdout:2/188: unlink d5/c7 0 2024-02-15T18:35:24.297 INFO:tasks.workunit.client.1.smithi121.stdout:2/189: stat d5/df/d27 0 2024-02-15T18:35:24.302 INFO:tasks.workunit.client.1.smithi121.stdout:1/299: getdents d0/d6/d7/d9/d14/d48 0 2024-02-15T18:35:24.303 INFO:tasks.workunit.client.1.smithi121.stdout:2/190: mknod d5/df/d27/c38 0 2024-02-15T18:35:24.303 INFO:tasks.workunit.client.1.smithi121.stdout:2/191: getdents d5/df/d27/d26/d33 0 2024-02-15T18:35:24.303 INFO:tasks.workunit.client.1.smithi121.stdout:2/192: write d5/f8 [3183428,12293] 0 2024-02-15T18:35:24.303 INFO:tasks.workunit.client.1.smithi121.stdout:2/193: dread - d5/df/f2b zero size 2024-02-15T18:35:24.304 INFO:tasks.workunit.client.1.smithi121.stdout:1/300: symlink d0/d6/d7/d9/d14/l64 0 2024-02-15T18:35:24.304 INFO:tasks.workunit.client.1.smithi121.stdout:1/301: chown d0/d6/d7/d9/d14/d48/f60 350930 1 2024-02-15T18:35:24.312 INFO:tasks.workunit.client.1.smithi121.stdout:2/194: creat d5/df/f39 x:0 0 0 2024-02-15T18:35:24.313 INFO:tasks.workunit.client.1.smithi121.stdout:1/302: creat d0/d6/d7/f65 x:0 0 0 2024-02-15T18:35:24.318 INFO:tasks.workunit.client.1.smithi121.stdout:1/303: rename d0/l17 to d0/d6/d7/d29/l66 0 2024-02-15T18:35:24.319 INFO:tasks.workunit.client.1.smithi121.stdout:1/304: write d0/d6/d7/d9/f20 [4689585,55228] 0 2024-02-15T18:35:24.319 INFO:tasks.workunit.client.1.smithi121.stdout:1/305: chown d0/d6/d7/c59 192 1 2024-02-15T18:35:24.327 INFO:tasks.workunit.client.1.smithi121.stdout:2/195: dwrite d5/df/d10/d15/d22/f37 [0,4194304] 0 2024-02-15T18:35:24.328 INFO:tasks.workunit.client.1.smithi121.stdout:1/306: symlink d0/d39/d4b/l67 0 2024-02-15T18:35:24.332 INFO:tasks.workunit.client.1.smithi121.stdout:2/196: mkdir d5/df/d27/d3a 0 2024-02-15T18:35:24.332 INFO:tasks.workunit.client.1.smithi121.stdout:2/197: write d5/df/d10/d15/d18/f36 [649676,113249] 0 2024-02-15T18:35:24.334 INFO:tasks.workunit.client.1.smithi121.stdout:2/198: write d5/df/d10/d15/d22/f37 [2600101,35101] 0 2024-02-15T18:35:24.336 INFO:tasks.workunit.client.1.smithi121.stdout:2/199: rename d5/df/d10/d15/d18/f36 to d5/f3b 0 2024-02-15T18:35:24.336 INFO:tasks.workunit.client.1.smithi121.stdout:2/200: truncate d5/df/f2b 52485 0 2024-02-15T18:35:24.336 INFO:tasks.workunit.client.1.smithi121.stdout:2/201: write d5/df/d27/f21 [912738,70282] 0 2024-02-15T18:35:24.336 INFO:tasks.workunit.client.1.smithi121.stdout:2/202: readlink d5/df/d10/d15/l34 0 2024-02-15T18:35:24.337 INFO:tasks.workunit.client.1.smithi121.stdout:1/307: dread d0/d6/d7/d9/f35 [0,4194304] 0 2024-02-15T18:35:24.337 INFO:tasks.workunit.client.1.smithi121.stdout:1/308: write d0/d6/f1a [1775370,53835] 0 2024-02-15T18:35:24.338 INFO:tasks.workunit.client.1.smithi121.stdout:2/203: creat d5/df/d10/d15/d22/f3c x:0 0 0 2024-02-15T18:35:24.338 INFO:tasks.workunit.client.1.smithi121.stdout:1/309: read d0/d6/d7/d9/f20 [743494,118872] 0 2024-02-15T18:35:24.339 INFO:tasks.workunit.client.1.smithi121.stdout:1/310: write d0/d6/d7/f26 [1905139,69969] 0 2024-02-15T18:35:24.339 INFO:tasks.workunit.client.1.smithi121.stdout:1/311: chown d0/d6/d7/d9/d51/d5f 2895418 1 2024-02-15T18:35:24.346 INFO:tasks.workunit.client.1.smithi121.stdout:2/204: dread d5/fd [0,4194304] 0 2024-02-15T18:35:24.348 INFO:tasks.workunit.client.1.smithi121.stdout:2/205: rename d5/df/d27/ce to d5/df/d10/d15/c3d 0 2024-02-15T18:35:24.355 INFO:tasks.workunit.client.1.smithi121.stdout:2/206: fdatasync d5/df/d10/d15/d22/f37 0 2024-02-15T18:35:24.357 INFO:tasks.workunit.client.1.smithi121.stdout:1/312: dwrite d0/f21 [0,4194304] 0 2024-02-15T18:35:24.357 INFO:tasks.workunit.client.1.smithi121.stdout:1/313: chown d0/d6/d7/d9/d14/d48 227 1 2024-02-15T18:35:24.357 INFO:tasks.workunit.client.1.smithi121.stdout:1/314: write d0/f53 [1185539,45060] 0 2024-02-15T18:35:24.357 INFO:tasks.workunit.client.1.smithi121.stdout:1/315: dread - d0/d6/d7/d29/f5d zero size 2024-02-15T18:35:24.357 INFO:tasks.workunit.client.1.smithi121.stdout:1/316: chown d0/d6/d7/c3c 897 1 2024-02-15T18:35:24.358 INFO:tasks.workunit.client.1.smithi121.stdout:2/207: dread d5/fd [0,4194304] 0 2024-02-15T18:35:24.358 INFO:tasks.workunit.client.1.smithi121.stdout:1/317: creat d0/d39/f68 x:0 0 0 2024-02-15T18:35:24.373 INFO:tasks.workunit.client.1.smithi121.stdout:1/318: fsync d0/d39/f68 0 2024-02-15T18:35:24.374 INFO:tasks.workunit.client.1.smithi121.stdout:1/319: mknod d0/d6/d7/d9/d51/c69 0 2024-02-15T18:35:24.374 INFO:tasks.workunit.client.1.smithi121.stdout:1/320: dread - d0/d6/d7/d9/d14/f58 zero size 2024-02-15T18:35:24.376 INFO:tasks.workunit.client.1.smithi121.stdout:2/208: dwrite d5/f8 [0,4194304] 0 2024-02-15T18:35:24.376 INFO:tasks.workunit.client.1.smithi121.stdout:2/209: fsync d5/df/f1b 0 2024-02-15T18:35:24.377 INFO:tasks.workunit.client.1.smithi121.stdout:1/321: write d0/d6/d7/d9/f23 [3400208,37543] 0 2024-02-15T18:35:24.377 INFO:tasks.workunit.client.1.smithi121.stdout:2/210: dread d5/fc [0,4194304] 0 2024-02-15T18:35:24.377 INFO:tasks.workunit.client.1.smithi121.stdout:2/211: write d5/df/d10/d15/d22/f3c [834246,104867] 0 2024-02-15T18:35:24.380 INFO:tasks.workunit.client.1.smithi121.stdout:2/212: rmdir d5/df/d10/d15 39 2024-02-15T18:35:24.381 INFO:tasks.workunit.client.1.smithi121.stdout:2/213: read d5/df/d10/d15/f1c [2535446,54315] 0 2024-02-15T18:35:24.381 INFO:tasks.workunit.client.1.smithi121.stdout:2/214: stat d5/df/d10/d15/l2e 0 2024-02-15T18:35:24.383 INFO:tasks.workunit.client.1.smithi121.stdout:0/365: sync 2024-02-15T18:35:24.383 INFO:tasks.workunit.client.1.smithi121.stdout:7/227: sync 2024-02-15T18:35:24.384 INFO:tasks.workunit.client.1.smithi121.stdout:7/228: write d0/f18 [1740607,90886] 0 2024-02-15T18:35:24.385 INFO:tasks.workunit.client.1.smithi121.stdout:2/215: rename d5/df/d27/d26 to d5/df/d27/d3e 0 2024-02-15T18:35:24.385 INFO:tasks.workunit.client.1.smithi121.stdout:2/216: write d5/df/d10/d15/f2d [1394635,103108] 0 2024-02-15T18:35:24.386 INFO:tasks.workunit.client.1.smithi121.stdout:2/217: dread d5/df/f2b [0,4194304] 0 2024-02-15T18:35:24.388 INFO:tasks.workunit.client.1.smithi121.stdout:2/218: write d5/df/d10/d15/d22/f37 [1047535,54827] 0 2024-02-15T18:35:24.388 INFO:tasks.workunit.client.1.smithi121.stdout:2/219: chown d5/df 230920432 1 2024-02-15T18:35:24.388 INFO:tasks.workunit.client.1.smithi121.stdout:0/366: rename d5/d6/d2a/d36/d44/c61 to d5/d6/d2a/d73/c79 0 2024-02-15T18:35:24.392 INFO:tasks.workunit.client.1.smithi121.stdout:7/229: mknod d0/d5/d11/c4f 0 2024-02-15T18:35:24.392 INFO:tasks.workunit.client.1.smithi121.stdout:7/230: fdatasync d0/f42 0 2024-02-15T18:35:24.393 INFO:tasks.workunit.client.1.smithi121.stdout:7/231: write d0/d5/d11/d1c/f23 [4311738,90771] 0 2024-02-15T18:35:24.393 INFO:tasks.workunit.client.1.smithi121.stdout:2/220: stat d5/df/d27/d3e/l28 0 2024-02-15T18:35:24.395 INFO:tasks.workunit.client.1.smithi121.stdout:0/367: creat d5/d6/d2a/d73/f7a x:0 0 0 2024-02-15T18:35:24.396 INFO:tasks.workunit.client.1.smithi121.stdout:7/232: symlink d0/d4a/l50 0 2024-02-15T18:35:24.396 INFO:tasks.workunit.client.1.smithi121.stdout:1/322: dwrite d0/f47 [0,4194304] 0 2024-02-15T18:35:24.397 INFO:tasks.workunit.client.1.smithi121.stdout:2/221: mkdir d5/df/d3f 0 2024-02-15T18:35:24.397 INFO:tasks.workunit.client.1.smithi121.stdout:2/222: fdatasync d5/fc 0 2024-02-15T18:35:24.402 INFO:tasks.workunit.client.1.smithi121.stdout:1/323: symlink d0/l6a 0 2024-02-15T18:35:24.403 INFO:tasks.workunit.client.1.smithi121.stdout:2/223: creat d5/df/d27/d3e/f40 x:0 0 0 2024-02-15T18:35:24.404 INFO:tasks.workunit.client.1.smithi121.stdout:2/224: stat d5/df/d10/l1d 0 2024-02-15T18:35:24.405 INFO:tasks.workunit.client.1.smithi121.stdout:0/368: link d5/d6/dd/c5c d5/d6/d2a/c7b 0 2024-02-15T18:35:24.406 INFO:tasks.workunit.client.1.smithi121.stdout:1/324: creat d0/d6/d7/d9/d14/f6b x:0 0 0 2024-02-15T18:35:24.407 INFO:tasks.workunit.client.1.smithi121.stdout:0/369: truncate d5/d6/f3a 1232206 0 2024-02-15T18:35:24.407 INFO:tasks.workunit.client.1.smithi121.stdout:0/370: chown d5/d6/d2a/d36/l57 12 1 2024-02-15T18:35:24.408 INFO:tasks.workunit.client.1.smithi121.stdout:1/325: creat d0/d6/d7/d9/d51/f6c x:0 0 0 2024-02-15T18:35:24.409 INFO:tasks.workunit.client.1.smithi121.stdout:0/371: mknod d5/d6/d2a/d36/c7c 0 2024-02-15T18:35:24.409 INFO:tasks.workunit.client.1.smithi121.stdout:1/326: write d0/d39/d4b/f61 [309457,19346] 0 2024-02-15T18:35:24.409 INFO:tasks.workunit.client.1.smithi121.stdout:1/327: fsync d0/d6/d7/f4c 0 2024-02-15T18:35:24.410 INFO:tasks.workunit.client.1.smithi121.stdout:1/328: write d0/d6/d7/f5e [245525,124760] 0 2024-02-15T18:35:24.411 INFO:tasks.workunit.client.1.smithi121.stdout:7/233: dwrite d0/d5/d11/d1c/f3e [0,4194304] 0 2024-02-15T18:35:24.420 INFO:tasks.workunit.client.1.smithi121.stdout:1/329: dread d0/f47 [0,4194304] 0 2024-02-15T18:35:24.423 INFO:tasks.workunit.client.1.smithi121.stdout:1/330: read d0/d6/d7/f4c [204363,20130] 0 2024-02-15T18:35:24.424 INFO:tasks.workunit.client.1.smithi121.stdout:1/331: link d0/d6/d7/d9/d14/d48/f60 d0/d39/d4b/f6d 0 2024-02-15T18:35:24.433 INFO:tasks.workunit.client.1.smithi121.stdout:2/225: dwrite d5/df/d10/d15/f2d [0,4194304] 0 2024-02-15T18:35:24.434 INFO:tasks.workunit.client.1.smithi121.stdout:7/234: dwrite d0/d5/f15 [0,4194304] 0 2024-02-15T18:35:24.434 INFO:tasks.workunit.client.1.smithi121.stdout:7/235: write d0/d5/d11/f48 [892764,11373] 0 2024-02-15T18:35:24.435 INFO:tasks.workunit.client.1.smithi121.stdout:2/226: write d5/f8 [1465493,64545] 0 2024-02-15T18:35:24.437 INFO:tasks.workunit.client.1.smithi121.stdout:0/372: rename d5/d6 to d5/d30/d3c/d7d 0 2024-02-15T18:35:24.438 INFO:tasks.workunit.client.1.smithi121.stdout:0/373: write d5/d30/d3c/d7d/dd/f2c [1836708,123362] 0 2024-02-15T18:35:24.438 INFO:tasks.workunit.client.1.smithi121.stdout:0/374: chown d5/d30/d3c/d7d/d2a/d36/d44/d71 10 1 2024-02-15T18:35:24.469 INFO:tasks.workunit.client.1.smithi121.stdout:7/236: creat d0/d5/d11/f51 x:0 0 0 2024-02-15T18:35:24.473 INFO:tasks.workunit.client.1.smithi121.stdout:1/332: rename d0/f47 to d0/d6/f6e 0 2024-02-15T18:35:24.473 INFO:tasks.workunit.client.1.smithi121.stdout:1/333: readlink d0/d6/d7/l37 0 2024-02-15T18:35:24.475 INFO:tasks.workunit.client.1.smithi121.stdout:0/375: creat d5/d30/d3c/d7d/dd/f7e x:0 0 0 2024-02-15T18:35:24.481 INFO:tasks.workunit.client.1.smithi121.stdout:7/237: creat d0/f52 x:0 0 0 2024-02-15T18:35:24.483 INFO:tasks.workunit.client.1.smithi121.stdout:1/334: getdents d0/d39 0 2024-02-15T18:35:24.484 INFO:tasks.workunit.client.1.smithi121.stdout:8/369: sync 2024-02-15T18:35:24.484 INFO:tasks.workunit.client.1.smithi121.stdout:8/370: truncate d0/d7/d36/f81 613361 0 2024-02-15T18:35:24.485 INFO:tasks.workunit.client.1.smithi121.stdout:0/376: creat d5/d30/d3c/d7d/dd/f7f x:0 0 0 2024-02-15T18:35:24.485 INFO:tasks.workunit.client.1.smithi121.stdout:0/377: stat d5/f3b 0 2024-02-15T18:35:24.485 INFO:tasks.workunit.client.1.smithi121.stdout:0/378: write d5/d30/d3c/d7d/dd/f2d [106858,34329] 0 2024-02-15T18:35:24.490 INFO:tasks.workunit.client.1.smithi121.stdout:1/335: mknod d0/d6/d7/d50/c6f 0 2024-02-15T18:35:24.490 INFO:tasks.workunit.client.1.smithi121.stdout:1/336: chown d0/d6/d7/d29/l34 270593 1 2024-02-15T18:35:24.490 INFO:tasks.workunit.client.1.smithi121.stdout:1/337: read - d0/d6/d7/d4e/f5b zero size 2024-02-15T18:35:24.492 INFO:tasks.workunit.client.1.smithi121.stdout:2/227: rmdir d5/df/d10/d15/d18 0 2024-02-15T18:35:24.495 INFO:tasks.workunit.client.1.smithi121.stdout:8/371: rename d0/d7/d3f to d0/d7/d36/d8c 0 2024-02-15T18:35:24.497 INFO:tasks.workunit.client.1.smithi121.stdout:2/228: symlink d5/df/d27/l41 0 2024-02-15T18:35:24.497 INFO:tasks.workunit.client.1.smithi121.stdout:2/229: write d5/df/f2a [809565,95163] 0 2024-02-15T18:35:24.498 INFO:tasks.workunit.client.1.smithi121.stdout:8/372: mknod d0/d7/de/c8d 0 2024-02-15T18:35:24.500 INFO:tasks.workunit.client.1.smithi121.stdout:0/379: truncate d5/d30/d3c/d7d/f1f 1191302 0 2024-02-15T18:35:24.501 INFO:tasks.workunit.client.1.smithi121.stdout:2/230: creat d5/df/d10/d15/d22/f42 x:0 0 0 2024-02-15T18:35:24.503 INFO:tasks.workunit.client.1.smithi121.stdout:8/373: creat d0/d7/de/f8e x:0 0 0 2024-02-15T18:35:24.504 INFO:tasks.workunit.client.1.smithi121.stdout:0/380: mknod d5/d30/d3c/d7d/c80 0 2024-02-15T18:35:24.506 INFO:tasks.workunit.client.1.smithi121.stdout:2/231: unlink d5/df/d27/c1f 0 2024-02-15T18:35:24.506 INFO:tasks.workunit.client.1.smithi121.stdout:2/232: chown d5/df/d10/d15/d22/f37 621 1 2024-02-15T18:35:24.508 INFO:tasks.workunit.client.1.smithi121.stdout:8/374: mkdir d0/d61/d8f 0 2024-02-15T18:35:24.508 INFO:tasks.workunit.client.1.smithi121.stdout:8/375: fsync d0/d7/d36/d8c/d45/f2c 0 2024-02-15T18:35:24.509 INFO:tasks.workunit.client.1.smithi121.stdout:7/238: dwrite d0/d5/d11/d1c/f44 [0,4194304] 0 2024-02-15T18:35:24.510 INFO:tasks.workunit.client.1.smithi121.stdout:2/233: creat d5/df/d27/d3a/f43 x:0 0 0 2024-02-15T18:35:24.513 INFO:tasks.workunit.client.1.smithi121.stdout:1/338: dwrite d0/d39/f68 [0,4194304] 0 2024-02-15T18:35:24.513 INFO:tasks.workunit.client.1.smithi121.stdout:1/339: chown d0/d39/d4b/l67 211008792 1 2024-02-15T18:35:24.513 INFO:tasks.workunit.client.1.smithi121.stdout:7/239: creat d0/d4a/f53 x:0 0 0 2024-02-15T18:35:24.515 INFO:tasks.workunit.client.1.smithi121.stdout:2/234: read d5/df/d10/d15/d22/f37 [2427906,84118] 0 2024-02-15T18:35:24.515 INFO:tasks.workunit.client.1.smithi121.stdout:2/235: chown d5/df/d10/d15/c1e 71 1 2024-02-15T18:35:24.515 INFO:tasks.workunit.client.1.smithi121.stdout:0/381: dread d5/d30/d3c/d7d/dd/f16 [0,4194304] 0 2024-02-15T18:35:24.516 INFO:tasks.workunit.client.1.smithi121.stdout:0/382: dread - d5/d30/d3c/d7d/dd/f4b zero size 2024-02-15T18:35:24.520 INFO:tasks.workunit.client.1.smithi121.stdout:5/264: sync 2024-02-15T18:35:24.520 INFO:tasks.workunit.client.1.smithi121.stdout:4/164: sync 2024-02-15T18:35:24.520 INFO:tasks.workunit.client.1.smithi121.stdout:9/266: sync 2024-02-15T18:35:24.520 INFO:tasks.workunit.client.1.smithi121.stdout:6/317: sync 2024-02-15T18:35:24.520 INFO:tasks.workunit.client.1.smithi121.stdout:7/240: dread d0/d33/f34 [0,4194304] 0 2024-02-15T18:35:24.520 INFO:tasks.workunit.client.1.smithi121.stdout:3/318: sync 2024-02-15T18:35:24.520 INFO:tasks.workunit.client.1.smithi121.stdout:3/319: chown d2/f7 2201435 1 2024-02-15T18:35:24.523 INFO:tasks.workunit.client.1.smithi121.stdout:4/165: rename d5/d7/d1a to d5/d31 0 2024-02-15T18:35:24.526 INFO:tasks.workunit.client.1.smithi121.stdout:3/320: mknod d2/d1a/d22/c6e 0 2024-02-15T18:35:24.531 INFO:tasks.workunit.client.1.smithi121.stdout:0/383: dread d5/d30/d3c/d7d/d2a/d36/f54 [0,4194304] 0 2024-02-15T18:35:24.531 INFO:tasks.workunit.client.1.smithi121.stdout:0/384: fsync d5/f40 0 2024-02-15T18:35:24.531 INFO:tasks.workunit.client.1.smithi121.stdout:0/385: read - d5/d30/d3c/d7d/f43 zero size 2024-02-15T18:35:24.531 INFO:tasks.workunit.client.1.smithi121.stdout:0/386: stat f2 0 2024-02-15T18:35:24.531 INFO:tasks.workunit.client.1.smithi121.stdout:5/265: rename d0/d16/d37 to d0/d2/d1e/d21/d47/d5a 0 2024-02-15T18:35:24.531 INFO:tasks.workunit.client.1.smithi121.stdout:3/321: truncate d2/f36 4425461 0 2024-02-15T18:35:24.536 INFO:tasks.workunit.client.1.smithi121.stdout:0/387: mkdir d5/d30/d3c/d7d/d2a/d73/d81 0 2024-02-15T18:35:24.536 INFO:tasks.workunit.client.1.smithi121.stdout:0/388: stat d5/d30/d3c/d7d/d2a/d36/d44/l6c 0 2024-02-15T18:35:24.536 INFO:tasks.workunit.client.1.smithi121.stdout:5/266: creat d0/d2/d18/f5b x:0 0 0 2024-02-15T18:35:24.542 INFO:tasks.workunit.client.1.smithi121.stdout:5/267: rmdir d0/d2 39 2024-02-15T18:35:24.542 INFO:tasks.workunit.client.1.smithi121.stdout:5/268: write d0/d2/d18/f27 [1141216,45780] 0 2024-02-15T18:35:24.542 INFO:tasks.workunit.client.1.smithi121.stdout:5/269: chown d0/d16/d34/f3a 2602 1 2024-02-15T18:35:24.542 INFO:tasks.workunit.client.1.smithi121.stdout:3/322: creat d2/d1a/d27/d2c/d25/d31/d58/f6f x:0 0 0 2024-02-15T18:35:24.545 INFO:tasks.workunit.client.1.smithi121.stdout:3/323: dread d2/d1a/d27/d2c/f19 [0,4194304] 0 2024-02-15T18:35:24.545 INFO:tasks.workunit.client.1.smithi121.stdout:3/324: fdatasync d2/d1a/d27/d2c/f56 0 2024-02-15T18:35:24.546 INFO:tasks.workunit.client.1.smithi121.stdout:8/376: dwrite d0/d3/f64 [0,4194304] 0 2024-02-15T18:35:24.547 INFO:tasks.workunit.client.1.smithi121.stdout:3/325: read d2/d5a/f5f [297359,77500] 0 2024-02-15T18:35:24.547 INFO:tasks.workunit.client.1.smithi121.stdout:5/270: creat d0/d2/d1e/d21/d47/d5a/f5c x:0 0 0 2024-02-15T18:35:24.550 INFO:tasks.workunit.client.1.smithi121.stdout:8/377: creat d0/d7/d36/d52/d7a/f90 x:0 0 0 2024-02-15T18:35:24.551 INFO:tasks.workunit.client.1.smithi121.stdout:8/378: chown d0/d7/de 4402914 1 2024-02-15T18:35:24.553 INFO:tasks.workunit.client.1.smithi121.stdout:3/326: unlink d2/d1a/d27/f29 0 2024-02-15T18:35:24.553 INFO:tasks.workunit.client.1.smithi121.stdout:3/327: dread - d2/d1a/d54/f59 zero size 2024-02-15T18:35:24.553 INFO:tasks.workunit.client.1.smithi121.stdout:3/328: stat d2/d1a/d54/f69 0 2024-02-15T18:35:24.554 INFO:tasks.workunit.client.1.smithi121.stdout:8/379: dread d0/d2a/f7d [0,4194304] 0 2024-02-15T18:35:24.555 INFO:tasks.workunit.client.1.smithi121.stdout:3/329: mkdir d2/d1a/d27/d70 0 2024-02-15T18:35:24.567 INFO:tasks.workunit.client.1.smithi121.stdout:1/340: dwrite d0/d6/d7/f1b [0,4194304] 0 2024-02-15T18:35:24.574 INFO:tasks.workunit.client.1.smithi121.stdout:1/341: creat d0/d6/d7/d9/d14/d48/f70 x:0 0 0 2024-02-15T18:35:24.576 INFO:tasks.workunit.client.1.smithi121.stdout:0/389: dwrite d5/d30/d3c/d7d/dd/f2c [0,4194304] 0 2024-02-15T18:35:24.576 INFO:tasks.workunit.client.1.smithi121.stdout:0/390: fsync f2 0 2024-02-15T18:35:24.576 INFO:tasks.workunit.client.1.smithi121.stdout:0/391: fsync d5/d30/d3c/d7d/d2a/d36/f54 0 2024-02-15T18:35:24.578 INFO:tasks.workunit.client.1.smithi121.stdout:1/342: rename d0/d6/f19 to d0/d6/d7/d9/d14/d3e/f71 0 2024-02-15T18:35:24.579 INFO:tasks.workunit.client.1.smithi121.stdout:0/392: truncate d5/d30/d3c/d7d/d2a/d4e/f5b 3651063 0 2024-02-15T18:35:24.579 INFO:tasks.workunit.client.1.smithi121.stdout:0/393: write d5/d30/d3c/d7d/dd/f4b [52043,110387] 0 2024-02-15T18:35:24.582 INFO:tasks.workunit.client.1.smithi121.stdout:0/394: creat d5/d30/d3c/d7d/d74/f82 x:0 0 0 2024-02-15T18:35:24.582 INFO:tasks.workunit.client.1.smithi121.stdout:4/166: dwrite d5/d9/d19/d2a/f2c [0,4194304] 0 2024-02-15T18:35:24.582 INFO:tasks.workunit.client.1.smithi121.stdout:2/236: dwrite f4 [0,4194304] 0 2024-02-15T18:35:24.582 INFO:tasks.workunit.client.1.smithi121.stdout:2/237: truncate d5/df/f2b 300982 0 2024-02-15T18:35:24.584 INFO:tasks.workunit.client.1.smithi121.stdout:6/318: dwrite d7/dc/d2c/f5c [0,4194304] 0 2024-02-15T18:35:24.585 INFO:tasks.workunit.client.1.smithi121.stdout:3/330: dwrite d2/f40 [4194304,4194304] 0 2024-02-15T18:35:24.587 INFO:tasks.workunit.client.1.smithi121.stdout:2/238: getdents d5/df/d10 0 2024-02-15T18:35:24.589 INFO:tasks.workunit.client.1.smithi121.stdout:6/319: mkdir d7/dc/d34/d42/d62/d63/d6f 0 2024-02-15T18:35:24.590 INFO:tasks.workunit.client.1.smithi121.stdout:8/380: dwrite d0/d7/d36/d52/f5b [0,4194304] 0 2024-02-15T18:35:24.599 INFO:tasks.workunit.client.1.smithi121.stdout:5/271: getdents d0/d2/d1e/d21/d47/d5a 0 2024-02-15T18:35:24.602 INFO:tasks.workunit.client.1.smithi121.stdout:3/331: link d2/d1a/d27/f2d d2/d1a/d27/d2c/d25/d31/d34/d4a/f71 0 2024-02-15T18:35:24.605 INFO:tasks.workunit.client.1.smithi121.stdout:1/343: dwrite d0/d6/f6e [0,4194304] 0 2024-02-15T18:35:24.609 INFO:tasks.workunit.client.1.smithi121.stdout:4/167: dwrite d5/d9/f20 [0,4194304] 0 2024-02-15T18:35:24.611 INFO:tasks.workunit.client.1.smithi121.stdout:0/395: dwrite d5/d30/f33 [0,4194304] 0 2024-02-15T18:35:24.612 INFO:tasks.workunit.client.1.smithi121.stdout:4/168: dread d5/d9/d1d/f24 [0,4194304] 0 2024-02-15T18:35:24.612 INFO:tasks.workunit.client.1.smithi121.stdout:6/320: truncate d7/f18 450505 0 2024-02-15T18:35:24.616 INFO:tasks.workunit.client.1.smithi121.stdout:2/239: dwrite d5/f8 [4194304,4194304] 0 2024-02-15T18:35:24.618 INFO:tasks.workunit.client.1.smithi121.stdout:2/240: write d5/f8 [7230371,128441] 0 2024-02-15T18:35:24.618 INFO:tasks.workunit.client.1.smithi121.stdout:2/241: fsync d5/df/d27/d3e/f40 0 2024-02-15T18:35:24.621 INFO:tasks.workunit.client.1.smithi121.stdout:8/381: link d0/d7/de/f8e d0/d7/de/d7c/f91 0 2024-02-15T18:35:24.621 INFO:tasks.workunit.client.1.smithi121.stdout:8/382: chown d0/d61/d66/f76 1503394428 1 2024-02-15T18:35:24.624 INFO:tasks.workunit.client.1.smithi121.stdout:5/272: mknod d0/d2/d1e/d21/c5d 0 2024-02-15T18:35:24.625 INFO:tasks.workunit.client.1.smithi121.stdout:3/332: mkdir d2/d1a/d6b/d72 0 2024-02-15T18:35:24.626 INFO:tasks.workunit.client.1.smithi121.stdout:1/344: mkdir d0/d6/d7/d72 0 2024-02-15T18:35:24.627 INFO:tasks.workunit.client.1.smithi121.stdout:3/333: write d2/f9 [4005626,107277] 0 2024-02-15T18:35:24.628 INFO:tasks.workunit.client.1.smithi121.stdout:0/396: dwrite d5/d30/d3c/d7d/d2a/f2f [4194304,4194304] 0 2024-02-15T18:35:24.629 INFO:tasks.workunit.client.1.smithi121.stdout:4/169: getdents d5/d31 0 2024-02-15T18:35:24.631 INFO:tasks.workunit.client.1.smithi121.stdout:6/321: rename d7/ca to d7/dc/d34/d42/d62/d63/d66/c70 0 2024-02-15T18:35:24.635 INFO:tasks.workunit.client.1.smithi121.stdout:8/383: creat d0/d61/d8f/f92 x:0 0 0 2024-02-15T18:35:24.637 INFO:tasks.workunit.client.1.smithi121.stdout:5/273: creat d0/d2/d1e/d21/d33/d4c/f5e x:0 0 0 2024-02-15T18:35:24.637 INFO:tasks.workunit.client.1.smithi121.stdout:5/274: chown d0/d16/d34/f3a 54982 1 2024-02-15T18:35:24.644 INFO:tasks.workunit.client.1.smithi121.stdout:1/345: link d0/d6/d7/d4e/f5b d0/f73 0 2024-02-15T18:35:24.646 INFO:tasks.workunit.client.1.smithi121.stdout:0/397: truncate d5/f3b 2874497 0 2024-02-15T18:35:24.646 INFO:tasks.workunit.client.1.smithi121.stdout:0/398: dread - d5/d30/d3c/d7d/d2a/d36/f51 zero size 2024-02-15T18:35:24.647 INFO:tasks.workunit.client.1.smithi121.stdout:0/399: read d5/d30/d5d/f5e [425290,122822] 0 2024-02-15T18:35:24.650 INFO:tasks.workunit.client.1.smithi121.stdout:4/170: creat d5/d9/d19/d2a/f32 x:0 0 0 2024-02-15T18:35:24.652 INFO:tasks.workunit.client.1.smithi121.stdout:5/275: dwrite d0/d2/d18/f27 [0,4194304] 0 2024-02-15T18:35:24.652 INFO:tasks.workunit.client.1.smithi121.stdout:6/322: rename d7/dc/d55/f5f to d7/dc/d34/f71 0 2024-02-15T18:35:24.655 INFO:tasks.workunit.client.1.smithi121.stdout:8/384: rename d0/d4a/l55 to d0/d4a/l93 0 2024-02-15T18:35:24.657 INFO:tasks.workunit.client.1.smithi121.stdout:1/346: symlink d0/d6/d7/d9/d14/d3e/l74 0 2024-02-15T18:35:24.661 INFO:tasks.workunit.client.1.smithi121.stdout:4/171: link d5/f10 d5/d31/d25/f33 0 2024-02-15T18:35:24.661 INFO:tasks.workunit.client.1.smithi121.stdout:4/172: chown f1 15681544 1 2024-02-15T18:35:24.664 INFO:tasks.workunit.client.1.smithi121.stdout:8/385: dread d0/f15 [0,4194304] 0 2024-02-15T18:35:24.665 INFO:tasks.workunit.client.1.smithi121.stdout:5/276: creat d0/f5f x:0 0 0 2024-02-15T18:35:24.665 INFO:tasks.workunit.client.1.smithi121.stdout:5/277: read - d0/f5f zero size 2024-02-15T18:35:24.666 INFO:tasks.workunit.client.1.smithi121.stdout:6/323: rmdir d7/d4b/d60 39 2024-02-15T18:35:24.666 INFO:tasks.workunit.client.1.smithi121.stdout:6/324: truncate d7/f5d 205504 0 2024-02-15T18:35:24.666 INFO:tasks.workunit.client.1.smithi121.stdout:6/325: write d7/dc/f15 [9324437,111339] 0 2024-02-15T18:35:24.666 INFO:tasks.workunit.client.1.smithi121.stdout:6/326: write d7/dc/f21 [1251266,70028] 0 2024-02-15T18:35:24.666 INFO:tasks.workunit.client.1.smithi121.stdout:6/327: readlink d7/l2b 0 2024-02-15T18:35:24.667 INFO:tasks.workunit.client.1.smithi121.stdout:1/347: mknod d0/c75 0 2024-02-15T18:35:24.667 INFO:tasks.workunit.client.1.smithi121.stdout:1/348: chown d0/f53 6646 1 2024-02-15T18:35:24.668 INFO:tasks.workunit.client.1.smithi121.stdout:1/349: read d0/d6/d7/d29/f41 [395129,99968] 0 2024-02-15T18:35:24.668 INFO:tasks.workunit.client.1.smithi121.stdout:0/400: symlink d5/d30/l83 0 2024-02-15T18:35:24.669 INFO:tasks.workunit.client.1.smithi121.stdout:4/173: mknod d5/d31/c34 0 2024-02-15T18:35:24.669 INFO:tasks.workunit.client.1.smithi121.stdout:8/386: truncate d0/d3/f64 3639922 0 2024-02-15T18:35:24.671 INFO:tasks.workunit.client.1.smithi121.stdout:8/387: write d0/d7/d36/d52/f5b [1449707,115909] 0 2024-02-15T18:35:24.671 INFO:tasks.workunit.client.1.smithi121.stdout:8/388: chown d0/d2a/d51/c71 38599 1 2024-02-15T18:35:24.682 INFO:tasks.workunit.client.1.smithi121.stdout:5/278: creat d0/f60 x:0 0 0 2024-02-15T18:35:24.684 INFO:tasks.workunit.client.1.smithi121.stdout:1/350: dwrite d0/d39/f5c [0,4194304] 0 2024-02-15T18:35:24.684 INFO:tasks.workunit.client.1.smithi121.stdout:6/328: creat d7/d4b/d59/f72 x:0 0 0 2024-02-15T18:35:24.684 INFO:tasks.workunit.client.1.smithi121.stdout:0/401: symlink d5/d30/d3c/d7d/d2a/d73/l84 0 2024-02-15T18:35:24.688 INFO:tasks.workunit.client.1.smithi121.stdout:4/174: mknod d5/d7/d1e/c35 0 2024-02-15T18:35:24.690 INFO:tasks.workunit.client.1.smithi121.stdout:8/389: truncate d0/f13 3351592 0 2024-02-15T18:35:24.690 INFO:tasks.workunit.client.1.smithi121.stdout:8/390: truncate d0/d7/d36/d8c/f49 775948 0 2024-02-15T18:35:24.690 INFO:tasks.workunit.client.1.smithi121.stdout:8/391: fdatasync d0/d3/d20/f78 0 2024-02-15T18:35:24.693 INFO:tasks.workunit.client.1.smithi121.stdout:5/279: write d0/fb [1709460,9202] 0 2024-02-15T18:35:24.694 INFO:tasks.workunit.client.1.smithi121.stdout:1/351: link d0/d6/d7/d9/d14/d48/f70 d0/d6/d7/d4e/f76 0 2024-02-15T18:35:24.695 INFO:tasks.workunit.client.1.smithi121.stdout:6/329: mkdir d7/d4b/d60/d73 0 2024-02-15T18:35:24.696 INFO:tasks.workunit.client.1.smithi121.stdout:4/175: dread d5/d9/d1d/f22 [0,4194304] 0 2024-02-15T18:35:24.696 INFO:tasks.workunit.client.1.smithi121.stdout:4/176: chown d5/d9/d1d/f22 118115845 1 2024-02-15T18:35:24.700 INFO:tasks.workunit.client.1.smithi121.stdout:0/402: rename d5/d30/d3c/d7d/f1f to d5/d30/d3c/d7d/d2a/d36/d44/d71/f85 0 2024-02-15T18:35:24.703 INFO:tasks.workunit.client.1.smithi121.stdout:8/392: link d0/d7/d36/d8c/d45/f89 d0/d7/d36/d8c/d45/f94 0 2024-02-15T18:35:24.704 INFO:tasks.workunit.client.1.smithi121.stdout:5/280: getdents d0/d2/d1e 0 2024-02-15T18:35:24.708 INFO:tasks.workunit.client.1.smithi121.stdout:1/352: getdents d0/d6/d7/d4e 0 2024-02-15T18:35:24.710 INFO:tasks.workunit.client.1.smithi121.stdout:6/330: creat d7/dc/d34/d42/d62/f74 x:0 0 0 2024-02-15T18:35:24.710 INFO:tasks.workunit.client.1.smithi121.stdout:1/353: read d0/d6/d7/d29/f41 [1575898,58196] 0 2024-02-15T18:35:24.711 INFO:tasks.workunit.client.1.smithi121.stdout:4/177: symlink d5/d9/d19/l36 0 2024-02-15T18:35:24.713 INFO:tasks.workunit.client.1.smithi121.stdout:8/393: getdents d0/d48 0 2024-02-15T18:35:24.717 INFO:tasks.workunit.client.1.smithi121.stdout:4/178: truncate d5/d9/d1d/f24 570766 0 2024-02-15T18:35:24.721 INFO:tasks.workunit.client.1.smithi121.stdout:4/179: rename d5/d9/d1d/f22 to d5/d31/d25/f37 0 2024-02-15T18:35:24.722 INFO:tasks.workunit.client.1.smithi121.stdout:4/180: creat d5/d7/f38 x:0 0 0 2024-02-15T18:35:24.724 INFO:tasks.workunit.client.1.smithi121.stdout:4/181: creat d5/d31/d25/f39 x:0 0 0 2024-02-15T18:35:24.726 INFO:tasks.workunit.client.1.smithi121.stdout:4/182: unlink d5/d9/d19/d2a/l2d 0 2024-02-15T18:35:24.726 INFO:tasks.workunit.client.1.smithi121.stdout:4/183: write d5/d31/f23 [220928,86461] 0 2024-02-15T18:35:24.728 INFO:tasks.workunit.client.1.smithi121.stdout:4/184: rename d5/d7/l16 to d5/l3a 0 2024-02-15T18:35:24.728 INFO:tasks.workunit.client.1.smithi121.stdout:0/403: dwrite f2 [4194304,4194304] 0 2024-02-15T18:35:24.729 INFO:tasks.workunit.client.1.smithi121.stdout:4/185: rmdir d5/d9/d19 39 2024-02-15T18:35:24.729 INFO:tasks.workunit.client.1.smithi121.stdout:6/331: fdatasync d7/dc/f15 0 2024-02-15T18:35:24.732 INFO:tasks.workunit.client.1.smithi121.stdout:0/404: link d5/d30/d3c/d7d/cc d5/d30/d3c/d7d/d74/c86 0 2024-02-15T18:35:24.732 INFO:tasks.workunit.client.1.smithi121.stdout:0/405: dread - d5/d30/d3c/d7d/d2a/d36/f51 zero size 2024-02-15T18:35:24.732 INFO:tasks.workunit.client.1.smithi121.stdout:0/406: write d5/d30/d3c/d7d/dd/f68 [98421,113933] 0 2024-02-15T18:35:24.735 INFO:tasks.workunit.client.1.smithi121.stdout:4/186: rename c4 to d5/d9/d1d/c3b 0 2024-02-15T18:35:24.740 INFO:tasks.workunit.client.1.smithi121.stdout:5/281: dwrite d0/d2/d1e/d21/d39/f51 [0,4194304] 0 2024-02-15T18:35:24.741 INFO:tasks.workunit.client.1.smithi121.stdout:8/394: dwrite d0/d7/de/f1a [4194304,4194304] 0 2024-02-15T18:35:24.741 INFO:tasks.workunit.client.1.smithi121.stdout:1/354: dwrite d0/d6/d7/f5e [0,4194304] 0 2024-02-15T18:35:24.742 INFO:tasks.workunit.client.1.smithi121.stdout:6/332: rmdir d7 39 2024-02-15T18:35:24.743 INFO:tasks.workunit.client.1.smithi121.stdout:0/407: symlink d5/d30/d3c/d7d/d2a/d36/d44/d71/l87 0 2024-02-15T18:35:24.745 INFO:tasks.workunit.client.1.smithi121.stdout:0/408: dread d5/d30/d3c/d7d/dd/f17 [0,4194304] 0 2024-02-15T18:35:24.750 INFO:tasks.workunit.client.1.smithi121.stdout:6/333: dread d7/dc/d2c/d36/f38 [0,4194304] 0 2024-02-15T18:35:24.751 INFO:tasks.workunit.client.1.smithi121.stdout:1/355: dread d0/d39/f5c [0,4194304] 0 2024-02-15T18:35:24.754 INFO:tasks.workunit.client.1.smithi121.stdout:4/187: rmdir d5/d9 39 2024-02-15T18:35:24.755 INFO:tasks.workunit.client.1.smithi121.stdout:4/188: truncate d5/d9/d1d/f2f 466649 0 2024-02-15T18:35:24.755 INFO:tasks.workunit.client.1.smithi121.stdout:4/189: truncate d5/d7/d14/f17 1474575 0 2024-02-15T18:35:24.757 INFO:tasks.workunit.client.1.smithi121.stdout:5/282: rename d0/d16/f58 to d0/d2/d1e/d4e/f61 0 2024-02-15T18:35:24.758 INFO:tasks.workunit.client.1.smithi121.stdout:0/409: rename d5/d30/d5d to d5/d30/d5d/d88 22 2024-02-15T18:35:24.758 INFO:tasks.workunit.client.1.smithi121.stdout:5/283: read d0/f22 [764035,71482] 0 2024-02-15T18:35:24.760 INFO:tasks.workunit.client.1.smithi121.stdout:6/334: dread d7/d4b/f53 [0,4194304] 0 2024-02-15T18:35:24.761 INFO:tasks.workunit.client.1.smithi121.stdout:6/335: chown d7/d4b/d60/d73 37 1 2024-02-15T18:35:24.764 INFO:tasks.workunit.client.1.smithi121.stdout:8/395: unlink d0/d4a/l93 0 2024-02-15T18:35:24.764 INFO:tasks.workunit.client.1.smithi121.stdout:8/396: stat d0/l14 0 2024-02-15T18:35:24.765 INFO:tasks.workunit.client.1.smithi121.stdout:4/190: creat d5/d7/f3c x:0 0 0 2024-02-15T18:35:24.765 INFO:tasks.workunit.client.1.smithi121.stdout:4/191: fdatasync d5/d7/fb 0 2024-02-15T18:35:24.766 INFO:tasks.workunit.client.1.smithi121.stdout:4/192: truncate d5/d31/d25/f39 816749 0 2024-02-15T18:35:24.766 INFO:tasks.workunit.client.1.smithi121.stdout:4/193: fdatasync f1 0 2024-02-15T18:35:24.766 INFO:tasks.workunit.client.1.smithi121.stdout:1/356: dwrite d0/d6/d7/d9/f20 [4194304,4194304] 0 2024-02-15T18:35:24.766 INFO:tasks.workunit.client.1.smithi121.stdout:1/357: truncate d0/d6/f12 2948785 0 2024-02-15T18:35:24.766 INFO:tasks.workunit.client.1.smithi121.stdout:1/358: readlink d0/d6/d7/d9/d14/l64 0 2024-02-15T18:35:24.768 INFO:tasks.workunit.client.1.smithi121.stdout:0/410: link d5/d30/d3c/d7d/d2a/d73/l84 d5/d30/d3c/d7d/d2a/d73/l89 0 2024-02-15T18:35:24.771 INFO:tasks.workunit.client.1.smithi121.stdout:5/284: mkdir d0/d2/d1e/d21/d40/d62 0 2024-02-15T18:35:24.772 INFO:tasks.workunit.client.1.smithi121.stdout:6/336: symlink d7/dc/d34/d42/d62/d63/d6f/l75 0 2024-02-15T18:35:24.772 INFO:tasks.workunit.client.1.smithi121.stdout:6/337: chown d7/dc/d34/d42/d62/d63 0 1 2024-02-15T18:35:24.774 INFO:tasks.workunit.client.1.smithi121.stdout:4/194: mkdir d5/d31/d3d 0 2024-02-15T18:35:24.775 INFO:tasks.workunit.client.1.smithi121.stdout:4/195: fdatasync d5/d7/f3c 0 2024-02-15T18:35:24.775 INFO:tasks.workunit.client.1.smithi121.stdout:4/196: chown d5 1 1 2024-02-15T18:35:24.775 INFO:tasks.workunit.client.1.smithi121.stdout:4/197: stat d5 0 2024-02-15T18:35:24.775 INFO:tasks.workunit.client.1.smithi121.stdout:4/198: fdatasync d5/d7/d14/f29 0 2024-02-15T18:35:24.775 INFO:tasks.workunit.client.1.smithi121.stdout:4/199: getdents d5/d9/d19/d28 0 2024-02-15T18:35:24.777 INFO:tasks.workunit.client.1.smithi121.stdout:1/359: mkdir d0/d6/d7/d29/d77 0 2024-02-15T18:35:24.778 INFO:tasks.workunit.client.1.smithi121.stdout:0/411: creat d5/d30/d3c/d7d/d2a/f8a x:0 0 0 2024-02-15T18:35:24.782 INFO:tasks.workunit.client.1.smithi121.stdout:5/285: symlink d0/d2/d1e/l63 0 2024-02-15T18:35:24.782 INFO:tasks.workunit.client.1.smithi121.stdout:5/286: read - d0/d2/d1e/d21/d33/d4c/f5e zero size 2024-02-15T18:35:24.782 INFO:tasks.workunit.client.1.smithi121.stdout:6/338: mkdir d7/dc/d76 0 2024-02-15T18:35:24.784 INFO:tasks.workunit.client.1.smithi121.stdout:4/200: mkdir d5/d31/d3e 0 2024-02-15T18:35:24.785 INFO:tasks.workunit.client.1.smithi121.stdout:0/412: creat d5/d30/d3c/d7d/d2a/d36/f8b x:0 0 0 2024-02-15T18:35:24.788 INFO:tasks.workunit.client.1.smithi121.stdout:8/397: dwrite d0/d2a/d51/f6b [0,4194304] 0 2024-02-15T18:35:24.788 INFO:tasks.workunit.client.1.smithi121.stdout:8/398: fdatasync d0/d7/d36/d52/d7a/f8a 0 2024-02-15T18:35:24.789 INFO:tasks.workunit.client.1.smithi121.stdout:6/339: link d7/dc/d2c/d36/f56 d7/dc/f77 0 2024-02-15T18:35:24.789 INFO:tasks.workunit.client.1.smithi121.stdout:6/340: fdatasync d7/dc/d2c/d36/f56 0 2024-02-15T18:35:24.790 INFO:tasks.workunit.client.1.smithi121.stdout:6/341: write d7/dc/d2c/d36/f49 [2242524,36795] 0 2024-02-15T18:35:24.806 INFO:tasks.workunit.client.1.smithi121.stdout:1/360: dwrite d0/d6/d7/d9/d14/d48/f60 [0,4194304] 0 2024-02-15T18:35:24.808 INFO:tasks.workunit.client.1.smithi121.stdout:5/287: dwrite d0/d16/d34/f3a [0,4194304] 0 2024-02-15T18:35:24.810 INFO:tasks.workunit.client.1.smithi121.stdout:4/201: dwrite d5/d7/f3c [0,4194304] 0 2024-02-15T18:35:24.815 INFO:tasks.workunit.client.1.smithi121.stdout:5/288: unlink d0/d2/l3f 0 2024-02-15T18:35:24.816 INFO:tasks.workunit.client.1.smithi121.stdout:8/399: dwrite d0/d61/d66/f80 [0,4194304] 0 2024-02-15T18:35:24.818 INFO:tasks.workunit.client.1.smithi121.stdout:4/202: write d5/d9/f20 [2765909,95199] 0 2024-02-15T18:35:24.818 INFO:tasks.workunit.client.1.smithi121.stdout:4/203: truncate d5/d7/f38 458754 0 2024-02-15T18:35:24.823 INFO:tasks.workunit.client.1.smithi121.stdout:5/289: dread d0/d2/d18/f31 [0,4194304] 0 2024-02-15T18:35:24.824 INFO:tasks.workunit.client.1.smithi121.stdout:5/290: chown d0/d2/d1e/d21/d33/d4c 67 1 2024-02-15T18:35:24.828 INFO:tasks.workunit.client.1.smithi121.stdout:0/413: dwrite d5/d30/d3c/d7d/dd/f68 [0,4194304] 0 2024-02-15T18:35:24.828 INFO:tasks.workunit.client.1.smithi121.stdout:0/414: truncate d5/d30/d3c/d7d/d2a/d36/f3d 301743 0 2024-02-15T18:35:24.830 INFO:tasks.workunit.client.1.smithi121.stdout:1/361: dwrite d0/d6/d7/d9/d14/d3e/f71 [0,4194304] 0 2024-02-15T18:35:24.832 INFO:tasks.workunit.client.1.smithi121.stdout:5/291: dread d0/d2/d18/f20 [0,4194304] 0 2024-02-15T18:35:24.832 INFO:tasks.workunit.client.1.smithi121.stdout:5/292: read d0/d2/fd [1420824,42939] 0 2024-02-15T18:35:24.836 INFO:tasks.workunit.client.1.smithi121.stdout:8/400: mkdir d0/d61/d95 0 2024-02-15T18:35:24.836 INFO:tasks.workunit.client.1.smithi121.stdout:8/401: readlink d0/d2a/d51/l4e 0 2024-02-15T18:35:24.841 INFO:tasks.workunit.client.1.smithi121.stdout:4/204: link d5/d9/d19/l36 d5/d9/l3f 0 2024-02-15T18:35:24.841 INFO:tasks.workunit.client.1.smithi121.stdout:4/205: truncate d5/d7/f38 1288366 0 2024-02-15T18:35:24.841 INFO:tasks.workunit.client.1.smithi121.stdout:4/206: readlink d5/d7/l1c 0 2024-02-15T18:35:24.841 INFO:tasks.workunit.client.1.smithi121.stdout:4/207: fdatasync d5/f30 0 2024-02-15T18:35:24.841 INFO:tasks.workunit.client.1.smithi121.stdout:4/208: dread - d5/d9/d19/d2a/f32 zero size 2024-02-15T18:35:24.842 INFO:tasks.workunit.client.1.smithi121.stdout:0/415: symlink d5/d30/d3c/d7d/d2a/d4e/l8c 0 2024-02-15T18:35:24.842 INFO:tasks.workunit.client.1.smithi121.stdout:0/416: stat d5/d30/d3c/d7d/d2a/l52 0 2024-02-15T18:35:24.844 INFO:tasks.workunit.client.1.smithi121.stdout:1/362: link d0/d6/d7/d9/f2d d0/d6/d7/d9/d14/d48/f78 0 2024-02-15T18:35:24.844 INFO:tasks.workunit.client.1.smithi121.stdout:1/363: fdatasync d0/d39/d4b/f6d 0 2024-02-15T18:35:24.844 INFO:tasks.workunit.client.1.smithi121.stdout:1/364: readlink d0/l4 0 2024-02-15T18:35:24.844 INFO:tasks.workunit.client.1.smithi121.stdout:5/293: stat d0/c3e 0 2024-02-15T18:35:24.845 INFO:tasks.workunit.client.1.smithi121.stdout:8/402: symlink d0/d4a/l96 0 2024-02-15T18:35:24.846 INFO:tasks.workunit.client.1.smithi121.stdout:4/209: mknod d5/d31/d3d/c40 0 2024-02-15T18:35:24.847 INFO:tasks.workunit.client.1.smithi121.stdout:1/365: mknod d0/c79 0 2024-02-15T18:35:24.847 INFO:tasks.workunit.client.1.smithi121.stdout:1/366: chown d0/d39/f68 2519 1 2024-02-15T18:35:24.850 INFO:tasks.workunit.client.1.smithi121.stdout:4/210: link d5/d31/f23 d5/d9/d1d/f41 0 2024-02-15T18:35:24.852 INFO:tasks.workunit.client.1.smithi121.stdout:4/211: dread d5/d31/d25/f39 [0,4194304] 0 2024-02-15T18:35:24.853 INFO:tasks.workunit.client.1.smithi121.stdout:4/212: fdatasync d5/d9/d1d/f2f 0 2024-02-15T18:35:24.853 INFO:tasks.workunit.client.1.smithi121.stdout:1/367: rename d0/d6/l40 to d0/d6/d7/d9/d14/l7a 0 2024-02-15T18:35:24.853 INFO:tasks.workunit.client.1.smithi121.stdout:1/368: stat d0/d6/d7/d9/d14/l49 0 2024-02-15T18:35:24.856 INFO:tasks.workunit.client.1.smithi121.stdout:4/213: rename d5/f10 to d5/d9/d1d/f42 0 2024-02-15T18:35:24.887 INFO:tasks.workunit.client.1.smithi121.stdout:1/369: creat d0/d6/d7/d9/f7b x:0 0 0 2024-02-15T18:35:24.890 INFO:tasks.workunit.client.1.smithi121.stdout:5/294: dwrite d0/d2/d1e/d21/d47/d5a/f5c [0,4194304] 0 2024-02-15T18:35:24.890 INFO:tasks.workunit.client.1.smithi121.stdout:5/295: write d0/d2/fd [4672848,3937] 0 2024-02-15T18:35:24.890 INFO:tasks.workunit.client.1.smithi121.stdout:1/370: creat d0/d6/d7/d9/d14/d48/f7c x:0 0 0 2024-02-15T18:35:24.891 INFO:tasks.workunit.client.1.smithi121.stdout:1/371: write d0/f1d [3471552,29543] 0 2024-02-15T18:35:24.891 INFO:tasks.workunit.client.1.smithi121.stdout:1/372: truncate d0/d39/d4b/f61 424608 0 2024-02-15T18:35:24.891 INFO:tasks.workunit.client.1.smithi121.stdout:1/373: chown d0/d39/c42 176 1 2024-02-15T18:35:24.893 INFO:tasks.workunit.client.1.smithi121.stdout:0/417: dwrite d5/d30/d3c/d7d/d2a/d36/f3d [0,4194304] 0 2024-02-15T18:35:24.895 INFO:tasks.workunit.client.1.smithi121.stdout:5/296: getdents d0/d2/d1e/d4e 0 2024-02-15T18:35:24.896 INFO:tasks.workunit.client.1.smithi121.stdout:8/403: dwrite d0/d7/d36/f81 [0,4194304] 0 2024-02-15T18:35:24.897 INFO:tasks.workunit.client.1.smithi121.stdout:8/404: write d0/d7/de/f2d [3245745,64271] 0 2024-02-15T18:35:24.899 INFO:tasks.workunit.client.1.smithi121.stdout:8/405: write d0/d2a/d51/f38 [932352,119362] 0 2024-02-15T18:35:24.899 INFO:tasks.workunit.client.1.smithi121.stdout:8/406: chown d0/d4a/f5f 684338 1 2024-02-15T18:35:24.899 INFO:tasks.workunit.client.1.smithi121.stdout:8/407: dread - d0/d7/d36/f5c zero size 2024-02-15T18:35:24.900 INFO:tasks.workunit.client.1.smithi121.stdout:8/408: read - d0/d61/d7b/f7f zero size 2024-02-15T18:35:24.900 INFO:tasks.workunit.client.1.smithi121.stdout:8/409: readlink d0/d61/l70 0 2024-02-15T18:35:24.902 INFO:tasks.workunit.client.1.smithi121.stdout:4/214: truncate d5/d9/f20 680853 0 2024-02-15T18:35:24.909 INFO:tasks.workunit.client.1.smithi121.stdout:1/374: mkdir d0/d39/d7d 0 2024-02-15T18:35:24.909 INFO:tasks.workunit.client.1.smithi121.stdout:1/375: write d0/d6/d7/d9/f35 [470870,33933] 0 2024-02-15T18:35:24.914 INFO:tasks.workunit.client.1.smithi121.stdout:0/418: mkdir d5/d30/d3c/d7d/d2a/d73/d8d 0 2024-02-15T18:35:24.914 INFO:tasks.workunit.client.1.smithi121.stdout:0/419: write d5/d30/d3c/d7d/f3a [2193495,95774] 0 2024-02-15T18:35:24.914 INFO:tasks.workunit.client.1.smithi121.stdout:0/420: chown d5/d30/d3c/d7d/d2a/d73/l77 4 1 2024-02-15T18:35:24.915 INFO:tasks.workunit.client.1.smithi121.stdout:0/421: dread d5/d30/d3c/d7d/dd/f4b [0,4194304] 0 2024-02-15T18:35:24.928 INFO:tasks.workunit.client.1.smithi121.stdout:5/297: creat d0/d2/d1e/d21/d47/f64 x:0 0 0 2024-02-15T18:35:24.929 INFO:tasks.workunit.client.1.smithi121.stdout:5/298: read d0/d2/d18/f31 [29270,70588] 0 2024-02-15T18:35:24.929 INFO:tasks.workunit.client.1.smithi121.stdout:5/299: read - d0/f60 zero size 2024-02-15T18:35:24.930 INFO:tasks.workunit.client.1.smithi121.stdout:5/300: write d0/fb [2163026,55058] 0 2024-02-15T18:35:24.930 INFO:tasks.workunit.client.1.smithi121.stdout:5/301: fdatasync d0/d16/d34/f43 0 2024-02-15T18:35:24.935 INFO:tasks.workunit.client.1.smithi121.stdout:4/215: symlink d5/l43 0 2024-02-15T18:35:24.937 INFO:tasks.workunit.client.1.smithi121.stdout:1/376: rename d0/d6/d7/d4e to d0/d6/d7/d50/d7e 0 2024-02-15T18:35:24.938 INFO:tasks.workunit.client.1.smithi121.stdout:0/422: symlink d5/l8e 0 2024-02-15T18:35:24.938 INFO:tasks.workunit.client.1.smithi121.stdout:0/423: read - d5/d30/d3c/d7d/d2a/f8a zero size 2024-02-15T18:35:24.940 INFO:tasks.workunit.client.1.smithi121.stdout:5/302: symlink d0/d2/d1e/d21/d47/l65 0 2024-02-15T18:35:24.940 INFO:tasks.workunit.client.1.smithi121.stdout:5/303: truncate d0/d2/d18/f5b 865343 0 2024-02-15T18:35:24.941 INFO:tasks.workunit.client.1.smithi121.stdout:4/216: symlink d5/d7/l44 0 2024-02-15T18:35:24.941 INFO:tasks.workunit.client.1.smithi121.stdout:4/217: fsync d5/d7/f3c 0 2024-02-15T18:35:24.941 INFO:tasks.workunit.client.1.smithi121.stdout:4/218: write d5/d9/d1d/f2f [436498,80409] 0 2024-02-15T18:35:24.942 INFO:tasks.workunit.client.1.smithi121.stdout:1/377: rename d0/d6/d7/d9/d51/c69 to d0/d6/d7/d50/d7e/c7f 0 2024-02-15T18:35:24.943 INFO:tasks.workunit.client.1.smithi121.stdout:0/424: truncate d5/f3b 985348 0 2024-02-15T18:35:24.944 INFO:tasks.workunit.client.1.smithi121.stdout:0/425: write d5/f40 [3542018,31953] 0 2024-02-15T18:35:24.945 INFO:tasks.workunit.client.1.smithi121.stdout:4/219: symlink d5/d31/l45 0 2024-02-15T18:35:24.945 INFO:tasks.workunit.client.1.smithi121.stdout:4/220: stat d5/d7/f38 0 2024-02-15T18:35:24.948 INFO:tasks.workunit.client.1.smithi121.stdout:0/426: creat d5/d30/d3c/f8f x:0 0 0 2024-02-15T18:35:24.948 INFO:tasks.workunit.client.1.smithi121.stdout:0/427: chown d5/d30/d3c/d7d/dd/f25 59 1 2024-02-15T18:35:24.949 INFO:tasks.workunit.client.1.smithi121.stdout:0/428: dread d5/d30/d3c/d7d/dd/f32 [0,4194304] 0 2024-02-15T18:35:24.949 INFO:tasks.workunit.client.1.smithi121.stdout:4/221: symlink d5/d9/d19/l46 0 2024-02-15T18:35:24.949 INFO:tasks.workunit.client.1.smithi121.stdout:4/222: truncate d5/d9/d19/d2a/f32 332141 0 2024-02-15T18:35:24.959 INFO:tasks.workunit.client.1.smithi121.stdout:4/223: creat d5/d31/d3e/f47 x:0 0 0 2024-02-15T18:35:24.962 INFO:tasks.workunit.client.1.smithi121.stdout:5/304: dwrite d0/d16/d34/f43 [0,4194304] 0 2024-02-15T18:35:24.963 INFO:tasks.workunit.client.1.smithi121.stdout:5/305: chown d0/d16 968 1 2024-02-15T18:35:24.963 INFO:tasks.workunit.client.1.smithi121.stdout:5/306: truncate d0/d16/d34/f45 194898 0 2024-02-15T18:35:24.963 INFO:tasks.workunit.client.1.smithi121.stdout:5/307: dread - d0/f4b zero size 2024-02-15T18:35:24.963 INFO:tasks.workunit.client.1.smithi121.stdout:5/308: truncate d0/fb 6229987 0 2024-02-15T18:35:24.967 INFO:tasks.workunit.client.1.smithi121.stdout:5/309: getdents d0/d2/d1e/d21/d47 0 2024-02-15T18:35:24.967 INFO:tasks.workunit.client.1.smithi121.stdout:5/310: dread - d0/d2/d1e/d21/d33/f56 zero size 2024-02-15T18:35:24.969 INFO:tasks.workunit.client.1.smithi121.stdout:1/378: dwrite d0/f21 [4194304,4194304] 0 2024-02-15T18:35:24.969 INFO:tasks.workunit.client.1.smithi121.stdout:5/311: dread d0/f22 [0,4194304] 0 2024-02-15T18:35:24.970 INFO:tasks.workunit.client.1.smithi121.stdout:1/379: mknod d0/d39/c80 0 2024-02-15T18:35:24.978 INFO:tasks.workunit.client.1.smithi121.stdout:5/312: mknod d0/d2/d1e/d21/d39/c66 0 2024-02-15T18:35:24.985 INFO:tasks.workunit.client.1.smithi121.stdout:5/313: symlink d0/d16/l67 0 2024-02-15T18:35:24.985 INFO:tasks.workunit.client.1.smithi121.stdout:4/224: dwrite d5/d7/d14/f17 [0,4194304] 0 2024-02-15T18:35:24.986 INFO:tasks.workunit.client.1.smithi121.stdout:5/314: getdents d0/d2/d18 0 2024-02-15T18:35:24.990 INFO:tasks.workunit.client.1.smithi121.stdout:1/380: dwrite d0/d6/d7/d50/d7e/f5b [0,4194304] 0 2024-02-15T18:35:24.990 INFO:tasks.workunit.client.1.smithi121.stdout:5/315: truncate d0/d2/d18/f29 746569 0 2024-02-15T18:35:24.990 INFO:tasks.workunit.client.1.smithi121.stdout:5/316: dread - d0/f5f zero size 2024-02-15T18:35:24.993 INFO:tasks.workunit.client.1.smithi121.stdout:4/225: dread d5/fa [0,4194304] 0 2024-02-15T18:35:24.995 INFO:tasks.workunit.client.1.smithi121.stdout:1/381: mknod d0/d6/d7/d9/d14/d48/c81 0 2024-02-15T18:35:25.001 INFO:tasks.workunit.client.1.smithi121.stdout:4/226: dread f1 [4194304,4194304] 0 2024-02-15T18:35:25.001 INFO:tasks.workunit.client.1.smithi121.stdout:4/227: chown d5/d31/d25/f39 5559 1 2024-02-15T18:35:25.002 INFO:tasks.workunit.client.1.smithi121.stdout:4/228: readlink d5/d7/l1c 0 2024-02-15T18:35:25.063 INFO:tasks.workunit.client.1.smithi121.stdout:5/317: creat d0/d16/f68 x:0 0 0 2024-02-15T18:35:25.064 INFO:tasks.workunit.client.1.smithi121.stdout:5/318: dread d0/d2/d18/f5b [0,4194304] 0 2024-02-15T18:35:25.064 INFO:tasks.workunit.client.1.smithi121.stdout:5/319: write d0/d2/d1e/d21/d33/f56 [529776,7928] 0 2024-02-15T18:35:25.071 INFO:tasks.workunit.client.1.smithi121.stdout:1/382: dread d0/d39/d4b/f55 [0,4194304] 0 2024-02-15T18:35:25.071 INFO:tasks.workunit.client.1.smithi121.stdout:1/383: fsync d0/d6/d7/d9/d14/f58 0 2024-02-15T18:35:25.071 INFO:tasks.workunit.client.1.smithi121.stdout:1/384: write d0/d6/d7/d29/f54 [4714540,114923] 0 2024-02-15T18:35:25.071 INFO:tasks.workunit.client.1.smithi121.stdout:1/385: chown d0/d6/d7/d50/c6f 34 1 2024-02-15T18:35:25.088 INFO:tasks.workunit.client.1.smithi121.stdout:4/229: creat d5/d7/d1e/f48 x:0 0 0 2024-02-15T18:35:25.106 INFO:tasks.workunit.client.1.smithi121.stdout:5/320: mknod d0/d2/d1e/d21/d40/d62/c69 0 2024-02-15T18:35:25.106 INFO:tasks.workunit.client.1.smithi121.stdout:5/321: chown d0/d16/f2b 256170 1 2024-02-15T18:35:25.108 INFO:tasks.workunit.client.1.smithi121.stdout:1/386: getdents d0/d6/d7/d9/d14/d3e 0 2024-02-15T18:35:25.108 INFO:tasks.workunit.client.1.smithi121.stdout:4/230: creat d5/d31/d3d/f49 x:0 0 0 2024-02-15T18:35:25.115 INFO:tasks.workunit.client.1.smithi121.stdout:5/322: dread d0/d2/d1e/d21/d47/d5a/f5c [0,4194304] 0 2024-02-15T18:35:25.115 INFO:tasks.workunit.client.1.smithi121.stdout:5/323: dread - d0/d16/f2b zero size 2024-02-15T18:35:25.115 INFO:tasks.workunit.client.1.smithi121.stdout:5/324: truncate d0/d16/f68 550396 0 2024-02-15T18:35:25.117 INFO:tasks.workunit.client.1.smithi121.stdout:5/325: read d0/d16/d34/f45 [114952,81060] 0 2024-02-15T18:35:25.117 INFO:tasks.workunit.client.1.smithi121.stdout:4/231: dread d5/d9/d19/d2a/f2c [0,4194304] 0 2024-02-15T18:35:25.136 INFO:tasks.workunit.client.1.smithi121.stdout:4/232: dwrite d5/d31/d3d/f49 [0,4194304] 0 2024-02-15T18:35:25.143 INFO:tasks.workunit.client.1.smithi121.stdout:7/241: sync 2024-02-15T18:35:25.144 INFO:tasks.workunit.client.1.smithi121.stdout:9/267: sync 2024-02-15T18:35:25.144 INFO:tasks.workunit.client.1.smithi121.stdout:9/268: chown d7/d8/d25/d30 249351373 1 2024-02-15T18:35:25.144 INFO:tasks.workunit.client.1.smithi121.stdout:9/269: write d7/d8/f47 [1148827,103258] 0 2024-02-15T18:35:25.416 INFO:tasks.workunit.client.1.smithi121.stdout:1/387: mkdir d0/d6/d7/d72/d82 0 2024-02-15T18:35:25.417 INFO:tasks.workunit.client.1.smithi121.stdout:5/326: unlink d0/f22 0 2024-02-15T18:35:25.417 INFO:tasks.workunit.client.1.smithi121.stdout:5/327: dread - d0/d2/d1e/d21/d47/f64 zero size 2024-02-15T18:35:25.428 INFO:tasks.workunit.client.1.smithi121.stdout:5/328: dread d0/d16/d34/f3a [0,4194304] 0 2024-02-15T18:35:25.430 INFO:tasks.workunit.client.1.smithi121.stdout:1/388: dread d0/f21 [0,4194304] 0 2024-02-15T18:35:25.431 INFO:tasks.workunit.client.1.smithi121.stdout:1/389: chown d0/d6/d7/d29/f5d 29807 1 2024-02-15T18:35:25.432 INFO:tasks.workunit.client.1.smithi121.stdout:1/390: dread d0/d39/d4b/f61 [0,4194304] 0 2024-02-15T18:35:25.444 INFO:tasks.workunit.client.1.smithi121.stdout:7/242: rename d0/d5/d11/l24 to d0/d5/l54 0 2024-02-15T18:35:25.444 INFO:tasks.workunit.client.1.smithi121.stdout:7/243: write d0/d5/f47 [1037996,15562] 0 2024-02-15T18:35:25.445 INFO:tasks.workunit.client.1.smithi121.stdout:9/270: mknod d7/d3a/d42/d56/c5c 0 2024-02-15T18:35:25.445 INFO:tasks.workunit.client.1.smithi121.stdout:9/271: fsync d7/d8/d25/d30/f32 0 2024-02-15T18:35:25.447 INFO:tasks.workunit.client.1.smithi121.stdout:5/329: rename d0/d16/d34/f3a to d0/d16/d34/f6a 0 2024-02-15T18:35:25.448 INFO:tasks.workunit.client.1.smithi121.stdout:1/391: creat d0/d6/d7/d9/d14/f83 x:0 0 0 2024-02-15T18:35:25.453 INFO:tasks.workunit.client.1.smithi121.stdout:7/244: truncate d0/d5/d11/f48 714243 0 2024-02-15T18:35:25.453 INFO:tasks.workunit.client.1.smithi121.stdout:7/245: write d0/d33/f34 [803397,42050] 0 2024-02-15T18:35:25.455 INFO:tasks.workunit.client.1.smithi121.stdout:1/392: mknod d0/d6/d7/d72/c84 0 2024-02-15T18:35:25.455 INFO:tasks.workunit.client.1.smithi121.stdout:7/246: mknod d0/c55 0 2024-02-15T18:35:25.455 INFO:tasks.workunit.client.1.smithi121.stdout:7/247: chown d0/d33/f34 41 1 2024-02-15T18:35:25.456 INFO:tasks.workunit.client.1.smithi121.stdout:7/248: mknod d0/d38/d3f/c56 0 2024-02-15T18:35:25.457 INFO:tasks.workunit.client.1.smithi121.stdout:7/249: readlink d0/l14 0 2024-02-15T18:35:25.457 INFO:tasks.workunit.client.1.smithi121.stdout:9/272: dread d7/d8/d25/d30/d33/d3e/f41 [0,4194304] 0 2024-02-15T18:35:25.457 INFO:tasks.workunit.client.1.smithi121.stdout:9/273: write d7/d8/d25/d30/d33/f58 [261985,94731] 0 2024-02-15T18:35:25.458 INFO:tasks.workunit.client.1.smithi121.stdout:9/274: getdents d7/d8 0 2024-02-15T18:35:25.458 INFO:tasks.workunit.client.1.smithi121.stdout:9/275: fdatasync d7/f1f 0 2024-02-15T18:35:25.464 INFO:tasks.workunit.client.1.smithi121.stdout:9/276: symlink d7/d8/d25/d30/d33/d3c/l5d 0 2024-02-15T18:35:25.468 INFO:tasks.workunit.client.1.smithi121.stdout:9/277: write d7/f1c [56497,84351] 0 2024-02-15T18:35:25.474 INFO:tasks.workunit.client.1.smithi121.stdout:8/410: sync 2024-02-15T18:35:25.474 INFO:tasks.workunit.client.1.smithi121.stdout:0/429: sync 2024-02-15T18:35:25.474 INFO:tasks.workunit.client.1.smithi121.stdout:3/334: sync 2024-02-15T18:35:25.474 INFO:tasks.workunit.client.1.smithi121.stdout:8/411: stat d0/d2a/f7d 0 2024-02-15T18:35:25.474 INFO:tasks.workunit.client.1.smithi121.stdout:3/335: stat d2/d1a/d27/d70 0 2024-02-15T18:35:25.474 INFO:tasks.workunit.client.1.smithi121.stdout:8/412: chown d0/d2a/d51/f6f 1 1 2024-02-15T18:35:25.474 INFO:tasks.workunit.client.1.smithi121.stdout:3/336: truncate d2/d1a/d27/f5e 71909 0 2024-02-15T18:35:25.474 INFO:tasks.workunit.client.1.smithi121.stdout:8/413: truncate d0/d7/de/f8e 284604 0 2024-02-15T18:35:25.474 INFO:tasks.workunit.client.1.smithi121.stdout:3/337: read - d2/d1a/d27/d2c/d25/d31/d58/f6f zero size 2024-02-15T18:35:25.474 INFO:tasks.workunit.client.1.smithi121.stdout:8/414: write d0/f2 [4447374,107948] 0 2024-02-15T18:35:25.475 INFO:tasks.workunit.client.1.smithi121.stdout:4/233: sync 2024-02-15T18:35:25.475 INFO:tasks.workunit.client.1.smithi121.stdout:4/234: write d5/d7/f38 [1667903,62393] 0 2024-02-15T18:35:25.475 INFO:tasks.workunit.client.1.smithi121.stdout:4/235: fdatasync d5/d7/f3c 0 2024-02-15T18:35:25.475 INFO:tasks.workunit.client.1.smithi121.stdout:4/236: dread - d5/d7/d1e/f48 zero size 2024-02-15T18:35:25.475 INFO:tasks.workunit.client.1.smithi121.stdout:2/242: sync 2024-02-15T18:35:25.475 INFO:tasks.workunit.client.1.smithi121.stdout:6/342: sync 2024-02-15T18:35:25.475 INFO:tasks.workunit.client.1.smithi121.stdout:6/343: chown d7/dc/d34/d42/c64 3373 1 2024-02-15T18:35:25.475 INFO:tasks.workunit.client.1.smithi121.stdout:6/344: write d7/dc/d34/f13 [746217,39473] 0 2024-02-15T18:35:25.477 INFO:tasks.workunit.client.1.smithi121.stdout:5/330: dwrite d0/d2/d18/f27 [4194304,4194304] 0 2024-02-15T18:35:25.478 INFO:tasks.workunit.client.1.smithi121.stdout:1/393: dwrite d0/f53 [0,4194304] 0 2024-02-15T18:35:25.478 INFO:tasks.workunit.client.1.smithi121.stdout:1/394: write d0/d6/d7/f4c [1244335,103261] 0 2024-02-15T18:35:25.483 INFO:tasks.workunit.client.1.smithi121.stdout:7/250: dwrite d0/d5/f2b [0,4194304] 0 2024-02-15T18:35:25.485 INFO:tasks.workunit.client.1.smithi121.stdout:0/430: dread d5/d30/d3c/d7d/dd/f68 [0,4194304] 0 2024-02-15T18:35:25.486 INFO:tasks.workunit.client.1.smithi121.stdout:0/431: write d5/d30/d3c/d7d/f3a [1458940,61405] 0 2024-02-15T18:35:25.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:24.592095+0000 mgr.smithi121.ckiryi (mgr.24385) 4 : cluster [DBG] pgmap v6: 65 pgs: 65 active+clean; 445 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:35:25.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:24.936450+0000 mon.smithi038 (mon.0) 588 : cluster [DBG] Standby manager daemon smithi038.khesbl started 2024-02-15T18:35:25.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:24.939274+0000 mon.smithi038 (mon.0) 589 : audit [DBG] from='mgr.? 172.21.15.38:0/4155076373' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi038.khesbl/crt"}]: dispatch 2024-02-15T18:35:25.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:24.939826+0000 mon.smithi038 (mon.0) 590 : audit [DBG] from='mgr.? 172.21.15.38:0/4155076373' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:35:25.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:24.944324+0000 mon.smithi038 (mon.0) 591 : audit [DBG] from='mgr.? 172.21.15.38:0/4155076373' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi038.khesbl/key"}]: dispatch 2024-02-15T18:35:25.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:24.944661+0000 mon.smithi038 (mon.0) 592 : audit [DBG] from='mgr.? 172.21.15.38:0/4155076373' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:35:25.702 INFO:tasks.workunit.client.1.smithi121.stdout:9/278: mknod d7/d8/c5e 0 2024-02-15T18:35:25.704 INFO:tasks.workunit.client.1.smithi121.stdout:8/415: creat d0/d2a/f97 x:0 0 0 2024-02-15T18:35:25.705 INFO:tasks.workunit.client.1.smithi121.stdout:4/237: mkdir d5/d31/d3d/d4a 0 2024-02-15T18:35:25.705 INFO:tasks.workunit.client.1.smithi121.stdout:4/238: chown d5/d7/l1c 59494 1 2024-02-15T18:35:25.707 INFO:tasks.workunit.client.1.smithi121.stdout:2/243: creat d5/df/d10/d15/d22/f44 x:0 0 0 2024-02-15T18:35:25.707 INFO:tasks.workunit.client.1.smithi121.stdout:2/244: chown d5/df/d10/f11 651780666 1 2024-02-15T18:35:25.709 INFO:tasks.workunit.client.1.smithi121.stdout:6/345: symlink d7/dc/d34/d42/d6e/l78 0 2024-02-15T18:35:25.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:24.592095+0000 mgr.smithi121.ckiryi (mgr.24385) 4 : cluster [DBG] pgmap v6: 65 pgs: 65 active+clean; 445 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:35:25.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:24.936450+0000 mon.smithi038 (mon.0) 588 : cluster [DBG] Standby manager daemon smithi038.khesbl started 2024-02-15T18:35:25.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:24.939274+0000 mon.smithi038 (mon.0) 589 : audit [DBG] from='mgr.? 172.21.15.38:0/4155076373' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi038.khesbl/crt"}]: dispatch 2024-02-15T18:35:25.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:24.939826+0000 mon.smithi038 (mon.0) 590 : audit [DBG] from='mgr.? 172.21.15.38:0/4155076373' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:35:25.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:24.944324+0000 mon.smithi038 (mon.0) 591 : audit [DBG] from='mgr.? 172.21.15.38:0/4155076373' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi038.khesbl/key"}]: dispatch 2024-02-15T18:35:25.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:24.944661+0000 mon.smithi038 (mon.0) 592 : audit [DBG] from='mgr.? 172.21.15.38:0/4155076373' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:35:25.972 INFO:tasks.workunit.client.1.smithi121.stdout:5/331: write d0/d2/d1e/d21/d33/f56 [130227,12126] 0 2024-02-15T18:35:25.973 INFO:tasks.workunit.client.1.smithi121.stdout:1/395: creat d0/d6/d7/d50/d7e/f85 x:0 0 0 2024-02-15T18:35:25.974 INFO:tasks.workunit.client.1.smithi121.stdout:7/251: symlink d0/d38/d3f/d43/l57 0 2024-02-15T18:35:25.974 INFO:tasks.workunit.client.1.smithi121.stdout:7/252: write d0/d5/d11/d1c/f1f [826991,119602] 0 2024-02-15T18:35:25.976 INFO:tasks.workunit.client.1.smithi121.stdout:1/396: dread d0/d6/d7/d9/d14/d3e/f43 [0,4194304] 0 2024-02-15T18:35:25.981 INFO:tasks.workunit.client.1.smithi121.stdout:0/432: link d5/d30/d3c/d7d/d2a/d73/l84 d5/d30/d3c/d7d/d2a/d36/d44/d71/l90 0 2024-02-15T18:35:25.988 INFO:tasks.workunit.client.1.smithi121.stdout:9/279: rmdir d7/d8/d18 39 2024-02-15T18:35:25.988 INFO:tasks.workunit.client.1.smithi121.stdout:9/280: truncate d7/f59 272078 0 2024-02-15T18:35:25.988 INFO:tasks.workunit.client.1.smithi121.stdout:9/281: write d7/d8/d36/f2f [249669,105276] 0 2024-02-15T18:35:25.989 INFO:tasks.workunit.client.1.smithi121.stdout:8/416: rename d0/d7/c21 to d0/d61/d8f/c98 0 2024-02-15T18:35:25.991 INFO:tasks.workunit.client.1.smithi121.stdout:4/239: link d5/d9/d19/d2a/c2b d5/d7/c4b 0 2024-02-15T18:35:25.992 INFO:tasks.workunit.client.1.smithi121.stdout:8/417: read d0/d7/d36/d8c/d45/f2c [6295345,98919] 0 2024-02-15T18:35:25.992 INFO:tasks.workunit.client.1.smithi121.stdout:6/346: symlink d7/dc/d2c/d36/l79 0 2024-02-15T18:35:25.993 INFO:tasks.workunit.client.1.smithi121.stdout:7/253: dwrite d0/d5/f15 [0,4194304] 0 2024-02-15T18:35:25.994 INFO:tasks.workunit.client.1.smithi121.stdout:1/397: creat d0/d39/d7d/f86 x:0 0 0 2024-02-15T18:35:25.996 INFO:tasks.workunit.client.1.smithi121.stdout:0/433: creat d5/d30/d3c/d7d/d2a/f91 x:0 0 0 2024-02-15T18:35:25.997 INFO:tasks.workunit.client.1.smithi121.stdout:1/398: read d0/f53 [2245889,91321] 0 2024-02-15T18:35:26.001 INFO:tasks.workunit.client.1.smithi121.stdout:9/282: symlink d7/d3a/d42/l5f 0 2024-02-15T18:35:26.002 INFO:tasks.workunit.client.1.smithi121.stdout:8/418: creat d0/d4a/f99 x:0 0 0 2024-02-15T18:35:26.002 INFO:tasks.workunit.client.1.smithi121.stdout:8/419: fdatasync d0/d7/f5e 0 2024-02-15T18:35:26.002 INFO:tasks.workunit.client.1.smithi121.stdout:8/420: chown d0/d7/d36/c4d 405116114 1 2024-02-15T18:35:26.003 INFO:tasks.workunit.client.1.smithi121.stdout:7/254: dread d0/d5/d11/d1c/f44 [0,4194304] 0 2024-02-15T18:35:26.004 INFO:tasks.workunit.client.1.smithi121.stdout:6/347: creat d7/dc/d34/d42/d62/d63/f7a x:0 0 0 2024-02-15T18:35:26.005 INFO:tasks.workunit.client.1.smithi121.stdout:5/332: mknod d0/d2/d1e/d21/c6b 0 2024-02-15T18:35:26.007 INFO:tasks.workunit.client.1.smithi121.stdout:0/434: creat d5/f92 x:0 0 0 2024-02-15T18:35:26.007 INFO:tasks.workunit.client.1.smithi121.stdout:0/435: fsync d5/d30/d5d/f5e 0 2024-02-15T18:35:26.007 INFO:tasks.workunit.client.1.smithi121.stdout:0/436: write d5/d30/d3c/d7d/d2a/d4e/f5a [4742849,92219] 0 2024-02-15T18:35:26.008 INFO:tasks.workunit.client.1.smithi121.stdout:0/437: chown d5/d30/d3c/d7d/d2a/d4e/f5a 62474 1 2024-02-15T18:35:26.008 INFO:tasks.workunit.client.1.smithi121.stdout:1/399: mknod d0/d6/d7/d50/c87 0 2024-02-15T18:35:26.010 INFO:tasks.workunit.client.1.smithi121.stdout:8/421: getdents d0/d3 0 2024-02-15T18:35:26.012 INFO:tasks.workunit.client.1.smithi121.stdout:6/348: rename d7/dc/d34/c2f to d7/d4b/c7b 0 2024-02-15T18:35:26.012 INFO:tasks.workunit.client.1.smithi121.stdout:5/333: mknod d0/d2/d1e/d21/d33/c6c 0 2024-02-15T18:35:26.013 INFO:tasks.workunit.client.1.smithi121.stdout:6/349: mkdir d7/dc/d34/d7c 0 2024-02-15T18:35:26.015 INFO:tasks.workunit.client.1.smithi121.stdout:0/438: rename d5/d30/d3c/d7d/d2a/d36/d44/d71/f85 to d5/d30/d3c/d7d/dd/f93 0 2024-02-15T18:35:26.016 INFO:tasks.workunit.client.1.smithi121.stdout:5/334: mknod d0/d2/c6d 0 2024-02-15T18:35:26.016 INFO:tasks.workunit.client.1.smithi121.stdout:5/335: dread - d0/d2/d1e/d21/d33/d4c/f5e zero size 2024-02-15T18:35:26.018 INFO:tasks.workunit.client.1.smithi121.stdout:6/350: symlink d7/dc/d34/d42/d62/d63/d6f/l7d 0 2024-02-15T18:35:26.019 INFO:tasks.workunit.client.1.smithi121.stdout:6/351: read d7/dc/f21 [587894,127603] 0 2024-02-15T18:35:26.019 INFO:tasks.workunit.client.1.smithi121.stdout:6/352: write d7/dc/d34/d42/d62/d63/f7a [75639,100032] 0 2024-02-15T18:35:26.021 INFO:tasks.workunit.client.1.smithi121.stdout:0/439: mknod d5/d30/d3c/d7d/d74/c94 0 2024-02-15T18:35:26.025 INFO:tasks.workunit.client.1.smithi121.stdout:8/422: rename d0/d7/de/f8e to d0/d7/d36/d52/d7a/f9a 0 2024-02-15T18:35:26.034 INFO:tasks.workunit.client.1.smithi121.stdout:9/283: dwrite d7/f1c [0,4194304] 0 2024-02-15T18:35:26.036 INFO:tasks.workunit.client.1.smithi121.stdout:7/255: dwrite d0/d5/f2b [0,4194304] 0 2024-02-15T18:35:26.036 INFO:tasks.workunit.client.1.smithi121.stdout:7/256: fdatasync d0/d4a/f53 0 2024-02-15T18:35:26.037 INFO:tasks.workunit.client.1.smithi121.stdout:7/257: chown d0/d5/d11/c13 11057 1 2024-02-15T18:35:26.037 INFO:tasks.workunit.client.1.smithi121.stdout:1/400: dwrite d0/d6/d7/d50/d7e/f85 [0,4194304] 0 2024-02-15T18:35:26.037 INFO:tasks.workunit.client.1.smithi121.stdout:1/401: chown d0/l36 262904879 1 2024-02-15T18:35:26.037 INFO:tasks.workunit.client.1.smithi121.stdout:7/258: symlink d0/d38/d3f/l58 0 2024-02-15T18:35:26.037 INFO:tasks.workunit.client.1.smithi121.stdout:9/284: read d7/d8/f10 [1046847,128881] 0 2024-02-15T18:35:26.041 INFO:tasks.workunit.client.1.smithi121.stdout:8/423: dwrite d0/d3/d20/f78 [0,4194304] 0 2024-02-15T18:35:26.043 INFO:tasks.workunit.client.1.smithi121.stdout:1/402: mkdir d0/d6/d7/d9/d14/d3e/d88 0 2024-02-15T18:35:26.044 INFO:tasks.workunit.client.1.smithi121.stdout:7/259: truncate d0/d33/f3b 1530467 0 2024-02-15T18:35:26.045 INFO:tasks.workunit.client.1.smithi121.stdout:9/285: mknod d7/d8/d25/d30/d33/d3e/c60 0 2024-02-15T18:35:26.056 INFO:tasks.workunit.client.1.smithi121.stdout:1/403: truncate d0/d6/d7/d9/d14/d48/f78 3800074 0 2024-02-15T18:35:26.057 INFO:tasks.workunit.client.1.smithi121.stdout:7/260: rmdir d0/d38/d3c 39 2024-02-15T18:35:26.058 INFO:tasks.workunit.client.1.smithi121.stdout:9/286: symlink d7/d8/d25/d30/d33/l61 0 2024-02-15T18:35:26.061 INFO:tasks.workunit.client.1.smithi121.stdout:7/261: rename d0/d5/d11/f51 to d0/d5/d4c/f59 0 2024-02-15T18:35:26.061 INFO:tasks.workunit.client.1.smithi121.stdout:7/262: readlink d0/d5/d11/d1c/l2e 0 2024-02-15T18:35:26.070 INFO:tasks.workunit.client.1.smithi121.stdout:7/263: dread d0/f29 [0,4194304] 0 2024-02-15T18:35:26.070 INFO:tasks.workunit.client.1.smithi121.stdout:7/264: fsync d0/d5/d11/d1c/f21 0 2024-02-15T18:35:26.070 INFO:tasks.workunit.client.1.smithi121.stdout:7/265: rmdir d0/d38/d3c 39 2024-02-15T18:35:26.072 INFO:tasks.workunit.client.1.smithi121.stdout:7/266: getdents d0/d38/d3c 0 2024-02-15T18:35:26.072 INFO:tasks.workunit.client.1.smithi121.stdout:7/267: chown d0/d5/f15 1857886303 1 2024-02-15T18:35:26.075 INFO:tasks.workunit.client.1.smithi121.stdout:9/287: dwrite d7/f57 [0,4194304] 0 2024-02-15T18:35:26.075 INFO:tasks.workunit.client.1.smithi121.stdout:9/288: chown d7/d8/f47 57214590 1 2024-02-15T18:35:26.075 INFO:tasks.workunit.client.1.smithi121.stdout:9/289: dread - d7/f4e zero size 2024-02-15T18:35:26.075 INFO:tasks.workunit.client.1.smithi121.stdout:9/290: read d7/d8/f3b [197851,9518] 0 2024-02-15T18:35:26.075 INFO:tasks.workunit.client.1.smithi121.stdout:9/291: write d7/d8/d25/d30/d33/f58 [624737,22321] 0 2024-02-15T18:35:26.075 INFO:tasks.workunit.client.1.smithi121.stdout:9/292: truncate d7/d8/f16 799489 0 2024-02-15T18:35:26.076 INFO:tasks.workunit.client.1.smithi121.stdout:9/293: write d7/d8/d25/d30/f4c [259143,78300] 0 2024-02-15T18:35:26.077 INFO:tasks.workunit.client.1.smithi121.stdout:9/294: stat d7/d8/d18/l21 0 2024-02-15T18:35:26.081 INFO:tasks.workunit.client.1.smithi121.stdout:9/295: read d7/d8/d18/f19 [1441648,32345] 0 2024-02-15T18:35:26.083 INFO:tasks.workunit.client.1.smithi121.stdout:9/296: creat d7/d8/d25/d30/f62 x:0 0 0 2024-02-15T18:35:26.092 INFO:tasks.workunit.client.1.smithi121.stdout:7/268: dwrite d0/d5/f2b [0,4194304] 0 2024-02-15T18:35:26.092 INFO:tasks.workunit.client.1.smithi121.stdout:7/269: chown d0/la 1660 1 2024-02-15T18:35:26.092 INFO:tasks.workunit.client.1.smithi121.stdout:7/270: chown d0/d5/d11/d1c/f35 8730145 1 2024-02-15T18:35:26.092 INFO:tasks.workunit.client.1.smithi121.stdout:7/271: getdents d0/d38/d3c 0 2024-02-15T18:35:26.092 INFO:tasks.workunit.client.1.smithi121.stdout:7/272: fdatasync d0/d5/d11/f48 0 2024-02-15T18:35:26.100 INFO:tasks.workunit.client.1.smithi121.stdout:9/297: dwrite d7/d8/d25/d30/d33/f58 [0,4194304] 0 2024-02-15T18:35:26.100 INFO:tasks.workunit.client.1.smithi121.stdout:9/298: chown d7/d8/d25/d30/f32 97630725 1 2024-02-15T18:35:26.100 INFO:tasks.workunit.client.1.smithi121.stdout:9/299: fsync d7/d8/d18/f53 0 2024-02-15T18:35:26.172 INFO:tasks.workunit.client.1.smithi121.stdout:9/300: dwrite d7/f1b [0,4194304] 0 2024-02-15T18:35:26.173 INFO:tasks.workunit.client.1.smithi121.stdout:9/301: creat d7/d8/d40/f63 x:0 0 0 2024-02-15T18:35:26.192 INFO:tasks.workunit.client.1.smithi121.stdout:9/302: dwrite d7/f23 [0,4194304] 0 2024-02-15T18:35:26.192 INFO:tasks.workunit.client.1.smithi121.stdout:9/303: readlink d7/d8/d18/l24 0 2024-02-15T18:35:26.193 INFO:tasks.workunit.client.1.smithi121.stdout:9/304: creat d7/d8/d25/d30/f64 x:0 0 0 2024-02-15T18:35:26.193 INFO:tasks.workunit.client.1.smithi121.stdout:9/305: fdatasync d7/d8/d36/f2f 0 2024-02-15T18:35:26.194 INFO:tasks.workunit.client.1.smithi121.stdout:9/306: creat d7/d8/f65 x:0 0 0 2024-02-15T18:35:26.195 INFO:tasks.workunit.client.1.smithi121.stdout:9/307: symlink d7/l66 0 2024-02-15T18:35:26.197 INFO:tasks.workunit.client.1.smithi121.stdout:9/308: rename d7/d8/d36/l37 to d7/d8/d25/l67 0 2024-02-15T18:35:26.198 INFO:tasks.workunit.client.1.smithi121.stdout:9/309: creat d7/d8/d40/f68 x:0 0 0 2024-02-15T18:35:26.214 INFO:tasks.workunit.client.1.smithi121.stdout:9/310: dwrite d7/f57 [0,4194304] 0 2024-02-15T18:35:26.214 INFO:tasks.workunit.client.1.smithi121.stdout:9/311: chown d7/d8/d18/l21 101416831 1 2024-02-15T18:35:26.228 INFO:tasks.workunit.client.1.smithi121.stdout:9/312: dwrite d7/f4e [0,4194304] 0 2024-02-15T18:35:26.228 INFO:tasks.workunit.client.1.smithi121.stdout:9/313: readlink d7/l29 0 2024-02-15T18:35:26.228 INFO:tasks.workunit.client.1.smithi121.stdout:9/314: write d7/d8/d36/f2f [1329286,81491] 0 2024-02-15T18:35:26.228 INFO:tasks.workunit.client.1.smithi121.stdout:9/315: readlink d7/d8/d25/l67 0 2024-02-15T18:35:26.230 INFO:tasks.workunit.client.1.smithi121.stdout:9/316: mknod d7/d26/c69 0 2024-02-15T18:35:26.232 INFO:tasks.workunit.client.1.smithi121.stdout:9/317: rename d7/d8/d40/f68 to d7/d3a/d42/f6a 0 2024-02-15T18:35:26.232 INFO:tasks.workunit.client.1.smithi121.stdout:9/318: chown d7/f13 6963729 1 2024-02-15T18:35:26.236 INFO:tasks.workunit.client.1.smithi121.stdout:9/319: creat d7/d26/f6b x:0 0 0 2024-02-15T18:35:26.237 INFO:tasks.workunit.client.1.smithi121.stdout:9/320: rename d7/d8/f3b to d7/d3a/d42/f6c 0 2024-02-15T18:35:26.238 INFO:tasks.workunit.client.1.smithi121.stdout:9/321: write f5 [1064523,15015] 0 2024-02-15T18:35:26.239 INFO:tasks.workunit.client.1.smithi121.stdout:9/322: creat d7/d8/d25/d30/d33/d3c/d3f/f6d x:0 0 0 2024-02-15T18:35:26.240 INFO:tasks.workunit.client.1.smithi121.stdout:9/323: creat d7/d8/d40/f6e x:0 0 0 2024-02-15T18:35:26.241 INFO:tasks.workunit.client.1.smithi121.stdout:9/324: mknod d7/d8/c6f 0 2024-02-15T18:35:26.242 INFO:tasks.workunit.client.1.smithi121.stdout:9/325: rename d7/d8/d25/d30/d33/d3c/f54 to d7/d8/d40/f70 0 2024-02-15T18:35:26.243 INFO:tasks.workunit.client.1.smithi121.stdout:9/326: link d7/c2a d7/d8/d25/c71 0 2024-02-15T18:35:26.243 INFO:tasks.workunit.client.1.smithi121.stdout:9/327: chown d7/d8/f16 313012298 1 2024-02-15T18:35:26.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:25.365283+0000 mon.smithi121 (mon.1) 44 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mgr metadata", "who": "smithi038.khesbl", "id": "smithi038.khesbl"}]: dispatch 2024-02-15T18:35:26.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:25.365354+0000 mon.smithi038 (mon.0) 593 : cluster [DBG] mgrmap e23: smithi121.ckiryi(active, since 7s), standbys: smithi038.khesbl 2024-02-15T18:35:26.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:25.674259+0000 mon.smithi038 (mon.0) 594 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:26.775 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:25.365283+0000 mon.smithi121 (mon.1) 44 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mgr metadata", "who": "smithi038.khesbl", "id": "smithi038.khesbl"}]: dispatch 2024-02-15T18:35:26.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:25.365354+0000 mon.smithi038 (mon.0) 593 : cluster [DBG] mgrmap e23: smithi121.ckiryi(active, since 7s), standbys: smithi038.khesbl 2024-02-15T18:35:26.776 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:25.674259+0000 mon.smithi038 (mon.0) 594 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:26.889 INFO:tasks.workunit.client.1.smithi121.stdout:4/240: unlink d5/d31/d25/f39 0 2024-02-15T18:35:26.889 INFO:tasks.workunit.client.1.smithi121.stdout:4/241: chown d5/d9/l1b 103119 1 2024-02-15T18:35:26.889 INFO:tasks.workunit.client.1.smithi121.stdout:6/353: unlink d7/f47 0 2024-02-15T18:35:26.891 INFO:tasks.workunit.client.1.smithi121.stdout:0/440: unlink d5/d30/d3c/d7d/d2a/f8a 0 2024-02-15T18:35:26.891 INFO:tasks.workunit.client.1.smithi121.stdout:0/441: readlink d5/d30/d3c/d7d/d2a/d36/d44/l48 0 2024-02-15T18:35:26.891 INFO:tasks.workunit.client.1.smithi121.stdout:6/354: dread d7/f18 [0,4194304] 0 2024-02-15T18:35:26.892 INFO:tasks.workunit.client.1.smithi121.stdout:4/242: dread d5/d9/d1d/f2f [0,4194304] 0 2024-02-15T18:35:26.892 INFO:tasks.workunit.client.1.smithi121.stdout:4/243: fsync d5/d31/f23 0 2024-02-15T18:35:26.892 INFO:tasks.workunit.client.1.smithi121.stdout:5/336: unlink d0/d2/d1e/d21/l50 0 2024-02-15T18:35:26.892 INFO:tasks.workunit.client.1.smithi121.stdout:5/337: chown d0/l23 826012 1 2024-02-15T18:35:26.893 INFO:tasks.workunit.client.1.smithi121.stdout:4/244: read d5/d7/d14/f17 [760848,3058] 0 2024-02-15T18:35:26.894 INFO:tasks.workunit.client.1.smithi121.stdout:8/424: unlink d0/f5 0 2024-02-15T18:35:26.894 INFO:tasks.workunit.client.1.smithi121.stdout:8/425: readlink d0/d7/d36/d8c/d45/l5a 0 2024-02-15T18:35:26.895 INFO:tasks.workunit.client.1.smithi121.stdout:1/404: rmdir d0/d6/d7/d72/d82 0 2024-02-15T18:35:26.895 INFO:tasks.workunit.client.1.smithi121.stdout:6/355: rmdir d7/d4b/d60 39 2024-02-15T18:35:26.896 INFO:tasks.workunit.client.1.smithi121.stdout:5/338: mkdir d0/d6e 0 2024-02-15T18:35:26.897 INFO:tasks.workunit.client.1.smithi121.stdout:5/339: chown d0/l28 39 1 2024-02-15T18:35:26.897 INFO:tasks.workunit.client.1.smithi121.stdout:5/340: dread - d0/f60 zero size 2024-02-15T18:35:26.897 INFO:tasks.workunit.client.1.smithi121.stdout:8/426: mknod d0/d48/d86/c9b 0 2024-02-15T18:35:26.898 INFO:tasks.workunit.client.1.smithi121.stdout:6/356: mknod d7/dc/c7e 0 2024-02-15T18:35:26.900 INFO:tasks.workunit.client.1.smithi121.stdout:5/341: symlink d0/d2/d1e/d4e/l6f 0 2024-02-15T18:35:26.901 INFO:tasks.workunit.client.1.smithi121.stdout:8/427: creat d0/d61/d95/f9c x:0 0 0 2024-02-15T18:35:26.902 INFO:tasks.workunit.client.1.smithi121.stdout:8/428: read d0/d7/de/f65 [452446,60127] 0 2024-02-15T18:35:26.902 INFO:tasks.workunit.client.1.smithi121.stdout:8/429: chown d0/d48 53307930 1 2024-02-15T18:35:26.904 INFO:tasks.workunit.client.1.smithi121.stdout:8/430: mknod d0/d7/c9d 0 2024-02-15T18:35:26.906 INFO:tasks.workunit.client.1.smithi121.stdout:5/342: mkdir d0/d2/d70 0 2024-02-15T18:35:26.907 INFO:tasks.workunit.client.1.smithi121.stdout:5/343: truncate d0/d2/d1e/d4e/f55 754923 0 2024-02-15T18:35:26.907 INFO:tasks.workunit.client.1.smithi121.stdout:8/431: rename d0/d3/d20/f33 to d0/d7/d36/d8c/d45/f9e 0 2024-02-15T18:35:26.909 INFO:tasks.workunit.client.1.smithi121.stdout:4/245: dread d5/d31/d3d/f49 [0,4194304] 0 2024-02-15T18:35:26.911 INFO:tasks.workunit.client.1.smithi121.stdout:4/246: mkdir d5/d9/d19/d4c 0 2024-02-15T18:35:26.918 INFO:tasks.workunit.client.1.smithi121.stdout:8/432: dread d0/f13 [0,4194304] 0 2024-02-15T18:35:26.921 INFO:tasks.workunit.client.1.smithi121.stdout:0/442: dwrite d5/d30/d3c/d7d/d2a/d36/f8b [0,4194304] 0 2024-02-15T18:35:26.921 INFO:tasks.workunit.client.1.smithi121.stdout:1/405: dread d0/d6/d7/d29/f54 [0,4194304] 0 2024-02-15T18:35:26.921 INFO:tasks.workunit.client.1.smithi121.stdout:0/443: fdatasync d5/d30/d3c/d7d/dd/f32 0 2024-02-15T18:35:26.925 INFO:tasks.workunit.client.1.smithi121.stdout:1/406: rename d0/d6/d7/d50/d7e/c7f to d0/d6/d7/d9/d51/d5f/c89 0 2024-02-15T18:35:26.927 INFO:tasks.workunit.client.1.smithi121.stdout:1/407: creat d0/d6/d7/d9/d14/f8a x:0 0 0 2024-02-15T18:35:26.927 INFO:tasks.workunit.client.1.smithi121.stdout:8/433: dread d0/d7/d36/d8c/d45/f79 [0,4194304] 0 2024-02-15T18:35:26.927 INFO:tasks.workunit.client.1.smithi121.stdout:8/434: dread - d0/d7/de/d7c/f82 zero size 2024-02-15T18:35:26.928 INFO:tasks.workunit.client.1.smithi121.stdout:0/444: rename d5/d30/f6f to d5/d30/d3c/d7d/d2a/f95 0 2024-02-15T18:35:26.928 INFO:tasks.workunit.client.1.smithi121.stdout:0/445: fsync d5/d30/d3c/d7d/dd/f47 0 2024-02-15T18:35:26.929 INFO:tasks.workunit.client.1.smithi121.stdout:1/408: mknod d0/d6/d7/d50/c8b 0 2024-02-15T18:35:26.930 INFO:tasks.workunit.client.1.smithi121.stdout:8/435: unlink d0/d48/d86/c9b 0 2024-02-15T18:35:26.930 INFO:tasks.workunit.client.1.smithi121.stdout:8/436: stat d0/d7/de/c8d 0 2024-02-15T18:35:26.931 INFO:tasks.workunit.client.1.smithi121.stdout:6/357: dwrite d7/f28 [0,4194304] 0 2024-02-15T18:35:26.937 INFO:tasks.workunit.client.1.smithi121.stdout:0/446: truncate d5/d30/d3c/d7d/dd/f34 796623 0 2024-02-15T18:35:26.938 INFO:tasks.workunit.client.1.smithi121.stdout:1/409: rmdir d0/d39/d4b 39 2024-02-15T18:35:26.939 INFO:tasks.workunit.client.1.smithi121.stdout:1/410: fsync d0/d6/d7/d29/f54 0 2024-02-15T18:35:26.939 INFO:tasks.workunit.client.1.smithi121.stdout:8/437: dread d0/d2a/d51/f38 [4194304,4194304] 0 2024-02-15T18:35:26.941 INFO:tasks.workunit.client.1.smithi121.stdout:4/247: dwrite d5/d7/f38 [0,4194304] 0 2024-02-15T18:35:26.942 INFO:tasks.workunit.client.1.smithi121.stdout:4/248: write d5/d31/d3d/f49 [176951,76896] 0 2024-02-15T18:35:26.946 INFO:tasks.workunit.client.1.smithi121.stdout:6/358: link d7/dc/d34/d42/d6e/l78 d7/d4b/l7f 0 2024-02-15T18:35:26.946 INFO:tasks.workunit.client.1.smithi121.stdout:6/359: write d7/dc/f25 [1527589,7107] 0 2024-02-15T18:35:26.947 INFO:tasks.workunit.client.1.smithi121.stdout:0/447: mkdir d5/d30/d3c/d7d/d74/d96 0 2024-02-15T18:35:26.947 INFO:tasks.workunit.client.1.smithi121.stdout:0/448: fsync d5/d30/d3c/d7d/d2a/d73/f7a 0 2024-02-15T18:35:26.948 INFO:tasks.workunit.client.1.smithi121.stdout:0/449: write d5/d30/d3c/d7d/dd/f2d [96694,114743] 0 2024-02-15T18:35:26.954 INFO:tasks.workunit.client.1.smithi121.stdout:1/411: rename d0/d6/d7/d9/d14/d48/c81 to d0/d39/c8c 0 2024-02-15T18:35:26.956 INFO:tasks.workunit.client.1.smithi121.stdout:8/438: creat d0/d7/de/d7c/f9f x:0 0 0 2024-02-15T18:35:26.961 INFO:tasks.workunit.client.1.smithi121.stdout:6/360: creat d7/dc/d34/d42/f80 x:0 0 0 2024-02-15T18:35:26.962 INFO:tasks.workunit.client.1.smithi121.stdout:0/450: truncate d5/d30/d3c/d7d/d2a/d4e/f5b 1428151 0 2024-02-15T18:35:26.964 INFO:tasks.workunit.client.1.smithi121.stdout:6/361: symlink d7/d4b/l81 0 2024-02-15T18:35:26.964 INFO:tasks.workunit.client.1.smithi121.stdout:6/362: chown d7/d4b/c7b 16 1 2024-02-15T18:35:26.964 INFO:tasks.workunit.client.1.smithi121.stdout:4/249: dwrite d5/d31/f23 [0,4194304] 0 2024-02-15T18:35:26.965 INFO:tasks.workunit.client.1.smithi121.stdout:4/250: chown d5/d9/d1d/f24 257890230 1 2024-02-15T18:35:26.965 INFO:tasks.workunit.client.1.smithi121.stdout:4/251: dread - d5/f30 zero size 2024-02-15T18:35:26.965 INFO:tasks.workunit.client.1.smithi121.stdout:0/451: write d5/d30/d3c/d7d/dd/f25 [1716418,40957] 0 2024-02-15T18:35:26.971 INFO:tasks.workunit.client.1.smithi121.stdout:8/439: dread d0/d3/d20/f78 [0,4194304] 0 2024-02-15T18:35:26.972 INFO:tasks.workunit.client.1.smithi121.stdout:4/252: mkdir d5/d9/d19/d2a/d4d 0 2024-02-15T18:35:26.973 INFO:tasks.workunit.client.1.smithi121.stdout:0/452: dread d5/d30/d3c/d7d/dd/f68 [0,4194304] 0 2024-02-15T18:35:26.975 INFO:tasks.workunit.client.1.smithi121.stdout:0/453: truncate d5/d30/d3c/d7d/dd/f16 3359852 0 2024-02-15T18:35:26.980 INFO:tasks.workunit.client.1.smithi121.stdout:0/454: mkdir d5/d30/d3c/d7d/d2a/d73/d97 0 2024-02-15T18:35:26.980 INFO:tasks.workunit.client.1.smithi121.stdout:0/455: creat d5/d30/d3c/d7d/d2a/d73/d97/f98 x:0 0 0 2024-02-15T18:35:26.981 INFO:tasks.workunit.client.1.smithi121.stdout:0/456: mknod d5/d30/d3c/d7d/c99 0 2024-02-15T18:35:26.988 INFO:tasks.workunit.client.1.smithi121.stdout:4/253: dread d5/fa [4194304,4194304] 0 2024-02-15T18:35:26.988 INFO:tasks.workunit.client.1.smithi121.stdout:6/363: dwrite d7/d4b/d59/f72 [0,4194304] 0 2024-02-15T18:35:26.991 INFO:tasks.workunit.client.1.smithi121.stdout:1/412: dwrite d0/fb [0,4194304] 0 2024-02-15T18:35:26.998 INFO:tasks.workunit.client.1.smithi121.stdout:0/457: dwrite d5/d30/d3c/d7d/d2a/d73/d97/f98 [0,4194304] 0 2024-02-15T18:35:26.999 INFO:tasks.workunit.client.1.smithi121.stdout:4/254: truncate d5/d9/d1d/f41 3635892 0 2024-02-15T18:35:26.999 INFO:tasks.workunit.client.1.smithi121.stdout:4/255: chown d5/d9/d19/d2a/f2c 3 1 2024-02-15T18:35:26.999 INFO:tasks.workunit.client.1.smithi121.stdout:4/256: fdatasync d5/d7/d1e/f48 0 2024-02-15T18:35:26.999 INFO:tasks.workunit.client.1.smithi121.stdout:4/257: write d5/d9/d19/d28/f21 [1635823,21360] 0 2024-02-15T18:35:27.004 INFO:tasks.workunit.client.1.smithi121.stdout:0/458: creat d5/d30/d3c/d7d/d2a/d73/f9a x:0 0 0 2024-02-15T18:35:27.006 INFO:tasks.workunit.client.1.smithi121.stdout:4/258: link d5/d31/d25/f33 d5/d9/d19/f4e 0 2024-02-15T18:35:27.010 INFO:tasks.workunit.client.1.smithi121.stdout:0/459: mkdir d5/d9b 0 2024-02-15T18:35:27.010 INFO:tasks.workunit.client.1.smithi121.stdout:4/259: mkdir d5/d31/d3d/d4f 0 2024-02-15T18:35:27.012 INFO:tasks.workunit.client.1.smithi121.stdout:4/260: mknod d5/d9/d19/d4c/c50 0 2024-02-15T18:35:27.013 INFO:tasks.workunit.client.1.smithi121.stdout:0/460: dread d5/f3b [0,4194304] 0 2024-02-15T18:35:27.013 INFO:tasks.workunit.client.1.smithi121.stdout:0/461: chown d5/d30/d3c/d7d/dd/f47 0 1 2024-02-15T18:35:27.013 INFO:tasks.workunit.client.1.smithi121.stdout:0/462: write d5/d30/d3c/d7d/d2a/d4e/f5a [5075504,58838] 0 2024-02-15T18:35:27.015 INFO:tasks.workunit.client.1.smithi121.stdout:4/261: rename d5/d31/d3e to d5/d31/d3d/d4f/d51 0 2024-02-15T18:35:27.025 INFO:tasks.workunit.client.1.smithi121.stdout:6/364: dwrite d7/dc/f15 [0,4194304] 0 2024-02-15T18:35:27.025 INFO:tasks.workunit.client.1.smithi121.stdout:4/262: dread f1 [0,4194304] 0 2024-02-15T18:35:27.028 INFO:tasks.workunit.client.1.smithi121.stdout:1/413: dwrite d0/d6/d7/d9/d14/f83 [0,4194304] 0 2024-02-15T18:35:27.031 INFO:tasks.workunit.client.1.smithi121.stdout:1/414: symlink d0/d6/d7/l8d 0 2024-02-15T18:35:27.035 INFO:tasks.workunit.client.1.smithi121.stdout:1/415: dread d0/f3d [0,4194304] 0 2024-02-15T18:35:27.041 INFO:tasks.workunit.client.1.smithi121.stdout:0/463: dwrite d5/d30/d3c/d7d/f56 [0,4194304] 0 2024-02-15T18:35:27.041 INFO:tasks.workunit.client.1.smithi121.stdout:0/464: fdatasync d5/d30/d3c/d7d/d2a/d36/f8b 0 2024-02-15T18:35:27.041 INFO:tasks.workunit.client.1.smithi121.stdout:0/465: stat d5/d30/d3c/d7d/d2a/f95 0 2024-02-15T18:35:27.041 INFO:tasks.workunit.client.1.smithi121.stdout:0/466: chown d5/d30/d3c/d7d/d2a/c7b 513062 1 2024-02-15T18:35:27.042 INFO:tasks.workunit.client.1.smithi121.stdout:4/263: dwrite d5/d9/d1d/f41 [4194304,4194304] 0 2024-02-15T18:35:27.042 INFO:tasks.workunit.client.1.smithi121.stdout:4/264: fsync d5/d7/f3c 0 2024-02-15T18:35:27.044 INFO:tasks.workunit.client.1.smithi121.stdout:4/265: symlink d5/d31/d25/l52 0 2024-02-15T18:35:27.044 INFO:tasks.workunit.client.1.smithi121.stdout:4/266: chown d5/d7/c4b 1862 1 2024-02-15T18:35:27.045 INFO:tasks.workunit.client.1.smithi121.stdout:4/267: creat d5/d7/d1e/f53 x:0 0 0 2024-02-15T18:35:27.046 INFO:tasks.workunit.client.1.smithi121.stdout:4/268: creat d5/d9/d19/d2a/d4d/f54 x:0 0 0 2024-02-15T18:35:27.048 INFO:tasks.workunit.client.1.smithi121.stdout:6/365: dwrite d7/dc/d34/f43 [0,4194304] 0 2024-02-15T18:35:27.048 INFO:tasks.workunit.client.1.smithi121.stdout:6/366: readlink d7/dc/d34/d42/l61 0 2024-02-15T18:35:27.051 INFO:tasks.workunit.client.1.smithi121.stdout:0/467: dread f2 [0,4194304] 0 2024-02-15T18:35:27.051 INFO:tasks.workunit.client.1.smithi121.stdout:4/269: creat d5/d7/d14/f55 x:0 0 0 2024-02-15T18:35:27.053 INFO:tasks.workunit.client.1.smithi121.stdout:8/440: unlink d0/d7/d36/d8c/d45/l6c 0 2024-02-15T18:35:27.053 INFO:tasks.workunit.client.1.smithi121.stdout:8/441: fsync d0/d3/d20/f8b 0 2024-02-15T18:35:27.053 INFO:tasks.workunit.client.1.smithi121.stdout:8/442: write d0/d7/d36/d8c/f4f [1013373,5843] 0 2024-02-15T18:35:27.053 INFO:tasks.workunit.client.1.smithi121.stdout:8/443: chown d0/d4a/f99 91 1 2024-02-15T18:35:27.053 INFO:tasks.workunit.client.1.smithi121.stdout:1/416: dwrite d0/d6/d7/d9/f7b [0,4194304] 0 2024-02-15T18:35:27.054 INFO:tasks.workunit.client.1.smithi121.stdout:6/367: mknod d7/dc/d76/c82 0 2024-02-15T18:35:27.054 INFO:tasks.workunit.client.1.smithi121.stdout:0/468: rmdir d5/d30/d3c/d7d/d2a/d36/d44 39 2024-02-15T18:35:27.054 INFO:tasks.workunit.client.1.smithi121.stdout:0/469: rename d5 to d5/d30/d3c/d7d/d2a/d73/d97/d9c 22 2024-02-15T18:35:27.054 INFO:tasks.workunit.client.1.smithi121.stdout:0/470: fdatasync d5/d30/d3c/d7d/dd/f2c 0 2024-02-15T18:35:27.054 INFO:tasks.workunit.client.1.smithi121.stdout:0/471: truncate d5/f6a 303074 0 2024-02-15T18:35:27.056 INFO:tasks.workunit.client.1.smithi121.stdout:4/270: creat d5/d9/d19/d28/f56 x:0 0 0 2024-02-15T18:35:27.070 INFO:tasks.workunit.client.1.smithi121.stdout:1/417: dwrite d0/d39/d4b/f6d [4194304,4194304] 0 2024-02-15T18:35:27.077 INFO:tasks.workunit.client.1.smithi121.stdout:6/368: dwrite d7/dc/f4c [0,4194304] 0 2024-02-15T18:35:27.077 INFO:tasks.workunit.client.1.smithi121.stdout:6/369: stat d7/dc/d2c/c65 0 2024-02-15T18:35:27.078 INFO:tasks.workunit.client.1.smithi121.stdout:6/370: getdents d7/d4b/d60/d73 0 2024-02-15T18:35:27.078 INFO:tasks.workunit.client.1.smithi121.stdout:6/371: readlink d7/dc/d2c/l5a 0 2024-02-15T18:35:27.113 INFO:tasks.workunit.client.1.smithi121.stdout:3/338: sync 2024-02-15T18:35:27.113 INFO:tasks.workunit.client.1.smithi121.stdout:2/245: sync 2024-02-15T18:35:27.113 INFO:tasks.workunit.client.1.smithi121.stdout:7/273: sync 2024-02-15T18:35:27.113 INFO:tasks.workunit.client.1.smithi121.stdout:9/328: sync 2024-02-15T18:35:27.114 INFO:tasks.workunit.client.1.smithi121.stdout:8/444: getdents d0/d4a 0 2024-02-15T18:35:27.114 INFO:tasks.workunit.client.1.smithi121.stdout:8/445: chown d0/d61/d7b/f7f 15 1 2024-02-15T18:35:27.114 INFO:tasks.workunit.client.1.smithi121.stdout:7/274: read d0/f29 [2866509,100535] 0 2024-02-15T18:35:27.129 INFO:tasks.workunit.client.1.smithi121.stdout:2/246: dwrite d5/f8 [0,4194304] 0 2024-02-15T18:35:27.150 INFO:tasks.workunit.client.1.smithi121.stdout:2/247: dwrite d5/df/d10/d15/d22/f37 [0,4194304] 0 2024-02-15T18:35:27.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:26.592681+0000 mgr.smithi121.ckiryi (mgr.24385) 5 : cluster [DBG] pgmap v7: 65 pgs: 65 active+clean; 898 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 59 MiB/s rd, 119 MiB/s wr, 281 op/s 2024-02-15T18:35:27.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:26.592681+0000 mgr.smithi121.ckiryi (mgr.24385) 5 : cluster [DBG] pgmap v7: 65 pgs: 65 active+clean; 898 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 59 MiB/s rd, 119 MiB/s wr, 281 op/s 2024-02-15T18:35:28.388 INFO:tasks.workunit.client.1.smithi121.stdout:4/271: mkdir d5/d9/d19/d57 0 2024-02-15T18:35:28.394 INFO:tasks.workunit.client.1.smithi121.stdout:1/418: rmdir d0/d6/d7/d72 39 2024-02-15T18:35:28.397 INFO:tasks.workunit.client.1.smithi121.stdout:6/372: creat d7/dc/d34/d42/d62/f83 x:0 0 0 2024-02-15T18:35:28.397 INFO:tasks.workunit.client.1.smithi121.stdout:6/373: fsync d7/d4b/f53 0 2024-02-15T18:35:28.398 INFO:tasks.workunit.client.1.smithi121.stdout:6/374: write d7/dc/d34/f26 [7376649,30061] 0 2024-02-15T18:35:28.398 INFO:tasks.workunit.client.1.smithi121.stdout:6/375: chown d7/dc/d34/c33 6 1 2024-02-15T18:35:28.399 INFO:tasks.workunit.client.1.smithi121.stdout:3/339: creat d2/d1a/d27/d2c/d25/d31/f73 x:0 0 0 2024-02-15T18:35:29.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:28.593065+0000 mgr.smithi121.ckiryi (mgr.24385) 6 : cluster [DBG] pgmap v8: 65 pgs: 65 active+clean; 898 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 46 MiB/s rd, 92 MiB/s wr, 219 op/s 2024-02-15T18:35:29.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:28 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:28.593065+0000 mgr.smithi121.ckiryi (mgr.24385) 6 : cluster [DBG] pgmap v8: 65 pgs: 65 active+clean; 898 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 46 MiB/s rd, 92 MiB/s wr, 219 op/s 2024-02-15T18:35:29.539 INFO:tasks.workunit.client.1.smithi121.stdout:9/329: creat d7/d8/d40/f72 x:0 0 0 2024-02-15T18:35:29.539 INFO:tasks.workunit.client.1.smithi121.stdout:9/330: chown d7/d8/d40/f70 6 1 2024-02-15T18:35:29.540 INFO:tasks.workunit.client.1.smithi121.stdout:8/446: creat d0/d7/d36/d8c/d45/d53/fa0 x:0 0 0 2024-02-15T18:35:29.542 INFO:tasks.workunit.client.1.smithi121.stdout:7/275: symlink d0/d38/d46/l5a 0 2024-02-15T18:35:29.543 INFO:tasks.workunit.client.1.smithi121.stdout:2/248: creat d5/df/f45 x:0 0 0 2024-02-15T18:35:29.544 INFO:tasks.workunit.client.1.smithi121.stdout:2/249: stat d5/df/d27/d3e/d35 0 2024-02-15T18:35:29.547 INFO:tasks.workunit.client.1.smithi121.stdout:9/331: dread d7/f4e [0,4194304] 0 2024-02-15T18:35:29.550 INFO:tasks.workunit.client.1.smithi121.stdout:9/332: dread d7/d8/d25/d30/f4c [0,4194304] 0 2024-02-15T18:35:29.550 INFO:tasks.workunit.client.1.smithi121.stdout:9/333: write d7/d3a/d42/f6a [257648,43754] 0 2024-02-15T18:35:29.567 INFO:tasks.workunit.client.1.smithi121.stdout:6/376: symlink d7/dc/d34/d42/d62/l84 0 2024-02-15T18:35:29.567 INFO:tasks.workunit.client.1.smithi121.stdout:3/340: truncate d2/d1a/d27/d2c/f56 989698 0 2024-02-15T18:35:29.568 INFO:tasks.workunit.client.1.smithi121.stdout:8/447: symlink d0/la1 0 2024-02-15T18:35:29.570 INFO:tasks.workunit.client.1.smithi121.stdout:7/276: mkdir d0/d4a/d5b 0 2024-02-15T18:35:29.570 INFO:tasks.workunit.client.1.smithi121.stdout:7/277: write d0/d5/d11/d1c/f1f [1846984,75692] 0 2024-02-15T18:35:29.570 INFO:tasks.workunit.client.1.smithi121.stdout:9/334: mkdir d7/d8/d18/d73 0 2024-02-15T18:35:29.571 INFO:tasks.workunit.client.1.smithi121.stdout:6/377: creat d7/dc/d34/d7c/f85 x:0 0 0 2024-02-15T18:35:29.571 INFO:tasks.workunit.client.1.smithi121.stdout:6/378: chown d7/dc/d55/l68 2269 1 2024-02-15T18:35:29.573 INFO:tasks.workunit.client.1.smithi121.stdout:8/448: rename d0/d4a/f5f to d0/d3/d20/fa2 0 2024-02-15T18:35:29.573 INFO:tasks.workunit.client.1.smithi121.stdout:8/449: stat d0/d48/f6a 0 2024-02-15T18:35:29.574 INFO:tasks.workunit.client.1.smithi121.stdout:8/450: read d0/d3/f64 [735426,47239] 0 2024-02-15T18:35:29.574 INFO:tasks.workunit.client.1.smithi121.stdout:8/451: truncate d0/d7/d36/d8c/d45/f3e 375020 0 2024-02-15T18:35:29.575 INFO:tasks.workunit.client.1.smithi121.stdout:6/379: creat d7/dc/d34/f86 x:0 0 0 2024-02-15T18:35:29.577 INFO:tasks.workunit.client.1.smithi121.stdout:3/341: dread d2/fe [0,4194304] 0 2024-02-15T18:35:29.578 INFO:tasks.workunit.client.1.smithi121.stdout:8/452: creat d0/d2a/fa3 x:0 0 0 2024-02-15T18:35:29.578 INFO:tasks.workunit.client.1.smithi121.stdout:8/453: stat d0/d2a/d51/l30 0 2024-02-15T18:35:29.580 INFO:tasks.workunit.client.1.smithi121.stdout:6/380: rename d7/dc/d34/d42/d62/d63/d66/c70 to d7/dc/d34/d42/d6e/c87 0 2024-02-15T18:35:29.582 INFO:tasks.workunit.client.1.smithi121.stdout:8/454: creat d0/d7/fa4 x:0 0 0 2024-02-15T18:35:29.584 INFO:tasks.workunit.client.1.smithi121.stdout:3/342: rename d2/d1a/c48 to d2/d1a/d27/d2c/d25/d31/c74 0 2024-02-15T18:35:29.584 INFO:tasks.workunit.client.1.smithi121.stdout:3/343: dread - d2/d1a/d54/f59 zero size 2024-02-15T18:35:29.586 INFO:tasks.workunit.client.1.smithi121.stdout:6/381: creat d7/dc/d34/d42/d62/d63/d6f/f88 x:0 0 0 2024-02-15T18:35:29.586 INFO:tasks.workunit.client.1.smithi121.stdout:6/382: chown d7/f24 0 1 2024-02-15T18:35:29.586 INFO:tasks.workunit.client.1.smithi121.stdout:6/383: chown d7/d4b/d59/f72 2 1 2024-02-15T18:35:29.587 INFO:tasks.workunit.client.1.smithi121.stdout:8/455: mkdir d0/d7/da5 0 2024-02-15T18:35:29.587 INFO:tasks.workunit.client.1.smithi121.stdout:9/335: dread d7/f1c [0,4194304] 0 2024-02-15T18:35:29.588 INFO:tasks.workunit.client.1.smithi121.stdout:3/344: getdents d2/d1a/d22 0 2024-02-15T18:35:29.588 INFO:tasks.workunit.client.1.smithi121.stdout:3/345: readlink d2/d1a/l55 0 2024-02-15T18:35:29.589 INFO:tasks.workunit.client.1.smithi121.stdout:6/384: getdents d7/dc/d76 0 2024-02-15T18:35:29.592 INFO:tasks.workunit.client.1.smithi121.stdout:9/336: creat d7/d8/d36/f74 x:0 0 0 2024-02-15T18:35:29.593 INFO:tasks.workunit.client.1.smithi121.stdout:7/278: dwrite d0/d5/d11/d1c/f35 [0,4194304] 0 2024-02-15T18:35:29.594 INFO:tasks.workunit.client.1.smithi121.stdout:3/346: creat d2/d1a/d6b/f75 x:0 0 0 2024-02-15T18:35:29.597 INFO:tasks.workunit.client.1.smithi121.stdout:9/337: dread d7/d8/f16 [0,4194304] 0 2024-02-15T18:35:29.598 INFO:tasks.workunit.client.1.smithi121.stdout:8/456: rename d0/d4a to d0/d61/da6 0 2024-02-15T18:35:29.598 INFO:tasks.workunit.client.1.smithi121.stdout:8/457: fdatasync d0/d7/d36/d8c/d45/f3e 0 2024-02-15T18:35:29.598 INFO:tasks.workunit.client.1.smithi121.stdout:8/458: stat d0/d7/d36/d52/d7a 0 2024-02-15T18:35:29.598 INFO:tasks.workunit.client.1.smithi121.stdout:9/338: rename d7/d8/d36 to d7/d8/d36/d75 22 2024-02-15T18:35:29.598 INFO:tasks.workunit.client.1.smithi121.stdout:9/339: fsync d7/d8/d18/f45 0 2024-02-15T18:35:29.605 INFO:tasks.workunit.client.1.smithi121.stdout:7/279: mkdir d0/d5/d11/d5c 0 2024-02-15T18:35:29.605 INFO:tasks.workunit.client.1.smithi121.stdout:7/280: truncate d0/d38/d3c/f4d 2730232 0 2024-02-15T18:35:29.606 INFO:tasks.workunit.client.1.smithi121.stdout:6/385: dwrite d7/dc/f21 [0,4194304] 0 2024-02-15T18:35:29.606 INFO:tasks.workunit.client.1.smithi121.stdout:8/459: getdents d0/d3/d20 0 2024-02-15T18:35:29.607 INFO:tasks.workunit.client.1.smithi121.stdout:6/386: read d7/dc/f25 [126703,24286] 0 2024-02-15T18:35:29.608 INFO:tasks.workunit.client.1.smithi121.stdout:9/340: symlink d7/d3a/d42/l76 0 2024-02-15T18:35:29.609 INFO:tasks.workunit.client.1.smithi121.stdout:9/341: dread - d7/d8/d40/f72 zero size 2024-02-15T18:35:29.609 INFO:tasks.workunit.client.1.smithi121.stdout:7/281: rename d0/d5/d11/d1c/f28 to d0/d33/f5d 0 2024-02-15T18:35:29.611 INFO:tasks.workunit.client.1.smithi121.stdout:8/460: mkdir d0/d7/da7 0 2024-02-15T18:35:29.612 INFO:tasks.workunit.client.1.smithi121.stdout:6/387: mkdir d7/dc/d34/d42/d62/d63/d6f/d89 0 2024-02-15T18:35:29.614 INFO:tasks.workunit.client.1.smithi121.stdout:9/342: mkdir d7/d26/d4f/d5b/d77 0 2024-02-15T18:35:29.614 INFO:tasks.workunit.client.1.smithi121.stdout:9/343: rename d7/d8/d25/d30/d33/d3e to d7/d8/d25/d30/d33/d3e/d78 22 2024-02-15T18:35:29.614 INFO:tasks.workunit.client.1.smithi121.stdout:9/344: dread - d7/d8/d25/d30/d33/d3c/d3f/f6d zero size 2024-02-15T18:35:29.615 INFO:tasks.workunit.client.1.smithi121.stdout:7/282: link d0/f18 d0/d5/f5e 0 2024-02-15T18:35:29.616 INFO:tasks.workunit.client.1.smithi121.stdout:9/345: write d7/d8/f10 [2150979,83011] 0 2024-02-15T18:35:29.616 INFO:tasks.workunit.client.1.smithi121.stdout:9/346: write d7/f4e [4608350,61991] 0 2024-02-15T18:35:29.620 INFO:tasks.workunit.client.1.smithi121.stdout:3/347: dwrite d2/d1a/d27/d2c/d25/d31/f57 [0,4194304] 0 2024-02-15T18:35:29.622 INFO:tasks.workunit.client.1.smithi121.stdout:6/388: creat d7/d4b/d60/d6c/f8a x:0 0 0 2024-02-15T18:35:29.623 INFO:tasks.workunit.client.1.smithi121.stdout:8/461: dread d0/d61/d66/f76 [0,4194304] 0 2024-02-15T18:35:29.626 INFO:tasks.workunit.client.1.smithi121.stdout:3/348: creat d2/d1a/d6b/d72/f76 x:0 0 0 2024-02-15T18:35:29.626 INFO:tasks.workunit.client.1.smithi121.stdout:3/349: chown d2/f9 11493270 1 2024-02-15T18:35:29.627 INFO:tasks.workunit.client.1.smithi121.stdout:6/389: symlink d7/l8b 0 2024-02-15T18:35:29.631 INFO:tasks.workunit.client.1.smithi121.stdout:9/347: dread d7/f1b [0,4194304] 0 2024-02-15T18:35:29.652 INFO:tasks.workunit.client.1.smithi121.stdout:6/390: dwrite d7/f24 [8388608,4194304] 0 2024-02-15T18:35:29.657 INFO:tasks.workunit.client.1.smithi121.stdout:8/462: rmdir d0/d48/d86 39 2024-02-15T18:35:29.660 INFO:tasks.workunit.client.1.smithi121.stdout:7/283: dwrite d0/d38/d3c/f4d [0,4194304] 0 2024-02-15T18:35:29.660 INFO:tasks.workunit.client.1.smithi121.stdout:0/472: sync 2024-02-15T18:35:29.660 INFO:tasks.workunit.client.1.smithi121.stdout:0/473: write d5/d30/d3c/d7d/dd/f47 [5216483,64766] 0 2024-02-15T18:35:29.661 INFO:tasks.workunit.client.1.smithi121.stdout:5/344: sync 2024-02-15T18:35:29.662 INFO:tasks.workunit.client.1.smithi121.stdout:7/284: dread d0/fb [0,4194304] 0 2024-02-15T18:35:29.663 INFO:tasks.workunit.client.1.smithi121.stdout:9/348: mkdir d7/d8/d25/d30/d33/d79 0 2024-02-15T18:35:29.663 INFO:tasks.workunit.client.1.smithi121.stdout:9/349: read - d7/d8/d25/d30/f64 zero size 2024-02-15T18:35:29.665 INFO:tasks.workunit.client.1.smithi121.stdout:6/391: getdents d7/dc/d34/d42 0 2024-02-15T18:35:29.667 INFO:tasks.workunit.client.1.smithi121.stdout:8/463: truncate d0/d7/d36/f81 4029038 0 2024-02-15T18:35:29.667 INFO:tasks.workunit.client.1.smithi121.stdout:8/464: write d0/d61/d8f/f92 [455538,42463] 0 2024-02-15T18:35:29.668 INFO:tasks.workunit.client.1.smithi121.stdout:3/350: dwrite d2/d1a/d27/d2c/d25/d30/f46 [0,4194304] 0 2024-02-15T18:35:29.671 INFO:tasks.workunit.client.1.smithi121.stdout:0/474: rmdir d5/d30/d3c 39 2024-02-15T18:35:29.671 INFO:tasks.workunit.client.1.smithi121.stdout:0/475: write d5/d30/d3c/d7d/dd/f68 [4843025,2255] 0 2024-02-15T18:35:29.672 INFO:tasks.workunit.client.1.smithi121.stdout:5/345: link d0/d2/l3d d0/d2/d1e/l71 0 2024-02-15T18:35:29.673 INFO:tasks.workunit.client.1.smithi121.stdout:7/285: mkdir d0/d5/d11/d1c/d5f 0 2024-02-15T18:35:29.676 INFO:tasks.workunit.client.1.smithi121.stdout:9/350: creat d7/d8/d40/d4b/f7a x:0 0 0 2024-02-15T18:35:29.677 INFO:tasks.workunit.client.1.smithi121.stdout:0/476: creat d5/d6b/f9d x:0 0 0 2024-02-15T18:35:29.677 INFO:tasks.workunit.client.1.smithi121.stdout:0/477: stat d5/d30/d3c/d7d/c80 0 2024-02-15T18:35:29.678 INFO:tasks.workunit.client.1.smithi121.stdout:7/286: mknod d0/d38/d46/c60 0 2024-02-15T18:35:29.680 INFO:tasks.workunit.client.1.smithi121.stdout:7/287: dread d0/fb [0,4194304] 0 2024-02-15T18:35:29.680 INFO:tasks.workunit.client.1.smithi121.stdout:0/478: link d5/d30/d3c/d7d/d74/c94 d5/d30/d3c/d7d/d74/d96/c9e 0 2024-02-15T18:35:29.682 INFO:tasks.workunit.client.1.smithi121.stdout:7/288: mkdir d0/d38/d61 0 2024-02-15T18:35:29.682 INFO:tasks.workunit.client.1.smithi121.stdout:7/289: dread - d0/d4a/f53 zero size 2024-02-15T18:35:29.682 INFO:tasks.workunit.client.1.smithi121.stdout:0/479: rmdir d5/d30/d3c/d7d/d2a/d4e 39 2024-02-15T18:35:29.683 INFO:tasks.workunit.client.1.smithi121.stdout:5/346: dread d0/d2/f14 [0,4194304] 0 2024-02-15T18:35:29.683 INFO:tasks.workunit.client.1.smithi121.stdout:5/347: read - d0/f4b zero size 2024-02-15T18:35:29.683 INFO:tasks.workunit.client.1.smithi121.stdout:7/290: mknod d0/d5/d4c/c62 0 2024-02-15T18:35:29.685 INFO:tasks.workunit.client.1.smithi121.stdout:0/480: link d5/c38 d5/d30/d3c/d7d/d74/d96/c9f 0 2024-02-15T18:35:29.686 INFO:tasks.workunit.client.1.smithi121.stdout:0/481: getdents d5/d30/d3c/d7d/d2a/d73/d97 0 2024-02-15T18:35:29.688 INFO:tasks.workunit.client.1.smithi121.stdout:5/348: mknod d0/d2/d1e/d21/d47/c72 0 2024-02-15T18:35:29.689 INFO:tasks.workunit.client.1.smithi121.stdout:0/482: mkdir d5/d6b/da0 0 2024-02-15T18:35:29.689 INFO:tasks.workunit.client.1.smithi121.stdout:0/483: chown d5/l21 7796371 1 2024-02-15T18:35:29.694 INFO:tasks.workunit.client.1.smithi121.stdout:5/349: creat d0/d2/d1e/d21/d40/d62/f73 x:0 0 0 2024-02-15T18:35:29.701 INFO:tasks.workunit.client.1.smithi121.stdout:6/392: dwrite d7/dc/d34/d42/d62/f74 [0,4194304] 0 2024-02-15T18:35:29.703 INFO:tasks.workunit.client.1.smithi121.stdout:6/393: mknod d7/dc/d34/d42/c8c 0 2024-02-15T18:35:29.706 INFO:tasks.workunit.client.1.smithi121.stdout:6/394: mkdir d7/dc/d34/d42/d62/d63/d8d 0 2024-02-15T18:35:29.710 INFO:tasks.workunit.client.1.smithi121.stdout:3/351: dwrite d2/d1a/d6b/d72/f76 [0,4194304] 0 2024-02-15T18:35:29.711 INFO:tasks.workunit.client.1.smithi121.stdout:3/352: read - d2/d1a/d27/d2c/d25/d31/f4e zero size 2024-02-15T18:35:29.727 INFO:tasks.workunit.client.1.smithi121.stdout:0/484: dwrite d5/d30/d3c/d7d/d2a/d73/f9a [0,4194304] 0 2024-02-15T18:35:29.729 INFO:tasks.workunit.client.1.smithi121.stdout:7/291: dwrite d0/d5/f15 [4194304,4194304] 0 2024-02-15T18:35:29.729 INFO:tasks.workunit.client.1.smithi121.stdout:5/350: dwrite d0/d2/d18/f29 [0,4194304] 0 2024-02-15T18:35:29.730 INFO:tasks.workunit.client.1.smithi121.stdout:6/395: dwrite d7/dc/d34/d42/d62/d63/f7a [0,4194304] 0 2024-02-15T18:35:29.730 INFO:tasks.workunit.client.1.smithi121.stdout:8/465: dwrite d0/d7/d36/d8c/d45/f9e [0,4194304] 0 2024-02-15T18:35:29.730 INFO:tasks.workunit.client.1.smithi121.stdout:8/466: stat d0/d7/d36/d8c 0 2024-02-15T18:35:29.730 INFO:tasks.workunit.client.1.smithi121.stdout:0/485: write d5/d30/d3c/d7d/d2a/d36/f3d [1978629,40315] 0 2024-02-15T18:35:29.730 INFO:tasks.workunit.client.1.smithi121.stdout:0/486: write d5/d30/d3c/d7d/d2a/f95 [316738,13941] 0 2024-02-15T18:35:29.738 INFO:tasks.workunit.client.1.smithi121.stdout:7/292: dread d0/d5/d11/d1c/f35 [0,4194304] 0 2024-02-15T18:35:29.743 INFO:tasks.workunit.client.1.smithi121.stdout:5/351: creat d0/d2/d1e/d21/d40/d62/f74 x:0 0 0 2024-02-15T18:35:29.744 INFO:tasks.workunit.client.1.smithi121.stdout:6/396: mkdir d7/dc/d8e 0 2024-02-15T18:35:29.747 INFO:tasks.workunit.client.1.smithi121.stdout:8/467: link d0/d61/d8f/f92 d0/d73/fa8 0 2024-02-15T18:35:29.750 INFO:tasks.workunit.client.1.smithi121.stdout:0/487: mknod d5/d30/d3c/d7d/d2a/d36/d44/d71/ca1 0 2024-02-15T18:35:29.750 INFO:tasks.workunit.client.1.smithi121.stdout:0/488: write d5/d30/d3c/f8f [417594,110833] 0 2024-02-15T18:35:29.750 INFO:tasks.workunit.client.1.smithi121.stdout:0/489: fsync d5/d30/d3c/d7d/dd/f17 0 2024-02-15T18:35:29.750 INFO:tasks.workunit.client.1.smithi121.stdout:5/352: fsync d0/d2/d1e/d21/d39/f53 0 2024-02-15T18:35:29.753 INFO:tasks.workunit.client.1.smithi121.stdout:6/397: creat d7/d4b/f8f x:0 0 0 2024-02-15T18:35:29.753 INFO:tasks.workunit.client.1.smithi121.stdout:6/398: fsync d7/f31 0 2024-02-15T18:35:29.755 INFO:tasks.workunit.client.1.smithi121.stdout:5/353: link d0/d2/d1e/d21/d40/d62/c69 d0/d2/d1e/d21/d33/d4c/c75 0 2024-02-15T18:35:29.756 INFO:tasks.workunit.client.1.smithi121.stdout:6/399: creat d7/dc/f90 x:0 0 0 2024-02-15T18:35:29.757 INFO:tasks.workunit.client.1.smithi121.stdout:0/490: fdatasync d5/d30/d3c/d7d/dd/f47 0 2024-02-15T18:35:29.757 INFO:tasks.workunit.client.1.smithi121.stdout:0/491: dread - d5/d30/d3c/d7d/f43 zero size 2024-02-15T18:35:29.757 INFO:tasks.workunit.client.1.smithi121.stdout:5/354: rename d0/c13 to d0/d2/c76 0 2024-02-15T18:35:29.760 INFO:tasks.workunit.client.1.smithi121.stdout:0/492: rename d5/d30/d3c/d7d/le to d5/d30/d3c/d7d/d2a/d73/d97/la2 0 2024-02-15T18:35:29.762 INFO:tasks.workunit.client.1.smithi121.stdout:5/355: link d0/d2/d1e/d21/d39/c3c d0/d2/d70/c77 0 2024-02-15T18:35:29.763 INFO:tasks.workunit.client.1.smithi121.stdout:5/356: link d0/d16/f68 d0/d2/d1e/d21/d40/f78 0 2024-02-15T18:35:29.764 INFO:tasks.workunit.client.1.smithi121.stdout:5/357: fsync d0/d16/d34/f6a 0 2024-02-15T18:35:29.764 INFO:tasks.workunit.client.1.smithi121.stdout:5/358: write d0/f60 [228056,122911] 0 2024-02-15T18:35:29.764 INFO:tasks.workunit.client.1.smithi121.stdout:0/493: write d5/d30/d3c/d7d/dd/f93 [2364982,93316] 0 2024-02-15T18:35:29.764 INFO:tasks.workunit.client.1.smithi121.stdout:0/494: truncate d5/d30/d3c/d7d/d2a/d36/f8b 5216473 0 2024-02-15T18:35:29.765 INFO:tasks.workunit.client.1.smithi121.stdout:0/495: symlink d5/d30/d3c/d7d/d74/la3 0 2024-02-15T18:35:29.767 INFO:tasks.workunit.client.1.smithi121.stdout:0/496: creat d5/fa4 x:0 0 0 2024-02-15T18:35:29.767 INFO:tasks.workunit.client.1.smithi121.stdout:8/468: dwrite d0/d2a/d51/f38 [0,4194304] 0 2024-02-15T18:35:29.769 INFO:tasks.workunit.client.1.smithi121.stdout:0/497: write d5/d30/d3c/d7d/f24 [3411223,41523] 0 2024-02-15T18:35:29.770 INFO:tasks.workunit.client.1.smithi121.stdout:0/498: fdatasync d5/d6b/f9d 0 2024-02-15T18:35:29.770 INFO:tasks.workunit.client.1.smithi121.stdout:0/499: readlink d5/d30/d3c/l3f 0 2024-02-15T18:35:29.770 INFO:tasks.workunit.client.1.smithi121.stdout:8/469: rename d0/d61/d8f/f92 to d0/d73/fa9 0 2024-02-15T18:35:29.783 INFO:tasks.workunit.client.1.smithi121.stdout:6/400: dwrite d7/dc/f90 [0,4194304] 0 2024-02-15T18:35:29.785 INFO:tasks.workunit.client.1.smithi121.stdout:8/470: dread d0/d7/de/f2d [0,4194304] 0 2024-02-15T18:35:29.787 INFO:tasks.workunit.client.1.smithi121.stdout:0/500: dread d5/d30/d3c/d7d/d2a/d4e/f5a [0,4194304] 0 2024-02-15T18:35:29.787 INFO:tasks.workunit.client.1.smithi121.stdout:6/401: read d7/dc/d2c/d36/f3e [1484351,111264] 0 2024-02-15T18:35:29.787 INFO:tasks.workunit.client.1.smithi121.stdout:5/359: dwrite d0/fb [4194304,4194304] 0 2024-02-15T18:35:29.787 INFO:tasks.workunit.client.1.smithi121.stdout:0/501: truncate d5/d30/d3c/d7d/dd/f4b 538747 0 2024-02-15T18:35:29.787 INFO:tasks.workunit.client.1.smithi121.stdout:0/502: chown d5/f40 1047 1 2024-02-15T18:35:29.788 INFO:tasks.workunit.client.1.smithi121.stdout:8/471: link d0/d61/d8f/c98 d0/d7/de/d67/caa 0 2024-02-15T18:35:29.788 INFO:tasks.workunit.client.1.smithi121.stdout:5/360: dread d0/d16/d34/f45 [0,4194304] 0 2024-02-15T18:35:29.789 INFO:tasks.workunit.client.1.smithi121.stdout:6/402: mkdir d7/dc/d34/d42/d6e/d91 0 2024-02-15T18:35:29.789 INFO:tasks.workunit.client.1.smithi121.stdout:6/403: write d7/dc/f77 [565526,112] 0 2024-02-15T18:35:29.793 INFO:tasks.workunit.client.1.smithi121.stdout:0/503: rmdir d5/d30/d3c/d7d/d2a/d36/d44/d71 39 2024-02-15T18:35:29.793 INFO:tasks.workunit.client.1.smithi121.stdout:0/504: chown d5/d30/d3c/d7d/d74/c94 17349842 1 2024-02-15T18:35:29.794 INFO:tasks.workunit.client.1.smithi121.stdout:8/472: creat d0/d61/d66/fab x:0 0 0 2024-02-15T18:35:29.795 INFO:tasks.workunit.client.1.smithi121.stdout:8/473: write d0/d3/d20/fa2 [3991483,2577] 0 2024-02-15T18:35:29.795 INFO:tasks.workunit.client.1.smithi121.stdout:8/474: chown d0/c1d 4015438 1 2024-02-15T18:35:29.799 INFO:tasks.workunit.client.1.smithi121.stdout:6/404: dread d7/f24 [0,4194304] 0 2024-02-15T18:35:29.799 INFO:tasks.workunit.client.1.smithi121.stdout:6/405: truncate d7/f5d 476767 0 2024-02-15T18:35:29.800 INFO:tasks.workunit.client.1.smithi121.stdout:6/406: truncate d7/d4b/d60/d6c/f8a 408026 0 2024-02-15T18:35:29.802 INFO:tasks.workunit.client.1.smithi121.stdout:5/361: dwrite d0/d2/d1e/d4e/f55 [0,4194304] 0 2024-02-15T18:35:29.802 INFO:tasks.workunit.client.1.smithi121.stdout:0/505: mknod d5/d30/d5d/ca5 0 2024-02-15T18:35:29.803 INFO:tasks.workunit.client.1.smithi121.stdout:8/475: mkdir d0/d61/d7b/dac 0 2024-02-15T18:35:29.804 INFO:tasks.workunit.client.1.smithi121.stdout:6/407: symlink d7/dc/d8e/l92 0 2024-02-15T18:35:29.805 INFO:tasks.workunit.client.1.smithi121.stdout:5/362: rmdir d0/d2/d18 39 2024-02-15T18:35:29.806 INFO:tasks.workunit.client.1.smithi121.stdout:8/476: rmdir d0/d61/da6 39 2024-02-15T18:35:29.807 INFO:tasks.workunit.client.1.smithi121.stdout:6/408: rmdir d7/dc/d34/d42/d62/d63/d66 39 2024-02-15T18:35:29.808 INFO:tasks.workunit.client.1.smithi121.stdout:5/363: creat d0/d2/d1e/d21/d40/f79 x:0 0 0 2024-02-15T18:35:29.809 INFO:tasks.workunit.client.1.smithi121.stdout:8/477: mknod d0/d2a/d51/cad 0 2024-02-15T18:35:29.810 INFO:tasks.workunit.client.1.smithi121.stdout:6/409: dread d7/dc/d2c/d36/f49 [0,4194304] 0 2024-02-15T18:35:29.810 INFO:tasks.workunit.client.1.smithi121.stdout:6/410: write d7/dc/f25 [2267236,19937] 0 2024-02-15T18:35:29.816 INFO:tasks.workunit.client.1.smithi121.stdout:5/364: mkdir d0/d16/d7a 0 2024-02-15T18:35:29.817 INFO:tasks.workunit.client.1.smithi121.stdout:6/411: mkdir d7/dc/d34/d42/d62/d93 0 2024-02-15T18:35:29.817 INFO:tasks.workunit.client.1.smithi121.stdout:6/412: fdatasync d7/dc/d34/f13 0 2024-02-15T18:35:29.818 INFO:tasks.workunit.client.1.smithi121.stdout:6/413: write d7/dc/d34/d42/d62/f83 [567826,102485] 0 2024-02-15T18:35:29.819 INFO:tasks.workunit.client.1.smithi121.stdout:5/365: creat d0/d2/f7b x:0 0 0 2024-02-15T18:35:29.820 INFO:tasks.workunit.client.1.smithi121.stdout:9/351: unlink d7/d8/d25/d30/d33/d3c/d3f/f6d 0 2024-02-15T18:35:29.820 INFO:tasks.workunit.client.1.smithi121.stdout:9/352: chown d7/d8/d18/f53 6531930 1 2024-02-15T18:35:29.821 INFO:tasks.workunit.client.1.smithi121.stdout:3/353: unlink d2/d1a/d22/f2b 0 2024-02-15T18:35:29.822 INFO:tasks.workunit.client.1.smithi121.stdout:7/293: unlink d0/d5/d11/d1c/l2e 0 2024-02-15T18:35:29.822 INFO:tasks.workunit.client.1.smithi121.stdout:7/294: chown d0/d4a/l50 13 1 2024-02-15T18:35:29.825 INFO:tasks.workunit.client.1.smithi121.stdout:7/295: write d0/f29 [2228425,92933] 0 2024-02-15T18:35:29.826 INFO:tasks.workunit.client.1.smithi121.stdout:5/366: dread d0/f60 [0,4194304] 0 2024-02-15T18:35:29.826 INFO:tasks.workunit.client.1.smithi121.stdout:5/367: write d0/fb [9139320,80134] 0 2024-02-15T18:35:29.832 INFO:tasks.workunit.client.1.smithi121.stdout:0/506: dwrite d5/f3b [0,4194304] 0 2024-02-15T18:35:29.833 INFO:tasks.workunit.client.1.smithi121.stdout:6/414: dread d7/dc/d34/f26 [4194304,4194304] 0 2024-02-15T18:35:29.833 INFO:tasks.workunit.client.1.smithi121.stdout:6/415: fsync d7/f11 0 2024-02-15T18:35:29.833 INFO:tasks.workunit.client.1.smithi121.stdout:6/416: write d7/dc/d34/f86 [312894,21771] 0 2024-02-15T18:35:29.833 INFO:tasks.workunit.client.1.smithi121.stdout:6/417: write d7/dc/d34/d42/f54 [2902933,121512] 0 2024-02-15T18:35:29.833 INFO:tasks.workunit.client.1.smithi121.stdout:6/418: dread - d7/dc/f37 zero size 2024-02-15T18:35:29.836 INFO:tasks.workunit.client.1.smithi121.stdout:3/354: creat d2/d1a/d27/f77 x:0 0 0 2024-02-15T18:35:29.837 INFO:tasks.workunit.client.1.smithi121.stdout:7/296: creat d0/d5/f63 x:0 0 0 2024-02-15T18:35:29.840 INFO:tasks.workunit.client.1.smithi121.stdout:6/419: mkdir d7/dc/d34/d42/d6e/d94 0 2024-02-15T18:35:29.840 INFO:tasks.workunit.client.1.smithi121.stdout:6/420: fdatasync d7/dc/f5e 0 2024-02-15T18:35:29.846 INFO:tasks.workunit.client.1.smithi121.stdout:9/353: dwrite d7/d8/d18/f53 [0,4194304] 0 2024-02-15T18:35:29.846 INFO:tasks.workunit.client.1.smithi121.stdout:9/354: readlink d7/d3a/d42/l76 0 2024-02-15T18:35:29.848 INFO:tasks.workunit.client.1.smithi121.stdout:3/355: dread d2/d1a/d6b/d72/f76 [0,4194304] 0 2024-02-15T18:35:29.851 INFO:tasks.workunit.client.1.smithi121.stdout:3/356: rename d2/d1a/d27/f2d to d2/d5a/f78 0 2024-02-15T18:35:29.851 INFO:tasks.workunit.client.1.smithi121.stdout:3/357: write d2/d1a/d27/f4b [900826,116128] 0 2024-02-15T18:35:29.851 INFO:tasks.workunit.client.1.smithi121.stdout:5/368: dwrite d0/f5f [0,4194304] 0 2024-02-15T18:35:29.851 INFO:tasks.workunit.client.1.smithi121.stdout:5/369: write d0/d2/d1e/d21/d39/f53 [784253,48570] 0 2024-02-15T18:35:29.851 INFO:tasks.workunit.client.1.smithi121.stdout:3/358: creat d2/d1a/d6b/f79 x:0 0 0 2024-02-15T18:35:29.853 INFO:tasks.workunit.client.1.smithi121.stdout:5/370: creat d0/d2/f7c x:0 0 0 2024-02-15T18:35:29.853 INFO:tasks.workunit.client.1.smithi121.stdout:5/371: fsync d0/d2/d1e/d21/d33/f56 0 2024-02-15T18:35:29.858 INFO:tasks.workunit.client.1.smithi121.stdout:6/421: dread d7/dc/d34/f86 [0,4194304] 0 2024-02-15T18:35:29.859 INFO:tasks.workunit.client.1.smithi121.stdout:9/355: dread d7/f4e [0,4194304] 0 2024-02-15T18:35:29.859 INFO:tasks.workunit.client.1.smithi121.stdout:0/507: dwrite d5/d30/d3c/d7d/d2a/f2f [4194304,4194304] 0 2024-02-15T18:35:29.873 INFO:tasks.workunit.client.1.smithi121.stdout:6/422: dread d7/dc/d34/f43 [0,4194304] 0 2024-02-15T18:35:29.874 INFO:tasks.workunit.client.1.smithi121.stdout:5/372: rename d0/d2/d1e/d21 to d0/d2/d1e/d4e/d7d 0 2024-02-15T18:35:29.877 INFO:tasks.workunit.client.1.smithi121.stdout:3/359: fsync d2/d1a/d27/d2c/d25/d31/d34/d4a/f71 0 2024-02-15T18:35:29.877 INFO:tasks.workunit.client.1.smithi121.stdout:3/360: chown d2/d1a/d27/d2c/d25/d31/f57 591746 1 2024-02-15T18:35:29.881 INFO:tasks.workunit.client.1.smithi121.stdout:9/356: dwrite d7/d8/d25/d30/f62 [0,4194304] 0 2024-02-15T18:35:29.882 INFO:tasks.workunit.client.1.smithi121.stdout:0/508: dwrite d5/d30/d3c/d7d/d2a/d36/f51 [0,4194304] 0 2024-02-15T18:35:29.882 INFO:tasks.workunit.client.1.smithi121.stdout:5/373: unlink d0/d2/d18/c36 0 2024-02-15T18:35:29.882 INFO:tasks.workunit.client.1.smithi121.stdout:6/423: write d7/dc/d2c/f5c [1727796,2875] 0 2024-02-15T18:35:29.882 INFO:tasks.workunit.client.1.smithi121.stdout:3/361: creat d2/d1a/f7a x:0 0 0 2024-02-15T18:35:29.889 INFO:tasks.workunit.client.1.smithi121.stdout:9/357: rmdir d7/d8/d25/d30/d33 39 2024-02-15T18:35:29.889 INFO:tasks.workunit.client.1.smithi121.stdout:9/358: truncate d7/d8/f10 4864989 0 2024-02-15T18:35:29.890 INFO:tasks.workunit.client.1.smithi121.stdout:5/374: mknod d0/d16/d7a/c7e 0 2024-02-15T18:35:29.891 INFO:tasks.workunit.client.1.smithi121.stdout:6/424: symlink d7/dc/d34/d42/d62/d63/l95 0 2024-02-15T18:35:29.893 INFO:tasks.workunit.client.1.smithi121.stdout:3/362: creat d2/d1a/d27/d2c/f7b x:0 0 0 2024-02-15T18:35:29.893 INFO:tasks.workunit.client.1.smithi121.stdout:9/359: dread d7/f1c [0,4194304] 0 2024-02-15T18:35:29.911 INFO:tasks.workunit.client.1.smithi121.stdout:3/363: dwrite d2/d1a/d54/f59 [0,4194304] 0 2024-02-15T18:35:29.911 INFO:tasks.workunit.client.1.smithi121.stdout:5/375: symlink d0/d2/d1e/d4e/d7d/d33/l7f 0 2024-02-15T18:35:29.913 INFO:tasks.workunit.client.1.smithi121.stdout:3/364: write d2/d1a/d27/d2c/d25/d30/f46 [653982,25540] 0 2024-02-15T18:35:29.914 INFO:tasks.workunit.client.1.smithi121.stdout:0/509: unlink d5/d30/d3c/d7d/d2a/d36/d44/d71/ca1 0 2024-02-15T18:35:29.917 INFO:tasks.workunit.client.1.smithi121.stdout:9/360: rmdir d7/d26/d4f/d5b 39 2024-02-15T18:35:29.917 INFO:tasks.workunit.client.1.smithi121.stdout:6/425: creat d7/dc/d8e/f96 x:0 0 0 2024-02-15T18:35:29.917 INFO:tasks.workunit.client.1.smithi121.stdout:9/361: fsync d7/d8/d18/f53 0 2024-02-15T18:35:29.921 INFO:tasks.workunit.client.1.smithi121.stdout:0/510: mkdir d5/d30/d3c/da6 0 2024-02-15T18:35:29.923 INFO:tasks.workunit.client.1.smithi121.stdout:9/362: creat d7/d8/d18/f7b x:0 0 0 2024-02-15T18:35:29.924 INFO:tasks.workunit.client.1.smithi121.stdout:0/511: mknod d5/d30/d3c/d7d/d2a/d36/d44/ca7 0 2024-02-15T18:35:29.924 INFO:tasks.workunit.client.1.smithi121.stdout:0/512: truncate d5/d30/d3c/d7d/d74/f82 28945 0 2024-02-15T18:35:29.925 INFO:tasks.workunit.client.1.smithi121.stdout:9/363: creat d7/d8/f7c x:0 0 0 2024-02-15T18:35:29.927 INFO:tasks.workunit.client.1.smithi121.stdout:0/513: rename d5/d30/d3c/d7d/dd/f7e to d5/d6b/fa8 0 2024-02-15T18:35:29.929 INFO:tasks.workunit.client.1.smithi121.stdout:1/419: creat d0/d6/d7/d72/f8e x:0 0 0 2024-02-15T18:35:29.930 INFO:tasks.workunit.client.1.smithi121.stdout:6/426: dread d7/dc/d34/f26 [4194304,4194304] 0 2024-02-15T18:35:29.931 INFO:tasks.workunit.client.1.smithi121.stdout:1/420: mknod d0/d6/d7/d9/c8f 0 2024-02-15T18:35:29.934 INFO:tasks.workunit.client.1.smithi121.stdout:5/376: dwrite d0/d16/d34/f6a [4194304,4194304] 0 2024-02-15T18:35:29.935 INFO:tasks.workunit.client.1.smithi121.stdout:1/421: write d0/d6/d7/f26 [427480,129902] 0 2024-02-15T18:35:29.937 INFO:tasks.workunit.client.1.smithi121.stdout:1/422: symlink d0/d6/d7/d9/l90 0 2024-02-15T18:35:29.937 INFO:tasks.workunit.client.1.smithi121.stdout:1/423: chown d0/d6/c15 232105740 1 2024-02-15T18:35:29.937 INFO:tasks.workunit.client.1.smithi121.stdout:1/424: chown d0/d6/d7/d9/c8f 0 1 2024-02-15T18:35:29.937 INFO:tasks.workunit.client.1.smithi121.stdout:1/425: write d0/d6/d7/d50/d7e/f5b [2007517,3095] 0 2024-02-15T18:35:29.939 INFO:tasks.workunit.client.1.smithi121.stdout:1/426: creat d0/d6/d7/d9/d14/d3e/f91 x:0 0 0 2024-02-15T18:35:29.939 INFO:tasks.workunit.client.1.smithi121.stdout:1/427: truncate d0/d6/d7/d9/d51/d5f/f62 518022 0 2024-02-15T18:35:30.001 INFO:tasks.workunit.client.1.smithi121.stdout:3/365: rename d2/d1a/d27/d2c to d2/d1a/d27/d7c 0 2024-02-15T18:35:30.001 INFO:tasks.workunit.client.1.smithi121.stdout:3/366: write d2/d1a/d27/f5e [638264,84118] 0 2024-02-15T18:35:30.002 INFO:tasks.workunit.client.1.smithi121.stdout:1/428: getdents d0/d6/d7/d9/d51/d5f 0 2024-02-15T18:35:30.002 INFO:tasks.workunit.client.1.smithi121.stdout:1/429: write d0/d6/d7/f26 [3923501,62157] 0 2024-02-15T18:35:30.002 INFO:tasks.workunit.client.1.smithi121.stdout:9/364: creat d7/d8/d36/f7d x:0 0 0 2024-02-15T18:35:30.006 INFO:tasks.workunit.client.1.smithi121.stdout:1/430: rename d0/d6/d7/f26 to d0/d39/d4b/f92 0 2024-02-15T18:35:30.006 INFO:tasks.workunit.client.1.smithi121.stdout:3/367: dread d2/d1a/d6b/d72/f76 [0,4194304] 0 2024-02-15T18:35:30.007 INFO:tasks.workunit.client.1.smithi121.stdout:6/427: dwrite d7/dc/d34/f43 [0,4194304] 0 2024-02-15T18:35:30.008 INFO:tasks.workunit.client.1.smithi121.stdout:6/428: write d7/dc/d2c/d36/f3c [228170,35562] 0 2024-02-15T18:35:30.014 INFO:tasks.workunit.client.1.smithi121.stdout:1/431: dread d0/d6/d7/d9/d14/d3e/f4a [0,4194304] 0 2024-02-15T18:35:30.017 INFO:tasks.workunit.client.1.smithi121.stdout:1/432: rename d0/d6/d7/f5e to d0/d6/d7/d29/f93 0 2024-02-15T18:35:30.017 INFO:tasks.workunit.client.1.smithi121.stdout:1/433: fdatasync d0/d6/f12 0 2024-02-15T18:35:30.019 INFO:tasks.workunit.client.1.smithi121.stdout:1/434: creat d0/d6/d7/f94 x:0 0 0 2024-02-15T18:35:30.035 INFO:tasks.workunit.client.1.smithi121.stdout:6/429: dwrite d7/f28 [0,4194304] 0 2024-02-15T18:35:30.035 INFO:tasks.workunit.client.1.smithi121.stdout:6/430: stat d7/dc/d34/d42/d6e/d94 0 2024-02-15T18:35:30.048 INFO:tasks.workunit.client.1.smithi121.stdout:1/435: fsync d0/f1d 0 2024-02-15T18:35:30.048 INFO:tasks.workunit.client.1.smithi121.stdout:1/436: fsync d0/d6/d7/d9/d14/d3e/f43 0 2024-02-15T18:35:30.049 INFO:tasks.workunit.client.1.smithi121.stdout:1/437: mknod d0/d6/d7/d50/d7e/c95 0 2024-02-15T18:35:30.052 INFO:tasks.workunit.client.1.smithi121.stdout:1/438: mknod d0/d39/c96 0 2024-02-15T18:35:30.066 INFO:tasks.workunit.client.1.smithi121.stdout:0/514: rmdir d5 39 2024-02-15T18:35:30.068 INFO:tasks.workunit.client.1.smithi121.stdout:5/377: creat d0/d2/d1e/d4e/f80 x:0 0 0 2024-02-15T18:35:30.069 INFO:tasks.workunit.client.1.smithi121.stdout:9/365: symlink d7/d26/d4f/d5b/l7e 0 2024-02-15T18:35:30.071 INFO:tasks.workunit.client.1.smithi121.stdout:9/366: dread d7/d8/d40/f52 [0,4194304] 0 2024-02-15T18:35:30.073 INFO:tasks.workunit.client.1.smithi121.stdout:1/439: dwrite d0/d6/d7/d50/d7e/f85 [0,4194304] 0 2024-02-15T18:35:30.089 INFO:tasks.workunit.client.1.smithi121.stdout:3/368: mkdir d2/d1a/d6b/d7d 0 2024-02-15T18:35:30.089 INFO:tasks.workunit.client.1.smithi121.stdout:6/431: creat d7/dc/d34/d42/d6e/f97 x:0 0 0 2024-02-15T18:35:30.091 INFO:tasks.workunit.client.1.smithi121.stdout:0/515: mknod d5/d9b/ca9 0 2024-02-15T18:35:30.091 INFO:tasks.workunit.client.1.smithi121.stdout:0/516: write d5/d30/d3c/d7d/f3a [2688593,37219] 0 2024-02-15T18:35:30.091 INFO:tasks.workunit.client.1.smithi121.stdout:0/517: write d5/d30/d3c/d7d/dd/f32 [1527126,30609] 0 2024-02-15T18:35:30.093 INFO:tasks.workunit.client.1.smithi121.stdout:3/369: mknod d2/d1a/d27/d7c/d25/d30/c7e 0 2024-02-15T18:35:30.095 INFO:tasks.workunit.client.1.smithi121.stdout:0/518: creat d5/d30/d3c/d7d/d74/d96/faa x:0 0 0 2024-02-15T18:35:30.095 INFO:tasks.workunit.client.1.smithi121.stdout:0/519: chown d5/d30/d3c/d7d/cc 272994428 1 2024-02-15T18:35:30.097 INFO:tasks.workunit.client.1.smithi121.stdout:5/378: creat d0/d16/f81 x:0 0 0 2024-02-15T18:35:30.099 INFO:tasks.workunit.client.1.smithi121.stdout:3/370: rename d2/f36 to d2/d1a/d27/f7f 0 2024-02-15T18:35:30.099 INFO:tasks.workunit.client.1.smithi121.stdout:0/520: mkdir d5/d6b/dab 0 2024-02-15T18:35:30.100 INFO:tasks.workunit.client.1.smithi121.stdout:9/367: dwrite d7/d8/d18/f7b [0,4194304] 0 2024-02-15T18:35:30.100 INFO:tasks.workunit.client.1.smithi121.stdout:5/379: getdents d0/d16/d34 0 2024-02-15T18:35:30.101 INFO:tasks.workunit.client.1.smithi121.stdout:9/368: read d7/f1b [1843408,26430] 0 2024-02-15T18:35:30.104 INFO:tasks.workunit.client.1.smithi121.stdout:1/440: dwrite d0/f3d [0,4194304] 0 2024-02-15T18:35:30.104 INFO:tasks.workunit.client.1.smithi121.stdout:1/441: write d0/d6/d7/d9/d14/d48/f70 [497572,116759] 0 2024-02-15T18:35:30.104 INFO:tasks.workunit.client.1.smithi121.stdout:3/371: creat d2/d1a/d27/d63/f80 x:0 0 0 2024-02-15T18:35:30.104 INFO:tasks.workunit.client.1.smithi121.stdout:6/432: dread d7/dc/f48 [0,4194304] 0 2024-02-15T18:35:30.104 INFO:tasks.workunit.client.1.smithi121.stdout:0/521: creat d5/d30/d5d/fac x:0 0 0 2024-02-15T18:35:30.107 INFO:tasks.workunit.client.1.smithi121.stdout:5/380: creat d0/f82 x:0 0 0 2024-02-15T18:35:30.107 INFO:tasks.workunit.client.1.smithi121.stdout:5/381: truncate d0/d16/f2b 191486 0 2024-02-15T18:35:30.107 INFO:tasks.workunit.client.1.smithi121.stdout:5/382: fdatasync d0/d2/d1e/d4e/d7d/d47/d5a/f5c 0 2024-02-15T18:35:30.107 INFO:tasks.workunit.client.1.smithi121.stdout:1/442: rename d0/c30 to d0/d6/d7/d29/d77/c97 0 2024-02-15T18:35:30.110 INFO:tasks.workunit.client.1.smithi121.stdout:9/369: link d7/f1b d7/d26/d4f/f7f 0 2024-02-15T18:35:30.112 INFO:tasks.workunit.client.1.smithi121.stdout:1/443: symlink d0/d6/d7/d72/l98 0 2024-02-15T18:35:30.112 INFO:tasks.workunit.client.1.smithi121.stdout:6/433: dread d7/f31 [0,4194304] 0 2024-02-15T18:35:30.112 INFO:tasks.workunit.client.1.smithi121.stdout:6/434: readlink d7/l3a 0 2024-02-15T18:35:30.115 INFO:tasks.workunit.client.1.smithi121.stdout:3/372: mkdir d2/d1a/d27/d63/d81 0 2024-02-15T18:35:30.115 INFO:tasks.workunit.client.1.smithi121.stdout:3/373: readlink d2/d1a/l24 0 2024-02-15T18:35:30.115 INFO:tasks.workunit.client.1.smithi121.stdout:3/374: truncate d2/d1a/d27/f77 214516 0 2024-02-15T18:35:30.115 INFO:tasks.workunit.client.1.smithi121.stdout:1/444: mknod d0/d6/d7/d72/c99 0 2024-02-15T18:35:30.116 INFO:tasks.workunit.client.1.smithi121.stdout:0/522: getdents d5/d30/d3c/d7d/d2a/d36/d44/d71 0 2024-02-15T18:35:30.116 INFO:tasks.workunit.client.1.smithi121.stdout:5/383: creat d0/d2/d70/f83 x:0 0 0 2024-02-15T18:35:30.118 INFO:tasks.workunit.client.1.smithi121.stdout:3/375: creat d2/d1a/d6b/f82 x:0 0 0 2024-02-15T18:35:30.118 INFO:tasks.workunit.client.1.smithi121.stdout:1/445: creat d0/d6/d7/d29/f9a x:0 0 0 2024-02-15T18:35:30.118 INFO:tasks.workunit.client.1.smithi121.stdout:1/446: truncate d0/d6/d7/d9/d51/f6c 657703 0 2024-02-15T18:35:30.119 INFO:tasks.workunit.client.1.smithi121.stdout:0/523: creat d5/d6b/dab/fad x:0 0 0 2024-02-15T18:35:30.119 INFO:tasks.workunit.client.1.smithi121.stdout:0/524: fsync d5/d30/d3c/d7d/dd/f7f 0 2024-02-15T18:35:30.119 INFO:tasks.workunit.client.1.smithi121.stdout:0/525: readlink d5/d30/d3c/d7d/dd/l28 0 2024-02-15T18:35:30.119 INFO:tasks.workunit.client.1.smithi121.stdout:5/384: chown d0/d2/d1e/d4e/d7d/l2a 1 1 2024-02-15T18:35:30.122 INFO:tasks.workunit.client.1.smithi121.stdout:9/370: unlink d7/l2e 0 2024-02-15T18:35:30.122 INFO:tasks.workunit.client.1.smithi121.stdout:1/447: rename d0/d39/c42 to d0/d6/d7/c9b 0 2024-02-15T18:35:30.124 INFO:tasks.workunit.client.1.smithi121.stdout:5/385: unlink d0/d2/d1e/d4e/d7d/d40/d62/f74 0 2024-02-15T18:35:30.124 INFO:tasks.workunit.client.1.smithi121.stdout:5/386: write d0/d2/d1e/f42 [4325401,80941] 0 2024-02-15T18:35:30.124 INFO:tasks.workunit.client.1.smithi121.stdout:5/387: fsync d0/d2/d18/f5b 0 2024-02-15T18:35:30.125 INFO:tasks.workunit.client.1.smithi121.stdout:9/371: creat d7/d8/d25/d30/f80 x:0 0 0 2024-02-15T18:35:30.125 INFO:tasks.workunit.client.1.smithi121.stdout:3/376: dread d2/d1a/d27/d7c/f19 [0,4194304] 0 2024-02-15T18:35:30.125 INFO:tasks.workunit.client.1.smithi121.stdout:3/377: fsync d2/d1a/d22/f35 0 2024-02-15T18:35:30.125 INFO:tasks.workunit.client.1.smithi121.stdout:3/378: chown d2/d1a/l1e 0 1 2024-02-15T18:35:30.126 INFO:tasks.workunit.client.1.smithi121.stdout:3/379: write d2/d1a/d6b/f79 [274053,120997] 0 2024-02-15T18:35:30.126 INFO:tasks.workunit.client.1.smithi121.stdout:3/380: chown d2/d1a/d27/d7c/c45 0 1 2024-02-15T18:35:30.126 INFO:tasks.workunit.client.1.smithi121.stdout:5/388: creat d0/d2/d1e/d4e/d7d/d47/f84 x:0 0 0 2024-02-15T18:35:30.128 INFO:tasks.workunit.client.1.smithi121.stdout:9/372: creat d7/d26/f81 x:0 0 0 2024-02-15T18:35:30.130 INFO:tasks.workunit.client.1.smithi121.stdout:3/381: unlink d2/d1a/d27/d7c/d25/d31/l33 0 2024-02-15T18:35:30.130 INFO:tasks.workunit.client.1.smithi121.stdout:9/373: dread d7/d8/d25/d30/f32 [0,4194304] 0 2024-02-15T18:35:30.131 INFO:tasks.workunit.client.1.smithi121.stdout:5/389: symlink d0/d2/d1e/d4e/d7d/d40/l85 0 2024-02-15T18:35:30.132 INFO:tasks.workunit.client.1.smithi121.stdout:9/374: mknod d7/d8/d18/d73/c82 0 2024-02-15T18:35:30.133 INFO:tasks.workunit.client.1.smithi121.stdout:3/382: creat d2/d1a/d6b/d7d/f83 x:0 0 0 2024-02-15T18:35:30.133 INFO:tasks.workunit.client.1.smithi121.stdout:3/383: stat d2/d1a/d27/d7c/d25/d31/f50 0 2024-02-15T18:35:30.134 INFO:tasks.workunit.client.1.smithi121.stdout:6/435: dwrite d7/dc/f21 [4194304,4194304] 0 2024-02-15T18:35:30.135 INFO:tasks.workunit.client.1.smithi121.stdout:9/375: link d7/d8/d18/f7b d7/d8/d18/f83 0 2024-02-15T18:35:30.135 INFO:tasks.workunit.client.1.smithi121.stdout:3/384: creat d2/d1a/d27/f84 x:0 0 0 2024-02-15T18:35:30.143 INFO:tasks.workunit.client.1.smithi121.stdout:6/436: symlink d7/d4b/d59/l98 0 2024-02-15T18:35:30.143 INFO:tasks.workunit.client.1.smithi121.stdout:9/376: creat d7/d8/d25/d30/d33/d3c/f84 x:0 0 0 2024-02-15T18:35:30.145 INFO:tasks.workunit.client.1.smithi121.stdout:0/526: dwrite d5/d30/d3c/d7d/dd/f22 [0,4194304] 0 2024-02-15T18:35:30.146 INFO:tasks.workunit.client.1.smithi121.stdout:9/377: symlink d7/d8/d25/d30/d33/d3e/l85 0 2024-02-15T18:35:30.148 INFO:tasks.workunit.client.1.smithi121.stdout:9/378: creat d7/d8/d25/d30/d33/d3e/f86 x:0 0 0 2024-02-15T18:35:30.148 INFO:tasks.workunit.client.1.smithi121.stdout:9/379: chown d7/c12 681695 1 2024-02-15T18:35:30.148 INFO:tasks.workunit.client.1.smithi121.stdout:9/380: write d7/d8/d18/f45 [549921,100395] 0 2024-02-15T18:35:30.148 INFO:tasks.workunit.client.1.smithi121.stdout:9/381: fsync d7/d8/f10 0 2024-02-15T18:35:30.153 INFO:tasks.workunit.client.1.smithi121.stdout:9/382: unlink d7/d8/d25/d30/d33/d3e/c46 0 2024-02-15T18:35:30.154 INFO:tasks.workunit.client.1.smithi121.stdout:0/527: write d5/d30/f33 [2319231,50847] 0 2024-02-15T18:35:30.156 INFO:tasks.workunit.client.1.smithi121.stdout:9/383: link d7/d3a/d42/f6c d7/d8/f87 0 2024-02-15T18:35:30.166 INFO:tasks.workunit.client.1.smithi121.stdout:5/390: dwrite d0/d2/fd [0,4194304] 0 2024-02-15T18:35:30.168 INFO:tasks.workunit.client.1.smithi121.stdout:5/391: symlink d0/d16/d34/l86 0 2024-02-15T18:35:30.168 INFO:tasks.workunit.client.1.smithi121.stdout:6/437: dwrite d7/dc/d34/d42/f54 [0,4194304] 0 2024-02-15T18:35:30.169 INFO:tasks.workunit.client.1.smithi121.stdout:1/448: dwrite d0/d39/d4b/f61 [0,4194304] 0 2024-02-15T18:35:30.169 INFO:tasks.workunit.client.1.smithi121.stdout:9/384: dread d7/d8/d18/f19 [0,4194304] 0 2024-02-15T18:35:30.169 INFO:tasks.workunit.client.1.smithi121.stdout:9/385: write d7/d8/d25/d30/d33/d3c/f84 [554596,10435] 0 2024-02-15T18:35:30.169 INFO:tasks.workunit.client.1.smithi121.stdout:5/392: getdents d0/d2/d1e 0 2024-02-15T18:35:30.170 INFO:tasks.workunit.client.1.smithi121.stdout:6/438: dread d7/f5d [0,4194304] 0 2024-02-15T18:35:30.170 INFO:tasks.workunit.client.1.smithi121.stdout:6/439: stat d7/dc/f4c 0 2024-02-15T18:35:30.172 INFO:tasks.workunit.client.1.smithi121.stdout:1/449: dread d0/d6/d7/f4c [0,4194304] 0 2024-02-15T18:35:30.172 INFO:tasks.workunit.client.1.smithi121.stdout:1/450: stat d0/d6/d7/d9/f23 0 2024-02-15T18:35:30.172 INFO:tasks.workunit.client.1.smithi121.stdout:1/451: stat d0/d39/d4b/f6d 0 2024-02-15T18:35:30.173 INFO:tasks.workunit.client.1.smithi121.stdout:0/528: dwrite d5/d30/d3c/d7d/dd/f17 [0,4194304] 0 2024-02-15T18:35:30.175 INFO:tasks.workunit.client.1.smithi121.stdout:1/452: creat d0/f9c x:0 0 0 2024-02-15T18:35:30.180 INFO:tasks.workunit.client.1.smithi121.stdout:1/453: mkdir d0/d6/d7/d29/d9d 0 2024-02-15T18:35:30.185 INFO:tasks.workunit.client.1.smithi121.stdout:9/386: creat d7/d8/d25/d30/d33/f88 x:0 0 0 2024-02-15T18:35:30.186 INFO:tasks.workunit.client.1.smithi121.stdout:5/393: link d0/d2/f7c d0/d2/d18/f87 0 2024-02-15T18:35:30.187 INFO:tasks.workunit.client.1.smithi121.stdout:6/440: creat d7/d4b/d59/f99 x:0 0 0 2024-02-15T18:35:30.192 INFO:tasks.workunit.client.1.smithi121.stdout:0/529: dwrite d5/d30/d3c/d7d/dd/f7f [0,4194304] 0 2024-02-15T18:35:30.192 INFO:tasks.workunit.client.1.smithi121.stdout:1/454: dread d0/d6/f6e [0,4194304] 0 2024-02-15T18:35:30.198 INFO:tasks.workunit.client.1.smithi121.stdout:9/387: rmdir d7/d3a/d42/d56 39 2024-02-15T18:35:30.198 INFO:tasks.workunit.client.1.smithi121.stdout:9/388: chown d7/d26/d4f/d5b/l7e 175234 1 2024-02-15T18:35:30.198 INFO:tasks.workunit.client.1.smithi121.stdout:9/389: chown d7/d8/ca 35707410 1 2024-02-15T18:35:30.198 INFO:tasks.workunit.client.1.smithi121.stdout:9/390: fdatasync d7/d8/f17 0 2024-02-15T18:35:30.198 INFO:tasks.workunit.client.1.smithi121.stdout:5/394: mkdir d0/d2/d70/d88 0 2024-02-15T18:35:30.198 INFO:tasks.workunit.client.1.smithi121.stdout:0/530: truncate f2 3999634 0 2024-02-15T18:35:30.200 INFO:tasks.workunit.client.1.smithi121.stdout:6/441: dread d7/dc/f4c [0,4194304] 0 2024-02-15T18:35:30.200 INFO:tasks.workunit.client.1.smithi121.stdout:5/395: creat d0/f89 x:0 0 0 2024-02-15T18:35:30.204 INFO:tasks.workunit.client.1.smithi121.stdout:6/442: dread d7/dc/d2c/d36/f3c [0,4194304] 0 2024-02-15T18:35:30.205 INFO:tasks.workunit.client.1.smithi121.stdout:0/531: creat d5/d30/d3c/d7d/d2a/d73/d8d/fae x:0 0 0 2024-02-15T18:35:30.206 INFO:tasks.workunit.client.1.smithi121.stdout:5/396: getdents d0/d2/d1e/d4e/d7d/d39 0 2024-02-15T18:35:30.209 INFO:tasks.workunit.client.1.smithi121.stdout:6/443: creat d7/dc/d34/d42/d6e/d94/f9a x:0 0 0 2024-02-15T18:35:30.210 INFO:tasks.workunit.client.1.smithi121.stdout:0/532: creat d5/d30/d3c/d7d/d2a/d73/d81/faf x:0 0 0 2024-02-15T18:35:30.211 INFO:tasks.workunit.client.1.smithi121.stdout:5/397: rmdir d0/d6e 0 2024-02-15T18:35:30.211 INFO:tasks.workunit.client.1.smithi121.stdout:5/398: read - d0/d2/d1e/d4e/d7d/d33/d4c/f5e zero size 2024-02-15T18:35:30.213 INFO:tasks.workunit.client.1.smithi121.stdout:0/533: link d5/d30/d3c/d7d/d2a/d4e/f5b d5/d30/d3c/d7d/dd/fb0 0 2024-02-15T18:35:30.214 INFO:tasks.workunit.client.1.smithi121.stdout:5/399: creat d0/d2/d70/f8a x:0 0 0 2024-02-15T18:35:30.214 INFO:tasks.workunit.client.1.smithi121.stdout:5/400: truncate d0/d16/d34/f45 952483 0 2024-02-15T18:35:30.215 INFO:tasks.workunit.client.1.smithi121.stdout:0/534: mkdir d5/d6b/dab/db1 0 2024-02-15T18:35:30.216 INFO:tasks.workunit.client.1.smithi121.stdout:5/401: link d0/d2/d1e/d4e/d7d/d40/f78 d0/d2/d1e/d4e/d7d/d33/d4c/f8b 0 2024-02-15T18:35:30.216 INFO:tasks.workunit.client.1.smithi121.stdout:5/402: fsync d0/d16/f2b 0 2024-02-15T18:35:30.216 INFO:tasks.workunit.client.1.smithi121.stdout:5/403: stat d0/d2/d1e/d4e/d7d/d39/c66 0 2024-02-15T18:35:30.217 INFO:tasks.workunit.client.1.smithi121.stdout:5/404: fsync d0/d2/f7c 0 2024-02-15T18:35:30.217 INFO:tasks.workunit.client.1.smithi121.stdout:0/535: chown d5/d30/d3c/d7d/d2a/l2e 0 1 2024-02-15T18:35:30.217 INFO:tasks.workunit.client.1.smithi121.stdout:0/536: chown d5/l53 8877893 1 2024-02-15T18:35:30.221 INFO:tasks.workunit.client.1.smithi121.stdout:6/444: dread d7/dc/d34/f71 [0,4194304] 0 2024-02-15T18:35:30.222 INFO:tasks.workunit.client.1.smithi121.stdout:1/455: dwrite d0/d6/d7/d29/f54 [0,4194304] 0 2024-02-15T18:35:30.225 INFO:tasks.workunit.client.1.smithi121.stdout:9/391: dwrite f5 [0,4194304] 0 2024-02-15T18:35:30.226 INFO:tasks.workunit.client.1.smithi121.stdout:0/537: symlink d5/d6b/dab/db1/lb2 0 2024-02-15T18:35:30.226 INFO:tasks.workunit.client.1.smithi121.stdout:1/456: mknod d0/d6/d7/d72/c9e 0 2024-02-15T18:35:30.226 INFO:tasks.workunit.client.1.smithi121.stdout:1/457: chown d0/l4 2051030 1 2024-02-15T18:35:30.227 INFO:tasks.workunit.client.1.smithi121.stdout:6/445: mkdir d7/dc/d34/d42/d62/d63/d8d/d9b 0 2024-02-15T18:35:30.227 INFO:tasks.workunit.client.1.smithi121.stdout:0/538: write d5/d30/f33 [2288632,107876] 0 2024-02-15T18:35:30.228 INFO:tasks.workunit.client.1.smithi121.stdout:0/539: fdatasync d5/d30/d3c/d7d/d2a/d36/f54 0 2024-02-15T18:35:30.229 INFO:tasks.workunit.client.1.smithi121.stdout:1/458: mknod d0/c9f 0 2024-02-15T18:35:30.229 INFO:tasks.workunit.client.1.smithi121.stdout:9/392: symlink d7/d8/d25/d30/d33/d3e/l89 0 2024-02-15T18:35:30.230 INFO:tasks.workunit.client.1.smithi121.stdout:1/459: symlink d0/d6/d7/d9/d14/la0 0 2024-02-15T18:35:30.230 INFO:tasks.workunit.client.1.smithi121.stdout:6/446: rmdir d7/dc/d34/d42/d62/d63/d8d/d9b 0 2024-02-15T18:35:30.232 INFO:tasks.workunit.client.1.smithi121.stdout:0/540: rename d5/d9b/ca9 to d5/d30/d3c/cb3 0 2024-02-15T18:35:30.232 INFO:tasks.workunit.client.1.smithi121.stdout:0/541: readlink d5/d30/d3c/d7d/d2a/d36/d44/l48 0 2024-02-15T18:35:30.235 INFO:tasks.workunit.client.1.smithi121.stdout:9/393: symlink d7/d26/l8a 0 2024-02-15T18:35:30.236 INFO:tasks.workunit.client.1.smithi121.stdout:6/447: rmdir d7/dc/d34/d42/d62/d93 0 2024-02-15T18:35:30.236 INFO:tasks.workunit.client.1.smithi121.stdout:6/448: stat d7/c29 0 2024-02-15T18:35:30.236 INFO:tasks.workunit.client.1.smithi121.stdout:6/449: stat d7/dc/d2c/d36/f3e 0 2024-02-15T18:35:30.236 INFO:tasks.workunit.client.1.smithi121.stdout:5/405: dwrite d0/d2/d1e/d4e/d7d/d33/d4c/f8b [0,4194304] 0 2024-02-15T18:35:30.248 INFO:tasks.workunit.client.1.smithi121.stdout:1/460: dwrite d0/d6/d7/d9/d51/d5f/f62 [0,4194304] 0 2024-02-15T18:35:30.248 INFO:tasks.workunit.client.1.smithi121.stdout:1/461: dread - d0/d6/d7/d9/d14/f58 zero size 2024-02-15T18:35:30.248 INFO:tasks.workunit.client.1.smithi121.stdout:1/462: chown d0/d6/d7/d29/f54 32500 1 2024-02-15T18:35:30.248 INFO:tasks.workunit.client.1.smithi121.stdout:1/463: fsync d0/d6/d7/f4c 0 2024-02-15T18:35:30.251 INFO:tasks.workunit.client.1.smithi121.stdout:5/406: dread d0/d2/fd [0,4194304] 0 2024-02-15T18:35:30.251 INFO:tasks.workunit.client.1.smithi121.stdout:1/464: write d0/d6/d7/d9/d14/d3e/f4a [3124533,57107] 0 2024-02-15T18:35:30.251 INFO:tasks.workunit.client.1.smithi121.stdout:1/465: dread - d0/d6/d7/d9/d14/f6b zero size 2024-02-15T18:35:30.251 INFO:tasks.workunit.client.1.smithi121.stdout:1/466: chown d0/d6/d7/c9b 1 1 2024-02-15T18:35:30.252 INFO:tasks.workunit.client.1.smithi121.stdout:0/542: rename d5/d30/d3c to d5/d6b/dab/db4 0 2024-02-15T18:35:30.257 INFO:tasks.workunit.client.1.smithi121.stdout:9/394: rename d7/d8/d18 to d7/d3a/d42/d56/d8b 0 2024-02-15T18:35:30.259 INFO:tasks.workunit.client.1.smithi121.stdout:0/543: creat d5/d6b/dab/db4/d7d/dd/fb5 x:0 0 0 2024-02-15T18:35:30.260 INFO:tasks.workunit.client.1.smithi121.stdout:9/395: symlink d7/d3a/l8c 0 2024-02-15T18:35:30.260 INFO:tasks.workunit.client.1.smithi121.stdout:9/396: readlink d7/d3a/d42/l76 0 2024-02-15T18:35:30.260 INFO:tasks.workunit.client.1.smithi121.stdout:0/544: stat d5/d6b/dab/db4/c60 0 2024-02-15T18:35:30.260 INFO:tasks.workunit.client.1.smithi121.stdout:0/545: readlink d5/d30/l83 0 2024-02-15T18:35:30.261 INFO:tasks.workunit.client.1.smithi121.stdout:9/397: symlink d7/d8/d25/d30/d33/l8d 0 2024-02-15T18:35:30.263 INFO:tasks.workunit.client.1.smithi121.stdout:0/546: write d5/d6b/dab/db4/d7d/dd/f7f [2978309,108938] 0 2024-02-15T18:35:30.264 INFO:tasks.workunit.client.1.smithi121.stdout:0/547: mknod d5/cb6 0 2024-02-15T18:35:30.266 INFO:tasks.workunit.client.1.smithi121.stdout:0/548: mkdir d5/d6b/dab/db4/d7d/d2a/d36/d44/db7 0 2024-02-15T18:35:30.267 INFO:tasks.workunit.client.1.smithi121.stdout:9/398: dread d7/d8/f10 [0,4194304] 0 2024-02-15T18:35:30.267 INFO:tasks.workunit.client.1.smithi121.stdout:9/399: truncate d7/d8/d25/d30/d33/f88 301517 0 2024-02-15T18:35:30.267 INFO:tasks.workunit.client.1.smithi121.stdout:9/400: chown d7/d8/d25/d30/d33/d3e/f41 3 1 2024-02-15T18:35:30.267 INFO:tasks.workunit.client.1.smithi121.stdout:0/549: mkdir d5/d6b/dab/db4/d7d/d2a/d73/d97/db8 0 2024-02-15T18:35:30.268 INFO:tasks.workunit.client.1.smithi121.stdout:0/550: symlink d5/d6b/dab/db4/d7d/d2a/d73/d81/lb9 0 2024-02-15T18:35:30.322 INFO:tasks.workunit.client.1.smithi121.stdout:5/407: dread d0/d2/d1e/d4e/d7d/d47/d5a/f5c [0,4194304] 0 2024-02-15T18:35:30.322 INFO:tasks.workunit.client.1.smithi121.stdout:5/408: rmdir d0/d2/d1e/d4e/d7d/d39 39 2024-02-15T18:35:30.322 INFO:tasks.workunit.client.1.smithi121.stdout:5/409: dread - d0/d2/f7b zero size 2024-02-15T18:35:30.323 INFO:tasks.workunit.client.1.smithi121.stdout:5/410: creat d0/d2/d1e/f8c x:0 0 0 2024-02-15T18:35:30.324 INFO:tasks.workunit.client.1.smithi121.stdout:5/411: read d0/f5f [4173962,112677] 0 2024-02-15T18:35:30.324 INFO:tasks.workunit.client.1.smithi121.stdout:5/412: chown d0/d2/d1e/d4e/d7d/c5d 32447 1 2024-02-15T18:35:30.324 INFO:tasks.workunit.client.1.smithi121.stdout:5/413: stat d0/d2/d1e/d4e/d7d/d33/f59 0 2024-02-15T18:35:30.324 INFO:tasks.workunit.client.1.smithi121.stdout:5/414: stat d0/fb 0 2024-02-15T18:35:30.326 INFO:tasks.workunit.client.1.smithi121.stdout:5/415: rename d0/d2/d1e/d4e/d7d/d33/d4c/f54 to d0/d2/d1e/d4e/d7d/d33/d4c/f8d 0 2024-02-15T18:35:30.328 INFO:tasks.workunit.client.1.smithi121.stdout:6/450: dwrite d7/dc/f15 [0,4194304] 0 2024-02-15T18:35:30.328 INFO:tasks.workunit.client.1.smithi121.stdout:6/451: dread - d7/dc/d34/d42/d62/f6b zero size 2024-02-15T18:35:30.328 INFO:tasks.workunit.client.1.smithi121.stdout:0/551: dwrite d5/d6b/dab/db4/d7d/d2a/d4e/f5a [0,4194304] 0 2024-02-15T18:35:30.328 INFO:tasks.workunit.client.1.smithi121.stdout:6/452: dread - d7/d4b/d59/f99 zero size 2024-02-15T18:35:30.329 INFO:tasks.workunit.client.1.smithi121.stdout:6/453: symlink d7/dc/d34/d42/d62/d63/d66/l9c 0 2024-02-15T18:35:30.331 INFO:tasks.workunit.client.1.smithi121.stdout:6/454: link d7/dc/f48 d7/dc/d2c/d36/f9d 0 2024-02-15T18:35:30.333 INFO:tasks.workunit.client.1.smithi121.stdout:9/401: dwrite d7/d8/d40/f52 [0,4194304] 0 2024-02-15T18:35:30.333 INFO:tasks.workunit.client.1.smithi121.stdout:6/455: write d7/dc/d34/d42/d62/f74 [705359,111911] 0 2024-02-15T18:35:30.333 INFO:tasks.workunit.client.1.smithi121.stdout:1/467: dwrite d0/d6/d7/d9/d14/d48/f70 [0,4194304] 0 2024-02-15T18:35:30.335 INFO:tasks.workunit.client.1.smithi121.stdout:9/402: mkdir d7/d3a/d8e 0 2024-02-15T18:35:30.336 INFO:tasks.workunit.client.1.smithi121.stdout:6/456: symlink d7/dc/d76/l9e 0 2024-02-15T18:35:30.338 INFO:tasks.workunit.client.1.smithi121.stdout:0/552: dread d5/d6b/dab/db4/d7d/f56 [0,4194304] 0 2024-02-15T18:35:30.338 INFO:tasks.workunit.client.1.smithi121.stdout:9/403: mkdir d7/d8/d25/d30/d33/d8f 0 2024-02-15T18:35:30.338 INFO:tasks.workunit.client.1.smithi121.stdout:9/404: chown d7/d8/d25/d30/d33/d3e 1010432 1 2024-02-15T18:35:30.338 INFO:tasks.workunit.client.1.smithi121.stdout:6/457: creat d7/dc/d34/d42/d6e/f9f x:0 0 0 2024-02-15T18:35:30.339 INFO:tasks.workunit.client.1.smithi121.stdout:6/458: read d7/dc/d34/f26 [7798943,115061] 0 2024-02-15T18:35:30.341 INFO:tasks.workunit.client.1.smithi121.stdout:6/459: creat d7/dc/d34/d42/d62/d63/d8d/fa0 x:0 0 0 2024-02-15T18:35:30.342 INFO:tasks.workunit.client.1.smithi121.stdout:9/405: write d7/d8/d25/d30/f62 [3558920,74631] 0 2024-02-15T18:35:30.342 INFO:tasks.workunit.client.1.smithi121.stdout:9/406: read - d7/d8/d36/f7d zero size 2024-02-15T18:35:30.343 INFO:tasks.workunit.client.1.smithi121.stdout:6/460: read d7/dc/d34/f1f [266276,8223] 0 2024-02-15T18:35:30.343 INFO:tasks.workunit.client.1.smithi121.stdout:6/461: readlink d7/d4b/l4d 0 2024-02-15T18:35:30.345 INFO:tasks.workunit.client.1.smithi121.stdout:1/468: link d0/d39/l46 d0/d6/d7/d29/la1 0 2024-02-15T18:35:30.345 INFO:tasks.workunit.client.1.smithi121.stdout:0/553: dread d5/d6b/dab/db4/d7d/f3a [0,4194304] 0 2024-02-15T18:35:30.346 INFO:tasks.workunit.client.1.smithi121.stdout:1/469: creat d0/d6/d7/d9/fa2 x:0 0 0 2024-02-15T18:35:30.349 INFO:tasks.workunit.client.1.smithi121.stdout:1/470: creat d0/d39/d7d/fa3 x:0 0 0 2024-02-15T18:35:30.349 INFO:tasks.workunit.client.1.smithi121.stdout:9/407: mknod d7/d8/d25/d30/d33/d8f/c90 0 2024-02-15T18:35:30.349 INFO:tasks.workunit.client.1.smithi121.stdout:9/408: chown d7/d8/c1a 104951 1 2024-02-15T18:35:30.349 INFO:tasks.workunit.client.1.smithi121.stdout:9/409: chown d7/d8/d25/d30/f80 506753354 1 2024-02-15T18:35:30.350 INFO:tasks.workunit.client.1.smithi121.stdout:0/554: symlink d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/lba 0 2024-02-15T18:35:30.352 INFO:tasks.workunit.client.1.smithi121.stdout:6/462: dread d7/dc/d2c/d36/f58 [0,4194304] 0 2024-02-15T18:35:30.352 INFO:tasks.workunit.client.1.smithi121.stdout:9/410: link d7/d8/d25/d30/d33/f58 d7/d26/d4f/d5b/d77/f91 0 2024-02-15T18:35:30.353 INFO:tasks.workunit.client.1.smithi121.stdout:6/463: stat d7/d4b/c7b 0 2024-02-15T18:35:30.353 INFO:tasks.workunit.client.1.smithi121.stdout:5/416: dwrite d0/f82 [0,4194304] 0 2024-02-15T18:35:30.355 INFO:tasks.workunit.client.1.smithi121.stdout:0/555: unlink d5/d6b/dab/db4/d7d/c80 0 2024-02-15T18:35:30.355 INFO:tasks.workunit.client.1.smithi121.stdout:0/556: fdatasync d5/d6b/dab/db4/d7d/dd/f2c 0 2024-02-15T18:35:30.356 INFO:tasks.workunit.client.1.smithi121.stdout:9/411: unlink d7/l27 0 2024-02-15T18:35:30.356 INFO:tasks.workunit.client.1.smithi121.stdout:0/557: read d5/d6b/dab/db4/d7d/d2a/d36/f3d [2189006,85673] 0 2024-02-15T18:35:30.356 INFO:tasks.workunit.client.1.smithi121.stdout:6/464: mkdir d7/dc/d34/d42/d62/d63/d6f/d89/da1 0 2024-02-15T18:35:30.362 INFO:tasks.workunit.client.1.smithi121.stdout:0/558: symlink d5/d6b/dab/db4/d7d/d2a/d36/d44/db7/lbb 0 2024-02-15T18:35:30.362 INFO:tasks.workunit.client.1.smithi121.stdout:5/417: creat d0/d2/f8e x:0 0 0 2024-02-15T18:35:30.364 INFO:tasks.workunit.client.1.smithi121.stdout:6/465: creat d7/d4b/d60/d73/fa2 x:0 0 0 2024-02-15T18:35:30.364 INFO:tasks.workunit.client.1.smithi121.stdout:0/559: rmdir d5/d6b/dab/db4/d7d/d2a/d4e 39 2024-02-15T18:35:30.365 INFO:tasks.workunit.client.1.smithi121.stdout:6/466: link d7/dc/d34/d42/c64 d7/dc/d34/d42/d62/d63/d8d/ca3 0 2024-02-15T18:35:30.366 INFO:tasks.workunit.client.1.smithi121.stdout:0/560: getdents d5/d6b/dab/db4/d7d/d74 0 2024-02-15T18:35:30.373 INFO:tasks.workunit.client.1.smithi121.stdout:1/471: dwrite d0/d6/d7/d29/f5d [0,4194304] 0 2024-02-15T18:35:30.373 INFO:tasks.workunit.client.1.smithi121.stdout:1/472: readlink d0/d6/d7/d9/d14/l49 0 2024-02-15T18:35:30.373 INFO:tasks.workunit.client.1.smithi121.stdout:1/473: fdatasync d0/d6/d7/d9/d14/d3e/f43 0 2024-02-15T18:35:30.375 INFO:tasks.workunit.client.1.smithi121.stdout:9/412: dwrite d7/f1f [0,4194304] 0 2024-02-15T18:35:30.376 INFO:tasks.workunit.client.1.smithi121.stdout:9/413: dread d7/f59 [0,4194304] 0 2024-02-15T18:35:30.376 INFO:tasks.workunit.client.1.smithi121.stdout:1/474: creat d0/d6/d7/d9/fa4 x:0 0 0 2024-02-15T18:35:30.377 INFO:tasks.workunit.client.1.smithi121.stdout:1/475: unlink d0/d6/d7/l44 0 2024-02-15T18:35:30.377 INFO:tasks.workunit.client.1.smithi121.stdout:1/476: getdents d0/d6/d7/d9/d14/d3e/d88 0 2024-02-15T18:35:30.382 INFO:tasks.workunit.client.1.smithi121.stdout:1/477: fsync d0/d39/d7d/fa3 0 2024-02-15T18:35:30.382 INFO:tasks.workunit.client.1.smithi121.stdout:1/478: truncate d0/d6/d7/d9/d14/d3e/f91 1029310 0 2024-02-15T18:35:30.383 INFO:tasks.workunit.client.1.smithi121.stdout:5/418: dwrite d0/d2/f7b [0,4194304] 0 2024-02-15T18:35:30.385 INFO:tasks.workunit.client.1.smithi121.stdout:0/561: truncate d5/d6b/dab/db4/d7d/d2a/d36/f8b 3293775 0 2024-02-15T18:35:30.388 INFO:tasks.workunit.client.1.smithi121.stdout:9/414: symlink d7/d3a/d8e/l92 0 2024-02-15T18:35:30.388 INFO:tasks.workunit.client.1.smithi121.stdout:6/467: dwrite d7/dc/d34/d7c/f85 [0,4194304] 0 2024-02-15T18:35:30.389 INFO:tasks.workunit.client.1.smithi121.stdout:0/562: creat d5/d9b/fbc x:0 0 0 2024-02-15T18:35:30.389 INFO:tasks.workunit.client.1.smithi121.stdout:9/415: mkdir d7/d8/d25/d30/d33/d8f/d93 0 2024-02-15T18:35:30.391 INFO:tasks.workunit.client.1.smithi121.stdout:1/479: symlink d0/d6/d7/d9/d51/la5 0 2024-02-15T18:35:30.392 INFO:tasks.workunit.client.1.smithi121.stdout:6/468: mknod d7/d4b/d60/d73/ca4 0 2024-02-15T18:35:30.393 INFO:tasks.workunit.client.1.smithi121.stdout:0/563: dread d5/d6b/dab/db4/d7d/dd/f34 [0,4194304] 0 2024-02-15T18:35:30.395 INFO:tasks.workunit.client.1.smithi121.stdout:9/416: read d7/d26/d4f/d5b/d77/f91 [3406736,87974] 0 2024-02-15T18:35:30.396 INFO:tasks.workunit.client.1.smithi121.stdout:6/469: creat d7/dc/d34/d42/d62/d63/d6f/d89/fa5 x:0 0 0 2024-02-15T18:35:30.397 INFO:tasks.workunit.client.1.smithi121.stdout:0/564: link d5/d6b/dab/db4/d7d/dd/c59 d5/d6b/dab/db4/d7d/d2a/d36/d44/db7/cbd 0 2024-02-15T18:35:30.397 INFO:tasks.workunit.client.1.smithi121.stdout:0/565: dread - d5/d9b/fbc zero size 2024-02-15T18:35:30.398 INFO:tasks.workunit.client.1.smithi121.stdout:0/566: write f2 [3633101,54362] 0 2024-02-15T18:35:30.399 INFO:tasks.workunit.client.1.smithi121.stdout:9/417: rename d7/d8/d25/l50 to d7/d26/d4f/l94 0 2024-02-15T18:35:30.399 INFO:tasks.workunit.client.1.smithi121.stdout:5/419: dwrite d0/d2/d1e/d4e/d7d/d40/f78 [0,4194304] 0 2024-02-15T18:35:30.400 INFO:tasks.workunit.client.1.smithi121.stdout:1/480: write d0/d6/d7/d50/d7e/f76 [758175,57847] 0 2024-02-15T18:35:30.400 INFO:tasks.workunit.client.1.smithi121.stdout:6/470: creat d7/d4b/d60/d6c/fa6 x:0 0 0 2024-02-15T18:35:30.401 INFO:tasks.workunit.client.1.smithi121.stdout:0/567: symlink d5/d6b/dab/db4/d7d/d2a/d36/d44/lbe 0 2024-02-15T18:35:30.406 INFO:tasks.workunit.client.1.smithi121.stdout:9/418: getdents d7/d8/d36 0 2024-02-15T18:35:30.407 INFO:tasks.workunit.client.1.smithi121.stdout:5/420: mkdir d0/d16/d34/d8f 0 2024-02-15T18:35:30.407 INFO:tasks.workunit.client.1.smithi121.stdout:5/421: truncate d0/d2/d1e/d4e/d7d/d33/f59 978064 0 2024-02-15T18:35:30.408 INFO:tasks.workunit.client.1.smithi121.stdout:6/471: rename d7/dc/d34/f3d to d7/dc/d34/d42/d62/d63/d8d/fa7 0 2024-02-15T18:35:30.408 INFO:tasks.workunit.client.1.smithi121.stdout:6/472: truncate d7/dc/d34/d42/d62/d63/d8d/fa7 924257 0 2024-02-15T18:35:30.409 INFO:tasks.workunit.client.1.smithi121.stdout:0/568: mkdir d5/d6b/dab/db1/dbf 0 2024-02-15T18:35:30.409 INFO:tasks.workunit.client.1.smithi121.stdout:6/473: write d7/dc/f21 [1730411,25463] 0 2024-02-15T18:35:30.411 INFO:tasks.workunit.client.1.smithi121.stdout:9/419: mknod d7/d3a/d8e/c95 0 2024-02-15T18:35:30.414 INFO:tasks.workunit.client.1.smithi121.stdout:6/474: creat d7/dc/d2c/d36/fa8 x:0 0 0 2024-02-15T18:35:30.415 INFO:tasks.workunit.client.1.smithi121.stdout:9/420: rmdir d7/d3a/d42 39 2024-02-15T18:35:30.415 INFO:tasks.workunit.client.1.smithi121.stdout:5/422: rename d0/d2/f25 to d0/d16/d34/d8f/f90 0 2024-02-15T18:35:30.417 INFO:tasks.workunit.client.1.smithi121.stdout:6/475: link d7/dc/d34/f43 d7/dc/d34/d42/d62/d63/d8d/fa9 0 2024-02-15T18:35:30.419 INFO:tasks.workunit.client.1.smithi121.stdout:1/481: dwrite d0/d6/d7/f1b [0,4194304] 0 2024-02-15T18:35:30.421 INFO:tasks.workunit.client.1.smithi121.stdout:9/421: mkdir d7/d8/d25/d30/d33/d3c/d96 0 2024-02-15T18:35:30.422 INFO:tasks.workunit.client.1.smithi121.stdout:5/423: creat d0/d2/d1e/d4e/f91 x:0 0 0 2024-02-15T18:35:30.422 INFO:tasks.workunit.client.1.smithi121.stdout:5/424: chown d0/d2/d1e/d4e/d7d/d40/d62 219809585 1 2024-02-15T18:35:30.422 INFO:tasks.workunit.client.1.smithi121.stdout:1/482: mkdir d0/d6/d7/d9/d14/d3e/d88/da6 0 2024-02-15T18:35:30.423 INFO:tasks.workunit.client.1.smithi121.stdout:6/476: rename d7/d4b/d59/f99 to d7/dc/d34/d7c/faa 0 2024-02-15T18:35:30.423 INFO:tasks.workunit.client.1.smithi121.stdout:6/477: chown d7/d4b/d60 0 1 2024-02-15T18:35:30.424 INFO:tasks.workunit.client.1.smithi121.stdout:6/478: write d7/d4b/d60/d6c/f8a [787869,31325] 0 2024-02-15T18:35:30.424 INFO:tasks.workunit.client.1.smithi121.stdout:1/483: symlink d0/d39/la7 0 2024-02-15T18:35:30.424 INFO:tasks.workunit.client.1.smithi121.stdout:1/484: chown d0/d6/d7/d9/d14/d3e/d88/da6 7369 1 2024-02-15T18:35:30.427 INFO:tasks.workunit.client.1.smithi121.stdout:1/485: write d0/d6/f1a [1374247,51506] 0 2024-02-15T18:35:30.429 INFO:tasks.workunit.client.1.smithi121.stdout:5/425: getdents d0 0 2024-02-15T18:35:30.429 INFO:tasks.workunit.client.1.smithi121.stdout:1/486: rename d0/d6/d7/d50/d7e/f5b to d0/d6/d7/d9/d14/d3e/d88/da6/fa8 0 2024-02-15T18:35:30.430 INFO:tasks.workunit.client.1.smithi121.stdout:6/479: getdents d7/dc/d8e 0 2024-02-15T18:35:30.433 INFO:tasks.workunit.client.1.smithi121.stdout:6/480: rename d7/d4b/l7f to d7/dc/d34/d7c/lab 0 2024-02-15T18:35:30.433 INFO:tasks.workunit.client.1.smithi121.stdout:1/487: link d0/d6/d7/d9/d51/f6c d0/d6/d7/d9/d14/d3e/d88/fa9 0 2024-02-15T18:35:30.434 INFO:tasks.workunit.client.1.smithi121.stdout:1/488: read d0/d6/d7/d9/d14/d3e/f91 [602172,120922] 0 2024-02-15T18:35:30.434 INFO:tasks.workunit.client.1.smithi121.stdout:6/481: mkdir d7/dc/d34/d42/d62/dac 0 2024-02-15T18:35:30.437 INFO:tasks.workunit.client.1.smithi121.stdout:1/489: mknod d0/d6/d7/d9/d14/d3e/d88/caa 0 2024-02-15T18:35:30.437 INFO:tasks.workunit.client.1.smithi121.stdout:1/490: chown d0/d6/d7/f1b 49742309 1 2024-02-15T18:35:30.437 INFO:tasks.workunit.client.1.smithi121.stdout:1/491: chown d0/d6 481178399 1 2024-02-15T18:35:30.439 INFO:tasks.workunit.client.1.smithi121.stdout:1/492: unlink d0/d6/d7/c2a 0 2024-02-15T18:35:30.442 INFO:tasks.workunit.client.1.smithi121.stdout:6/482: unlink d7/dc/d34/d42/d62/d63/d6f/d89/fa5 0 2024-02-15T18:35:30.442 INFO:tasks.workunit.client.1.smithi121.stdout:6/483: chown d7/c51 52170506 1 2024-02-15T18:35:30.442 INFO:tasks.workunit.client.1.smithi121.stdout:6/484: dread - d7/dc/d34/d42/d6e/d94/f9a zero size 2024-02-15T18:35:30.442 INFO:tasks.workunit.client.1.smithi121.stdout:6/485: chown d7/dc/d34/d42/d62/d63/d8d/fa7 0 1 2024-02-15T18:35:30.445 INFO:tasks.workunit.client.1.smithi121.stdout:1/493: symlink d0/lab 0 2024-02-15T18:35:30.445 INFO:tasks.workunit.client.1.smithi121.stdout:1/494: fsync d0/d39/d4b/f61 0 2024-02-15T18:35:30.446 INFO:tasks.workunit.client.1.smithi121.stdout:1/495: unlink d0/d6/d7/c3c 0 2024-02-15T18:35:30.447 INFO:tasks.workunit.client.1.smithi121.stdout:9/422: dwrite f5 [0,4194304] 0 2024-02-15T18:35:30.447 INFO:tasks.workunit.client.1.smithi121.stdout:5/426: dwrite d0/d2/d18/f29 [0,4194304] 0 2024-02-15T18:35:30.448 INFO:tasks.workunit.client.1.smithi121.stdout:9/423: mknod d7/d8/d25/d30/d33/d3c/c97 0 2024-02-15T18:35:30.449 INFO:tasks.workunit.client.1.smithi121.stdout:1/496: write d0/d6/d7/d29/f93 [166317,34725] 0 2024-02-15T18:35:30.450 INFO:tasks.workunit.client.1.smithi121.stdout:5/427: unlink d0/l23 0 2024-02-15T18:35:30.450 INFO:tasks.workunit.client.1.smithi121.stdout:1/497: write d0/d6/d7/d9/d14/d3e/f4a [2623564,52978] 0 2024-02-15T18:35:30.451 INFO:tasks.workunit.client.1.smithi121.stdout:9/424: dread d7/d3a/d42/f6c [0,4194304] 0 2024-02-15T18:35:30.456 INFO:tasks.workunit.client.1.smithi121.stdout:5/428: creat d0/d2/d70/f92 x:0 0 0 2024-02-15T18:35:30.462 INFO:tasks.workunit.client.1.smithi121.stdout:6/486: dwrite d7/dc/d34/f86 [0,4194304] 0 2024-02-15T18:35:30.462 INFO:tasks.workunit.client.1.smithi121.stdout:6/487: fsync d7/dc/d34/d42/f54 0 2024-02-15T18:35:30.463 INFO:tasks.workunit.client.1.smithi121.stdout:6/488: readlink d7/dc/d2c/d36/l4f 0 2024-02-15T18:35:30.463 INFO:tasks.workunit.client.1.smithi121.stdout:1/498: mkdir d0/d39/d4b/dac 0 2024-02-15T18:35:30.463 INFO:tasks.workunit.client.1.smithi121.stdout:9/425: dread f5 [0,4194304] 0 2024-02-15T18:35:30.463 INFO:tasks.workunit.client.1.smithi121.stdout:9/426: stat d7/f4e 0 2024-02-15T18:35:30.463 INFO:tasks.workunit.client.1.smithi121.stdout:5/429: link d0/f60 d0/f93 0 2024-02-15T18:35:30.466 INFO:tasks.workunit.client.1.smithi121.stdout:1/499: unlink d0/c56 0 2024-02-15T18:35:30.466 INFO:tasks.workunit.client.1.smithi121.stdout:6/489: creat d7/dc/d34/d42/d6e/fad x:0 0 0 2024-02-15T18:35:30.466 INFO:tasks.workunit.client.1.smithi121.stdout:6/490: chown d7/dc/c23 31855124 1 2024-02-15T18:35:30.467 INFO:tasks.workunit.client.1.smithi121.stdout:6/491: read d7/dc/d34/d42/d62/d63/d8d/fa7 [819877,106235] 0 2024-02-15T18:35:30.467 INFO:tasks.workunit.client.1.smithi121.stdout:9/427: creat d7/d3a/d8e/f98 x:0 0 0 2024-02-15T18:35:30.468 INFO:tasks.workunit.client.1.smithi121.stdout:6/492: write d7/f18 [313901,15916] 0 2024-02-15T18:35:30.468 INFO:tasks.workunit.client.1.smithi121.stdout:6/493: truncate d7/dc/d34/d7c/faa 424927 0 2024-02-15T18:35:30.470 INFO:tasks.workunit.client.1.smithi121.stdout:5/430: creat d0/d16/f94 x:0 0 0 2024-02-15T18:35:30.470 INFO:tasks.workunit.client.1.smithi121.stdout:5/431: read - d0/d2/d70/f92 zero size 2024-02-15T18:35:30.476 INFO:tasks.workunit.client.1.smithi121.stdout:6/494: mkdir d7/dae 0 2024-02-15T18:35:30.477 INFO:tasks.workunit.client.1.smithi121.stdout:5/432: creat d0/d2/d1e/d4e/d7d/d33/d4c/f95 x:0 0 0 2024-02-15T18:35:30.477 INFO:tasks.workunit.client.1.smithi121.stdout:5/433: write d0/d16/f94 [630435,121740] 0 2024-02-15T18:35:30.478 INFO:tasks.workunit.client.1.smithi121.stdout:5/434: readlink d0/d2/d1e/d4e/d7d/d33/l7f 0 2024-02-15T18:35:30.478 INFO:tasks.workunit.client.1.smithi121.stdout:6/495: mknod d7/dae/caf 0 2024-02-15T18:35:30.481 INFO:tasks.workunit.client.1.smithi121.stdout:6/496: getdents d7/dc/d34/d7c 0 2024-02-15T18:35:30.488 INFO:tasks.workunit.client.1.smithi121.stdout:1/500: dwrite d0/d6/d7/d9/d14/f8a [0,4194304] 0 2024-02-15T18:35:30.488 INFO:tasks.workunit.client.1.smithi121.stdout:1/501: write d0/d6/d7/f65 [1040920,26493] 0 2024-02-15T18:35:30.488 INFO:tasks.workunit.client.1.smithi121.stdout:9/428: dwrite d7/d8/f16 [0,4194304] 0 2024-02-15T18:35:30.493 INFO:tasks.workunit.client.1.smithi121.stdout:5/435: rmdir d0/d2/d1e/d4e/d7d/d33/d4c 39 2024-02-15T18:35:30.496 INFO:tasks.workunit.client.1.smithi121.stdout:5/436: getdents d0/d16 0 2024-02-15T18:35:30.497 INFO:tasks.workunit.client.1.smithi121.stdout:5/437: creat d0/d2/d70/d88/f96 x:0 0 0 2024-02-15T18:35:30.497 INFO:tasks.workunit.client.1.smithi121.stdout:5/438: truncate d0/d16/f81 1015462 0 2024-02-15T18:35:30.499 INFO:tasks.workunit.client.1.smithi121.stdout:5/439: link d0/d2/d70/f92 d0/d16/d34/d8f/f97 0 2024-02-15T18:35:30.501 INFO:tasks.workunit.client.1.smithi121.stdout:9/429: unlink d7/cc 0 2024-02-15T18:35:30.502 INFO:tasks.workunit.client.1.smithi121.stdout:9/430: rename d7/d8/d25/d30/d33/d3c/d96 to d7/d8/d99 0 2024-02-15T18:35:30.503 INFO:tasks.workunit.client.1.smithi121.stdout:9/431: symlink d7/d8/d25/d30/d33/d79/l9a 0 2024-02-15T18:35:30.503 INFO:tasks.workunit.client.1.smithi121.stdout:9/432: fsync d7/d8/d25/d30/f62 0 2024-02-15T18:35:30.503 INFO:tasks.workunit.client.1.smithi121.stdout:9/433: dread - d7/d8/d40/f72 zero size 2024-02-15T18:35:30.504 INFO:tasks.workunit.client.1.smithi121.stdout:6/497: dwrite d7/d4b/d60/d6c/f8a [0,4194304] 0 2024-02-15T18:35:30.504 INFO:tasks.workunit.client.1.smithi121.stdout:6/498: dread - d7/dc/d34/d42/d6e/f97 zero size 2024-02-15T18:35:30.507 INFO:tasks.workunit.client.1.smithi121.stdout:6/499: rename d7/dc/d34/f1f to d7/dc/d34/d42/d62/fb0 0 2024-02-15T18:35:30.508 INFO:tasks.workunit.client.1.smithi121.stdout:6/500: creat d7/dc/d34/d42/d62/d63/d6f/fb1 x:0 0 0 2024-02-15T18:35:30.508 INFO:tasks.workunit.client.1.smithi121.stdout:6/501: write d7/dc/d34/f13 [2462656,22978] 0 2024-02-15T18:35:30.509 INFO:tasks.workunit.client.1.smithi121.stdout:5/440: dread d0/d2/d1e/d4e/d7d/d39/f51 [0,4194304] 0 2024-02-15T18:35:30.509 INFO:tasks.workunit.client.1.smithi121.stdout:5/441: dread - d0/d2/f8e zero size 2024-02-15T18:35:30.509 INFO:tasks.workunit.client.1.smithi121.stdout:6/502: creat d7/d4b/d60/d73/fb2 x:0 0 0 2024-02-15T18:35:30.509 INFO:tasks.workunit.client.1.smithi121.stdout:5/442: creat d0/d16/f98 x:0 0 0 2024-02-15T18:35:30.510 INFO:tasks.workunit.client.1.smithi121.stdout:6/503: read d7/dc/f5e [1951645,104968] 0 2024-02-15T18:35:30.510 INFO:tasks.workunit.client.1.smithi121.stdout:6/504: readlink d7/l2b 0 2024-02-15T18:35:30.511 INFO:tasks.workunit.client.1.smithi121.stdout:1/502: truncate d0/d6/d7/f65 115396 0 2024-02-15T18:35:30.514 INFO:tasks.workunit.client.1.smithi121.stdout:1/503: creat d0/d6/d7/d29/d77/fad x:0 0 0 2024-02-15T18:35:30.516 INFO:tasks.workunit.client.1.smithi121.stdout:1/504: creat d0/fae x:0 0 0 2024-02-15T18:35:30.516 INFO:tasks.workunit.client.1.smithi121.stdout:6/505: creat d7/dc/d34/d42/d62/dac/fb3 x:0 0 0 2024-02-15T18:35:30.517 INFO:tasks.workunit.client.1.smithi121.stdout:1/505: mkdir d0/d6/d7/d9/d14/d3e/d88/daf 0 2024-02-15T18:35:30.517 INFO:tasks.workunit.client.1.smithi121.stdout:6/506: getdents d7/dc/d34/d42/d62/d63 0 2024-02-15T18:35:30.522 INFO:tasks.workunit.client.1.smithi121.stdout:1/506: rmdir d0/d6 39 2024-02-15T18:35:30.530 INFO:tasks.workunit.client.1.smithi121.stdout:9/434: dwrite d7/f1f [4194304,4194304] 0 2024-02-15T18:35:30.534 INFO:tasks.workunit.client.1.smithi121.stdout:9/435: rmdir d7/d8/d99 0 2024-02-15T18:35:30.534 INFO:tasks.workunit.client.1.smithi121.stdout:9/436: chown d7/f1b 7889 1 2024-02-15T18:35:30.537 INFO:tasks.workunit.client.1.smithi121.stdout:5/443: dwrite d0/d2/d1e/d4e/d7d/d40/d62/f73 [0,4194304] 0 2024-02-15T18:35:30.537 INFO:tasks.workunit.client.1.smithi121.stdout:5/444: chown d0/d2/c46 6768 1 2024-02-15T18:35:30.537 INFO:tasks.workunit.client.1.smithi121.stdout:5/445: write d0/d16/f2b [956514,96233] 0 2024-02-15T18:35:30.542 INFO:tasks.workunit.client.1.smithi121.stdout:1/507: dwrite d0/fae [0,4194304] 0 2024-02-15T18:35:30.548 INFO:tasks.workunit.client.1.smithi121.stdout:9/437: dread d7/f1f [0,4194304] 0 2024-02-15T18:35:30.550 INFO:tasks.workunit.client.1.smithi121.stdout:9/438: mknod d7/d26/c9b 0 2024-02-15T18:35:30.550 INFO:tasks.workunit.client.1.smithi121.stdout:9/439: truncate d7/d8/d25/d30/f64 612694 0 2024-02-15T18:35:30.551 INFO:tasks.workunit.client.1.smithi121.stdout:9/440: mknod d7/d8/d25/d30/d33/d8f/c9c 0 2024-02-15T18:35:30.551 INFO:tasks.workunit.client.1.smithi121.stdout:1/508: dread d0/d6/d7/d29/f41 [0,4194304] 0 2024-02-15T18:35:30.551 INFO:tasks.workunit.client.1.smithi121.stdout:9/441: rmdir d7/d8/d25/d30/d33/d3c/d3f 39 2024-02-15T18:35:30.552 INFO:tasks.workunit.client.1.smithi121.stdout:9/442: creat d7/d3a/f9d x:0 0 0 2024-02-15T18:35:30.553 INFO:tasks.workunit.client.1.smithi121.stdout:9/443: rmdir d7/d8 39 2024-02-15T18:35:30.554 INFO:tasks.workunit.client.1.smithi121.stdout:1/509: unlink d0/d6/d7/d9/f20 0 2024-02-15T18:35:30.556 INFO:tasks.workunit.client.1.smithi121.stdout:1/510: symlink d0/d6/d7/d9/lb0 0 2024-02-15T18:35:30.558 INFO:tasks.workunit.client.1.smithi121.stdout:1/511: rename d0/f3d to d0/d6/fb1 0 2024-02-15T18:35:30.564 INFO:tasks.workunit.client.1.smithi121.stdout:1/512: dread d0/d6/d7/d9/d14/f8a [0,4194304] 0 2024-02-15T18:35:30.564 INFO:tasks.workunit.client.1.smithi121.stdout:1/513: dread - d0/d6/d7/d9/d14/f6b zero size 2024-02-15T18:35:30.566 INFO:tasks.workunit.client.1.smithi121.stdout:1/514: unlink d0/d6/d7/d9/d14/d48/f7c 0 2024-02-15T18:35:30.566 INFO:tasks.workunit.client.1.smithi121.stdout:1/515: chown d0/d6/d7/d29/l4f 5 1 2024-02-15T18:35:30.567 INFO:tasks.workunit.client.1.smithi121.stdout:1/516: mknod d0/d6/d7/d29/cb2 0 2024-02-15T18:35:30.570 INFO:tasks.workunit.client.1.smithi121.stdout:1/517: rename d0/d6/d7/d9/f7b to d0/d39/fb3 0 2024-02-15T18:35:30.570 INFO:tasks.workunit.client.1.smithi121.stdout:1/518: write d0/d6/d7/d9/d14/f3b [742497,53285] 0 2024-02-15T18:35:30.570 INFO:tasks.workunit.client.1.smithi121.stdout:1/519: chown d0/d6/d7/l37 1127 1 2024-02-15T18:35:30.571 INFO:tasks.workunit.client.1.smithi121.stdout:1/520: mkdir d0/d6/d7/d9/d14/d3e/d88/daf/db4 0 2024-02-15T18:35:30.572 INFO:tasks.workunit.client.1.smithi121.stdout:1/521: write d0/d39/d4b/f6d [3239108,62027] 0 2024-02-15T18:35:30.573 INFO:tasks.workunit.client.1.smithi121.stdout:1/522: truncate d0/d6/d7/d29/f41 591152 0 2024-02-15T18:35:30.573 INFO:tasks.workunit.client.1.smithi121.stdout:1/523: chown d0/d6/d7/d9/d14/la0 861137 1 2024-02-15T18:35:30.579 INFO:tasks.workunit.client.1.smithi121.stdout:1/524: dread d0/d6/d7/d9/d14/d48/f78 [0,4194304] 0 2024-02-15T18:35:30.580 INFO:tasks.workunit.client.1.smithi121.stdout:1/525: creat d0/d6/d7/d9/d14/d3e/fb5 x:0 0 0 2024-02-15T18:35:30.580 INFO:tasks.workunit.client.1.smithi121.stdout:1/526: dread - d0/d6/d7/d29/d77/fad zero size 2024-02-15T18:35:30.597 INFO:tasks.workunit.client.1.smithi121.stdout:1/527: dwrite d0/d6/d7/d29/f5d [0,4194304] 0 2024-02-15T18:35:30.597 INFO:tasks.workunit.client.1.smithi121.stdout:1/528: chown d0/d6/d7/f65 2 1 2024-02-15T18:35:30.601 INFO:tasks.workunit.client.1.smithi121.stdout:1/529: mkdir d0/d6/d7/d9/d14/db6 0 2024-02-15T18:35:30.602 INFO:tasks.workunit.client.1.smithi121.stdout:1/530: mknod d0/d6/d7/d9/d14/cb7 0 2024-02-15T18:35:30.610 INFO:tasks.workunit.client.1.smithi121.stdout:1/531: dread d0/d39/d4b/f92 [0,4194304] 0 2024-02-15T18:35:30.611 INFO:tasks.workunit.client.1.smithi121.stdout:1/532: creat d0/d6/d7/d9/d14/db6/fb8 x:0 0 0 2024-02-15T18:35:30.612 INFO:tasks.workunit.client.1.smithi121.stdout:1/533: symlink d0/d6/d7/d29/lb9 0 2024-02-15T18:35:30.614 INFO:tasks.workunit.client.1.smithi121.stdout:1/534: rename d0/d39/d4b/f55 to d0/d6/d7/d9/d14/db6/fba 0 2024-02-15T18:35:30.614 INFO:tasks.workunit.client.1.smithi121.stdout:1/535: write d0/d6/d7/d29/f9a [314611,109136] 0 2024-02-15T18:35:30.628 INFO:tasks.workunit.client.1.smithi121.stdout:1/536: dwrite d0/d6/d7/d9/d14/db6/fba [0,4194304] 0 2024-02-15T18:35:30.628 INFO:tasks.workunit.client.1.smithi121.stdout:1/537: write d0/d39/d7d/fa3 [485281,55172] 0 2024-02-15T18:35:30.631 INFO:tasks.workunit.client.1.smithi121.stdout:1/538: creat d0/d6/d7/d9/d14/fbb x:0 0 0 2024-02-15T18:35:30.635 INFO:tasks.workunit.client.1.smithi121.stdout:1/539: creat d0/fbc x:0 0 0 2024-02-15T18:35:30.635 INFO:tasks.workunit.client.1.smithi121.stdout:1/540: write d0/fae [4903345,46607] 0 2024-02-15T18:35:30.646 INFO:tasks.workunit.client.1.smithi121.stdout:1/541: dread d0/d6/d7/d29/f5d [0,4194304] 0 2024-02-15T18:35:30.659 INFO:tasks.workunit.client.1.smithi121.stdout:1/542: dwrite d0/f9c [0,4194304] 0 2024-02-15T18:35:30.663 INFO:tasks.workunit.client.1.smithi121.stdout:1/543: rename d0/d6/f12 to d0/d6/d7/d9/d14/d3e/d88/da6/fbd 0 2024-02-15T18:35:30.663 INFO:tasks.workunit.client.1.smithi121.stdout:1/544: truncate d0/f9c 4388426 0 2024-02-15T18:35:30.663 INFO:tasks.workunit.client.1.smithi121.stdout:1/545: fsync d0/d6/d7/d50/d7e/f76 0 2024-02-15T18:35:30.670 INFO:tasks.workunit.client.1.smithi121.stdout:1/546: dread d0/d6/d7/d50/d7e/f76 [0,4194304] 0 2024-02-15T18:35:30.672 INFO:tasks.workunit.client.1.smithi121.stdout:1/547: rename d0/d6/d7/d9/d14/d3e/f43 to d0/fbe 0 2024-02-15T18:35:30.675 INFO:tasks.workunit.client.1.smithi121.stdout:1/548: mknod d0/d39/d4b/cbf 0 2024-02-15T18:35:30.675 INFO:tasks.workunit.client.1.smithi121.stdout:1/549: chown d0/d6/d7/f4c 3347 1 2024-02-15T18:35:30.675 INFO:tasks.workunit.client.1.smithi121.stdout:1/550: chown d0/d6/d7/d29/l4f 5 1 2024-02-15T18:35:30.675 INFO:tasks.workunit.client.1.smithi121.stdout:1/551: chown d0/d6/d7/d50 0 1 2024-02-15T18:35:30.684 INFO:tasks.workunit.client.1.smithi121.stdout:1/552: dread d0/d6/d7/d9/d14/d3e/f71 [0,4194304] 0 2024-02-15T18:35:30.701 INFO:tasks.workunit.client.1.smithi121.stdout:1/553: dwrite d0/d6/d7/d50/d7e/f85 [0,4194304] 0 2024-02-15T18:35:30.701 INFO:tasks.workunit.client.1.smithi121.stdout:1/554: chown d0/f9c 2901 1 2024-02-15T18:35:30.701 INFO:tasks.workunit.client.1.smithi121.stdout:1/555: chown d0/d6/d7/d9/d14/d48/f70 14 1 2024-02-15T18:35:30.701 INFO:tasks.workunit.client.1.smithi121.stdout:1/556: chown d0/d6 0 1 2024-02-15T18:35:31.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:30.593618+0000 mgr.smithi121.ckiryi (mgr.24385) 7 : cluster [DBG] pgmap v9: 65 pgs: 65 active+clean; 1.0 GiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 48 MiB/s rd, 99 MiB/s wr, 241 op/s 2024-02-15T18:35:31.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:30 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:30.593618+0000 mgr.smithi121.ckiryi (mgr.24385) 7 : cluster [DBG] pgmap v9: 65 pgs: 65 active+clean; 1.0 GiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 48 MiB/s rd, 99 MiB/s wr, 241 op/s 2024-02-15T18:35:31.455 INFO:tasks.workunit.client.1.smithi121.stdout:9/444: sync 2024-02-15T18:35:31.455 INFO:tasks.workunit.client.1.smithi121.stdout:9/445: readlink d7/d26/d4f/d5b/l7e 0 2024-02-15T18:35:31.456 INFO:tasks.workunit.client.1.smithi121.stdout:9/446: mkdir d7/d3a/d8e/d9e 0 2024-02-15T18:35:31.456 INFO:tasks.workunit.client.1.smithi121.stdout:9/447: rename d7/d3a/d8e to d7/d3a/d8e/d9f 22 2024-02-15T18:35:31.458 INFO:tasks.workunit.client.1.smithi121.stdout:9/448: rename d7/d8/d25/d30/f4c to d7/d8/d25/d30/d33/d79/fa0 0 2024-02-15T18:35:31.459 INFO:tasks.workunit.client.1.smithi121.stdout:9/449: mknod d7/d26/d4f/d5b/d77/ca1 0 2024-02-15T18:35:31.459 INFO:tasks.workunit.client.1.smithi121.stdout:9/450: truncate d7/d8/d40/f63 722556 0 2024-02-15T18:35:31.461 INFO:tasks.workunit.client.1.smithi121.stdout:9/451: rmdir d7/d8/d40/d4b 39 2024-02-15T18:35:31.461 INFO:tasks.workunit.client.1.smithi121.stdout:9/452: mknod d7/d3a/d8e/ca2 0 2024-02-15T18:35:31.463 INFO:tasks.workunit.client.1.smithi121.stdout:9/453: truncate d7/d8/d40/f52 3181463 0 2024-02-15T18:35:31.471 INFO:tasks.workunit.client.1.smithi121.stdout:9/454: fsync d7/d8/d25/d30/f32 0 2024-02-15T18:35:31.471 INFO:tasks.workunit.client.1.smithi121.stdout:9/455: fdatasync d7/d8/f10 0 2024-02-15T18:35:31.471 INFO:tasks.workunit.client.1.smithi121.stdout:9/456: mkdir d7/d8/d36/da3 0 2024-02-15T18:35:31.472 INFO:tasks.workunit.client.1.smithi121.stdout:9/457: read d7/d8/f10 [4634209,97440] 0 2024-02-15T18:35:31.473 INFO:tasks.workunit.client.1.smithi121.stdout:9/458: mknod d7/d3a/d42/d56/ca4 0 2024-02-15T18:35:31.473 INFO:tasks.workunit.client.1.smithi121.stdout:9/459: dread - d7/d26/f81 zero size 2024-02-15T18:35:31.491 INFO:tasks.workunit.client.1.smithi121.stdout:9/460: dwrite d7/d8/f16 [0,4194304] 0 2024-02-15T18:35:31.494 INFO:tasks.workunit.client.1.smithi121.stdout:9/461: rename d7/d8/d25/d30/d33/d3e to d7/d8/d25/d30/da5 0 2024-02-15T18:35:31.516 INFO:tasks.workunit.client.1.smithi121.stdout:9/462: dwrite d7/d8/d36/f74 [0,4194304] 0 2024-02-15T18:35:31.516 INFO:tasks.workunit.client.1.smithi121.stdout:9/463: dread - d7/d8/d25/d30/f80 zero size 2024-02-15T18:35:31.518 INFO:tasks.workunit.client.1.smithi121.stdout:9/464: rename d7/d8/f17 to d7/d26/d4f/d5b/d77/fa6 0 2024-02-15T18:35:31.518 INFO:tasks.workunit.client.1.smithi121.stdout:9/465: chown d7/l14 15192 1 2024-02-15T18:35:31.519 INFO:tasks.workunit.client.1.smithi121.stdout:9/466: symlink d7/d3a/d42/d56/d8b/la7 0 2024-02-15T18:35:31.519 INFO:tasks.workunit.client.1.smithi121.stdout:9/467: readlink d7/d3a/d8e/l92 0 2024-02-15T18:35:31.521 INFO:tasks.workunit.client.1.smithi121.stdout:9/468: creat d7/d8/fa8 x:0 0 0 2024-02-15T18:35:31.523 INFO:tasks.workunit.client.1.smithi121.stdout:9/469: dread d7/d8/d25/d30/d33/d3c/f84 [0,4194304] 0 2024-02-15T18:35:31.523 INFO:tasks.workunit.client.1.smithi121.stdout:9/470: dread - d7/d26/f6b zero size 2024-02-15T18:35:31.528 INFO:tasks.workunit.client.1.smithi121.stdout:9/471: rename d7/d3a/d42/d56 to d7/d3a/d8e/d9e/da9 0 2024-02-15T18:35:31.529 INFO:tasks.workunit.client.1.smithi121.stdout:9/472: symlink d7/d8/d40/laa 0 2024-02-15T18:35:31.529 INFO:tasks.workunit.client.1.smithi121.stdout:9/473: fdatasync d7/d3a/d8e/d9e/da9/d8b/f19 0 2024-02-15T18:35:31.530 INFO:tasks.workunit.client.1.smithi121.stdout:9/474: truncate d7/f1c 1553910 0 2024-02-15T18:35:31.530 INFO:tasks.workunit.client.1.smithi121.stdout:9/475: readlink d7/d8/d25/d30/d33/l49 0 2024-02-15T18:35:31.530 INFO:tasks.workunit.client.1.smithi121.stdout:9/476: chown d7/d8/d25/d30/d33/c35 3284895 1 2024-02-15T18:35:31.537 INFO:tasks.workunit.client.1.smithi121.stdout:9/477: dread d7/d26/d4f/d5b/d77/fa6 [0,4194304] 0 2024-02-15T18:35:31.537 INFO:tasks.workunit.client.1.smithi121.stdout:9/478: read d7/d3a/d42/f6c [102202,26111] 0 2024-02-15T18:35:31.539 INFO:tasks.workunit.client.1.smithi121.stdout:9/479: creat d7/d3a/fab x:0 0 0 2024-02-15T18:35:31.553 INFO:tasks.workunit.client.1.smithi121.stdout:9/480: dwrite d7/d3a/d8e/d9e/da9/d8b/f83 [4194304,4194304] 0 2024-02-15T18:35:31.554 INFO:tasks.workunit.client.1.smithi121.stdout:9/481: write d7/d8/f1d [3428046,63144] 0 2024-02-15T18:35:31.556 INFO:tasks.workunit.client.1.smithi121.stdout:9/482: link d7/d8/d25/d30/d33/c39 d7/d8/d40/d4b/cac 0 2024-02-15T18:35:31.557 INFO:tasks.workunit.client.1.smithi121.stdout:9/483: creat d7/d3a/d8e/d9e/da9/d8b/d73/fad x:0 0 0 2024-02-15T18:35:31.558 INFO:tasks.workunit.client.1.smithi121.stdout:9/484: truncate d7/d8/d40/f52 3383810 0 2024-02-15T18:35:31.559 INFO:tasks.workunit.client.1.smithi121.stdout:0/569: sync 2024-02-15T18:35:31.559 INFO:tasks.workunit.client.1.smithi121.stdout:5/446: sync 2024-02-15T18:35:31.560 INFO:tasks.workunit.client.1.smithi121.stdout:6/507: sync 2024-02-15T18:35:31.560 INFO:tasks.workunit.client.1.smithi121.stdout:3/385: sync 2024-02-15T18:35:31.560 INFO:tasks.workunit.client.1.smithi121.stdout:2/250: sync 2024-02-15T18:35:31.560 INFO:tasks.workunit.client.1.smithi121.stdout:1/557: sync 2024-02-15T18:35:31.560 INFO:tasks.workunit.client.1.smithi121.stdout:1/558: chown d0/d6/d7/d29/d77/fad 2558186 1 2024-02-15T18:35:31.560 INFO:tasks.workunit.client.1.smithi121.stdout:0/570: chown d5/d6b/dab/db4/d7d/dd/f22 5643358 1 2024-02-15T18:35:31.560 INFO:tasks.workunit.client.1.smithi121.stdout:0/571: fdatasync d5/d6b/dab/db4/d7d/d2a/f91 0 2024-02-15T18:35:31.560 INFO:tasks.workunit.client.1.smithi121.stdout:8/478: sync 2024-02-15T18:35:31.560 INFO:tasks.workunit.client.1.smithi121.stdout:5/447: write d0/d2/d1e/d4e/d7d/d33/f56 [1462395,19985] 0 2024-02-15T18:35:31.560 INFO:tasks.workunit.client.1.smithi121.stdout:0/572: fsync d5/d6b/dab/db4/d7d/d2a/f91 0 2024-02-15T18:35:31.560 INFO:tasks.workunit.client.1.smithi121.stdout:3/386: write d2/d1a/d27/d7c/f1b [4807851,35951] 0 2024-02-15T18:35:31.560 INFO:tasks.workunit.client.1.smithi121.stdout:4/272: sync 2024-02-15T18:35:31.560 INFO:tasks.workunit.client.1.smithi121.stdout:4/273: chown d5/c15 0 1 2024-02-15T18:35:31.560 INFO:tasks.workunit.client.1.smithi121.stdout:4/274: fsync f1 0 2024-02-15T18:35:31.560 INFO:tasks.workunit.client.1.smithi121.stdout:7/297: sync 2024-02-15T18:35:31.561 INFO:tasks.workunit.client.1.smithi121.stdout:6/508: getdents d7/dc/d2c/d36 0 2024-02-15T18:35:31.562 INFO:tasks.workunit.client.1.smithi121.stdout:6/509: dread - d7/d4b/d60/d73/fb2 zero size 2024-02-15T18:35:31.562 INFO:tasks.workunit.client.1.smithi121.stdout:6/510: write d7/dc/d34/d42/d62/dac/fb3 [306844,79418] 0 2024-02-15T18:35:31.562 INFO:tasks.workunit.client.1.smithi121.stdout:6/511: read - d7/dc/d34/d42/d62/d63/d6f/f88 zero size 2024-02-15T18:35:31.565 INFO:tasks.workunit.client.1.smithi121.stdout:8/479: unlink d0/d73/f85 0 2024-02-15T18:35:31.566 INFO:tasks.workunit.client.1.smithi121.stdout:3/387: stat d2/d1a/d27/d7c/f56 0 2024-02-15T18:35:31.566 INFO:tasks.workunit.client.1.smithi121.stdout:2/251: rename d5/df/d27/d3e/d33 to d5/df/d10/d15/d46 0 2024-02-15T18:35:31.566 INFO:tasks.workunit.client.1.smithi121.stdout:2/252: chown d5/df/d10/l1d 142670 1 2024-02-15T18:35:31.568 INFO:tasks.workunit.client.1.smithi121.stdout:7/298: creat d0/d5/d11/d5c/f64 x:0 0 0 2024-02-15T18:35:31.572 INFO:tasks.workunit.client.1.smithi121.stdout:6/512: dread d7/dc/d2c/d36/f58 [0,4194304] 0 2024-02-15T18:35:31.574 INFO:tasks.workunit.client.1.smithi121.stdout:3/388: write d2/d1a/f1f [851109,50309] 0 2024-02-15T18:35:31.574 INFO:tasks.workunit.client.1.smithi121.stdout:8/480: rmdir d0/d7/de/d7c 39 2024-02-15T18:35:31.574 INFO:tasks.workunit.client.1.smithi121.stdout:8/481: fdatasync d0/d7/f5e 0 2024-02-15T18:35:31.575 INFO:tasks.workunit.client.1.smithi121.stdout:0/573: symlink d5/d6b/dab/db4/d7d/d2a/lc0 0 2024-02-15T18:35:31.575 INFO:tasks.workunit.client.1.smithi121.stdout:8/482: dread d0/d7/d36/d52/d7a/f9a [0,4194304] 0 2024-02-15T18:35:31.575 INFO:tasks.workunit.client.1.smithi121.stdout:2/253: mkdir d5/df/d27/d3e/d47 0 2024-02-15T18:35:31.578 INFO:tasks.workunit.client.1.smithi121.stdout:7/299: rmdir d0/d38/d3f 39 2024-02-15T18:35:31.582 INFO:tasks.workunit.client.1.smithi121.stdout:0/574: creat d5/d6b/dab/db1/fc1 x:0 0 0 2024-02-15T18:35:31.583 INFO:tasks.workunit.client.1.smithi121.stdout:8/483: creat d0/d2a/fae x:0 0 0 2024-02-15T18:35:31.584 INFO:tasks.workunit.client.1.smithi121.stdout:2/254: truncate d5/f8 3869607 0 2024-02-15T18:35:31.585 INFO:tasks.workunit.client.1.smithi121.stdout:9/485: dwrite d7/d8/d36/f74 [0,4194304] 0 2024-02-15T18:35:31.589 INFO:tasks.workunit.client.1.smithi121.stdout:7/300: link d0/d5/d11/d1c/f35 d0/d4a/d5b/f65 0 2024-02-15T18:35:31.589 INFO:tasks.workunit.client.1.smithi121.stdout:7/301: chown d0/d5/f26 776 1 2024-02-15T18:35:31.589 INFO:tasks.workunit.client.1.smithi121.stdout:7/302: fdatasync d0/d5/d11/d1c/f23 0 2024-02-15T18:35:31.590 INFO:tasks.workunit.client.1.smithi121.stdout:8/484: creat d0/d7/de/d7c/faf x:0 0 0 2024-02-15T18:35:31.590 INFO:tasks.workunit.client.1.smithi121.stdout:7/303: dread d0/d5/d11/f48 [0,4194304] 0 2024-02-15T18:35:31.591 INFO:tasks.workunit.client.1.smithi121.stdout:9/486: mknod d7/d3a/d42/cae 0 2024-02-15T18:35:31.592 INFO:tasks.workunit.client.1.smithi121.stdout:6/513: unlink d7/dc/d2c/f5c 0 2024-02-15T18:35:31.593 INFO:tasks.workunit.client.1.smithi121.stdout:8/485: symlink d0/d61/lb0 0 2024-02-15T18:35:31.593 INFO:tasks.workunit.client.1.smithi121.stdout:0/575: dread d5/d6b/dab/db4/d7d/dd/f68 [4194304,4194304] 0 2024-02-15T18:35:31.594 INFO:tasks.workunit.client.1.smithi121.stdout:8/486: stat d0/d7/d36/f81 0 2024-02-15T18:35:31.594 INFO:tasks.workunit.client.1.smithi121.stdout:5/448: dwrite d0/d2/d1e/d4e/d7d/d39/f51 [0,4194304] 0 2024-02-15T18:35:31.594 INFO:tasks.workunit.client.1.smithi121.stdout:7/304: creat d0/d5/f66 x:0 0 0 2024-02-15T18:35:31.595 INFO:tasks.workunit.client.1.smithi121.stdout:9/487: mkdir d7/d8/d40/daf 0 2024-02-15T18:35:31.596 INFO:tasks.workunit.client.1.smithi121.stdout:8/487: mkdir d0/d7/db1 0 2024-02-15T18:35:31.598 INFO:tasks.workunit.client.1.smithi121.stdout:0/576: mknod d5/d6b/dab/db4/d7d/d2a/d73/d97/cc2 0 2024-02-15T18:35:31.598 INFO:tasks.workunit.client.1.smithi121.stdout:7/305: write d0/d38/d3c/f4b [1189790,33551] 0 2024-02-15T18:35:31.598 INFO:tasks.workunit.client.1.smithi121.stdout:1/559: rename d0/d6/d7/d29 to d0/d39/d7d/dc0 0 2024-02-15T18:35:31.601 INFO:tasks.workunit.client.1.smithi121.stdout:6/514: read d7/dc/f5e [2718609,50304] 0 2024-02-15T18:35:31.602 INFO:tasks.workunit.client.1.smithi121.stdout:1/560: unlink d0/d39/f5c 0 2024-02-15T18:35:31.603 INFO:tasks.workunit.client.1.smithi121.stdout:8/488: creat d0/d61/d7b/fb2 x:0 0 0 2024-02-15T18:35:31.605 INFO:tasks.workunit.client.1.smithi121.stdout:1/561: mknod d0/d39/d4b/cc1 0 2024-02-15T18:35:31.605 INFO:tasks.workunit.client.1.smithi121.stdout:1/562: chown d0/d6/d7/d9/d14/f83 1585 1 2024-02-15T18:35:31.606 INFO:tasks.workunit.client.1.smithi121.stdout:0/577: mknod d5/d6b/dab/db4/d7d/d74/d96/cc3 0 2024-02-15T18:35:31.606 INFO:tasks.workunit.client.1.smithi121.stdout:0/578: fdatasync d5/d6b/dab/db4/d7d/dd/f22 0 2024-02-15T18:35:31.607 INFO:tasks.workunit.client.1.smithi121.stdout:7/306: mkdir d0/d5/d11/d67 0 2024-02-15T18:35:31.607 INFO:tasks.workunit.client.1.smithi121.stdout:7/307: chown d0/d5/f9 66236 1 2024-02-15T18:35:31.609 INFO:tasks.workunit.client.1.smithi121.stdout:6/515: creat d7/dc/d2c/d36/fb4 x:0 0 0 2024-02-15T18:35:31.609 INFO:tasks.workunit.client.1.smithi121.stdout:7/308: write d0/d5/f26 [6650772,119552] 0 2024-02-15T18:35:31.612 INFO:tasks.workunit.client.1.smithi121.stdout:0/579: mkdir d5/d6b/dab/db4/d7d/d74/d96/dc4 0 2024-02-15T18:35:31.614 INFO:tasks.workunit.client.1.smithi121.stdout:8/489: dread d0/d7/d36/d8c/d45/f2c [0,4194304] 0 2024-02-15T18:35:31.614 INFO:tasks.workunit.client.1.smithi121.stdout:3/389: rename d2/d1a/d27/d70 to d2/d1a/d22/d85 0 2024-02-15T18:35:31.615 INFO:tasks.workunit.client.1.smithi121.stdout:6/516: symlink d7/dc/d34/d42/lb5 0 2024-02-15T18:35:31.616 INFO:tasks.workunit.client.1.smithi121.stdout:7/309: truncate d0/f29 2936518 0 2024-02-15T18:35:31.617 INFO:tasks.workunit.client.1.smithi121.stdout:0/580: creat d5/d6b/dab/db4/d7d/d2a/d73/d8d/fc5 x:0 0 0 2024-02-15T18:35:31.618 INFO:tasks.workunit.client.1.smithi121.stdout:3/390: truncate d2/d1a/d22/f35 544574 0 2024-02-15T18:35:31.619 INFO:tasks.workunit.client.1.smithi121.stdout:8/490: mknod d0/d61/d66/cb3 0 2024-02-15T18:35:31.621 INFO:tasks.workunit.client.1.smithi121.stdout:7/310: rename d0/l2f to d0/d5/d11/l68 0 2024-02-15T18:35:31.621 INFO:tasks.workunit.client.1.smithi121.stdout:7/311: dread - d0/d5/d4c/f59 zero size 2024-02-15T18:35:31.622 INFO:tasks.workunit.client.1.smithi121.stdout:5/449: dwrite d0/d16/d34/d8f/f90 [0,4194304] 0 2024-02-15T18:35:31.624 INFO:tasks.workunit.client.1.smithi121.stdout:3/391: mknod d2/d1a/d27/d7c/d25/d31/d34/c86 0 2024-02-15T18:35:31.624 INFO:tasks.workunit.client.1.smithi121.stdout:3/392: write d2/d1a/d6b/d7d/f83 [550696,53770] 0 2024-02-15T18:35:31.624 INFO:tasks.workunit.client.1.smithi121.stdout:8/491: creat d0/d7/d36/d52/fb4 x:0 0 0 2024-02-15T18:35:31.624 INFO:tasks.workunit.client.1.smithi121.stdout:8/492: fsync d0/d3/f64 0 2024-02-15T18:35:31.625 INFO:tasks.workunit.client.1.smithi121.stdout:3/393: write d2/d1a/d27/f7f [3651687,116885] 0 2024-02-15T18:35:31.626 INFO:tasks.workunit.client.1.smithi121.stdout:3/394: readlink l0 0 2024-02-15T18:35:31.626 INFO:tasks.workunit.client.1.smithi121.stdout:3/395: truncate d2/d1a/d27/d7c/f7b 276375 0 2024-02-15T18:35:31.626 INFO:tasks.workunit.client.1.smithi121.stdout:0/581: rename d5/d6b/f9d to d5/d6b/dab/db4/d7d/d74/fc6 0 2024-02-15T18:35:31.628 INFO:tasks.workunit.client.1.smithi121.stdout:8/493: unlink d0/d73/fa8 0 2024-02-15T18:35:31.629 INFO:tasks.workunit.client.1.smithi121.stdout:0/582: getdents d5/d6b/dab/db4/d7d/d2a/d4e 0 2024-02-15T18:35:31.631 INFO:tasks.workunit.client.1.smithi121.stdout:7/312: dread d0/d5/d11/d1c/f1f [0,4194304] 0 2024-02-15T18:35:31.633 INFO:tasks.workunit.client.1.smithi121.stdout:0/583: creat d5/d6b/dab/db4/d7d/d2a/d36/d44/fc7 x:0 0 0 2024-02-15T18:35:31.633 INFO:tasks.workunit.client.1.smithi121.stdout:0/584: chown d5/d6b/dab/db4/d7d/d2a/lc0 123394030 1 2024-02-15T18:35:31.637 INFO:tasks.workunit.client.1.smithi121.stdout:1/563: rmdir d0 39 2024-02-15T18:35:31.639 INFO:tasks.workunit.client.1.smithi121.stdout:9/488: truncate d7/d8/d40/f52 3104214 0 2024-02-15T18:35:31.641 INFO:tasks.workunit.client.1.smithi121.stdout:3/396: truncate d2/d1a/d22/f42 3438240 0 2024-02-15T18:35:31.641 INFO:tasks.workunit.client.1.smithi121.stdout:3/397: chown d2/d1a/d27/d7c/d25/d30/f41 124934 1 2024-02-15T18:35:31.641 INFO:tasks.workunit.client.1.smithi121.stdout:7/313: creat d0/f69 x:0 0 0 2024-02-15T18:35:31.644 INFO:tasks.workunit.client.1.smithi121.stdout:3/398: mknod d2/d1a/d27/d63/d81/c87 0 2024-02-15T18:35:31.644 INFO:tasks.workunit.client.1.smithi121.stdout:7/314: symlink d0/l6a 0 2024-02-15T18:35:31.645 INFO:tasks.workunit.client.1.smithi121.stdout:3/399: mkdir d2/d1a/d27/d7c/d25/d88 0 2024-02-15T18:35:31.646 INFO:tasks.workunit.client.1.smithi121.stdout:7/315: symlink d0/d4a/d5b/l6b 0 2024-02-15T18:35:31.647 INFO:tasks.workunit.client.1.smithi121.stdout:3/400: read d2/f7 [4479219,4523] 0 2024-02-15T18:35:31.647 INFO:tasks.workunit.client.1.smithi121.stdout:1/564: rename d0/d39/c45 to d0/d6/d7/d9/d51/d5f/cc2 0 2024-02-15T18:35:31.649 INFO:tasks.workunit.client.1.smithi121.stdout:3/401: creat d2/d1a/f89 x:0 0 0 2024-02-15T18:35:31.649 INFO:tasks.workunit.client.1.smithi121.stdout:6/517: dwrite d7/dc/f4c [4194304,4194304] 0 2024-02-15T18:35:31.649 INFO:tasks.workunit.client.1.smithi121.stdout:5/450: dwrite d0/d2/f7b [4194304,4194304] 0 2024-02-15T18:35:31.651 INFO:tasks.workunit.client.1.smithi121.stdout:1/565: mknod d0/d39/d4b/cc3 0 2024-02-15T18:35:31.651 INFO:tasks.workunit.client.1.smithi121.stdout:1/566: stat d0/d6/f1a 0 2024-02-15T18:35:31.651 INFO:tasks.workunit.client.1.smithi121.stdout:6/518: mknod d7/dc/d2c/cb6 0 2024-02-15T18:35:31.655 INFO:tasks.workunit.client.1.smithi121.stdout:1/567: dread d0/d6/d7/d9/d14/f8a [0,4194304] 0 2024-02-15T18:35:31.656 INFO:tasks.workunit.client.1.smithi121.stdout:1/568: fdatasync d0/d6/d7/f4c 0 2024-02-15T18:35:31.656 INFO:tasks.workunit.client.1.smithi121.stdout:1/569: readlink d0/d39/la7 0 2024-02-15T18:35:31.657 INFO:tasks.workunit.client.1.smithi121.stdout:1/570: creat d0/d6/d7/fc4 x:0 0 0 2024-02-15T18:35:31.657 INFO:tasks.workunit.client.1.smithi121.stdout:3/402: dread d2/d1a/d27/d7c/f2f [0,4194304] 0 2024-02-15T18:35:31.660 INFO:tasks.workunit.client.1.smithi121.stdout:1/571: unlink d0/d6/d7/c9b 0 2024-02-15T18:35:31.661 INFO:tasks.workunit.client.1.smithi121.stdout:1/572: fsync d0/d6/d7/d9/d14/f6b 0 2024-02-15T18:35:31.662 INFO:tasks.workunit.client.1.smithi121.stdout:5/451: symlink d0/d2/d1e/d4e/d7d/d40/l99 0 2024-02-15T18:35:31.667 INFO:tasks.workunit.client.1.smithi121.stdout:6/519: creat d7/dc/d34/d42/d62/d63/d6f/d89/fb7 x:0 0 0 2024-02-15T18:35:31.671 INFO:tasks.workunit.client.1.smithi121.stdout:3/403: mknod d2/d1a/d27/d7c/d25/d88/c8a 0 2024-02-15T18:35:31.673 INFO:tasks.workunit.client.1.smithi121.stdout:3/404: symlink d2/d1a/d6b/d72/l8b 0 2024-02-15T18:35:31.675 INFO:tasks.workunit.client.1.smithi121.stdout:3/405: mknod d2/d1a/d27/d7c/c8c 0 2024-02-15T18:35:31.675 INFO:tasks.workunit.client.1.smithi121.stdout:3/406: read - d2/d1a/d27/d7c/d25/d31/d34/d4a/f5c zero size 2024-02-15T18:35:31.677 INFO:tasks.workunit.client.1.smithi121.stdout:3/407: creat d2/d1a/d27/d7c/d25/d31/d34/d4a/f8d x:0 0 0 2024-02-15T18:35:31.677 INFO:tasks.workunit.client.1.smithi121.stdout:3/408: chown d2/d1a/d27/d7c/d25/d31/d34/c86 59 1 2024-02-15T18:35:31.677 INFO:tasks.workunit.client.1.smithi121.stdout:3/409: chown d2/cf 0 1 2024-02-15T18:35:31.678 INFO:tasks.workunit.client.1.smithi121.stdout:3/410: creat d2/d1a/d6b/d7d/f8e x:0 0 0 2024-02-15T18:35:31.687 INFO:tasks.workunit.client.1.smithi121.stdout:1/573: dwrite d0/d6/d7/fc4 [0,4194304] 0 2024-02-15T18:35:31.688 INFO:tasks.workunit.client.1.smithi121.stdout:1/574: read d0/d6/d7/d50/d7e/f85 [3461713,124196] 0 2024-02-15T18:35:31.697 INFO:tasks.workunit.client.1.smithi121.stdout:6/520: dwrite d7/dc/d34/d42/d62/d63/d6f/fb1 [0,4194304] 0 2024-02-15T18:35:31.699 INFO:tasks.workunit.client.1.smithi121.stdout:5/452: dwrite d0/d16/d34/d8f/f90 [0,4194304] 0 2024-02-15T18:35:31.700 INFO:tasks.workunit.client.1.smithi121.stdout:5/453: symlink d0/d2/d1e/d4e/d7d/d33/l9a 0 2024-02-15T18:35:31.702 INFO:tasks.workunit.client.1.smithi121.stdout:6/521: unlink d7/dc/c7e 0 2024-02-15T18:35:31.703 INFO:tasks.workunit.client.1.smithi121.stdout:6/522: write d7/dc/d2c/d36/f38 [618474,84009] 0 2024-02-15T18:35:31.703 INFO:tasks.workunit.client.1.smithi121.stdout:5/454: rmdir d0/d2/d1e/d4e/d7d/d47/d5a 39 2024-02-15T18:35:31.707 INFO:tasks.workunit.client.1.smithi121.stdout:5/455: creat d0/f9b x:0 0 0 2024-02-15T18:35:31.709 INFO:tasks.workunit.client.1.smithi121.stdout:5/456: creat d0/d2/d70/d88/f9c x:0 0 0 2024-02-15T18:35:31.709 INFO:tasks.workunit.client.1.smithi121.stdout:5/457: dread - d0/f4b zero size 2024-02-15T18:35:31.717 INFO:tasks.workunit.client.1.smithi121.stdout:5/458: dread d0/d16/d34/d8f/f90 [0,4194304] 0 2024-02-15T18:35:31.721 INFO:tasks.workunit.client.1.smithi121.stdout:6/523: dwrite d7/dc/d2c/d36/f9d [0,4194304] 0 2024-02-15T18:35:31.722 INFO:tasks.workunit.client.1.smithi121.stdout:6/524: creat d7/dc/d2c/d36/fb8 x:0 0 0 2024-02-15T18:35:31.725 INFO:tasks.workunit.client.1.smithi121.stdout:5/459: dread d0/d2/d18/f29 [0,4194304] 0 2024-02-15T18:35:31.727 INFO:tasks.workunit.client.1.smithi121.stdout:5/460: getdents d0/d2/d1e/d4e/d7d/d47 0 2024-02-15T18:35:31.732 INFO:tasks.workunit.client.1.smithi121.stdout:6/525: rmdir d7 39 2024-02-15T18:35:31.732 INFO:tasks.workunit.client.1.smithi121.stdout:6/526: dread - d7/dc/d2c/d36/fb4 zero size 2024-02-15T18:35:31.733 INFO:tasks.workunit.client.1.smithi121.stdout:6/527: read d7/dc/d34/f43 [177338,22026] 0 2024-02-15T18:35:31.733 INFO:tasks.workunit.client.1.smithi121.stdout:6/528: read d7/dc/f90 [3273268,52008] 0 2024-02-15T18:35:31.742 INFO:tasks.workunit.client.1.smithi121.stdout:5/461: dwrite d0/f5f [0,4194304] 0 2024-02-15T18:35:31.744 INFO:tasks.workunit.client.1.smithi121.stdout:5/462: unlink d0/d16/d34/f44 0 2024-02-15T18:35:31.746 INFO:tasks.workunit.client.1.smithi121.stdout:5/463: dread d0/f93 [0,4194304] 0 2024-02-15T18:35:31.746 INFO:tasks.workunit.client.1.smithi121.stdout:5/464: truncate d0/d16/f94 1353705 0 2024-02-15T18:35:31.746 INFO:tasks.workunit.client.1.smithi121.stdout:5/465: dread d0/d2/d18/f5b [0,4194304] 0 2024-02-15T18:35:31.749 INFO:tasks.workunit.client.1.smithi121.stdout:5/466: rename d0/d2/d1e/d4e/d7d/d33 to d0/d2/d1e/d4e/d7d/d9d 0 2024-02-15T18:35:31.751 INFO:tasks.workunit.client.1.smithi121.stdout:5/467: rename d0/d16/f2b to d0/d2/d18/f9e 0 2024-02-15T18:35:31.751 INFO:tasks.workunit.client.1.smithi121.stdout:5/468: readlink d0/d2/d1e/l1f 0 2024-02-15T18:35:31.751 INFO:tasks.workunit.client.1.smithi121.stdout:5/469: chown d0/d16/d34 67728626 1 2024-02-15T18:35:31.751 INFO:tasks.workunit.client.1.smithi121.stdout:5/470: write d0/d2/d1e/d4e/d7d/d9d/d4c/f8d [585288,31590] 0 2024-02-15T18:35:31.751 INFO:tasks.workunit.client.1.smithi121.stdout:5/471: dread - d0/d2/d70/f92 zero size 2024-02-15T18:35:31.752 INFO:tasks.workunit.client.1.smithi121.stdout:5/472: mknod d0/d2/d1e/d4e/d7d/d40/c9f 0 2024-02-15T18:35:31.752 INFO:tasks.workunit.client.1.smithi121.stdout:5/473: stat d0/d16/l67 0 2024-02-15T18:35:31.753 INFO:tasks.workunit.client.1.smithi121.stdout:5/474: read - d0/d16/d34/d8f/f97 zero size 2024-02-15T18:35:31.768 INFO:tasks.workunit.client.1.smithi121.stdout:5/475: dwrite d0/d16/f98 [0,4194304] 0 2024-02-15T18:35:31.771 INFO:tasks.workunit.client.1.smithi121.stdout:5/476: link d0/d2/d1e/d4e/d7d/d9d/f59 d0/d2/d1e/d4e/d7d/d47/d5a/fa0 0 2024-02-15T18:35:31.772 INFO:tasks.workunit.client.1.smithi121.stdout:5/477: mkdir d0/d16/da1 0 2024-02-15T18:35:31.777 INFO:tasks.workunit.client.1.smithi121.stdout:5/478: creat d0/d2/d1e/fa2 x:0 0 0 2024-02-15T18:35:31.777 INFO:tasks.workunit.client.1.smithi121.stdout:5/479: write d0/d16/f94 [1598715,50571] 0 2024-02-15T18:35:31.778 INFO:tasks.workunit.client.1.smithi121.stdout:5/480: mknod d0/d2/d1e/ca3 0 2024-02-15T18:35:31.779 INFO:tasks.workunit.client.1.smithi121.stdout:5/481: rmdir d0/d2/d1e/d4e/d7d/d9d 39 2024-02-15T18:35:31.779 INFO:tasks.workunit.client.1.smithi121.stdout:5/482: chown d0/d2/d1e/d4e/d7d/d39/c66 1048305537 1 2024-02-15T18:35:31.794 INFO:tasks.workunit.client.1.smithi121.stdout:5/483: dwrite d0/d2/d1e/d4e/d7d/d47/f64 [0,4194304] 0 2024-02-15T18:35:31.796 INFO:tasks.workunit.client.1.smithi121.stdout:5/484: mkdir d0/d16/d34/da4 0 2024-02-15T18:35:31.804 INFO:tasks.workunit.client.1.smithi121.stdout:5/485: dread d0/f82 [0,4194304] 0 2024-02-15T18:35:31.811 INFO:tasks.workunit.client.1.smithi121.stdout:5/486: dread d0/f5f [0,4194304] 0 2024-02-15T18:35:31.812 INFO:tasks.workunit.client.1.smithi121.stdout:5/487: link d0/d2/d1e/d4e/d7d/d39/f51 d0/d2/d1e/d4e/d7d/d47/fa5 0 2024-02-15T18:35:31.813 INFO:tasks.workunit.client.1.smithi121.stdout:5/488: readlink d0/d2/d1e/l1f 0 2024-02-15T18:35:31.813 INFO:tasks.workunit.client.1.smithi121.stdout:5/489: creat d0/d16/d34/d8f/fa6 x:0 0 0 2024-02-15T18:35:31.814 INFO:tasks.workunit.client.1.smithi121.stdout:5/490: mkdir d0/d2/d1e/d4e/d7d/d47/d5a/da7 0 2024-02-15T18:35:31.815 INFO:tasks.workunit.client.1.smithi121.stdout:5/491: getdents d0/d2/d1e/d4e/d7d/d9d 0 2024-02-15T18:35:31.833 INFO:tasks.workunit.client.1.smithi121.stdout:5/492: dwrite d0/d16/f98 [0,4194304] 0 2024-02-15T18:35:31.834 INFO:tasks.workunit.client.1.smithi121.stdout:5/493: creat d0/d16/da1/fa8 x:0 0 0 2024-02-15T18:35:31.835 INFO:tasks.workunit.client.1.smithi121.stdout:5/494: symlink d0/d2/d1e/d4e/d7d/d40/la9 0 2024-02-15T18:35:31.835 INFO:tasks.workunit.client.1.smithi121.stdout:5/495: write d0/d2/d1e/d4e/d7d/f2c [1761335,1903] 0 2024-02-15T18:35:31.851 INFO:tasks.workunit.client.1.smithi121.stdout:5/496: dwrite d0/d2/d1e/d4e/d7d/d9d/d4c/f95 [0,4194304] 0 2024-02-15T18:35:31.853 INFO:tasks.workunit.client.1.smithi121.stdout:5/497: mkdir d0/d2/d1e/d4e/d7d/d9d/d4c/daa 0 2024-02-15T18:35:31.853 INFO:tasks.workunit.client.1.smithi121.stdout:5/498: readlink d0/d2/d1e/d4e/d7d/d47/l65 0 2024-02-15T18:35:31.853 INFO:tasks.workunit.client.1.smithi121.stdout:5/499: stat d0/d2/d1e/d4e/d7d/l2a 0 2024-02-15T18:35:31.854 INFO:tasks.workunit.client.1.smithi121.stdout:5/500: symlink d0/d2/d1e/d4e/d7d/d9d/d4c/daa/lab 0 2024-02-15T18:35:31.854 INFO:tasks.workunit.client.1.smithi121.stdout:5/501: dread - d0/f4b zero size 2024-02-15T18:35:31.855 INFO:tasks.workunit.client.1.smithi121.stdout:5/502: write d0/d2/d1e/d4e/d7d/d39/f53 [758156,86481] 0 2024-02-15T18:35:31.856 INFO:tasks.workunit.client.1.smithi121.stdout:5/503: symlink d0/d2/d1e/d4e/d7d/d39/lac 0 2024-02-15T18:35:31.856 INFO:tasks.workunit.client.1.smithi121.stdout:5/504: truncate d0/d16/f94 2195629 0 2024-02-15T18:35:31.856 INFO:tasks.workunit.client.1.smithi121.stdout:5/505: fdatasync d0/d16/f68 0 2024-02-15T18:35:31.861 INFO:tasks.workunit.client.1.smithi121.stdout:5/506: dread d0/d2/f14 [0,4194304] 0 2024-02-15T18:35:31.862 INFO:tasks.workunit.client.1.smithi121.stdout:5/507: symlink d0/d2/d1e/d4e/d7d/d47/d5a/lad 0 2024-02-15T18:35:31.862 INFO:tasks.workunit.client.1.smithi121.stdout:5/508: creat d0/d2/d1e/d4e/d7d/d40/d62/fae x:0 0 0 2024-02-15T18:35:31.864 INFO:tasks.workunit.client.1.smithi121.stdout:5/509: rmdir d0/d16 39 2024-02-15T18:35:31.864 INFO:tasks.workunit.client.1.smithi121.stdout:5/510: chown d0/d2/c10 2 1 2024-02-15T18:35:31.866 INFO:tasks.workunit.client.1.smithi121.stdout:5/511: rename d0/d2/d1e/d4e/d7d/d40/d62/f73 to d0/d2/faf 0 2024-02-15T18:35:31.884 INFO:tasks.workunit.client.1.smithi121.stdout:5/512: dwrite d0/d16/d34/d8f/fa6 [0,4194304] 0 2024-02-15T18:35:31.884 INFO:tasks.workunit.client.1.smithi121.stdout:5/513: fsync d0/d2/d1e/d4e/d7d/d9d/f56 0 2024-02-15T18:35:31.893 INFO:tasks.workunit.client.1.smithi121.stdout:5/514: link d0/d2/d18/l57 d0/d2/d18/lb0 0 2024-02-15T18:35:31.894 INFO:tasks.workunit.client.1.smithi121.stdout:5/515: truncate d0/d2/d18/f9e 243767 0 2024-02-15T18:35:31.908 INFO:tasks.workunit.client.1.smithi121.stdout:5/516: dwrite d0/d16/d34/d8f/f90 [4194304,4194304] 0 2024-02-15T18:35:31.910 INFO:tasks.workunit.client.1.smithi121.stdout:5/517: mkdir d0/d2/d1e/d4e/d7d/d40/d62/db1 0 2024-02-15T18:35:31.911 INFO:tasks.workunit.client.1.smithi121.stdout:5/518: symlink d0/d2/d1e/d4e/d7d/d9d/lb2 0 2024-02-15T18:35:31.912 INFO:tasks.workunit.client.1.smithi121.stdout:5/519: truncate d0/d2/d1e/d4e/d7d/d9d/f59 1162057 0 2024-02-15T18:35:31.913 INFO:tasks.workunit.client.1.smithi121.stdout:5/520: write d0/d2/faf [3876208,98226] 0 2024-02-15T18:35:31.930 INFO:tasks.workunit.client.1.smithi121.stdout:5/521: dwrite d0/d2/faf [0,4194304] 0 2024-02-15T18:35:31.932 INFO:tasks.workunit.client.1.smithi121.stdout:5/522: mkdir d0/d2/d1e/d4e/d7d/d47/d5a/db3 0 2024-02-15T18:35:31.932 INFO:tasks.workunit.client.1.smithi121.stdout:5/523: chown d0/d2/d1e/d4e 1571243 1 2024-02-15T18:35:31.944 INFO:tasks.workunit.client.1.smithi121.stdout:5/524: dwrite d0/d2/f7c [0,4194304] 0 2024-02-15T18:35:31.945 INFO:tasks.workunit.client.1.smithi121.stdout:5/525: write d0/d2/d18/f29 [2926603,30610] 0 2024-02-15T18:35:31.966 INFO:tasks.workunit.client.1.smithi121.stdout:5/526: dwrite d0/d2/d1e/d4e/d7d/d9d/f59 [0,4194304] 0 2024-02-15T18:35:31.968 INFO:tasks.workunit.client.1.smithi121.stdout:5/527: creat d0/d2/d70/d88/fb4 x:0 0 0 2024-02-15T18:35:31.972 INFO:tasks.workunit.client.1.smithi121.stdout:5/528: unlink d0/d2/d1e/d4e/d7d/d40/la9 0 2024-02-15T18:35:31.972 INFO:tasks.workunit.client.1.smithi121.stdout:5/529: chown d0/d2/l3d 11 1 2024-02-15T18:35:31.973 INFO:tasks.workunit.client.1.smithi121.stdout:5/530: mkdir d0/d16/d7a/db5 0 2024-02-15T18:35:31.973 INFO:tasks.workunit.client.1.smithi121.stdout:5/531: readlink d0/d2/d1e/l63 0 2024-02-15T18:35:31.975 INFO:tasks.workunit.client.1.smithi121.stdout:5/532: symlink d0/d2/d1e/d4e/d7d/lb6 0 2024-02-15T18:35:31.975 INFO:tasks.workunit.client.1.smithi121.stdout:5/533: dread - d0/d2/d70/d88/fb4 zero size 2024-02-15T18:35:31.976 INFO:tasks.workunit.client.1.smithi121.stdout:5/534: unlink d0/d2/d1e/ca3 0 2024-02-15T18:35:31.978 INFO:tasks.workunit.client.1.smithi121.stdout:5/535: link d0/c1c d0/d2/d1e/cb7 0 2024-02-15T18:35:31.994 INFO:tasks.workunit.client.1.smithi121.stdout:5/536: dwrite d0/f5f [4194304,4194304] 0 2024-02-15T18:35:32.009 INFO:tasks.workunit.client.1.smithi121.stdout:5/537: dwrite d0/d2/d18/f27 [0,4194304] 0 2024-02-15T18:35:32.010 INFO:tasks.workunit.client.1.smithi121.stdout:5/538: rmdir d0/d2/d70/d88 39 2024-02-15T18:35:32.012 INFO:tasks.workunit.client.1.smithi121.stdout:5/539: creat d0/d2/d1e/d4e/d7d/fb8 x:0 0 0 2024-02-15T18:35:32.014 INFO:tasks.workunit.client.1.smithi121.stdout:5/540: rmdir d0/d2/d70 39 2024-02-15T18:35:32.017 INFO:tasks.workunit.client.1.smithi121.stdout:5/541: read d0/d2/f7c [3859134,15637] 0 2024-02-15T18:35:32.018 INFO:tasks.workunit.client.1.smithi121.stdout:5/542: write d0/d2/d1e/d4e/d7d/d47/f64 [526567,64871] 0 2024-02-15T18:35:32.033 INFO:tasks.workunit.client.1.smithi121.stdout:5/543: dwrite d0/d2/d18/f5b [0,4194304] 0 2024-02-15T18:35:32.033 INFO:tasks.workunit.client.1.smithi121.stdout:5/544: getdents d0/d2/d1e/d4e/d7d/d40/d62/db1 0 2024-02-15T18:35:32.035 INFO:tasks.workunit.client.1.smithi121.stdout:5/545: mkdir d0/d2/d1e/d4e/d7d/d40/db9 0 2024-02-15T18:35:32.035 INFO:tasks.workunit.client.1.smithi121.stdout:5/546: mknod d0/d16/da1/cba 0 2024-02-15T18:35:32.036 INFO:tasks.workunit.client.1.smithi121.stdout:5/547: mknod d0/d2/d1e/d4e/d7d/d47/d5a/cbb 0 2024-02-15T18:35:32.051 INFO:tasks.workunit.client.1.smithi121.stdout:5/548: dwrite d0/d2/d1e/d4e/d7d/d9d/d4c/f8b [0,4194304] 0 2024-02-15T18:35:32.060 INFO:tasks.workunit.client.1.smithi121.stdout:5/549: dread d0/fb [4194304,4194304] 0 2024-02-15T18:35:32.060 INFO:tasks.workunit.client.1.smithi121.stdout:5/550: fdatasync d0/d2/d70/f92 0 2024-02-15T18:35:32.069 INFO:tasks.workunit.client.1.smithi121.stdout:5/551: dread d0/d2/d1e/d4e/d7d/d47/fa5 [0,4194304] 0 2024-02-15T18:35:32.071 INFO:tasks.workunit.client.1.smithi121.stdout:5/552: mkdir d0/d2/d1e/d4e/d7d/d47/d5a/dbc 0 2024-02-15T18:35:32.072 INFO:tasks.workunit.client.1.smithi121.stdout:5/553: creat d0/d2/d1e/d4e/d7d/d47/d5a/da7/fbd x:0 0 0 2024-02-15T18:35:32.072 INFO:tasks.workunit.client.1.smithi121.stdout:5/554: write d0/d2/d18/f31 [376432,113437] 0 2024-02-15T18:35:32.073 INFO:tasks.workunit.client.1.smithi121.stdout:5/555: fsync d0/d2/d1e/fa2 0 2024-02-15T18:35:32.086 INFO:tasks.workunit.client.1.smithi121.stdout:5/556: dwrite d0/d2/f7b [8388608,4194304] 0 2024-02-15T18:35:32.086 INFO:tasks.workunit.client.1.smithi121.stdout:5/557: readlink d0/d16/d34/l49 0 2024-02-15T18:35:32.086 INFO:tasks.workunit.client.1.smithi121.stdout:5/558: fsync d0/d2/d1e/d4e/d7d/d40/f79 0 2024-02-15T18:35:32.086 INFO:tasks.workunit.client.1.smithi121.stdout:5/559: fdatasync d0/d2/faf 0 2024-02-15T18:35:32.087 INFO:tasks.workunit.client.1.smithi121.stdout:5/560: fsync d0/d2/d1e/d4e/d7d/d9d/d4c/f95 0 2024-02-15T18:35:32.087 INFO:tasks.workunit.client.1.smithi121.stdout:5/561: fdatasync d0/f4b 0 2024-02-15T18:35:32.087 INFO:tasks.workunit.client.1.smithi121.stdout:5/562: rmdir d0/d16/d7a 39 2024-02-15T18:35:32.088 INFO:tasks.workunit.client.1.smithi121.stdout:5/563: mknod d0/d16/d7a/cbe 0 2024-02-15T18:35:32.094 INFO:tasks.workunit.client.1.smithi121.stdout:5/564: dread d0/d2/d1e/d4e/d7d/d40/f78 [0,4194304] 0 2024-02-15T18:35:32.096 INFO:tasks.workunit.client.1.smithi121.stdout:5/565: write d0/d2/f7b [1835535,129984] 0 2024-02-15T18:35:32.113 INFO:tasks.workunit.client.1.smithi121.stdout:5/566: dwrite d0/d2/d18/f27 [0,4194304] 0 2024-02-15T18:35:32.113 INFO:tasks.workunit.client.1.smithi121.stdout:5/567: fdatasync d0/d16/d34/f6a 0 2024-02-15T18:35:32.113 INFO:tasks.workunit.client.1.smithi121.stdout:5/568: write d0/d2/d1e/d4e/d7d/d47/d5a/fa0 [5171458,93367] 0 2024-02-15T18:35:32.113 INFO:tasks.workunit.client.1.smithi121.stdout:5/569: stat d0/d2/d1e/d4e/d7d/d39/f53 0 2024-02-15T18:35:32.113 INFO:tasks.workunit.client.1.smithi121.stdout:5/570: creat d0/d16/d34/d8f/fbf x:0 0 0 2024-02-15T18:35:32.113 INFO:tasks.workunit.client.1.smithi121.stdout:5/571: readlink d0/d16/d34/l49 0 2024-02-15T18:35:32.132 INFO:tasks.workunit.client.1.smithi121.stdout:5/572: dwrite d0/f93 [0,4194304] 0 2024-02-15T18:35:32.132 INFO:tasks.workunit.client.1.smithi121.stdout:5/573: dread - d0/d2/d70/f92 zero size 2024-02-15T18:35:32.135 INFO:tasks.workunit.client.1.smithi121.stdout:5/574: creat d0/d2/d70/fc0 x:0 0 0 2024-02-15T18:35:32.137 INFO:tasks.workunit.client.1.smithi121.stdout:5/575: rename d0/d2/d70/f92 to d0/d2/d1e/d4e/d7d/d40/d62/db1/fc1 0 2024-02-15T18:35:32.155 INFO:tasks.workunit.client.1.smithi121.stdout:5/576: dwrite d0/d2/fd [4194304,4194304] 0 2024-02-15T18:35:32.156 INFO:tasks.workunit.client.1.smithi121.stdout:5/577: rename d0/d2/c6d to d0/d2/d1e/d4e/d7d/d47/d5a/db3/cc2 0 2024-02-15T18:35:32.156 INFO:tasks.workunit.client.1.smithi121.stdout:5/578: readlink d0/d16/d34/l49 0 2024-02-15T18:35:32.166 INFO:tasks.workunit.client.1.smithi121.stdout:5/579: dread d0/d2/d1e/d4e/d7d/d47/d5a/fa0 [0,4194304] 0 2024-02-15T18:35:32.168 INFO:tasks.workunit.client.1.smithi121.stdout:5/580: write d0/d16/d34/d8f/fa6 [2284611,80763] 0 2024-02-15T18:35:32.169 INFO:tasks.workunit.client.1.smithi121.stdout:5/581: creat d0/d2/d1e/d4e/d7d/d40/d62/db1/fc3 x:0 0 0 2024-02-15T18:35:32.194 INFO:tasks.workunit.client.1.smithi121.stdout:5/582: dwrite d0/d16/d34/d8f/fa6 [0,4194304] 0 2024-02-15T18:35:32.210 INFO:tasks.workunit.client.1.smithi121.stdout:5/583: dwrite d0/d2/d1e/d4e/d7d/d39/f51 [0,4194304] 0 2024-02-15T18:35:32.212 INFO:tasks.workunit.client.1.smithi121.stdout:5/584: creat d0/d2/d1e/d4e/d7d/d9d/fc4 x:0 0 0 2024-02-15T18:35:32.212 INFO:tasks.workunit.client.1.smithi121.stdout:5/585: dread - d0/d2/d70/d88/f96 zero size 2024-02-15T18:35:32.212 INFO:tasks.workunit.client.1.smithi121.stdout:5/586: write d0/f9b [417582,66776] 0 2024-02-15T18:35:32.212 INFO:tasks.workunit.client.1.smithi121.stdout:5/587: stat d0/d2/d1e/d4e/d7d/d47/fa5 0 2024-02-15T18:35:32.226 INFO:tasks.workunit.client.1.smithi121.stdout:5/588: dwrite d0/d2/d70/f8a [0,4194304] 0 2024-02-15T18:35:32.226 INFO:tasks.workunit.client.1.smithi121.stdout:5/589: chown d0/fb 2450774 1 2024-02-15T18:35:32.243 INFO:tasks.workunit.client.1.smithi121.stdout:5/590: dwrite d0/d16/d34/f45 [0,4194304] 0 2024-02-15T18:35:32.245 INFO:tasks.workunit.client.1.smithi121.stdout:5/591: dread d0/d2/d1e/d4e/d7d/d9d/d4c/f8d [0,4194304] 0 2024-02-15T18:35:32.246 INFO:tasks.workunit.client.1.smithi121.stdout:5/592: getdents d0/d16/d7a 0 2024-02-15T18:35:33.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:32.593960+0000 mgr.smithi121.ckiryi (mgr.24385) 8 : cluster [DBG] pgmap v10: 65 pgs: 65 active+clean; 1.0 GiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 44 MiB/s rd, 91 MiB/s wr, 221 op/s 2024-02-15T18:35:33.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:32.593960+0000 mgr.smithi121.ckiryi (mgr.24385) 8 : cluster [DBG] pgmap v10: 65 pgs: 65 active+clean; 1.0 GiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 44 MiB/s rd, 91 MiB/s wr, 221 op/s 2024-02-15T18:35:35.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:33.692735+0000 mon.smithi038 (mon.0) 595 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:35.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:34.001035+0000 mon.smithi038 (mon.0) 596 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:35.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:34.007554+0000 mon.smithi038 (mon.0) 597 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:35.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:34.014798+0000 mon.smithi038 (mon.0) 598 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:35.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:34.026278+0000 mon.smithi038 (mon.0) 599 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:35.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:34.033861+0000 mon.smithi038 (mon.0) 600 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:35.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:35:34.043924+0000 mgr.smithi121.ckiryi (mgr.24385) 9 : cephadm [INF] Reconfiguring prometheus.smithi038 (dependencies changed)... 2024-02-15T18:35:35.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:35:34.055793+0000 mgr.smithi121.ckiryi (mgr.24385) 10 : cephadm [INF] Reconfiguring daemon prometheus.smithi038 on smithi038 2024-02-15T18:35:35.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:34.594351+0000 mgr.smithi121.ckiryi (mgr.24385) 11 : cluster [DBG] pgmap v11: 65 pgs: 65 active+clean; 1.0 GiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 44 MiB/s rd, 91 MiB/s wr, 221 op/s 2024-02-15T18:35:35.029 INFO:tasks.workunit.client.1.smithi121.stdout:6/529: sync 2024-02-15T18:35:35.029 INFO:tasks.workunit.client.1.smithi121.stdout:5/593: sync 2024-02-15T18:35:35.029 INFO:tasks.workunit.client.1.smithi121.stdout:5/594: stat d0/d2/d18 0 2024-02-15T18:35:35.029 INFO:tasks.workunit.client.1.smithi121.stdout:7/316: sync 2024-02-15T18:35:35.030 INFO:tasks.workunit.client.1.smithi121.stdout:9/489: sync 2024-02-15T18:35:35.030 INFO:tasks.workunit.client.1.smithi121.stdout:3/411: sync 2024-02-15T18:35:35.030 INFO:tasks.workunit.client.1.smithi121.stdout:3/412: fsync d2/d5a/f5f 0 2024-02-15T18:35:35.030 INFO:tasks.workunit.client.1.smithi121.stdout:3/413: truncate d2/d1a/d54/f69 290799 0 2024-02-15T18:35:35.032 INFO:tasks.workunit.client.1.smithi121.stdout:0/585: sync 2024-02-15T18:35:35.032 INFO:tasks.workunit.client.1.smithi121.stdout:1/575: sync 2024-02-15T18:35:35.032 INFO:tasks.workunit.client.1.smithi121.stdout:1/576: fsync d0/d6/d7/d9/d14/d48/f60 0 2024-02-15T18:35:35.033 INFO:tasks.workunit.client.1.smithi121.stdout:2/255: sync 2024-02-15T18:35:35.033 INFO:tasks.workunit.client.1.smithi121.stdout:4/275: sync 2024-02-15T18:35:35.033 INFO:tasks.workunit.client.1.smithi121.stdout:4/276: stat d5/d9 0 2024-02-15T18:35:35.033 INFO:tasks.workunit.client.1.smithi121.stdout:8/494: sync 2024-02-15T18:35:35.041 INFO:tasks.workunit.client.1.smithi121.stdout:4/277: dread d5/d9/d1d/f2f [0,4194304] 0 2024-02-15T18:35:35.044 INFO:tasks.workunit.client.1.smithi121.stdout:0/586: write d5/f40 [2846326,4449] 0 2024-02-15T18:35:35.044 INFO:tasks.workunit.client.1.smithi121.stdout:6/530: dread d7/dc/d2c/d36/f9d [0,4194304] 0 2024-02-15T18:35:35.044 INFO:tasks.workunit.client.1.smithi121.stdout:6/531: chown d7/dc/d34/d42 120035148 1 2024-02-15T18:35:35.044 INFO:tasks.workunit.client.1.smithi121.stdout:6/532: readlink d7/l8b 0 2024-02-15T18:35:35.048 INFO:tasks.workunit.client.1.smithi121.stdout:4/278: read d5/d9/d19/d28/f21 [344167,101602] 0 2024-02-15T18:35:35.048 INFO:tasks.workunit.client.1.smithi121.stdout:4/279: write d5/d31/d3d/d4f/d51/f47 [985028,126604] 0 2024-02-15T18:35:35.049 INFO:tasks.workunit.client.1.smithi121.stdout:1/577: dread d0/d6/d7/d50/d7e/f85 [0,4194304] 0 2024-02-15T18:35:35.058 INFO:tasks.workunit.client.1.smithi121.stdout:5/595: getdents d0/d16/d34 0 2024-02-15T18:35:35.058 INFO:tasks.workunit.client.1.smithi121.stdout:9/490: symlink d7/d26/d4f/lb0 0 2024-02-15T18:35:35.058 INFO:tasks.workunit.client.1.smithi121.stdout:9/491: chown d7/d26/d4f/l94 7656943 1 2024-02-15T18:35:35.059 INFO:tasks.workunit.client.1.smithi121.stdout:8/495: rename d0/d7/d36/l58 to d0/d7/db1/lb5 0 2024-02-15T18:35:35.059 INFO:tasks.workunit.client.1.smithi121.stdout:7/317: symlink d0/d5/d11/d67/l6c 0 2024-02-15T18:35:35.059 INFO:tasks.workunit.client.1.smithi121.stdout:0/587: mkdir d5/d6b/dab/db4/d7d/d2a/d73/d8d/dc8 0 2024-02-15T18:35:35.059 INFO:tasks.workunit.client.1.smithi121.stdout:4/280: getdents d5/d31/d3d/d4f/d51 0 2024-02-15T18:35:35.062 INFO:tasks.workunit.client.1.smithi121.stdout:7/318: read d0/f27 [2220665,52317] 0 2024-02-15T18:35:35.063 INFO:tasks.workunit.client.1.smithi121.stdout:7/319: chown d0/d5/l1a 105250 1 2024-02-15T18:35:35.065 INFO:tasks.workunit.client.1.smithi121.stdout:5/596: link d0/d2/d1e/d4e/d7d/d47/d5a/l4a d0/d2/d1e/d4e/d7d/d39/lc5 0 2024-02-15T18:35:35.066 INFO:tasks.workunit.client.1.smithi121.stdout:0/588: write d5/d6b/dab/db4/d7d/f24 [7040449,91373] 0 2024-02-15T18:35:35.067 INFO:tasks.workunit.client.1.smithi121.stdout:9/492: truncate d7/d3a/d8e/d9e/da9/d8b/f53 2306497 0 2024-02-15T18:35:35.067 INFO:tasks.workunit.client.1.smithi121.stdout:9/493: write d7/d8/d36/f74 [4279259,8260] 0 2024-02-15T18:35:35.069 INFO:tasks.workunit.client.1.smithi121.stdout:8/496: mkdir d0/d61/db6 0 2024-02-15T18:35:35.069 INFO:tasks.workunit.client.1.smithi121.stdout:7/320: mknod d0/d33/c6d 0 2024-02-15T18:35:35.069 INFO:tasks.workunit.client.1.smithi121.stdout:7/321: readlink d0/d5/d11/l30 0 2024-02-15T18:35:35.070 INFO:tasks.workunit.client.1.smithi121.stdout:2/256: dwrite d5/df/d10/f11 [0,4194304] 0 2024-02-15T18:35:35.071 INFO:tasks.workunit.client.1.smithi121.stdout:5/597: rename d0/d2/d1e/d4e/d7d/d9d/c6c to d0/d2/d1e/d4e/d7d/d47/cc6 0 2024-02-15T18:35:35.074 INFO:tasks.workunit.client.1.smithi121.stdout:3/414: dwrite d2/d1a/d27/d7c/d25/d31/d34/d4a/f8d [0,4194304] 0 2024-02-15T18:35:35.074 INFO:tasks.workunit.client.1.smithi121.stdout:3/415: readlink l0 0 2024-02-15T18:35:35.074 INFO:tasks.workunit.client.1.smithi121.stdout:3/416: fdatasync d2/d1a/d27/d7c/d25/d30/f46 0 2024-02-15T18:35:35.074 INFO:tasks.workunit.client.1.smithi121.stdout:3/417: write d2/d5a/f67 [255363,33276] 0 2024-02-15T18:35:35.074 INFO:tasks.workunit.client.1.smithi121.stdout:3/418: dread - d2/d1a/d6b/d7d/f8e zero size 2024-02-15T18:35:35.074 INFO:tasks.workunit.client.1.smithi121.stdout:3/419: readlink d2/d1a/d6b/d72/l8b 0 2024-02-15T18:35:35.074 INFO:tasks.workunit.client.1.smithi121.stdout:3/420: readlink d2/d1a/d27/d7c/l4c 0 2024-02-15T18:35:35.075 INFO:tasks.workunit.client.1.smithi121.stdout:0/589: mknod d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/cc9 0 2024-02-15T18:35:35.075 INFO:tasks.workunit.client.1.smithi121.stdout:0/590: chown d5/d6b/dab/db4/d7d/d74/c75 15 1 2024-02-15T18:35:35.075 INFO:tasks.workunit.client.1.smithi121.stdout:0/591: chown d5/d6b/dab/db4/d7d/d2a/d73/d8d/fc5 4125036 1 2024-02-15T18:35:35.075 INFO:tasks.workunit.client.1.smithi121.stdout:0/592: readlink d5/d6b/dab/db4/d7d/d2a/d36/d44/l65 0 2024-02-15T18:35:35.075 INFO:tasks.workunit.client.1.smithi121.stdout:6/533: dwrite d7/dc/d34/d42/d62/dac/fb3 [0,4194304] 0 2024-02-15T18:35:35.079 INFO:tasks.workunit.client.1.smithi121.stdout:8/497: dread d0/d7/d36/d52/f5b [0,4194304] 0 2024-02-15T18:35:35.081 INFO:tasks.workunit.client.1.smithi121.stdout:4/281: dwrite d5/d9/d19/d28/f21 [0,4194304] 0 2024-02-15T18:35:35.081 INFO:tasks.workunit.client.1.smithi121.stdout:4/282: truncate d5/d31/d3d/d4f/d51/f47 1526129 0 2024-02-15T18:35:35.081 INFO:tasks.workunit.client.1.smithi121.stdout:4/283: stat d5/l43 0 2024-02-15T18:35:35.081 INFO:tasks.workunit.client.1.smithi121.stdout:9/494: creat d7/d26/d4f/d5b/fb1 x:0 0 0 2024-02-15T18:35:35.081 INFO:tasks.workunit.client.1.smithi121.stdout:7/322: creat d0/d33/f6e x:0 0 0 2024-02-15T18:35:35.082 INFO:tasks.workunit.client.1.smithi121.stdout:8/498: write d0/d7/d36/d8c/d45/f3a [1830273,17588] 0 2024-02-15T18:35:35.082 INFO:tasks.workunit.client.1.smithi121.stdout:2/257: rmdir d5/df 39 2024-02-15T18:35:35.091 INFO:tasks.workunit.client.1.smithi121.stdout:5/598: getdents d0/d2/d1e/d4e/d7d/d9d 0 2024-02-15T18:35:35.095 INFO:tasks.workunit.client.1.smithi121.stdout:3/421: rmdir d2/d1a/d27 39 2024-02-15T18:35:35.097 INFO:tasks.workunit.client.1.smithi121.stdout:0/593: mkdir d5/d6b/dab/db4/dca 0 2024-02-15T18:35:35.098 INFO:tasks.workunit.client.1.smithi121.stdout:8/499: dwrite d0/d2a/d51/f57 [0,4194304] 0 2024-02-15T18:35:35.099 INFO:tasks.workunit.client.1.smithi121.stdout:8/500: fdatasync d0/d7/de/f1c 0 2024-02-15T18:35:35.100 INFO:tasks.workunit.client.1.smithi121.stdout:6/534: mkdir d7/dc/d34/d42/d62/d63/d6f/db9 0 2024-02-15T18:35:35.102 INFO:tasks.workunit.client.1.smithi121.stdout:8/501: dread d0/f13 [0,4194304] 0 2024-02-15T18:35:35.107 INFO:tasks.workunit.client.1.smithi121.stdout:4/284: symlink d5/d9/d19/d2a/d4d/l58 0 2024-02-15T18:35:35.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:33.692735+0000 mon.smithi038 (mon.0) 595 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:35.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:34.001035+0000 mon.smithi038 (mon.0) 596 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:35.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:34.007554+0000 mon.smithi038 (mon.0) 597 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:35.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:34.014798+0000 mon.smithi038 (mon.0) 598 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:35.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:34.026278+0000 mon.smithi038 (mon.0) 599 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:35.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:34.033861+0000 mon.smithi038 (mon.0) 600 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:35.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:35:34.043924+0000 mgr.smithi121.ckiryi (mgr.24385) 9 : cephadm [INF] Reconfiguring prometheus.smithi038 (dependencies changed)... 2024-02-15T18:35:35.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:35:34.055793+0000 mgr.smithi121.ckiryi (mgr.24385) 10 : cephadm [INF] Reconfiguring daemon prometheus.smithi038 on smithi038 2024-02-15T18:35:35.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:34.594351+0000 mgr.smithi121.ckiryi (mgr.24385) 11 : cluster [DBG] pgmap v11: 65 pgs: 65 active+clean; 1.0 GiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 44 MiB/s rd, 91 MiB/s wr, 221 op/s 2024-02-15T18:35:35.109 INFO:tasks.workunit.client.1.smithi121.stdout:7/323: mkdir d0/d5/d4c/d6f 0 2024-02-15T18:35:35.110 INFO:tasks.workunit.client.1.smithi121.stdout:7/324: write d0/d5/f15 [810363,118780] 0 2024-02-15T18:35:35.113 INFO:tasks.workunit.client.1.smithi121.stdout:2/258: mknod d5/df/d10/d15/d46/c48 0 2024-02-15T18:35:35.115 INFO:tasks.workunit.client.1.smithi121.stdout:9/495: unlink d7/c44 0 2024-02-15T18:35:35.115 INFO:tasks.workunit.client.1.smithi121.stdout:5/599: rmdir d0/d2/d1e/d4e/d7d/d9d 39 2024-02-15T18:35:35.118 INFO:tasks.workunit.client.1.smithi121.stdout:0/594: truncate d5/d6b/dab/db4/d7d/dd/f16 655396 0 2024-02-15T18:35:35.118 INFO:tasks.workunit.client.1.smithi121.stdout:0/595: write d5/d6b/dab/fad [341719,128696] 0 2024-02-15T18:35:35.118 INFO:tasks.workunit.client.1.smithi121.stdout:6/535: getdents d7/dc/d34/d42/d6e 0 2024-02-15T18:35:35.120 INFO:tasks.workunit.client.1.smithi121.stdout:5/600: dread d0/d16/f68 [0,4194304] 0 2024-02-15T18:35:35.121 INFO:tasks.workunit.client.1.smithi121.stdout:4/285: rmdir d5/d31/d25 39 2024-02-15T18:35:35.122 INFO:tasks.workunit.client.1.smithi121.stdout:4/286: write d5/d9/d19/d28/f56 [223027,130252] 0 2024-02-15T18:35:35.123 INFO:tasks.workunit.client.1.smithi121.stdout:2/259: mkdir d5/d49 0 2024-02-15T18:35:35.124 INFO:tasks.workunit.client.1.smithi121.stdout:2/260: fdatasync d5/df/f2a 0 2024-02-15T18:35:35.124 INFO:tasks.workunit.client.1.smithi121.stdout:9/496: symlink d7/d8/d25/d30/d33/d3c/lb2 0 2024-02-15T18:35:35.124 INFO:tasks.workunit.client.1.smithi121.stdout:9/497: chown d7/d8/ca 43 1 2024-02-15T18:35:35.126 INFO:tasks.workunit.client.1.smithi121.stdout:0/596: rmdir d5/d6b 39 2024-02-15T18:35:35.126 INFO:tasks.workunit.client.1.smithi121.stdout:6/536: mknod d7/dc/d34/d42/d62/d63/cba 0 2024-02-15T18:35:35.128 INFO:tasks.workunit.client.1.smithi121.stdout:2/261: symlink d5/df/d27/d3e/d35/l4a 0 2024-02-15T18:35:35.129 INFO:tasks.workunit.client.1.smithi121.stdout:9/498: mknod d7/d8/d25/d30/d33/d3c/cb3 0 2024-02-15T18:35:35.130 INFO:tasks.workunit.client.1.smithi121.stdout:4/287: dread d5/d7/fb [0,4194304] 0 2024-02-15T18:35:35.130 INFO:tasks.workunit.client.1.smithi121.stdout:5/601: dread d0/d2/d1e/d4e/d7d/d47/d5a/fa0 [4194304,4194304] 0 2024-02-15T18:35:35.130 INFO:tasks.workunit.client.1.smithi121.stdout:2/262: creat d5/df/d10/d15/d22/f4b x:0 0 0 2024-02-15T18:35:35.130 INFO:tasks.workunit.client.1.smithi121.stdout:5/602: fsync d0/d2/d1e/d4e/d7d/d9d/f59 0 2024-02-15T18:35:35.132 INFO:tasks.workunit.client.1.smithi121.stdout:9/499: creat d7/d8/d25/d30/d33/d8f/d93/fb4 x:0 0 0 2024-02-15T18:35:35.132 INFO:tasks.workunit.client.1.smithi121.stdout:9/500: write d7/d26/f81 [777765,63063] 0 2024-02-15T18:35:35.132 INFO:tasks.workunit.client.1.smithi121.stdout:9/501: chown d7/d26/d4f/d5b/d77/ca1 3920 1 2024-02-15T18:35:35.132 INFO:tasks.workunit.client.1.smithi121.stdout:9/502: truncate d7/d3a/f9d 201606 0 2024-02-15T18:35:35.132 INFO:tasks.workunit.client.1.smithi121.stdout:9/503: chown d7/d8/d25/d30/da5/c4d 1621308361 1 2024-02-15T18:35:35.133 INFO:tasks.workunit.client.1.smithi121.stdout:2/263: unlink d5/df/d27/d3e/f40 0 2024-02-15T18:35:35.134 INFO:tasks.workunit.client.1.smithi121.stdout:4/288: creat d5/d9/d19/d2a/f59 x:0 0 0 2024-02-15T18:35:35.134 INFO:tasks.workunit.client.1.smithi121.stdout:4/289: chown d5/d31/d25/l52 451 1 2024-02-15T18:35:35.135 INFO:tasks.workunit.client.1.smithi121.stdout:5/603: getdents d0/d2/d1e/d4e/d7d/d39 0 2024-02-15T18:35:35.136 INFO:tasks.workunit.client.1.smithi121.stdout:9/504: mknod d7/d3a/d8e/d9e/cb5 0 2024-02-15T18:35:35.138 INFO:tasks.workunit.client.1.smithi121.stdout:4/290: mkdir d5/d31/d5a 0 2024-02-15T18:35:35.138 INFO:tasks.workunit.client.1.smithi121.stdout:7/325: dwrite d0/f27 [4194304,4194304] 0 2024-02-15T18:35:35.140 INFO:tasks.workunit.client.1.smithi121.stdout:7/326: truncate d0/d4a/d5b/f65 1604276 0 2024-02-15T18:35:35.140 INFO:tasks.workunit.client.1.smithi121.stdout:9/505: dread d7/d8/d40/f63 [0,4194304] 0 2024-02-15T18:35:35.140 INFO:tasks.workunit.client.1.smithi121.stdout:9/506: fsync d7/f57 0 2024-02-15T18:35:35.143 INFO:tasks.workunit.client.1.smithi121.stdout:3/422: dwrite d2/d1a/d22/f6a [0,4194304] 0 2024-02-15T18:35:35.149 INFO:tasks.workunit.client.1.smithi121.stdout:2/264: dread d5/f8 [0,4194304] 0 2024-02-15T18:35:35.153 INFO:tasks.workunit.client.1.smithi121.stdout:3/423: rename d2/d1a/d27/d7c/c17 to d2/d1a/c8f 0 2024-02-15T18:35:35.155 INFO:tasks.workunit.client.1.smithi121.stdout:2/265: mkdir d5/df/d10/d4c 0 2024-02-15T18:35:35.157 INFO:tasks.workunit.client.1.smithi121.stdout:3/424: unlink d2/d1a/l24 0 2024-02-15T18:35:35.158 INFO:tasks.workunit.client.1.smithi121.stdout:7/327: write d0/d5/f15 [2214253,81498] 0 2024-02-15T18:35:35.159 INFO:tasks.workunit.client.1.smithi121.stdout:3/425: symlink d2/d1a/d27/d7c/d25/d31/d34/d4a/l90 0 2024-02-15T18:35:35.159 INFO:tasks.workunit.client.1.smithi121.stdout:5/604: read d0/d2/d18/f27 [7682736,128453] 0 2024-02-15T18:35:35.159 INFO:tasks.workunit.client.1.smithi121.stdout:2/266: write d5/f8 [3429438,86045] 0 2024-02-15T18:35:35.159 INFO:tasks.workunit.client.1.smithi121.stdout:7/328: symlink d0/d5/d11/l70 0 2024-02-15T18:35:35.161 INFO:tasks.workunit.client.1.smithi121.stdout:0/597: dwrite d5/d6b/dab/db4/d7d/dd/f7f [4194304,4194304] 0 2024-02-15T18:35:35.161 INFO:tasks.workunit.client.1.smithi121.stdout:5/605: creat d0/d16/da1/fc7 x:0 0 0 2024-02-15T18:35:35.164 INFO:tasks.workunit.client.1.smithi121.stdout:7/329: creat d0/d5/d11/f71 x:0 0 0 2024-02-15T18:35:35.165 INFO:tasks.workunit.client.1.smithi121.stdout:0/598: truncate d5/d6b/dab/db4/d7d/dd/f2c 123054 0 2024-02-15T18:35:35.167 INFO:tasks.workunit.client.1.smithi121.stdout:5/606: unlink d0/d2/d1e/l63 0 2024-02-15T18:35:35.168 INFO:tasks.workunit.client.1.smithi121.stdout:0/599: creat d5/d6b/fcb x:0 0 0 2024-02-15T18:35:35.169 INFO:tasks.workunit.client.1.smithi121.stdout:0/600: creat d5/d6b/dab/db4/d7d/d2a/d4e/fcc x:0 0 0 2024-02-15T18:35:35.170 INFO:tasks.workunit.client.1.smithi121.stdout:7/330: read d0/d5/d11/f40 [6777998,58803] 0 2024-02-15T18:35:35.171 INFO:tasks.workunit.client.1.smithi121.stdout:6/537: dwrite d7/d4b/d60/d6c/f8a [0,4194304] 0 2024-02-15T18:35:35.171 INFO:tasks.workunit.client.1.smithi121.stdout:0/601: creat d5/d6b/dab/db4/d7d/d2a/d73/fcd x:0 0 0 2024-02-15T18:35:35.171 INFO:tasks.workunit.client.1.smithi121.stdout:0/602: chown d5/d6b/dab/db4/d7d/d2a/d36/f51 315730 1 2024-02-15T18:35:35.173 INFO:tasks.workunit.client.1.smithi121.stdout:6/538: rmdir d7/dae 39 2024-02-15T18:35:35.175 INFO:tasks.workunit.client.1.smithi121.stdout:0/603: creat d5/d6b/dab/db1/fce x:0 0 0 2024-02-15T18:35:35.187 INFO:tasks.workunit.client.1.smithi121.stdout:9/507: dwrite d7/d8/f16 [0,4194304] 0 2024-02-15T18:35:35.188 INFO:tasks.workunit.client.1.smithi121.stdout:9/508: symlink d7/d8/d40/d4b/lb6 0 2024-02-15T18:35:35.189 INFO:tasks.workunit.client.1.smithi121.stdout:9/509: symlink d7/d8/d40/d4b/lb7 0 2024-02-15T18:35:35.190 INFO:tasks.workunit.client.1.smithi121.stdout:4/291: dwrite d5/d9/d1d/f41 [4194304,4194304] 0 2024-02-15T18:35:35.191 INFO:tasks.workunit.client.1.smithi121.stdout:9/510: rename d7/d8/d25/d30/d33/d3c/cb3 to d7/d8/d25/d30/d33/d8f/cb8 0 2024-02-15T18:35:35.192 INFO:tasks.workunit.client.1.smithi121.stdout:4/292: symlink d5/d9/d19/d57/l5b 0 2024-02-15T18:35:35.197 INFO:tasks.workunit.client.1.smithi121.stdout:9/511: dread d7/d8/d36/f2f [0,4194304] 0 2024-02-15T18:35:35.197 INFO:tasks.workunit.client.1.smithi121.stdout:9/512: write d7/d8/d25/d30/d33/d8f/d93/fb4 [274402,17590] 0 2024-02-15T18:35:35.204 INFO:tasks.workunit.client.1.smithi121.stdout:4/293: mkdir d5/d9/d19/d5c 0 2024-02-15T18:35:35.206 INFO:tasks.workunit.client.1.smithi121.stdout:9/513: rename d7/d26/c28 to d7/d8/d25/d30/d33/d8f/d93/cb9 0 2024-02-15T18:35:35.207 INFO:tasks.workunit.client.1.smithi121.stdout:3/426: dwrite d2/d1a/d27/d7c/d25/d31/f57 [4194304,4194304] 0 2024-02-15T18:35:35.208 INFO:tasks.workunit.client.1.smithi121.stdout:3/427: write d2/d1a/d27/d7c/f2f [3672902,42552] 0 2024-02-15T18:35:35.209 INFO:tasks.workunit.client.1.smithi121.stdout:9/514: rename d7/d3a/d42 to d7/d26/dba 0 2024-02-15T18:35:35.210 INFO:tasks.workunit.client.1.smithi121.stdout:9/515: chown d7/c12 212936 1 2024-02-15T18:35:35.210 INFO:tasks.workunit.client.1.smithi121.stdout:9/516: dread - d7/d8/d36/f7d zero size 2024-02-15T18:35:35.213 INFO:tasks.workunit.client.1.smithi121.stdout:2/267: dwrite d5/df/f2a [0,4194304] 0 2024-02-15T18:35:35.215 INFO:tasks.workunit.client.1.smithi121.stdout:9/517: rename d7/f59 to d7/d3a/d8e/d9e/da9/d8b/fbb 0 2024-02-15T18:35:35.216 INFO:tasks.workunit.client.1.smithi121.stdout:7/331: dwrite d0/d5/d11/d1c/f1f [0,4194304] 0 2024-02-15T18:35:35.217 INFO:tasks.workunit.client.1.smithi121.stdout:6/539: dwrite d7/d4b/d60/d6c/f8a [4194304,4194304] 0 2024-02-15T18:35:35.217 INFO:tasks.workunit.client.1.smithi121.stdout:6/540: fdatasync d7/dc/d34/d42/d62/d63/d6f/fb1 0 2024-02-15T18:35:35.218 INFO:tasks.workunit.client.1.smithi121.stdout:6/541: fdatasync d7/dc/d34/f4a 0 2024-02-15T18:35:35.218 INFO:tasks.workunit.client.1.smithi121.stdout:0/604: dwrite d5/d6b/dab/db4/d7d/dd/f2d [0,4194304] 0 2024-02-15T18:35:35.218 INFO:tasks.workunit.client.1.smithi121.stdout:5/607: dwrite d0/f4b [0,4194304] 0 2024-02-15T18:35:35.219 INFO:tasks.workunit.client.1.smithi121.stdout:2/268: rename d5/df/d10/d15 to d5/d49/d4d 0 2024-02-15T18:35:35.220 INFO:tasks.workunit.client.1.smithi121.stdout:2/269: write d5/f8 [3827380,20241] 0 2024-02-15T18:35:35.223 INFO:tasks.workunit.client.1.smithi121.stdout:9/518: creat d7/d8/d25/d30/da5/fbc x:0 0 0 2024-02-15T18:35:35.228 INFO:tasks.workunit.client.1.smithi121.stdout:4/294: dwrite d5/d7/f38 [0,4194304] 0 2024-02-15T18:35:35.235 INFO:tasks.workunit.client.1.smithi121.stdout:6/542: mknod d7/dc/d34/d42/d62/d63/cbb 0 2024-02-15T18:35:35.235 INFO:tasks.workunit.client.1.smithi121.stdout:3/428: write d2/d1a/d22/f42 [1460047,64936] 0 2024-02-15T18:35:35.245 INFO:tasks.workunit.client.1.smithi121.stdout:0/605: unlink d5/d6b/dab/db4/d7d/dd/c29 0 2024-02-15T18:35:35.245 INFO:tasks.workunit.client.1.smithi121.stdout:5/608: stat d0/d2/d1e/d4e/d7d/l24 0 2024-02-15T18:35:35.246 INFO:tasks.workunit.client.1.smithi121.stdout:7/332: rename d0/d5/f9 to d0/d38/d61/f72 0 2024-02-15T18:35:35.247 INFO:tasks.workunit.client.1.smithi121.stdout:9/519: mknod d7/d26/d4f/cbd 0 2024-02-15T18:35:35.249 INFO:tasks.workunit.client.1.smithi121.stdout:0/606: mknod d5/d6b/dab/db4/d7d/dd/ccf 0 2024-02-15T18:35:35.249 INFO:tasks.workunit.client.1.smithi121.stdout:0/607: truncate d5/d6b/dab/db4/d7d/dd/f34 1371090 0 2024-02-15T18:35:35.250 INFO:tasks.workunit.client.1.smithi121.stdout:5/609: creat d0/d2/d1e/d4e/d7d/d40/fc8 x:0 0 0 2024-02-15T18:35:35.250 INFO:tasks.workunit.client.1.smithi121.stdout:5/610: write d0/d2/d1e/d4e/d7d/d39/f53 [1589118,120749] 0 2024-02-15T18:35:35.252 INFO:tasks.workunit.client.1.smithi121.stdout:2/270: rename d5/c23 to d5/df/d27/d3a/c4e 0 2024-02-15T18:35:35.254 INFO:tasks.workunit.client.1.smithi121.stdout:0/608: creat d5/d6b/dab/db4/d7d/dd/fd0 x:0 0 0 2024-02-15T18:35:35.256 INFO:tasks.workunit.client.1.smithi121.stdout:9/520: rename d7/d8/d36/f74 to d7/d8/fbe 0 2024-02-15T18:35:35.258 INFO:tasks.workunit.client.1.smithi121.stdout:2/271: creat d5/d49/d4d/d22/f4f x:0 0 0 2024-02-15T18:35:35.258 INFO:tasks.workunit.client.1.smithi121.stdout:2/272: dread - d5/d49/d4d/d22/f4f zero size 2024-02-15T18:35:35.258 INFO:tasks.workunit.client.1.smithi121.stdout:0/609: symlink d5/d6b/dab/db4/d7d/d2a/d73/d81/ld1 0 2024-02-15T18:35:35.262 INFO:tasks.workunit.client.1.smithi121.stdout:9/521: dread d7/d26/d4f/d5b/d77/fa6 [0,4194304] 0 2024-02-15T18:35:35.265 INFO:tasks.workunit.client.1.smithi121.stdout:4/295: dwrite d5/d9/d1d/f42 [0,4194304] 0 2024-02-15T18:35:35.267 INFO:tasks.workunit.client.1.smithi121.stdout:9/522: dread f5 [0,4194304] 0 2024-02-15T18:35:35.267 INFO:tasks.workunit.client.1.smithi121.stdout:4/296: rename d5/d9/d19/d2a/f59 to d5/d31/d3d/d4f/d51/f5d 0 2024-02-15T18:35:35.268 INFO:tasks.workunit.client.1.smithi121.stdout:9/523: mknod d7/d3a/d8e/d9e/da9/cbf 0 2024-02-15T18:35:35.269 INFO:tasks.workunit.client.1.smithi121.stdout:4/297: mknod d5/d9/d19/d2a/d4d/c5e 0 2024-02-15T18:35:35.270 INFO:tasks.workunit.client.1.smithi121.stdout:6/543: dwrite d7/d4b/d60/d73/fa2 [0,4194304] 0 2024-02-15T18:35:35.270 INFO:tasks.workunit.client.1.smithi121.stdout:9/524: unlink d7/c2a 0 2024-02-15T18:35:35.271 INFO:tasks.workunit.client.1.smithi121.stdout:4/298: truncate d5/d7/fb 3863758 0 2024-02-15T18:35:35.271 INFO:tasks.workunit.client.1.smithi121.stdout:4/299: fdatasync d5/d31/d25/f33 0 2024-02-15T18:35:35.271 INFO:tasks.workunit.client.1.smithi121.stdout:4/300: chown d5/d31/d3d/d4f/d51 437934426 1 2024-02-15T18:35:35.271 INFO:tasks.workunit.client.1.smithi121.stdout:4/301: write d5/f30 [731641,53379] 0 2024-02-15T18:35:35.271 INFO:tasks.workunit.client.1.smithi121.stdout:2/273: dread d5/df/d27/f21 [0,4194304] 0 2024-02-15T18:35:35.272 INFO:tasks.workunit.client.1.smithi121.stdout:2/274: write d5/df/d27/d3a/f43 [1008631,91461] 0 2024-02-15T18:35:35.272 INFO:tasks.workunit.client.1.smithi121.stdout:9/525: creat d7/fc0 x:0 0 0 2024-02-15T18:35:35.272 INFO:tasks.workunit.client.1.smithi121.stdout:9/526: readlink d7/d8/d25/l67 0 2024-02-15T18:35:35.273 INFO:tasks.workunit.client.1.smithi121.stdout:4/302: mknod d5/d9/d19/d28/c5f 0 2024-02-15T18:35:35.274 INFO:tasks.workunit.client.1.smithi121.stdout:2/275: creat d5/df/d10/d4c/f50 x:0 0 0 2024-02-15T18:35:35.275 INFO:tasks.workunit.client.1.smithi121.stdout:2/276: stat d5/df/d10/c2c 0 2024-02-15T18:35:35.275 INFO:tasks.workunit.client.1.smithi121.stdout:2/277: dread d5/fc [0,4194304] 0 2024-02-15T18:35:35.275 INFO:tasks.workunit.client.1.smithi121.stdout:4/303: creat d5/d31/d3d/d4a/f60 x:0 0 0 2024-02-15T18:35:35.283 INFO:tasks.workunit.client.1.smithi121.stdout:4/304: dread d5/d31/d25/f37 [0,4194304] 0 2024-02-15T18:35:35.284 INFO:tasks.workunit.client.1.smithi121.stdout:3/429: dwrite d2/d1a/d27/d7c/d25/d31/d34/d4a/f8d [0,4194304] 0 2024-02-15T18:35:35.284 INFO:tasks.workunit.client.1.smithi121.stdout:3/430: dread - d2/d1a/d27/d7c/d25/d31/d58/f6f zero size 2024-02-15T18:35:35.291 INFO:tasks.workunit.client.1.smithi121.stdout:5/611: dwrite d0/d2/d70/d88/f96 [0,4194304] 0 2024-02-15T18:35:35.292 INFO:tasks.workunit.client.1.smithi121.stdout:5/612: fdatasync d0/d2/d1e/fa2 0 2024-02-15T18:35:35.292 INFO:tasks.workunit.client.1.smithi121.stdout:0/610: dwrite d5/d6b/dab/db4/d7d/dd/f2d [0,4194304] 0 2024-02-15T18:35:35.292 INFO:tasks.workunit.client.1.smithi121.stdout:0/611: write d5/d6b/dab/db4/d7d/dd/f2c [525548,63036] 0 2024-02-15T18:35:35.294 INFO:tasks.workunit.client.1.smithi121.stdout:2/278: dwrite d5/d49/d4d/d22/f44 [0,4194304] 0 2024-02-15T18:35:35.296 INFO:tasks.workunit.client.1.smithi121.stdout:5/613: rmdir d0/d2/d1e/d4e/d7d/d9d/d4c/daa 39 2024-02-15T18:35:35.302 INFO:tasks.workunit.client.1.smithi121.stdout:4/305: dwrite d5/d9/d1d/f2f [0,4194304] 0 2024-02-15T18:35:35.302 INFO:tasks.workunit.client.1.smithi121.stdout:4/306: chown d5/d7/d14/f17 892 1 2024-02-15T18:35:35.314 INFO:tasks.workunit.client.1.smithi121.stdout:0/612: write d5/d6b/dab/db4/d7d/d74/f82 [1065969,84632] 0 2024-02-15T18:35:35.314 INFO:tasks.workunit.client.1.smithi121.stdout:0/613: chown d5/d6b/dab/db4/d7d/dd/ccf 49523521 1 2024-02-15T18:35:35.321 INFO:tasks.workunit.client.1.smithi121.stdout:5/614: creat d0/d2/d70/fc9 x:0 0 0 2024-02-15T18:35:35.321 INFO:tasks.workunit.client.1.smithi121.stdout:4/307: creat d5/d9/d19/d4c/f61 x:0 0 0 2024-02-15T18:35:35.321 INFO:tasks.workunit.client.1.smithi121.stdout:4/308: chown d5/d9 3479835 1 2024-02-15T18:35:35.321 INFO:tasks.workunit.client.1.smithi121.stdout:4/309: chown d5/d31/d25 11826 1 2024-02-15T18:35:35.322 INFO:tasks.workunit.client.1.smithi121.stdout:2/279: rmdir d5/df 39 2024-02-15T18:35:35.323 INFO:tasks.workunit.client.1.smithi121.stdout:5/615: getdents d0/d2/d1e/d4e/d7d/d40/d62/db1 0 2024-02-15T18:35:35.326 INFO:tasks.workunit.client.1.smithi121.stdout:5/616: mkdir d0/d16/d7a/db5/dca 0 2024-02-15T18:35:35.326 INFO:tasks.workunit.client.1.smithi121.stdout:5/617: truncate d0/d2/d1e/d4e/d7d/d40/d62/db1/fc3 804423 0 2024-02-15T18:35:35.327 INFO:tasks.workunit.client.1.smithi121.stdout:0/614: dread d5/d6b/dab/db4/d7d/d2a/d4e/f5a [0,4194304] 0 2024-02-15T18:35:35.327 INFO:tasks.workunit.client.1.smithi121.stdout:0/615: dread - d5/d6b/fcb zero size 2024-02-15T18:35:35.329 INFO:tasks.workunit.client.1.smithi121.stdout:5/618: creat d0/d16/d7a/db5/fcb x:0 0 0 2024-02-15T18:35:35.331 INFO:tasks.workunit.client.1.smithi121.stdout:4/310: dread d5/d9/d19/d2a/f2c [0,4194304] 0 2024-02-15T18:35:35.336 INFO:tasks.workunit.client.1.smithi121.stdout:4/311: rename d5/d7/l44 to d5/d9/d19/d28/l62 0 2024-02-15T18:35:35.338 INFO:tasks.workunit.client.1.smithi121.stdout:4/312: read f1 [4588462,53705] 0 2024-02-15T18:35:35.344 INFO:tasks.workunit.client.1.smithi121.stdout:4/313: dread d5/d9/f20 [0,4194304] 0 2024-02-15T18:35:35.345 INFO:tasks.workunit.client.1.smithi121.stdout:2/280: dwrite d5/df/f45 [0,4194304] 0 2024-02-15T18:35:35.345 INFO:tasks.workunit.client.1.smithi121.stdout:2/281: dread - d5/d49/d4d/d22/f4f zero size 2024-02-15T18:35:35.353 INFO:tasks.workunit.client.1.smithi121.stdout:2/282: truncate d5/d49/d4d/f1c 2470002 0 2024-02-15T18:35:35.356 INFO:tasks.workunit.client.1.smithi121.stdout:2/283: mkdir d5/df/d27/d51 0 2024-02-15T18:35:35.365 INFO:tasks.workunit.client.1.smithi121.stdout:4/314: dwrite d5/d9/d19/d4c/f61 [0,4194304] 0 2024-02-15T18:35:35.366 INFO:tasks.workunit.client.1.smithi121.stdout:0/616: dwrite d5/d6b/dab/db4/f8f [0,4194304] 0 2024-02-15T18:35:35.366 INFO:tasks.workunit.client.1.smithi121.stdout:0/617: stat d5/d6b/c6e 0 2024-02-15T18:35:35.373 INFO:tasks.workunit.client.1.smithi121.stdout:0/618: mknod d5/d6b/dab/db4/d7d/d2a/d73/d8d/dc8/cd2 0 2024-02-15T18:35:35.384 INFO:tasks.workunit.client.1.smithi121.stdout:2/284: dwrite d5/d49/d4d/d22/f42 [0,4194304] 0 2024-02-15T18:35:35.389 INFO:tasks.workunit.client.1.smithi121.stdout:2/285: truncate d5/f8 1218710 0 2024-02-15T18:35:35.390 INFO:tasks.workunit.client.1.smithi121.stdout:2/286: write d5/df/d27/d3a/f43 [1258004,93784] 0 2024-02-15T18:35:35.394 INFO:tasks.workunit.client.1.smithi121.stdout:2/287: write d5/df/f2a [3619620,5385] 0 2024-02-15T18:35:35.394 INFO:tasks.workunit.client.1.smithi121.stdout:2/288: chown d5/df/l20 1794116 1 2024-02-15T18:35:35.394 INFO:tasks.workunit.client.1.smithi121.stdout:2/289: write d5/d49/d4d/d22/f44 [4460342,51235] 0 2024-02-15T18:35:35.394 INFO:tasks.workunit.client.1.smithi121.stdout:2/290: write d5/d49/d4d/f2d [4885658,30819] 0 2024-02-15T18:35:35.395 INFO:tasks.workunit.client.1.smithi121.stdout:2/291: truncate f4 1839989 0 2024-02-15T18:35:35.397 INFO:tasks.workunit.client.1.smithi121.stdout:2/292: symlink d5/df/d27/l52 0 2024-02-15T18:35:35.397 INFO:tasks.workunit.client.1.smithi121.stdout:2/293: fsync d5/d49/d4d/d22/f42 0 2024-02-15T18:35:35.398 INFO:tasks.workunit.client.1.smithi121.stdout:0/619: dwrite d5/f92 [0,4194304] 0 2024-02-15T18:35:35.398 INFO:tasks.workunit.client.1.smithi121.stdout:2/294: creat d5/df/d27/d3e/f53 x:0 0 0 2024-02-15T18:35:35.402 INFO:tasks.workunit.client.1.smithi121.stdout:4/315: dwrite d5/d7/f38 [0,4194304] 0 2024-02-15T18:35:35.403 INFO:tasks.workunit.client.1.smithi121.stdout:0/620: creat d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/fd3 x:0 0 0 2024-02-15T18:35:35.403 INFO:tasks.workunit.client.1.smithi121.stdout:0/621: dread - d5/d6b/fa8 zero size 2024-02-15T18:35:35.403 INFO:tasks.workunit.client.1.smithi121.stdout:2/295: mkdir d5/df/d54 0 2024-02-15T18:35:35.403 INFO:tasks.workunit.client.1.smithi121.stdout:2/296: stat d5/df/d27/d3e/d35 0 2024-02-15T18:35:35.403 INFO:tasks.workunit.client.1.smithi121.stdout:2/297: readlink l1 0 2024-02-15T18:35:35.404 INFO:tasks.workunit.client.1.smithi121.stdout:2/298: write d5/fd [939946,48274] 0 2024-02-15T18:35:35.405 INFO:tasks.workunit.client.1.smithi121.stdout:0/622: rmdir d5/d6b/dab/db4/d7d 39 2024-02-15T18:35:35.407 INFO:tasks.workunit.client.1.smithi121.stdout:2/299: write d5/d49/d4d/f2d [2183618,98410] 0 2024-02-15T18:35:35.410 INFO:tasks.workunit.client.1.smithi121.stdout:2/300: symlink d5/df/d27/l55 0 2024-02-15T18:35:35.410 INFO:tasks.workunit.client.1.smithi121.stdout:2/301: rename d5 to d5/df/d27/d3e/d35/d56 22 2024-02-15T18:35:35.410 INFO:tasks.workunit.client.1.smithi121.stdout:4/316: rmdir d5/d9/d19/d2a 39 2024-02-15T18:35:35.416 INFO:tasks.workunit.client.1.smithi121.stdout:2/302: link d5/df/d27/d3e/d35/l4a d5/df/d3f/l57 0 2024-02-15T18:35:35.418 INFO:tasks.workunit.client.1.smithi121.stdout:4/317: mkdir d5/d31/d3d/d4f/d51/d63 0 2024-02-15T18:35:35.418 INFO:tasks.workunit.client.1.smithi121.stdout:2/303: mkdir d5/d49/d4d/d58 0 2024-02-15T18:35:35.420 INFO:tasks.workunit.client.1.smithi121.stdout:2/304: write d5/df/f1b [72609,116259] 0 2024-02-15T18:35:35.420 INFO:tasks.workunit.client.1.smithi121.stdout:2/305: getdents d5/df/d27/d51 0 2024-02-15T18:35:35.421 INFO:tasks.workunit.client.1.smithi121.stdout:2/306: creat d5/df/d27/d3e/f59 x:0 0 0 2024-02-15T18:35:35.430 INFO:tasks.workunit.client.1.smithi121.stdout:4/318: dread d5/f6 [0,4194304] 0 2024-02-15T18:35:35.431 INFO:tasks.workunit.client.1.smithi121.stdout:4/319: write d5/d9/d19/d28/f21 [5106544,49095] 0 2024-02-15T18:35:35.431 INFO:tasks.workunit.client.1.smithi121.stdout:0/623: dwrite f2 [4194304,4194304] 0 2024-02-15T18:35:35.431 INFO:tasks.workunit.client.1.smithi121.stdout:0/624: dread - d5/d6b/dab/db4/d7d/dd/fd0 zero size 2024-02-15T18:35:35.441 INFO:tasks.workunit.client.1.smithi121.stdout:2/307: dwrite d5/d49/d4d/d22/f37 [0,4194304] 0 2024-02-15T18:35:35.441 INFO:tasks.workunit.client.1.smithi121.stdout:2/308: readlink d5/df/d27/l55 0 2024-02-15T18:35:35.442 INFO:tasks.workunit.client.1.smithi121.stdout:2/309: getdents d5/df/d10/d4c 0 2024-02-15T18:35:35.444 INFO:tasks.workunit.client.1.smithi121.stdout:2/310: symlink d5/d49/d4d/l5a 0 2024-02-15T18:35:35.444 INFO:tasks.workunit.client.1.smithi121.stdout:2/311: readlink d5/df/d3f/l57 0 2024-02-15T18:35:35.448 INFO:tasks.workunit.client.1.smithi121.stdout:0/625: dread d5/d6b/dab/db4/d7d/d2a/d36/f51 [0,4194304] 0 2024-02-15T18:35:35.448 INFO:tasks.workunit.client.1.smithi121.stdout:0/626: readlink d5/d30/l41 0 2024-02-15T18:35:35.451 INFO:tasks.workunit.client.1.smithi121.stdout:0/627: rename d5/d6b/dab/db4/d7d/dd/f47 to d5/d6b/dab/db4/d7d/d2a/d36/d44/db7/fd4 0 2024-02-15T18:35:35.455 INFO:tasks.workunit.client.1.smithi121.stdout:4/320: dwrite d5/d9/d19/f4e [0,4194304] 0 2024-02-15T18:35:35.455 INFO:tasks.workunit.client.1.smithi121.stdout:4/321: fdatasync d5/d9/d19/d4c/f61 0 2024-02-15T18:35:35.481 INFO:tasks.workunit.client.1.smithi121.stdout:4/322: dwrite d5/d7/f3c [4194304,4194304] 0 2024-02-15T18:35:35.483 INFO:tasks.workunit.client.1.smithi121.stdout:0/628: dwrite d5/d6b/dab/db4/d7d/d2a/d73/d97/f98 [0,4194304] 0 2024-02-15T18:35:35.483 INFO:tasks.workunit.client.1.smithi121.stdout:0/629: truncate d5/d6b/dab/db4/d7d/d2a/f95 704696 0 2024-02-15T18:35:35.484 INFO:tasks.workunit.client.1.smithi121.stdout:4/323: creat d5/f64 x:0 0 0 2024-02-15T18:35:35.488 INFO:tasks.workunit.client.1.smithi121.stdout:0/630: rename d5/d6b/dab/db4/d7d/d2a/d4e/l8c to d5/d6b/dab/db4/d7d/d2a/ld5 0 2024-02-15T18:35:35.488 INFO:tasks.workunit.client.1.smithi121.stdout:2/312: dwrite d5/d49/d4d/f1c [0,4194304] 0 2024-02-15T18:35:35.491 INFO:tasks.workunit.client.1.smithi121.stdout:0/631: creat d5/d6b/dab/db4/d7d/d2a/d73/d81/fd6 x:0 0 0 2024-02-15T18:35:35.492 INFO:tasks.workunit.client.1.smithi121.stdout:2/313: symlink d5/d49/d4d/d22/l5b 0 2024-02-15T18:35:35.494 INFO:tasks.workunit.client.1.smithi121.stdout:0/632: rename d5/d6b/dab/db4/d7d/d2a/d73/d81/lb9 to d5/d6b/dab/db4/d7d/d2a/d36/ld7 0 2024-02-15T18:35:35.498 INFO:tasks.workunit.client.1.smithi121.stdout:2/314: fsync d5/fd 0 2024-02-15T18:35:35.502 INFO:tasks.workunit.client.1.smithi121.stdout:2/315: creat d5/df/d27/d3e/d47/f5c x:0 0 0 2024-02-15T18:35:35.520 INFO:tasks.workunit.client.1.smithi121.stdout:4/324: dwrite d5/d9/d19/d2a/d4d/f54 [0,4194304] 0 2024-02-15T18:35:35.520 INFO:tasks.workunit.client.1.smithi121.stdout:4/325: fsync d5/d9/d19/d2a/d4d/f54 0 2024-02-15T18:35:35.520 INFO:tasks.workunit.client.1.smithi121.stdout:4/326: fsync d5/d7/f38 0 2024-02-15T18:35:35.520 INFO:tasks.workunit.client.1.smithi121.stdout:4/327: readlink d5/d9/l3f 0 2024-02-15T18:35:35.520 INFO:tasks.workunit.client.1.smithi121.stdout:0/633: dwrite d5/d6b/dab/db4/d7d/d2a/d73/fcd [0,4194304] 0 2024-02-15T18:35:35.523 INFO:tasks.workunit.client.1.smithi121.stdout:0/634: write d5/f27 [3768989,17230] 0 2024-02-15T18:35:35.523 INFO:tasks.workunit.client.1.smithi121.stdout:2/316: dread d5/df/f45 [0,4194304] 0 2024-02-15T18:35:35.523 INFO:tasks.workunit.client.1.smithi121.stdout:2/317: chown d5/df/d27/f21 0 1 2024-02-15T18:35:35.524 INFO:tasks.workunit.client.1.smithi121.stdout:4/328: mknod d5/d7/d14/c65 0 2024-02-15T18:35:35.528 INFO:tasks.workunit.client.1.smithi121.stdout:2/318: creat d5/df/d54/f5d x:0 0 0 2024-02-15T18:35:35.528 INFO:tasks.workunit.client.1.smithi121.stdout:0/635: symlink d5/d9b/ld8 0 2024-02-15T18:35:35.528 INFO:tasks.workunit.client.1.smithi121.stdout:4/329: stat d5/d7/c4b 0 2024-02-15T18:35:35.531 INFO:tasks.workunit.client.1.smithi121.stdout:0/636: link d5/d6b/dab/db4/d7d/d2a/d36/d44/db7/fd4 d5/d6b/dab/db1/dbf/fd9 0 2024-02-15T18:35:35.532 INFO:tasks.workunit.client.1.smithi121.stdout:4/330: mknod d5/c66 0 2024-02-15T18:35:35.536 INFO:tasks.workunit.client.1.smithi121.stdout:2/319: write d5/d49/d4d/f2d [2653283,95507] 0 2024-02-15T18:35:35.538 INFO:tasks.workunit.client.1.smithi121.stdout:0/637: creat d5/d6b/dab/db4/d7d/d2a/fda x:0 0 0 2024-02-15T18:35:35.539 INFO:tasks.workunit.client.1.smithi121.stdout:2/320: symlink d5/df/d27/d3e/d35/l5e 0 2024-02-15T18:35:35.542 INFO:tasks.workunit.client.1.smithi121.stdout:2/321: creat d5/df/d27/f5f x:0 0 0 2024-02-15T18:35:35.542 INFO:tasks.workunit.client.1.smithi121.stdout:2/322: truncate d5/df/d27/f5f 4207 0 2024-02-15T18:35:35.545 INFO:tasks.workunit.client.1.smithi121.stdout:2/323: write d5/df/f45 [1239923,90262] 0 2024-02-15T18:35:35.545 INFO:tasks.workunit.client.1.smithi121.stdout:2/324: stat d5/df/d10/f11 0 2024-02-15T18:35:35.545 INFO:tasks.workunit.client.1.smithi121.stdout:2/325: write d5/df/d27/d3e/f59 [400751,121811] 0 2024-02-15T18:35:35.550 INFO:tasks.workunit.client.1.smithi121.stdout:2/326: unlink d5/df/f45 0 2024-02-15T18:35:35.553 INFO:tasks.workunit.client.1.smithi121.stdout:2/327: rename d5/df/d27/d51 to d5/df/d60 0 2024-02-15T18:35:35.554 INFO:tasks.workunit.client.1.smithi121.stdout:2/328: readlink d5/df/d10/l30 0 2024-02-15T18:35:35.554 INFO:tasks.workunit.client.1.smithi121.stdout:4/331: dwrite d5/d7/d1e/f53 [0,4194304] 0 2024-02-15T18:35:35.557 INFO:tasks.workunit.client.1.smithi121.stdout:4/332: symlink d5/d9/d1d/d26/l67 0 2024-02-15T18:35:35.558 INFO:tasks.workunit.client.1.smithi121.stdout:4/333: fdatasync d5/d31/f23 0 2024-02-15T18:35:35.563 INFO:tasks.workunit.client.1.smithi121.stdout:4/334: unlink d5/d9/d19/d28/f56 0 2024-02-15T18:35:35.569 INFO:tasks.workunit.client.1.smithi121.stdout:4/335: rmdir d5/d31/d5a 0 2024-02-15T18:35:35.578 INFO:tasks.workunit.client.1.smithi121.stdout:4/336: dread d5/d9/d1d/f2f [0,4194304] 0 2024-02-15T18:35:35.580 INFO:tasks.workunit.client.1.smithi121.stdout:4/337: link d5/d7/f38 d5/d9/d19/d2a/f68 0 2024-02-15T18:35:35.597 INFO:tasks.workunit.client.1.smithi121.stdout:4/338: dwrite d5/d9/d19/d2a/f68 [0,4194304] 0 2024-02-15T18:35:35.598 INFO:tasks.workunit.client.1.smithi121.stdout:4/339: creat d5/d9/d19/d5c/f69 x:0 0 0 2024-02-15T18:35:35.616 INFO:tasks.workunit.client.1.smithi121.stdout:4/340: dwrite d5/d9/d1d/f2f [0,4194304] 0 2024-02-15T18:35:35.617 INFO:tasks.workunit.client.1.smithi121.stdout:4/341: creat d5/d7/f6a x:0 0 0 2024-02-15T18:35:35.631 INFO:tasks.workunit.client.1.smithi121.stdout:4/342: dwrite d5/d9/f20 [0,4194304] 0 2024-02-15T18:35:35.654 INFO:tasks.workunit.client.1.smithi121.stdout:4/343: dwrite d5/d7/d14/f17 [0,4194304] 0 2024-02-15T18:35:35.655 INFO:tasks.workunit.client.1.smithi121.stdout:4/344: symlink d5/l6b 0 2024-02-15T18:35:35.657 INFO:tasks.workunit.client.1.smithi121.stdout:4/345: truncate d5/d31/d25/f37 1623997 0 2024-02-15T18:35:35.672 INFO:tasks.workunit.client.1.smithi121.stdout:4/346: dwrite d5/d9/d19/d2a/f2c [0,4194304] 0 2024-02-15T18:35:36.660 INFO:tasks.workunit.client.1.smithi121.stdout:1/578: sync 2024-02-15T18:35:36.660 INFO:tasks.workunit.client.1.smithi121.stdout:1/579: getdents d0/d6/d7/d9/d14/d3e 0 2024-02-15T18:35:36.663 INFO:tasks.workunit.client.1.smithi121.stdout:1/580: mknod d0/d6/d7/d9/d14/d3e/d88/daf/db4/cc5 0 2024-02-15T18:35:36.663 INFO:tasks.workunit.client.1.smithi121.stdout:1/581: write d0/d39/d4b/f61 [4866176,55743] 0 2024-02-15T18:35:36.691 INFO:tasks.workunit.client.1.smithi121.stdout:1/582: dwrite d0/d6/d7/d9/d14/d3e/d88/da6/fbd [0,4194304] 0 2024-02-15T18:35:36.692 INFO:tasks.workunit.client.1.smithi121.stdout:1/583: creat d0/d6/d7/d9/d14/d3e/d88/fc6 x:0 0 0 2024-02-15T18:35:36.693 INFO:tasks.workunit.client.1.smithi121.stdout:1/584: write d0/d6/d7/d9/d14/d3e/f71 [1415741,102412] 0 2024-02-15T18:35:36.695 INFO:tasks.workunit.client.1.smithi121.stdout:1/585: mknod d0/d6/d7/d9/d14/d48/cc7 0 2024-02-15T18:35:36.696 INFO:tasks.workunit.client.1.smithi121.stdout:1/586: write d0/d6/d7/d9/d14/db6/fba [642809,109147] 0 2024-02-15T18:35:36.696 INFO:tasks.workunit.client.1.smithi121.stdout:1/587: stat d0/fbc 0 2024-02-15T18:35:36.696 INFO:tasks.workunit.client.1.smithi121.stdout:1/588: chown d0/d6/d7/d9/d14/d3e/d88 315603 1 2024-02-15T18:35:36.697 INFO:tasks.workunit.client.1.smithi121.stdout:1/589: mknod d0/d39/d7d/cc8 0 2024-02-15T18:35:36.698 INFO:tasks.workunit.client.1.smithi121.stdout:1/590: creat d0/d6/d7/d50/d7e/fc9 x:0 0 0 2024-02-15T18:35:36.700 INFO:tasks.workunit.client.1.smithi121.stdout:1/591: unlink d0/d6/d7/d9/d51/la5 0 2024-02-15T18:35:37.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:36.594781+0000 mgr.smithi121.ckiryi (mgr.24385) 12 : cluster [DBG] pgmap v12: 65 pgs: 65 active+clean; 1.2 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 62 MiB/s rd, 124 MiB/s wr, 348 op/s 2024-02-15T18:35:37.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:36 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:36.594781+0000 mgr.smithi121.ckiryi (mgr.24385) 12 : cluster [DBG] pgmap v12: 65 pgs: 65 active+clean; 1.2 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 62 MiB/s rd, 124 MiB/s wr, 348 op/s 2024-02-15T18:35:37.231 INFO:tasks.workunit.client.1.smithi121.stdout:8/502: sync 2024-02-15T18:35:37.231 INFO:tasks.workunit.client.1.smithi121.stdout:8/503: readlink d0/d2a/d51/l4e 0 2024-02-15T18:35:37.231 INFO:tasks.workunit.client.1.smithi121.stdout:8/504: chown d0/d7/c69 9 1 2024-02-15T18:35:37.232 INFO:tasks.workunit.client.1.smithi121.stdout:8/505: dread d0/d7/de/d7c/f91 [0,4194304] 0 2024-02-15T18:35:37.233 INFO:tasks.workunit.client.1.smithi121.stdout:8/506: creat d0/d73/fb7 x:0 0 0 2024-02-15T18:35:37.233 INFO:tasks.workunit.client.1.smithi121.stdout:8/507: chown d0/d61/d66/f6d 7271 1 2024-02-15T18:35:37.233 INFO:tasks.workunit.client.1.smithi121.stdout:8/508: chown d0/d61/d66 3 1 2024-02-15T18:35:37.233 INFO:tasks.workunit.client.1.smithi121.stdout:8/509: chown d0/d7/f10 34 1 2024-02-15T18:35:37.234 INFO:tasks.workunit.client.1.smithi121.stdout:8/510: mknod d0/d3/cb8 0 2024-02-15T18:35:37.234 INFO:tasks.workunit.client.1.smithi121.stdout:8/511: readlink d0/l14 0 2024-02-15T18:35:37.252 INFO:tasks.workunit.client.1.smithi121.stdout:8/512: dwrite d0/d7/f10 [4194304,4194304] 0 2024-02-15T18:35:37.252 INFO:tasks.workunit.client.1.smithi121.stdout:8/513: write d0/d7/d36/d8c/f49 [1657456,90727] 0 2024-02-15T18:35:37.252 INFO:tasks.workunit.client.1.smithi121.stdout:8/514: stat d0/d3 0 2024-02-15T18:35:37.255 INFO:tasks.workunit.client.1.smithi121.stdout:8/515: truncate d0/f37 1155714 0 2024-02-15T18:35:37.268 INFO:tasks.workunit.client.1.smithi121.stdout:8/516: dwrite d0/d61/d66/fab [0,4194304] 0 2024-02-15T18:35:37.271 INFO:tasks.workunit.client.1.smithi121.stdout:8/517: creat d0/d7/d36/fb9 x:0 0 0 2024-02-15T18:35:37.286 INFO:tasks.workunit.client.1.smithi121.stdout:8/518: dwrite d0/d3/f64 [0,4194304] 0 2024-02-15T18:35:37.286 INFO:tasks.workunit.client.1.smithi121.stdout:8/519: fdatasync d0/d2a/d51/f6f 0 2024-02-15T18:35:37.289 INFO:tasks.workunit.client.1.smithi121.stdout:8/520: creat d0/d7/d36/fba x:0 0 0 2024-02-15T18:35:37.290 INFO:tasks.workunit.client.1.smithi121.stdout:8/521: write d0/d7/d36/d8c/d45/f9e [4179628,23985] 0 2024-02-15T18:35:37.290 INFO:tasks.workunit.client.1.smithi121.stdout:8/522: chown d0/d7/de/d7c/f91 64136 1 2024-02-15T18:35:37.290 INFO:tasks.workunit.client.1.smithi121.stdout:8/523: fsync d0/d7/f5e 0 2024-02-15T18:35:37.290 INFO:tasks.workunit.client.1.smithi121.stdout:8/524: write d0/d61/d66/f6d [2563380,50367] 0 2024-02-15T18:35:37.291 INFO:tasks.workunit.client.1.smithi121.stdout:8/525: rmdir d0/d7 39 2024-02-15T18:35:37.292 INFO:tasks.workunit.client.1.smithi121.stdout:8/526: chown d0/d7/d36/c4d 7357722 1 2024-02-15T18:35:37.295 INFO:tasks.workunit.client.1.smithi121.stdout:0/638: sync 2024-02-15T18:35:37.295 INFO:tasks.workunit.client.1.smithi121.stdout:2/329: sync 2024-02-15T18:35:37.296 INFO:tasks.workunit.client.1.smithi121.stdout:6/544: sync 2024-02-15T18:35:37.296 INFO:tasks.workunit.client.1.smithi121.stdout:1/592: sync 2024-02-15T18:35:37.296 INFO:tasks.workunit.client.1.smithi121.stdout:7/333: sync 2024-02-15T18:35:37.296 INFO:tasks.workunit.client.1.smithi121.stdout:3/431: sync 2024-02-15T18:35:37.296 INFO:tasks.workunit.client.1.smithi121.stdout:5/619: sync 2024-02-15T18:35:37.296 INFO:tasks.workunit.client.1.smithi121.stdout:4/347: sync 2024-02-15T18:35:37.296 INFO:tasks.workunit.client.1.smithi121.stdout:5/620: chown d0/d2/d70/c77 2 1 2024-02-15T18:35:37.296 INFO:tasks.workunit.client.1.smithi121.stdout:2/330: write d5/df/f1b [1253085,76554] 0 2024-02-15T18:35:37.296 INFO:tasks.workunit.client.1.smithi121.stdout:9/527: sync 2024-02-15T18:35:37.296 INFO:tasks.workunit.client.1.smithi121.stdout:5/621: fsync d0/d2/d1e/d4e/d7d/d9d/d4c/f8d 0 2024-02-15T18:35:37.296 INFO:tasks.workunit.client.1.smithi121.stdout:4/348: write d5/d31/d3d/d4f/d51/f5d [655133,101277] 0 2024-02-15T18:35:37.296 INFO:tasks.workunit.client.1.smithi121.stdout:4/349: readlink d5/d9/l1b 0 2024-02-15T18:35:37.296 INFO:tasks.workunit.client.1.smithi121.stdout:4/350: chown d5/d31/c34 103 1 2024-02-15T18:35:37.296 INFO:tasks.workunit.client.1.smithi121.stdout:2/331: write d5/df/d27/f21 [1900245,84561] 0 2024-02-15T18:35:37.297 INFO:tasks.workunit.client.1.smithi121.stdout:4/351: stat d5/d9/d19/d2a/c2b 0 2024-02-15T18:35:37.297 INFO:tasks.workunit.client.1.smithi121.stdout:6/545: truncate d7/dc/d34/d42/d62/dac/fb3 2319229 0 2024-02-15T18:35:37.298 INFO:tasks.workunit.client.1.smithi121.stdout:6/546: write d7/dc/d2c/d36/f3e [2249720,100357] 0 2024-02-15T18:35:37.302 INFO:tasks.workunit.client.1.smithi121.stdout:0/639: mkdir d5/d6b/dab/db4/d7d/d2a/d73/d81/ddb 0 2024-02-15T18:35:37.303 INFO:tasks.workunit.client.1.smithi121.stdout:7/334: creat d0/d5/d11/d1c/d5f/f73 x:0 0 0 2024-02-15T18:35:37.303 INFO:tasks.workunit.client.1.smithi121.stdout:7/335: readlink d0/d33/l36 0 2024-02-15T18:35:37.308 INFO:tasks.workunit.client.1.smithi121.stdout:3/432: rename d2/d1a/d6b/d72/f76 to d2/d1a/d6b/f91 0 2024-02-15T18:35:37.310 INFO:tasks.workunit.client.1.smithi121.stdout:5/622: mkdir d0/d16/d34/da4/dcc 0 2024-02-15T18:35:37.313 INFO:tasks.workunit.client.1.smithi121.stdout:9/528: mknod d7/d26/d4f/d5b/d77/cc1 0 2024-02-15T18:35:37.313 INFO:tasks.workunit.client.1.smithi121.stdout:9/529: truncate d7/d3a/d8e/f98 189230 0 2024-02-15T18:35:37.314 INFO:tasks.workunit.client.1.smithi121.stdout:7/336: write d0/d5/d11/d1c/f3e [3248942,123953] 0 2024-02-15T18:35:37.317 INFO:tasks.workunit.client.1.smithi121.stdout:4/352: rename d5/d7/c18 to d5/d7/d14/c6c 0 2024-02-15T18:35:37.320 INFO:tasks.workunit.client.1.smithi121.stdout:8/527: dwrite d0/d7/d36/d8c/d45/f3e [0,4194304] 0 2024-02-15T18:35:37.322 INFO:tasks.workunit.client.1.smithi121.stdout:5/623: creat d0/d16/d34/d8f/fcd x:0 0 0 2024-02-15T18:35:37.323 INFO:tasks.workunit.client.1.smithi121.stdout:7/337: mkdir d0/d4a/d5b/d74 0 2024-02-15T18:35:37.323 INFO:tasks.workunit.client.1.smithi121.stdout:7/338: write d0/d5/f66 [665631,28769] 0 2024-02-15T18:35:37.324 INFO:tasks.workunit.client.1.smithi121.stdout:6/547: dwrite d7/dc/d34/d42/d6e/d94/f9a [0,4194304] 0 2024-02-15T18:35:37.324 INFO:tasks.workunit.client.1.smithi121.stdout:6/548: read d7/dc/d34/d42/d62/d63/d8d/fa7 [265611,12709] 0 2024-02-15T18:35:37.327 INFO:tasks.workunit.client.1.smithi121.stdout:4/353: symlink d5/d9/d19/d2a/d4d/l6d 0 2024-02-15T18:35:37.327 INFO:tasks.workunit.client.1.smithi121.stdout:4/354: truncate d5/d7/d14/f55 989184 0 2024-02-15T18:35:37.327 INFO:tasks.workunit.client.1.smithi121.stdout:4/355: write d5/d31/d3d/d4a/f60 [1002126,57516] 0 2024-02-15T18:35:37.327 INFO:tasks.workunit.client.1.smithi121.stdout:4/356: fdatasync d5/d7/d14/f55 0 2024-02-15T18:35:37.331 INFO:tasks.workunit.client.1.smithi121.stdout:2/332: rename d5/df/d27/d3e/d47 to d5/df/d27/d61 0 2024-02-15T18:35:37.331 INFO:tasks.workunit.client.1.smithi121.stdout:2/333: chown d5/df/l12 1447096981 1 2024-02-15T18:35:37.331 INFO:tasks.workunit.client.1.smithi121.stdout:2/334: write d5/d49/d4d/d22/f37 [4741507,56826] 0 2024-02-15T18:35:37.332 INFO:tasks.workunit.client.1.smithi121.stdout:0/640: dwrite d5/f3b [0,4194304] 0 2024-02-15T18:35:37.332 INFO:tasks.workunit.client.1.smithi121.stdout:0/641: chown d5/d6b/dab/db4/d7d/d2a/f91 194651 1 2024-02-15T18:35:37.334 INFO:tasks.workunit.client.1.smithi121.stdout:5/624: creat d0/d16/d34/da4/dcc/fce x:0 0 0 2024-02-15T18:35:37.335 INFO:tasks.workunit.client.1.smithi121.stdout:7/339: truncate d0/d38/d61/f72 2600321 0 2024-02-15T18:35:37.338 INFO:tasks.workunit.client.1.smithi121.stdout:6/549: mknod d7/d4b/d59/cbc 0 2024-02-15T18:35:37.338 INFO:tasks.workunit.client.1.smithi121.stdout:6/550: fsync d7/dc/f37 0 2024-02-15T18:35:37.340 INFO:tasks.workunit.client.1.smithi121.stdout:2/335: creat d5/df/d27/d3e/f62 x:0 0 0 2024-02-15T18:35:37.340 INFO:tasks.workunit.client.1.smithi121.stdout:2/336: dread - d5/df/d27/d61/f5c zero size 2024-02-15T18:35:37.340 INFO:tasks.workunit.client.1.smithi121.stdout:2/337: dread - d5/d49/d4d/d22/f4f zero size 2024-02-15T18:35:37.340 INFO:tasks.workunit.client.1.smithi121.stdout:2/338: truncate d5/d49/d4d/d22/f4b 507619 0 2024-02-15T18:35:37.340 INFO:tasks.workunit.client.1.smithi121.stdout:2/339: chown d5/d49/d4d/d58 0 1 2024-02-15T18:35:37.341 INFO:tasks.workunit.client.1.smithi121.stdout:0/642: symlink d5/d6b/dab/db4/d7d/d2a/d73/d81/ldc 0 2024-02-15T18:35:37.343 INFO:tasks.workunit.client.1.smithi121.stdout:8/528: dwrite d0/d2a/f97 [0,4194304] 0 2024-02-15T18:35:37.343 INFO:tasks.workunit.client.1.smithi121.stdout:5/625: chown d0/d2/d1e/d4e/d7d/c2f 1026524602 1 2024-02-15T18:35:37.344 INFO:tasks.workunit.client.1.smithi121.stdout:6/551: creat d7/dc/d34/fbd x:0 0 0 2024-02-15T18:35:37.344 INFO:tasks.workunit.client.1.smithi121.stdout:6/552: stat d7/f8 0 2024-02-15T18:35:37.344 INFO:tasks.workunit.client.1.smithi121.stdout:6/553: stat d7/dc/d8e/f96 0 2024-02-15T18:35:37.344 INFO:tasks.workunit.client.1.smithi121.stdout:6/554: getdents d7/dc/d8e 0 2024-02-15T18:35:37.346 INFO:tasks.workunit.client.1.smithi121.stdout:9/530: rename f5 to d7/d8/d25/fc2 0 2024-02-15T18:35:37.346 INFO:tasks.workunit.client.1.smithi121.stdout:9/531: dread - d7/d8/d36/f7d zero size 2024-02-15T18:35:37.351 INFO:tasks.workunit.client.1.smithi121.stdout:4/357: dwrite d5/d9/d19/f4e [0,4194304] 0 2024-02-15T18:35:37.355 INFO:tasks.workunit.client.1.smithi121.stdout:2/340: mkdir d5/d49/d4d/d58/d63 0 2024-02-15T18:35:37.359 INFO:tasks.workunit.client.1.smithi121.stdout:6/555: mknod d7/dc/d34/d42/d6e/d91/cbe 0 2024-02-15T18:35:37.364 INFO:tasks.workunit.client.1.smithi121.stdout:7/340: rename d0/la to d0/d5/d11/d67/l75 0 2024-02-15T18:35:37.364 INFO:tasks.workunit.client.1.smithi121.stdout:7/341: readlink d0/d33/l36 0 2024-02-15T18:35:37.364 INFO:tasks.workunit.client.1.smithi121.stdout:7/342: truncate d0/d38/d3f/d43/f4e 332676 0 2024-02-15T18:35:37.365 INFO:tasks.workunit.client.1.smithi121.stdout:4/358: symlink d5/d7/d1e/l6e 0 2024-02-15T18:35:37.365 INFO:tasks.workunit.client.1.smithi121.stdout:2/341: symlink d5/d49/d4d/l64 0 2024-02-15T18:35:37.368 INFO:tasks.workunit.client.1.smithi121.stdout:8/529: rename d0/d3/d20 to d0/d7/db1/dbb 0 2024-02-15T18:35:37.368 INFO:tasks.workunit.client.1.smithi121.stdout:8/530: dread - d0/d7/db1/dbb/f8b zero size 2024-02-15T18:35:37.369 INFO:tasks.workunit.client.1.smithi121.stdout:7/343: mkdir d0/d5/d76 0 2024-02-15T18:35:37.369 INFO:tasks.workunit.client.1.smithi121.stdout:7/344: readlink d0/l14 0 2024-02-15T18:35:37.369 INFO:tasks.workunit.client.1.smithi121.stdout:4/359: symlink d5/d31/d3d/d4a/l6f 0 2024-02-15T18:35:37.369 INFO:tasks.workunit.client.1.smithi121.stdout:4/360: dread - d5/d7/d1e/f48 zero size 2024-02-15T18:35:37.371 INFO:tasks.workunit.client.1.smithi121.stdout:0/643: dwrite d5/d6b/dab/db4/d7d/dd/f22 [0,4194304] 0 2024-02-15T18:35:37.371 INFO:tasks.workunit.client.1.smithi121.stdout:2/342: creat d5/f65 x:0 0 0 2024-02-15T18:35:37.372 INFO:tasks.workunit.client.1.smithi121.stdout:8/531: creat d0/d7/de/fbc x:0 0 0 2024-02-15T18:35:37.372 INFO:tasks.workunit.client.1.smithi121.stdout:8/532: chown d0/d7/de/d7c/f82 2 1 2024-02-15T18:35:37.372 INFO:tasks.workunit.client.1.smithi121.stdout:9/532: dwrite d7/d8/f38 [0,4194304] 0 2024-02-15T18:35:37.373 INFO:tasks.workunit.client.1.smithi121.stdout:7/345: getdents d0/d4a 0 2024-02-15T18:35:37.374 INFO:tasks.workunit.client.1.smithi121.stdout:4/361: mknod d5/d7/d14/c70 0 2024-02-15T18:35:37.374 INFO:tasks.workunit.client.1.smithi121.stdout:0/644: creat d5/d30/fdd x:0 0 0 2024-02-15T18:35:37.376 INFO:tasks.workunit.client.1.smithi121.stdout:2/343: mknod d5/df/d3f/c66 0 2024-02-15T18:35:37.376 INFO:tasks.workunit.client.1.smithi121.stdout:2/344: fsync d5/d49/d4d/d22/f42 0 2024-02-15T18:35:37.379 INFO:tasks.workunit.client.1.smithi121.stdout:7/346: rename d0/d5/f66 to d0/f77 0 2024-02-15T18:35:37.383 INFO:tasks.workunit.client.1.smithi121.stdout:6/556: dwrite d7/f11 [0,4194304] 0 2024-02-15T18:35:37.383 INFO:tasks.workunit.client.1.smithi121.stdout:2/345: symlink d5/df/d10/l67 0 2024-02-15T18:35:37.383 INFO:tasks.workunit.client.1.smithi121.stdout:2/346: chown d5/fd 67 1 2024-02-15T18:35:37.384 INFO:tasks.workunit.client.1.smithi121.stdout:9/533: write d7/f23 [3710966,107956] 0 2024-02-15T18:35:37.386 INFO:tasks.workunit.client.1.smithi121.stdout:8/533: dread d0/d7/de/f1a [4194304,4194304] 0 2024-02-15T18:35:37.386 INFO:tasks.workunit.client.1.smithi121.stdout:8/534: read - d0/d7/d36/f5c zero size 2024-02-15T18:35:37.390 INFO:tasks.workunit.client.1.smithi121.stdout:4/362: dread d5/d9/d1d/f42 [0,4194304] 0 2024-02-15T18:35:37.392 INFO:tasks.workunit.client.1.smithi121.stdout:7/347: link d0/d33/f3b d0/d38/d61/f78 0 2024-02-15T18:35:37.392 INFO:tasks.workunit.client.1.smithi121.stdout:7/348: dread - d0/d33/f6e zero size 2024-02-15T18:35:37.393 INFO:tasks.workunit.client.1.smithi121.stdout:7/349: fdatasync d0/d5/f26 0 2024-02-15T18:35:37.393 INFO:tasks.workunit.client.1.smithi121.stdout:7/350: write d0/d33/f6e [91156,112368] 0 2024-02-15T18:35:37.393 INFO:tasks.workunit.client.1.smithi121.stdout:7/351: write d0/d5/f63 [166855,58889] 0 2024-02-15T18:35:37.393 INFO:tasks.workunit.client.1.smithi121.stdout:6/557: link d7/dc/d34/c69 d7/dae/cbf 0 2024-02-15T18:35:37.393 INFO:tasks.workunit.client.1.smithi121.stdout:6/558: write d7/f8 [3715484,111594] 0 2024-02-15T18:35:37.393 INFO:tasks.workunit.client.1.smithi121.stdout:6/559: fdatasync d7/dc/d34/d42/d6e/fad 0 2024-02-15T18:35:37.394 INFO:tasks.workunit.client.1.smithi121.stdout:8/535: creat d0/fbd x:0 0 0 2024-02-15T18:35:37.394 INFO:tasks.workunit.client.1.smithi121.stdout:8/536: write d0/d73/fb7 [7129,44457] 0 2024-02-15T18:35:37.396 INFO:tasks.workunit.client.1.smithi121.stdout:7/352: mkdir d0/d5/d11/d79 0 2024-02-15T18:35:37.397 INFO:tasks.workunit.client.1.smithi121.stdout:6/560: mknod d7/dc/d34/d42/d6e/d94/cc0 0 2024-02-15T18:35:37.398 INFO:tasks.workunit.client.1.smithi121.stdout:8/537: rename d0/d7/d36/d8c/d45/f94 to d0/d7/de/d67/fbe 0 2024-02-15T18:35:37.400 INFO:tasks.workunit.client.1.smithi121.stdout:0/645: dwrite d5/d30/f33 [0,4194304] 0 2024-02-15T18:35:37.401 INFO:tasks.workunit.client.1.smithi121.stdout:5/626: unlink d0/d2/d18/f27 0 2024-02-15T18:35:37.402 INFO:tasks.workunit.client.1.smithi121.stdout:4/363: unlink d5/d7/d14/c65 0 2024-02-15T18:35:37.402 INFO:tasks.workunit.client.1.smithi121.stdout:4/364: write d5/f30 [742445,113980] 0 2024-02-15T18:35:37.411 INFO:tasks.workunit.client.1.smithi121.stdout:2/347: dwrite d5/d49/d4d/d22/f4b [0,4194304] 0 2024-02-15T18:35:37.411 INFO:tasks.workunit.client.1.smithi121.stdout:2/348: readlink d5/d49/d4d/l64 0 2024-02-15T18:35:37.411 INFO:tasks.workunit.client.1.smithi121.stdout:8/538: symlink d0/d7/d36/d52/lbf 0 2024-02-15T18:35:37.411 INFO:tasks.workunit.client.1.smithi121.stdout:5/627: mknod d0/d2/d1e/d4e/d7d/d9d/d4c/ccf 0 2024-02-15T18:35:37.413 INFO:tasks.workunit.client.1.smithi121.stdout:4/365: rename d5/d7/d1e to d5/d31/d3d/d4f/d51/d71 0 2024-02-15T18:35:37.417 INFO:tasks.workunit.client.1.smithi121.stdout:2/349: write d5/df/f2a [2436130,127688] 0 2024-02-15T18:35:37.418 INFO:tasks.workunit.client.1.smithi121.stdout:2/350: symlink d5/df/l68 0 2024-02-15T18:35:37.418 INFO:tasks.workunit.client.1.smithi121.stdout:8/539: getdents d0/d2a/d51 0 2024-02-15T18:35:37.418 INFO:tasks.workunit.client.1.smithi121.stdout:2/351: dread - d5/df/f39 zero size 2024-02-15T18:35:37.418 INFO:tasks.workunit.client.1.smithi121.stdout:8/540: fsync d0/d2a/f97 0 2024-02-15T18:35:37.419 INFO:tasks.workunit.client.1.smithi121.stdout:4/366: mkdir d5/d9/d1d/d26/d72 0 2024-02-15T18:35:37.420 INFO:tasks.workunit.client.1.smithi121.stdout:2/352: truncate d5/d49/d4d/d22/f44 2536172 0 2024-02-15T18:35:37.421 INFO:tasks.workunit.client.1.smithi121.stdout:4/367: truncate d5/d9/f20 929772 0 2024-02-15T18:35:37.423 INFO:tasks.workunit.client.1.smithi121.stdout:2/353: creat d5/df/d3f/f69 x:0 0 0 2024-02-15T18:35:37.435 INFO:tasks.workunit.client.1.smithi121.stdout:2/354: dread d5/d49/d4d/d22/f37 [0,4194304] 0 2024-02-15T18:35:37.436 INFO:tasks.workunit.client.1.smithi121.stdout:2/355: chown d5/df/d3f/l57 124407 1 2024-02-15T18:35:37.436 INFO:tasks.workunit.client.1.smithi121.stdout:6/561: dwrite d7/dc/d2c/d36/f9d [0,4194304] 0 2024-02-15T18:35:37.436 INFO:tasks.workunit.client.1.smithi121.stdout:6/562: fdatasync d7/dc/d34/d7c/faa 0 2024-02-15T18:35:37.437 INFO:tasks.workunit.client.1.smithi121.stdout:2/356: getdents d5/df/d27/d3a 0 2024-02-15T18:35:37.439 INFO:tasks.workunit.client.1.smithi121.stdout:6/563: mknod d7/dc/d34/d42/d6e/cc1 0 2024-02-15T18:35:37.447 INFO:tasks.workunit.client.1.smithi121.stdout:7/353: getdents d0/d5/d11 0 2024-02-15T18:35:37.466 INFO:tasks.workunit.client.1.smithi121.stdout:7/354: dread d0/d5/d11/d1c/f1f [0,4194304] 0 2024-02-15T18:35:37.468 INFO:tasks.workunit.client.1.smithi121.stdout:5/628: dwrite d0/d16/d34/f43 [0,4194304] 0 2024-02-15T18:35:37.470 INFO:tasks.workunit.client.1.smithi121.stdout:5/629: truncate d0/d2/d1e/d4e/d7d/d9d/f56 47644 0 2024-02-15T18:35:37.471 INFO:tasks.workunit.client.1.smithi121.stdout:4/368: dwrite d5/d7/d14/f55 [0,4194304] 0 2024-02-15T18:35:37.472 INFO:tasks.workunit.client.1.smithi121.stdout:0/646: dwrite d5/d6b/dab/db4/d7d/d2a/fda [0,4194304] 0 2024-02-15T18:35:37.472 INFO:tasks.workunit.client.1.smithi121.stdout:5/630: symlink d0/d2/d1e/d4e/d7d/d9d/d4c/daa/ld0 0 2024-02-15T18:35:37.472 INFO:tasks.workunit.client.1.smithi121.stdout:8/541: dwrite d0/d7/d36/d8c/d45/f89 [0,4194304] 0 2024-02-15T18:35:37.472 INFO:tasks.workunit.client.1.smithi121.stdout:6/564: dwrite d7/f18 [0,4194304] 0 2024-02-15T18:35:37.472 INFO:tasks.workunit.client.1.smithi121.stdout:8/542: readlink d0/d61/d66/l44 0 2024-02-15T18:35:37.474 INFO:tasks.workunit.client.1.smithi121.stdout:2/357: dwrite d5/d49/d4d/d22/f3c [0,4194304] 0 2024-02-15T18:35:37.480 INFO:tasks.workunit.client.1.smithi121.stdout:4/369: mkdir d5/d9/d1d/d26/d73 0 2024-02-15T18:35:37.484 INFO:tasks.workunit.client.1.smithi121.stdout:0/647: rename d5/d6b/dab/db4/d7d/dd/f2d to d5/d6b/dab/db4/d7d/d2a/d73/d8d/fde 0 2024-02-15T18:35:37.484 INFO:tasks.workunit.client.1.smithi121.stdout:0/648: truncate d5/fa4 602829 0 2024-02-15T18:35:37.486 INFO:tasks.workunit.client.1.smithi121.stdout:5/631: getdents d0/d2/d1e 0 2024-02-15T18:35:37.490 INFO:tasks.workunit.client.1.smithi121.stdout:6/565: truncate d7/f31 4088663 0 2024-02-15T18:35:37.494 INFO:tasks.workunit.client.1.smithi121.stdout:7/355: dwrite d0/d5/d11/d1c/f44 [0,4194304] 0 2024-02-15T18:35:37.495 INFO:tasks.workunit.client.1.smithi121.stdout:7/356: dread d0/d5/d11/f48 [0,4194304] 0 2024-02-15T18:35:37.499 INFO:tasks.workunit.client.1.smithi121.stdout:2/358: dwrite d5/d49/d4d/d22/f4b [0,4194304] 0 2024-02-15T18:35:37.499 INFO:tasks.workunit.client.1.smithi121.stdout:2/359: dread - d5/df/d10/d4c/f50 zero size 2024-02-15T18:35:37.499 INFO:tasks.workunit.client.1.smithi121.stdout:2/360: read - d5/df/f39 zero size 2024-02-15T18:35:37.515 INFO:tasks.workunit.client.1.smithi121.stdout:0/649: mknod d5/d6b/dab/db4/d7d/d2a/d73/cdf 0 2024-02-15T18:35:37.515 INFO:tasks.workunit.client.1.smithi121.stdout:4/370: creat d5/d9/d19/d28/f74 x:0 0 0 2024-02-15T18:35:37.515 INFO:tasks.workunit.client.1.smithi121.stdout:5/632: creat d0/d2/d1e/d4e/d7d/d9d/d4c/daa/fd1 x:0 0 0 2024-02-15T18:35:37.515 INFO:tasks.workunit.client.1.smithi121.stdout:5/633: chown d0/d2/d18/f31 8823 1 2024-02-15T18:35:37.515 INFO:tasks.workunit.client.1.smithi121.stdout:4/371: chown d5/d7/d14/f17 24348380 1 2024-02-15T18:35:37.515 INFO:tasks.workunit.client.1.smithi121.stdout:4/372: chown d5/d7/d14/f17 9410 1 2024-02-15T18:35:37.515 INFO:tasks.workunit.client.1.smithi121.stdout:4/373: fsync d5/d31/d3d/d4f/d51/f47 0 2024-02-15T18:35:37.517 INFO:tasks.workunit.client.1.smithi121.stdout:7/357: rmdir d0/d5 39 2024-02-15T18:35:37.520 INFO:tasks.workunit.client.1.smithi121.stdout:7/358: dread d0/d5/f47 [0,4194304] 0 2024-02-15T18:35:37.520 INFO:tasks.workunit.client.1.smithi121.stdout:7/359: chown d0/d5/d11/l68 58 1 2024-02-15T18:35:37.520 INFO:tasks.workunit.client.1.smithi121.stdout:7/360: fdatasync d0/d38/d3f/d43/f4e 0 2024-02-15T18:35:37.521 INFO:tasks.workunit.client.1.smithi121.stdout:7/361: creat d0/d5/d11/d1c/f7a x:0 0 0 2024-02-15T18:35:37.521 INFO:tasks.workunit.client.1.smithi121.stdout:7/362: write d0/d5/d11/f40 [9269674,103848] 0 2024-02-15T18:35:37.525 INFO:tasks.workunit.client.1.smithi121.stdout:7/363: read d0/d38/d61/f78 [82221,111965] 0 2024-02-15T18:35:37.533 INFO:tasks.workunit.client.1.smithi121.stdout:7/364: rename d0/d5/d11/d1c/c3d to d0/c7b 0 2024-02-15T18:35:37.534 INFO:tasks.workunit.client.1.smithi121.stdout:7/365: write d0/f77 [348767,28312] 0 2024-02-15T18:35:37.537 INFO:tasks.workunit.client.1.smithi121.stdout:7/366: creat d0/d5/d11/d1c/d5f/f7c x:0 0 0 2024-02-15T18:35:37.537 INFO:tasks.workunit.client.1.smithi121.stdout:7/367: read d0/d5/d11/d1c/f1f [1178480,91502] 0 2024-02-15T18:35:37.538 INFO:tasks.workunit.client.1.smithi121.stdout:7/368: creat d0/d5/d11/d1c/d5f/f7d x:0 0 0 2024-02-15T18:35:37.539 INFO:tasks.workunit.client.1.smithi121.stdout:7/369: dread - d0/d5/d11/f71 zero size 2024-02-15T18:35:37.539 INFO:tasks.workunit.client.1.smithi121.stdout:7/370: fdatasync d0/d5/d11/d1c/f3e 0 2024-02-15T18:35:37.545 INFO:tasks.workunit.client.1.smithi121.stdout:5/634: dwrite d0/d16/d7a/db5/fcb [0,4194304] 0 2024-02-15T18:35:37.546 INFO:tasks.workunit.client.1.smithi121.stdout:5/635: chown d0/d16/d7a/db5/fcb 11856515 1 2024-02-15T18:35:37.546 INFO:tasks.workunit.client.1.smithi121.stdout:5/636: dread - d0/d2/d1e/d4e/d7d/d47/d5a/da7/fbd zero size 2024-02-15T18:35:37.546 INFO:tasks.workunit.client.1.smithi121.stdout:7/371: dread d0/d5/f63 [0,4194304] 0 2024-02-15T18:35:37.546 INFO:tasks.workunit.client.1.smithi121.stdout:7/372: chown d0/d33/f34 12462735 1 2024-02-15T18:35:37.546 INFO:tasks.workunit.client.1.smithi121.stdout:7/373: readlink d0/l14 0 2024-02-15T18:35:37.546 INFO:tasks.workunit.client.1.smithi121.stdout:7/374: readlink d0/d5/l1a 0 2024-02-15T18:35:37.546 INFO:tasks.workunit.client.1.smithi121.stdout:4/374: dwrite d5/d7/d14/f29 [0,4194304] 0 2024-02-15T18:35:37.559 INFO:tasks.workunit.client.1.smithi121.stdout:5/637: truncate d0/d2/d70/f8a 2463253 0 2024-02-15T18:35:37.563 INFO:tasks.workunit.client.1.smithi121.stdout:4/375: rename d5/d9/d19/d2a/f32 to d5/d7/f75 0 2024-02-15T18:35:37.564 INFO:tasks.workunit.client.1.smithi121.stdout:5/638: mknod d0/d16/da1/cd2 0 2024-02-15T18:35:37.564 INFO:tasks.workunit.client.1.smithi121.stdout:5/639: readlink d0/d2/l3d 0 2024-02-15T18:35:37.566 INFO:tasks.workunit.client.1.smithi121.stdout:7/375: rmdir d0/d4a/d5b/d74 0 2024-02-15T18:35:37.566 INFO:tasks.workunit.client.1.smithi121.stdout:7/376: truncate d0/d5/d11/d1c/f21 4725533 0 2024-02-15T18:35:37.567 INFO:tasks.workunit.client.1.smithi121.stdout:4/376: mkdir d5/d9/d76 0 2024-02-15T18:35:37.569 INFO:tasks.workunit.client.1.smithi121.stdout:5/640: mknod d0/d2/d1e/d4e/d7d/d40/db9/cd3 0 2024-02-15T18:35:37.570 INFO:tasks.workunit.client.1.smithi121.stdout:7/377: dread d0/d33/f6e [0,4194304] 0 2024-02-15T18:35:37.572 INFO:tasks.workunit.client.1.smithi121.stdout:7/378: rmdir d0/d4a/d5b 39 2024-02-15T18:35:37.575 INFO:tasks.workunit.client.1.smithi121.stdout:5/641: dread d0/d2/d1e/d4e/d7d/d39/f53 [0,4194304] 0 2024-02-15T18:35:37.576 INFO:tasks.workunit.client.1.smithi121.stdout:5/642: creat d0/d16/da1/fd4 x:0 0 0 2024-02-15T18:35:37.581 INFO:tasks.workunit.client.1.smithi121.stdout:5/643: dread d0/d2/d1e/d4e/d7d/f2c [0,4194304] 0 2024-02-15T18:35:37.593 INFO:tasks.workunit.client.1.smithi121.stdout:4/377: dwrite d5/fa [0,4194304] 0 2024-02-15T18:35:37.593 INFO:tasks.workunit.client.1.smithi121.stdout:4/378: fsync d5/d7/f6a 0 2024-02-15T18:35:37.598 INFO:tasks.workunit.client.1.smithi121.stdout:5/644: rmdir d0/d16/da1 39 2024-02-15T18:35:37.598 INFO:tasks.workunit.client.1.smithi121.stdout:5/645: fsync d0/d16/da1/fc7 0 2024-02-15T18:35:37.602 INFO:tasks.workunit.client.1.smithi121.stdout:4/379: creat d5/d9/d19/d57/f77 x:0 0 0 2024-02-15T18:35:37.606 INFO:tasks.workunit.client.1.smithi121.stdout:4/380: link d5/d9/d19/d57/f77 d5/d9/d1d/d26/d72/f78 0 2024-02-15T18:35:37.618 INFO:tasks.workunit.client.1.smithi121.stdout:4/381: dread f1 [0,4194304] 0 2024-02-15T18:35:37.618 INFO:tasks.workunit.client.1.smithi121.stdout:4/382: write d5/d9/f2e [817419,129426] 0 2024-02-15T18:35:37.619 INFO:tasks.workunit.client.1.smithi121.stdout:4/383: mknod d5/d31/d25/c79 0 2024-02-15T18:35:37.620 INFO:tasks.workunit.client.1.smithi121.stdout:4/384: symlink d5/d9/d19/d4c/l7a 0 2024-02-15T18:35:37.621 INFO:tasks.workunit.client.1.smithi121.stdout:4/385: symlink d5/d9/d1d/d26/d73/l7b 0 2024-02-15T18:35:37.622 INFO:tasks.workunit.client.1.smithi121.stdout:4/386: mknod d5/d9/d19/c7c 0 2024-02-15T18:35:37.628 INFO:tasks.workunit.client.1.smithi121.stdout:4/387: truncate d5/d7/d14/f55 3802112 0 2024-02-15T18:35:37.631 INFO:tasks.workunit.client.1.smithi121.stdout:5/646: dwrite d0/d2/d18/f20 [0,4194304] 0 2024-02-15T18:35:37.631 INFO:tasks.workunit.client.1.smithi121.stdout:5/647: truncate d0/d16/da1/fc7 535501 0 2024-02-15T18:35:37.637 INFO:tasks.workunit.client.1.smithi121.stdout:5/648: symlink d0/d16/d34/d8f/ld5 0 2024-02-15T18:35:37.637 INFO:tasks.workunit.client.1.smithi121.stdout:5/649: stat d0/d2/d1e/d4e/d7d/d9d/lb2 0 2024-02-15T18:35:37.639 INFO:tasks.workunit.client.1.smithi121.stdout:5/650: mkdir d0/d16/d7a/db5/dd6 0 2024-02-15T18:35:37.639 INFO:tasks.workunit.client.1.smithi121.stdout:5/651: stat d0/d2/d1e 0 2024-02-15T18:35:37.640 INFO:tasks.workunit.client.1.smithi121.stdout:5/652: symlink d0/d16/d7a/ld7 0 2024-02-15T18:35:37.640 INFO:tasks.workunit.client.1.smithi121.stdout:5/653: truncate d0/d2/d1e/d4e/d7d/d40/d62/db1/fc1 793143 0 2024-02-15T18:35:37.641 INFO:tasks.workunit.client.1.smithi121.stdout:5/654: creat d0/d2/fd8 x:0 0 0 2024-02-15T18:35:37.641 INFO:tasks.workunit.client.1.smithi121.stdout:5/655: write d0/d2/d1e/d4e/d7d/d9d/fc4 [805522,111353] 0 2024-02-15T18:35:37.641 INFO:tasks.workunit.client.1.smithi121.stdout:5/656: readlink d0/d2/l2e 0 2024-02-15T18:35:37.641 INFO:tasks.workunit.client.1.smithi121.stdout:4/388: dread d5/d9/d1d/f2f [0,4194304] 0 2024-02-15T18:35:37.646 INFO:tasks.workunit.client.1.smithi121.stdout:4/389: mknod d5/d31/d3d/d4f/d51/c7d 0 2024-02-15T18:35:37.646 INFO:tasks.workunit.client.1.smithi121.stdout:4/390: chown d5/d9/d19/d5c/f69 0 1 2024-02-15T18:35:37.648 INFO:tasks.workunit.client.1.smithi121.stdout:4/391: getdents d5/d7 0 2024-02-15T18:35:37.648 INFO:tasks.workunit.client.1.smithi121.stdout:4/392: chown d5/d9/d19/d5c 13 1 2024-02-15T18:35:37.649 INFO:tasks.workunit.client.1.smithi121.stdout:4/393: mkdir d5/d31/d3d/d4a/d7e 0 2024-02-15T18:35:37.649 INFO:tasks.workunit.client.1.smithi121.stdout:4/394: dread - d5/f64 zero size 2024-02-15T18:35:37.651 INFO:tasks.workunit.client.1.smithi121.stdout:4/395: write f1 [233517,55761] 0 2024-02-15T18:35:37.651 INFO:tasks.workunit.client.1.smithi121.stdout:4/396: stat d5/d7/d14/c70 0 2024-02-15T18:35:37.653 INFO:tasks.workunit.client.1.smithi121.stdout:4/397: creat d5/d31/d3d/d4a/d7e/f7f x:0 0 0 2024-02-15T18:35:37.653 INFO:tasks.workunit.client.1.smithi121.stdout:4/398: chown d5/c12 8 1 2024-02-15T18:35:37.654 INFO:tasks.workunit.client.1.smithi121.stdout:4/399: mkdir d5/d31/d3d/d4f/d51/d80 0 2024-02-15T18:35:37.654 INFO:tasks.workunit.client.1.smithi121.stdout:4/400: write d5/d31/d3d/d4f/d51/d71/f48 [971118,115374] 0 2024-02-15T18:35:37.654 INFO:tasks.workunit.client.1.smithi121.stdout:4/401: chown d5/d9/d19 784427791 1 2024-02-15T18:35:37.654 INFO:tasks.workunit.client.1.smithi121.stdout:4/402: fsync d5/fa 0 2024-02-15T18:35:37.656 INFO:tasks.workunit.client.1.smithi121.stdout:4/403: mkdir d5/d9/d1d/d26/d73/d81 0 2024-02-15T18:35:37.661 INFO:tasks.workunit.client.1.smithi121.stdout:4/404: write d5/d7/f38 [1879628,27017] 0 2024-02-15T18:35:37.662 INFO:tasks.workunit.client.1.smithi121.stdout:5/657: dwrite d0/d16/d34/f6a [0,4194304] 0 2024-02-15T18:35:37.662 INFO:tasks.workunit.client.1.smithi121.stdout:5/658: fsync d0/d2/d1e/d4e/f80 0 2024-02-15T18:35:37.667 INFO:tasks.workunit.client.1.smithi121.stdout:5/659: rename d0/d16/f68 to d0/d2/d1e/d4e/d7d/d9d/d4c/fd9 0 2024-02-15T18:35:37.670 INFO:tasks.workunit.client.1.smithi121.stdout:4/405: unlink d5/d31/d3d/d4f/d51/d71/f53 0 2024-02-15T18:35:37.670 INFO:tasks.workunit.client.1.smithi121.stdout:5/660: rmdir d0/d16/d34/d8f 39 2024-02-15T18:35:37.670 INFO:tasks.workunit.client.1.smithi121.stdout:5/661: getdents d0/d2/d1e/d4e/d7d/d47/d5a/dbc 0 2024-02-15T18:35:37.671 INFO:tasks.workunit.client.1.smithi121.stdout:4/406: mkdir d5/d9/d19/d4c/d82 0 2024-02-15T18:35:37.672 INFO:tasks.workunit.client.1.smithi121.stdout:5/662: mknod d0/d2/d1e/d4e/cda 0 2024-02-15T18:35:37.674 INFO:tasks.workunit.client.1.smithi121.stdout:5/663: rmdir d0 39 2024-02-15T18:35:37.675 INFO:tasks.workunit.client.1.smithi121.stdout:4/407: write d5/d31/d3d/f49 [800368,12917] 0 2024-02-15T18:35:37.677 INFO:tasks.workunit.client.1.smithi121.stdout:5/664: symlink d0/d2/d70/ldb 0 2024-02-15T18:35:37.677 INFO:tasks.workunit.client.1.smithi121.stdout:5/665: chown d0/d2/d1e/d4e/d7d/l24 12 1 2024-02-15T18:35:37.680 INFO:tasks.workunit.client.1.smithi121.stdout:4/408: dread d5/f6 [0,4194304] 0 2024-02-15T18:35:37.682 INFO:tasks.workunit.client.1.smithi121.stdout:5/666: read d0/d2/d1e/d4e/d7d/d47/d5a/fa0 [3878779,71767] 0 2024-02-15T18:35:37.686 INFO:tasks.workunit.client.1.smithi121.stdout:4/409: mknod d5/c83 0 2024-02-15T18:35:37.687 INFO:tasks.workunit.client.1.smithi121.stdout:5/667: creat d0/d16/d34/d8f/fdc x:0 0 0 2024-02-15T18:35:37.688 INFO:tasks.workunit.client.1.smithi121.stdout:4/410: symlink d5/d9/d1d/d26/d73/d81/l84 0 2024-02-15T18:35:37.690 INFO:tasks.workunit.client.1.smithi121.stdout:5/668: unlink d0/d2/d1e/d4e/d7d/d39/c3c 0 2024-02-15T18:35:37.692 INFO:tasks.workunit.client.1.smithi121.stdout:4/411: dread d5/d31/d25/f37 [0,4194304] 0 2024-02-15T18:35:37.693 INFO:tasks.workunit.client.1.smithi121.stdout:5/669: unlink d0/f9b 0 2024-02-15T18:35:37.693 INFO:tasks.workunit.client.1.smithi121.stdout:5/670: write d0/d2/d1e/d4e/d7d/f2c [147526,16373] 0 2024-02-15T18:35:37.693 INFO:tasks.workunit.client.1.smithi121.stdout:4/412: symlink d5/d31/d3d/d4f/d51/d80/l85 0 2024-02-15T18:35:37.694 INFO:tasks.workunit.client.1.smithi121.stdout:5/671: creat d0/d2/d1e/d4e/d7d/d40/d62/db1/fdd x:0 0 0 2024-02-15T18:35:37.694 INFO:tasks.workunit.client.1.smithi121.stdout:5/672: readlink d0/d2/d1e/d4e/d7d/lb6 0 2024-02-15T18:35:37.700 INFO:tasks.workunit.client.1.smithi121.stdout:4/413: dread d5/d31/f23 [4194304,4194304] 0 2024-02-15T18:35:37.701 INFO:tasks.workunit.client.1.smithi121.stdout:4/414: symlink d5/d9/d1d/d26/l86 0 2024-02-15T18:35:37.703 INFO:tasks.workunit.client.1.smithi121.stdout:4/415: rmdir d5/d7/d14 39 2024-02-15T18:35:37.719 INFO:tasks.workunit.client.1.smithi121.stdout:5/673: dwrite d0/fb [8388608,4194304] 0 2024-02-15T18:35:37.719 INFO:tasks.workunit.client.1.smithi121.stdout:5/674: write d0/d2/faf [4932987,28708] 0 2024-02-15T18:35:37.719 INFO:tasks.workunit.client.1.smithi121.stdout:5/675: write d0/d2/d1e/d4e/f91 [750122,69237] 0 2024-02-15T18:35:37.719 INFO:tasks.workunit.client.1.smithi121.stdout:4/416: dwrite d5/fa [8388608,4194304] 0 2024-02-15T18:35:37.722 INFO:tasks.workunit.client.1.smithi121.stdout:4/417: mkdir d5/d9/d76/d87 0 2024-02-15T18:35:37.722 INFO:tasks.workunit.client.1.smithi121.stdout:5/676: fsync d0/d2/d1e/d4e/f91 0 2024-02-15T18:35:37.723 INFO:tasks.workunit.client.1.smithi121.stdout:4/418: write d5/d9/d1d/f41 [7207824,28083] 0 2024-02-15T18:35:37.730 INFO:tasks.workunit.client.1.smithi121.stdout:5/677: link d0/d2/d1e/d4e/d7d/d40/d62/c69 d0/cde 0 2024-02-15T18:35:37.739 INFO:tasks.workunit.client.1.smithi121.stdout:1/593: sync 2024-02-15T18:35:37.739 INFO:tasks.workunit.client.1.smithi121.stdout:1/594: dread - d0/fbc zero size 2024-02-15T18:35:37.741 INFO:tasks.workunit.client.1.smithi121.stdout:1/595: creat d0/d6/d7/d9/d14/d48/fca x:0 0 0 2024-02-15T18:35:37.744 INFO:tasks.workunit.client.1.smithi121.stdout:1/596: symlink d0/d6/d7/d9/lcb 0 2024-02-15T18:35:37.746 INFO:tasks.workunit.client.1.smithi121.stdout:4/419: dwrite d5/d9/d1d/f2f [0,4194304] 0 2024-02-15T18:35:37.748 INFO:tasks.workunit.client.1.smithi121.stdout:4/420: mknod d5/d9/d76/c88 0 2024-02-15T18:35:37.755 INFO:tasks.workunit.client.1.smithi121.stdout:4/421: dread d5/d9/f2e [0,4194304] 0 2024-02-15T18:35:37.755 INFO:tasks.workunit.client.1.smithi121.stdout:5/678: dwrite d0/d2/d1e/d4e/d7d/d47/fa5 [0,4194304] 0 2024-02-15T18:35:37.757 INFO:tasks.workunit.client.1.smithi121.stdout:5/679: mknod d0/d2/d70/d88/cdf 0 2024-02-15T18:35:37.759 INFO:tasks.workunit.client.1.smithi121.stdout:5/680: rmdir d0/d2/d1e/d4e/d7d/d40/db9 39 2024-02-15T18:35:37.761 INFO:tasks.workunit.client.1.smithi121.stdout:5/681: creat d0/d2/d1e/fe0 x:0 0 0 2024-02-15T18:35:37.761 INFO:tasks.workunit.client.1.smithi121.stdout:1/597: dwrite d0/d6/d7/d9/d14/fbb [0,4194304] 0 2024-02-15T18:35:37.793 INFO:tasks.workunit.client.1.smithi121.stdout:5/682: dwrite d0/d2/d1e/d4e/d7d/fb8 [0,4194304] 0 2024-02-15T18:35:37.795 INFO:tasks.workunit.client.1.smithi121.stdout:5/683: read d0/d2/d1e/d4e/d7d/d39/f51 [3133169,12169] 0 2024-02-15T18:35:37.795 INFO:tasks.workunit.client.1.smithi121.stdout:4/422: dwrite d5/d31/d3d/f49 [0,4194304] 0 2024-02-15T18:35:37.795 INFO:tasks.workunit.client.1.smithi121.stdout:4/423: truncate d5/d9/d19/d4c/f61 901974 0 2024-02-15T18:35:37.796 INFO:tasks.workunit.client.1.smithi121.stdout:4/424: chown d5/d9 4342 1 2024-02-15T18:35:37.797 INFO:tasks.workunit.client.1.smithi121.stdout:5/684: mkdir d0/d2/d70/de1 0 2024-02-15T18:35:37.797 INFO:tasks.workunit.client.1.smithi121.stdout:5/685: chown d0/d2/d1e/d4e/d7d/d47/d5a/da7 1506 1 2024-02-15T18:35:37.799 INFO:tasks.workunit.client.1.smithi121.stdout:5/686: truncate d0/d2/d1e/d4e/d7d/d47/d5a/f5c 3978039 0 2024-02-15T18:35:37.799 INFO:tasks.workunit.client.1.smithi121.stdout:5/687: stat d0/d2/fd8 0 2024-02-15T18:35:37.799 INFO:tasks.workunit.client.1.smithi121.stdout:5/688: read - d0/d2/d1e/d4e/f61 zero size 2024-02-15T18:35:37.799 INFO:tasks.workunit.client.1.smithi121.stdout:5/689: dread - d0/d2/d1e/d4e/d7d/d9d/d4c/daa/fd1 zero size 2024-02-15T18:35:37.799 INFO:tasks.workunit.client.1.smithi121.stdout:5/690: write d0/d16/d34/d8f/fbf [815146,81281] 0 2024-02-15T18:35:37.809 INFO:tasks.workunit.client.1.smithi121.stdout:5/691: dread d0/d2/d1e/f42 [0,4194304] 0 2024-02-15T18:35:37.812 INFO:tasks.workunit.client.1.smithi121.stdout:5/692: readlink d0/d2/d70/ldb 0 2024-02-15T18:35:37.812 INFO:tasks.workunit.client.1.smithi121.stdout:4/425: unlink d5/d9/d19/d4c/f61 0 2024-02-15T18:35:37.814 INFO:tasks.workunit.client.1.smithi121.stdout:4/426: mknod d5/d9/d1d/d26/d72/c89 0 2024-02-15T18:35:37.816 INFO:tasks.workunit.client.1.smithi121.stdout:5/693: link d0/d2/f14 d0/d2/d1e/d4e/d7d/fe2 0 2024-02-15T18:35:37.816 INFO:tasks.workunit.client.1.smithi121.stdout:5/694: write d0/d2/d1e/d4e/d7d/d47/f84 [105019,14643] 0 2024-02-15T18:35:37.818 INFO:tasks.workunit.client.1.smithi121.stdout:5/695: mknod d0/d2/d1e/d4e/d7d/d40/db9/ce3 0 2024-02-15T18:35:37.822 INFO:tasks.workunit.client.1.smithi121.stdout:5/696: unlink d0/d16/d34/l49 0 2024-02-15T18:35:37.823 INFO:tasks.workunit.client.1.smithi121.stdout:5/697: write d0/d2/d1e/d4e/d7d/f2c [2356688,97306] 0 2024-02-15T18:35:37.823 INFO:tasks.workunit.client.1.smithi121.stdout:4/427: dread d5/d31/f23 [0,4194304] 0 2024-02-15T18:35:37.824 INFO:tasks.workunit.client.1.smithi121.stdout:4/428: stat d5/d31/d3d/f49 0 2024-02-15T18:35:37.827 INFO:tasks.workunit.client.1.smithi121.stdout:4/429: mkdir d5/d31/d3d/d4f/d8a 0 2024-02-15T18:35:37.827 INFO:tasks.workunit.client.1.smithi121.stdout:4/430: write f1 [9236841,8638] 0 2024-02-15T18:35:37.844 INFO:tasks.workunit.client.1.smithi121.stdout:5/698: dwrite d0/d2/d1e/d4e/d7d/d9d/d4c/fd9 [4194304,4194304] 0 2024-02-15T18:35:37.848 INFO:tasks.workunit.client.1.smithi121.stdout:5/699: rmdir d0/d2/d1e/d4e/d7d/d40 39 2024-02-15T18:35:37.848 INFO:tasks.workunit.client.1.smithi121.stdout:5/700: dread - d0/d2/f8e zero size 2024-02-15T18:35:37.850 INFO:tasks.workunit.client.1.smithi121.stdout:5/701: rename d0/d2/d1e/d4e/d7d/d40/d62/c69 to d0/d2/d1e/d4e/d7d/d40/ce4 0 2024-02-15T18:35:37.853 INFO:tasks.workunit.client.1.smithi121.stdout:4/431: dwrite d5/d9/d1d/f41 [0,4194304] 0 2024-02-15T18:35:37.853 INFO:tasks.workunit.client.1.smithi121.stdout:4/432: write d5/f64 [459280,14199] 0 2024-02-15T18:35:37.854 INFO:tasks.workunit.client.1.smithi121.stdout:4/433: getdents d5/d9/d1d/d26/d73/d81 0 2024-02-15T18:35:37.886 INFO:tasks.workunit.client.1.smithi121.stdout:5/702: dwrite d0/d2/d18/f20 [0,4194304] 0 2024-02-15T18:35:37.887 INFO:tasks.workunit.client.1.smithi121.stdout:5/703: symlink d0/d2/d1e/d4e/d7d/d40/le5 0 2024-02-15T18:35:37.890 INFO:tasks.workunit.client.1.smithi121.stdout:5/704: rename d0/d2/d1e/d4e/d7d/d39/lc5 to d0/d16/le6 0 2024-02-15T18:35:37.891 INFO:tasks.workunit.client.1.smithi121.stdout:4/434: dwrite d5/d9/d19/d2a/d4d/f54 [0,4194304] 0 2024-02-15T18:35:37.891 INFO:tasks.workunit.client.1.smithi121.stdout:4/435: read - d5/d31/d3d/d4a/d7e/f7f zero size 2024-02-15T18:35:37.892 INFO:tasks.workunit.client.1.smithi121.stdout:5/705: rename d0/d16/d34/f6a to d0/d2/d1e/d4e/d7d/d39/fe7 0 2024-02-15T18:35:37.892 INFO:tasks.workunit.client.1.smithi121.stdout:5/706: readlink d0/d16/l67 0 2024-02-15T18:35:37.893 INFO:tasks.workunit.client.1.smithi121.stdout:5/707: rmdir d0/d2/d18 39 2024-02-15T18:35:37.895 INFO:tasks.workunit.client.1.smithi121.stdout:5/708: rename d0/d2/d1e/d4e/d7d/d39/l3b to d0/d16/d34/da4/le8 0 2024-02-15T18:35:37.905 INFO:tasks.workunit.client.1.smithi121.stdout:4/436: dread d5/f64 [0,4194304] 0 2024-02-15T18:35:37.905 INFO:tasks.workunit.client.1.smithi121.stdout:4/437: readlink d5/l3a 0 2024-02-15T18:35:37.907 INFO:tasks.workunit.client.1.smithi121.stdout:4/438: unlink d5/d9/d1d/c3b 0 2024-02-15T18:35:37.908 INFO:tasks.workunit.client.1.smithi121.stdout:4/439: truncate d5/f6 1052639 0 2024-02-15T18:35:37.911 INFO:tasks.workunit.client.1.smithi121.stdout:4/440: dread d5/d9/f2e [0,4194304] 0 2024-02-15T18:35:37.914 INFO:tasks.workunit.client.1.smithi121.stdout:5/709: dwrite d0/d2/d1e/d4e/d7d/d9d/d4c/f8d [0,4194304] 0 2024-02-15T18:35:37.919 INFO:tasks.workunit.client.1.smithi121.stdout:5/710: rmdir d0/d2/d1e/d4e 39 2024-02-15T18:35:37.927 INFO:tasks.workunit.client.1.smithi121.stdout:5/711: creat d0/d2/d1e/d4e/d7d/d47/fe9 x:0 0 0 2024-02-15T18:35:37.928 INFO:tasks.workunit.client.1.smithi121.stdout:5/712: creat d0/d16/d34/da4/fea x:0 0 0 2024-02-15T18:35:37.931 INFO:tasks.workunit.client.1.smithi121.stdout:4/441: dwrite d5/d31/d25/f37 [0,4194304] 0 2024-02-15T18:35:37.931 INFO:tasks.workunit.client.1.smithi121.stdout:5/713: rename d0/d2/d70/fc9 to d0/d2/d1e/d4e/d7d/d40/d62/feb 0 2024-02-15T18:35:37.935 INFO:tasks.workunit.client.1.smithi121.stdout:5/714: truncate d0/d16/d34/d8f/f97 259786 0 2024-02-15T18:35:37.936 INFO:tasks.workunit.client.1.smithi121.stdout:4/442: truncate f1 764112 0 2024-02-15T18:35:37.936 INFO:tasks.workunit.client.1.smithi121.stdout:4/443: chown d5/d9/d1d 421843 1 2024-02-15T18:35:37.938 INFO:tasks.workunit.client.1.smithi121.stdout:5/715: creat d0/d2/d1e/d4e/d7d/d47/d5a/da7/fec x:0 0 0 2024-02-15T18:35:37.939 INFO:tasks.workunit.client.1.smithi121.stdout:5/716: mknod d0/d2/d1e/d4e/d7d/d47/d5a/ced 0 2024-02-15T18:35:37.947 INFO:tasks.workunit.client.1.smithi121.stdout:5/717: dread d0/d2/d1e/d4e/d7d/d9d/d4c/f95 [0,4194304] 0 2024-02-15T18:35:37.954 INFO:tasks.workunit.client.1.smithi121.stdout:4/444: dwrite d5/d9/f20 [0,4194304] 0 2024-02-15T18:35:38.002 INFO:tasks.workunit.client.1.smithi121.stdout:4/445: dwrite d5/d7/f75 [0,4194304] 0 2024-02-15T18:35:38.004 INFO:tasks.workunit.client.1.smithi121.stdout:4/446: creat d5/d9/d19/d4c/f8b x:0 0 0 2024-02-15T18:35:38.005 INFO:tasks.workunit.client.1.smithi121.stdout:4/447: symlink d5/d9/d1d/d26/d72/l8c 0 2024-02-15T18:35:38.006 INFO:tasks.workunit.client.1.smithi121.stdout:4/448: write d5/d7/d14/f29 [212518,20204] 0 2024-02-15T18:35:38.008 INFO:tasks.workunit.client.1.smithi121.stdout:4/449: truncate d5/d31/d3d/d4f/d51/d71/f48 121958 0 2024-02-15T18:35:38.009 INFO:tasks.workunit.client.1.smithi121.stdout:4/450: mkdir d5/d9/d19/d4c/d8d 0 2024-02-15T18:35:38.011 INFO:tasks.workunit.client.1.smithi121.stdout:4/451: unlink d5/d7/fb 0 2024-02-15T18:35:38.026 INFO:tasks.workunit.client.1.smithi121.stdout:4/452: dwrite d5/d9/d19/d28/f21 [0,4194304] 0 2024-02-15T18:35:38.027 INFO:tasks.workunit.client.1.smithi121.stdout:4/453: chown d5/d9/d19/d28/l62 2771 1 2024-02-15T18:35:38.027 INFO:tasks.workunit.client.1.smithi121.stdout:4/454: readlink d5/d7/l1c 0 2024-02-15T18:35:38.027 INFO:tasks.workunit.client.1.smithi121.stdout:4/455: link d5/d31/f23 d5/d9/d19/d4c/d82/f8e 0 2024-02-15T18:35:38.031 INFO:tasks.workunit.client.1.smithi121.stdout:4/456: write d5/d7/d14/f55 [2524226,124292] 0 2024-02-15T18:35:38.031 INFO:tasks.workunit.client.1.smithi121.stdout:4/457: write d5/d9/d19/d57/f77 [822082,87287] 0 2024-02-15T18:35:38.032 INFO:tasks.workunit.client.1.smithi121.stdout:4/458: mknod d5/d31/d3d/d4f/d51/c8f 0 2024-02-15T18:35:38.033 INFO:tasks.workunit.client.1.smithi121.stdout:4/459: write d5/d31/f23 [4774208,59685] 0 2024-02-15T18:35:38.033 INFO:tasks.workunit.client.1.smithi121.stdout:4/460: write d5/d7/d14/f17 [5186396,64162] 0 2024-02-15T18:35:38.033 INFO:tasks.workunit.client.1.smithi121.stdout:4/461: write d5/d31/d3d/d4a/f60 [1845152,81104] 0 2024-02-15T18:35:38.040 INFO:tasks.workunit.client.1.smithi121.stdout:0/650: sync 2024-02-15T18:35:38.040 INFO:tasks.workunit.client.1.smithi121.stdout:9/534: sync 2024-02-15T18:35:38.040 INFO:tasks.workunit.client.1.smithi121.stdout:3/433: sync 2024-02-15T18:35:38.040 INFO:tasks.workunit.client.1.smithi121.stdout:2/361: sync 2024-02-15T18:35:38.040 INFO:tasks.workunit.client.1.smithi121.stdout:8/543: sync 2024-02-15T18:35:38.040 INFO:tasks.workunit.client.1.smithi121.stdout:0/651: chown d5/d6b/dab/db4/d7d/dd/c66 942 1 2024-02-15T18:35:38.041 INFO:tasks.workunit.client.1.smithi121.stdout:6/566: sync 2024-02-15T18:35:38.042 INFO:tasks.workunit.client.1.smithi121.stdout:0/652: write d5/d6b/dab/db1/dbf/fd9 [4308095,90923] 0 2024-02-15T18:35:38.042 INFO:tasks.workunit.client.1.smithi121.stdout:9/535: rmdir d7/d3a 39 2024-02-15T18:35:38.044 INFO:tasks.workunit.client.1.smithi121.stdout:0/653: write d5/d6b/dab/db4/d7d/f11 [95370,23673] 0 2024-02-15T18:35:38.045 INFO:tasks.workunit.client.1.smithi121.stdout:3/434: unlink d2/d1a/d27/d7c/d25/d88/c8a 0 2024-02-15T18:35:38.046 INFO:tasks.workunit.client.1.smithi121.stdout:2/362: mkdir d5/df/d27/d3e/d6a 0 2024-02-15T18:35:38.053 INFO:tasks.workunit.client.1.smithi121.stdout:8/544: link d0/d2a/f7d d0/d7/d36/fc0 0 2024-02-15T18:35:38.053 INFO:tasks.workunit.client.1.smithi121.stdout:6/567: link d7/dc/d34/d42/lb5 d7/dc/d76/lc2 0 2024-02-15T18:35:38.054 INFO:tasks.workunit.client.1.smithi121.stdout:9/536: unlink d7/d3a/d8e/d9e/da9/d8b/l2c 0 2024-02-15T18:35:38.054 INFO:tasks.workunit.client.1.smithi121.stdout:6/568: dread d7/dc/d34/d7c/faa [0,4194304] 0 2024-02-15T18:35:38.054 INFO:tasks.workunit.client.1.smithi121.stdout:0/654: mkdir d5/d6b/dab/db4/d7d/d2a/d73/d8d/dc8/de0 0 2024-02-15T18:35:38.054 INFO:tasks.workunit.client.1.smithi121.stdout:0/655: chown d5/d6b/dab/db4/d7d/dd/l28 43508 1 2024-02-15T18:35:38.058 INFO:tasks.workunit.client.1.smithi121.stdout:9/537: rename d7/d3a/d8e/c95 to d7/cc3 0 2024-02-15T18:35:38.059 INFO:tasks.workunit.client.1.smithi121.stdout:6/569: getdents d7/dc/d2c/d36 0 2024-02-15T18:35:38.059 INFO:tasks.workunit.client.1.smithi121.stdout:0/656: creat d5/d30/d5d/fe1 x:0 0 0 2024-02-15T18:35:38.062 INFO:tasks.workunit.client.1.smithi121.stdout:0/657: creat d5/d6b/dab/db4/d7d/d74/d96/fe2 x:0 0 0 2024-02-15T18:35:38.062 INFO:tasks.workunit.client.1.smithi121.stdout:9/538: write d7/d8/d25/d30/d33/d3c/f84 [112542,59237] 0 2024-02-15T18:35:38.063 INFO:tasks.workunit.client.1.smithi121.stdout:9/539: fdatasync d7/d26/d4f/f7f 0 2024-02-15T18:35:38.063 INFO:tasks.workunit.client.1.smithi121.stdout:4/462: dwrite d5/f30 [0,4194304] 0 2024-02-15T18:35:38.063 INFO:tasks.workunit.client.1.smithi121.stdout:0/658: write d5/d6b/dab/db4/d7d/dd/f17 [1502600,113855] 0 2024-02-15T18:35:38.064 INFO:tasks.workunit.client.1.smithi121.stdout:9/540: chown d7/d8/d25/c71 10 1 2024-02-15T18:35:38.069 INFO:tasks.workunit.client.1.smithi121.stdout:9/541: rename d7/d8/d25/d30/d33/d3c/d3f to d7/d8/d40/dc4 0 2024-02-15T18:35:38.069 INFO:tasks.workunit.client.1.smithi121.stdout:9/542: chown d7/d26/c69 740357 1 2024-02-15T18:35:38.072 INFO:tasks.workunit.client.1.smithi121.stdout:9/543: rename d7/d3a/d8e/d9e/da9/d8b/f45 to d7/d8/d36/fc5 0 2024-02-15T18:35:38.074 INFO:tasks.workunit.client.1.smithi121.stdout:9/544: link l2 d7/d8/d36/da3/lc6 0 2024-02-15T18:35:38.074 INFO:tasks.workunit.client.1.smithi121.stdout:3/435: dwrite d2/f7 [0,4194304] 0 2024-02-15T18:35:38.075 INFO:tasks.workunit.client.1.smithi121.stdout:9/545: dread d7/d3a/d8e/f98 [0,4194304] 0 2024-02-15T18:35:38.077 INFO:tasks.workunit.client.1.smithi121.stdout:3/436: write d2/d1a/d22/f6a [244329,46824] 0 2024-02-15T18:35:38.077 INFO:tasks.workunit.client.1.smithi121.stdout:3/437: fsync d2/d1a/d27/d7c/d25/d31/f43 0 2024-02-15T18:35:38.077 INFO:tasks.workunit.client.1.smithi121.stdout:3/438: write d2/d1a/d6b/f75 [732337,76555] 0 2024-02-15T18:35:38.086 INFO:tasks.workunit.client.1.smithi121.stdout:4/463: dwrite d5/d31/d3d/d4f/d51/d71/f48 [0,4194304] 0 2024-02-15T18:35:38.091 INFO:tasks.workunit.client.1.smithi121.stdout:0/659: dwrite d5/d6b/dab/db4/d7d/d2a/d73/d97/f98 [4194304,4194304] 0 2024-02-15T18:35:38.091 INFO:tasks.workunit.client.1.smithi121.stdout:3/439: fsync d2/d1a/d6b/f75 0 2024-02-15T18:35:38.093 INFO:tasks.workunit.client.1.smithi121.stdout:0/660: mkdir d5/d6b/dab/db4/d7d/d74/d96/de3 0 2024-02-15T18:35:38.095 INFO:tasks.workunit.client.1.smithi121.stdout:0/661: mkdir d5/d6b/dab/db4/d7d/d74/d96/de4 0 2024-02-15T18:35:38.096 INFO:tasks.workunit.client.1.smithi121.stdout:3/440: write d2/f40 [689057,79606] 0 2024-02-15T18:35:38.098 INFO:tasks.workunit.client.1.smithi121.stdout:9/546: dwrite d7/d26/f81 [0,4194304] 0 2024-02-15T18:35:38.100 INFO:tasks.workunit.client.1.smithi121.stdout:9/547: write d7/d26/d4f/d5b/d77/f91 [1205099,48438] 0 2024-02-15T18:35:38.104 INFO:tasks.workunit.client.1.smithi121.stdout:9/548: mkdir d7/d3a/d8e/d9e/dc7 0 2024-02-15T18:35:38.104 INFO:tasks.workunit.client.1.smithi121.stdout:9/549: chown d7/d26/d4f/l94 7057051 1 2024-02-15T18:35:38.104 INFO:tasks.workunit.client.1.smithi121.stdout:9/550: write d7/d8/fa8 [954343,72006] 0 2024-02-15T18:35:38.105 INFO:tasks.workunit.client.1.smithi121.stdout:9/551: symlink d7/lc8 0 2024-02-15T18:35:38.107 INFO:tasks.workunit.client.1.smithi121.stdout:9/552: link d7/d3a/d8e/d9e/da9/d8b/la7 d7/d26/dba/lc9 0 2024-02-15T18:35:38.108 INFO:tasks.workunit.client.1.smithi121.stdout:9/553: dread d7/d8/f87 [0,4194304] 0 2024-02-15T18:35:38.109 INFO:tasks.workunit.client.1.smithi121.stdout:9/554: mknod d7/d3a/d8e/cca 0 2024-02-15T18:35:38.118 INFO:tasks.workunit.client.1.smithi121.stdout:3/441: dwrite d2/d1a/d27/d63/f80 [0,4194304] 0 2024-02-15T18:35:38.119 INFO:tasks.workunit.client.1.smithi121.stdout:0/662: dwrite d5/d6b/dab/db4/d7d/d2a/d4e/f5a [4194304,4194304] 0 2024-02-15T18:35:38.119 INFO:tasks.workunit.client.1.smithi121.stdout:4/464: dwrite d5/d31/d25/f37 [0,4194304] 0 2024-02-15T18:35:38.119 INFO:tasks.workunit.client.1.smithi121.stdout:4/465: fdatasync d5/d9/d19/d5c/f69 0 2024-02-15T18:35:38.119 INFO:tasks.workunit.client.1.smithi121.stdout:4/466: fsync d5/d9/d1d/f41 0 2024-02-15T18:35:38.119 INFO:tasks.workunit.client.1.smithi121.stdout:4/467: chown d5/d31/d3d/d4f/d51 2608 1 2024-02-15T18:35:38.123 INFO:tasks.workunit.client.1.smithi121.stdout:3/442: rename d2/d1a/d27/d7c/f3a to d2/d1a/f92 0 2024-02-15T18:35:38.123 INFO:tasks.workunit.client.1.smithi121.stdout:3/443: chown d2/d1a/d27/d7c/c45 45 1 2024-02-15T18:35:38.123 INFO:tasks.workunit.client.1.smithi121.stdout:3/444: chown d2/cf 1 1 2024-02-15T18:35:38.124 INFO:tasks.workunit.client.1.smithi121.stdout:3/445: dread - d2/d1a/d6b/f82 zero size 2024-02-15T18:35:38.125 INFO:tasks.workunit.client.1.smithi121.stdout:3/446: mknod d2/d1a/d6b/d72/c93 0 2024-02-15T18:35:38.126 INFO:tasks.workunit.client.1.smithi121.stdout:3/447: symlink d2/d1a/d54/l94 0 2024-02-15T18:35:38.126 INFO:tasks.workunit.client.1.smithi121.stdout:3/448: truncate d2/d1a/f89 471464 0 2024-02-15T18:35:38.130 INFO:tasks.workunit.client.1.smithi121.stdout:9/555: dwrite d7/d8/d36/f2f [0,4194304] 0 2024-02-15T18:35:38.131 INFO:tasks.workunit.client.1.smithi121.stdout:9/556: write d7/d8/f10 [3325204,61510] 0 2024-02-15T18:35:38.132 INFO:tasks.workunit.client.1.smithi121.stdout:9/557: chown d7/d8/d25/fc2 923709 1 2024-02-15T18:35:38.144 INFO:tasks.workunit.client.1.smithi121.stdout:4/468: dwrite d5/d9/d19/d4c/f8b [0,4194304] 0 2024-02-15T18:35:38.153 INFO:tasks.workunit.client.1.smithi121.stdout:3/449: dwrite d2/d1a/d22/f35 [0,4194304] 0 2024-02-15T18:35:38.153 INFO:tasks.workunit.client.1.smithi121.stdout:3/450: read - d2/d1a/d27/d7c/d25/d31/d34/d4a/f6d zero size 2024-02-15T18:35:38.156 INFO:tasks.workunit.client.1.smithi121.stdout:3/451: truncate d2/d1a/d27/f7f 421394 0 2024-02-15T18:35:38.157 INFO:tasks.workunit.client.1.smithi121.stdout:3/452: rmdir d2/d1a/d27/d7c/d25/d31/d34 39 2024-02-15T18:35:38.157 INFO:tasks.workunit.client.1.smithi121.stdout:3/453: write d2/d1a/d27/d7c/d25/d30/f5b [1043473,4158] 0 2024-02-15T18:35:38.157 INFO:tasks.workunit.client.1.smithi121.stdout:3/454: creat d2/d5a/f95 x:0 0 0 2024-02-15T18:35:38.166 INFO:tasks.workunit.client.1.smithi121.stdout:9/558: dwrite d7/f13 [0,4194304] 0 2024-02-15T18:35:38.166 INFO:tasks.workunit.client.1.smithi121.stdout:9/559: dread - d7/d8/d40/f6e zero size 2024-02-15T18:35:38.169 INFO:tasks.workunit.client.1.smithi121.stdout:9/560: creat d7/d3a/d8e/d9e/fcb x:0 0 0 2024-02-15T18:35:38.169 INFO:tasks.workunit.client.1.smithi121.stdout:9/561: fdatasync d7/d3a/d8e/d9e/da9/d8b/fbb 0 2024-02-15T18:35:38.169 INFO:tasks.workunit.client.1.smithi121.stdout:9/562: fdatasync d7/f1c 0 2024-02-15T18:35:38.169 INFO:tasks.workunit.client.1.smithi121.stdout:4/469: dwrite d5/d7/d14/f55 [0,4194304] 0 2024-02-15T18:35:38.171 INFO:tasks.workunit.client.1.smithi121.stdout:9/563: dread d7/d8/d25/d30/d33/f88 [0,4194304] 0 2024-02-15T18:35:38.171 INFO:tasks.workunit.client.1.smithi121.stdout:9/564: truncate d7/d26/dba/f6c 708911 0 2024-02-15T18:35:38.172 INFO:tasks.workunit.client.1.smithi121.stdout:4/470: mknod d5/d31/d25/c90 0 2024-02-15T18:35:38.172 INFO:tasks.workunit.client.1.smithi121.stdout:9/565: dread d7/d8/d36/fc5 [0,4194304] 0 2024-02-15T18:35:38.175 INFO:tasks.workunit.client.1.smithi121.stdout:4/471: symlink d5/d9/d19/d2a/d4d/l91 0 2024-02-15T18:35:38.175 INFO:tasks.workunit.client.1.smithi121.stdout:4/472: fsync d5/d9/d1d/f2f 0 2024-02-15T18:35:38.175 INFO:tasks.workunit.client.1.smithi121.stdout:4/473: readlink d5/d7/l1c 0 2024-02-15T18:35:38.175 INFO:tasks.workunit.client.1.smithi121.stdout:4/474: chown d5/d7/d14/f17 220 1 2024-02-15T18:35:38.175 INFO:tasks.workunit.client.1.smithi121.stdout:4/475: fdatasync d5/d31/d3d/d4f/d51/f5d 0 2024-02-15T18:35:38.176 INFO:tasks.workunit.client.1.smithi121.stdout:9/566: mkdir d7/dcc 0 2024-02-15T18:35:38.177 INFO:tasks.workunit.client.1.smithi121.stdout:4/476: symlink d5/d9/d76/l92 0 2024-02-15T18:35:38.180 INFO:tasks.workunit.client.1.smithi121.stdout:4/477: dread f1 [0,4194304] 0 2024-02-15T18:35:38.184 INFO:tasks.workunit.client.1.smithi121.stdout:9/567: rename d7/d26/d4f/l94 to d7/d26/lcd 0 2024-02-15T18:35:38.184 INFO:tasks.workunit.client.1.smithi121.stdout:9/568: read d7/d3a/d8e/f98 [134874,73788] 0 2024-02-15T18:35:38.184 INFO:tasks.workunit.client.1.smithi121.stdout:9/569: dread - d7/d8/d40/f70 zero size 2024-02-15T18:35:38.185 INFO:tasks.workunit.client.1.smithi121.stdout:4/478: symlink d5/d31/d3d/d4f/d51/d71/l93 0 2024-02-15T18:35:38.185 INFO:tasks.workunit.client.1.smithi121.stdout:4/479: chown d5/d9/d1d/f41 24 1 2024-02-15T18:35:38.189 INFO:tasks.workunit.client.1.smithi121.stdout:9/570: mkdir d7/d8/d36/dce 0 2024-02-15T18:35:38.189 INFO:tasks.workunit.client.1.smithi121.stdout:4/480: symlink d5/d31/d3d/l94 0 2024-02-15T18:35:38.190 INFO:tasks.workunit.client.1.smithi121.stdout:9/571: mknod d7/d3a/d8e/d9e/da9/d8b/d73/ccf 0 2024-02-15T18:35:38.191 INFO:tasks.workunit.client.1.smithi121.stdout:4/481: mkdir d5/d9/d76/d87/d95 0 2024-02-15T18:35:38.191 INFO:tasks.workunit.client.1.smithi121.stdout:4/482: dread - d5/d31/d3d/d4a/d7e/f7f zero size 2024-02-15T18:35:38.192 INFO:tasks.workunit.client.1.smithi121.stdout:9/572: unlink d7/d8/f38 0 2024-02-15T18:35:38.192 INFO:tasks.workunit.client.1.smithi121.stdout:9/573: chown d7/d3a/d8e/d9e/da9/d8b/d73/c82 3011089 1 2024-02-15T18:35:38.192 INFO:tasks.workunit.client.1.smithi121.stdout:4/483: creat d5/d9/d19/d2a/f96 x:0 0 0 2024-02-15T18:35:38.193 INFO:tasks.workunit.client.1.smithi121.stdout:9/574: mkdir d7/d8/d25/d30/d33/d3c/dd0 0 2024-02-15T18:35:38.194 INFO:tasks.workunit.client.1.smithi121.stdout:4/484: creat d5/d31/d3d/d4f/d8a/f97 x:0 0 0 2024-02-15T18:35:38.194 INFO:tasks.workunit.client.1.smithi121.stdout:4/485: readlink d5/d9/d1d/d26/l86 0 2024-02-15T18:35:38.195 INFO:tasks.workunit.client.1.smithi121.stdout:4/486: mkdir d5/d9/d19/d5c/d98 0 2024-02-15T18:35:38.195 INFO:tasks.workunit.client.1.smithi121.stdout:4/487: fdatasync d5/d7/d14/f17 0 2024-02-15T18:35:38.197 INFO:tasks.workunit.client.1.smithi121.stdout:4/488: creat d5/d31/d3d/d4f/d51/d63/f99 x:0 0 0 2024-02-15T18:35:38.197 INFO:tasks.workunit.client.1.smithi121.stdout:9/575: dread d7/f1f [0,4194304] 0 2024-02-15T18:35:38.198 INFO:tasks.workunit.client.1.smithi121.stdout:9/576: creat d7/d8/d25/d30/fd1 x:0 0 0 2024-02-15T18:35:38.199 INFO:tasks.workunit.client.1.smithi121.stdout:9/577: write d7/d3a/d8e/d9e/fcb [246605,36929] 0 2024-02-15T18:35:38.199 INFO:tasks.workunit.client.1.smithi121.stdout:7/379: sync 2024-02-15T18:35:38.199 INFO:tasks.workunit.client.1.smithi121.stdout:9/578: creat d7/d26/fd2 x:0 0 0 2024-02-15T18:35:38.200 INFO:tasks.workunit.client.1.smithi121.stdout:9/579: chown d7/f1f 52 1 2024-02-15T18:35:38.200 INFO:tasks.workunit.client.1.smithi121.stdout:9/580: truncate d7/d8/fa8 1342440 0 2024-02-15T18:35:38.200 INFO:tasks.workunit.client.1.smithi121.stdout:9/581: creat d7/d8/d25/d30/d33/d79/fd3 x:0 0 0 2024-02-15T18:35:38.204 INFO:tasks.workunit.client.1.smithi121.stdout:9/582: dread d7/d8/d25/fc2 [0,4194304] 0 2024-02-15T18:35:38.204 INFO:tasks.workunit.client.1.smithi121.stdout:9/583: readlink d7/d3a/d8e/d9e/da9/d8b/l21 0 2024-02-15T18:35:38.205 INFO:tasks.workunit.client.1.smithi121.stdout:7/380: dread d0/d38/d61/f78 [0,4194304] 0 2024-02-15T18:35:38.207 INFO:tasks.workunit.client.1.smithi121.stdout:4/489: dread d5/d9/d19/d4c/d82/f8e [0,4194304] 0 2024-02-15T18:35:38.207 INFO:tasks.workunit.client.1.smithi121.stdout:7/381: creat d0/d4a/f7e x:0 0 0 2024-02-15T18:35:38.208 INFO:tasks.workunit.client.1.smithi121.stdout:4/490: stat d5/d7/l1c 0 2024-02-15T18:35:38.210 INFO:tasks.workunit.client.1.smithi121.stdout:7/382: creat d0/d5/d11/d67/f7f x:0 0 0 2024-02-15T18:35:38.216 INFO:tasks.workunit.client.1.smithi121.stdout:4/491: mkdir d5/d9/d19/d4c/d9a 0 2024-02-15T18:35:38.218 INFO:tasks.workunit.client.1.smithi121.stdout:7/383: creat d0/d38/d3c/f80 x:0 0 0 2024-02-15T18:35:38.218 INFO:tasks.workunit.client.1.smithi121.stdout:7/384: dread - d0/d5/d4c/f59 zero size 2024-02-15T18:35:38.219 INFO:tasks.workunit.client.1.smithi121.stdout:7/385: creat d0/d5/f81 x:0 0 0 2024-02-15T18:35:38.222 INFO:tasks.workunit.client.1.smithi121.stdout:7/386: rename d0/d38/d3f/d43/f4e to d0/d5/d11/d1c/d5f/f82 0 2024-02-15T18:35:38.222 INFO:tasks.workunit.client.1.smithi121.stdout:7/387: dread - d0/d5/d4c/f59 zero size 2024-02-15T18:35:38.222 INFO:tasks.workunit.client.1.smithi121.stdout:7/388: readlink d0/d5/d11/d67/l6c 0 2024-02-15T18:35:38.223 INFO:tasks.workunit.client.1.smithi121.stdout:7/389: mknod d0/d5/d11/d1c/c83 0 2024-02-15T18:35:38.224 INFO:tasks.workunit.client.1.smithi121.stdout:7/390: symlink d0/d5/d11/d1c/d5f/l84 0 2024-02-15T18:35:38.231 INFO:tasks.workunit.client.1.smithi121.stdout:7/391: dread d0/d4a/d5b/f65 [0,4194304] 0 2024-02-15T18:35:38.238 INFO:tasks.workunit.client.1.smithi121.stdout:7/392: dread d0/d33/f34 [0,4194304] 0 2024-02-15T18:35:38.238 INFO:tasks.workunit.client.1.smithi121.stdout:4/492: dwrite d5/d31/d3d/d4f/d51/f47 [0,4194304] 0 2024-02-15T18:35:38.238 INFO:tasks.workunit.client.1.smithi121.stdout:4/493: chown d5/d31/d3d/d4f/d51/d80/l85 7 1 2024-02-15T18:35:38.238 INFO:tasks.workunit.client.1.smithi121.stdout:4/494: chown d5/d9/d19/d2a/d4d/l6d 132587433 1 2024-02-15T18:35:38.239 INFO:tasks.workunit.client.1.smithi121.stdout:7/393: mknod d0/d5/d11/c85 0 2024-02-15T18:35:38.241 INFO:tasks.workunit.client.1.smithi121.stdout:9/584: dwrite d7/d8/d40/f63 [0,4194304] 0 2024-02-15T18:35:38.241 INFO:tasks.workunit.client.1.smithi121.stdout:9/585: write d7/d26/fd2 [952193,41589] 0 2024-02-15T18:35:38.242 INFO:tasks.workunit.client.1.smithi121.stdout:9/586: mkdir d7/d8/d25/d30/d33/d8f/dd4 0 2024-02-15T18:35:38.249 INFO:tasks.workunit.client.1.smithi121.stdout:7/394: dread d0/d38/d3c/f4b [0,4194304] 0 2024-02-15T18:35:38.251 INFO:tasks.workunit.client.1.smithi121.stdout:7/395: stat d0/d5/l54 0 2024-02-15T18:35:38.252 INFO:tasks.workunit.client.1.smithi121.stdout:7/396: link d0/d5/d11/d1c/f21 d0/d5/d11/d79/f86 0 2024-02-15T18:35:38.252 INFO:tasks.workunit.client.1.smithi121.stdout:7/397: stat d0/d38/d3c 0 2024-02-15T18:35:38.258 INFO:tasks.workunit.client.1.smithi121.stdout:9/587: dwrite d7/d8/f7c [0,4194304] 0 2024-02-15T18:35:38.263 INFO:tasks.workunit.client.1.smithi121.stdout:9/588: creat d7/d8/d25/d30/d33/d8f/fd5 x:0 0 0 2024-02-15T18:35:38.263 INFO:tasks.workunit.client.1.smithi121.stdout:9/589: write d7/d8/d25/d30/d33/d79/fa0 [1128735,85019] 0 2024-02-15T18:35:38.268 INFO:tasks.workunit.client.1.smithi121.stdout:9/590: write d7/f23 [2290121,108250] 0 2024-02-15T18:35:38.269 INFO:tasks.workunit.client.1.smithi121.stdout:9/591: write d7/d8/d25/d30/d33/f88 [681798,60898] 0 2024-02-15T18:35:38.269 INFO:tasks.workunit.client.1.smithi121.stdout:9/592: write d7/d8/d25/d30/da5/f41 [4610360,62728] 0 2024-02-15T18:35:38.269 INFO:tasks.workunit.client.1.smithi121.stdout:9/593: creat d7/d26/fd6 x:0 0 0 2024-02-15T18:35:38.270 INFO:tasks.workunit.client.1.smithi121.stdout:9/594: mknod d7/d26/d4f/d5b/cd7 0 2024-02-15T18:35:38.270 INFO:tasks.workunit.client.1.smithi121.stdout:9/595: truncate d7/d26/d4f/d5b/fb1 1045575 0 2024-02-15T18:35:38.270 INFO:tasks.workunit.client.1.smithi121.stdout:9/596: readlink d7/d3a/l8c 0 2024-02-15T18:35:38.271 INFO:tasks.workunit.client.1.smithi121.stdout:9/597: unlink d7/d3a/d8e/ca2 0 2024-02-15T18:35:38.271 INFO:tasks.workunit.client.1.smithi121.stdout:9/598: write d7/d26/dba/f6c [127763,29407] 0 2024-02-15T18:35:38.275 INFO:tasks.workunit.client.1.smithi121.stdout:7/398: dwrite d0/d5/d11/d1c/f21 [0,4194304] 0 2024-02-15T18:35:38.275 INFO:tasks.workunit.client.1.smithi121.stdout:7/399: chown d0/d5/d11 258328069 1 2024-02-15T18:35:38.275 INFO:tasks.workunit.client.1.smithi121.stdout:7/400: dread - d0/d5/d11/d67/f7f zero size 2024-02-15T18:35:38.279 INFO:tasks.workunit.client.1.smithi121.stdout:9/599: truncate d7/d26/fd2 680359 0 2024-02-15T18:35:38.283 INFO:tasks.workunit.client.1.smithi121.stdout:7/401: write d0/d5/d11/d1c/f3e [1009274,45436] 0 2024-02-15T18:35:38.283 INFO:tasks.workunit.client.1.smithi121.stdout:9/600: dread d7/d8/d25/fc2 [0,4194304] 0 2024-02-15T18:35:38.283 INFO:tasks.workunit.client.1.smithi121.stdout:9/601: stat d7/d8/d25/d30/d33/c5a 0 2024-02-15T18:35:38.283 INFO:tasks.workunit.client.1.smithi121.stdout:7/402: read d0/d5/d11/d1c/f35 [993761,18090] 0 2024-02-15T18:35:38.283 INFO:tasks.workunit.client.1.smithi121.stdout:7/403: read - d0/d5/d11/d1c/d5f/f7c zero size 2024-02-15T18:35:38.283 INFO:tasks.workunit.client.1.smithi121.stdout:7/404: write d0/d38/d3c/f80 [336525,97440] 0 2024-02-15T18:35:38.284 INFO:tasks.workunit.client.1.smithi121.stdout:9/602: rename d7/d26/d4f/d5b/fb1 to d7/d8/d25/d30/da5/fd8 0 2024-02-15T18:35:38.286 INFO:tasks.workunit.client.1.smithi121.stdout:7/405: link d0/d5/d11/d1c/f1f d0/d38/d46/f87 0 2024-02-15T18:35:38.287 INFO:tasks.workunit.client.1.smithi121.stdout:9/603: mkdir d7/d8/d25/d30/d33/d3c/dd0/dd9 0 2024-02-15T18:35:38.289 INFO:tasks.workunit.client.1.smithi121.stdout:7/406: symlink d0/d38/l88 0 2024-02-15T18:35:38.289 INFO:tasks.workunit.client.1.smithi121.stdout:7/407: readlink d0/l6a 0 2024-02-15T18:35:38.289 INFO:tasks.workunit.client.1.smithi121.stdout:9/604: creat d7/d8/d25/d30/d33/d8f/fda x:0 0 0 2024-02-15T18:35:38.293 INFO:tasks.workunit.client.1.smithi121.stdout:7/408: link d0/d5/d11/d1c/f23 d0/d4a/d5b/f89 0 2024-02-15T18:35:38.293 INFO:tasks.workunit.client.1.smithi121.stdout:7/409: dread - d0/f52 zero size 2024-02-15T18:35:38.293 INFO:tasks.workunit.client.1.smithi121.stdout:9/605: symlink d7/d8/d25/d30/d33/d8f/ldb 0 2024-02-15T18:35:38.294 INFO:tasks.workunit.client.1.smithi121.stdout:7/410: creat d0/d38/d61/f8a x:0 0 0 2024-02-15T18:35:38.294 INFO:tasks.workunit.client.1.smithi121.stdout:7/411: truncate d0/d5/d11/d1c/f21 5182460 0 2024-02-15T18:35:38.295 INFO:tasks.workunit.client.1.smithi121.stdout:9/606: creat d7/d3a/fdc x:0 0 0 2024-02-15T18:35:38.296 INFO:tasks.workunit.client.1.smithi121.stdout:7/412: unlink d0/d5/d11/d1c/d5f/f7c 0 2024-02-15T18:35:38.297 INFO:tasks.workunit.client.1.smithi121.stdout:7/413: mknod d0/d5/d76/c8b 0 2024-02-15T18:35:38.297 INFO:tasks.workunit.client.1.smithi121.stdout:7/414: stat d0/d5/d11/d1c/f23 0 2024-02-15T18:35:38.297 INFO:tasks.workunit.client.1.smithi121.stdout:7/415: truncate d0/d5/d11/f48 1339078 0 2024-02-15T18:35:38.297 INFO:tasks.workunit.client.1.smithi121.stdout:7/416: truncate d0/d5/d11/d1c/d5f/f82 501039 0 2024-02-15T18:35:38.298 INFO:tasks.workunit.client.1.smithi121.stdout:7/417: mknod d0/d5/d11/d67/c8c 0 2024-02-15T18:35:38.299 INFO:tasks.workunit.client.1.smithi121.stdout:7/418: fdatasync d0/d38/d61/f8a 0 2024-02-15T18:35:38.302 INFO:tasks.workunit.client.1.smithi121.stdout:9/607: dread d7/d8/f16 [0,4194304] 0 2024-02-15T18:35:38.302 INFO:tasks.workunit.client.1.smithi121.stdout:9/608: readlink d7/d8/d40/d4b/lb7 0 2024-02-15T18:35:38.303 INFO:tasks.workunit.client.1.smithi121.stdout:9/609: write d7/d8/d25/d30/d33/d79/fa0 [908,39067] 0 2024-02-15T18:35:38.305 INFO:tasks.workunit.client.1.smithi121.stdout:9/610: truncate d7/d26/f81 3713263 0 2024-02-15T18:35:38.305 INFO:tasks.workunit.client.1.smithi121.stdout:9/611: chown d7/d8/d36/f7d 167830 1 2024-02-15T18:35:38.305 INFO:tasks.workunit.client.1.smithi121.stdout:9/612: truncate d7/d26/f6b 96687 0 2024-02-15T18:35:38.305 INFO:tasks.workunit.client.1.smithi121.stdout:9/613: truncate d7/d8/d25/d30/d33/d8f/fda 536355 0 2024-02-15T18:35:38.306 INFO:tasks.workunit.client.1.smithi121.stdout:7/419: dread d0/d5/d11/d79/f86 [0,4194304] 0 2024-02-15T18:35:38.321 INFO:tasks.workunit.client.1.smithi121.stdout:9/614: dwrite d7/d26/fd6 [0,4194304] 0 2024-02-15T18:35:38.321 INFO:tasks.workunit.client.1.smithi121.stdout:7/420: rename d0/d38/d46/c60 to d0/c8d 0 2024-02-15T18:35:38.322 INFO:tasks.workunit.client.1.smithi121.stdout:7/421: read - d0/d5/d11/d1c/d5f/f7d zero size 2024-02-15T18:35:38.325 INFO:tasks.workunit.client.1.smithi121.stdout:9/615: unlink d7/d8/d25/d30/d33/d8f/fda 0 2024-02-15T18:35:38.326 INFO:tasks.workunit.client.1.smithi121.stdout:7/422: creat d0/d38/d3f/d43/f8e x:0 0 0 2024-02-15T18:35:38.328 INFO:tasks.workunit.client.1.smithi121.stdout:9/616: creat d7/d3a/d8e/d9e/da9/d8b/d73/fdd x:0 0 0 2024-02-15T18:35:38.332 INFO:tasks.workunit.client.1.smithi121.stdout:7/423: rename d0/d5/d11/d1c/f7a to d0/d38/d3f/d43/f8f 0 2024-02-15T18:35:38.332 INFO:tasks.workunit.client.1.smithi121.stdout:7/424: write d0/f42 [880695,29408] 0 2024-02-15T18:35:38.333 INFO:tasks.workunit.client.1.smithi121.stdout:7/425: write d0/f27 [6077172,52606] 0 2024-02-15T18:35:38.333 INFO:tasks.workunit.client.1.smithi121.stdout:9/617: creat d7/d8/d25/d30/d33/d79/fde x:0 0 0 2024-02-15T18:35:38.333 INFO:tasks.workunit.client.1.smithi121.stdout:9/618: fsync d7/fc0 0 2024-02-15T18:35:38.334 INFO:tasks.workunit.client.1.smithi121.stdout:7/426: truncate d0/d5/f5e 898838 0 2024-02-15T18:35:38.342 INFO:tasks.workunit.client.1.smithi121.stdout:9/619: dread d7/d8/d25/d30/d33/f58 [0,4194304] 0 2024-02-15T18:35:38.342 INFO:tasks.workunit.client.1.smithi121.stdout:1/598: sync 2024-02-15T18:35:38.343 INFO:tasks.workunit.client.1.smithi121.stdout:9/620: symlink d7/dcc/ldf 0 2024-02-15T18:35:38.344 INFO:tasks.workunit.client.1.smithi121.stdout:9/621: mknod d7/d3a/d8e/d9e/ce0 0 2024-02-15T18:35:38.351 INFO:tasks.workunit.client.1.smithi121.stdout:7/427: dwrite d0/d4a/d5b/f89 [0,4194304] 0 2024-02-15T18:35:38.351 INFO:tasks.workunit.client.1.smithi121.stdout:7/428: dread - d0/d4a/f7e zero size 2024-02-15T18:35:38.370 INFO:tasks.workunit.client.1.smithi121.stdout:9/622: dwrite d7/f1f [0,4194304] 0 2024-02-15T18:35:38.370 INFO:tasks.workunit.client.1.smithi121.stdout:9/623: truncate d7/d8/d40/d4b/f7a 310921 0 2024-02-15T18:35:38.372 INFO:tasks.workunit.client.1.smithi121.stdout:1/599: dwrite d0/d6/d7/d9/f35 [4194304,4194304] 0 2024-02-15T18:35:38.376 INFO:tasks.workunit.client.1.smithi121.stdout:1/600: rmdir d0/d6/d7/d9/d14 39 2024-02-15T18:35:38.378 INFO:tasks.workunit.client.1.smithi121.stdout:1/601: rename d0/d6/d7/d9/lcb to d0/d6/d7/d9/d14/d48/lcc 0 2024-02-15T18:35:38.382 INFO:tasks.workunit.client.1.smithi121.stdout:1/602: link d0/d6/d7/d9/lb0 d0/d6/d7/lcd 0 2024-02-15T18:35:38.385 INFO:tasks.workunit.client.1.smithi121.stdout:1/603: dread d0/d6/d7/d9/d14/d48/f78 [0,4194304] 0 2024-02-15T18:35:38.385 INFO:tasks.workunit.client.1.smithi121.stdout:1/604: fsync d0/d6/d7/d9/d14/f58 0 2024-02-15T18:35:38.387 INFO:tasks.workunit.client.1.smithi121.stdout:1/605: creat d0/d39/d7d/dc0/d9d/fce x:0 0 0 2024-02-15T18:35:38.389 INFO:tasks.workunit.client.1.smithi121.stdout:7/429: dwrite d0/f18 [0,4194304] 0 2024-02-15T18:35:38.389 INFO:tasks.workunit.client.1.smithi121.stdout:1/606: read d0/d6/d7/fc4 [2017051,68149] 0 2024-02-15T18:35:38.390 INFO:tasks.workunit.client.1.smithi121.stdout:9/624: dwrite d7/d8/d25/d30/d33/d3c/f84 [0,4194304] 0 2024-02-15T18:35:38.390 INFO:tasks.workunit.client.1.smithi121.stdout:9/625: chown d7/d3a/d8e 3139 1 2024-02-15T18:35:38.392 INFO:tasks.workunit.client.1.smithi121.stdout:9/626: creat d7/d3a/d8e/d9e/dc7/fe1 x:0 0 0 2024-02-15T18:35:38.395 INFO:tasks.workunit.client.1.smithi121.stdout:9/627: truncate d7/f1b 750335 0 2024-02-15T18:35:38.395 INFO:tasks.workunit.client.1.smithi121.stdout:9/628: fsync d7/d8/fa8 0 2024-02-15T18:35:38.396 INFO:tasks.workunit.client.1.smithi121.stdout:9/629: mknod d7/d8/d25/d30/ce2 0 2024-02-15T18:35:38.403 INFO:tasks.workunit.client.1.smithi121.stdout:9/630: dread d7/f1c [0,4194304] 0 2024-02-15T18:35:38.409 INFO:tasks.workunit.client.1.smithi121.stdout:7/430: dwrite d0/d38/d3f/d43/f8e [0,4194304] 0 2024-02-15T18:35:38.414 INFO:tasks.workunit.client.1.smithi121.stdout:7/431: dread d0/d38/d61/f78 [0,4194304] 0 2024-02-15T18:35:38.415 INFO:tasks.workunit.client.1.smithi121.stdout:7/432: creat d0/d5/d11/d1c/d5f/f90 x:0 0 0 2024-02-15T18:35:38.417 INFO:tasks.workunit.client.1.smithi121.stdout:7/433: creat d0/d5/d11/f91 x:0 0 0 2024-02-15T18:35:38.418 INFO:tasks.workunit.client.1.smithi121.stdout:7/434: creat d0/d4a/d5b/f92 x:0 0 0 2024-02-15T18:35:38.419 INFO:tasks.workunit.client.1.smithi121.stdout:7/435: mknod d0/d5/d4c/d6f/c93 0 2024-02-15T18:35:38.424 INFO:tasks.workunit.client.1.smithi121.stdout:1/607: dwrite d0/d6/d7/d9/f2d [0,4194304] 0 2024-02-15T18:35:38.424 INFO:tasks.workunit.client.1.smithi121.stdout:1/608: write d0/d6/d7/f94 [953411,54256] 0 2024-02-15T18:35:38.424 INFO:tasks.workunit.client.1.smithi121.stdout:1/609: chown d0/d39/d4b/cc3 108 1 2024-02-15T18:35:38.426 INFO:tasks.workunit.client.1.smithi121.stdout:1/610: write d0/d39/f68 [571289,101320] 0 2024-02-15T18:35:38.426 INFO:tasks.workunit.client.1.smithi121.stdout:1/611: fdatasync d0/d6/d7/d9/d14/fbb 0 2024-02-15T18:35:38.426 INFO:tasks.workunit.client.1.smithi121.stdout:1/612: chown d0/d6/d7/d9/d14/f83 393 1 2024-02-15T18:35:38.426 INFO:tasks.workunit.client.1.smithi121.stdout:9/631: dwrite d7/d26/f81 [0,4194304] 0 2024-02-15T18:35:38.428 INFO:tasks.workunit.client.1.smithi121.stdout:9/632: mknod d7/d3a/d8e/d9e/dc7/ce3 0 2024-02-15T18:35:38.429 INFO:tasks.workunit.client.1.smithi121.stdout:9/633: write d7/d8/d36/fc5 [194314,110356] 0 2024-02-15T18:35:38.429 INFO:tasks.workunit.client.1.smithi121.stdout:9/634: stat d7/d26/f6b 0 2024-02-15T18:35:38.452 INFO:tasks.workunit.client.1.smithi121.stdout:9/635: dwrite d7/d8/d25/d30/da5/fd8 [0,4194304] 0 2024-02-15T18:35:38.456 INFO:tasks.workunit.client.1.smithi121.stdout:9/636: write d7/d8/f7c [1439071,117891] 0 2024-02-15T18:35:38.456 INFO:tasks.workunit.client.1.smithi121.stdout:9/637: chown d7/d26/dba/lc9 992 1 2024-02-15T18:35:38.458 INFO:tasks.workunit.client.1.smithi121.stdout:9/638: link d7/d8/d25/c71 d7/d3a/d8e/d9e/da9/ce4 0 2024-02-15T18:35:38.460 INFO:tasks.workunit.client.1.smithi121.stdout:9/639: mkdir d7/d3a/d8e/de5 0 2024-02-15T18:35:38.460 INFO:tasks.workunit.client.1.smithi121.stdout:9/640: fdatasync d7/d26/f81 0 2024-02-15T18:35:38.460 INFO:tasks.workunit.client.1.smithi121.stdout:9/641: chown d7/d3a/d8e/d9e/da9/c5c 94529255 1 2024-02-15T18:35:38.461 INFO:tasks.workunit.client.1.smithi121.stdout:9/642: chown d7/d3a 21561530 1 2024-02-15T18:35:38.463 INFO:tasks.workunit.client.1.smithi121.stdout:9/643: write d7/d8/f1d [2600490,7084] 0 2024-02-15T18:35:38.463 INFO:tasks.workunit.client.1.smithi121.stdout:9/644: readlink d7/dcc/ldf 0 2024-02-15T18:35:38.467 INFO:tasks.workunit.client.1.smithi121.stdout:1/613: rmdir d0/d6 39 2024-02-15T18:35:38.469 INFO:tasks.workunit.client.1.smithi121.stdout:1/614: rmdir d0/d6/d7/d9/d14/db6 39 2024-02-15T18:35:38.489 INFO:tasks.workunit.client.1.smithi121.stdout:9/645: dwrite d7/d8/d25/d30/f62 [0,4194304] 0 2024-02-15T18:35:38.499 INFO:tasks.workunit.client.1.smithi121.stdout:9/646: dread d7/f13 [0,4194304] 0 2024-02-15T18:35:38.501 INFO:tasks.workunit.client.1.smithi121.stdout:9/647: creat d7/d26/d4f/d5b/d77/fe6 x:0 0 0 2024-02-15T18:35:38.502 INFO:tasks.workunit.client.1.smithi121.stdout:9/648: write d7/d8/d40/f52 [1550174,12903] 0 2024-02-15T18:35:38.520 INFO:tasks.workunit.client.1.smithi121.stdout:9/649: dwrite d7/d26/dba/f6c [0,4194304] 0 2024-02-15T18:35:38.529 INFO:tasks.workunit.client.1.smithi121.stdout:9/650: dread d7/d26/f81 [0,4194304] 0 2024-02-15T18:35:38.530 INFO:tasks.workunit.client.1.smithi121.stdout:9/651: unlink d7/f1f 0 2024-02-15T18:35:38.530 INFO:tasks.workunit.client.1.smithi121.stdout:9/652: chown d7/d8/d25/d30/d33/d3c/f84 7781528 1 2024-02-15T18:35:38.531 INFO:tasks.workunit.client.1.smithi121.stdout:9/653: chown d7/d8/d25/d30/d33/d8f/d93/fb4 341 1 2024-02-15T18:35:38.532 INFO:tasks.workunit.client.1.smithi121.stdout:9/654: rename d7/d3a/d8e/f98 to d7/d3a/d8e/d9e/fe7 0 2024-02-15T18:35:38.533 INFO:tasks.workunit.client.1.smithi121.stdout:9/655: creat d7/d3a/d8e/d9e/dc7/fe8 x:0 0 0 2024-02-15T18:35:38.553 INFO:tasks.workunit.client.1.smithi121.stdout:9/656: dwrite d7/d8/d36/f7d [0,4194304] 0 2024-02-15T18:35:38.555 INFO:tasks.workunit.client.1.smithi121.stdout:9/657: creat d7/d8/d40/d4b/fe9 x:0 0 0 2024-02-15T18:35:38.558 INFO:tasks.workunit.client.1.smithi121.stdout:9/658: symlink d7/d3a/d8e/d9e/dc7/lea 0 2024-02-15T18:35:38.559 INFO:tasks.workunit.client.1.smithi121.stdout:9/659: creat d7/d8/d25/d30/d33/feb x:0 0 0 2024-02-15T18:35:38.560 INFO:tasks.workunit.client.1.smithi121.stdout:9/660: truncate d7/d8/d25/d30/da5/fbc 940421 0 2024-02-15T18:35:38.571 INFO:tasks.workunit.client.1.smithi121.stdout:5/718: sync 2024-02-15T18:35:38.572 INFO:tasks.workunit.client.1.smithi121.stdout:5/719: truncate d0/d2/d70/fc0 46443 0 2024-02-15T18:35:38.574 INFO:tasks.workunit.client.1.smithi121.stdout:5/720: symlink d0/d2/d1e/d4e/d7d/d47/d5a/db3/lee 0 2024-02-15T18:35:38.588 INFO:tasks.workunit.client.1.smithi121.stdout:5/721: dread d0/d2/d1e/d4e/d7d/d9d/d4c/f95 [0,4194304] 0 2024-02-15T18:35:38.590 INFO:tasks.workunit.client.1.smithi121.stdout:5/722: link d0/d2/d18/c1b d0/d2/d1e/d4e/d7d/d9d/d4c/daa/cef 0 2024-02-15T18:35:38.592 INFO:tasks.workunit.client.1.smithi121.stdout:5/723: dread d0/d2/d1e/d4e/d7d/d39/f53 [0,4194304] 0 2024-02-15T18:35:38.594 INFO:tasks.workunit.client.1.smithi121.stdout:5/724: creat d0/d2/d70/d88/ff0 x:0 0 0 2024-02-15T18:35:38.594 INFO:tasks.workunit.client.1.smithi121.stdout:5/725: chown d0/d2/d1e/d4e/d7d/d9d/d4c/f8b 102122 1 2024-02-15T18:35:38.594 INFO:tasks.workunit.client.1.smithi121.stdout:5/726: readlink d0/d2/d1e/l71 0 2024-02-15T18:35:38.594 INFO:tasks.workunit.client.1.smithi121.stdout:5/727: stat d0/cf 0 2024-02-15T18:35:38.596 INFO:tasks.workunit.client.1.smithi121.stdout:5/728: mknod d0/d2/d1e/d4e/d7d/d40/cf1 0 2024-02-15T18:35:38.596 INFO:tasks.workunit.client.1.smithi121.stdout:5/729: fdatasync d0/d2/d1e/f8c 0 2024-02-15T18:35:38.598 INFO:tasks.workunit.client.1.smithi121.stdout:9/661: dwrite d7/d26/f6b [0,4194304] 0 2024-02-15T18:35:38.607 INFO:tasks.workunit.client.1.smithi121.stdout:5/730: dread d0/d2/d1e/d4e/f55 [0,4194304] 0 2024-02-15T18:35:38.607 INFO:tasks.workunit.client.1.smithi121.stdout:5/731: readlink d0/d2/l3d 0 2024-02-15T18:35:38.611 INFO:tasks.workunit.client.1.smithi121.stdout:5/732: write d0/d2/d18/f29 [506504,36319] 0 2024-02-15T18:35:38.611 INFO:tasks.workunit.client.1.smithi121.stdout:9/662: dread d7/d3a/d8e/d9e/da9/d8b/f83 [4194304,4194304] 0 2024-02-15T18:35:38.614 INFO:tasks.workunit.client.1.smithi121.stdout:9/663: symlink d7/d26/dba/lec 0 2024-02-15T18:35:38.614 INFO:tasks.workunit.client.1.smithi121.stdout:5/733: rmdir d0/d16/d7a/db5 39 2024-02-15T18:35:38.614 INFO:tasks.workunit.client.1.smithi121.stdout:5/734: readlink d0/d2/d1e/d4e/d7d/d9d/d4c/daa/lab 0 2024-02-15T18:35:38.617 INFO:tasks.workunit.client.1.smithi121.stdout:5/735: mkdir d0/d2/d1e/d4e/d7d/d39/df2 0 2024-02-15T18:35:38.617 INFO:tasks.workunit.client.1.smithi121.stdout:5/736: getdents d0/d16/d7a/db5/dca 0 2024-02-15T18:35:38.617 INFO:tasks.workunit.client.1.smithi121.stdout:5/737: dread - d0/d2/d1e/d4e/d7d/d40/f79 zero size 2024-02-15T18:35:38.617 INFO:tasks.workunit.client.1.smithi121.stdout:9/664: truncate d7/d26/d4f/d5b/d77/f91 3741860 0 2024-02-15T18:35:38.620 INFO:tasks.workunit.client.1.smithi121.stdout:5/738: creat d0/d16/d34/ff3 x:0 0 0 2024-02-15T18:35:38.620 INFO:tasks.workunit.client.1.smithi121.stdout:9/665: getdents d7/d8/d36/da3 0 2024-02-15T18:35:38.621 INFO:tasks.workunit.client.1.smithi121.stdout:5/739: symlink d0/d2/d70/de1/lf4 0 2024-02-15T18:35:38.626 INFO:tasks.workunit.client.1.smithi121.stdout:9/666: dread d7/f13 [0,4194304] 0 2024-02-15T18:35:38.641 INFO:tasks.workunit.client.1.smithi121.stdout:5/740: dwrite d0/d2/d1e/d4e/d7d/d9d/fc4 [0,4194304] 0 2024-02-15T18:35:38.641 INFO:tasks.workunit.client.1.smithi121.stdout:5/741: write d0/d2/faf [5420483,88294] 0 2024-02-15T18:35:38.648 INFO:tasks.workunit.client.1.smithi121.stdout:9/667: truncate d7/d26/d4f/d5b/d77/f91 2893973 0 2024-02-15T18:35:38.648 INFO:tasks.workunit.client.1.smithi121.stdout:9/668: write d7/d26/d4f/f7f [1271529,34163] 0 2024-02-15T18:35:38.648 INFO:tasks.workunit.client.1.smithi121.stdout:9/669: read - d7/d8/d25/d30/d33/d79/fd3 zero size 2024-02-15T18:35:38.648 INFO:tasks.workunit.client.1.smithi121.stdout:9/670: fdatasync d7/d3a/d8e/d9e/da9/d8b/d73/fad 0 2024-02-15T18:35:38.648 INFO:tasks.workunit.client.1.smithi121.stdout:9/671: truncate d7/d8/d25/d30/da5/fd8 4489872 0 2024-02-15T18:35:38.675 INFO:tasks.workunit.client.1.smithi121.stdout:9/672: dwrite d7/d8/d25/d30/d33/f88 [0,4194304] 0 2024-02-15T18:35:38.691 INFO:tasks.workunit.client.1.smithi121.stdout:5/742: dwrite d0/d2/d70/fc0 [0,4194304] 0 2024-02-15T18:35:38.694 INFO:tasks.workunit.client.1.smithi121.stdout:5/743: creat d0/d2/d1e/d4e/d7d/d47/d5a/ff5 x:0 0 0 2024-02-15T18:35:38.697 INFO:tasks.workunit.client.1.smithi121.stdout:5/744: write d0/d2/d1e/d4e/d7d/fe2 [75053,108748] 0 2024-02-15T18:35:38.700 INFO:tasks.workunit.client.1.smithi121.stdout:9/673: dwrite d7/d8/d25/d30/f64 [0,4194304] 0 2024-02-15T18:35:38.704 INFO:tasks.workunit.client.1.smithi121.stdout:9/674: creat d7/d3a/d8e/d9e/fed x:0 0 0 2024-02-15T18:35:38.728 INFO:tasks.workunit.client.1.smithi121.stdout:5/745: dwrite d0/d16/d34/f43 [4194304,4194304] 0 2024-02-15T18:35:38.729 INFO:tasks.workunit.client.1.smithi121.stdout:9/675: dwrite d7/d3a/d8e/d9e/da9/d8b/fbb [0,4194304] 0 2024-02-15T18:35:38.731 INFO:tasks.workunit.client.1.smithi121.stdout:5/746: link d0/d16/d7a/db5/fcb d0/d16/d34/d8f/ff6 0 2024-02-15T18:35:38.733 INFO:tasks.workunit.client.1.smithi121.stdout:9/676: rmdir d7/d8/d25 39 2024-02-15T18:35:38.738 INFO:tasks.workunit.client.1.smithi121.stdout:5/747: unlink d0/d2/fd8 0 2024-02-15T18:35:38.739 INFO:tasks.workunit.client.1.smithi121.stdout:9/677: creat d7/d26/dba/fee x:0 0 0 2024-02-15T18:35:38.744 INFO:tasks.workunit.client.1.smithi121.stdout:9/678: truncate d7/d26/fd6 3613995 0 2024-02-15T18:35:38.744 INFO:tasks.workunit.client.1.smithi121.stdout:9/679: chown d7/d3a/d8e/d9e/cb5 836250 1 2024-02-15T18:35:38.755 INFO:tasks.workunit.client.1.smithi121.stdout:5/748: dwrite d0/d16/d34/da4/dcc/fce [0,4194304] 0 2024-02-15T18:35:38.755 INFO:tasks.workunit.client.1.smithi121.stdout:5/749: chown d0/d2/d1e/d4e/d7d/d40/d62/db1 14 1 2024-02-15T18:35:38.764 INFO:tasks.workunit.client.1.smithi121.stdout:5/750: dread d0/d2/d18/f29 [0,4194304] 0 2024-02-15T18:35:38.765 INFO:tasks.workunit.client.1.smithi121.stdout:5/751: rename d0/d16/d7a/c7e to d0/d16/d34/d8f/cf7 0 2024-02-15T18:35:38.766 INFO:tasks.workunit.client.1.smithi121.stdout:5/752: creat d0/d16/d34/d8f/ff8 x:0 0 0 2024-02-15T18:35:38.766 INFO:tasks.workunit.client.1.smithi121.stdout:5/753: chown d0/d2/c10 23 1 2024-02-15T18:35:38.766 INFO:tasks.workunit.client.1.smithi121.stdout:5/754: rmdir d0/d2/d1e/d4e/d7d/d47/d5a/db3 39 2024-02-15T18:35:38.767 INFO:tasks.workunit.client.1.smithi121.stdout:5/755: rmdir d0/d16/d34/da4/dcc 39 2024-02-15T18:35:38.769 INFO:tasks.workunit.client.1.smithi121.stdout:5/756: symlink d0/d2/d70/d88/lf9 0 2024-02-15T18:35:38.770 INFO:tasks.workunit.client.1.smithi121.stdout:5/757: symlink d0/d16/d7a/lfa 0 2024-02-15T18:35:38.792 INFO:tasks.workunit.client.1.smithi121.stdout:2/363: sync 2024-02-15T18:35:38.993 INFO:tasks.workunit.client.1.smithi121.stdout:0/663: sync 2024-02-15T18:35:38.993 INFO:tasks.workunit.client.1.smithi121.stdout:6/570: sync 2024-02-15T18:35:38.993 INFO:tasks.workunit.client.1.smithi121.stdout:8/545: sync 2024-02-15T18:35:38.994 INFO:tasks.workunit.client.1.smithi121.stdout:0/664: chown d5/d6b/dab/db4/d7d/dd/c5c 1843698 1 2024-02-15T18:35:38.994 INFO:tasks.workunit.client.1.smithi121.stdout:6/571: creat d7/dc/d34/d42/d6e/fc3 x:0 0 0 2024-02-15T18:35:38.996 INFO:tasks.workunit.client.1.smithi121.stdout:6/572: read d7/d4b/d60/d6c/f8a [3079503,87503] 0 2024-02-15T18:35:38.998 INFO:tasks.workunit.client.1.smithi121.stdout:8/546: dread d0/f2 [0,4194304] 0 2024-02-15T18:35:39.000 INFO:tasks.workunit.client.1.smithi121.stdout:6/573: rename d7/dc/f25 to d7/dc/d34/d42/d62/dac/fc4 0 2024-02-15T18:35:39.001 INFO:tasks.workunit.client.1.smithi121.stdout:6/574: creat d7/dc/d34/d42/d6e/fc5 x:0 0 0 2024-02-15T18:35:39.001 INFO:tasks.workunit.client.1.smithi121.stdout:6/575: truncate d7/dc/f48 4879845 0 2024-02-15T18:35:39.003 INFO:tasks.workunit.client.1.smithi121.stdout:6/576: rename d7/dc/d34/d42/d6e/d91/cbe to d7/dae/cc6 0 2024-02-15T18:35:39.005 INFO:tasks.workunit.client.1.smithi121.stdout:6/577: unlink d7/dc/d34/d42/l61 0 2024-02-15T18:35:39.018 INFO:tasks.workunit.client.1.smithi121.stdout:0/665: dwrite d5/d6b/dab/db4/d7d/d2a/f91 [0,4194304] 0 2024-02-15T18:35:39.022 INFO:tasks.workunit.client.1.smithi121.stdout:6/578: dwrite d7/dc/d34/d42/d62/d63/d6f/fb1 [0,4194304] 0 2024-02-15T18:35:39.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:38 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:38.595172+0000 mgr.smithi121.ckiryi (mgr.24385) 13 : cluster [DBG] pgmap v13: 65 pgs: 65 active+clean; 1.2 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 27 MiB/s rd, 55 MiB/s wr, 185 op/s 2024-02-15T18:35:39.024 INFO:tasks.workunit.client.1.smithi121.stdout:6/579: creat d7/dc/d34/d42/d6e/d91/fc7 x:0 0 0 2024-02-15T18:35:39.024 INFO:tasks.workunit.client.1.smithi121.stdout:6/580: chown d7/dc/c1b 400 1 2024-02-15T18:35:39.031 INFO:tasks.workunit.client.1.smithi121.stdout:8/547: dwrite d0/d7/de/f2d [4194304,4194304] 0 2024-02-15T18:35:39.033 INFO:tasks.workunit.client.1.smithi121.stdout:8/548: getdents d0/d7/d36 0 2024-02-15T18:35:39.034 INFO:tasks.workunit.client.1.smithi121.stdout:8/549: truncate d0/d2a/f7d 1083056 0 2024-02-15T18:35:39.043 INFO:tasks.workunit.client.1.smithi121.stdout:8/550: dread d0/d7/db1/dbb/f78 [0,4194304] 0 2024-02-15T18:35:39.043 INFO:tasks.workunit.client.1.smithi121.stdout:8/551: fdatasync d0/f13 0 2024-02-15T18:35:39.044 INFO:tasks.workunit.client.1.smithi121.stdout:0/666: dwrite d5/d6b/dab/db4/d7d/d2a/f91 [0,4194304] 0 2024-02-15T18:35:39.052 INFO:tasks.workunit.client.1.smithi121.stdout:6/581: dwrite d7/dc/d34/d7c/f85 [0,4194304] 0 2024-02-15T18:35:39.052 INFO:tasks.workunit.client.1.smithi121.stdout:6/582: fdatasync d7/f8 0 2024-02-15T18:35:39.052 INFO:tasks.workunit.client.1.smithi121.stdout:6/583: symlink d7/dc/d34/d42/d62/d63/lc8 0 2024-02-15T18:35:39.054 INFO:tasks.workunit.client.1.smithi121.stdout:6/584: rename d7/dc/d8e to d7/d4b/d60/d73/dc9 0 2024-02-15T18:35:39.054 INFO:tasks.workunit.client.1.smithi121.stdout:6/585: fsync d7/dc/d34/f86 0 2024-02-15T18:35:39.055 INFO:tasks.workunit.client.1.smithi121.stdout:6/586: mknod d7/dc/d34/d42/d6e/cca 0 2024-02-15T18:35:39.070 INFO:tasks.workunit.client.1.smithi121.stdout:8/552: dwrite d0/d2a/f4b [0,4194304] 0 2024-02-15T18:35:39.071 INFO:tasks.workunit.client.1.smithi121.stdout:6/587: dread d7/dc/d34/f13 [0,4194304] 0 2024-02-15T18:35:39.071 INFO:tasks.workunit.client.1.smithi121.stdout:0/667: dwrite d5/d6b/dab/db4/d7d/d74/fc6 [0,4194304] 0 2024-02-15T18:35:39.074 INFO:tasks.workunit.client.1.smithi121.stdout:8/553: mknod d0/d7/da7/cc1 0 2024-02-15T18:35:39.074 INFO:tasks.workunit.client.1.smithi121.stdout:8/554: chown d0/d73 14219199 1 2024-02-15T18:35:39.074 INFO:tasks.workunit.client.1.smithi121.stdout:8/555: write d0/d61/d66/f6d [458552,34941] 0 2024-02-15T18:35:39.075 INFO:tasks.workunit.client.1.smithi121.stdout:8/556: write d0/d61/da6/f99 [75949,7574] 0 2024-02-15T18:35:39.075 INFO:tasks.workunit.client.1.smithi121.stdout:8/557: readlink d0/d7/d36/d8c/d45/l5a 0 2024-02-15T18:35:39.077 INFO:tasks.workunit.client.1.smithi121.stdout:0/668: creat d5/d6b/dab/db4/d7d/d2a/d4e/fe5 x:0 0 0 2024-02-15T18:35:39.082 INFO:tasks.workunit.client.1.smithi121.stdout:0/669: truncate d5/d6b/dab/db4/d7d/dd/f7f 8217451 0 2024-02-15T18:35:39.082 INFO:tasks.workunit.client.1.smithi121.stdout:0/670: truncate d5/d6b/dab/db4/d7d/dd/fb0 1691033 0 2024-02-15T18:35:39.085 INFO:tasks.workunit.client.1.smithi121.stdout:0/671: getdents d5/d6b/dab/db4/d7d 0 2024-02-15T18:35:39.085 INFO:tasks.workunit.client.1.smithi121.stdout:0/672: getdents d5/d6b/dab/db4/da6 0 2024-02-15T18:35:39.091 INFO:tasks.workunit.client.1.smithi121.stdout:0/673: link d5/d6b/dab/db4/d7d/d2a/d36/f54 d5/d30/fe6 0 2024-02-15T18:35:39.092 INFO:tasks.workunit.client.1.smithi121.stdout:8/558: dwrite d0/d7/de/d7c/f91 [0,4194304] 0 2024-02-15T18:35:39.092 INFO:tasks.workunit.client.1.smithi121.stdout:8/559: readlink d0/d48/d86/l87 0 2024-02-15T18:35:39.102 INFO:tasks.workunit.client.1.smithi121.stdout:6/588: dwrite d7/f31 [0,4194304] 0 2024-02-15T18:35:39.103 INFO:tasks.workunit.client.1.smithi121.stdout:0/674: dread f2 [4194304,4194304] 0 2024-02-15T18:35:39.103 INFO:tasks.workunit.client.1.smithi121.stdout:6/589: truncate d7/dc/f90 4359721 0 2024-02-15T18:35:39.103 INFO:tasks.workunit.client.1.smithi121.stdout:0/675: dread - d5/d6b/dab/db4/d7d/d2a/d36/d44/fc7 zero size 2024-02-15T18:35:39.103 INFO:tasks.workunit.client.1.smithi121.stdout:6/590: getdents d7/dc/d34/d42/d62/d63/d6f/d89 0 2024-02-15T18:35:39.104 INFO:tasks.workunit.client.1.smithi121.stdout:6/591: chown d7/dae/cbf 218613541 1 2024-02-15T18:35:39.104 INFO:tasks.workunit.client.1.smithi121.stdout:6/592: fdatasync d7/dc/d34/d42/d62/fb0 0 2024-02-15T18:35:39.104 INFO:tasks.workunit.client.1.smithi121.stdout:0/676: symlink d5/d6b/dab/db4/dca/le7 0 2024-02-15T18:35:39.105 INFO:tasks.workunit.client.1.smithi121.stdout:6/593: symlink d7/dc/d34/d42/lcb 0 2024-02-15T18:35:39.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:38 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:38.595172+0000 mgr.smithi121.ckiryi (mgr.24385) 13 : cluster [DBG] pgmap v13: 65 pgs: 65 active+clean; 1.2 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 27 MiB/s rd, 55 MiB/s wr, 185 op/s 2024-02-15T18:35:39.108 INFO:tasks.workunit.client.1.smithi121.stdout:8/560: truncate d0/d7/d36/d52/f5b 2112618 0 2024-02-15T18:35:39.114 INFO:tasks.workunit.client.1.smithi121.stdout:6/594: dread d7/dc/d2c/d36/f9d [0,4194304] 0 2024-02-15T18:35:39.117 INFO:tasks.workunit.client.1.smithi121.stdout:6/595: creat d7/dc/d34/d42/d62/d63/d6f/d89/fcc x:0 0 0 2024-02-15T18:35:39.117 INFO:tasks.workunit.client.1.smithi121.stdout:6/596: fsync d7/dc/d34/d42/d62/f74 0 2024-02-15T18:35:39.118 INFO:tasks.workunit.client.1.smithi121.stdout:8/561: dread d0/d7/f10 [0,4194304] 0 2024-02-15T18:35:39.118 INFO:tasks.workunit.client.1.smithi121.stdout:8/562: fdatasync d0/f8 0 2024-02-15T18:35:39.118 INFO:tasks.workunit.client.1.smithi121.stdout:8/563: chown d0/d7 970 1 2024-02-15T18:35:39.118 INFO:tasks.workunit.client.1.smithi121.stdout:6/597: symlink d7/dae/lcd 0 2024-02-15T18:35:39.122 INFO:tasks.workunit.client.1.smithi121.stdout:0/677: dwrite d5/d6b/dab/db4/d7d/f56 [0,4194304] 0 2024-02-15T18:35:39.122 INFO:tasks.workunit.client.1.smithi121.stdout:0/678: fsync d5/d6b/dab/db4/d7d/d2a/d73/d8d/fde 0 2024-02-15T18:35:39.124 INFO:tasks.workunit.client.1.smithi121.stdout:6/598: mkdir d7/dc/d34/d42/d62/d63/d66/dce 0 2024-02-15T18:35:39.125 INFO:tasks.workunit.client.1.smithi121.stdout:0/679: symlink d5/d6b/dab/db4/d7d/d2a/le8 0 2024-02-15T18:35:39.127 INFO:tasks.workunit.client.1.smithi121.stdout:8/564: dread d0/d7/d36/f81 [0,4194304] 0 2024-02-15T18:35:39.133 INFO:tasks.workunit.client.1.smithi121.stdout:6/599: dread d7/dc/d34/d42/d62/d63/d8d/fa9 [0,4194304] 0 2024-02-15T18:35:39.134 INFO:tasks.workunit.client.1.smithi121.stdout:6/600: truncate d7/d4b/d60/d73/fb2 282701 0 2024-02-15T18:35:39.134 INFO:tasks.workunit.client.1.smithi121.stdout:6/601: read d7/d4b/d59/f72 [1200195,25404] 0 2024-02-15T18:35:39.136 INFO:tasks.workunit.client.1.smithi121.stdout:6/602: getdents d7/dc/d34/d42/d6e/d91 0 2024-02-15T18:35:39.144 INFO:tasks.workunit.client.1.smithi121.stdout:6/603: dread d7/dc/d34/d42/d62/d63/d8d/fa9 [0,4194304] 0 2024-02-15T18:35:39.144 INFO:tasks.workunit.client.1.smithi121.stdout:6/604: chown d7/dc/d34/d7c/lab 899 1 2024-02-15T18:35:39.145 INFO:tasks.workunit.client.1.smithi121.stdout:0/680: dwrite d5/d6b/dab/db4/d7d/d2a/d36/f3d [0,4194304] 0 2024-02-15T18:35:39.146 INFO:tasks.workunit.client.1.smithi121.stdout:6/605: link d7/dc/d34/l12 d7/dc/d34/d42/d62/d63/d66/dce/lcf 0 2024-02-15T18:35:39.146 INFO:tasks.workunit.client.1.smithi121.stdout:6/606: write d7/dc/d34/d42/d62/d63/d8d/fa7 [1287920,70809] 0 2024-02-15T18:35:39.147 INFO:tasks.workunit.client.1.smithi121.stdout:0/681: mknod d5/d6b/dab/db1/dbf/ce9 0 2024-02-15T18:35:39.148 INFO:tasks.workunit.client.1.smithi121.stdout:6/607: rmdir d7/dc/d34/d42/d62/d63/d66/dce 39 2024-02-15T18:35:39.150 INFO:tasks.workunit.client.1.smithi121.stdout:0/682: rename d5/d6b/dab/db4/d7d/d2a/d73/d81/faf to d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/fea 0 2024-02-15T18:35:39.151 INFO:tasks.workunit.client.1.smithi121.stdout:0/683: write d5/d30/f4f [183417,74237] 0 2024-02-15T18:35:39.152 INFO:tasks.workunit.client.1.smithi121.stdout:6/608: creat d7/dae/fd0 x:0 0 0 2024-02-15T18:35:39.152 INFO:tasks.workunit.client.1.smithi121.stdout:8/565: dwrite d0/f2 [0,4194304] 0 2024-02-15T18:35:39.153 INFO:tasks.workunit.client.1.smithi121.stdout:0/684: getdents d5/d6b/dab/db4/d7d/d2a/d4e 0 2024-02-15T18:35:39.160 INFO:tasks.workunit.client.1.smithi121.stdout:6/609: link d7/dc/d34/d42/d6e/fc5 d7/dc/d34/fd1 0 2024-02-15T18:35:39.169 INFO:tasks.workunit.client.1.smithi121.stdout:3/455: sync 2024-02-15T18:35:39.169 INFO:tasks.workunit.client.1.smithi121.stdout:4/495: sync 2024-02-15T18:35:39.171 INFO:tasks.workunit.client.1.smithi121.stdout:8/566: link d0/d2a/d51/l29 d0/d2a/lc2 0 2024-02-15T18:35:39.172 INFO:tasks.workunit.client.1.smithi121.stdout:0/685: mkdir d5/d6b/dab/db4/d7d/d2a/d73/d81/ddb/deb 0 2024-02-15T18:35:39.174 INFO:tasks.workunit.client.1.smithi121.stdout:6/610: symlink d7/dc/d34/d42/d62/d63/ld2 0 2024-02-15T18:35:39.176 INFO:tasks.workunit.client.1.smithi121.stdout:3/456: mkdir d2/d1a/d27/d7c/d25/d88/d96 0 2024-02-15T18:35:39.177 INFO:tasks.workunit.client.1.smithi121.stdout:8/567: creat d0/d61/da6/fc3 x:0 0 0 2024-02-15T18:35:39.177 INFO:tasks.workunit.client.1.smithi121.stdout:8/568: readlink d0/d7/d36/l68 0 2024-02-15T18:35:39.179 INFO:tasks.workunit.client.1.smithi121.stdout:0/686: unlink d5/d6b/dab/db4/d7d/d2a/d36/f3d 0 2024-02-15T18:35:39.179 INFO:tasks.workunit.client.1.smithi121.stdout:0/687: dread - d5/d30/fdd zero size 2024-02-15T18:35:39.179 INFO:tasks.workunit.client.1.smithi121.stdout:0/688: readlink d5/l53 0 2024-02-15T18:35:39.180 INFO:tasks.workunit.client.1.smithi121.stdout:6/611: link d7/dae/cc6 d7/d4b/cd3 0 2024-02-15T18:35:39.181 INFO:tasks.workunit.client.1.smithi121.stdout:8/569: mknod d0/d61/d8f/cc4 0 2024-02-15T18:35:39.181 INFO:tasks.workunit.client.1.smithi121.stdout:8/570: chown d0/d7/de/f4c 18583 1 2024-02-15T18:35:39.182 INFO:tasks.workunit.client.1.smithi121.stdout:6/612: symlink d7/dc/d34/d42/d6e/d94/ld4 0 2024-02-15T18:35:39.184 INFO:tasks.workunit.client.1.smithi121.stdout:8/571: unlink d0/d2a/d51/f6f 0 2024-02-15T18:35:39.184 INFO:tasks.workunit.client.1.smithi121.stdout:8/572: chown d0/d7/d36 1440689 1 2024-02-15T18:35:39.185 INFO:tasks.workunit.client.1.smithi121.stdout:8/573: getdents d0/d61/d7b/dac 0 2024-02-15T18:35:39.185 INFO:tasks.workunit.client.1.smithi121.stdout:6/613: mkdir d7/d4b/d60/d6c/dd5 0 2024-02-15T18:35:39.186 INFO:tasks.workunit.client.1.smithi121.stdout:4/496: dwrite d5/f6 [0,4194304] 0 2024-02-15T18:35:39.186 INFO:tasks.workunit.client.1.smithi121.stdout:6/614: write d7/dc/d34/d7c/faa [964597,67453] 0 2024-02-15T18:35:39.186 INFO:tasks.workunit.client.1.smithi121.stdout:8/574: mknod d0/d48/cc5 0 2024-02-15T18:35:39.188 INFO:tasks.workunit.client.1.smithi121.stdout:4/497: truncate d5/d9/d1d/f24 124220 0 2024-02-15T18:35:39.189 INFO:tasks.workunit.client.1.smithi121.stdout:4/498: chown d5/d9/d19/d4c/f8b 964219105 1 2024-02-15T18:35:39.189 INFO:tasks.workunit.client.1.smithi121.stdout:6/615: rmdir d7/dc/d34/d42/d62/d63/d66/dce 39 2024-02-15T18:35:39.189 INFO:tasks.workunit.client.1.smithi121.stdout:6/616: fdatasync d7/dc/d34/d42/d62/d63/d8d/fa9 0 2024-02-15T18:35:39.190 INFO:tasks.workunit.client.1.smithi121.stdout:8/575: rmdir d0/d61/d7b 39 2024-02-15T18:35:39.191 INFO:tasks.workunit.client.1.smithi121.stdout:4/499: write d5/fa [9057430,118602] 0 2024-02-15T18:35:39.192 INFO:tasks.workunit.client.1.smithi121.stdout:6/617: link d7/f11 d7/dc/d34/d42/d62/d63/d66/dce/fd6 0 2024-02-15T18:35:39.193 INFO:tasks.workunit.client.1.smithi121.stdout:8/576: truncate d0/d7/f10 6881129 0 2024-02-15T18:35:39.194 INFO:tasks.workunit.client.1.smithi121.stdout:4/500: mkdir d5/d31/d3d/d9b 0 2024-02-15T18:35:39.194 INFO:tasks.workunit.client.1.smithi121.stdout:4/501: fsync d5/d9/d19/f4e 0 2024-02-15T18:35:39.195 INFO:tasks.workunit.client.1.smithi121.stdout:8/577: mknod d0/d3/cc6 0 2024-02-15T18:35:39.196 INFO:tasks.workunit.client.1.smithi121.stdout:6/618: dread d7/d4b/d60/d73/fb2 [0,4194304] 0 2024-02-15T18:35:39.196 INFO:tasks.workunit.client.1.smithi121.stdout:6/619: fsync d7/dc/f5e 0 2024-02-15T18:35:39.196 INFO:tasks.workunit.client.1.smithi121.stdout:4/502: mkdir d5/d31/d3d/d4f/d51/d9c 0 2024-02-15T18:35:39.198 INFO:tasks.workunit.client.1.smithi121.stdout:8/578: unlink d0/d7/d36/d8c/d45/l5a 0 2024-02-15T18:35:39.200 INFO:tasks.workunit.client.1.smithi121.stdout:6/620: symlink d7/d4b/d59/ld7 0 2024-02-15T18:35:39.200 INFO:tasks.workunit.client.1.smithi121.stdout:8/579: write d0/d61/d66/f3b [800027,78252] 0 2024-02-15T18:35:39.201 INFO:tasks.workunit.client.1.smithi121.stdout:4/503: mkdir d5/d9/d1d/d26/d9d 0 2024-02-15T18:35:39.206 INFO:tasks.workunit.client.1.smithi121.stdout:3/457: dwrite d2/d1a/d22/f6a [4194304,4194304] 0 2024-02-15T18:35:39.213 INFO:tasks.workunit.client.1.smithi121.stdout:8/580: dread d0/d2a/d51/f6b [4194304,4194304] 0 2024-02-15T18:35:39.218 INFO:tasks.workunit.client.1.smithi121.stdout:8/581: creat d0/d7/d36/d52/fc7 x:0 0 0 2024-02-15T18:35:39.219 INFO:tasks.workunit.client.1.smithi121.stdout:4/504: dwrite d5/d9/d19/d2a/f2c [4194304,4194304] 0 2024-02-15T18:35:39.220 INFO:tasks.workunit.client.1.smithi121.stdout:3/458: getdents d2/d1a/d27/d7c 0 2024-02-15T18:35:39.220 INFO:tasks.workunit.client.1.smithi121.stdout:8/582: creat d0/d2a/d51/fc8 x:0 0 0 2024-02-15T18:35:39.220 INFO:tasks.workunit.client.1.smithi121.stdout:3/459: chown d2/d1a/d27/d7c/d25/d30/f41 1675 1 2024-02-15T18:35:39.220 INFO:tasks.workunit.client.1.smithi121.stdout:3/460: chown d2/d5a/f78 543154873 1 2024-02-15T18:35:39.224 INFO:tasks.workunit.client.1.smithi121.stdout:6/621: dwrite d7/dc/d34/d42/d62/d63/d6f/d89/fb7 [0,4194304] 0 2024-02-15T18:35:39.226 INFO:tasks.workunit.client.1.smithi121.stdout:4/505: rmdir d5/d9/d1d/d26/d73/d81 39 2024-02-15T18:35:39.226 INFO:tasks.workunit.client.1.smithi121.stdout:4/506: stat d5/d31/d3d/d4f/d51/d63 0 2024-02-15T18:35:39.226 INFO:tasks.workunit.client.1.smithi121.stdout:4/507: getdents d5/d9/d19/d4c/d9a 0 2024-02-15T18:35:39.227 INFO:tasks.workunit.client.1.smithi121.stdout:8/583: unlink d0/l19 0 2024-02-15T18:35:39.228 INFO:tasks.workunit.client.1.smithi121.stdout:3/461: mkdir d2/d1a/d27/d7c/d25/d31/d97 0 2024-02-15T18:35:39.232 INFO:tasks.workunit.client.1.smithi121.stdout:6/622: symlink d7/dc/d34/d42/d62/d63/ld8 0 2024-02-15T18:35:39.233 INFO:tasks.workunit.client.1.smithi121.stdout:4/508: unlink d5/d9/d76/l92 0 2024-02-15T18:35:39.233 INFO:tasks.workunit.client.1.smithi121.stdout:4/509: fsync d5/d9/d1d/f24 0 2024-02-15T18:35:39.235 INFO:tasks.workunit.client.1.smithi121.stdout:8/584: mkdir d0/d3/dc9 0 2024-02-15T18:35:39.235 INFO:tasks.workunit.client.1.smithi121.stdout:3/462: symlink d2/d1a/d27/d7c/d25/d88/d96/l98 0 2024-02-15T18:35:39.237 INFO:tasks.workunit.client.1.smithi121.stdout:6/623: rename d7/dc/d2c/d36/f6a to d7/dc/d34/d42/d62/fd9 0 2024-02-15T18:35:39.239 INFO:tasks.workunit.client.1.smithi121.stdout:8/585: rmdir d0/d61/d95 39 2024-02-15T18:35:39.241 INFO:tasks.workunit.client.1.smithi121.stdout:3/463: link d2/d5a/f78 d2/d1a/d27/d7c/d25/d31/d34/d4a/f99 0 2024-02-15T18:35:39.247 INFO:tasks.workunit.client.1.smithi121.stdout:4/510: rename d5/d7 to d5/d9/d76/d9e 0 2024-02-15T18:35:39.249 INFO:tasks.workunit.client.1.smithi121.stdout:6/624: unlink d7/dae/fd0 0 2024-02-15T18:35:39.251 INFO:tasks.workunit.client.1.smithi121.stdout:8/586: write d0/d73/fa9 [1461091,295] 0 2024-02-15T18:35:39.253 INFO:tasks.workunit.client.1.smithi121.stdout:4/511: mknod d5/d31/d3d/d9b/c9f 0 2024-02-15T18:35:39.254 INFO:tasks.workunit.client.1.smithi121.stdout:3/464: mkdir d2/d1a/d27/d7c/d25/d88/d9a 0 2024-02-15T18:35:39.256 INFO:tasks.workunit.client.1.smithi121.stdout:8/587: rename d0/d61/db6 to d0/d2a/d51/dca 0 2024-02-15T18:35:39.276 INFO:tasks.workunit.client.1.smithi121.stdout:8/588: rmdir d0/d2a/d51/dca 0 2024-02-15T18:35:39.276 INFO:tasks.workunit.client.1.smithi121.stdout:8/589: truncate d0/fbd 743887 0 2024-02-15T18:35:39.276 INFO:tasks.workunit.client.1.smithi121.stdout:8/590: stat d0/d7/d36/f5c 0 2024-02-15T18:35:39.278 INFO:tasks.workunit.client.1.smithi121.stdout:8/591: rename d0/d7/de/d7c to d0/d7/da5/dcb 0 2024-02-15T18:35:39.279 INFO:tasks.workunit.client.1.smithi121.stdout:8/592: creat d0/d7/d36/d8c/d45/fcc x:0 0 0 2024-02-15T18:35:39.280 INFO:tasks.workunit.client.1.smithi121.stdout:8/593: mkdir d0/d7/d36/d8c/d45/dcd 0 2024-02-15T18:35:39.280 INFO:tasks.workunit.client.1.smithi121.stdout:6/625: dwrite d7/dc/d34/d42/d62/d63/d6f/f88 [0,4194304] 0 2024-02-15T18:35:39.280 INFO:tasks.workunit.client.1.smithi121.stdout:6/626: readlink d7/dc/d34/d7c/lab 0 2024-02-15T18:35:39.281 INFO:tasks.workunit.client.1.smithi121.stdout:6/627: readlink d7/dc/d34/d42/d62/l84 0 2024-02-15T18:35:39.281 INFO:tasks.workunit.client.1.smithi121.stdout:4/512: dwrite d5/d9/d1d/f24 [0,4194304] 0 2024-02-15T18:35:39.281 INFO:tasks.workunit.client.1.smithi121.stdout:3/465: dwrite d2/d1a/d27/d7c/d25/d30/f41 [0,4194304] 0 2024-02-15T18:35:39.282 INFO:tasks.workunit.client.1.smithi121.stdout:8/594: unlink d0/d61/da6/fc3 0 2024-02-15T18:35:39.282 INFO:tasks.workunit.client.1.smithi121.stdout:8/595: chown d0/d2a/fa3 29 1 2024-02-15T18:35:39.285 INFO:tasks.workunit.client.1.smithi121.stdout:6/628: creat d7/dc/d34/d42/d62/d63/d6f/d89/da1/fda x:0 0 0 2024-02-15T18:35:39.288 INFO:tasks.workunit.client.1.smithi121.stdout:6/629: write d7/dc/f15 [7244413,21605] 0 2024-02-15T18:35:39.290 INFO:tasks.workunit.client.1.smithi121.stdout:4/513: dread d5/f6 [0,4194304] 0 2024-02-15T18:35:39.290 INFO:tasks.workunit.client.1.smithi121.stdout:3/466: dread d2/d1a/d27/d7c/f7b [0,4194304] 0 2024-02-15T18:35:39.290 INFO:tasks.workunit.client.1.smithi121.stdout:3/467: fsync d2/d1a/d6b/f79 0 2024-02-15T18:35:39.291 INFO:tasks.workunit.client.1.smithi121.stdout:6/630: unlink d7/dc/d34/l12 0 2024-02-15T18:35:39.291 INFO:tasks.workunit.client.1.smithi121.stdout:6/631: dread - d7/dc/d34/fd1 zero size 2024-02-15T18:35:39.296 INFO:tasks.workunit.client.1.smithi121.stdout:4/514: mkdir d5/d9/d19/da0 0 2024-02-15T18:35:39.296 INFO:tasks.workunit.client.1.smithi121.stdout:4/515: fsync d5/f6 0 2024-02-15T18:35:39.297 INFO:tasks.workunit.client.1.smithi121.stdout:8/596: write d0/d48/f6a [826621,46980] 0 2024-02-15T18:35:39.298 INFO:tasks.workunit.client.1.smithi121.stdout:3/468: getdents d2/d1a/d27/d7c/d25/d88 0 2024-02-15T18:35:39.299 INFO:tasks.workunit.client.1.smithi121.stdout:3/469: read d2/d1a/d27/d7c/f56 [541598,74659] 0 2024-02-15T18:35:39.300 INFO:tasks.workunit.client.1.smithi121.stdout:6/632: creat d7/dc/d34/d42/d62/fdb x:0 0 0 2024-02-15T18:35:39.302 INFO:tasks.workunit.client.1.smithi121.stdout:4/516: mknod d5/d9/ca1 0 2024-02-15T18:35:39.303 INFO:tasks.workunit.client.1.smithi121.stdout:8/597: unlink d0/d61/lb0 0 2024-02-15T18:35:39.307 INFO:tasks.workunit.client.1.smithi121.stdout:3/470: mknod d2/d1a/d27/d7c/d25/d88/c9b 0 2024-02-15T18:35:39.307 INFO:tasks.workunit.client.1.smithi121.stdout:6/633: mknod d7/dc/d2c/cdc 0 2024-02-15T18:35:39.307 INFO:tasks.workunit.client.1.smithi121.stdout:8/598: rmdir d0/d7/d36/d8c 39 2024-02-15T18:35:39.310 INFO:tasks.workunit.client.1.smithi121.stdout:6/634: mknod d7/d4b/d59/cdd 0 2024-02-15T18:35:39.311 INFO:tasks.workunit.client.1.smithi121.stdout:3/471: dread d2/d1a/d27/d7c/d25/d31/f50 [0,4194304] 0 2024-02-15T18:35:39.312 INFO:tasks.workunit.client.1.smithi121.stdout:6/635: dread d7/d4b/d60/d73/fb2 [0,4194304] 0 2024-02-15T18:35:39.312 INFO:tasks.workunit.client.1.smithi121.stdout:6/636: dread - d7/dc/d2c/d36/fa8 zero size 2024-02-15T18:35:39.312 INFO:tasks.workunit.client.1.smithi121.stdout:3/472: creat d2/d1a/d27/d7c/d25/d31/d34/d4a/f9c x:0 0 0 2024-02-15T18:35:39.325 INFO:tasks.workunit.client.1.smithi121.stdout:4/517: dwrite d5/d9/d19/d4c/d82/f8e [4194304,4194304] 0 2024-02-15T18:35:39.325 INFO:tasks.workunit.client.1.smithi121.stdout:4/518: dread - d5/d9/d19/d28/f74 zero size 2024-02-15T18:35:39.326 INFO:tasks.workunit.client.1.smithi121.stdout:4/519: write d5/d9/d1d/f42 [1965671,65601] 0 2024-02-15T18:35:39.327 INFO:tasks.workunit.client.1.smithi121.stdout:4/520: rmdir d5/d31/d3d/d4f 39 2024-02-15T18:35:39.330 INFO:tasks.workunit.client.1.smithi121.stdout:4/521: dread d5/d9/f2e [0,4194304] 0 2024-02-15T18:35:39.341 INFO:tasks.workunit.client.1.smithi121.stdout:6/637: dwrite d7/dc/d34/fbd [0,4194304] 0 2024-02-15T18:35:39.341 INFO:tasks.workunit.client.1.smithi121.stdout:6/638: fdatasync d7/f8 0 2024-02-15T18:35:39.342 INFO:tasks.workunit.client.1.smithi121.stdout:4/522: write d5/d9/d76/d9e/d14/f29 [1616518,66865] 0 2024-02-15T18:35:39.344 INFO:tasks.workunit.client.1.smithi121.stdout:3/473: dwrite d2/d1a/f37 [0,4194304] 0 2024-02-15T18:35:39.344 INFO:tasks.workunit.client.1.smithi121.stdout:4/523: mknod d5/d9/d19/d4c/d9a/ca2 0 2024-02-15T18:35:39.345 INFO:tasks.workunit.client.1.smithi121.stdout:8/599: dwrite d0/d7/da5/dcb/faf [0,4194304] 0 2024-02-15T18:35:39.346 INFO:tasks.workunit.client.1.smithi121.stdout:3/474: mknod d2/d1a/d54/c9d 0 2024-02-15T18:35:39.348 INFO:tasks.workunit.client.1.smithi121.stdout:4/524: link d5/c83 d5/d31/d3d/d4a/ca3 0 2024-02-15T18:35:39.360 INFO:tasks.workunit.client.1.smithi121.stdout:3/475: symlink d2/d1a/d27/d7c/d25/d88/d96/l9e 0 2024-02-15T18:35:39.362 INFO:tasks.workunit.client.1.smithi121.stdout:4/525: symlink d5/d31/d25/la4 0 2024-02-15T18:35:39.362 INFO:tasks.workunit.client.1.smithi121.stdout:6/639: dwrite d7/dc/d34/d42/d62/d63/f7a [0,4194304] 0 2024-02-15T18:35:39.362 INFO:tasks.workunit.client.1.smithi121.stdout:3/476: mkdir d2/d1a/d22/d85/d9f 0 2024-02-15T18:35:39.362 INFO:tasks.workunit.client.1.smithi121.stdout:3/477: truncate d2/d1a/f92 430321 0 2024-02-15T18:35:39.362 INFO:tasks.workunit.client.1.smithi121.stdout:6/640: creat d7/dc/d34/d42/d62/fde x:0 0 0 2024-02-15T18:35:39.364 INFO:tasks.workunit.client.1.smithi121.stdout:8/600: dwrite d0/d61/d66/fab [4194304,4194304] 0 2024-02-15T18:35:39.372 INFO:tasks.workunit.client.1.smithi121.stdout:6/641: mknod d7/dc/d34/d42/cdf 0 2024-02-15T18:35:39.373 INFO:tasks.workunit.client.1.smithi121.stdout:8/601: mkdir d0/dce 0 2024-02-15T18:35:39.374 INFO:tasks.workunit.client.1.smithi121.stdout:4/526: dread d5/d9/d19/d4c/f8b [0,4194304] 0 2024-02-15T18:35:39.374 INFO:tasks.workunit.client.1.smithi121.stdout:6/642: mknod d7/dc/d34/d42/d62/d63/d6f/ce0 0 2024-02-15T18:35:39.376 INFO:tasks.workunit.client.1.smithi121.stdout:6/643: read d7/dc/d34/d42/d62/d63/f7a [302143,32876] 0 2024-02-15T18:35:39.376 INFO:tasks.workunit.client.1.smithi121.stdout:8/602: rename d0/d7/da5/dcb/f82 to d0/d7/d36/fcf 0 2024-02-15T18:35:39.378 INFO:tasks.workunit.client.1.smithi121.stdout:4/527: creat d5/d31/d3d/d9b/fa5 x:0 0 0 2024-02-15T18:35:39.382 INFO:tasks.workunit.client.1.smithi121.stdout:7/436: sync 2024-02-15T18:35:39.383 INFO:tasks.workunit.client.1.smithi121.stdout:4/528: read d5/d31/f23 [3718154,103625] 0 2024-02-15T18:35:39.383 INFO:tasks.workunit.client.1.smithi121.stdout:4/529: read - d5/d9/d19/d2a/f96 zero size 2024-02-15T18:35:39.383 INFO:tasks.workunit.client.1.smithi121.stdout:8/603: symlink d0/d3/dc9/ld0 0 2024-02-15T18:35:39.386 INFO:tasks.workunit.client.1.smithi121.stdout:4/530: unlink d5/d9/d19/d57/f77 0 2024-02-15T18:35:39.388 INFO:tasks.workunit.client.1.smithi121.stdout:4/531: link c3 d5/d9/d19/ca6 0 2024-02-15T18:35:39.389 INFO:tasks.workunit.client.1.smithi121.stdout:4/532: creat d5/d9/d19/d4c/d9a/fa7 x:0 0 0 2024-02-15T18:35:39.389 INFO:tasks.workunit.client.1.smithi121.stdout:4/533: write d5/d9/f2e [1281689,101464] 0 2024-02-15T18:35:39.394 INFO:tasks.workunit.client.1.smithi121.stdout:6/644: dwrite d7/dc/d34/d7c/faa [0,4194304] 0 2024-02-15T18:35:39.398 INFO:tasks.workunit.client.1.smithi121.stdout:6/645: creat d7/dc/d55/fe1 x:0 0 0 2024-02-15T18:35:39.398 INFO:tasks.workunit.client.1.smithi121.stdout:7/437: dread d0/d5/f2b [0,4194304] 0 2024-02-15T18:35:39.398 INFO:tasks.workunit.client.1.smithi121.stdout:4/534: dread d5/f30 [0,4194304] 0 2024-02-15T18:35:39.398 INFO:tasks.workunit.client.1.smithi121.stdout:4/535: chown d5/d9/d76/d87 174619 1 2024-02-15T18:35:39.399 INFO:tasks.workunit.client.1.smithi121.stdout:6/646: rename d7/l41 to d7/dc/d76/le2 0 2024-02-15T18:35:39.406 INFO:tasks.workunit.client.1.smithi121.stdout:4/536: write d5/d9/d1d/d26/d72/f78 [1412931,14862] 0 2024-02-15T18:35:39.406 INFO:tasks.workunit.client.1.smithi121.stdout:6/647: creat d7/dc/d34/d42/d62/d63/d8d/fe3 x:0 0 0 2024-02-15T18:35:39.409 INFO:tasks.workunit.client.1.smithi121.stdout:6/648: symlink d7/le4 0 2024-02-15T18:35:39.409 INFO:tasks.workunit.client.1.smithi121.stdout:6/649: chown d7/d4b/f53 190557 1 2024-02-15T18:35:39.410 INFO:tasks.workunit.client.1.smithi121.stdout:4/537: creat d5/d9/d1d/d26/d72/fa8 x:0 0 0 2024-02-15T18:35:39.411 INFO:tasks.workunit.client.1.smithi121.stdout:6/650: unlink d7/dc/d34/d42/d62/d63/d8d/fe3 0 2024-02-15T18:35:39.411 INFO:tasks.workunit.client.1.smithi121.stdout:7/438: dwrite d0/d5/f81 [0,4194304] 0 2024-02-15T18:35:39.413 INFO:tasks.workunit.client.1.smithi121.stdout:4/538: getdents d5/d9/d19/d4c/d82 0 2024-02-15T18:35:39.414 INFO:tasks.workunit.client.1.smithi121.stdout:6/651: unlink d7/dc/d2c/d36/f58 0 2024-02-15T18:35:39.416 INFO:tasks.workunit.client.1.smithi121.stdout:7/439: truncate d0/d5/d11/d1c/f3e 3697243 0 2024-02-15T18:35:39.417 INFO:tasks.workunit.client.1.smithi121.stdout:4/539: mknod d5/d31/d3d/d9b/ca9 0 2024-02-15T18:35:39.418 INFO:tasks.workunit.client.1.smithi121.stdout:4/540: creat d5/d31/d3d/d4f/d8a/faa x:0 0 0 2024-02-15T18:35:39.427 INFO:tasks.workunit.client.1.smithi121.stdout:4/541: truncate d5/d31/d25/f37 1304767 0 2024-02-15T18:35:39.438 INFO:tasks.workunit.client.1.smithi121.stdout:4/542: dread d5/d31/d25/f37 [0,4194304] 0 2024-02-15T18:35:39.438 INFO:tasks.workunit.client.1.smithi121.stdout:4/543: dread - d5/d31/d3d/d4f/d8a/faa zero size 2024-02-15T18:35:39.438 INFO:tasks.workunit.client.1.smithi121.stdout:4/544: read - d5/d9/d19/d5c/f69 zero size 2024-02-15T18:35:39.439 INFO:tasks.workunit.client.1.smithi121.stdout:4/545: creat d5/d9/d76/d87/fab x:0 0 0 2024-02-15T18:35:39.439 INFO:tasks.workunit.client.1.smithi121.stdout:4/546: dread - d5/d9/d19/d28/f74 zero size 2024-02-15T18:35:39.439 INFO:tasks.workunit.client.1.smithi121.stdout:4/547: mknod d5/d9/d19/cac 0 2024-02-15T18:35:39.445 INFO:tasks.workunit.client.1.smithi121.stdout:4/548: dread d5/d9/d1d/f42 [0,4194304] 0 2024-02-15T18:35:39.446 INFO:tasks.workunit.client.1.smithi121.stdout:4/549: creat d5/d9/d1d/d26/d73/d81/fad x:0 0 0 2024-02-15T18:35:39.446 INFO:tasks.workunit.client.1.smithi121.stdout:4/550: fsync d5/d31/f23 0 2024-02-15T18:35:39.447 INFO:tasks.workunit.client.1.smithi121.stdout:4/551: fsync d5/d31/d3d/d4f/d51/d63/f99 0 2024-02-15T18:35:39.448 INFO:tasks.workunit.client.1.smithi121.stdout:4/552: creat d5/d31/d3d/d4f/d51/d9c/fae x:0 0 0 2024-02-15T18:35:39.448 INFO:tasks.workunit.client.1.smithi121.stdout:4/553: read - d5/d9/d19/d4c/d9a/fa7 zero size 2024-02-15T18:35:39.448 INFO:tasks.workunit.client.1.smithi121.stdout:4/554: truncate d5/d9/d76/d9e/f75 1608357 0 2024-02-15T18:35:39.462 INFO:tasks.workunit.client.1.smithi121.stdout:4/555: dwrite d5/d9/d19/d28/f21 [0,4194304] 0 2024-02-15T18:35:39.462 INFO:tasks.workunit.client.1.smithi121.stdout:4/556: chown d5/d9/d1d/d26/l86 0 1 2024-02-15T18:35:39.463 INFO:tasks.workunit.client.1.smithi121.stdout:4/557: getdents d5/d9/d19/d5c 0 2024-02-15T18:35:39.465 INFO:tasks.workunit.client.1.smithi121.stdout:4/558: getdents d5/d9/d19/d57 0 2024-02-15T18:35:39.475 INFO:tasks.workunit.client.1.smithi121.stdout:4/559: dread d5/d9/d19/d28/f21 [0,4194304] 0 2024-02-15T18:35:39.475 INFO:tasks.workunit.client.1.smithi121.stdout:4/560: chown d5/d9/d19/d57 64585 1 2024-02-15T18:35:39.476 INFO:tasks.workunit.client.1.smithi121.stdout:4/561: creat d5/d31/d3d/d4f/faf x:0 0 0 2024-02-15T18:35:39.476 INFO:tasks.workunit.client.1.smithi121.stdout:4/562: fdatasync d5/d9/d1d/f42 0 2024-02-15T18:35:39.478 INFO:tasks.workunit.client.1.smithi121.stdout:4/563: rename d5/d9/d76/d9e/d14/c70 to d5/d9/d1d/d26/d73/d81/cb0 0 2024-02-15T18:35:39.482 INFO:tasks.workunit.client.1.smithi121.stdout:4/564: mknod d5/d9/d19/cb1 0 2024-02-15T18:35:39.483 INFO:tasks.workunit.client.1.smithi121.stdout:4/565: fdatasync d5/d9/d76/d9e/d14/f17 0 2024-02-15T18:35:39.483 INFO:tasks.workunit.client.1.smithi121.stdout:4/566: dread - d5/d9/d1d/d26/d73/d81/fad zero size 2024-02-15T18:35:39.483 INFO:tasks.workunit.client.1.smithi121.stdout:4/567: dread - d5/d31/d3d/d4f/d51/d9c/fae zero size 2024-02-15T18:35:39.483 INFO:tasks.workunit.client.1.smithi121.stdout:4/568: symlink d5/d9/d19/d4c/d8d/lb2 0 2024-02-15T18:35:39.483 INFO:tasks.workunit.client.1.smithi121.stdout:4/569: write d5/d9/d19/d2a/f2c [8750981,119879] 0 2024-02-15T18:35:39.484 INFO:tasks.workunit.client.1.smithi121.stdout:4/570: write d5/d31/d3d/d4f/faf [408445,81196] 0 2024-02-15T18:35:39.484 INFO:tasks.workunit.client.1.smithi121.stdout:4/571: chown d5/d31/d3d/d4a/d7e/f7f 7837 1 2024-02-15T18:35:39.486 INFO:tasks.workunit.client.1.smithi121.stdout:4/572: creat d5/d9/d19/d28/fb3 x:0 0 0 2024-02-15T18:35:39.507 INFO:tasks.workunit.client.1.smithi121.stdout:4/573: dwrite d5/f30 [4194304,4194304] 0 2024-02-15T18:35:39.509 INFO:tasks.workunit.client.1.smithi121.stdout:4/574: getdents d5/d9/d19/d4c 0 2024-02-15T18:35:39.519 INFO:tasks.workunit.client.1.smithi121.stdout:4/575: dread d5/d9/d1d/f24 [0,4194304] 0 2024-02-15T18:35:39.555 INFO:tasks.workunit.client.1.smithi121.stdout:4/576: dwrite d5/d31/d3d/d4f/d51/d9c/fae [0,4194304] 0 2024-02-15T18:35:39.580 INFO:tasks.workunit.client.1.smithi121.stdout:4/577: dwrite d5/d31/d25/f33 [0,4194304] 0 2024-02-15T18:35:39.585 INFO:tasks.workunit.client.1.smithi121.stdout:4/578: rename d5/d31/d3d/d4f/d51/d9c to d5/d9/d76/d87/db4 0 2024-02-15T18:35:39.585 INFO:tasks.workunit.client.1.smithi121.stdout:4/579: fsync d5/d9/f20 0 2024-02-15T18:35:39.586 INFO:tasks.workunit.client.1.smithi121.stdout:4/580: creat d5/d31/d3d/d4f/d8a/fb5 x:0 0 0 2024-02-15T18:35:39.589 INFO:tasks.workunit.client.1.smithi121.stdout:4/581: mkdir d5/d31/d25/db6 0 2024-02-15T18:35:39.589 INFO:tasks.workunit.client.1.smithi121.stdout:4/582: fdatasync d5/d9/d19/d4c/d82/f8e 0 2024-02-15T18:35:39.614 INFO:tasks.workunit.client.1.smithi121.stdout:4/583: dwrite d5/d31/d25/f37 [0,4194304] 0 2024-02-15T18:35:39.616 INFO:tasks.workunit.client.1.smithi121.stdout:4/584: mknod d5/d9/d1d/cb7 0 2024-02-15T18:35:39.617 INFO:tasks.workunit.client.1.smithi121.stdout:4/585: truncate d5/d9/d19/d2a/d4d/f54 1383121 0 2024-02-15T18:35:39.618 INFO:tasks.workunit.client.1.smithi121.stdout:4/586: symlink d5/d9/d19/d2a/d4d/lb8 0 2024-02-15T18:35:39.619 INFO:tasks.workunit.client.1.smithi121.stdout:4/587: chown d5/d9/d19/d5c/d98 35175600 1 2024-02-15T18:35:39.620 INFO:tasks.workunit.client.1.smithi121.stdout:4/588: creat d5/d9/d76/d9e/fb9 x:0 0 0 2024-02-15T18:35:39.639 INFO:tasks.workunit.client.1.smithi121.stdout:1/615: sync 2024-02-15T18:35:39.639 INFO:tasks.workunit.client.1.smithi121.stdout:1/616: chown d0/d6/f1a 677089320 1 2024-02-15T18:35:39.641 INFO:tasks.workunit.client.1.smithi121.stdout:1/617: mkdir d0/d39/d7d/dc0/dcf 0 2024-02-15T18:35:39.641 INFO:tasks.workunit.client.1.smithi121.stdout:4/589: write d5/d9/d19/d2a/d4d/f54 [1364842,27336] 0 2024-02-15T18:35:39.645 INFO:tasks.workunit.client.1.smithi121.stdout:1/618: creat d0/d6/d7/d9/d14/d3e/fd0 x:0 0 0 2024-02-15T18:35:39.645 INFO:tasks.workunit.client.1.smithi121.stdout:4/590: symlink d5/d9/d1d/d26/lba 0 2024-02-15T18:35:39.645 INFO:tasks.workunit.client.1.smithi121.stdout:4/591: truncate d5/d9/d19/d5c/f69 670339 0 2024-02-15T18:35:39.645 INFO:tasks.workunit.client.1.smithi121.stdout:4/592: write d5/d9/d19/d2a/d4d/f54 [2197505,113565] 0 2024-02-15T18:35:39.653 INFO:tasks.workunit.client.1.smithi121.stdout:1/619: write d0/d39/f68 [3583847,22167] 0 2024-02-15T18:35:39.653 INFO:tasks.workunit.client.1.smithi121.stdout:1/620: chown d0/d39/d7d/dc0/cb2 6441 1 2024-02-15T18:35:39.655 INFO:tasks.workunit.client.1.smithi121.stdout:1/621: creat d0/d39/d7d/dc0/d9d/fd1 x:0 0 0 2024-02-15T18:35:39.657 INFO:tasks.workunit.client.1.smithi121.stdout:1/622: link d0/f1d d0/d39/d4b/dac/fd2 0 2024-02-15T18:35:39.662 INFO:tasks.workunit.client.1.smithi121.stdout:1/623: write d0/d39/d7d/dc0/f5d [4050853,84391] 0 2024-02-15T18:35:39.665 INFO:tasks.workunit.client.1.smithi121.stdout:1/624: unlink d0/fbc 0 2024-02-15T18:35:39.665 INFO:tasks.workunit.client.1.smithi121.stdout:4/593: dwrite d5/d9/d19/d4c/d9a/fa7 [0,4194304] 0 2024-02-15T18:35:39.666 INFO:tasks.workunit.client.1.smithi121.stdout:1/625: read d0/d6/d7/d9/d51/f6c [449359,60618] 0 2024-02-15T18:35:39.666 INFO:tasks.workunit.client.1.smithi121.stdout:1/626: stat d0/d6/d7/d9/d14/d3e/d88/caa 0 2024-02-15T18:35:39.701 INFO:tasks.workunit.client.1.smithi121.stdout:4/594: dwrite d5/d31/d3d/f49 [0,4194304] 0 2024-02-15T18:35:39.701 INFO:tasks.workunit.client.1.smithi121.stdout:1/627: dwrite d0/d6/d7/d9/d14/d3e/f71 [0,4194304] 0 2024-02-15T18:35:39.701 INFO:tasks.workunit.client.1.smithi121.stdout:1/628: chown d0/d6/d7/d9/d51/d5f/c89 18532 1 2024-02-15T18:35:39.702 INFO:tasks.workunit.client.1.smithi121.stdout:4/595: symlink d5/d9/d1d/d26/d73/d81/lbb 0 2024-02-15T18:35:39.705 INFO:tasks.workunit.client.1.smithi121.stdout:4/596: rename d5/d9/d19/d4c/f8b to d5/d31/d3d/d4f/d51/d71/fbc 0 2024-02-15T18:35:39.706 INFO:tasks.workunit.client.1.smithi121.stdout:4/597: mknod d5/d9/d76/d9e/d14/cbd 0 2024-02-15T18:35:39.706 INFO:tasks.workunit.client.1.smithi121.stdout:4/598: truncate d5/d9/d1d/f24 4926848 0 2024-02-15T18:35:39.706 INFO:tasks.workunit.client.1.smithi121.stdout:4/599: write d5/d9/d1d/d26/d73/d81/fad [119416,96487] 0 2024-02-15T18:35:39.724 INFO:tasks.workunit.client.1.smithi121.stdout:4/600: dwrite d5/d9/d19/d2a/f2c [4194304,4194304] 0 2024-02-15T18:35:39.724 INFO:tasks.workunit.client.1.smithi121.stdout:4/601: dread - d5/d9/d19/d28/fb3 zero size 2024-02-15T18:35:39.726 INFO:tasks.workunit.client.1.smithi121.stdout:4/602: rmdir d5/d31/d3d/d9b 39 2024-02-15T18:35:39.726 INFO:tasks.workunit.client.1.smithi121.stdout:1/629: dwrite d0/d39/d4b/f92 [0,4194304] 0 2024-02-15T18:35:39.728 INFO:tasks.workunit.client.1.smithi121.stdout:1/630: read d0/f21 [7294170,79337] 0 2024-02-15T18:35:39.733 INFO:tasks.workunit.client.1.smithi121.stdout:1/631: unlink d0/d6/d7/d72/c9e 0 2024-02-15T18:35:39.748 INFO:tasks.workunit.client.1.smithi121.stdout:4/603: dwrite d5/d31/d25/f37 [0,4194304] 0 2024-02-15T18:35:39.749 INFO:tasks.workunit.client.1.smithi121.stdout:4/604: stat d5/d31/d25/c90 0 2024-02-15T18:35:39.749 INFO:tasks.workunit.client.1.smithi121.stdout:4/605: stat d5/d31/d3d/d4a/d7e/f7f 0 2024-02-15T18:35:39.749 INFO:tasks.workunit.client.1.smithi121.stdout:4/606: dread - d5/d31/d3d/d4f/d8a/fb5 zero size 2024-02-15T18:35:39.755 INFO:tasks.workunit.client.1.smithi121.stdout:1/632: dwrite d0/d6/d7/d9/d14/d48/f78 [0,4194304] 0 2024-02-15T18:35:39.756 INFO:tasks.workunit.client.1.smithi121.stdout:4/607: rmdir d5 39 2024-02-15T18:35:39.756 INFO:tasks.workunit.client.1.smithi121.stdout:1/633: truncate d0/d39/fb3 2392317 0 2024-02-15T18:35:39.756 INFO:tasks.workunit.client.1.smithi121.stdout:1/634: rename d0/d6/d7 to d0/d6/d7/d72/dd3 22 2024-02-15T18:35:39.760 INFO:tasks.workunit.client.1.smithi121.stdout:1/635: rename d0/f73 to d0/d39/d7d/dc0/fd4 0 2024-02-15T18:35:39.762 INFO:tasks.workunit.client.1.smithi121.stdout:1/636: rename d0/d6/f1a to d0/d6/d7/d9/d14/db6/fd5 0 2024-02-15T18:35:39.764 INFO:tasks.workunit.client.1.smithi121.stdout:1/637: getdents d0/d6/d7/d9/d14/d3e/d88/daf/db4 0 2024-02-15T18:35:39.766 INFO:tasks.workunit.client.1.smithi121.stdout:1/638: creat d0/d39/d7d/dc0/fd6 x:0 0 0 2024-02-15T18:35:39.768 INFO:tasks.workunit.client.1.smithi121.stdout:1/639: creat d0/d6/d7/d50/fd7 x:0 0 0 2024-02-15T18:35:39.768 INFO:tasks.workunit.client.1.smithi121.stdout:1/640: write d0/d6/d7/d72/f8e [337800,125432] 0 2024-02-15T18:35:39.768 INFO:tasks.workunit.client.1.smithi121.stdout:1/641: chown d0/d6/d7/d9/d14/d3e/d88/daf/db4 27013 1 2024-02-15T18:35:39.770 INFO:tasks.workunit.client.1.smithi121.stdout:1/642: rename d0/d39/d4b/cc1 to d0/d6/d7/d9/d14/d3e/d88/da6/cd8 0 2024-02-15T18:35:39.772 INFO:tasks.workunit.client.1.smithi121.stdout:1/643: mkdir d0/d6/d7/d72/dd9 0 2024-02-15T18:35:39.773 INFO:tasks.workunit.client.1.smithi121.stdout:1/644: symlink d0/d6/d7/d9/d14/d3e/lda 0 2024-02-15T18:35:39.773 INFO:tasks.workunit.client.1.smithi121.stdout:1/645: fsync d0/d6/d7/d9/f23 0 2024-02-15T18:35:39.776 INFO:tasks.workunit.client.1.smithi121.stdout:1/646: symlink d0/d39/d7d/dc0/d9d/ldb 0 2024-02-15T18:35:39.776 INFO:tasks.workunit.client.1.smithi121.stdout:1/647: fsync d0/d6/d7/d9/d51/d5f/f62 0 2024-02-15T18:35:39.790 INFO:tasks.workunit.client.1.smithi121.stdout:1/648: rename d0/d39/d4b/f6d to d0/d6/d7/d9/fdc 0 2024-02-15T18:35:39.792 INFO:tasks.workunit.client.1.smithi121.stdout:1/649: symlink d0/d39/d4b/dac/ldd 0 2024-02-15T18:35:39.792 INFO:tasks.workunit.client.1.smithi121.stdout:1/650: stat d0/d6/d7/d9/d14/d48/fca 0 2024-02-15T18:35:39.792 INFO:tasks.workunit.client.1.smithi121.stdout:1/651: write d0/d39/d7d/dc0/d9d/fd1 [221327,80303] 0 2024-02-15T18:35:39.794 INFO:tasks.workunit.client.1.smithi121.stdout:1/652: unlink d0/d6/d7/d9/d14/d3e/d88/fc6 0 2024-02-15T18:35:39.807 INFO:tasks.workunit.client.1.smithi121.stdout:1/653: dwrite d0/f9c [4194304,4194304] 0 2024-02-15T18:35:39.824 INFO:tasks.workunit.client.1.smithi121.stdout:1/654: dwrite d0/d6/d7/d50/d7e/fc9 [0,4194304] 0 2024-02-15T18:35:39.826 INFO:tasks.workunit.client.1.smithi121.stdout:1/655: creat d0/d39/d4b/fde x:0 0 0 2024-02-15T18:35:39.836 INFO:tasks.workunit.client.1.smithi121.stdout:1/656: link d0/d39/d4b/f61 d0/d6/fdf 0 2024-02-15T18:35:39.836 INFO:tasks.workunit.client.1.smithi121.stdout:1/657: stat d0/d6/d7/d9/d14/d3e/d88/daf 0 2024-02-15T18:35:39.836 INFO:tasks.workunit.client.1.smithi121.stdout:1/658: write d0/d39/d7d/dc0/d9d/fce [1014243,38717] 0 2024-02-15T18:35:39.837 INFO:tasks.workunit.client.1.smithi121.stdout:1/659: readlink d0/d6/d7/l8d 0 2024-02-15T18:35:39.856 INFO:tasks.workunit.client.1.smithi121.stdout:1/660: dwrite d0/d39/f68 [0,4194304] 0 2024-02-15T18:35:39.863 INFO:tasks.workunit.client.1.smithi121.stdout:1/661: mknod d0/d6/d7/d9/d14/d3e/d88/da6/ce0 0 2024-02-15T18:35:39.870 INFO:tasks.workunit.client.1.smithi121.stdout:1/662: creat d0/d6/d7/d9/d14/db6/fe1 x:0 0 0 2024-02-15T18:35:39.872 INFO:tasks.workunit.client.1.smithi121.stdout:1/663: link d0/d6/d7/d9/fa4 d0/d6/fe2 0 2024-02-15T18:35:39.872 INFO:tasks.workunit.client.1.smithi121.stdout:1/664: truncate d0/d6/d7/d9/d14/f6b 179037 0 2024-02-15T18:35:39.872 INFO:tasks.workunit.client.1.smithi121.stdout:1/665: fdatasync d0/d6/d7/fc4 0 2024-02-15T18:35:39.874 INFO:tasks.workunit.client.1.smithi121.stdout:1/666: mknod d0/d6/d7/d9/d14/d3e/d88/daf/db4/ce3 0 2024-02-15T18:35:39.876 INFO:tasks.workunit.client.1.smithi121.stdout:1/667: rename d0/d6/d7/d9/d14/l7a to d0/d6/d7/d50/le4 0 2024-02-15T18:35:39.877 INFO:tasks.workunit.client.1.smithi121.stdout:1/668: creat d0/d39/d7d/dc0/fe5 x:0 0 0 2024-02-15T18:35:39.878 INFO:tasks.workunit.client.1.smithi121.stdout:1/669: truncate d0/d6/d7/f94 954302 0 2024-02-15T18:35:39.884 INFO:tasks.workunit.client.1.smithi121.stdout:2/364: sync 2024-02-15T18:35:39.884 INFO:tasks.workunit.client.1.smithi121.stdout:5/758: sync 2024-02-15T18:35:39.884 INFO:tasks.workunit.client.1.smithi121.stdout:9/680: sync 2024-02-15T18:35:39.884 INFO:tasks.workunit.client.1.smithi121.stdout:2/365: dread - d5/df/f39 zero size 2024-02-15T18:35:39.888 INFO:tasks.workunit.client.1.smithi121.stdout:5/759: truncate d0/d16/d34/da4/dcc/fce 1611260 0 2024-02-15T18:35:39.888 INFO:tasks.workunit.client.1.smithi121.stdout:5/760: write d0/d2/f8e [859202,4559] 0 2024-02-15T18:35:39.889 INFO:tasks.workunit.client.1.smithi121.stdout:9/681: mkdir d7/d26/def 0 2024-02-15T18:35:39.891 INFO:tasks.workunit.client.1.smithi121.stdout:9/682: creat d7/d8/d40/daf/ff0 x:0 0 0 2024-02-15T18:35:39.891 INFO:tasks.workunit.client.1.smithi121.stdout:9/683: chown d7/d8/d40/d4b/f7a 62455914 1 2024-02-15T18:35:39.894 INFO:tasks.workunit.client.1.smithi121.stdout:9/684: rename d7/d3a/d8e/d9e/fe7 to d7/d8/d25/d30/d33/d79/ff1 0 2024-02-15T18:35:39.895 INFO:tasks.workunit.client.1.smithi121.stdout:9/685: creat d7/d8/d25/d30/d33/d3c/dd0/ff2 x:0 0 0 2024-02-15T18:35:39.895 INFO:tasks.workunit.client.1.smithi121.stdout:9/686: write d7/d8/d25/d30/f80 [348436,13070] 0 2024-02-15T18:35:39.902 INFO:tasks.workunit.client.1.smithi121.stdout:5/761: dread d0/d2/d70/d88/f96 [0,4194304] 0 2024-02-15T18:35:39.910 INFO:tasks.workunit.client.1.smithi121.stdout:1/670: dwrite d0/d6/d7/d9/d51/f6c [0,4194304] 0 2024-02-15T18:35:39.911 INFO:tasks.workunit.client.1.smithi121.stdout:2/366: dwrite d5/df/d27/f21 [0,4194304] 0 2024-02-15T18:35:39.911 INFO:tasks.workunit.client.1.smithi121.stdout:2/367: stat d5/df/d27/d61 0 2024-02-15T18:35:39.912 INFO:tasks.workunit.client.1.smithi121.stdout:9/687: dwrite d7/d8/d40/f70 [0,4194304] 0 2024-02-15T18:35:39.915 INFO:tasks.workunit.client.1.smithi121.stdout:2/368: rmdir d5/df/d27/d61 39 2024-02-15T18:35:39.917 INFO:tasks.workunit.client.1.smithi121.stdout:1/671: truncate d0/d6/f6e 2709597 0 2024-02-15T18:35:39.919 INFO:tasks.workunit.client.1.smithi121.stdout:2/369: creat d5/df/d27/d61/f6b x:0 0 0 2024-02-15T18:35:39.920 INFO:tasks.workunit.client.1.smithi121.stdout:1/672: mkdir d0/d6/d7/d9/d14/d3e/d88/da6/de6 0 2024-02-15T18:35:39.921 INFO:tasks.workunit.client.1.smithi121.stdout:5/762: dwrite d0/d16/d34/d8f/ff8 [0,4194304] 0 2024-02-15T18:35:39.921 INFO:tasks.workunit.client.1.smithi121.stdout:2/370: unlink d5/f8 0 2024-02-15T18:35:39.924 INFO:tasks.workunit.client.1.smithi121.stdout:5/763: symlink d0/d2/d1e/d4e/d7d/d40/lfb 0 2024-02-15T18:35:39.928 INFO:tasks.workunit.client.1.smithi121.stdout:2/371: creat d5/df/f6c x:0 0 0 2024-02-15T18:35:39.928 INFO:tasks.workunit.client.1.smithi121.stdout:2/372: fdatasync d5/fd 0 2024-02-15T18:35:39.928 INFO:tasks.workunit.client.1.smithi121.stdout:2/373: chown d5/df/d10/f11 1311633222 1 2024-02-15T18:35:39.928 INFO:tasks.workunit.client.1.smithi121.stdout:2/374: fsync d5/df/d10/d4c/f50 0 2024-02-15T18:35:39.928 INFO:tasks.workunit.client.1.smithi121.stdout:2/375: chown d5/df/d10/d4c/f50 102426 1 2024-02-15T18:35:39.931 INFO:tasks.workunit.client.1.smithi121.stdout:5/764: rename d0/d2/d1e/f42 to d0/d2/d1e/d4e/d7d/d9d/d4c/daa/ffc 0 2024-02-15T18:35:39.932 INFO:tasks.workunit.client.1.smithi121.stdout:2/376: truncate d5/fc 156820 0 2024-02-15T18:35:39.932 INFO:tasks.workunit.client.1.smithi121.stdout:2/377: fsync d5/df/f1b 0 2024-02-15T18:35:39.932 INFO:tasks.workunit.client.1.smithi121.stdout:2/378: readlink d5/d49/d4d/l2e 0 2024-02-15T18:35:39.933 INFO:tasks.workunit.client.1.smithi121.stdout:2/379: write d5/df/d54/f5d [613977,41227] 0 2024-02-15T18:35:39.933 INFO:tasks.workunit.client.1.smithi121.stdout:2/380: readlink d5/df/l12 0 2024-02-15T18:35:39.933 INFO:tasks.workunit.client.1.smithi121.stdout:2/381: fsync d5/df/d3f/f69 0 2024-02-15T18:35:39.933 INFO:tasks.workunit.client.1.smithi121.stdout:5/765: creat d0/d2/d1e/d4e/d7d/d47/d5a/db3/ffd x:0 0 0 2024-02-15T18:35:39.935 INFO:tasks.workunit.client.1.smithi121.stdout:2/382: creat d5/df/d10/f6d x:0 0 0 2024-02-15T18:35:39.935 INFO:tasks.workunit.client.1.smithi121.stdout:5/766: write d0/d2/d1e/d4e/d7d/d9d/d4c/f8b [7939704,115898] 0 2024-02-15T18:35:39.943 INFO:tasks.workunit.client.1.smithi121.stdout:2/383: dread d5/df/d54/f5d [0,4194304] 0 2024-02-15T18:35:39.943 INFO:tasks.workunit.client.1.smithi121.stdout:2/384: write d5/d49/d4d/d22/f4f [941178,3047] 0 2024-02-15T18:35:39.944 INFO:tasks.workunit.client.1.smithi121.stdout:2/385: symlink d5/df/d27/d3e/d6a/l6e 0 2024-02-15T18:35:39.944 INFO:tasks.workunit.client.1.smithi121.stdout:2/386: truncate d5/df/d27/d3e/f53 636253 0 2024-02-15T18:35:39.947 INFO:tasks.workunit.client.1.smithi121.stdout:2/387: read d5/d49/d4d/d22/f42 [3574433,44947] 0 2024-02-15T18:35:39.948 INFO:tasks.workunit.client.1.smithi121.stdout:2/388: read d5/df/d27/d3a/f43 [285757,9004] 0 2024-02-15T18:35:39.951 INFO:tasks.workunit.client.1.smithi121.stdout:5/767: dwrite d0/d2/d70/fc0 [0,4194304] 0 2024-02-15T18:35:39.954 INFO:tasks.workunit.client.1.smithi121.stdout:5/768: rename d0/d2/d1e/d4e/d7d/d40/l99 to d0/d2/d70/d88/lfe 0 2024-02-15T18:35:39.962 INFO:tasks.workunit.client.1.smithi121.stdout:2/389: dwrite d5/d49/d4d/d22/f44 [0,4194304] 0 2024-02-15T18:35:39.966 INFO:tasks.workunit.client.1.smithi121.stdout:5/769: truncate d0/d2/d1e/d4e/d7d/d9d/d4c/f8b 2431869 0 2024-02-15T18:35:39.968 INFO:tasks.workunit.client.1.smithi121.stdout:5/770: mknod d0/d2/d1e/d4e/d7d/d47/d5a/dbc/cff 0 2024-02-15T18:35:39.981 INFO:tasks.workunit.client.1.smithi121.stdout:1/673: creat d0/d6/d7/d9/d14/db6/fe7 x:0 0 0 2024-02-15T18:35:39.984 INFO:tasks.workunit.client.1.smithi121.stdout:1/674: truncate d0/d6/d7/d9/f35 3676893 0 2024-02-15T18:35:39.984 INFO:tasks.workunit.client.1.smithi121.stdout:1/675: chown d0/d6/d7/d9/d14/d3e/d88/daf/db4/cc5 52353899 1 2024-02-15T18:35:39.987 INFO:tasks.workunit.client.1.smithi121.stdout:5/771: dwrite d0/fb [8388608,4194304] 0 2024-02-15T18:35:39.987 INFO:tasks.workunit.client.1.smithi121.stdout:5/772: chown d0/d2/d1e/d4e/d7d/d39/lac 530017 1 2024-02-15T18:35:39.988 INFO:tasks.workunit.client.1.smithi121.stdout:5/773: write d0/d2/d1e/d4e/d7d/d9d/f59 [6007930,95698] 0 2024-02-15T18:35:40.009 INFO:tasks.workunit.client.1.smithi121.stdout:1/676: dwrite d0/f9c [4194304,4194304] 0 2024-02-15T18:35:40.009 INFO:tasks.workunit.client.1.smithi121.stdout:5/774: dwrite d0/d2/f8e [0,4194304] 0 2024-02-15T18:35:40.014 INFO:tasks.workunit.client.1.smithi121.stdout:1/677: mkdir d0/d6/d7/d9/d14/d3e/d88/daf/db4/de8 0 2024-02-15T18:35:40.014 INFO:tasks.workunit.client.1.smithi121.stdout:1/678: chown d0/d39 2648808 1 2024-02-15T18:35:40.016 INFO:tasks.workunit.client.1.smithi121.stdout:1/679: getdents d0/d39/d7d/dc0/d77 0 2024-02-15T18:35:40.028 INFO:tasks.workunit.client.1.smithi121.stdout:0/689: sync 2024-02-15T18:35:40.028 INFO:tasks.workunit.client.1.smithi121.stdout:8/604: sync 2024-02-15T18:35:40.028 INFO:tasks.workunit.client.1.smithi121.stdout:7/440: sync 2024-02-15T18:35:40.028 INFO:tasks.workunit.client.1.smithi121.stdout:6/652: sync 2024-02-15T18:35:40.028 INFO:tasks.workunit.client.1.smithi121.stdout:3/478: sync 2024-02-15T18:35:40.028 INFO:tasks.workunit.client.1.smithi121.stdout:3/479: fdatasync d2/d1a/d22/f42 0 2024-02-15T18:35:40.028 INFO:tasks.workunit.client.1.smithi121.stdout:3/480: chown d2/d1a/l1e 7655934 1 2024-02-15T18:35:40.041 INFO:tasks.workunit.client.1.smithi121.stdout:1/680: dwrite d0/d6/d7/d9/d14/f8a [0,4194304] 0 2024-02-15T18:35:40.042 INFO:tasks.workunit.client.1.smithi121.stdout:1/681: getdents d0/d6/d7/d9/d14/d3e/d88/daf/db4 0 2024-02-15T18:35:40.042 INFO:tasks.workunit.client.1.smithi121.stdout:1/682: chown d0/fb 154298852 1 2024-02-15T18:35:40.060 INFO:tasks.workunit.client.1.smithi121.stdout:8/605: dwrite d0/d7/de/f1c [0,4194304] 0 2024-02-15T18:35:40.063 INFO:tasks.workunit.client.1.smithi121.stdout:1/683: dread d0/d39/fb3 [0,4194304] 0 2024-02-15T18:35:40.063 INFO:tasks.workunit.client.1.smithi121.stdout:1/684: fdatasync d0/d6/d7/d50/fd7 0 2024-02-15T18:35:40.063 INFO:tasks.workunit.client.1.smithi121.stdout:1/685: stat d0/d6/d7/d9/d51/d5f 0 2024-02-15T18:35:40.064 INFO:tasks.workunit.client.1.smithi121.stdout:5/775: link d0/l12 d0/d2/d70/d88/l100 0 2024-02-15T18:35:40.065 INFO:tasks.workunit.client.1.smithi121.stdout:1/686: link d0/c75 d0/d6/d7/d9/d51/ce9 0 2024-02-15T18:35:40.065 INFO:tasks.workunit.client.1.smithi121.stdout:1/687: chown d0/lab 52415888 1 2024-02-15T18:35:40.066 INFO:tasks.workunit.client.1.smithi121.stdout:3/481: dwrite d2/d1a/d27/d7c/d25/d31/d58/f61 [0,4194304] 0 2024-02-15T18:35:40.066 INFO:tasks.workunit.client.1.smithi121.stdout:3/482: write d2/d5a/f5f [654923,40723] 0 2024-02-15T18:35:40.068 INFO:tasks.workunit.client.1.smithi121.stdout:1/688: mknod d0/d6/d7/d9/cea 0 2024-02-15T18:35:40.071 INFO:tasks.workunit.client.1.smithi121.stdout:7/441: dwrite d0/d38/d3f/d43/f8f [0,4194304] 0 2024-02-15T18:35:40.071 INFO:tasks.workunit.client.1.smithi121.stdout:1/689: symlink d0/d6/d7/d9/leb 0 2024-02-15T18:35:40.074 INFO:tasks.workunit.client.1.smithi121.stdout:6/653: creat d7/dc/d34/d42/d62/dac/fe5 x:0 0 0 2024-02-15T18:35:40.077 INFO:tasks.workunit.client.1.smithi121.stdout:6/654: write d7/dc/d34/d42/d62/d63/f7a [3190931,27447] 0 2024-02-15T18:35:40.077 INFO:tasks.workunit.client.1.smithi121.stdout:6/655: chown d7/dae/cbf 350622062 1 2024-02-15T18:35:40.079 INFO:tasks.workunit.client.1.smithi121.stdout:0/690: creat d5/d6b/dab/db4/fec x:0 0 0 2024-02-15T18:35:40.095 INFO:tasks.workunit.client.1.smithi121.stdout:8/606: creat d0/d61/d95/fd1 x:0 0 0 2024-02-15T18:35:40.099 INFO:tasks.workunit.client.1.smithi121.stdout:0/691: dwrite d5/d6b/dab/db4/d7d/d2a/d73/d8d/fc5 [0,4194304] 0 2024-02-15T18:35:40.106 INFO:tasks.workunit.client.1.smithi121.stdout:3/483: stat d2/d1a/d22/f47 0 2024-02-15T18:35:40.106 INFO:tasks.workunit.client.1.smithi121.stdout:3/484: write d2/d1a/d6b/d7d/f8e [613827,72540] 0 2024-02-15T18:35:40.106 INFO:tasks.workunit.client.1.smithi121.stdout:3/485: chown d2/d1a/d54/f59 74839836 1 2024-02-15T18:35:40.106 INFO:tasks.workunit.client.1.smithi121.stdout:1/690: rmdir d0/d39/d7d/dc0 39 2024-02-15T18:35:40.106 INFO:tasks.workunit.client.1.smithi121.stdout:1/691: write d0/d6/d7/d9/fdc [8412070,115778] 0 2024-02-15T18:35:40.106 INFO:tasks.workunit.client.1.smithi121.stdout:1/692: chown d0/d39/d7d/dc0/c2e 169951133 1 2024-02-15T18:35:40.108 INFO:tasks.workunit.client.1.smithi121.stdout:7/442: creat d0/d5/d4c/d6f/f94 x:0 0 0 2024-02-15T18:35:40.109 INFO:tasks.workunit.client.1.smithi121.stdout:1/693: read d0/f1d [754147,83836] 0 2024-02-15T18:35:40.110 INFO:tasks.workunit.client.1.smithi121.stdout:1/694: mkdir d0/d6/d7/d50/d7e/dec 0 2024-02-15T18:35:40.110 INFO:tasks.workunit.client.1.smithi121.stdout:6/656: symlink d7/dc/d34/d42/d6e/le6 0 2024-02-15T18:35:40.110 INFO:tasks.workunit.client.1.smithi121.stdout:6/657: write d7/dc/f37 [467397,26814] 0 2024-02-15T18:35:40.111 INFO:tasks.workunit.client.1.smithi121.stdout:8/607: mknod d0/d3/dc9/cd2 0 2024-02-15T18:35:40.112 INFO:tasks.workunit.client.1.smithi121.stdout:1/695: mknod d0/d6/d7/d72/dd9/ced 0 2024-02-15T18:35:40.113 INFO:tasks.workunit.client.1.smithi121.stdout:1/696: dread - d0/d39/d7d/f86 zero size 2024-02-15T18:35:40.119 INFO:tasks.workunit.client.1.smithi121.stdout:7/443: rename d0/d5/d11/l70 to d0/d5/d4c/d6f/l95 0 2024-02-15T18:35:40.119 INFO:tasks.workunit.client.1.smithi121.stdout:6/658: creat d7/dc/d34/d42/d62/d63/d8d/fe7 x:0 0 0 2024-02-15T18:35:40.120 INFO:tasks.workunit.client.1.smithi121.stdout:8/608: creat d0/d7/d36/d8c/d45/fd3 x:0 0 0 2024-02-15T18:35:40.123 INFO:tasks.workunit.client.1.smithi121.stdout:0/692: dwrite d5/d6b/dab/db4/d7d/dd/f93 [0,4194304] 0 2024-02-15T18:35:40.123 INFO:tasks.workunit.client.1.smithi121.stdout:7/444: creat d0/d38/d46/f96 x:0 0 0 2024-02-15T18:35:40.123 INFO:tasks.workunit.client.1.smithi121.stdout:7/445: fsync d0/d5/d11/d5c/f64 0 2024-02-15T18:35:40.125 INFO:tasks.workunit.client.1.smithi121.stdout:0/693: write d5/d6b/dab/db1/dbf/fd9 [3769456,66395] 0 2024-02-15T18:35:40.125 INFO:tasks.workunit.client.1.smithi121.stdout:0/694: chown d5/d6b/dab/db4/d7d/d2a/f2b 15949 1 2024-02-15T18:35:40.128 INFO:tasks.workunit.client.1.smithi121.stdout:1/697: rmdir d0/d6/d7/d72 39 2024-02-15T18:35:40.136 INFO:tasks.workunit.client.1.smithi121.stdout:1/698: truncate d0/d39/f68 1701191 0 2024-02-15T18:35:40.141 INFO:tasks.workunit.client.1.smithi121.stdout:8/609: mkdir d0/d48/dd4 0 2024-02-15T18:35:40.141 INFO:tasks.workunit.client.1.smithi121.stdout:8/610: truncate d0/d7/d36/fb9 923447 0 2024-02-15T18:35:40.142 INFO:tasks.workunit.client.1.smithi121.stdout:8/611: readlink d0/d2a/d51/l29 0 2024-02-15T18:35:40.143 INFO:tasks.workunit.client.1.smithi121.stdout:0/695: rename d5/d6b/dab/db4/d7d/d2a/d73/d8d/fde to d5/fed 0 2024-02-15T18:35:40.143 INFO:tasks.workunit.client.1.smithi121.stdout:0/696: truncate d5/d6b/dab/fad 700700 0 2024-02-15T18:35:40.144 INFO:tasks.workunit.client.1.smithi121.stdout:8/612: creat d0/d2a/fd5 x:0 0 0 2024-02-15T18:35:40.146 INFO:tasks.workunit.client.1.smithi121.stdout:0/697: creat d5/d6b/dab/db4/d7d/d2a/d73/d97/db8/fee x:0 0 0 2024-02-15T18:35:40.147 INFO:tasks.workunit.client.1.smithi121.stdout:1/699: mknod d0/d39/cee 0 2024-02-15T18:35:40.148 INFO:tasks.workunit.client.1.smithi121.stdout:8/613: unlink d0/d7/de/fbc 0 2024-02-15T18:35:40.153 INFO:tasks.workunit.client.1.smithi121.stdout:1/700: fsync d0/d6/d7/d9/fdc 0 2024-02-15T18:35:40.153 INFO:tasks.workunit.client.1.smithi121.stdout:1/701: fsync d0/d6/d7/d9/d14/d48/f78 0 2024-02-15T18:35:40.156 INFO:tasks.workunit.client.1.smithi121.stdout:0/698: dread d5/d6b/dab/db4/d7d/d2a/f2f [0,4194304] 0 2024-02-15T18:35:40.157 INFO:tasks.workunit.client.1.smithi121.stdout:0/699: creat d5/d30/fef x:0 0 0 2024-02-15T18:35:40.157 INFO:tasks.workunit.client.1.smithi121.stdout:7/446: dwrite d0/d5/d11/d1c/f1f [0,4194304] 0 2024-02-15T18:35:40.158 INFO:tasks.workunit.client.1.smithi121.stdout:0/700: write d5/d30/fe6 [2751485,32849] 0 2024-02-15T18:35:40.160 INFO:tasks.workunit.client.1.smithi121.stdout:0/701: symlink d5/d6b/dab/db4/d7d/d2a/d73/d8d/dc8/de0/lf0 0 2024-02-15T18:35:40.160 INFO:tasks.workunit.client.1.smithi121.stdout:3/486: dwrite d2/d1a/f89 [0,4194304] 0 2024-02-15T18:35:40.161 INFO:tasks.workunit.client.1.smithi121.stdout:6/659: dwrite d7/d4b/f53 [4194304,4194304] 0 2024-02-15T18:35:40.161 INFO:tasks.workunit.client.1.smithi121.stdout:7/447: write d0/d5/d11/d1c/f1f [2879719,51873] 0 2024-02-15T18:35:40.161 INFO:tasks.workunit.client.1.smithi121.stdout:7/448: chown d0/d5/f15 130128 1 2024-02-15T18:35:40.162 INFO:tasks.workunit.client.1.smithi121.stdout:1/702: creat d0/fef x:0 0 0 2024-02-15T18:35:40.162 INFO:tasks.workunit.client.1.smithi121.stdout:0/702: creat d5/d6b/dab/db4/d7d/d74/d96/de4/ff1 x:0 0 0 2024-02-15T18:35:40.166 INFO:tasks.workunit.client.1.smithi121.stdout:6/660: mkdir d7/dc/d34/d42/d62/dac/de8 0 2024-02-15T18:35:40.166 INFO:tasks.workunit.client.1.smithi121.stdout:6/661: truncate d7/dc/d34/d42/d6e/fc5 1001025 0 2024-02-15T18:35:40.171 INFO:tasks.workunit.client.1.smithi121.stdout:0/703: creat d5/d6b/dab/db4/d7d/d2a/d4e/ff2 x:0 0 0 2024-02-15T18:35:40.172 INFO:tasks.workunit.client.1.smithi121.stdout:6/662: getdents d7/dc/d2c 0 2024-02-15T18:35:40.173 INFO:tasks.workunit.client.1.smithi121.stdout:4/608: sync 2024-02-15T18:35:40.173 INFO:tasks.workunit.client.1.smithi121.stdout:6/663: mknod d7/dc/d34/d42/d62/d63/d66/ce9 0 2024-02-15T18:35:40.174 INFO:tasks.workunit.client.1.smithi121.stdout:6/664: creat d7/dc/d34/d42/d62/d63/d6f/d89/fea x:0 0 0 2024-02-15T18:35:40.175 INFO:tasks.workunit.client.1.smithi121.stdout:6/665: creat d7/d4b/d60/d6c/feb x:0 0 0 2024-02-15T18:35:40.177 INFO:tasks.workunit.client.1.smithi121.stdout:6/666: dread d7/dc/d34/d42/d6e/fc5 [0,4194304] 0 2024-02-15T18:35:40.181 INFO:tasks.workunit.client.1.smithi121.stdout:1/703: dwrite d0/d6/d7/d50/d7e/f85 [0,4194304] 0 2024-02-15T18:35:40.181 INFO:tasks.workunit.client.1.smithi121.stdout:0/704: dread d5/d6b/dab/db4/f8f [0,4194304] 0 2024-02-15T18:35:40.183 INFO:tasks.workunit.client.1.smithi121.stdout:6/667: creat d7/d4b/d59/fec x:0 0 0 2024-02-15T18:35:40.185 INFO:tasks.workunit.client.1.smithi121.stdout:1/704: mknod d0/d6/d7/d9/d51/d5f/cf0 0 2024-02-15T18:35:40.201 INFO:tasks.workunit.client.1.smithi121.stdout:1/705: mknod d0/d39/d7d/dc0/d9d/cf1 0 2024-02-15T18:35:40.235 INFO:tasks.workunit.client.1.smithi121.stdout:4/609: write d5/d9/d1d/d26/d72/fa8 [832983,44903] 0 2024-02-15T18:35:40.235 INFO:tasks.workunit.client.1.smithi121.stdout:4/610: write d5/d9/d76/d9e/f38 [4566385,59901] 0 2024-02-15T18:35:40.235 INFO:tasks.workunit.client.1.smithi121.stdout:1/706: mknod d0/d6/d7/d9/d14/cf2 0 2024-02-15T18:35:40.235 INFO:tasks.workunit.client.1.smithi121.stdout:1/707: chown d0/d6/d7/d9/d51/d5f/cf0 29056 1 2024-02-15T18:35:40.235 INFO:tasks.workunit.client.1.smithi121.stdout:1/708: chown d0/f9c 346984 1 2024-02-15T18:35:40.235 INFO:tasks.workunit.client.1.smithi121.stdout:0/705: truncate d5/d6b/dab/db4/d7d/f11 5786667 0 2024-02-15T18:35:40.235 INFO:tasks.workunit.client.1.smithi121.stdout:0/706: dread - d5/d6b/dab/db4/fec zero size 2024-02-15T18:35:40.236 INFO:tasks.workunit.client.1.smithi121.stdout:6/668: mknod d7/dc/d34/d42/d62/d63/ced 0 2024-02-15T18:35:40.238 INFO:tasks.workunit.client.1.smithi121.stdout:0/707: rmdir d5/d6b/dab/db4/d7d/d74/d96 39 2024-02-15T18:35:40.240 INFO:tasks.workunit.client.1.smithi121.stdout:6/669: creat d7/dc/d34/d42/fee x:0 0 0 2024-02-15T18:35:40.240 INFO:tasks.workunit.client.1.smithi121.stdout:0/708: rmdir d5/d6b/dab/db4/d7d/d2a 39 2024-02-15T18:35:40.242 INFO:tasks.workunit.client.1.smithi121.stdout:6/670: truncate d7/dc/d34/d42/d6e/d94/f9a 955745 0 2024-02-15T18:35:40.242 INFO:tasks.workunit.client.1.smithi121.stdout:6/671: truncate d7/dc/d34/d42/d62/d63/d8d/fa9 4899567 0 2024-02-15T18:35:40.242 INFO:tasks.workunit.client.1.smithi121.stdout:6/672: chown d7/dc/d34/d42/d62/d63/d66/l9c 8 1 2024-02-15T18:35:40.242 INFO:tasks.workunit.client.1.smithi121.stdout:6/673: stat d7/dc/d34/d42/f54 0 2024-02-15T18:35:40.242 INFO:tasks.workunit.client.1.smithi121.stdout:0/709: write d5/d6b/dab/db4/d7d/d2a/d36/f51 [1861124,82961] 0 2024-02-15T18:35:40.245 INFO:tasks.workunit.client.1.smithi121.stdout:4/611: unlink d5/d31/d3d/d4f/d8a/f97 0 2024-02-15T18:35:40.249 INFO:tasks.workunit.client.1.smithi121.stdout:5/776: creat d0/d2/d18/f101 x:0 0 0 2024-02-15T18:35:40.249 INFO:tasks.workunit.client.1.smithi121.stdout:5/777: chown d0/d2/d1e/d4e/d7d/d40/db9/cd3 0 1 2024-02-15T18:35:40.249 INFO:tasks.workunit.client.1.smithi121.stdout:5/778: fsync d0/d2/d18/f87 0 2024-02-15T18:35:40.249 INFO:tasks.workunit.client.1.smithi121.stdout:5/779: fsync d0/d16/da1/fd4 0 2024-02-15T18:35:40.250 INFO:tasks.workunit.client.1.smithi121.stdout:0/710: getdents d5/d6b/dab/db4/d7d/dd 0 2024-02-15T18:35:40.250 INFO:tasks.workunit.client.1.smithi121.stdout:5/780: read d0/d2/d70/d88/f96 [918774,76002] 0 2024-02-15T18:35:40.251 INFO:tasks.workunit.client.1.smithi121.stdout:0/711: mkdir d5/d6b/dab/db4/d7d/d2a/d73/d8d/dc8/df3 0 2024-02-15T18:35:40.252 INFO:tasks.workunit.client.1.smithi121.stdout:0/712: write d5/d9b/fbc [866001,53131] 0 2024-02-15T18:35:40.253 INFO:tasks.workunit.client.1.smithi121.stdout:5/781: creat d0/d16/f102 x:0 0 0 2024-02-15T18:35:40.253 INFO:tasks.workunit.client.1.smithi121.stdout:5/782: stat d0/d2/d1e/f8c 0 2024-02-15T18:35:40.253 INFO:tasks.workunit.client.1.smithi121.stdout:1/709: link d0/d39/d7d/cc8 d0/d6/d7/d9/d14/db6/cf3 0 2024-02-15T18:35:40.253 INFO:tasks.workunit.client.1.smithi121.stdout:1/710: fdatasync d0/d6/d7/d9/d14/d3e/f71 0 2024-02-15T18:35:40.256 INFO:tasks.workunit.client.1.smithi121.stdout:5/783: read d0/d2/faf [5293437,105168] 0 2024-02-15T18:35:40.257 INFO:tasks.workunit.client.1.smithi121.stdout:0/713: write d5/d6b/dab/db4/d7d/d2a/d73/f9a [411203,121407] 0 2024-02-15T18:35:40.260 INFO:tasks.workunit.client.1.smithi121.stdout:0/714: unlink d5/d6b/dab/db4/d7d/d2a/d36/d44/db7/fd4 0 2024-02-15T18:35:40.260 INFO:tasks.workunit.client.1.smithi121.stdout:0/715: write d5/d6b/dab/db4/d7d/d74/d96/de4/ff1 [159479,42982] 0 2024-02-15T18:35:40.263 INFO:tasks.workunit.client.1.smithi121.stdout:0/716: mkdir d5/d6b/dab/db4/d7d/d2a/d73/d97/df4 0 2024-02-15T18:35:40.263 INFO:tasks.workunit.client.1.smithi121.stdout:0/717: fdatasync d5/d6b/dab/db4/d7d/f11 0 2024-02-15T18:35:40.265 INFO:tasks.workunit.client.1.smithi121.stdout:0/718: creat d5/d6b/dab/db4/d7d/d2a/d73/d97/ff5 x:0 0 0 2024-02-15T18:35:40.267 INFO:tasks.workunit.client.1.smithi121.stdout:1/711: write d0/d6/d7/d9/d14/fbb [3432426,59657] 0 2024-02-15T18:35:40.267 INFO:tasks.workunit.client.1.smithi121.stdout:1/712: chown d0/d39 15120455 1 2024-02-15T18:35:40.270 INFO:tasks.workunit.client.1.smithi121.stdout:1/713: link d0/d6/d7/d9/d14/d48/fca d0/d39/d7d/dc0/d77/ff4 0 2024-02-15T18:35:40.270 INFO:tasks.workunit.client.1.smithi121.stdout:1/714: chown d0/d6/d7/d9/f2d 65 1 2024-02-15T18:35:40.270 INFO:tasks.workunit.client.1.smithi121.stdout:1/715: chown d0/fb 209 1 2024-02-15T18:35:40.273 INFO:tasks.workunit.client.1.smithi121.stdout:4/612: dwrite d5/d9/d76/d9e/fb9 [0,4194304] 0 2024-02-15T18:35:40.276 INFO:tasks.workunit.client.1.smithi121.stdout:6/674: dwrite d7/dc/d34/d42/d62/f6b [0,4194304] 0 2024-02-15T18:35:40.276 INFO:tasks.workunit.client.1.smithi121.stdout:5/784: dwrite d0/d2/d18/f5b [4194304,4194304] 0 2024-02-15T18:35:40.277 INFO:tasks.workunit.client.1.smithi121.stdout:4/613: mknod d5/d9/d19/d5c/cbe 0 2024-02-15T18:35:40.278 INFO:tasks.workunit.client.1.smithi121.stdout:1/716: write d0/d6/d7/d9/d14/d48/f78 [1814634,87017] 0 2024-02-15T18:35:40.279 INFO:tasks.workunit.client.1.smithi121.stdout:6/675: unlink d7/dc/f48 0 2024-02-15T18:35:40.279 INFO:tasks.workunit.client.1.smithi121.stdout:6/676: mknod d7/cef 0 2024-02-15T18:35:40.285 INFO:tasks.workunit.client.1.smithi121.stdout:5/785: unlink d0/d2/d1e/d4e/d7d/d39/f53 0 2024-02-15T18:35:40.286 INFO:tasks.workunit.client.1.smithi121.stdout:4/614: symlink d5/d9/d76/d9e/lbf 0 2024-02-15T18:35:40.288 INFO:tasks.workunit.client.1.smithi121.stdout:5/786: rename d0/d2/d1e/d4e/d7d/d47/f84 to d0/d2/d1e/d4e/d7d/d47/d5a/da7/f103 0 2024-02-15T18:35:40.290 INFO:tasks.workunit.client.1.smithi121.stdout:0/719: dread d5/d6b/dab/db4/d7d/d2a/d73/f9a [0,4194304] 0 2024-02-15T18:35:40.290 INFO:tasks.workunit.client.1.smithi121.stdout:0/720: readlink d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/l72 0 2024-02-15T18:35:40.291 INFO:tasks.workunit.client.1.smithi121.stdout:0/721: creat d5/d30/ff6 x:0 0 0 2024-02-15T18:35:40.291 INFO:tasks.workunit.client.1.smithi121.stdout:0/722: chown d5/d6b/dab/db4/d7d/d2a/d73/f9a 1666147781 1 2024-02-15T18:35:40.295 INFO:tasks.workunit.client.1.smithi121.stdout:0/723: link d5/d6b/dab/db4/d7d/dd/f68 d5/d6b/dab/ff7 0 2024-02-15T18:35:40.301 INFO:tasks.workunit.client.1.smithi121.stdout:6/677: dwrite d7/dc/d34/d42/d62/f83 [0,4194304] 0 2024-02-15T18:35:40.301 INFO:tasks.workunit.client.1.smithi121.stdout:6/678: read d7/dc/d34/fd1 [691810,45494] 0 2024-02-15T18:35:40.301 INFO:tasks.workunit.client.1.smithi121.stdout:6/679: fsync d7/dc/d2c/d36/f49 0 2024-02-15T18:35:40.302 INFO:tasks.workunit.client.1.smithi121.stdout:1/717: dwrite d0/d6/d7/f94 [0,4194304] 0 2024-02-15T18:35:40.302 INFO:tasks.workunit.client.1.smithi121.stdout:1/718: truncate d0/d39/d7d/dc0/d77/ff4 1042241 0 2024-02-15T18:35:40.305 INFO:tasks.workunit.client.1.smithi121.stdout:6/680: mkdir d7/dc/d34/d42/d62/d63/d6f/db9/df0 0 2024-02-15T18:35:40.305 INFO:tasks.workunit.client.1.smithi121.stdout:6/681: truncate d7/dc/d34/d42/d62/d63/d6f/d89/fcc 647442 0 2024-02-15T18:35:40.305 INFO:tasks.workunit.client.1.smithi121.stdout:4/615: rename d5/d31/d3d/c40 to d5/d9/cc0 0 2024-02-15T18:35:40.306 INFO:tasks.workunit.client.1.smithi121.stdout:6/682: mkdir d7/dc/d34/d42/d62/d63/d6f/d89/df1 0 2024-02-15T18:35:40.307 INFO:tasks.workunit.client.1.smithi121.stdout:6/683: mknod d7/dc/d34/d42/d62/cf2 0 2024-02-15T18:35:40.308 INFO:tasks.workunit.client.1.smithi121.stdout:6/684: truncate d7/d4b/d59/f72 3384376 0 2024-02-15T18:35:40.309 INFO:tasks.workunit.client.1.smithi121.stdout:6/685: stat d7/dc/d34/d42/c64 0 2024-02-15T18:35:40.309 INFO:tasks.workunit.client.1.smithi121.stdout:6/686: write d7/dc/d34/d42/d62/d63/d6f/f88 [4863441,42702] 0 2024-02-15T18:35:40.315 INFO:tasks.workunit.client.1.smithi121.stdout:1/719: mkdir d0/d39/d4b/dac/df5 0 2024-02-15T18:35:40.316 INFO:tasks.workunit.client.1.smithi121.stdout:4/616: mknod d5/d31/d3d/d4f/cc1 0 2024-02-15T18:35:40.317 INFO:tasks.workunit.client.1.smithi121.stdout:1/720: truncate d0/d6/d7/d50/d7e/fc9 1719696 0 2024-02-15T18:35:40.317 INFO:tasks.workunit.client.1.smithi121.stdout:4/617: symlink d5/d31/lc2 0 2024-02-15T18:35:40.319 INFO:tasks.workunit.client.1.smithi121.stdout:0/724: dwrite d5/d6b/fcb [0,4194304] 0 2024-02-15T18:35:40.320 INFO:tasks.workunit.client.1.smithi121.stdout:5/787: dwrite d0/d16/d34/d8f/ff8 [0,4194304] 0 2024-02-15T18:35:40.322 INFO:tasks.workunit.client.1.smithi121.stdout:0/725: mknod d5/d6b/dab/db4/d7d/dd/cf8 0 2024-02-15T18:35:40.324 INFO:tasks.workunit.client.1.smithi121.stdout:6/687: dwrite d7/d4b/f8f [0,4194304] 0 2024-02-15T18:35:40.324 INFO:tasks.workunit.client.1.smithi121.stdout:0/726: rename d5/d6b/dab/db4/d7d/d2a/d73/fcd to d5/d6b/dab/db4/ff9 0 2024-02-15T18:35:40.325 INFO:tasks.workunit.client.1.smithi121.stdout:0/727: read d5/d6b/dab/db4/d7d/d74/d96/de4/ff1 [128199,61168] 0 2024-02-15T18:35:40.331 INFO:tasks.workunit.client.1.smithi121.stdout:6/688: creat d7/dc/ff3 x:0 0 0 2024-02-15T18:35:40.332 INFO:tasks.workunit.client.1.smithi121.stdout:0/728: creat d5/d6b/dab/db4/d7d/d74/d96/de3/ffa x:0 0 0 2024-02-15T18:35:40.334 INFO:tasks.workunit.client.1.smithi121.stdout:0/729: creat d5/d6b/dab/db4/d7d/d2a/d73/d97/ffb x:0 0 0 2024-02-15T18:35:40.337 INFO:tasks.workunit.client.1.smithi121.stdout:6/689: dread d7/dc/d2c/d36/f56 [0,4194304] 0 2024-02-15T18:35:40.341 INFO:tasks.workunit.client.1.smithi121.stdout:4/618: dread d5/d9/d76/d9e/d14/f17 [0,4194304] 0 2024-02-15T18:35:40.345 INFO:tasks.workunit.client.1.smithi121.stdout:1/721: dwrite d0/d6/d7/d9/d14/d3e/fb5 [0,4194304] 0 2024-02-15T18:35:40.350 INFO:tasks.workunit.client.1.smithi121.stdout:1/722: rename d0/d6/d7/d9/d14/d3e/d88/da6/ce0 to d0/d39/d4b/cf6 0 2024-02-15T18:35:40.351 INFO:tasks.workunit.client.1.smithi121.stdout:6/690: dread d7/f18 [0,4194304] 0 2024-02-15T18:35:40.351 INFO:tasks.workunit.client.1.smithi121.stdout:6/691: truncate d7/dc/d34/d42/d6e/d91/fc7 43555 0 2024-02-15T18:35:40.351 INFO:tasks.workunit.client.1.smithi121.stdout:1/723: mkdir d0/d6/d7/d9/d14/d3e/d88/da6/df7 0 2024-02-15T18:35:40.352 INFO:tasks.workunit.client.1.smithi121.stdout:6/692: symlink d7/d4b/d60/d6c/dd5/lf4 0 2024-02-15T18:35:40.352 INFO:tasks.workunit.client.1.smithi121.stdout:6/693: dread - d7/d4b/d60/d6c/fa6 zero size 2024-02-15T18:35:40.353 INFO:tasks.workunit.client.1.smithi121.stdout:1/724: creat d0/d6/d7/d9/d14/db6/ff8 x:0 0 0 2024-02-15T18:35:40.354 INFO:tasks.workunit.client.1.smithi121.stdout:6/694: mknod d7/dc/d34/d42/d62/d63/d6f/d89/df1/cf5 0 2024-02-15T18:35:40.355 INFO:tasks.workunit.client.1.smithi121.stdout:6/695: creat d7/dc/d34/d42/d6e/d94/ff6 x:0 0 0 2024-02-15T18:35:40.356 INFO:tasks.workunit.client.1.smithi121.stdout:6/696: fsync d7/dc/d34/d42/d62/f83 0 2024-02-15T18:35:40.358 INFO:tasks.workunit.client.1.smithi121.stdout:5/788: dwrite d0/d16/f81 [0,4194304] 0 2024-02-15T18:35:40.360 INFO:tasks.workunit.client.1.smithi121.stdout:5/789: rmdir d0/d16/d7a/db5/dd6 0 2024-02-15T18:35:40.365 INFO:tasks.workunit.client.1.smithi121.stdout:0/730: dwrite d5/d6b/dab/db4/d7d/d2a/f2f [4194304,4194304] 0 2024-02-15T18:35:40.366 INFO:tasks.workunit.client.1.smithi121.stdout:0/731: rename d5/fed to d5/d6b/dab/db4/d7d/d2a/d36/ffc 0 2024-02-15T18:35:40.370 INFO:tasks.workunit.client.1.smithi121.stdout:0/732: symlink d5/d6b/dab/db4/d7d/d2a/lfd 0 2024-02-15T18:35:40.371 INFO:tasks.workunit.client.1.smithi121.stdout:4/619: dwrite d5/d31/d3d/d4f/d51/f5d [0,4194304] 0 2024-02-15T18:35:40.375 INFO:tasks.workunit.client.1.smithi121.stdout:0/733: link d5/d6b/dab/db4/d7d/l58 d5/d6b/dab/db4/d7d/d2a/d36/d44/lfe 0 2024-02-15T18:35:40.375 INFO:tasks.workunit.client.1.smithi121.stdout:0/734: chown d5/fa4 182701824 1 2024-02-15T18:35:40.376 INFO:tasks.workunit.client.1.smithi121.stdout:0/735: creat d5/d6b/dab/db4/d7d/fff x:0 0 0 2024-02-15T18:35:40.378 INFO:tasks.workunit.client.1.smithi121.stdout:0/736: mkdir d5/d6b/dab/db4/d7d/d2a/d4e/d100 0 2024-02-15T18:35:40.379 INFO:tasks.workunit.client.1.smithi121.stdout:0/737: creat d5/d30/f101 x:0 0 0 2024-02-15T18:35:40.379 INFO:tasks.workunit.client.1.smithi121.stdout:0/738: truncate d5/d6b/dab/db4/d7d/d2a/d73/d97/ff5 824078 0 2024-02-15T18:35:40.379 INFO:tasks.workunit.client.1.smithi121.stdout:1/725: write d0/d39/fb3 [2657852,67279] 0 2024-02-15T18:35:40.380 INFO:tasks.workunit.client.1.smithi121.stdout:0/739: creat d5/d6b/dab/db4/d7d/d2a/f102 x:0 0 0 2024-02-15T18:35:40.387 INFO:tasks.workunit.client.1.smithi121.stdout:4/620: mkdir d5/d9/d19/d4c/dc3 0 2024-02-15T18:35:40.388 INFO:tasks.workunit.client.1.smithi121.stdout:1/726: unlink d0/l4 0 2024-02-15T18:35:40.390 INFO:tasks.workunit.client.1.smithi121.stdout:4/621: symlink d5/d9/d76/lc4 0 2024-02-15T18:35:40.393 INFO:tasks.workunit.client.1.smithi121.stdout:4/622: unlink d5/d9/d19/d5c/f69 0 2024-02-15T18:35:40.397 INFO:tasks.workunit.client.1.smithi121.stdout:6/697: dwrite d7/d4b/d60/d73/fb2 [0,4194304] 0 2024-02-15T18:35:40.398 INFO:tasks.workunit.client.1.smithi121.stdout:6/698: read d7/dc/d2c/d36/f56 [83658,45717] 0 2024-02-15T18:35:40.399 INFO:tasks.workunit.client.1.smithi121.stdout:0/740: dwrite d5/d6b/fcb [0,4194304] 0 2024-02-15T18:35:40.400 INFO:tasks.workunit.client.1.smithi121.stdout:6/699: creat d7/d4b/d60/ff7 x:0 0 0 2024-02-15T18:35:40.401 INFO:tasks.workunit.client.1.smithi121.stdout:6/700: mkdir d7/dc/d55/df8 0 2024-02-15T18:35:40.403 INFO:tasks.workunit.client.1.smithi121.stdout:5/790: dwrite d0/d2/d1e/d4e/d7d/d9d/d4c/daa/fd1 [0,4194304] 0 2024-02-15T18:35:40.403 INFO:tasks.workunit.client.1.smithi121.stdout:0/741: write d5/d6b/dab/db4/d7d/dd/f7f [1934783,33886] 0 2024-02-15T18:35:40.407 INFO:tasks.workunit.client.1.smithi121.stdout:0/742: link d5/d6b/dab/db4/d7d/d2a/d73/d8d/fae d5/d6b/dab/db4/d7d/d2a/d73/d81/ddb/deb/f103 0 2024-02-15T18:35:40.407 INFO:tasks.workunit.client.1.smithi121.stdout:0/743: fdatasync d5/d30/f4f 0 2024-02-15T18:35:40.435 INFO:tasks.workunit.client.1.smithi121.stdout:4/623: dwrite d5/d9/d76/d9e/f3c [0,4194304] 0 2024-02-15T18:35:40.435 INFO:tasks.workunit.client.1.smithi121.stdout:4/624: write d5/f6 [2901068,54894] 0 2024-02-15T18:35:40.437 INFO:tasks.workunit.client.1.smithi121.stdout:4/625: mknod d5/d9/d76/cc5 0 2024-02-15T18:35:40.437 INFO:tasks.workunit.client.1.smithi121.stdout:4/626: chown d5/d31/d3d/d4f/d8a/faa 1984 1 2024-02-15T18:35:40.439 INFO:tasks.workunit.client.1.smithi121.stdout:4/627: creat d5/d31/d3d/d4f/d51/d71/fc6 x:0 0 0 2024-02-15T18:35:40.439 INFO:tasks.workunit.client.1.smithi121.stdout:4/628: write d5/d9/d76/d9e/fb9 [4955204,71944] 0 2024-02-15T18:35:40.440 INFO:tasks.workunit.client.1.smithi121.stdout:4/629: rmdir d5/d9/d19/d4c/d82 39 2024-02-15T18:35:40.444 INFO:tasks.workunit.client.1.smithi121.stdout:0/744: dwrite d5/d6b/dab/db4/fec [0,4194304] 0 2024-02-15T18:35:40.444 INFO:tasks.workunit.client.1.smithi121.stdout:0/745: truncate d5/d6b/dab/db4/d7d/d2a/d73/d97/ffb 752247 0 2024-02-15T18:35:40.445 INFO:tasks.workunit.client.1.smithi121.stdout:6/701: dwrite d7/dc/d34/d42/d62/d63/d6f/d89/fcc [0,4194304] 0 2024-02-15T18:35:40.445 INFO:tasks.workunit.client.1.smithi121.stdout:1/727: dwrite d0/d6/d7/d9/d14/f6b [0,4194304] 0 2024-02-15T18:35:40.447 INFO:tasks.workunit.client.1.smithi121.stdout:0/746: rmdir d5/d6b/dab/db4/d7d/d2a/d36/d44/db7 39 2024-02-15T18:35:40.448 INFO:tasks.workunit.client.1.smithi121.stdout:0/747: write d5/d6b/dab/db4/d7d/dd/fd0 [772409,64923] 0 2024-02-15T18:35:40.448 INFO:tasks.workunit.client.1.smithi121.stdout:0/748: chown d5/d6b/dab/db4/d7d/d2a/c7b 3 1 2024-02-15T18:35:40.448 INFO:tasks.workunit.client.1.smithi121.stdout:5/791: dwrite d0/d16/f81 [0,4194304] 0 2024-02-15T18:35:40.456 INFO:tasks.workunit.client.1.smithi121.stdout:6/702: symlink d7/dc/d34/d42/d62/lf9 0 2024-02-15T18:35:40.461 INFO:tasks.workunit.client.1.smithi121.stdout:1/728: truncate d0/d39/d7d/dc0/fd4 2276006 0 2024-02-15T18:35:40.462 INFO:tasks.workunit.client.1.smithi121.stdout:5/792: dread d0/d2/d1e/d4e/d7d/d9d/d4c/daa/ffc [0,4194304] 0 2024-02-15T18:35:40.478 INFO:tasks.workunit.client.1.smithi121.stdout:0/749: dwrite d5/d6b/dab/db4/d7d/d2a/d73/d97/f98 [4194304,4194304] 0 2024-02-15T18:35:40.478 INFO:tasks.workunit.client.1.smithi121.stdout:4/630: dwrite d5/d9/d76/d87/db4/fae [0,4194304] 0 2024-02-15T18:35:40.480 INFO:tasks.workunit.client.1.smithi121.stdout:6/703: dwrite d7/dc/d34/fbd [0,4194304] 0 2024-02-15T18:35:40.480 INFO:tasks.workunit.client.1.smithi121.stdout:0/750: link d5/d6b/dab/db4/d7d/d2a/d4e/c70 d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/c104 0 2024-02-15T18:35:40.481 INFO:tasks.workunit.client.1.smithi121.stdout:0/751: stat d5/d6b/dab/db4/d7d/d2a/d36/d44/lfe 0 2024-02-15T18:35:40.481 INFO:tasks.workunit.client.1.smithi121.stdout:0/752: chown d5/d6b/dab/db4/d7d/dd/l4a 463 1 2024-02-15T18:35:40.481 INFO:tasks.workunit.client.1.smithi121.stdout:0/753: chown d5/d6b/dab/db4/d7d/d2a/d73/d8d/dc8/de0/lf0 367160 1 2024-02-15T18:35:40.481 INFO:tasks.workunit.client.1.smithi121.stdout:0/754: readlink d5/d30/l83 0 2024-02-15T18:35:40.483 INFO:tasks.workunit.client.1.smithi121.stdout:1/729: truncate d0/d6/d7/f94 3338981 0 2024-02-15T18:35:40.491 INFO:tasks.workunit.client.1.smithi121.stdout:0/755: dread d5/d6b/dab/db4/d7d/f24 [4194304,4194304] 0 2024-02-15T18:35:40.492 INFO:tasks.workunit.client.1.smithi121.stdout:0/756: symlink d5/d6b/dab/db4/l105 0 2024-02-15T18:35:40.493 INFO:tasks.workunit.client.1.smithi121.stdout:0/757: stat d5/l53 0 2024-02-15T18:35:40.493 INFO:tasks.workunit.client.1.smithi121.stdout:0/758: truncate d5/d9b/fbc 1011631 0 2024-02-15T18:35:40.570 INFO:tasks.workunit.client.1.smithi121.stdout:4/631: link d5/d31/d3d/d9b/c9f d5/d9/d76/d87/cc7 0 2024-02-15T18:35:40.580 INFO:tasks.workunit.client.1.smithi121.stdout:5/793: rename d0/d2/d18/lb0 to d0/d2/d70/l104 0 2024-02-15T18:35:40.580 INFO:tasks.workunit.client.1.smithi121.stdout:5/794: fsync d0/d2/d70/fc0 0 2024-02-15T18:35:40.582 INFO:tasks.workunit.client.1.smithi121.stdout:6/704: rename d7/dc/d34/d42/d6e to d7/d4b/d60/d6c/dd5/dfa 0 2024-02-15T18:35:40.583 INFO:tasks.workunit.client.1.smithi121.stdout:0/759: rename d5/d6b/dab/db4/d7d/dd/ccf to d5/d6b/dab/db4/d7d/c106 0 2024-02-15T18:35:40.587 INFO:tasks.workunit.client.1.smithi121.stdout:0/760: rmdir d5/d6b/dab/db4/d7d/d2a/d73/d97 39 2024-02-15T18:35:40.587 INFO:tasks.workunit.client.1.smithi121.stdout:0/761: fsync d5/d30/fe6 0 2024-02-15T18:35:40.590 INFO:tasks.workunit.client.1.smithi121.stdout:6/705: truncate d7/dc/d34/d42/d62/d63/d66/dce/fd6 1483600 0 2024-02-15T18:35:40.592 INFO:tasks.workunit.client.1.smithi121.stdout:6/706: symlink d7/dc/d76/lfb 0 2024-02-15T18:35:40.601 INFO:tasks.workunit.client.1.smithi121.stdout:4/632: mkdir d5/d31/d3d/d4f/d51/d63/dc8 0 2024-02-15T18:35:40.608 INFO:tasks.workunit.client.1.smithi121.stdout:4/633: symlink d5/d31/d3d/d4f/d51/d63/dc8/lc9 0 2024-02-15T18:35:40.609 INFO:tasks.workunit.client.1.smithi121.stdout:5/795: mknod d0/c105 0 2024-02-15T18:35:40.609 INFO:tasks.workunit.client.1.smithi121.stdout:5/796: write d0/d2/d1e/d4e/d7d/f2c [2754622,10719] 0 2024-02-15T18:35:40.610 INFO:tasks.workunit.client.1.smithi121.stdout:4/634: rmdir d5/d31/d25 39 2024-02-15T18:35:40.611 INFO:tasks.workunit.client.1.smithi121.stdout:5/797: link d0/d2/d1e/d4e/d7d/d9d/d4c/c75 d0/d2/d70/de1/c106 0 2024-02-15T18:35:40.614 INFO:tasks.workunit.client.1.smithi121.stdout:4/635: mkdir d5/d31/d3d/d4f/d51/d71/dca 0 2024-02-15T18:35:40.614 INFO:tasks.workunit.client.1.smithi121.stdout:4/636: fdatasync d5/d9/d19/d2a/f68 0 2024-02-15T18:35:40.615 INFO:tasks.workunit.client.1.smithi121.stdout:6/707: dwrite d7/dc/ff3 [0,4194304] 0 2024-02-15T18:35:40.615 INFO:tasks.workunit.client.1.smithi121.stdout:5/798: creat d0/d2/d1e/d4e/d7d/d39/f107 x:0 0 0 2024-02-15T18:35:40.618 INFO:tasks.workunit.client.1.smithi121.stdout:6/708: creat d7/dc/d34/d42/d62/d63/d6f/db9/df0/ffc x:0 0 0 2024-02-15T18:35:40.620 INFO:tasks.workunit.client.1.smithi121.stdout:6/709: unlink d7/dc/d76/l9e 0 2024-02-15T18:35:40.620 INFO:tasks.workunit.client.1.smithi121.stdout:0/762: dwrite d5/d6b/dab/db4/d7d/d74/f82 [0,4194304] 0 2024-02-15T18:35:40.621 INFO:tasks.workunit.client.1.smithi121.stdout:6/710: creat d7/d4b/d60/d6c/dd5/dfa/ffd x:0 0 0 2024-02-15T18:35:40.624 INFO:tasks.workunit.client.1.smithi121.stdout:0/763: truncate d5/d6b/dab/db4/d7d/dd/f22 233061 0 2024-02-15T18:35:40.624 INFO:tasks.workunit.client.1.smithi121.stdout:0/764: truncate d5/d6b/dab/db4/d7d/d2a/d73/d97/ff5 1519915 0 2024-02-15T18:35:40.627 INFO:tasks.workunit.client.1.smithi121.stdout:0/765: mkdir d5/d107 0 2024-02-15T18:35:40.627 INFO:tasks.workunit.client.1.smithi121.stdout:0/766: write d5/d6b/dab/db4/d7d/d2a/d36/f54 [4261665,112083] 0 2024-02-15T18:35:40.627 INFO:tasks.workunit.client.1.smithi121.stdout:0/767: write d5/d6b/dab/db4/d7d/d2a/d73/d97/ffb [1155041,52439] 0 2024-02-15T18:35:40.627 INFO:tasks.workunit.client.1.smithi121.stdout:0/768: fdatasync d5/f27 0 2024-02-15T18:35:40.627 INFO:tasks.workunit.client.1.smithi121.stdout:6/711: link d7/d4b/c7b d7/dc/cfe 0 2024-02-15T18:35:40.627 INFO:tasks.workunit.client.1.smithi121.stdout:6/712: write d7/dc/d34/d42/d62/d63/d8d/fa0 [189656,75728] 0 2024-02-15T18:35:40.640 INFO:tasks.workunit.client.1.smithi121.stdout:6/713: truncate d7/dc/ff3 1553178 0 2024-02-15T18:35:40.641 INFO:tasks.workunit.client.1.smithi121.stdout:6/714: readlink d7/dc/d34/d42/d62/d63/l95 0 2024-02-15T18:35:40.666 INFO:tasks.workunit.client.1.smithi121.stdout:0/769: dwrite d5/d6b/dab/db4/d7d/d2a/d36/d44/fc7 [0,4194304] 0 2024-02-15T18:35:40.667 INFO:tasks.workunit.client.1.smithi121.stdout:4/637: dwrite d5/d31/d3d/d4f/d51/d71/fbc [0,4194304] 0 2024-02-15T18:35:40.671 INFO:tasks.workunit.client.1.smithi121.stdout:5/799: dwrite d0/f60 [0,4194304] 0 2024-02-15T18:35:40.671 INFO:tasks.workunit.client.1.smithi121.stdout:0/770: creat d5/d6b/dab/db4/d7d/d2a/d36/d44/db7/f108 x:0 0 0 2024-02-15T18:35:40.671 INFO:tasks.workunit.client.1.smithi121.stdout:0/771: chown d5/d6b/dab/db4/d7d/d74/d96/c9e 0 1 2024-02-15T18:35:40.671 INFO:tasks.workunit.client.1.smithi121.stdout:0/772: chown d5/d30/d5d/c62 15838431 1 2024-02-15T18:35:40.671 INFO:tasks.workunit.client.1.smithi121.stdout:4/638: write d5/d31/d3d/d4a/f60 [480821,111120] 0 2024-02-15T18:35:40.674 INFO:tasks.workunit.client.1.smithi121.stdout:6/715: dwrite d7/d4b/d60/d73/fb2 [0,4194304] 0 2024-02-15T18:35:40.674 INFO:tasks.workunit.client.1.smithi121.stdout:6/716: chown d7/dc/d34/d42/d62/dac/fc4 207765 1 2024-02-15T18:35:40.674 INFO:tasks.workunit.client.1.smithi121.stdout:6/717: chown d7/dc/f77 28 1 2024-02-15T18:35:40.674 INFO:tasks.workunit.client.1.smithi121.stdout:6/718: fdatasync d7/d4b/d59/fec 0 2024-02-15T18:35:40.674 INFO:tasks.workunit.client.1.smithi121.stdout:5/800: symlink d0/d16/d34/da4/l108 0 2024-02-15T18:35:40.674 INFO:tasks.workunit.client.1.smithi121.stdout:5/801: write d0/d2/d1e/d4e/d7d/d9d/d4c/f5e [659296,2442] 0 2024-02-15T18:35:40.674 INFO:tasks.workunit.client.1.smithi121.stdout:5/802: readlink d0/d16/le6 0 2024-02-15T18:35:40.674 INFO:tasks.workunit.client.1.smithi121.stdout:4/639: truncate d5/d9/d1d/f41 1765899 0 2024-02-15T18:35:40.675 INFO:tasks.workunit.client.1.smithi121.stdout:4/640: truncate d5/d9/d76/d9e/f6a 180200 0 2024-02-15T18:35:40.676 INFO:tasks.workunit.client.1.smithi121.stdout:0/773: read d5/d6b/dab/db4/d7d/dd/f68 [2470547,127199] 0 2024-02-15T18:35:40.678 INFO:tasks.workunit.client.1.smithi121.stdout:5/803: symlink d0/d16/d34/d8f/l109 0 2024-02-15T18:35:40.679 INFO:tasks.workunit.client.1.smithi121.stdout:4/641: mknod d5/d9/d19/d4c/d9a/ccb 0 2024-02-15T18:35:40.684 INFO:tasks.workunit.client.1.smithi121.stdout:5/804: creat d0/d2/d1e/f10a x:0 0 0 2024-02-15T18:35:40.686 INFO:tasks.workunit.client.1.smithi121.stdout:0/774: dread d5/d6b/dab/db4/d7d/d74/fc6 [0,4194304] 0 2024-02-15T18:35:40.686 INFO:tasks.workunit.client.1.smithi121.stdout:0/775: chown d5/d6b/dab/db4/cb3 1563434 1 2024-02-15T18:35:40.686 INFO:tasks.workunit.client.1.smithi121.stdout:5/805: write d0/f4b [3822744,116171] 0 2024-02-15T18:35:40.686 INFO:tasks.workunit.client.1.smithi121.stdout:5/806: fsync d0/d16/d34/f43 0 2024-02-15T18:35:40.690 INFO:tasks.workunit.client.1.smithi121.stdout:5/807: creat d0/d16/d34/da4/f10b x:0 0 0 2024-02-15T18:35:40.690 INFO:tasks.workunit.client.1.smithi121.stdout:5/808: truncate d0/d2/d1e/f10a 720073 0 2024-02-15T18:35:40.690 INFO:tasks.workunit.client.1.smithi121.stdout:5/809: dread - d0/d2/d18/f101 zero size 2024-02-15T18:35:40.691 INFO:tasks.workunit.client.1.smithi121.stdout:6/719: dwrite d7/dc/d34/d42/d62/d63/d6f/f88 [4194304,4194304] 0 2024-02-15T18:35:40.693 INFO:tasks.workunit.client.1.smithi121.stdout:6/720: rmdir d7/dc/d76 39 2024-02-15T18:35:40.693 INFO:tasks.workunit.client.1.smithi121.stdout:6/721: chown d7/dc/c22 2287 1 2024-02-15T18:35:40.705 INFO:tasks.workunit.client.1.smithi121.stdout:5/810: dread d0/d2/d1e/d4e/d7d/d9d/fc4 [0,4194304] 0 2024-02-15T18:35:40.714 INFO:tasks.workunit.client.1.smithi121.stdout:0/776: dwrite d5/d6b/dab/db4/d7d/d2a/d73/d97/ff5 [0,4194304] 0 2024-02-15T18:35:40.718 INFO:tasks.workunit.client.1.smithi121.stdout:0/777: mkdir d5/d107/d109 0 2024-02-15T18:35:40.721 INFO:tasks.workunit.client.1.smithi121.stdout:6/722: dwrite d7/dc/d2c/d36/f9d [0,4194304] 0 2024-02-15T18:35:40.721 INFO:tasks.workunit.client.1.smithi121.stdout:6/723: read - d7/dc/d34/d42/fee zero size 2024-02-15T18:35:40.725 INFO:tasks.workunit.client.1.smithi121.stdout:6/724: truncate d7/dc/d34/d42/f54 2363861 0 2024-02-15T18:35:40.728 INFO:tasks.workunit.client.1.smithi121.stdout:0/778: dread d5/d6b/dab/db4/d7d/d2a/d36/f54 [0,4194304] 0 2024-02-15T18:35:40.730 INFO:tasks.workunit.client.1.smithi121.stdout:0/779: mkdir d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/d10a 0 2024-02-15T18:35:40.731 INFO:tasks.workunit.client.1.smithi121.stdout:9/688: sync 2024-02-15T18:35:40.732 INFO:tasks.workunit.client.1.smithi121.stdout:2/390: sync 2024-02-15T18:35:40.737 INFO:tasks.workunit.client.1.smithi121.stdout:0/780: rmdir d5/d6b/dab/db4/d7d/d2a/d73/d97/db8 39 2024-02-15T18:35:40.737 INFO:tasks.workunit.client.1.smithi121.stdout:2/391: mknod d5/df/d27/d3e/c6f 0 2024-02-15T18:35:40.737 INFO:tasks.workunit.client.1.smithi121.stdout:9/689: creat d7/d3a/d8e/d9e/da9/ff3 x:0 0 0 2024-02-15T18:35:40.737 INFO:tasks.workunit.client.1.smithi121.stdout:9/690: stat d7/d8/ca 0 2024-02-15T18:35:40.743 INFO:tasks.workunit.client.1.smithi121.stdout:9/691: symlink d7/d8/d25/d30/d33/d8f/d93/lf4 0 2024-02-15T18:35:40.743 INFO:tasks.workunit.client.1.smithi121.stdout:0/781: symlink d5/d30/d5d/l10b 0 2024-02-15T18:35:40.744 INFO:tasks.workunit.client.1.smithi121.stdout:2/392: rmdir d5/df/d27/d61 39 2024-02-15T18:35:40.744 INFO:tasks.workunit.client.1.smithi121.stdout:2/393: stat d5/df/d27/d61 0 2024-02-15T18:35:40.744 INFO:tasks.workunit.client.1.smithi121.stdout:2/394: write d5/df/d54/f5d [1144601,38250] 0 2024-02-15T18:35:40.744 INFO:tasks.workunit.client.1.smithi121.stdout:5/811: rename d0/d2/d70/d88 to d0/d2/d10c 0 2024-02-15T18:35:40.745 INFO:tasks.workunit.client.1.smithi121.stdout:0/782: symlink d5/d6b/dab/db4/d7d/d2a/d4e/d100/l10c 0 2024-02-15T18:35:40.745 INFO:tasks.workunit.client.1.smithi121.stdout:9/692: creat d7/d26/d4f/ff5 x:0 0 0 2024-02-15T18:35:40.745 INFO:tasks.workunit.client.1.smithi121.stdout:9/693: fdatasync d7/d8/fa8 0 2024-02-15T18:35:40.745 INFO:tasks.workunit.client.1.smithi121.stdout:9/694: fdatasync d7/f23 0 2024-02-15T18:35:40.747 INFO:tasks.workunit.client.1.smithi121.stdout:5/812: creat d0/d2/d18/f10d x:0 0 0 2024-02-15T18:35:40.748 INFO:tasks.workunit.client.1.smithi121.stdout:9/695: mkdir d7/df6 0 2024-02-15T18:35:40.750 INFO:tasks.workunit.client.1.smithi121.stdout:9/696: creat d7/d8/d36/da3/ff7 x:0 0 0 2024-02-15T18:35:40.752 INFO:tasks.workunit.client.1.smithi121.stdout:6/725: dwrite d7/dc/d55/fe1 [0,4194304] 0 2024-02-15T18:35:40.753 INFO:tasks.workunit.client.1.smithi121.stdout:6/726: rename d7 to d7/d4b/d60/d6c/dd5/dff 22 2024-02-15T18:35:40.753 INFO:tasks.workunit.client.1.smithi121.stdout:6/727: dread - d7/d4b/d60/d6c/feb zero size 2024-02-15T18:35:40.756 INFO:tasks.workunit.client.1.smithi121.stdout:9/697: mknod d7/d3a/d8e/d9e/cf8 0 2024-02-15T18:35:40.756 INFO:tasks.workunit.client.1.smithi121.stdout:6/728: mkdir d7/dc/d2c/d100 0 2024-02-15T18:35:40.767 INFO:tasks.workunit.client.1.smithi121.stdout:6/729: dread d7/dc/ff3 [0,4194304] 0 2024-02-15T18:35:40.768 INFO:tasks.workunit.client.1.smithi121.stdout:6/730: creat d7/dc/d34/d42/d62/d63/d6f/f101 x:0 0 0 2024-02-15T18:35:40.769 INFO:tasks.workunit.client.1.smithi121.stdout:9/698: write d7/d8/f1d [2255621,91983] 0 2024-02-15T18:35:40.770 INFO:tasks.workunit.client.1.smithi121.stdout:5/813: dwrite d0/d16/d34/da4/fea [0,4194304] 0 2024-02-15T18:35:40.770 INFO:tasks.workunit.client.1.smithi121.stdout:5/814: write d0/d2/d1e/f8c [874180,99964] 0 2024-02-15T18:35:40.771 INFO:tasks.workunit.client.1.smithi121.stdout:6/731: rename d7/dc/f77 to d7/f102 0 2024-02-15T18:35:40.773 INFO:tasks.workunit.client.1.smithi121.stdout:2/395: dwrite d5/d49/d4d/d22/f4b [0,4194304] 0 2024-02-15T18:35:40.773 INFO:tasks.workunit.client.1.smithi121.stdout:2/396: dread - d5/df/d27/d61/f6b zero size 2024-02-15T18:35:40.773 INFO:tasks.workunit.client.1.smithi121.stdout:2/397: stat d5/df/d10/d4c 0 2024-02-15T18:35:40.774 INFO:tasks.workunit.client.1.smithi121.stdout:6/732: unlink d7/dc/d34/d42/d62/d63/d6f/d89/fcc 0 2024-02-15T18:35:40.777 INFO:tasks.workunit.client.1.smithi121.stdout:0/783: rmdir d5/d30 39 2024-02-15T18:35:40.778 INFO:tasks.workunit.client.1.smithi121.stdout:5/815: symlink d0/d16/d7a/db5/l10e 0 2024-02-15T18:35:40.778 INFO:tasks.workunit.client.1.smithi121.stdout:5/816: write d0/d2/d1e/d4e/d7d/d40/fc8 [443616,17658] 0 2024-02-15T18:35:40.778 INFO:tasks.workunit.client.1.smithi121.stdout:5/817: truncate d0/d2/f14 823418 0 2024-02-15T18:35:40.778 INFO:tasks.workunit.client.1.smithi121.stdout:5/818: write d0/d2/d1e/d4e/d7d/d40/d62/db1/fdd [999897,54887] 0 2024-02-15T18:35:40.781 INFO:tasks.workunit.client.1.smithi121.stdout:2/398: rmdir d5/d49/d4d/d22 39 2024-02-15T18:35:40.782 INFO:tasks.workunit.client.1.smithi121.stdout:6/733: mkdir d7/dc/d76/d103 0 2024-02-15T18:35:40.782 INFO:tasks.workunit.client.1.smithi121.stdout:6/734: dread - d7/d4b/d60/d6c/dd5/dfa/fad zero size 2024-02-15T18:35:40.782 INFO:tasks.workunit.client.1.smithi121.stdout:6/735: readlink d7/dc/d34/d42/d62/d63/ld2 0 2024-02-15T18:35:40.783 INFO:tasks.workunit.client.1.smithi121.stdout:0/784: rename d5/d6b/dab/db4/d7d/dd/c14 to d5/d6b/dab/db4/d7d/dd/c10d 0 2024-02-15T18:35:40.784 INFO:tasks.workunit.client.1.smithi121.stdout:6/736: write d7/dc/d34/f26 [6307867,57980] 0 2024-02-15T18:35:40.785 INFO:tasks.workunit.client.1.smithi121.stdout:2/399: creat d5/df/d27/d3e/d35/f70 x:0 0 0 2024-02-15T18:35:40.787 INFO:tasks.workunit.client.1.smithi121.stdout:0/785: rename d5/d6b/dab/db4/d7d/d2a/d36/d44/l6c to d5/d6b/dab/db4/d7d/l10e 0 2024-02-15T18:35:40.788 INFO:tasks.workunit.client.1.smithi121.stdout:0/786: chown d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/lba 240399 1 2024-02-15T18:35:40.792 INFO:tasks.workunit.client.1.smithi121.stdout:0/787: write d5/d6b/dab/db4/d7d/d2a/d73/d97/ff5 [574057,41072] 0 2024-02-15T18:35:40.792 INFO:tasks.workunit.client.1.smithi121.stdout:2/400: read d5/d49/d4d/f1c [755560,85924] 0 2024-02-15T18:35:40.793 INFO:tasks.workunit.client.1.smithi121.stdout:2/401: read d5/d49/d4d/f2d [2843070,105739] 0 2024-02-15T18:35:40.794 INFO:tasks.workunit.client.1.smithi121.stdout:0/788: rmdir d5/d6b/dab/db4/d7d/d2a/d73/d8d/dc8/df3 0 2024-02-15T18:35:40.794 INFO:tasks.workunit.client.1.smithi121.stdout:0/789: chown d5/d6b/dab/db4/d7d/dd/fb0 325756 1 2024-02-15T18:35:40.796 INFO:tasks.workunit.client.1.smithi121.stdout:9/699: dwrite d7/d8/d25/d30/da5/fd8 [0,4194304] 0 2024-02-15T18:35:40.797 INFO:tasks.workunit.client.1.smithi121.stdout:9/700: write d7/d8/d25/d30/da5/f41 [4429031,130561] 0 2024-02-15T18:35:40.797 INFO:tasks.workunit.client.1.smithi121.stdout:9/701: write d7/d3a/d8e/d9e/da9/d8b/d73/fad [910671,31858] 0 2024-02-15T18:35:40.798 INFO:tasks.workunit.client.1.smithi121.stdout:0/790: getdents d5/d6b/dab/db4/d7d/d2a/d36/d44/db7 0 2024-02-15T18:35:40.802 INFO:tasks.workunit.client.1.smithi121.stdout:9/702: mkdir d7/d8/d36/df9 0 2024-02-15T18:35:40.803 INFO:tasks.workunit.client.1.smithi121.stdout:0/791: symlink d5/d9b/l10f 0 2024-02-15T18:35:40.805 INFO:tasks.workunit.client.1.smithi121.stdout:9/703: creat d7/d3a/d8e/d9e/da9/d8b/d73/ffa x:0 0 0 2024-02-15T18:35:40.806 INFO:tasks.workunit.client.1.smithi121.stdout:6/737: dwrite d7/dc/d2c/d36/f9d [4194304,4194304] 0 2024-02-15T18:35:40.807 INFO:tasks.workunit.client.1.smithi121.stdout:0/792: mknod d5/d9b/c110 0 2024-02-15T18:35:40.807 INFO:tasks.workunit.client.1.smithi121.stdout:0/793: dread - d5/d6b/dab/db4/d7d/d2a/d4e/fcc zero size 2024-02-15T18:35:40.808 INFO:tasks.workunit.client.1.smithi121.stdout:9/704: dread d7/d8/d25/d30/d33/d79/fa0 [0,4194304] 0 2024-02-15T18:35:40.810 INFO:tasks.workunit.client.1.smithi121.stdout:9/705: write d7/d8/d36/f7d [1688447,31148] 0 2024-02-15T18:35:40.811 INFO:tasks.workunit.client.1.smithi121.stdout:9/706: read d7/d8/f87 [1432045,19420] 0 2024-02-15T18:35:40.811 INFO:tasks.workunit.client.1.smithi121.stdout:9/707: fsync d7/d8/d25/d30/d33/d3c/f84 0 2024-02-15T18:35:40.811 INFO:tasks.workunit.client.1.smithi121.stdout:0/794: dread d5/d6b/dab/db4/d7d/f3a [0,4194304] 0 2024-02-15T18:35:40.811 INFO:tasks.workunit.client.1.smithi121.stdout:0/795: chown d5/d6b/dab/db4/d7d/dd/f32 4360 1 2024-02-15T18:35:40.816 INFO:tasks.workunit.client.1.smithi121.stdout:2/402: rename d5/df/d27/d3e/d35 to d5/d49/d71 0 2024-02-15T18:35:40.817 INFO:tasks.workunit.client.1.smithi121.stdout:6/738: creat d7/dc/d55/f104 x:0 0 0 2024-02-15T18:35:40.820 INFO:tasks.workunit.client.1.smithi121.stdout:0/796: link d5/d30/d5d/f5e d5/d30/d5d/f111 0 2024-02-15T18:35:40.823 INFO:tasks.workunit.client.1.smithi121.stdout:9/708: rename d7/d8/d40/f52 to d7/d3a/d8e/d9e/da9/d8b/ffb 0 2024-02-15T18:35:40.824 INFO:tasks.workunit.client.1.smithi121.stdout:9/709: write d7/d8/d36/f7d [3581885,40379] 0 2024-02-15T18:35:40.825 INFO:tasks.workunit.client.1.smithi121.stdout:2/403: rename d5/df/d27/f5f to d5/df/d27/f72 0 2024-02-15T18:35:40.825 INFO:tasks.workunit.client.1.smithi121.stdout:6/739: creat d7/dc/d34/d42/d62/d63/d8d/f105 x:0 0 0 2024-02-15T18:35:40.826 INFO:tasks.workunit.client.1.smithi121.stdout:0/797: rmdir d5/d6b/dab/db1/dbf 39 2024-02-15T18:35:40.829 INFO:tasks.workunit.client.1.smithi121.stdout:9/710: mknod d7/d3a/d8e/d9e/cfc 0 2024-02-15T18:35:40.829 INFO:tasks.workunit.client.1.smithi121.stdout:6/740: rmdir d7/d4b/d60/d6c/dd5 39 2024-02-15T18:35:40.831 INFO:tasks.workunit.client.1.smithi121.stdout:6/741: creat d7/dc/d34/d42/d62/d63/f106 x:0 0 0 2024-02-15T18:35:40.833 INFO:tasks.workunit.client.1.smithi121.stdout:6/742: symlink d7/dc/l107 0 2024-02-15T18:35:40.833 INFO:tasks.workunit.client.1.smithi121.stdout:6/743: dread - d7/d4b/d60/d6c/fa6 zero size 2024-02-15T18:35:40.842 INFO:tasks.workunit.client.1.smithi121.stdout:6/744: write d7/f31 [596428,73435] 0 2024-02-15T18:35:40.842 INFO:tasks.workunit.client.1.smithi121.stdout:6/745: getdents d7/d4b/d60/d6c/dd5/dfa/d91 0 2024-02-15T18:35:40.843 INFO:tasks.workunit.client.1.smithi121.stdout:6/746: creat d7/dc/f108 x:0 0 0 2024-02-15T18:35:40.845 INFO:tasks.workunit.client.1.smithi121.stdout:6/747: symlink d7/d4b/d60/d6c/dd5/dfa/d91/l109 0 2024-02-15T18:35:40.856 INFO:tasks.workunit.client.1.smithi121.stdout:2/404: dwrite d5/d49/d4d/f2d [0,4194304] 0 2024-02-15T18:35:40.856 INFO:tasks.workunit.client.1.smithi121.stdout:9/711: dwrite d7/d8/d25/d30/f80 [0,4194304] 0 2024-02-15T18:35:40.856 INFO:tasks.workunit.client.1.smithi121.stdout:2/405: mknod d5/df/d27/d61/c73 0 2024-02-15T18:35:40.857 INFO:tasks.workunit.client.1.smithi121.stdout:0/798: dwrite d5/f92 [0,4194304] 0 2024-02-15T18:35:40.858 INFO:tasks.workunit.client.1.smithi121.stdout:9/712: truncate d7/d26/fd6 2836233 0 2024-02-15T18:35:40.858 INFO:tasks.workunit.client.1.smithi121.stdout:9/713: readlink d7/d8/d25/d30/d33/l8d 0 2024-02-15T18:35:40.859 INFO:tasks.workunit.client.1.smithi121.stdout:2/406: symlink d5/d49/d71/l74 0 2024-02-15T18:35:40.859 INFO:tasks.workunit.client.1.smithi121.stdout:2/407: write d5/df/d27/f72 [604760,78263] 0 2024-02-15T18:35:40.896 INFO:tasks.workunit.client.1.smithi121.stdout:9/714: creat d7/d8/d36/da3/ffd x:0 0 0 2024-02-15T18:35:40.897 INFO:tasks.workunit.client.1.smithi121.stdout:2/408: creat d5/d49/d4d/f75 x:0 0 0 2024-02-15T18:35:40.898 INFO:tasks.workunit.client.1.smithi121.stdout:9/715: creat d7/d8/d36/dce/ffe x:0 0 0 2024-02-15T18:35:40.898 INFO:tasks.workunit.client.1.smithi121.stdout:9/716: truncate d7/d8/d40/f6e 722864 0 2024-02-15T18:35:40.898 INFO:tasks.workunit.client.1.smithi121.stdout:9/717: dread - d7/d8/d36/da3/ffd zero size 2024-02-15T18:35:40.898 INFO:tasks.workunit.client.1.smithi121.stdout:2/409: rmdir d5/df/d27/d3a 39 2024-02-15T18:35:40.900 INFO:tasks.workunit.client.1.smithi121.stdout:9/718: unlink d7/d8/d25/fc2 0 2024-02-15T18:35:40.900 INFO:tasks.workunit.client.1.smithi121.stdout:9/719: truncate d7/d8/d40/f72 839093 0 2024-02-15T18:35:40.900 INFO:tasks.workunit.client.1.smithi121.stdout:9/720: dread - d7/d26/d4f/d5b/d77/fe6 zero size 2024-02-15T18:35:40.900 INFO:tasks.workunit.client.1.smithi121.stdout:9/721: stat d7/d8/d25/d30/d33/feb 0 2024-02-15T18:35:40.909 INFO:tasks.workunit.client.1.smithi121.stdout:2/410: write f4 [2023130,80297] 0 2024-02-15T18:35:40.909 INFO:tasks.workunit.client.1.smithi121.stdout:2/411: chown f4 217 1 2024-02-15T18:35:40.909 INFO:tasks.workunit.client.1.smithi121.stdout:2/412: read d5/d49/d4d/d22/f4b [1650211,7105] 0 2024-02-15T18:35:40.918 INFO:tasks.workunit.client.1.smithi121.stdout:9/722: dwrite d7/d8/d25/d30/d33/d3c/dd0/ff2 [0,4194304] 0 2024-02-15T18:35:40.921 INFO:tasks.workunit.client.1.smithi121.stdout:2/413: dread d5/d49/d4d/d22/f3c [0,4194304] 0 2024-02-15T18:35:40.922 INFO:tasks.workunit.client.1.smithi121.stdout:9/723: rename d7/d3a/d8e/d9e/da9/c5c to d7/d8/d25/d30/d33/d3c/dd0/cff 0 2024-02-15T18:35:40.922 INFO:tasks.workunit.client.1.smithi121.stdout:9/724: dread - d7/d3a/d8e/d9e/dc7/fe1 zero size 2024-02-15T18:35:40.922 INFO:tasks.workunit.client.1.smithi121.stdout:9/725: fdatasync d7/d8/fa8 0 2024-02-15T18:35:40.923 INFO:tasks.workunit.client.1.smithi121.stdout:9/726: dread - d7/d26/d4f/d5b/d77/fe6 zero size 2024-02-15T18:35:40.924 INFO:tasks.workunit.client.1.smithi121.stdout:2/414: creat d5/df/d27/f76 x:0 0 0 2024-02-15T18:35:40.925 INFO:tasks.workunit.client.1.smithi121.stdout:9/727: rmdir d7/d8/d25/d30/d33 39 2024-02-15T18:35:40.935 INFO:tasks.workunit.client.1.smithi121.stdout:2/415: dread d5/d49/d4d/d22/f4b [0,4194304] 0 2024-02-15T18:35:40.939 INFO:tasks.workunit.client.1.smithi121.stdout:2/416: getdents d5/d49 0 2024-02-15T18:35:40.939 INFO:tasks.workunit.client.1.smithi121.stdout:2/417: write d5/f3b [1068865,61277] 0 2024-02-15T18:35:40.940 INFO:tasks.workunit.client.1.smithi121.stdout:2/418: creat d5/df/d10/d4c/f77 x:0 0 0 2024-02-15T18:35:40.940 INFO:tasks.workunit.client.1.smithi121.stdout:9/728: dwrite d7/d3a/d8e/d9e/da9/d8b/d73/fdd [0,4194304] 0 2024-02-15T18:35:40.940 INFO:tasks.workunit.client.1.smithi121.stdout:2/419: creat d5/f78 x:0 0 0 2024-02-15T18:35:41.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:40.595742+0000 mgr.smithi121.ckiryi (mgr.24385) 14 : cluster [DBG] pgmap v14: 65 pgs: 65 active+clean; 1.6 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 49 MiB/s rd, 109 MiB/s wr, 385 op/s 2024-02-15T18:35:41.024 INFO:tasks.workunit.client.1.smithi121.stdout:3/487: sync 2024-02-15T18:35:41.024 INFO:tasks.workunit.client.1.smithi121.stdout:8/614: sync 2024-02-15T18:35:41.024 INFO:tasks.workunit.client.1.smithi121.stdout:8/615: chown d0/d7/db1/dbb/fa2 53944 1 2024-02-15T18:35:41.032 INFO:tasks.workunit.client.1.smithi121.stdout:9/729: creat d7/d3a/d8e/d9e/da9/d8b/d73/f100 x:0 0 0 2024-02-15T18:35:41.032 INFO:tasks.workunit.client.1.smithi121.stdout:9/730: write d7/d8/d25/d30/d33/f58 [3707089,89643] 0 2024-02-15T18:35:41.032 INFO:tasks.workunit.client.1.smithi121.stdout:9/731: fdatasync d7/d8/fbe 0 2024-02-15T18:35:41.032 INFO:tasks.workunit.client.1.smithi121.stdout:2/420: creat d5/df/d27/f79 x:0 0 0 2024-02-15T18:35:41.033 INFO:tasks.workunit.client.1.smithi121.stdout:3/488: dread d2/d1a/d54/f69 [0,4194304] 0 2024-02-15T18:35:41.034 INFO:tasks.workunit.client.1.smithi121.stdout:8/616: creat d0/d2a/fd6 x:0 0 0 2024-02-15T18:35:41.035 INFO:tasks.workunit.client.1.smithi121.stdout:2/421: link d5/d49/d71/l74 d5/df/d27/d3e/d6a/l7a 0 2024-02-15T18:35:41.036 INFO:tasks.workunit.client.1.smithi121.stdout:3/489: getdents d2/d1a/d27/d63/d81 0 2024-02-15T18:35:41.038 INFO:tasks.workunit.client.1.smithi121.stdout:2/422: symlink d5/df/d27/d3e/d6a/l7b 0 2024-02-15T18:35:41.039 INFO:tasks.workunit.client.1.smithi121.stdout:2/423: symlink d5/df/d10/d4c/l7c 0 2024-02-15T18:35:41.045 INFO:tasks.workunit.client.1.smithi121.stdout:3/490: dread d2/d1a/d27/d7c/d25/d31/d34/d4a/f99 [0,4194304] 0 2024-02-15T18:35:41.046 INFO:tasks.workunit.client.1.smithi121.stdout:3/491: creat d2/d1a/d27/d7c/d25/d31/d34/d4a/fa0 x:0 0 0 2024-02-15T18:35:41.046 INFO:tasks.workunit.client.1.smithi121.stdout:3/492: dread - d2/d1a/d27/f6c zero size 2024-02-15T18:35:41.046 INFO:tasks.workunit.client.1.smithi121.stdout:3/493: write d2/d1a/f7a [80790,32407] 0 2024-02-15T18:35:41.052 INFO:tasks.workunit.client.1.smithi121.stdout:9/732: dwrite d7/f1c [0,4194304] 0 2024-02-15T18:35:41.052 INFO:tasks.workunit.client.1.smithi121.stdout:9/733: chown d7/d8/d25/d30/d33/f58 22371039 1 2024-02-15T18:35:41.056 INFO:tasks.workunit.client.1.smithi121.stdout:3/494: dread d2/d1a/d22/f42 [0,4194304] 0 2024-02-15T18:35:41.058 INFO:tasks.workunit.client.1.smithi121.stdout:2/424: dwrite d5/d49/d71/f70 [0,4194304] 0 2024-02-15T18:35:41.058 INFO:tasks.workunit.client.1.smithi121.stdout:9/734: write d7/d8/d25/d30/f62 [1440189,54229] 0 2024-02-15T18:35:41.058 INFO:tasks.workunit.client.1.smithi121.stdout:9/735: chown d7/d8/d25/d30/da5/fbc 5287 1 2024-02-15T18:35:41.058 INFO:tasks.workunit.client.1.smithi121.stdout:9/736: stat d7/dcc/ldf 0 2024-02-15T18:35:41.059 INFO:tasks.workunit.client.1.smithi121.stdout:3/495: dread d2/d1a/d6b/f75 [0,4194304] 0 2024-02-15T18:35:41.062 INFO:tasks.workunit.client.1.smithi121.stdout:3/496: link d2/d1a/d27/d7c/f7b d2/d1a/d6b/d7d/fa1 0 2024-02-15T18:35:41.062 INFO:tasks.workunit.client.1.smithi121.stdout:3/497: stat d2/f7 0 2024-02-15T18:35:41.064 INFO:tasks.workunit.client.1.smithi121.stdout:2/425: dread f4 [0,4194304] 0 2024-02-15T18:35:41.066 INFO:tasks.workunit.client.1.smithi121.stdout:2/426: rename d5/df/d27/f72 to d5/df/f7d 0 2024-02-15T18:35:41.066 INFO:tasks.workunit.client.1.smithi121.stdout:2/427: chown d5/df/d27/d3e/f59 6 1 2024-02-15T18:35:41.083 INFO:tasks.workunit.client.1.smithi121.stdout:9/737: dwrite d7/d8/d25/d30/da5/fd8 [0,4194304] 0 2024-02-15T18:35:41.084 INFO:tasks.workunit.client.1.smithi121.stdout:9/738: truncate d7/d8/d36/f7d 825320 0 2024-02-15T18:35:41.084 INFO:tasks.workunit.client.1.smithi121.stdout:2/428: dwrite d5/df/d27/d3e/f62 [0,4194304] 0 2024-02-15T18:35:41.087 INFO:tasks.workunit.client.1.smithi121.stdout:9/739: mkdir d7/d3a/d8e/d9e/dc7/d101 0 2024-02-15T18:35:41.087 INFO:tasks.workunit.client.1.smithi121.stdout:9/740: write d7/d8/d25/d30/d33/d79/fa0 [364257,26183] 0 2024-02-15T18:35:41.087 INFO:tasks.workunit.client.1.smithi121.stdout:9/741: chown d7/d8/d25/d30/d33/d3c/f84 150585638 1 2024-02-15T18:35:41.089 INFO:tasks.workunit.client.1.smithi121.stdout:9/742: dread d7/d8/d25/d30/d33/d8f/d93/fb4 [0,4194304] 0 2024-02-15T18:35:41.089 INFO:tasks.workunit.client.1.smithi121.stdout:9/743: stat d7/d3a/d8e/d9e/da9/ff3 0 2024-02-15T18:35:41.092 INFO:tasks.workunit.client.1.smithi121.stdout:9/744: write d7/d8/d36/f7d [1410713,127836] 0 2024-02-15T18:35:41.094 INFO:tasks.workunit.client.1.smithi121.stdout:9/745: mknod d7/d8/d40/daf/c102 0 2024-02-15T18:35:41.095 INFO:tasks.workunit.client.1.smithi121.stdout:9/746: write d7/d8/d36/f2f [2096652,93743] 0 2024-02-15T18:35:41.095 INFO:tasks.workunit.client.1.smithi121.stdout:9/747: truncate d7/f57 4399644 0 2024-02-15T18:35:41.097 INFO:tasks.workunit.client.1.smithi121.stdout:9/748: creat d7/d8/d36/dce/f103 x:0 0 0 2024-02-15T18:35:41.098 INFO:tasks.workunit.client.1.smithi121.stdout:9/749: dread d7/d3a/d8e/d9e/fcb [0,4194304] 0 2024-02-15T18:35:41.101 INFO:tasks.workunit.client.1.smithi121.stdout:9/750: rename d7/d26/d4f/d5b/l7e to d7/d8/d25/d30/d33/d79/l104 0 2024-02-15T18:35:41.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:40.595742+0000 mgr.smithi121.ckiryi (mgr.24385) 14 : cluster [DBG] pgmap v14: 65 pgs: 65 active+clean; 1.6 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 49 MiB/s rd, 109 MiB/s wr, 385 op/s 2024-02-15T18:35:41.117 INFO:tasks.workunit.client.1.smithi121.stdout:9/751: dwrite d7/d26/f81 [4194304,4194304] 0 2024-02-15T18:35:41.118 INFO:tasks.workunit.client.1.smithi121.stdout:9/752: stat d7/d8/d25/d30/d33/d3c/dd0/dd9 0 2024-02-15T18:35:41.121 INFO:tasks.workunit.client.1.smithi121.stdout:9/753: creat d7/d8/d36/dce/f105 x:0 0 0 2024-02-15T18:35:41.123 INFO:tasks.workunit.client.1.smithi121.stdout:9/754: rename d7/d8/d36/dce/f105 to d7/d3a/d8e/d9e/da9/d8b/f106 0 2024-02-15T18:35:41.123 INFO:tasks.workunit.client.1.smithi121.stdout:9/755: dread - d7/d8/d36/dce/f103 zero size 2024-02-15T18:35:41.123 INFO:tasks.workunit.client.1.smithi121.stdout:9/756: dread - d7/d26/d4f/d5b/d77/fe6 zero size 2024-02-15T18:35:41.137 INFO:tasks.workunit.client.1.smithi121.stdout:9/757: dwrite d7/d8/d40/f6e [0,4194304] 0 2024-02-15T18:35:41.140 INFO:tasks.workunit.client.1.smithi121.stdout:9/758: mknod d7/d8/d40/c107 0 2024-02-15T18:35:41.141 INFO:tasks.workunit.client.1.smithi121.stdout:9/759: mkdir d7/d8/d25/d30/da5/d108 0 2024-02-15T18:35:41.143 INFO:tasks.workunit.client.1.smithi121.stdout:9/760: creat d7/d3a/d8e/d9e/dc7/d101/f109 x:0 0 0 2024-02-15T18:35:41.145 INFO:tasks.workunit.client.1.smithi121.stdout:9/761: unlink d7/l66 0 2024-02-15T18:35:41.165 INFO:tasks.workunit.client.1.smithi121.stdout:9/762: dwrite d7/f23 [0,4194304] 0 2024-02-15T18:35:41.165 INFO:tasks.workunit.client.1.smithi121.stdout:9/763: chown d7/d26/c69 926433 1 2024-02-15T18:35:41.167 INFO:tasks.workunit.client.1.smithi121.stdout:9/764: unlink d7/d26/dba/fee 0 2024-02-15T18:35:41.169 INFO:tasks.workunit.client.1.smithi121.stdout:9/765: rename d7/d8/f10 to d7/d3a/f10a 0 2024-02-15T18:35:41.170 INFO:tasks.workunit.client.1.smithi121.stdout:9/766: write d7/d8/d25/d30/da5/f41 [1441282,77386] 0 2024-02-15T18:35:41.185 INFO:tasks.workunit.client.1.smithi121.stdout:1/730: sync 2024-02-15T18:35:41.185 INFO:tasks.workunit.client.1.smithi121.stdout:7/449: sync 2024-02-15T18:35:41.185 INFO:tasks.workunit.client.1.smithi121.stdout:7/450: write d0/d33/f5d [1169314,30982] 0 2024-02-15T18:35:41.186 INFO:tasks.workunit.client.1.smithi121.stdout:7/451: write d0/d5/f5e [2103125,68748] 0 2024-02-15T18:35:41.186 INFO:tasks.workunit.client.1.smithi121.stdout:7/452: chown d0/d38/d3c/f4d 1094390305 1 2024-02-15T18:35:41.186 INFO:tasks.workunit.client.1.smithi121.stdout:7/453: chown d0/d5/d11/d1c/f23 14 1 2024-02-15T18:35:41.187 INFO:tasks.workunit.client.1.smithi121.stdout:9/767: dwrite d7/d8/f47 [0,4194304] 0 2024-02-15T18:35:41.187 INFO:tasks.workunit.client.1.smithi121.stdout:9/768: getdents d7/d26 0 2024-02-15T18:35:41.188 INFO:tasks.workunit.client.1.smithi121.stdout:9/769: chown d7/d3a/d8e/d9e/da9/cbf 10 1 2024-02-15T18:35:41.193 INFO:tasks.workunit.client.1.smithi121.stdout:9/770: unlink d7/d8/d25/d30/d33/d8f/fd5 0 2024-02-15T18:35:41.195 INFO:tasks.workunit.client.1.smithi121.stdout:9/771: rename d7/c9 to d7/d26/dba/c10b 0 2024-02-15T18:35:41.197 INFO:tasks.workunit.client.1.smithi121.stdout:1/731: dread d0/d6/d7/d9/d14/d3e/d88/da6/fbd [0,4194304] 0 2024-02-15T18:35:41.198 INFO:tasks.workunit.client.1.smithi121.stdout:7/454: write d0/d38/d3f/d43/f8f [1892973,41987] 0 2024-02-15T18:35:41.199 INFO:tasks.workunit.client.1.smithi121.stdout:1/732: mknod d0/d6/d7/d9/d14/d3e/d88/daf/db4/de8/cf9 0 2024-02-15T18:35:41.201 INFO:tasks.workunit.client.1.smithi121.stdout:9/772: mknod d7/d3a/d8e/d9e/da9/d8b/d73/c10c 0 2024-02-15T18:35:41.205 INFO:tasks.workunit.client.1.smithi121.stdout:7/455: rename d0/ce to d0/d5/c97 0 2024-02-15T18:35:41.206 INFO:tasks.workunit.client.1.smithi121.stdout:9/773: creat d7/d8/d36/f10d x:0 0 0 2024-02-15T18:35:41.213 INFO:tasks.workunit.client.1.smithi121.stdout:9/774: dread d7/d8/d40/f63 [0,4194304] 0 2024-02-15T18:35:41.217 INFO:tasks.workunit.client.1.smithi121.stdout:1/733: truncate d0/f1d 3910015 0 2024-02-15T18:35:41.219 INFO:tasks.workunit.client.1.smithi121.stdout:1/734: unlink d0/d6/d7/d9/d14/f6b 0 2024-02-15T18:35:41.219 INFO:tasks.workunit.client.1.smithi121.stdout:1/735: fsync d0/d6/d7/d9/f2d 0 2024-02-15T18:35:41.230 INFO:tasks.workunit.client.1.smithi121.stdout:7/456: dwrite d0/d5/d11/d1c/f3e [4194304,4194304] 0 2024-02-15T18:35:41.230 INFO:tasks.workunit.client.1.smithi121.stdout:7/457: dread - d0/d4a/f53 zero size 2024-02-15T18:35:41.231 INFO:tasks.workunit.client.1.smithi121.stdout:9/775: dwrite d7/d8/d36/dce/f103 [0,4194304] 0 2024-02-15T18:35:41.231 INFO:tasks.workunit.client.1.smithi121.stdout:9/776: read d7/d8/d25/d30/f80 [1442501,59695] 0 2024-02-15T18:35:41.231 INFO:tasks.workunit.client.1.smithi121.stdout:9/777: write d7/d26/fd6 [3409988,35538] 0 2024-02-15T18:35:41.235 INFO:tasks.workunit.client.1.smithi121.stdout:9/778: truncate d7/f4e 3756117 0 2024-02-15T18:35:41.236 INFO:tasks.workunit.client.1.smithi121.stdout:1/736: dread d0/d6/d7/d9/d14/d3e/fb5 [0,4194304] 0 2024-02-15T18:35:41.236 INFO:tasks.workunit.client.1.smithi121.stdout:1/737: fsync d0/d39/d7d/dc0/d77/ff4 0 2024-02-15T18:35:41.236 INFO:tasks.workunit.client.1.smithi121.stdout:1/738: write d0/d6/d7/d9/d14/db6/ff8 [668545,71257] 0 2024-02-15T18:35:41.237 INFO:tasks.workunit.client.1.smithi121.stdout:9/779: creat d7/d3a/d8e/d9e/da9/f10e x:0 0 0 2024-02-15T18:35:41.237 INFO:tasks.workunit.client.1.smithi121.stdout:9/780: chown d7/d8/d40/d4b 366853 1 2024-02-15T18:35:41.239 INFO:tasks.workunit.client.1.smithi121.stdout:9/781: unlink d7/d8/d25/d30/f80 0 2024-02-15T18:35:41.240 INFO:tasks.workunit.client.1.smithi121.stdout:9/782: mkdir d7/d8/d25/d30/da5/d108/d10f 0 2024-02-15T18:35:41.241 INFO:tasks.workunit.client.1.smithi121.stdout:9/783: rename d7/d8/d25/d30/da5/d108/d10f to d7/d8/d36/da3/d110 0 2024-02-15T18:35:41.244 INFO:tasks.workunit.client.1.smithi121.stdout:9/784: rename d7/d3a/d8e/d9e/da9/d8b/l21 to d7/d26/d4f/d5b/l111 0 2024-02-15T18:35:41.244 INFO:tasks.workunit.client.1.smithi121.stdout:9/785: truncate d7/d3a/d8e/d9e/dc7/fe8 956270 0 2024-02-15T18:35:41.247 INFO:tasks.workunit.client.1.smithi121.stdout:1/739: dread d0/d6/d7/d9/d51/d5f/f62 [0,4194304] 0 2024-02-15T18:35:41.248 INFO:tasks.workunit.client.1.smithi121.stdout:7/458: dwrite d0/d5/d11/d5c/f64 [0,4194304] 0 2024-02-15T18:35:41.249 INFO:tasks.workunit.client.1.smithi121.stdout:1/740: write d0/d39/d7d/dc0/f93 [502690,96122] 0 2024-02-15T18:35:41.249 INFO:tasks.workunit.client.1.smithi121.stdout:1/741: truncate d0/d6/d7/d9/d14/d3e/fd0 391979 0 2024-02-15T18:35:41.249 INFO:tasks.workunit.client.1.smithi121.stdout:1/742: fdatasync d0/f1d 0 2024-02-15T18:35:41.251 INFO:tasks.workunit.client.1.smithi121.stdout:7/459: getdents d0/d38/d46 0 2024-02-15T18:35:41.252 INFO:tasks.workunit.client.1.smithi121.stdout:7/460: getdents d0/d38/d3c 0 2024-02-15T18:35:41.252 INFO:tasks.workunit.client.1.smithi121.stdout:7/461: stat d0/d38/d3c/f4d 0 2024-02-15T18:35:41.253 INFO:tasks.workunit.client.1.smithi121.stdout:7/462: rmdir d0/d5/d4c 39 2024-02-15T18:35:41.263 INFO:tasks.workunit.client.1.smithi121.stdout:9/786: dwrite d7/f1c [4194304,4194304] 0 2024-02-15T18:35:41.263 INFO:tasks.workunit.client.1.smithi121.stdout:9/787: read - d7/d3a/d8e/d9e/dc7/d101/f109 zero size 2024-02-15T18:35:41.263 INFO:tasks.workunit.client.1.smithi121.stdout:9/788: readlink d7/d8/d40/d4b/lb6 0 2024-02-15T18:35:41.263 INFO:tasks.workunit.client.1.smithi121.stdout:9/789: fdatasync d7/f23 0 2024-02-15T18:35:41.266 INFO:tasks.workunit.client.1.smithi121.stdout:9/790: rename d7/d3a/fab to d7/d8/d36/df9/f112 0 2024-02-15T18:35:41.266 INFO:tasks.workunit.client.1.smithi121.stdout:7/463: dread d0/d5/d11/d1c/f23 [0,4194304] 0 2024-02-15T18:35:41.267 INFO:tasks.workunit.client.1.smithi121.stdout:9/791: creat d7/f113 x:0 0 0 2024-02-15T18:35:41.268 INFO:tasks.workunit.client.1.smithi121.stdout:7/464: mknod d0/c98 0 2024-02-15T18:35:41.273 INFO:tasks.workunit.client.1.smithi121.stdout:9/792: unlink d7/d8/d25/d30/d33/d79/fd3 0 2024-02-15T18:35:41.273 INFO:tasks.workunit.client.1.smithi121.stdout:9/793: read - d7/d26/d4f/d5b/d77/fe6 zero size 2024-02-15T18:35:41.273 INFO:tasks.workunit.client.1.smithi121.stdout:7/465: symlink d0/d5/d11/d67/l99 0 2024-02-15T18:35:41.274 INFO:tasks.workunit.client.1.smithi121.stdout:1/743: dwrite d0/d6/d7/d9/d14/f58 [0,4194304] 0 2024-02-15T18:35:41.275 INFO:tasks.workunit.client.1.smithi121.stdout:1/744: fdatasync d0/d6/d7/d9/d51/f6c 0 2024-02-15T18:35:41.276 INFO:tasks.workunit.client.1.smithi121.stdout:9/794: rename d7/d3a/d8e/d9e/cf8 to d7/d8/d25/d30/d33/d8f/dd4/c114 0 2024-02-15T18:35:41.276 INFO:tasks.workunit.client.1.smithi121.stdout:9/795: fsync d7/d8/d40/f70 0 2024-02-15T18:35:41.277 INFO:tasks.workunit.client.1.smithi121.stdout:7/466: getdents d0/d33 0 2024-02-15T18:35:41.277 INFO:tasks.workunit.client.1.smithi121.stdout:7/467: write d0/d5/d11/d1c/d5f/f82 [812726,81563] 0 2024-02-15T18:35:41.278 INFO:tasks.workunit.client.1.smithi121.stdout:9/796: truncate d7/d8/d40/f63 1380154 0 2024-02-15T18:35:41.279 INFO:tasks.workunit.client.1.smithi121.stdout:7/468: mkdir d0/d4a/d5b/d9a 0 2024-02-15T18:35:41.280 INFO:tasks.workunit.client.1.smithi121.stdout:9/797: mknod d7/d3a/d8e/d9e/dc7/d101/c115 0 2024-02-15T18:35:41.281 INFO:tasks.workunit.client.1.smithi121.stdout:7/469: rename d0/d5/f2b to d0/d5/d11/d79/f9b 0 2024-02-15T18:35:41.290 INFO:tasks.workunit.client.1.smithi121.stdout:7/470: dread d0/d38/d46/f87 [0,4194304] 0 2024-02-15T18:35:41.290 INFO:tasks.workunit.client.1.smithi121.stdout:7/471: chown d0/d5/d11/d1c 3 1 2024-02-15T18:35:41.291 INFO:tasks.workunit.client.1.smithi121.stdout:7/472: mknod d0/d38/d46/c9c 0 2024-02-15T18:35:41.292 INFO:tasks.workunit.client.1.smithi121.stdout:7/473: dread d0/d33/f34 [0,4194304] 0 2024-02-15T18:35:41.294 INFO:tasks.workunit.client.1.smithi121.stdout:7/474: creat d0/d4a/d5b/d9a/f9d x:0 0 0 2024-02-15T18:35:41.298 INFO:tasks.workunit.client.1.smithi121.stdout:9/798: dwrite d7/d8/d36/fc5 [0,4194304] 0 2024-02-15T18:35:41.298 INFO:tasks.workunit.client.1.smithi121.stdout:9/799: dread - d7/d3a/d8e/d9e/da9/d8b/f106 zero size 2024-02-15T18:35:41.300 INFO:tasks.workunit.client.1.smithi121.stdout:9/800: mkdir d7/d26/d4f/d116 0 2024-02-15T18:35:41.301 INFO:tasks.workunit.client.1.smithi121.stdout:9/801: creat d7/d26/d4f/d5b/f117 x:0 0 0 2024-02-15T18:35:41.303 INFO:tasks.workunit.client.1.smithi121.stdout:9/802: link d7/d3a/d8e/d9e/fcb d7/d3a/d8e/f118 0 2024-02-15T18:35:41.303 INFO:tasks.workunit.client.1.smithi121.stdout:9/803: mknod d7/d3a/d8e/d9e/da9/d8b/c119 0 2024-02-15T18:35:41.304 INFO:tasks.workunit.client.1.smithi121.stdout:9/804: write d7/d3a/d8e/d9e/da9/d8b/d73/fad [499993,108357] 0 2024-02-15T18:35:41.315 INFO:tasks.workunit.client.1.smithi121.stdout:7/475: dwrite d0/d33/f5d [0,4194304] 0 2024-02-15T18:35:41.315 INFO:tasks.workunit.client.1.smithi121.stdout:7/476: read - d0/d5/d4c/d6f/f94 zero size 2024-02-15T18:35:41.315 INFO:tasks.workunit.client.1.smithi121.stdout:7/477: chown d0/d5/d11/d79/f86 3 1 2024-02-15T18:35:41.315 INFO:tasks.workunit.client.1.smithi121.stdout:7/478: write d0/d5/d11/f71 [966763,84886] 0 2024-02-15T18:35:41.315 INFO:tasks.workunit.client.1.smithi121.stdout:7/479: write d0/f69 [955611,77448] 0 2024-02-15T18:35:41.317 INFO:tasks.workunit.client.1.smithi121.stdout:7/480: symlink d0/d38/l9e 0 2024-02-15T18:35:41.317 INFO:tasks.workunit.client.1.smithi121.stdout:7/481: chown d0/d38/d61/f8a 9387749 1 2024-02-15T18:35:41.318 INFO:tasks.workunit.client.1.smithi121.stdout:7/482: rename d0/d5/d11/f71 to d0/d38/d3c/f9f 0 2024-02-15T18:35:41.318 INFO:tasks.workunit.client.1.smithi121.stdout:7/483: stat d0/l49 0 2024-02-15T18:35:41.325 INFO:tasks.workunit.client.1.smithi121.stdout:9/805: dwrite d7/d3a/fdc [0,4194304] 0 2024-02-15T18:35:41.325 INFO:tasks.workunit.client.1.smithi121.stdout:9/806: fsync d7/d3a/d8e/f118 0 2024-02-15T18:35:41.325 INFO:tasks.workunit.client.1.smithi121.stdout:1/745: link d0/d6/d7/d9/d14/l49 d0/d6/d7/d50/lfa 0 2024-02-15T18:35:41.327 INFO:tasks.workunit.client.1.smithi121.stdout:9/807: write d7/d8/d36/f2f [3516952,114033] 0 2024-02-15T18:35:41.327 INFO:tasks.workunit.client.1.smithi121.stdout:9/808: fsync d7/d8/d25/d30/d33/f88 0 2024-02-15T18:35:41.327 INFO:tasks.workunit.client.1.smithi121.stdout:7/484: getdents d0/d38/d3c 0 2024-02-15T18:35:41.332 INFO:tasks.workunit.client.1.smithi121.stdout:1/746: creat d0/d6/d7/d9/d14/d3e/d88/da6/df7/ffb x:0 0 0 2024-02-15T18:35:41.333 INFO:tasks.workunit.client.1.smithi121.stdout:9/809: rename d7/d8/f47 to d7/df6/f11a 0 2024-02-15T18:35:41.336 INFO:tasks.workunit.client.1.smithi121.stdout:7/485: creat d0/d38/fa0 x:0 0 0 2024-02-15T18:35:41.336 INFO:tasks.workunit.client.1.smithi121.stdout:9/810: write d7/d8/f16 [1414425,51413] 0 2024-02-15T18:35:41.336 INFO:tasks.workunit.client.1.smithi121.stdout:9/811: stat d7/d3a/d8e/d9e/dc7/d101/c115 0 2024-02-15T18:35:41.342 INFO:tasks.workunit.client.1.smithi121.stdout:7/486: rmdir d0/d5 39 2024-02-15T18:35:41.345 INFO:tasks.workunit.client.1.smithi121.stdout:9/812: rename d7/f23 to d7/d8/d36/df9/f11b 0 2024-02-15T18:35:41.347 INFO:tasks.workunit.client.1.smithi121.stdout:7/487: mknod d0/d38/ca1 0 2024-02-15T18:35:41.348 INFO:tasks.workunit.client.1.smithi121.stdout:9/813: mknod d7/d8/d36/c11c 0 2024-02-15T18:35:41.349 INFO:tasks.workunit.client.1.smithi121.stdout:7/488: mkdir d0/d4a/da2 0 2024-02-15T18:35:41.350 INFO:tasks.workunit.client.1.smithi121.stdout:7/489: rmdir d0/d5/d4c/d6f 39 2024-02-15T18:35:41.350 INFO:tasks.workunit.client.1.smithi121.stdout:7/490: dread - d0/d5/d11/f91 zero size 2024-02-15T18:35:41.350 INFO:tasks.workunit.client.1.smithi121.stdout:7/491: chown d0/d5/d11/d67/c8c 168259941 1 2024-02-15T18:35:41.351 INFO:tasks.workunit.client.1.smithi121.stdout:1/747: dwrite d0/d6/d7/f4c [0,4194304] 0 2024-02-15T18:35:41.352 INFO:tasks.workunit.client.1.smithi121.stdout:1/748: truncate d0/d6/d7/d9/d14/db6/fe1 335983 0 2024-02-15T18:35:41.353 INFO:tasks.workunit.client.1.smithi121.stdout:7/492: unlink d0/d38/d61/f72 0 2024-02-15T18:35:41.353 INFO:tasks.workunit.client.1.smithi121.stdout:7/493: write d0/d38/d46/f87 [1230202,27506] 0 2024-02-15T18:35:41.358 INFO:tasks.workunit.client.1.smithi121.stdout:1/749: unlink d0/d6/f6e 0 2024-02-15T18:35:41.358 INFO:tasks.workunit.client.1.smithi121.stdout:1/750: chown d0/d6/d7/d9/f35 0 1 2024-02-15T18:35:41.358 INFO:tasks.workunit.client.1.smithi121.stdout:9/814: dread d7/d8/f16 [0,4194304] 0 2024-02-15T18:35:41.363 INFO:tasks.workunit.client.1.smithi121.stdout:1/751: mknod d0/d6/d7/cfc 0 2024-02-15T18:35:41.363 INFO:tasks.workunit.client.1.smithi121.stdout:1/752: write d0/d39/d7d/dc0/fe5 [721896,80295] 0 2024-02-15T18:35:41.364 INFO:tasks.workunit.client.1.smithi121.stdout:7/494: dread d0/f18 [0,4194304] 0 2024-02-15T18:35:41.373 INFO:tasks.workunit.client.1.smithi121.stdout:9/815: write d7/d3a/d8e/d9e/da9/d8b/f53 [595534,54143] 0 2024-02-15T18:35:41.373 INFO:tasks.workunit.client.1.smithi121.stdout:9/816: fdatasync d7/d8/d25/d30/d33/d79/fde 0 2024-02-15T18:35:41.374 INFO:tasks.workunit.client.1.smithi121.stdout:9/817: mkdir d7/d3a/d8e/d9e/dc7/d11d 0 2024-02-15T18:35:41.377 INFO:tasks.workunit.client.1.smithi121.stdout:9/818: unlink d7/d3a/d8e/d9e/fcb 0 2024-02-15T18:35:41.382 INFO:tasks.workunit.client.1.smithi121.stdout:5/819: sync 2024-02-15T18:35:41.382 INFO:tasks.workunit.client.1.smithi121.stdout:4/642: sync 2024-02-15T18:35:41.383 INFO:tasks.workunit.client.1.smithi121.stdout:7/495: dwrite d0/d5/d11/d1c/d5f/f73 [0,4194304] 0 2024-02-15T18:35:41.383 INFO:tasks.workunit.client.1.smithi121.stdout:4/643: write d5/d31/d3d/d4a/f60 [690079,105324] 0 2024-02-15T18:35:41.383 INFO:tasks.workunit.client.1.smithi121.stdout:4/644: chown d5/d31/d3d/d4f/d51/d63 36523140 1 2024-02-15T18:35:41.384 INFO:tasks.workunit.client.1.smithi121.stdout:5/820: truncate d0/d2/f7b 2845851 0 2024-02-15T18:35:41.384 INFO:tasks.workunit.client.1.smithi121.stdout:5/821: fdatasync d0/d2/d18/f9e 0 2024-02-15T18:35:41.385 INFO:tasks.workunit.client.1.smithi121.stdout:4/645: write d5/f64 [127156,66221] 0 2024-02-15T18:35:41.385 INFO:tasks.workunit.client.1.smithi121.stdout:4/646: readlink d5/d9/d19/d2a/d4d/lb8 0 2024-02-15T18:35:41.387 INFO:tasks.workunit.client.1.smithi121.stdout:7/496: dread d0/d5/f5e [0,4194304] 0 2024-02-15T18:35:41.388 INFO:tasks.workunit.client.1.smithi121.stdout:5/822: symlink d0/d2/d1e/d4e/d7d/d39/df2/l10f 0 2024-02-15T18:35:41.390 INFO:tasks.workunit.client.1.smithi121.stdout:4/647: mkdir d5/d9/d19/d4c/d9a/dcc 0 2024-02-15T18:35:41.390 INFO:tasks.workunit.client.1.smithi121.stdout:7/497: creat d0/d5/d11/d5c/fa3 x:0 0 0 2024-02-15T18:35:41.390 INFO:tasks.workunit.client.1.smithi121.stdout:7/498: readlink d0/l49 0 2024-02-15T18:35:41.397 INFO:tasks.workunit.client.1.smithi121.stdout:5/823: dread d0/f5f [4194304,4194304] 0 2024-02-15T18:35:41.398 INFO:tasks.workunit.client.1.smithi121.stdout:9/819: dwrite d7/d26/fd6 [0,4194304] 0 2024-02-15T18:35:41.400 INFO:tasks.workunit.client.1.smithi121.stdout:9/820: unlink d7/d8/d25/d30/d33/d79/l104 0 2024-02-15T18:35:41.401 INFO:tasks.workunit.client.1.smithi121.stdout:4/648: dread d5/d9/d76/d9e/f38 [0,4194304] 0 2024-02-15T18:35:41.403 INFO:tasks.workunit.client.1.smithi121.stdout:9/821: unlink d7/d26/dba/l76 0 2024-02-15T18:35:41.404 INFO:tasks.workunit.client.1.smithi121.stdout:9/822: read d7/d8/fbe [3735893,32076] 0 2024-02-15T18:35:41.404 INFO:tasks.workunit.client.1.smithi121.stdout:9/823: chown d7/dcc/ldf 128557 1 2024-02-15T18:35:41.404 INFO:tasks.workunit.client.1.smithi121.stdout:5/824: symlink d0/d2/d1e/d4e/d7d/d47/d5a/dbc/l110 0 2024-02-15T18:35:41.408 INFO:tasks.workunit.client.1.smithi121.stdout:4/649: truncate d5/d9/d1d/f41 153555 0 2024-02-15T18:35:41.414 INFO:tasks.workunit.client.1.smithi121.stdout:5/825: unlink d0/d2/d1e/d4e/d7d/d9d/d4c/daa/fd1 0 2024-02-15T18:35:41.417 INFO:tasks.workunit.client.1.smithi121.stdout:5/826: getdents d0/d2/d1e/d4e/d7d/d47/d5a/db3 0 2024-02-15T18:35:41.417 INFO:tasks.workunit.client.1.smithi121.stdout:5/827: fsync d0/d2/d1e/d4e/d7d/d47/f64 0 2024-02-15T18:35:41.417 INFO:tasks.workunit.client.1.smithi121.stdout:5/828: truncate d0/d16/f98 4567903 0 2024-02-15T18:35:41.424 INFO:tasks.workunit.client.1.smithi121.stdout:4/650: dwrite d5/d9/d1d/f24 [0,4194304] 0 2024-02-15T18:35:41.442 INFO:tasks.workunit.client.1.smithi121.stdout:5/829: dwrite d0/d2/d1e/d4e/d7d/d9d/d4c/fd9 [0,4194304] 0 2024-02-15T18:35:41.449 INFO:tasks.workunit.client.1.smithi121.stdout:4/651: dwrite d5/d9/d76/d9e/d14/f55 [0,4194304] 0 2024-02-15T18:35:41.453 INFO:tasks.workunit.client.1.smithi121.stdout:5/830: dread d0/d2/d1e/d4e/d7d/d9d/d4c/fd9 [0,4194304] 0 2024-02-15T18:35:41.453 INFO:tasks.workunit.client.1.smithi121.stdout:5/831: write d0/d2/f14 [857337,126332] 0 2024-02-15T18:35:41.453 INFO:tasks.workunit.client.1.smithi121.stdout:5/832: readlink d0/d2/l11 0 2024-02-15T18:35:41.453 INFO:tasks.workunit.client.1.smithi121.stdout:5/833: truncate d0/f89 570016 0 2024-02-15T18:35:41.465 INFO:tasks.workunit.client.1.smithi121.stdout:5/834: dread d0/d2/f8e [0,4194304] 0 2024-02-15T18:35:41.465 INFO:tasks.workunit.client.1.smithi121.stdout:5/835: fdatasync d0/d2/f7b 0 2024-02-15T18:35:41.465 INFO:tasks.workunit.client.1.smithi121.stdout:5/836: write d0/d2/d18/f10d [823981,113465] 0 2024-02-15T18:35:41.471 INFO:tasks.workunit.client.1.smithi121.stdout:5/837: mknod d0/d2/d1e/d4e/d7d/d39/c111 0 2024-02-15T18:35:41.474 INFO:tasks.workunit.client.1.smithi121.stdout:4/652: dwrite d5/d9/d76/d87/db4/fae [0,4194304] 0 2024-02-15T18:35:41.476 INFO:tasks.workunit.client.1.smithi121.stdout:5/838: write d0/d16/d7a/db5/fcb [3715577,88667] 0 2024-02-15T18:35:41.476 INFO:tasks.workunit.client.1.smithi121.stdout:5/839: getdents d0/d2/d1e/d4e/d7d/d9d 0 2024-02-15T18:35:41.476 INFO:tasks.workunit.client.1.smithi121.stdout:4/653: dread d5/f64 [0,4194304] 0 2024-02-15T18:35:41.477 INFO:tasks.workunit.client.1.smithi121.stdout:5/840: read d0/d2/faf [4882727,60264] 0 2024-02-15T18:35:41.477 INFO:tasks.workunit.client.1.smithi121.stdout:5/841: fsync d0/d2/d1e/d4e/d7d/d47/d5a/f5c 0 2024-02-15T18:35:41.480 INFO:tasks.workunit.client.1.smithi121.stdout:4/654: unlink d5/d31/d3d/d4f/d8a/faa 0 2024-02-15T18:35:41.480 INFO:tasks.workunit.client.1.smithi121.stdout:4/655: write d5/d9/d19/d28/fb3 [462093,58087] 0 2024-02-15T18:35:41.483 INFO:tasks.workunit.client.1.smithi121.stdout:5/842: mknod d0/d2/d1e/d4e/d7d/d47/d5a/da7/c112 0 2024-02-15T18:35:41.484 INFO:tasks.workunit.client.1.smithi121.stdout:5/843: truncate d0/d16/d34/d8f/f90 9368647 0 2024-02-15T18:35:41.488 INFO:tasks.workunit.client.1.smithi121.stdout:4/656: dread d5/d31/d3d/d4f/d51/d71/f48 [0,4194304] 0 2024-02-15T18:35:41.519 INFO:tasks.workunit.client.1.smithi121.stdout:5/844: mknod d0/d2/d1e/d4e/d7d/c113 0 2024-02-15T18:35:41.521 INFO:tasks.workunit.client.1.smithi121.stdout:5/845: write d0/d2/f14 [255695,58800] 0 2024-02-15T18:35:41.524 INFO:tasks.workunit.client.1.smithi121.stdout:5/846: link d0/d2/d1e/d4e/d7d/d9d/d4c/daa/lab d0/d16/d34/l114 0 2024-02-15T18:35:41.526 INFO:tasks.workunit.client.1.smithi121.stdout:5/847: dread d0/d2/d1e/d4e/d7d/d47/d5a/da7/f103 [0,4194304] 0 2024-02-15T18:35:41.549 INFO:tasks.workunit.client.1.smithi121.stdout:3/498: sync 2024-02-15T18:35:41.549 INFO:tasks.workunit.client.1.smithi121.stdout:8/617: sync 2024-02-15T18:35:41.549 INFO:tasks.workunit.client.1.smithi121.stdout:8/618: chown d0/d61/d95/f9c 359623301 1 2024-02-15T18:35:41.549 INFO:tasks.workunit.client.1.smithi121.stdout:3/499: truncate d2/d1a/d27/d7c/d25/d31/d34/d4a/f9c 388837 0 2024-02-15T18:35:41.549 INFO:tasks.workunit.client.1.smithi121.stdout:8/619: read - d0/d61/d7b/fb2 zero size 2024-02-15T18:35:41.549 INFO:tasks.workunit.client.1.smithi121.stdout:3/500: chown d2/d1a/d27/d7c/f7b 2966 1 2024-02-15T18:35:41.549 INFO:tasks.workunit.client.1.smithi121.stdout:8/620: chown d0/d2a/d51/c41 0 1 2024-02-15T18:35:41.550 INFO:tasks.workunit.client.1.smithi121.stdout:3/501: fdatasync d2/d1a/d27/d7c/d25/d31/f50 0 2024-02-15T18:35:41.550 INFO:tasks.workunit.client.1.smithi121.stdout:3/502: fsync d2/d1a/d27/f7f 0 2024-02-15T18:35:41.550 INFO:tasks.workunit.client.1.smithi121.stdout:3/503: chown d2/d1a/d22/f47 10804157 1 2024-02-15T18:35:41.550 INFO:tasks.workunit.client.1.smithi121.stdout:0/799: sync 2024-02-15T18:35:41.550 INFO:tasks.workunit.client.1.smithi121.stdout:6/748: sync 2024-02-15T18:35:41.553 INFO:tasks.workunit.client.1.smithi121.stdout:0/800: dread d5/d6b/dab/db4/d7d/dd/f4b [0,4194304] 0 2024-02-15T18:35:41.553 INFO:tasks.workunit.client.1.smithi121.stdout:0/801: chown d5/d6b/dab/db1/dbf 0 1 2024-02-15T18:35:41.553 INFO:tasks.workunit.client.1.smithi121.stdout:6/749: mkdir d7/dc/d34/d42/d62/dac/d10a 0 2024-02-15T18:35:41.553 INFO:tasks.workunit.client.1.smithi121.stdout:6/750: truncate d7/d4b/d60/d6c/dd5/dfa/f97 27993 0 2024-02-15T18:35:41.556 INFO:tasks.workunit.client.1.smithi121.stdout:5/848: rename d0/d2/d18/f31 to d0/d16/d7a/f115 0 2024-02-15T18:35:41.556 INFO:tasks.workunit.client.1.smithi121.stdout:5/849: write d0/d2/d1e/d4e/d7d/fb8 [4797379,69919] 0 2024-02-15T18:35:41.557 INFO:tasks.workunit.client.1.smithi121.stdout:6/751: creat d7/dc/d34/d42/d62/d63/d6f/f10b x:0 0 0 2024-02-15T18:35:41.558 INFO:tasks.workunit.client.1.smithi121.stdout:3/504: rename d2/f40 to d2/d1a/d27/fa2 0 2024-02-15T18:35:41.558 INFO:tasks.workunit.client.1.smithi121.stdout:3/505: truncate d2/d5a/f78 5141534 0 2024-02-15T18:35:41.558 INFO:tasks.workunit.client.1.smithi121.stdout:3/506: write d2/d1a/f7a [427880,21702] 0 2024-02-15T18:35:41.561 INFO:tasks.workunit.client.1.smithi121.stdout:0/802: unlink d5/d6b/dab/db4/d7d/dd/l28 0 2024-02-15T18:35:41.562 INFO:tasks.workunit.client.1.smithi121.stdout:6/752: rename d7/dc/d2c/l5a to d7/dc/d34/d42/d62/d63/d6f/db9/l10c 0 2024-02-15T18:35:41.562 INFO:tasks.workunit.client.1.smithi121.stdout:6/753: fsync d7/d4b/d60/d6c/dd5/dfa/fc3 0 2024-02-15T18:35:41.562 INFO:tasks.workunit.client.1.smithi121.stdout:3/507: creat d2/d1a/d22/d85/fa3 x:0 0 0 2024-02-15T18:35:41.562 INFO:tasks.workunit.client.1.smithi121.stdout:3/508: chown d2/d1a/d22/d85 32104 1 2024-02-15T18:35:41.563 INFO:tasks.workunit.client.1.smithi121.stdout:0/803: symlink d5/d6b/dab/db4/d7d/d74/d96/l112 0 2024-02-15T18:35:41.563 INFO:tasks.workunit.client.1.smithi121.stdout:0/804: chown d5/d6b/dab/db4/d7d/dd/c10d 80849178 1 2024-02-15T18:35:41.563 INFO:tasks.workunit.client.1.smithi121.stdout:0/805: stat d5/d6b/dab/db4/d7d/d2a/f95 0 2024-02-15T18:35:41.566 INFO:tasks.workunit.client.1.smithi121.stdout:0/806: creat d5/d107/d109/f113 x:0 0 0 2024-02-15T18:35:41.566 INFO:tasks.workunit.client.1.smithi121.stdout:5/850: dread d0/d16/d34/d8f/ff6 [0,4194304] 0 2024-02-15T18:35:41.568 INFO:tasks.workunit.client.1.smithi121.stdout:6/754: mknod d7/dc/d34/d42/d62/d63/d66/dce/c10d 0 2024-02-15T18:35:41.568 INFO:tasks.workunit.client.1.smithi121.stdout:3/509: write d2/d1a/f89 [831045,113491] 0 2024-02-15T18:35:41.569 INFO:tasks.workunit.client.1.smithi121.stdout:8/621: dwrite d0/d61/d95/f9c [0,4194304] 0 2024-02-15T18:35:41.572 INFO:tasks.workunit.client.1.smithi121.stdout:5/851: dread d0/d16/f94 [0,4194304] 0 2024-02-15T18:35:41.574 INFO:tasks.workunit.client.1.smithi121.stdout:3/510: truncate d2/d1a/d27/d7c/d25/d31/d58/f61 2064087 0 2024-02-15T18:35:41.574 INFO:tasks.workunit.client.1.smithi121.stdout:3/511: readlink d2/d1a/l1e 0 2024-02-15T18:35:41.574 INFO:tasks.workunit.client.1.smithi121.stdout:0/807: mknod d5/d6b/dab/db4/dca/c114 0 2024-02-15T18:35:41.574 INFO:tasks.workunit.client.1.smithi121.stdout:0/808: chown d5/d30/d5d/fe1 208441090 1 2024-02-15T18:35:41.575 INFO:tasks.workunit.client.1.smithi121.stdout:6/755: creat d7/dc/d34/d42/d62/d63/d6f/d89/f10e x:0 0 0 2024-02-15T18:35:41.575 INFO:tasks.workunit.client.1.smithi121.stdout:6/756: write d7/dc/d34/d42/d62/d63/d6f/fb1 [5161438,50788] 0 2024-02-15T18:35:41.576 INFO:tasks.workunit.client.1.smithi121.stdout:5/852: mkdir d0/d16/d7a/db5/d116 0 2024-02-15T18:35:41.576 INFO:tasks.workunit.client.1.smithi121.stdout:8/622: creat d0/d61/d95/fd7 x:0 0 0 2024-02-15T18:35:41.576 INFO:tasks.workunit.client.1.smithi121.stdout:8/623: truncate d0/d7/d36/d52/fc7 368820 0 2024-02-15T18:35:41.577 INFO:tasks.workunit.client.1.smithi121.stdout:8/624: chown d0/d7/d36/d52 231385 1 2024-02-15T18:35:41.577 INFO:tasks.workunit.client.1.smithi121.stdout:5/853: write d0/f4b [740660,43835] 0 2024-02-15T18:35:41.584 INFO:tasks.workunit.client.1.smithi121.stdout:8/625: link d0/d2a/f4b d0/d61/d7b/fd8 0 2024-02-15T18:35:41.585 INFO:tasks.workunit.client.1.smithi121.stdout:5/854: dread d0/d16/d7a/f115 [0,4194304] 0 2024-02-15T18:35:41.588 INFO:tasks.workunit.client.1.smithi121.stdout:5/855: creat d0/d16/d34/da4/f117 x:0 0 0 2024-02-15T18:35:41.592 INFO:tasks.workunit.client.1.smithi121.stdout:8/626: creat d0/d7/de/fd9 x:0 0 0 2024-02-15T18:35:41.599 INFO:tasks.workunit.client.1.smithi121.stdout:3/512: rename d2/d1a/d27/d7c to d2/d1a/d27/d63/da4 0 2024-02-15T18:35:41.601 INFO:tasks.workunit.client.1.smithi121.stdout:8/627: link d0/l24 d0/d7/d36/d8c/d45/dcd/lda 0 2024-02-15T18:35:41.603 INFO:tasks.workunit.client.1.smithi121.stdout:3/513: creat d2/d1a/d6b/fa5 x:0 0 0 2024-02-15T18:35:41.603 INFO:tasks.workunit.client.1.smithi121.stdout:3/514: fsync d2/d1a/d22/f47 0 2024-02-15T18:35:41.603 INFO:tasks.workunit.client.1.smithi121.stdout:3/515: fsync d2/d1a/d27/fa2 0 2024-02-15T18:35:41.603 INFO:tasks.workunit.client.1.smithi121.stdout:3/516: dread - d2/d1a/d27/d63/da4/d25/d31/d34/d4a/fa0 zero size 2024-02-15T18:35:41.603 INFO:tasks.workunit.client.1.smithi121.stdout:3/517: dread - d2/d1a/d6b/fa5 zero size 2024-02-15T18:35:41.604 INFO:tasks.workunit.client.1.smithi121.stdout:8/628: creat d0/d7/fdb x:0 0 0 2024-02-15T18:35:41.609 INFO:tasks.workunit.client.1.smithi121.stdout:3/518: unlink d2/d1a/d27/d63/da4/d25/d31/d34/d4a/f8d 0 2024-02-15T18:35:41.609 INFO:tasks.workunit.client.1.smithi121.stdout:3/519: write d2/d5a/f95 [125863,72204] 0 2024-02-15T18:35:41.609 INFO:tasks.workunit.client.1.smithi121.stdout:3/520: stat d2/d1a/d27/f4b 0 2024-02-15T18:35:41.610 INFO:tasks.workunit.client.1.smithi121.stdout:3/521: mknod d2/d1a/d27/d63/ca6 0 2024-02-15T18:35:41.613 INFO:tasks.workunit.client.1.smithi121.stdout:3/522: rename d2/d1a/d22 to d2/d1a/d27/da7 0 2024-02-15T18:35:41.613 INFO:tasks.workunit.client.1.smithi121.stdout:3/523: write d2/d1a/d27/f84 [272333,106735] 0 2024-02-15T18:35:41.614 INFO:tasks.workunit.client.1.smithi121.stdout:3/524: mkdir d2/d1a/da8 0 2024-02-15T18:35:41.616 INFO:tasks.workunit.client.1.smithi121.stdout:5/856: dwrite d0/d2/d1e/d4e/d7d/d39/f107 [0,4194304] 0 2024-02-15T18:35:41.616 INFO:tasks.workunit.client.1.smithi121.stdout:3/525: symlink d2/d1a/d27/d63/da4/d25/d31/la9 0 2024-02-15T18:35:41.616 INFO:tasks.workunit.client.1.smithi121.stdout:3/526: chown d2/d1a/d6b/f91 1940 1 2024-02-15T18:35:41.620 INFO:tasks.workunit.client.1.smithi121.stdout:6/757: dwrite d7/dc/d34/d42/d62/dac/fe5 [0,4194304] 0 2024-02-15T18:35:41.620 INFO:tasks.workunit.client.1.smithi121.stdout:6/758: truncate d7/f31 4383247 0 2024-02-15T18:35:41.621 INFO:tasks.workunit.client.1.smithi121.stdout:0/809: dwrite d5/d30/d5d/f5e [0,4194304] 0 2024-02-15T18:35:41.623 INFO:tasks.workunit.client.1.smithi121.stdout:8/629: dwrite d0/d7/d36/d52/fb4 [0,4194304] 0 2024-02-15T18:35:41.625 INFO:tasks.workunit.client.1.smithi121.stdout:5/857: write d0/d2/d18/f29 [1559995,45388] 0 2024-02-15T18:35:41.628 INFO:tasks.workunit.client.1.smithi121.stdout:0/810: creat d5/d6b/dab/db4/d7d/d2a/d36/d44/f115 x:0 0 0 2024-02-15T18:35:41.629 INFO:tasks.workunit.client.1.smithi121.stdout:8/630: write d0/d7/d36/d8c/d45/f3e [2622017,47329] 0 2024-02-15T18:35:41.629 INFO:tasks.workunit.client.1.smithi121.stdout:8/631: dread - d0/d7/d36/d52/d7a/f8a zero size 2024-02-15T18:35:41.630 INFO:tasks.workunit.client.1.smithi121.stdout:3/527: fdatasync d2/d5a/f95 0 2024-02-15T18:35:41.630 INFO:tasks.workunit.client.1.smithi121.stdout:3/528: fdatasync d2/d1a/f1f 0 2024-02-15T18:35:41.634 INFO:tasks.workunit.client.1.smithi121.stdout:0/811: symlink d5/d30/l116 0 2024-02-15T18:35:41.634 INFO:tasks.workunit.client.1.smithi121.stdout:0/812: fdatasync d5/d6b/dab/db4/d7d/d2a/fda 0 2024-02-15T18:35:41.634 INFO:tasks.workunit.client.1.smithi121.stdout:0/813: write d5/d6b/dab/db4/d7d/f43 [417124,39791] 0 2024-02-15T18:35:41.640 INFO:tasks.workunit.client.1.smithi121.stdout:3/529: symlink d2/d1a/d27/d63/da4/d25/laa 0 2024-02-15T18:35:41.642 INFO:tasks.workunit.client.1.smithi121.stdout:0/814: truncate d5/fa4 336892 0 2024-02-15T18:35:41.642 INFO:tasks.workunit.client.1.smithi121.stdout:0/815: chown d5/d6b/dab/db4/d7d/d2a/d73/d97/df4 2045592 1 2024-02-15T18:35:41.642 INFO:tasks.workunit.client.1.smithi121.stdout:0/816: fsync d5/d6b/dab/db4/d7d/d2a/d4e/f5a 0 2024-02-15T18:35:41.642 INFO:tasks.workunit.client.1.smithi121.stdout:3/530: mknod d2/d1a/d27/d63/da4/d25/d88/d9a/cab 0 2024-02-15T18:35:41.643 INFO:tasks.workunit.client.1.smithi121.stdout:6/759: dwrite d7/d4b/d60/d6c/dd5/dfa/f9f [0,4194304] 0 2024-02-15T18:35:41.643 INFO:tasks.workunit.client.1.smithi121.stdout:6/760: read - d7/dc/d34/d42/d62/d63/d6f/f10b zero size 2024-02-15T18:35:41.644 INFO:tasks.workunit.client.1.smithi121.stdout:8/632: dread d0/d7/d36/f81 [0,4194304] 0 2024-02-15T18:35:41.645 INFO:tasks.workunit.client.1.smithi121.stdout:3/531: stat d2/c66 0 2024-02-15T18:35:41.647 INFO:tasks.workunit.client.1.smithi121.stdout:0/817: link d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/fd3 d5/d6b/dab/db1/dbf/f117 0 2024-02-15T18:35:41.647 INFO:tasks.workunit.client.1.smithi121.stdout:0/818: readlink d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/l72 0 2024-02-15T18:35:41.647 INFO:tasks.workunit.client.1.smithi121.stdout:0/819: write d5/d6b/dab/db1/dbf/f117 [562769,125116] 0 2024-02-15T18:35:41.649 INFO:tasks.workunit.client.1.smithi121.stdout:6/761: truncate d7/dc/d34/d42/f54 2781935 0 2024-02-15T18:35:41.650 INFO:tasks.workunit.client.1.smithi121.stdout:6/762: fsync d7/dc/d34/d42/d62/d63/d6f/f88 0 2024-02-15T18:35:41.650 INFO:tasks.workunit.client.1.smithi121.stdout:3/532: write d2/d1a/d27/d63/da4/d25/d31/f57 [6740807,40391] 0 2024-02-15T18:35:41.654 INFO:tasks.workunit.client.1.smithi121.stdout:3/533: creat d2/d1a/d27/d63/d81/fac x:0 0 0 2024-02-15T18:35:41.654 INFO:tasks.workunit.client.1.smithi121.stdout:6/763: readlink d7/dc/d34/d42/d62/d63/d66/dce/lcf 0 2024-02-15T18:35:41.654 INFO:tasks.workunit.client.1.smithi121.stdout:6/764: write d7/d4b/d60/d6c/dd5/dfa/d94/f9a [1322530,70265] 0 2024-02-15T18:35:41.659 INFO:tasks.workunit.client.1.smithi121.stdout:6/765: creat d7/d4b/d60/d6c/dd5/dfa/d91/f10f x:0 0 0 2024-02-15T18:35:41.659 INFO:tasks.workunit.client.1.smithi121.stdout:3/534: creat d2/d1a/d27/da7/fad x:0 0 0 2024-02-15T18:35:41.661 INFO:tasks.workunit.client.1.smithi121.stdout:3/535: creat d2/d1a/d27/d63/da4/d25/d31/d97/fae x:0 0 0 2024-02-15T18:35:41.661 INFO:tasks.workunit.client.1.smithi121.stdout:3/536: chown d2/d1a/d27/d63/da4/f56 2700 1 2024-02-15T18:35:41.662 INFO:tasks.workunit.client.1.smithi121.stdout:8/633: dread d0/d61/d66/f3b [0,4194304] 0 2024-02-15T18:35:41.664 INFO:tasks.workunit.client.1.smithi121.stdout:6/766: symlink d7/dc/d34/d42/d62/d63/d66/dce/l110 0 2024-02-15T18:35:41.664 INFO:tasks.workunit.client.1.smithi121.stdout:6/767: dread - d7/dc/d34/d42/fee zero size 2024-02-15T18:35:41.664 INFO:tasks.workunit.client.1.smithi121.stdout:3/537: mkdir d2/d1a/d27/d63/da4/d25/d31/d97/daf 0 2024-02-15T18:35:41.665 INFO:tasks.workunit.client.1.smithi121.stdout:3/538: creat d2/d1a/d27/d63/da4/d25/fb0 x:0 0 0 2024-02-15T18:35:41.666 INFO:tasks.workunit.client.1.smithi121.stdout:3/539: chown d2/d1a/d27/d63/da4/d25/d88/d9a 31677735 1 2024-02-15T18:35:41.667 INFO:tasks.workunit.client.1.smithi121.stdout:6/768: dread d7/dc/d34/f4a [0,4194304] 0 2024-02-15T18:35:41.670 INFO:tasks.workunit.client.1.smithi121.stdout:6/769: rename d7/d4b/d60/d6c/dd5/dfa/le6 to d7/dc/d34/d42/d62/dac/d10a/l111 0 2024-02-15T18:35:41.672 INFO:tasks.workunit.client.1.smithi121.stdout:6/770: creat d7/dc/d34/d42/d62/dac/de8/f112 x:0 0 0 2024-02-15T18:35:41.691 INFO:tasks.workunit.client.1.smithi121.stdout:8/634: dwrite d0/f2 [0,4194304] 0 2024-02-15T18:35:41.691 INFO:tasks.workunit.client.1.smithi121.stdout:0/820: dwrite d5/d6b/dab/db4/d7d/d2a/d73/d97/db8/fee [0,4194304] 0 2024-02-15T18:35:41.693 INFO:tasks.workunit.client.1.smithi121.stdout:6/771: dread d7/dc/d2c/d36/f38 [0,4194304] 0 2024-02-15T18:35:41.697 INFO:tasks.workunit.client.1.smithi121.stdout:5/858: dwrite d0/d2/f14 [0,4194304] 0 2024-02-15T18:35:41.697 INFO:tasks.workunit.client.1.smithi121.stdout:8/635: mknod d0/d7/d36/d52/d7a/cdc 0 2024-02-15T18:35:41.697 INFO:tasks.workunit.client.1.smithi121.stdout:8/636: chown d0/d7/d36/d8c/d45/fd3 14 1 2024-02-15T18:35:41.698 INFO:tasks.workunit.client.1.smithi121.stdout:6/772: mkdir d7/dc/d34/d42/d62/d63/d6f/db9/d113 0 2024-02-15T18:35:41.698 INFO:tasks.workunit.client.1.smithi121.stdout:6/773: readlink d7/l2b 0 2024-02-15T18:35:41.702 INFO:tasks.workunit.client.1.smithi121.stdout:8/637: write d0/d61/d66/f80 [2214043,7137] 0 2024-02-15T18:35:41.704 INFO:tasks.workunit.client.1.smithi121.stdout:3/540: dread d2/d1a/f89 [0,4194304] 0 2024-02-15T18:35:41.704 INFO:tasks.workunit.client.1.smithi121.stdout:8/638: dread d0/d7/d36/fb9 [0,4194304] 0 2024-02-15T18:35:41.716 INFO:tasks.workunit.client.1.smithi121.stdout:0/821: link d5/d6b/dab/db4/d7d/dd/fd0 d5/d30/f118 0 2024-02-15T18:35:41.716 INFO:tasks.workunit.client.1.smithi121.stdout:0/822: write d5/d6b/dab/db4/d7d/dd/f22 [259268,71041] 0 2024-02-15T18:35:41.716 INFO:tasks.workunit.client.1.smithi121.stdout:0/823: chown d5/d6b/dab/db4/d7d/d2a/d36/d44/lbe 295029346 1 2024-02-15T18:35:41.720 INFO:tasks.workunit.client.1.smithi121.stdout:3/541: mknod d2/d1a/d54/cb1 0 2024-02-15T18:35:41.720 INFO:tasks.workunit.client.1.smithi121.stdout:6/774: truncate d7/dc/d34/d42/d62/d63/d8d/fa7 765129 0 2024-02-15T18:35:41.720 INFO:tasks.workunit.client.1.smithi121.stdout:0/824: mknod d5/d6b/dab/db4/dca/c119 0 2024-02-15T18:35:41.720 INFO:tasks.workunit.client.1.smithi121.stdout:0/825: chown d5/d30/f101 1041521 1 2024-02-15T18:35:41.721 INFO:tasks.workunit.client.1.smithi121.stdout:0/826: chown d5/d6b/dab/db4/d7d/dd/f16 24 1 2024-02-15T18:35:41.721 INFO:tasks.workunit.client.1.smithi121.stdout:3/542: mknod d2/d1a/d27/d63/da4/d25/d31/d97/cb2 0 2024-02-15T18:35:41.723 INFO:tasks.workunit.client.1.smithi121.stdout:0/827: symlink d5/d107/l11a 0 2024-02-15T18:35:41.723 INFO:tasks.workunit.client.1.smithi121.stdout:0/828: fdatasync d5/d30/f33 0 2024-02-15T18:35:41.724 INFO:tasks.workunit.client.1.smithi121.stdout:6/775: getdents d7/dc/d2c/d36 0 2024-02-15T18:35:41.724 INFO:tasks.workunit.client.1.smithi121.stdout:6/776: readlink d7/d4b/d59/ld7 0 2024-02-15T18:35:41.726 INFO:tasks.workunit.client.1.smithi121.stdout:6/777: read d7/f11 [1387546,51128] 0 2024-02-15T18:35:41.728 INFO:tasks.workunit.client.1.smithi121.stdout:0/829: rename d5/d6b/dab/db4/d7d/d2a/d73/f9a to d5/d30/d5d/f11b 0 2024-02-15T18:35:41.729 INFO:tasks.workunit.client.1.smithi121.stdout:5/859: symlink d0/l118 0 2024-02-15T18:35:41.729 INFO:tasks.workunit.client.1.smithi121.stdout:8/639: dwrite d0/d7/d36/d8c/f49 [0,4194304] 0 2024-02-15T18:35:41.735 INFO:tasks.workunit.client.1.smithi121.stdout:0/830: mkdir d5/d11c 0 2024-02-15T18:35:41.738 INFO:tasks.workunit.client.1.smithi121.stdout:5/860: dread d0/d2/fd [4194304,4194304] 0 2024-02-15T18:35:41.739 INFO:tasks.workunit.client.1.smithi121.stdout:5/861: write d0/d2/d1e/f8c [306727,95856] 0 2024-02-15T18:35:41.739 INFO:tasks.workunit.client.1.smithi121.stdout:6/778: write d7/dc/d34/f26 [1787538,19130] 0 2024-02-15T18:35:41.739 INFO:tasks.workunit.client.1.smithi121.stdout:5/862: fsync d0/d2/d1e/d4e/d7d/d47/d5a/db3/ffd 0 2024-02-15T18:35:41.739 INFO:tasks.workunit.client.1.smithi121.stdout:6/779: write d7/dc/d34/d42/d62/fb0 [856489,30169] 0 2024-02-15T18:35:41.739 INFO:tasks.workunit.client.1.smithi121.stdout:8/640: creat d0/d61/d8f/fdd x:0 0 0 2024-02-15T18:35:41.739 INFO:tasks.workunit.client.1.smithi121.stdout:0/831: creat d5/d6b/dab/db4/d7d/d2a/d4e/f11d x:0 0 0 2024-02-15T18:35:41.740 INFO:tasks.workunit.client.1.smithi121.stdout:3/543: dwrite d2/d1a/d6b/f82 [0,4194304] 0 2024-02-15T18:35:41.740 INFO:tasks.workunit.client.1.smithi121.stdout:6/780: write d7/dc/d34/f86 [345123,52888] 0 2024-02-15T18:35:41.741 INFO:tasks.workunit.client.1.smithi121.stdout:5/863: truncate d0/d2/d1e/d4e/d7d/d47/d5a/f5c 1494725 0 2024-02-15T18:35:41.741 INFO:tasks.workunit.client.1.smithi121.stdout:8/641: dread d0/d61/da6/f99 [0,4194304] 0 2024-02-15T18:35:41.742 INFO:tasks.workunit.client.1.smithi121.stdout:0/832: write d5/d6b/dab/db4/d7d/d2a/d4e/f5a [7296083,98268] 0 2024-02-15T18:35:41.742 INFO:tasks.workunit.client.1.smithi121.stdout:3/544: dread d2/d1a/d27/f4b [0,4194304] 0 2024-02-15T18:35:41.742 INFO:tasks.workunit.client.1.smithi121.stdout:5/864: getdents d0/d2/d70/de1 0 2024-02-15T18:35:41.744 INFO:tasks.workunit.client.1.smithi121.stdout:3/545: write d2/d1a/d27/fa2 [7223466,56545] 0 2024-02-15T18:35:41.745 INFO:tasks.workunit.client.1.smithi121.stdout:6/781: mknod d7/dc/d2c/d36/c114 0 2024-02-15T18:35:41.747 INFO:tasks.workunit.client.1.smithi121.stdout:8/642: rmdir d0/d2a/d51 39 2024-02-15T18:35:41.747 INFO:tasks.workunit.client.1.smithi121.stdout:8/643: fdatasync d0/d7/d36/d52/d7a/f8a 0 2024-02-15T18:35:41.749 INFO:tasks.workunit.client.1.smithi121.stdout:5/865: rename d0/cde to d0/d16/d34/da4/dcc/c119 0 2024-02-15T18:35:41.750 INFO:tasks.workunit.client.1.smithi121.stdout:3/546: mkdir d2/d1a/d27/d63/da4/d25/d30/db3 0 2024-02-15T18:35:41.750 INFO:tasks.workunit.client.1.smithi121.stdout:6/782: mknod d7/dc/d34/d42/d62/d63/d8d/c115 0 2024-02-15T18:35:41.750 INFO:tasks.workunit.client.1.smithi121.stdout:6/783: chown d7/d4b/d60/d6c/dd5/dfa/d91/f10f 387210635 1 2024-02-15T18:35:41.752 INFO:tasks.workunit.client.1.smithi121.stdout:3/547: write d2/d1a/d27/da7/f64 [446932,98118] 0 2024-02-15T18:35:41.760 INFO:tasks.workunit.client.1.smithi121.stdout:5/866: symlink d0/d2/d1e/d4e/d7d/d9d/l11a 0 2024-02-15T18:35:41.765 INFO:tasks.workunit.client.1.smithi121.stdout:6/784: dread d7/dc/d34/d42/d62/d63/d6f/d89/fb7 [0,4194304] 0 2024-02-15T18:35:41.781 INFO:tasks.workunit.client.1.smithi121.stdout:3/548: dwrite d2/d1a/d27/d63/da4/d25/d31/f4e [0,4194304] 0 2024-02-15T18:35:41.781 INFO:tasks.workunit.client.1.smithi121.stdout:3/549: chown d2/d1a/f92 12506189 1 2024-02-15T18:35:41.784 INFO:tasks.workunit.client.1.smithi121.stdout:3/550: creat d2/d1a/d27/d63/da4/d25/d31/d58/fb4 x:0 0 0 2024-02-15T18:35:41.784 INFO:tasks.workunit.client.1.smithi121.stdout:8/644: dwrite d0/d7/db1/dbb/f8b [0,4194304] 0 2024-02-15T18:35:41.786 INFO:tasks.workunit.client.1.smithi121.stdout:5/867: dwrite d0/d2/d1e/d4e/f91 [0,4194304] 0 2024-02-15T18:35:41.787 INFO:tasks.workunit.client.1.smithi121.stdout:8/645: creat d0/d7/da5/dcb/fde x:0 0 0 2024-02-15T18:35:41.787 INFO:tasks.workunit.client.1.smithi121.stdout:3/551: dread d2/d1a/d6b/f79 [0,4194304] 0 2024-02-15T18:35:41.787 INFO:tasks.workunit.client.1.smithi121.stdout:3/552: chown d2/d1a/d27/da7/d85/fa3 29 1 2024-02-15T18:35:41.788 INFO:tasks.workunit.client.1.smithi121.stdout:6/785: dwrite d7/dc/f15 [8388608,4194304] 0 2024-02-15T18:35:41.796 INFO:tasks.workunit.client.1.smithi121.stdout:2/429: sync 2024-02-15T18:35:41.796 INFO:tasks.workunit.client.1.smithi121.stdout:1/753: sync 2024-02-15T18:35:41.797 INFO:tasks.workunit.client.1.smithi121.stdout:5/868: rmdir d0/d2/d70/de1 39 2024-02-15T18:35:41.801 INFO:tasks.workunit.client.1.smithi121.stdout:3/553: dread d2/f7 [0,4194304] 0 2024-02-15T18:35:41.801 INFO:tasks.workunit.client.1.smithi121.stdout:5/869: read d0/d2/d1e/d4e/d7d/d9d/d4c/f95 [147365,116957] 0 2024-02-15T18:35:41.801 INFO:tasks.workunit.client.1.smithi121.stdout:5/870: write d0/d2/faf [5893645,7980] 0 2024-02-15T18:35:41.802 INFO:tasks.workunit.client.1.smithi121.stdout:3/554: mknod d2/d1a/d54/cb5 0 2024-02-15T18:35:41.802 INFO:tasks.workunit.client.1.smithi121.stdout:3/555: read d2/d1a/f89 [3346540,54843] 0 2024-02-15T18:35:41.804 INFO:tasks.workunit.client.1.smithi121.stdout:3/556: rename d2/d1a/d54/cb1 to d2/d1a/d27/d63/da4/d25/d31/d34/cb6 0 2024-02-15T18:35:41.805 INFO:tasks.workunit.client.1.smithi121.stdout:3/557: creat d2/d1a/d27/d63/da4/d25/d31/d34/d4a/fb7 x:0 0 0 2024-02-15T18:35:41.806 INFO:tasks.workunit.client.1.smithi121.stdout:3/558: creat d2/d1a/d27/d63/d81/fb8 x:0 0 0 2024-02-15T18:35:41.807 INFO:tasks.workunit.client.1.smithi121.stdout:3/559: creat d2/d1a/d27/da7/fb9 x:0 0 0 2024-02-15T18:35:41.808 INFO:tasks.workunit.client.1.smithi121.stdout:3/560: creat d2/d1a/d27/d63/da4/d25/d31/d97/daf/fba x:0 0 0 2024-02-15T18:35:41.810 INFO:tasks.workunit.client.1.smithi121.stdout:3/561: dread d2/d5a/f5f [0,4194304] 0 2024-02-15T18:35:41.811 INFO:tasks.workunit.client.1.smithi121.stdout:1/754: dread d0/d6/d7/d50/d7e/f76 [0,4194304] 0 2024-02-15T18:35:41.812 INFO:tasks.workunit.client.1.smithi121.stdout:3/562: write d2/d1a/d27/da7/f47 [3970660,63241] 0 2024-02-15T18:35:41.812 INFO:tasks.workunit.client.1.smithi121.stdout:1/755: write d0/d6/d7/d9/d14/d3e/d88/da6/fbd [2913877,101502] 0 2024-02-15T18:35:41.813 INFO:tasks.workunit.client.1.smithi121.stdout:1/756: mkdir d0/d6/d7/d72/dd9/dfd 0 2024-02-15T18:35:41.814 INFO:tasks.workunit.client.1.smithi121.stdout:3/563: symlink d2/d1a/d27/da7/lbb 0 2024-02-15T18:35:41.814 INFO:tasks.workunit.client.1.smithi121.stdout:1/757: getdents d0/d39/d4b/dac/df5 0 2024-02-15T18:35:41.814 INFO:tasks.workunit.client.1.smithi121.stdout:3/564: chown d2/d1a/d27/da7/f47 16366858 1 2024-02-15T18:35:41.814 INFO:tasks.workunit.client.1.smithi121.stdout:1/758: symlink d0/d6/d7/d72/dd9/lfe 0 2024-02-15T18:35:41.814 INFO:tasks.workunit.client.1.smithi121.stdout:3/565: link d2/d1a/d27/da7/f47 d2/d1a/d27/d63/da4/d25/d31/d97/daf/fbc 0 2024-02-15T18:35:41.815 INFO:tasks.workunit.client.1.smithi121.stdout:3/566: truncate d2/d1a/d54/f69 1086014 0 2024-02-15T18:35:41.816 INFO:tasks.workunit.client.1.smithi121.stdout:1/759: rename d0/d39/f68 to d0/d39/d7d/dc0/fff 0 2024-02-15T18:35:41.818 INFO:tasks.workunit.client.1.smithi121.stdout:6/786: dwrite d7/dc/d34/d42/d62/d63/d8d/fe7 [0,4194304] 0 2024-02-15T18:35:41.818 INFO:tasks.workunit.client.1.smithi121.stdout:1/760: creat d0/d6/d7/d9/d51/f100 x:0 0 0 2024-02-15T18:35:41.818 INFO:tasks.workunit.client.1.smithi121.stdout:2/430: dwrite d5/df/d27/d3e/f53 [0,4194304] 0 2024-02-15T18:35:41.819 INFO:tasks.workunit.client.1.smithi121.stdout:8/646: dwrite d0/d7/de/f1c [4194304,4194304] 0 2024-02-15T18:35:41.820 INFO:tasks.workunit.client.1.smithi121.stdout:1/761: rmdir d0/d6/d7/d50 39 2024-02-15T18:35:41.820 INFO:tasks.workunit.client.1.smithi121.stdout:1/762: chown d0/d6/d7/d9/d14/d48/cc7 3 1 2024-02-15T18:35:41.825 INFO:tasks.workunit.client.1.smithi121.stdout:1/763: creat d0/d6/d7/d9/d14/d48/f101 x:0 0 0 2024-02-15T18:35:41.826 INFO:tasks.workunit.client.1.smithi121.stdout:6/787: mkdir d7/dc/d34/d42/d62/d63/d6f/d89/da1/d116 0 2024-02-15T18:35:41.827 INFO:tasks.workunit.client.1.smithi121.stdout:2/431: creat d5/df/d10/f7e x:0 0 0 2024-02-15T18:35:41.828 INFO:tasks.workunit.client.1.smithi121.stdout:8/647: stat d0/d61/d66/f76 0 2024-02-15T18:35:41.829 INFO:tasks.workunit.client.1.smithi121.stdout:6/788: getdents d7/dc/d34/d42 0 2024-02-15T18:35:41.830 INFO:tasks.workunit.client.1.smithi121.stdout:6/789: fsync d7/dc/f37 0 2024-02-15T18:35:41.831 INFO:tasks.workunit.client.1.smithi121.stdout:2/432: creat d5/d49/f7f x:0 0 0 2024-02-15T18:35:41.831 INFO:tasks.workunit.client.1.smithi121.stdout:2/433: chown d5/df/d27/l52 2592161 1 2024-02-15T18:35:41.831 INFO:tasks.workunit.client.1.smithi121.stdout:2/434: truncate d5/df/f39 296718 0 2024-02-15T18:35:41.831 INFO:tasks.workunit.client.1.smithi121.stdout:2/435: write d5/df/f6c [717224,104773] 0 2024-02-15T18:35:41.832 INFO:tasks.workunit.client.1.smithi121.stdout:8/648: mknod d0/d61/d7b/cdf 0 2024-02-15T18:35:41.833 INFO:tasks.workunit.client.1.smithi121.stdout:8/649: write d0/d73/fa9 [320510,30669] 0 2024-02-15T18:35:41.833 INFO:tasks.workunit.client.1.smithi121.stdout:6/790: symlink d7/d4b/d60/d73/l117 0 2024-02-15T18:35:41.833 INFO:tasks.workunit.client.1.smithi121.stdout:6/791: chown d7/dc/d2c/cb6 390696 1 2024-02-15T18:35:41.838 INFO:tasks.workunit.client.1.smithi121.stdout:8/650: link d0/d61/da6/c88 d0/d7/db1/ce0 0 2024-02-15T18:35:41.838 INFO:tasks.workunit.client.1.smithi121.stdout:8/651: stat d0/fbd 0 2024-02-15T18:35:41.840 INFO:tasks.workunit.client.1.smithi121.stdout:6/792: link d7/dc/d34/d42/d62/dac/fe5 d7/dc/d2c/f118 0 2024-02-15T18:35:41.842 INFO:tasks.workunit.client.1.smithi121.stdout:8/652: unlink d0/d7/f5e 0 2024-02-15T18:35:41.842 INFO:tasks.workunit.client.1.smithi121.stdout:8/653: write d0/d7/d36/d52/f5b [2348133,26683] 0 2024-02-15T18:35:41.842 INFO:tasks.workunit.client.1.smithi121.stdout:8/654: read - d0/d7/d36/d52/d7a/f8a zero size 2024-02-15T18:35:41.842 INFO:tasks.workunit.client.1.smithi121.stdout:6/793: stat d7/dae/cbf 0 2024-02-15T18:35:41.843 INFO:tasks.workunit.client.1.smithi121.stdout:8/655: symlink d0/d7/db1/le1 0 2024-02-15T18:35:41.843 INFO:tasks.workunit.client.1.smithi121.stdout:8/656: chown d0/d7/d36 8 1 2024-02-15T18:35:41.844 INFO:tasks.workunit.client.1.smithi121.stdout:1/764: mknod d0/d39/d4b/c102 0 2024-02-15T18:35:41.844 INFO:tasks.workunit.client.1.smithi121.stdout:8/657: mkdir d0/d7/db1/dbb/de2 0 2024-02-15T18:35:41.851 INFO:tasks.workunit.client.1.smithi121.stdout:6/794: dread d7/dc/d34/d42/d62/dac/fc4 [0,4194304] 0 2024-02-15T18:35:41.851 INFO:tasks.workunit.client.1.smithi121.stdout:6/795: mkdir d7/dc/d2c/d36/d119 0 2024-02-15T18:35:41.852 INFO:tasks.workunit.client.1.smithi121.stdout:6/796: chown d7/dc/d34/d42/d62/dac/de8 38348323 1 2024-02-15T18:35:41.852 INFO:tasks.workunit.client.1.smithi121.stdout:6/797: stat d7/dc/d34/d42/lb5 0 2024-02-15T18:35:41.854 INFO:tasks.workunit.client.1.smithi121.stdout:2/436: dwrite d5/d49/d4d/f2d [4194304,4194304] 0 2024-02-15T18:35:41.854 INFO:tasks.workunit.client.1.smithi121.stdout:2/437: chown d5/d49/d4d/d22 327854229 1 2024-02-15T18:35:41.854 INFO:tasks.workunit.client.1.smithi121.stdout:6/798: creat d7/dc/d34/d7c/f11a x:0 0 0 2024-02-15T18:35:41.854 INFO:tasks.workunit.client.1.smithi121.stdout:6/799: read - d7/dc/d34/d42/d62/d63/d6f/f101 zero size 2024-02-15T18:35:41.855 INFO:tasks.workunit.client.1.smithi121.stdout:6/800: symlink d7/dc/l11b 0 2024-02-15T18:35:41.855 INFO:tasks.workunit.client.1.smithi121.stdout:6/801: write d7/dc/d34/d42/d62/d63/d8d/f105 [779236,90688] 0 2024-02-15T18:35:41.855 INFO:tasks.workunit.client.1.smithi121.stdout:6/802: dread - d7/dc/d34/d42/d62/d63/d6f/d89/da1/fda zero size 2024-02-15T18:35:41.855 INFO:tasks.workunit.client.1.smithi121.stdout:6/803: read d7/d4b/f8f [2847049,57222] 0 2024-02-15T18:35:41.856 INFO:tasks.workunit.client.1.smithi121.stdout:2/438: dread d5/f3b [0,4194304] 0 2024-02-15T18:35:41.856 INFO:tasks.workunit.client.1.smithi121.stdout:2/439: write d5/f78 [594052,102452] 0 2024-02-15T18:35:41.856 INFO:tasks.workunit.client.1.smithi121.stdout:2/440: stat d5/df/d27/l52 0 2024-02-15T18:35:41.857 INFO:tasks.workunit.client.1.smithi121.stdout:6/804: creat d7/f11c x:0 0 0 2024-02-15T18:35:41.859 INFO:tasks.workunit.client.1.smithi121.stdout:6/805: creat d7/dae/f11d x:0 0 0 2024-02-15T18:35:41.859 INFO:tasks.workunit.client.1.smithi121.stdout:6/806: write d7/d4b/d60/d6c/dd5/dfa/fad [708181,1052] 0 2024-02-15T18:35:41.860 INFO:tasks.workunit.client.1.smithi121.stdout:2/441: unlink d5/df/f7d 0 2024-02-15T18:35:41.861 INFO:tasks.workunit.client.1.smithi121.stdout:2/442: write d5/fd [2674576,33833] 0 2024-02-15T18:35:41.861 INFO:tasks.workunit.client.1.smithi121.stdout:2/443: write d5/df/d10/d4c/f77 [325317,102352] 0 2024-02-15T18:35:41.861 INFO:tasks.workunit.client.1.smithi121.stdout:2/444: chown d5/df/f6c 215569526 1 2024-02-15T18:35:41.862 INFO:tasks.workunit.client.1.smithi121.stdout:6/807: truncate d7/dc/d34/f26 4626299 0 2024-02-15T18:35:41.863 INFO:tasks.workunit.client.1.smithi121.stdout:6/808: dread d7/dc/d34/d42/d62/d63/d8d/fa7 [0,4194304] 0 2024-02-15T18:35:41.864 INFO:tasks.workunit.client.1.smithi121.stdout:1/765: rename d0/d6 to d0/d39/d7d/dc0/d77/d103 0 2024-02-15T18:35:41.864 INFO:tasks.workunit.client.1.smithi121.stdout:1/766: chown d0/d39/d7d/dc0/d77/d103/d7/f4c 164 1 2024-02-15T18:35:41.864 INFO:tasks.workunit.client.1.smithi121.stdout:6/809: write d7/dc/d34/f13 [2314495,54894] 0 2024-02-15T18:35:41.866 INFO:tasks.workunit.client.1.smithi121.stdout:8/658: rename d0/d7/db1/ce0 to d0/d7/de/ce3 0 2024-02-15T18:35:41.867 INFO:tasks.workunit.client.1.smithi121.stdout:1/767: getdents d0/d39/d7d/dc0/d77/d103/d7 0 2024-02-15T18:35:41.868 INFO:tasks.workunit.client.1.smithi121.stdout:6/810: symlink d7/dc/d34/d42/d62/dac/l11e 0 2024-02-15T18:35:41.868 INFO:tasks.workunit.client.1.smithi121.stdout:6/811: chown d7/dc/d34/d42/d62/d63/d6f/d89/da1/fda 4414361 1 2024-02-15T18:35:41.870 INFO:tasks.workunit.client.1.smithi121.stdout:8/659: rename d0/d7/d36/c4d to d0/d61/ce4 0 2024-02-15T18:35:41.872 INFO:tasks.workunit.client.1.smithi121.stdout:2/445: dread d5/df/d27/f21 [0,4194304] 0 2024-02-15T18:35:41.872 INFO:tasks.workunit.client.1.smithi121.stdout:2/446: chown d5/d49/d4d/d22 63 1 2024-02-15T18:35:41.873 INFO:tasks.workunit.client.1.smithi121.stdout:8/660: dread d0/d2a/f7d [0,4194304] 0 2024-02-15T18:35:41.874 INFO:tasks.workunit.client.1.smithi121.stdout:1/768: dread d0/d39/d7d/dc0/fff [0,4194304] 0 2024-02-15T18:35:41.874 INFO:tasks.workunit.client.1.smithi121.stdout:1/769: fdatasync d0/d39/d7d/dc0/d77/d103/d7/d9/d14/f3b 0 2024-02-15T18:35:41.879 INFO:tasks.workunit.client.1.smithi121.stdout:2/447: creat d5/d49/d4d/d22/f80 x:0 0 0 2024-02-15T18:35:41.879 INFO:tasks.workunit.client.1.smithi121.stdout:2/448: chown d5/df/d3f/l57 107884 1 2024-02-15T18:35:41.882 INFO:tasks.workunit.client.1.smithi121.stdout:2/449: write d5/d49/d4d/f1c [2594073,58874] 0 2024-02-15T18:35:41.882 INFO:tasks.workunit.client.1.smithi121.stdout:8/661: creat d0/d7/db1/dbb/de2/fe5 x:0 0 0 2024-02-15T18:35:41.882 INFO:tasks.workunit.client.1.smithi121.stdout:8/662: chown d0/d7/d36/d8c 4 1 2024-02-15T18:35:41.889 INFO:tasks.workunit.client.1.smithi121.stdout:1/770: dread d0/fae [0,4194304] 0 2024-02-15T18:35:41.891 INFO:tasks.workunit.client.1.smithi121.stdout:6/812: dwrite d7/dc/ff3 [0,4194304] 0 2024-02-15T18:35:41.891 INFO:tasks.workunit.client.1.smithi121.stdout:6/813: dread - d7/dc/d34/d42/d62/d63/d6f/f101 zero size 2024-02-15T18:35:41.891 INFO:tasks.workunit.client.1.smithi121.stdout:1/771: write d0/d39/fb3 [1904803,91065] 0 2024-02-15T18:35:41.892 INFO:tasks.workunit.client.1.smithi121.stdout:1/772: creat d0/f104 x:0 0 0 2024-02-15T18:35:41.894 INFO:tasks.workunit.client.1.smithi121.stdout:1/773: rmdir d0/d39/d7d/dc0/d77/d103/d7/d9/d14/d3e 39 2024-02-15T18:35:41.897 INFO:tasks.workunit.client.1.smithi121.stdout:6/814: dread d7/d4b/d60/d6c/dd5/dfa/d94/f9a [0,4194304] 0 2024-02-15T18:35:41.897 INFO:tasks.workunit.client.1.smithi121.stdout:6/815: read - d7/d4b/d60/ff7 zero size 2024-02-15T18:35:41.898 INFO:tasks.workunit.client.1.smithi121.stdout:1/774: mknod d0/d39/d4b/dac/df5/c105 0 2024-02-15T18:35:41.899 INFO:tasks.workunit.client.1.smithi121.stdout:6/816: rename d7/dc/d34/d42/d62/d63/d8d/fa0 to d7/dc/d2c/f11f 0 2024-02-15T18:35:41.899 INFO:tasks.workunit.client.1.smithi121.stdout:1/775: creat d0/d39/d7d/dc0/dcf/f106 x:0 0 0 2024-02-15T18:35:41.900 INFO:tasks.workunit.client.1.smithi121.stdout:6/817: creat d7/dc/d34/d42/d62/dac/f120 x:0 0 0 2024-02-15T18:35:41.902 INFO:tasks.workunit.client.1.smithi121.stdout:1/776: creat d0/d39/d7d/dc0/d9d/f107 x:0 0 0 2024-02-15T18:35:41.905 INFO:tasks.workunit.client.1.smithi121.stdout:1/777: mkdir d0/d39/d7d/dc0/d77/d103/d7/d9/d14/d48/d108 0 2024-02-15T18:35:41.907 INFO:tasks.workunit.client.1.smithi121.stdout:8/663: dwrite d0/f13 [4194304,4194304] 0 2024-02-15T18:35:41.908 INFO:tasks.workunit.client.1.smithi121.stdout:1/778: creat d0/d39/d7d/dc0/d77/d103/d7/d9/d14/d3e/d88/da6/f109 x:0 0 0 2024-02-15T18:35:41.908 INFO:tasks.workunit.client.1.smithi121.stdout:1/779: write d0/d39/d7d/dc0/d77/d103/d7/d9/f35 [4222118,121028] 0 2024-02-15T18:35:41.908 INFO:tasks.workunit.client.1.smithi121.stdout:1/780: dread - d0/d39/d7d/dc0/d77/d103/d7/d9/d14/d3e/d88/da6/df7/ffb zero size 2024-02-15T18:35:41.908 INFO:tasks.workunit.client.1.smithi121.stdout:2/450: dwrite d5/df/d10/f7e [0,4194304] 0 2024-02-15T18:35:41.919 INFO:tasks.workunit.client.1.smithi121.stdout:8/664: dread d0/d7/de/f1c [0,4194304] 0 2024-02-15T18:35:41.920 INFO:tasks.workunit.client.1.smithi121.stdout:8/665: rename d0/d73/fa9 to d0/d61/d7b/dac/fe6 0 2024-02-15T18:35:41.926 INFO:tasks.workunit.client.1.smithi121.stdout:6/818: dwrite d7/d4b/d60/d6c/feb [0,4194304] 0 2024-02-15T18:35:41.927 INFO:tasks.workunit.client.1.smithi121.stdout:2/451: dwrite d5/d49/d4d/d22/f3c [4194304,4194304] 0 2024-02-15T18:35:41.928 INFO:tasks.workunit.client.1.smithi121.stdout:6/819: creat d7/dc/d2c/d36/f121 x:0 0 0 2024-02-15T18:35:41.930 INFO:tasks.workunit.client.1.smithi121.stdout:8/666: dread d0/d7/d36/d8c/d45/f89 [0,4194304] 0 2024-02-15T18:35:41.946 INFO:tasks.workunit.client.1.smithi121.stdout:1/781: write d0/d39/d7d/dc0/fd4 [2506271,78847] 0 2024-02-15T18:35:41.955 INFO:tasks.workunit.client.1.smithi121.stdout:2/452: rmdir d5/d49 39 2024-02-15T18:35:41.955 INFO:tasks.workunit.client.1.smithi121.stdout:2/453: stat d5/df/f6c 0 2024-02-15T18:35:41.955 INFO:tasks.workunit.client.1.smithi121.stdout:2/454: fdatasync d5/d49/f7f 0 2024-02-15T18:35:41.956 INFO:tasks.workunit.client.1.smithi121.stdout:6/820: link d7/dc/d34/d42/d62/d63/d6f/l75 d7/d4b/d59/l122 0 2024-02-15T18:35:41.956 INFO:tasks.workunit.client.1.smithi121.stdout:6/821: fsync d7/f102 0 2024-02-15T18:35:41.956 INFO:tasks.workunit.client.1.smithi121.stdout:6/822: readlink d7/dc/d34/d42/d62/lf9 0 2024-02-15T18:35:41.957 INFO:tasks.workunit.client.1.smithi121.stdout:6/823: write d7/dc/d34/d7c/f85 [3708703,22724] 0 2024-02-15T18:35:41.957 INFO:tasks.workunit.client.1.smithi121.stdout:8/667: creat d0/d48/dd4/fe7 x:0 0 0 2024-02-15T18:35:41.957 INFO:tasks.workunit.client.1.smithi121.stdout:8/668: dread - d0/d7/da5/dcb/f9f zero size 2024-02-15T18:35:41.959 INFO:tasks.workunit.client.1.smithi121.stdout:6/824: symlink d7/dc/l123 0 2024-02-15T18:35:41.969 INFO:tasks.workunit.client.1.smithi121.stdout:6/825: dread d7/dc/d34/d42/f54 [0,4194304] 0 2024-02-15T18:35:41.971 INFO:tasks.workunit.client.1.smithi121.stdout:1/782: dwrite d0/d39/d7d/dc0/d77/d103/d7/fc4 [0,4194304] 0 2024-02-15T18:35:41.972 INFO:tasks.workunit.client.1.smithi121.stdout:1/783: read d0/d39/d7d/dc0/d77/ff4 [253692,128523] 0 2024-02-15T18:35:41.972 INFO:tasks.workunit.client.1.smithi121.stdout:1/784: chown d0/d39/d7d/dc0/d77/d103/d7/d9/fa4 2 1 2024-02-15T18:35:41.973 INFO:tasks.workunit.client.1.smithi121.stdout:1/785: write d0/d39/d7d/dc0/d77/d103/d7/d9/fa4 [416576,33502] 0 2024-02-15T18:35:41.973 INFO:tasks.workunit.client.1.smithi121.stdout:1/786: truncate d0/d39/d7d/dc0/d9d/f107 182004 0 2024-02-15T18:35:41.973 INFO:tasks.workunit.client.1.smithi121.stdout:6/826: read d7/dc/d34/fbd [2069334,91569] 0 2024-02-15T18:35:41.973 INFO:tasks.workunit.client.1.smithi121.stdout:6/827: chown d7/f24 3 1 2024-02-15T18:35:41.974 INFO:tasks.workunit.client.1.smithi121.stdout:6/828: dread - d7/dc/d34/d42/d62/d63/d6f/d89/f10e zero size 2024-02-15T18:35:41.974 INFO:tasks.workunit.client.1.smithi121.stdout:6/829: chown d7/dc/l11b 799 1 2024-02-15T18:35:41.974 INFO:tasks.workunit.client.1.smithi121.stdout:2/455: dwrite d5/df/d27/d61/f5c [0,4194304] 0 2024-02-15T18:35:41.974 INFO:tasks.workunit.client.1.smithi121.stdout:1/787: creat d0/d39/f10a x:0 0 0 2024-02-15T18:35:41.976 INFO:tasks.workunit.client.1.smithi121.stdout:6/830: symlink d7/dc/d76/l124 0 2024-02-15T18:35:41.977 INFO:tasks.workunit.client.1.smithi121.stdout:2/456: getdents d5/df/d27/d61 0 2024-02-15T18:35:41.978 INFO:tasks.workunit.client.1.smithi121.stdout:1/788: rmdir d0/d39/d7d/dc0/d77/d103/d7/d9/d14/d3e/d88/da6/de6 0 2024-02-15T18:35:41.979 INFO:tasks.workunit.client.1.smithi121.stdout:6/831: link d7/dc/c23 d7/dc/d34/d42/d62/d63/d8d/c125 0 2024-02-15T18:35:41.979 INFO:tasks.workunit.client.1.smithi121.stdout:6/832: fsync d7/d4b/d59/fec 0 2024-02-15T18:35:41.979 INFO:tasks.workunit.client.1.smithi121.stdout:6/833: readlink d7/dc/d34/d42/d62/d63/lc8 0 2024-02-15T18:35:41.980 INFO:tasks.workunit.client.1.smithi121.stdout:2/457: mkdir d5/d49/d4d/d22/d81 0 2024-02-15T18:35:41.981 INFO:tasks.workunit.client.1.smithi121.stdout:2/458: truncate d5/df/d10/f11 3155461 0 2024-02-15T18:35:41.984 INFO:tasks.workunit.client.1.smithi121.stdout:2/459: mknod d5/df/d54/c82 0 2024-02-15T18:35:41.989 INFO:tasks.workunit.client.1.smithi121.stdout:1/789: dread d0/d39/d4b/f61 [0,4194304] 0 2024-02-15T18:35:41.991 INFO:tasks.workunit.client.1.smithi121.stdout:2/460: write d5/d49/d4d/f2d [6483440,94878] 0 2024-02-15T18:35:41.992 INFO:tasks.workunit.client.1.smithi121.stdout:1/790: creat d0/d39/d7d/dc0/d77/d103/d7/d50/f10b x:0 0 0 2024-02-15T18:35:41.994 INFO:tasks.workunit.client.1.smithi121.stdout:2/461: getdents d5/d49/d4d 0 2024-02-15T18:35:41.994 INFO:tasks.workunit.client.1.smithi121.stdout:2/462: write d5/df/d27/f79 [577172,45900] 0 2024-02-15T18:35:41.994 INFO:tasks.workunit.client.1.smithi121.stdout:2/463: chown d5/fc 61731365 1 2024-02-15T18:35:41.994 INFO:tasks.workunit.client.1.smithi121.stdout:2/464: readlink d5/df/d3f/l57 0 2024-02-15T18:35:41.996 INFO:tasks.workunit.client.1.smithi121.stdout:2/465: rename d5/d49/d4d/l25 to d5/d49/d4d/d22/l83 0 2024-02-15T18:35:41.998 INFO:tasks.workunit.client.1.smithi121.stdout:2/466: creat d5/d49/f84 x:0 0 0 2024-02-15T18:35:41.998 INFO:tasks.workunit.client.1.smithi121.stdout:2/467: symlink d5/df/d27/d3e/d6a/l85 0 2024-02-15T18:35:41.999 INFO:tasks.workunit.client.1.smithi121.stdout:2/468: readlink d5/df/l68 0 2024-02-15T18:35:42.003 INFO:tasks.workunit.client.1.smithi121.stdout:6/834: dwrite d7/d4b/d60/d6c/dd5/dfa/fc3 [0,4194304] 0 2024-02-15T18:35:42.003 INFO:tasks.workunit.client.1.smithi121.stdout:6/835: chown d7/d4b/d59/fec 3 1 2024-02-15T18:35:42.010 INFO:tasks.workunit.client.1.smithi121.stdout:1/791: dwrite d0/d39/d7d/dc0/f41 [0,4194304] 0 2024-02-15T18:35:42.010 INFO:tasks.workunit.client.1.smithi121.stdout:2/469: rmdir d5/df/d27 39 2024-02-15T18:35:42.011 INFO:tasks.workunit.client.1.smithi121.stdout:1/792: symlink d0/d39/d7d/dc0/d77/d103/d7/d9/d14/d3e/d88/daf/db4/de8/l10c 0 2024-02-15T18:35:42.011 INFO:tasks.workunit.client.1.smithi121.stdout:2/470: dread d5/df/f39 [0,4194304] 0 2024-02-15T18:35:42.012 INFO:tasks.workunit.client.1.smithi121.stdout:1/793: rmdir d0/d39/d4b/dac/df5 39 2024-02-15T18:35:42.013 INFO:tasks.workunit.client.1.smithi121.stdout:2/471: creat d5/df/d60/f86 x:0 0 0 2024-02-15T18:35:42.015 INFO:tasks.workunit.client.1.smithi121.stdout:1/794: mknod d0/d39/d7d/dc0/d77/d103/d7/d9/d51/d5f/c10d 0 2024-02-15T18:35:42.015 INFO:tasks.workunit.client.1.smithi121.stdout:1/795: read - d0/d39/f10a zero size 2024-02-15T18:35:42.015 INFO:tasks.workunit.client.1.smithi121.stdout:2/472: rename d5/d49/d4d/d22/f42 to d5/d49/d4d/d58/f87 0 2024-02-15T18:35:42.015 INFO:tasks.workunit.client.1.smithi121.stdout:2/473: chown d5/df/d27/d61/c73 109 1 2024-02-15T18:35:42.016 INFO:tasks.workunit.client.1.smithi121.stdout:1/796: symlink d0/d39/d7d/dc0/d77/d103/d7/d72/dd9/l10e 0 2024-02-15T18:35:42.018 INFO:tasks.workunit.client.1.smithi121.stdout:1/797: truncate d0/d39/d7d/dc0/d77/d103/fdf 1314451 0 2024-02-15T18:35:42.018 INFO:tasks.workunit.client.1.smithi121.stdout:2/474: rename d5/df/d54/f5d to d5/df/d3f/f88 0 2024-02-15T18:35:42.018 INFO:tasks.workunit.client.1.smithi121.stdout:1/798: write d0/fef [347760,95705] 0 2024-02-15T18:35:42.022 INFO:tasks.workunit.client.1.smithi121.stdout:1/799: rmdir d0/d39/d7d/dc0/d77/d103/d7/d9/d14/d3e 39 2024-02-15T18:35:42.022 INFO:tasks.workunit.client.1.smithi121.stdout:1/800: chown d0/d39/d7d/dc0/fe5 10333490 1 2024-02-15T18:35:42.032 INFO:tasks.workunit.client.1.smithi121.stdout:2/475: dwrite d5/df/d10/f6d [0,4194304] 0 2024-02-15T18:35:42.032 INFO:tasks.workunit.client.1.smithi121.stdout:2/476: chown d5/df/d3f/l57 5036625 1 2024-02-15T18:35:42.032 INFO:tasks.workunit.client.1.smithi121.stdout:2/477: dread - d5/d49/d4d/d22/f80 zero size 2024-02-15T18:35:42.035 INFO:tasks.workunit.client.1.smithi121.stdout:2/478: dread d5/df/d27/d3a/f43 [0,4194304] 0 2024-02-15T18:35:42.036 INFO:tasks.workunit.client.1.smithi121.stdout:2/479: write d5/df/d27/d3a/f43 [64187,114771] 0 2024-02-15T18:35:42.037 INFO:tasks.workunit.client.1.smithi121.stdout:2/480: write d5/df/d27/d61/f5c [3479279,123621] 0 2024-02-15T18:35:42.038 INFO:tasks.workunit.client.1.smithi121.stdout:1/801: unlink d0/d39/d7d/dc0/d77/d103/d7/d9/d14/d3e/f71 0 2024-02-15T18:35:42.038 INFO:tasks.workunit.client.1.smithi121.stdout:2/481: write d5/df/f1b [314869,117304] 0 2024-02-15T18:35:42.041 INFO:tasks.workunit.client.1.smithi121.stdout:1/802: rename d0/d39/d7d/dc0/d77/d103/d7/d9/d14/d3e/d88/da6/fa8 to d0/d39/d7d/dc0/d77/d103/d7/d50/d7e/dec/f10f 0 2024-02-15T18:35:42.044 INFO:tasks.workunit.client.1.smithi121.stdout:2/482: truncate d5/df/d10/f6d 1069662 0 2024-02-15T18:35:42.044 INFO:tasks.workunit.client.1.smithi121.stdout:2/483: chown d5/d49/d4d/d22/f4b 3768 1 2024-02-15T18:35:42.052 INFO:tasks.workunit.client.1.smithi121.stdout:2/484: unlink d5/df/d10/f11 0 2024-02-15T18:35:42.053 INFO:tasks.workunit.client.1.smithi121.stdout:2/485: mkdir d5/d89 0 2024-02-15T18:35:42.053 INFO:tasks.workunit.client.1.smithi121.stdout:2/486: read d5/df/f1b [1323998,113861] 0 2024-02-15T18:35:42.069 INFO:tasks.workunit.client.1.smithi121.stdout:1/803: dwrite d0/d39/d7d/dc0/f5d [4194304,4194304] 0 2024-02-15T18:35:42.079 INFO:tasks.workunit.client.1.smithi121.stdout:1/804: dread d0/d39/d7d/dc0/d77/d103/d7/d9/d14/db6/fd5 [0,4194304] 0 2024-02-15T18:35:42.081 INFO:tasks.workunit.client.1.smithi121.stdout:1/805: mknod d0/d39/d7d/dc0/d77/d103/d7/d9/d14/d3e/d88/da6/c110 0 2024-02-15T18:35:42.092 INFO:tasks.workunit.client.1.smithi121.stdout:1/806: rename d0/d39/d7d/dc0/d77/d103/d7 to d0/d39/d4b/dac/d111 0 2024-02-15T18:35:42.092 INFO:tasks.workunit.client.1.smithi121.stdout:1/807: dread - d0/d39/f10a zero size 2024-02-15T18:35:42.093 INFO:tasks.workunit.client.1.smithi121.stdout:7/499: sync 2024-02-15T18:35:42.094 INFO:tasks.workunit.client.1.smithi121.stdout:1/808: creat d0/d39/d4b/dac/d111/d72/dd9/dfd/f112 x:0 0 0 2024-02-15T18:35:42.097 INFO:tasks.workunit.client.1.smithi121.stdout:1/809: mknod d0/d39/d4b/dac/d111/d50/c113 0 2024-02-15T18:35:42.098 INFO:tasks.workunit.client.1.smithi121.stdout:1/810: mknod d0/d39/d4b/dac/d111/d9/d14/c114 0 2024-02-15T18:35:42.105 INFO:tasks.workunit.client.1.smithi121.stdout:1/811: symlink d0/d39/d4b/dac/d111/d9/d14/d48/d108/l115 0 2024-02-15T18:35:42.107 INFO:tasks.workunit.client.1.smithi121.stdout:1/812: rename d0/d39/d4b/dac/d111/d9/d14/d48/fca to d0/d39/d4b/dac/d111/d9/d14/d3e/f116 0 2024-02-15T18:35:42.109 INFO:tasks.workunit.client.1.smithi121.stdout:1/813: unlink d0/fef 0 2024-02-15T18:35:42.109 INFO:tasks.workunit.client.1.smithi121.stdout:1/814: stat d0/f9c 0 2024-02-15T18:35:42.109 INFO:tasks.workunit.client.1.smithi121.stdout:1/815: write d0/d39/d7d/dc0/d77/ff4 [1500736,36975] 0 2024-02-15T18:35:42.109 INFO:tasks.workunit.client.1.smithi121.stdout:7/500: dwrite d0/d5/d11/d79/f86 [4194304,4194304] 0 2024-02-15T18:35:42.110 INFO:tasks.workunit.client.1.smithi121.stdout:7/501: getdents d0/d4a/d5b 0 2024-02-15T18:35:42.113 INFO:tasks.workunit.client.1.smithi121.stdout:1/816: symlink d0/l117 0 2024-02-15T18:35:42.113 INFO:tasks.workunit.client.1.smithi121.stdout:1/817: chown d0/d39/d7d/dc0/d77/d103/fe2 4561055 1 2024-02-15T18:35:42.113 INFO:tasks.workunit.client.1.smithi121.stdout:7/502: dread d0/d5/d11/d1c/f35 [0,4194304] 0 2024-02-15T18:35:42.114 INFO:tasks.workunit.client.1.smithi121.stdout:1/818: getdents d0/d39/d7d/dc0/d77 0 2024-02-15T18:35:42.114 INFO:tasks.workunit.client.1.smithi121.stdout:7/503: symlink d0/d33/la4 0 2024-02-15T18:35:42.115 INFO:tasks.workunit.client.1.smithi121.stdout:7/504: creat d0/d4a/d5b/d9a/fa5 x:0 0 0 2024-02-15T18:35:42.117 INFO:tasks.workunit.client.1.smithi121.stdout:1/819: stat d0/d39/d4b/dac/d111/cc 0 2024-02-15T18:35:42.117 INFO:tasks.workunit.client.1.smithi121.stdout:1/820: readlink d0/d39/d4b/dac/d111/d50/le4 0 2024-02-15T18:35:42.117 INFO:tasks.workunit.client.1.smithi121.stdout:1/821: read d0/d39/d7d/dc0/fff [1635695,9716] 0 2024-02-15T18:35:42.118 INFO:tasks.workunit.client.1.smithi121.stdout:7/505: rename d0/d5/d11/d67 to d0/d4a/d5b/da6 0 2024-02-15T18:35:42.120 INFO:tasks.workunit.client.1.smithi121.stdout:7/506: rename d0/d4a/d5b/d9a/fa5 to d0/d5/d11/d5c/fa7 0 2024-02-15T18:35:42.126 INFO:tasks.workunit.client.1.smithi121.stdout:4/657: sync 2024-02-15T18:35:42.127 INFO:tasks.workunit.client.1.smithi121.stdout:9/824: sync 2024-02-15T18:35:42.127 INFO:tasks.workunit.client.1.smithi121.stdout:4/658: fdatasync d5/d9/d76/d9e/d14/f29 0 2024-02-15T18:35:42.128 INFO:tasks.workunit.client.1.smithi121.stdout:9/825: creat d7/d8/d25/f11e x:0 0 0 2024-02-15T18:35:42.129 INFO:tasks.workunit.client.1.smithi121.stdout:4/659: link d5/d9/f20 d5/d31/d3d/d4a/d7e/fcd 0 2024-02-15T18:35:42.137 INFO:tasks.workunit.client.1.smithi121.stdout:1/822: dwrite d0/d39/d4b/dac/d111/d9/d14/db6/fb8 [0,4194304] 0 2024-02-15T18:35:42.139 INFO:tasks.workunit.client.1.smithi121.stdout:1/823: dread d0/d39/d4b/dac/d111/d9/d14/d3e/fd0 [0,4194304] 0 2024-02-15T18:35:42.139 INFO:tasks.workunit.client.1.smithi121.stdout:1/824: chown d0/d39/d4b/dac/d111/d9/d14/d3e/d88/fa9 1420 1 2024-02-15T18:35:42.139 INFO:tasks.workunit.client.1.smithi121.stdout:9/826: write d7/d8/d40/d4b/f7a [233039,115612] 0 2024-02-15T18:35:42.139 INFO:tasks.workunit.client.1.smithi121.stdout:9/827: read d7/d26/d4f/d5b/d77/f91 [2315937,26817] 0 2024-02-15T18:35:42.145 INFO:tasks.workunit.client.1.smithi121.stdout:1/825: rename d0/d39/d4b/dac/d111/d9/d14/d3e/fd0 to d0/d39/d4b/dac/d111/d9/d51/f118 0 2024-02-15T18:35:42.145 INFO:tasks.workunit.client.1.smithi121.stdout:1/826: truncate d0/d39/d4b/dac/d111/d9/d14/f3b 1799916 0 2024-02-15T18:35:42.147 INFO:tasks.workunit.client.1.smithi121.stdout:9/828: dread d7/f13 [0,4194304] 0 2024-02-15T18:35:42.149 INFO:tasks.workunit.client.1.smithi121.stdout:9/829: write d7/d26/fd6 [1044487,30852] 0 2024-02-15T18:35:42.149 INFO:tasks.workunit.client.1.smithi121.stdout:9/830: readlink d7/d8/d40/dc4/l51 0 2024-02-15T18:35:42.149 INFO:tasks.workunit.client.1.smithi121.stdout:9/831: chown d7/d8/f1d 106407568 1 2024-02-15T18:35:42.149 INFO:tasks.workunit.client.1.smithi121.stdout:9/832: readlink d7/d8/d25/d30/da5/l85 0 2024-02-15T18:35:42.150 INFO:tasks.workunit.client.1.smithi121.stdout:4/660: dwrite d5/d31/d3d/d9b/fa5 [0,4194304] 0 2024-02-15T18:35:42.152 INFO:tasks.workunit.client.1.smithi121.stdout:4/661: creat d5/d31/d3d/d4f/d8a/fce x:0 0 0 2024-02-15T18:35:42.157 INFO:tasks.workunit.client.1.smithi121.stdout:9/833: read d7/d3a/d8e/d9e/da9/d8b/d73/fad [702811,13033] 0 2024-02-15T18:35:42.157 INFO:tasks.workunit.client.1.smithi121.stdout:9/834: chown d7/d8/d36/c11c 643 1 2024-02-15T18:35:42.158 INFO:tasks.workunit.client.1.smithi121.stdout:9/835: creat d7/d26/d4f/f11f x:0 0 0 2024-02-15T18:35:42.158 INFO:tasks.workunit.client.1.smithi121.stdout:9/836: write d7/d3a/d8e/d9e/dc7/fe1 [768358,122825] 0 2024-02-15T18:35:42.158 INFO:tasks.workunit.client.1.smithi121.stdout:9/837: stat d7/d3a/d8e/d9e/da9/d8b/f83 0 2024-02-15T18:35:42.160 INFO:tasks.workunit.client.1.smithi121.stdout:4/662: dread d5/d9/d19/d28/f21 [0,4194304] 0 2024-02-15T18:35:42.160 INFO:tasks.workunit.client.1.smithi121.stdout:4/663: dread - d5/d31/d3d/d4f/d51/d71/fc6 zero size 2024-02-15T18:35:42.164 INFO:tasks.workunit.client.1.smithi121.stdout:4/664: rename d5/d9/d76/d9e/d14/f17 to d5/d9/d19/d5c/fcf 0 2024-02-15T18:35:42.165 INFO:tasks.workunit.client.1.smithi121.stdout:4/665: write d5/d9/f20 [1313964,101922] 0 2024-02-15T18:35:42.165 INFO:tasks.workunit.client.1.smithi121.stdout:4/666: fsync d5/d9/d76/d9e/d14/f55 0 2024-02-15T18:35:42.166 INFO:tasks.workunit.client.1.smithi121.stdout:1/827: dwrite d0/d39/d4b/dac/d111/f1b [0,4194304] 0 2024-02-15T18:35:42.166 INFO:tasks.workunit.client.1.smithi121.stdout:1/828: readlink d0/d39/d7d/dc0/l4f 0 2024-02-15T18:35:42.166 INFO:tasks.workunit.client.1.smithi121.stdout:4/667: getdents d5/d31/d25 0 2024-02-15T18:35:42.167 INFO:tasks.workunit.client.1.smithi121.stdout:4/668: readlink d5/d9/d19/d2a/d4d/l6d 0 2024-02-15T18:35:42.167 INFO:tasks.workunit.client.1.smithi121.stdout:1/829: dread d0/d39/d4b/dac/d111/d9/fa4 [0,4194304] 0 2024-02-15T18:35:42.168 INFO:tasks.workunit.client.1.smithi121.stdout:4/669: symlink d5/d31/d3d/d4f/d51/d80/ld0 0 2024-02-15T18:35:42.170 INFO:tasks.workunit.client.1.smithi121.stdout:4/670: creat d5/d31/d25/db6/fd1 x:0 0 0 2024-02-15T18:35:42.171 INFO:tasks.workunit.client.1.smithi121.stdout:9/838: dread d7/d8/d25/d30/d33/f88 [0,4194304] 0 2024-02-15T18:35:42.172 INFO:tasks.workunit.client.1.smithi121.stdout:9/839: symlink d7/d8/d36/dce/l120 0 2024-02-15T18:35:42.174 INFO:tasks.workunit.client.1.smithi121.stdout:9/840: creat d7/d3a/f121 x:0 0 0 2024-02-15T18:35:42.174 INFO:tasks.workunit.client.1.smithi121.stdout:9/841: stat d7/d8/d36/da3/d110 0 2024-02-15T18:35:42.175 INFO:tasks.workunit.client.1.smithi121.stdout:1/830: dread d0/d39/d4b/dac/d111/d9/d14/d3e/d88/da6/fbd [0,4194304] 0 2024-02-15T18:35:42.178 INFO:tasks.workunit.client.1.smithi121.stdout:9/842: unlink d7/d26/c9b 0 2024-02-15T18:35:42.181 INFO:tasks.workunit.client.1.smithi121.stdout:9/843: rename d7/d8/d40/d4b to d7/d3a/d8e/d9e/da9/d8b/d122 0 2024-02-15T18:35:42.182 INFO:tasks.workunit.client.1.smithi121.stdout:1/831: write d0/d39/fb3 [620449,110923] 0 2024-02-15T18:35:42.183 INFO:tasks.workunit.client.1.smithi121.stdout:9/844: mkdir d7/d26/d4f/d5b/d77/d123 0 2024-02-15T18:35:42.183 INFO:tasks.workunit.client.1.smithi121.stdout:9/845: chown d7/d8/d25/d30/da5/d108 6487 1 2024-02-15T18:35:42.184 INFO:tasks.workunit.client.1.smithi121.stdout:9/846: symlink d7/d8/d25/d30/d33/d8f/l124 0 2024-02-15T18:35:42.185 INFO:tasks.workunit.client.1.smithi121.stdout:1/832: creat d0/d39/d7d/dc0/d77/f119 x:0 0 0 2024-02-15T18:35:42.187 INFO:tasks.workunit.client.1.smithi121.stdout:1/833: creat d0/d39/d4b/dac/d111/d9/d14/d3e/d88/da6/df7/f11a x:0 0 0 2024-02-15T18:35:42.188 INFO:tasks.workunit.client.1.smithi121.stdout:9/847: creat d7/f125 x:0 0 0 2024-02-15T18:35:42.190 INFO:tasks.workunit.client.1.smithi121.stdout:1/834: rename d0/d39/d4b/dac/d111/d9/d51/ce9 to d0/d39/d7d/dc0/d77/c11b 0 2024-02-15T18:35:42.196 INFO:tasks.workunit.client.1.smithi121.stdout:4/671: dwrite d5/d9/d76/d9e/f3c [0,4194304] 0 2024-02-15T18:35:42.196 INFO:tasks.workunit.client.1.smithi121.stdout:1/835: symlink d0/d39/d4b/l11c 0 2024-02-15T18:35:42.198 INFO:tasks.workunit.client.1.smithi121.stdout:4/672: creat d5/d31/d25/fd2 x:0 0 0 2024-02-15T18:35:42.199 INFO:tasks.workunit.client.1.smithi121.stdout:4/673: dread - d5/d31/d3d/d4a/d7e/f7f zero size 2024-02-15T18:35:42.199 INFO:tasks.workunit.client.1.smithi121.stdout:4/674: dread - d5/d31/d3d/d4f/d51/d63/f99 zero size 2024-02-15T18:35:42.202 INFO:tasks.workunit.client.1.smithi121.stdout:4/675: dread d5/d9/d1d/d26/d73/d81/fad [0,4194304] 0 2024-02-15T18:35:42.205 INFO:tasks.workunit.client.1.smithi121.stdout:4/676: mkdir d5/d31/d3d/d4f/dd3 0 2024-02-15T18:35:42.207 INFO:tasks.workunit.client.1.smithi121.stdout:9/848: dwrite d7/d8/d36/f7d [0,4194304] 0 2024-02-15T18:35:42.209 INFO:tasks.workunit.client.1.smithi121.stdout:9/849: symlink d7/d8/d25/d30/d33/d3c/dd0/dd9/l126 0 2024-02-15T18:35:42.209 INFO:tasks.workunit.client.1.smithi121.stdout:9/850: dread - d7/f113 zero size 2024-02-15T18:35:42.209 INFO:tasks.workunit.client.1.smithi121.stdout:9/851: fdatasync d7/d3a/d8e/d9e/da9/d8b/fbb 0 2024-02-15T18:35:42.210 INFO:tasks.workunit.client.1.smithi121.stdout:9/852: truncate d7/d3a/f9d 857408 0 2024-02-15T18:35:42.211 INFO:tasks.workunit.client.1.smithi121.stdout:9/853: truncate d7/d3a/fdc 3472553 0 2024-02-15T18:35:42.215 INFO:tasks.workunit.client.1.smithi121.stdout:9/854: creat d7/d8/d25/d30/f127 x:0 0 0 2024-02-15T18:35:42.216 INFO:tasks.workunit.client.1.smithi121.stdout:9/855: creat d7/d3a/d8e/f128 x:0 0 0 2024-02-15T18:35:42.218 INFO:tasks.workunit.client.1.smithi121.stdout:1/836: dwrite d0/d39/d4b/dac/d111/d9/d14/db6/fe1 [0,4194304] 0 2024-02-15T18:35:42.218 INFO:tasks.workunit.client.1.smithi121.stdout:1/837: truncate d0/f104 848585 0 2024-02-15T18:35:42.226 INFO:tasks.workunit.client.1.smithi121.stdout:1/838: getdents d0/d39/d7d/dc0/d9d 0 2024-02-15T18:35:42.227 INFO:tasks.workunit.client.1.smithi121.stdout:9/856: read d7/d8/d40/f70 [3588944,128225] 0 2024-02-15T18:35:42.227 INFO:tasks.workunit.client.1.smithi121.stdout:9/857: stat d7/d8/d36/f7d 0 2024-02-15T18:35:42.231 INFO:tasks.workunit.client.1.smithi121.stdout:4/677: dwrite d5/d9/d76/d9e/d14/f55 [0,4194304] 0 2024-02-15T18:35:42.232 INFO:tasks.workunit.client.1.smithi121.stdout:9/858: dread d7/d8/d25/d30/d33/f88 [0,4194304] 0 2024-02-15T18:35:42.232 INFO:tasks.workunit.client.1.smithi121.stdout:9/859: dread - d7/d3a/d8e/d9e/da9/d8b/d122/fe9 zero size 2024-02-15T18:35:42.232 INFO:tasks.workunit.client.1.smithi121.stdout:4/678: write d5/d31/d3d/d4f/d51/d71/f48 [4902448,82655] 0 2024-02-15T18:35:42.259 INFO:tasks.workunit.client.1.smithi121.stdout:1/839: dwrite d0/d39/d4b/dac/d111/d50/d7e/fc9 [0,4194304] 0 2024-02-15T18:35:42.259 INFO:tasks.workunit.client.1.smithi121.stdout:1/840: chown d0/d39/d7d/dc0/d9d/f107 16247261 1 2024-02-15T18:35:42.260 INFO:tasks.workunit.client.1.smithi121.stdout:1/841: write d0/d39/d4b/dac/d111/d9/fa2 [312564,69863] 0 2024-02-15T18:35:42.260 INFO:tasks.workunit.client.1.smithi121.stdout:9/860: dwrite d7/d8/d40/f6e [0,4194304] 0 2024-02-15T18:35:42.260 INFO:tasks.workunit.client.1.smithi121.stdout:4/679: dwrite d5/d9/d76/d9e/f3c [4194304,4194304] 0 2024-02-15T18:35:42.261 INFO:tasks.workunit.client.1.smithi121.stdout:1/842: mknod d0/d39/d4b/dac/df5/c11d 0 2024-02-15T18:35:42.263 INFO:tasks.workunit.client.1.smithi121.stdout:4/680: getdents d5/d9/d76/d9e/d14 0 2024-02-15T18:35:42.265 INFO:tasks.workunit.client.1.smithi121.stdout:9/861: unlink d7/d8/d40/f72 0 2024-02-15T18:35:42.266 INFO:tasks.workunit.client.1.smithi121.stdout:9/862: fdatasync d7/d3a/d8e/d9e/da9/d8b/f7b 0 2024-02-15T18:35:42.266 INFO:tasks.workunit.client.1.smithi121.stdout:9/863: truncate d7/d3a/d8e/d9e/da9/d8b/d73/ffa 411750 0 2024-02-15T18:35:42.266 INFO:tasks.workunit.client.1.smithi121.stdout:9/864: chown d7/d3a/d8e/d9e/da9/d8b 289517 1 2024-02-15T18:35:42.266 INFO:tasks.workunit.client.1.smithi121.stdout:9/865: chown d7/d3a/d8e/d9e/dc7/d101/c115 511852779 1 2024-02-15T18:35:42.266 INFO:tasks.workunit.client.1.smithi121.stdout:9/866: chown d7/d26/dba/lec 6255618 1 2024-02-15T18:35:42.266 INFO:tasks.workunit.client.1.smithi121.stdout:9/867: chown d7/d8/d40 4917536 1 2024-02-15T18:35:42.266 INFO:tasks.workunit.client.1.smithi121.stdout:9/868: chown d7/l29 148 1 2024-02-15T18:35:42.266 INFO:tasks.workunit.client.1.smithi121.stdout:1/843: mkdir d0/d39/d4b/dac/d111/d9/d14/db6/d11e 0 2024-02-15T18:35:42.267 INFO:tasks.workunit.client.1.smithi121.stdout:9/869: truncate d7/f1c 6004941 0 2024-02-15T18:35:42.267 INFO:tasks.workunit.client.1.smithi121.stdout:9/870: mkdir d7/d8/d25/d30/d33/d79/d129 0 2024-02-15T18:35:42.268 INFO:tasks.workunit.client.1.smithi121.stdout:1/844: mknod d0/d39/d4b/dac/d111/d9/d14/d3e/d88/daf/db4/de8/c11f 0 2024-02-15T18:35:42.270 INFO:tasks.workunit.client.1.smithi121.stdout:9/871: link d7/d3a/d8e/d9e/da9/d8b/l24 d7/d8/d25/d30/da5/d108/l12a 0 2024-02-15T18:35:42.273 INFO:tasks.workunit.client.1.smithi121.stdout:9/872: creat d7/d8/d36/da3/d110/f12b x:0 0 0 2024-02-15T18:35:42.274 INFO:tasks.workunit.client.1.smithi121.stdout:9/873: mkdir d7/d8/d25/d30/da5/d12c 0 2024-02-15T18:35:42.274 INFO:tasks.workunit.client.1.smithi121.stdout:9/874: fdatasync d7/d26/d4f/f7f 0 2024-02-15T18:35:42.295 INFO:tasks.workunit.client.1.smithi121.stdout:4/681: dwrite d5/d9/d19/f4e [0,4194304] 0 2024-02-15T18:35:42.295 INFO:tasks.workunit.client.1.smithi121.stdout:4/682: chown d5/d9/d1d/d26/d73/d81/fad 723640 1 2024-02-15T18:35:42.295 INFO:tasks.workunit.client.1.smithi121.stdout:1/845: dwrite d0/d39/d4b/dac/d111/d9/fa4 [0,4194304] 0 2024-02-15T18:35:42.295 INFO:tasks.workunit.client.1.smithi121.stdout:1/846: fsync d0/d39/d4b/dac/d111/d9/d14/d3e/d88/fa9 0 2024-02-15T18:35:42.296 INFO:tasks.workunit.client.1.smithi121.stdout:9/875: dwrite d7/f113 [0,4194304] 0 2024-02-15T18:35:42.296 INFO:tasks.workunit.client.1.smithi121.stdout:4/683: truncate d5/d9/d1d/f41 56375 0 2024-02-15T18:35:42.296 INFO:tasks.workunit.client.1.smithi121.stdout:4/684: readlink d5/d9/d1d/d26/d73/l7b 0 2024-02-15T18:35:42.296 INFO:tasks.workunit.client.1.smithi121.stdout:4/685: dread - d5/d31/d3d/d4f/d51/d71/fc6 zero size 2024-02-15T18:35:42.300 INFO:tasks.workunit.client.1.smithi121.stdout:1/847: rmdir d0/d39/d4b/dac/d111 39 2024-02-15T18:35:42.305 INFO:tasks.workunit.client.1.smithi121.stdout:9/876: dread d7/d8/d36/f7d [0,4194304] 0 2024-02-15T18:35:42.305 INFO:tasks.workunit.client.1.smithi121.stdout:4/686: dread d5/d31/d3d/d4a/f60 [0,4194304] 0 2024-02-15T18:35:42.306 INFO:tasks.workunit.client.1.smithi121.stdout:9/877: mkdir d7/d8/d25/d12d 0 2024-02-15T18:35:42.307 INFO:tasks.workunit.client.1.smithi121.stdout:9/878: getdents d7/d8/d25/d30/da5/d108 0 2024-02-15T18:35:42.308 INFO:tasks.workunit.client.1.smithi121.stdout:9/879: creat d7/d8/d36/da3/f12e x:0 0 0 2024-02-15T18:35:42.309 INFO:tasks.workunit.client.1.smithi121.stdout:1/848: dread d0/d39/d7d/dc0/f93 [0,4194304] 0 2024-02-15T18:35:42.310 INFO:tasks.workunit.client.1.smithi121.stdout:9/880: creat d7/d26/def/f12f x:0 0 0 2024-02-15T18:35:42.310 INFO:tasks.workunit.client.1.smithi121.stdout:4/687: creat d5/d9/d1d/d26/d73/d81/fd4 x:0 0 0 2024-02-15T18:35:42.311 INFO:tasks.workunit.client.1.smithi121.stdout:9/881: rename d7/d8/d36/da3/ffd to d7/d8/d40/f130 0 2024-02-15T18:35:42.312 INFO:tasks.workunit.client.1.smithi121.stdout:1/849: link d0/d39/d4b/dac/d111/d9/d51/d5f/c10d d0/d39/d7d/c120 0 2024-02-15T18:35:42.312 INFO:tasks.workunit.client.1.smithi121.stdout:1/850: chown d0/d39/d4b/dac/d111/d9/d14/d3e/d88/da6/f109 28528 1 2024-02-15T18:35:42.312 INFO:tasks.workunit.client.1.smithi121.stdout:9/882: creat d7/d8/d36/dce/f131 x:0 0 0 2024-02-15T18:35:42.314 INFO:tasks.workunit.client.1.smithi121.stdout:9/883: creat d7/d8/d25/d30/d33/f132 x:0 0 0 2024-02-15T18:35:42.314 INFO:tasks.workunit.client.1.smithi121.stdout:9/884: write d7/d26/d4f/d5b/d77/fa6 [4866088,104307] 0 2024-02-15T18:35:42.314 INFO:tasks.workunit.client.1.smithi121.stdout:9/885: stat d7/d8/c6f 0 2024-02-15T18:35:42.314 INFO:tasks.workunit.client.1.smithi121.stdout:9/886: read - d7/d3a/d8e/f128 zero size 2024-02-15T18:35:42.316 INFO:tasks.workunit.client.1.smithi121.stdout:9/887: rmdir d7/d3a/d8e/d9e/da9 39 2024-02-15T18:35:42.317 INFO:tasks.workunit.client.1.smithi121.stdout:4/688: creat d5/d31/d3d/d4f/d51/d63/dc8/fd5 x:0 0 0 2024-02-15T18:35:42.318 INFO:tasks.workunit.client.1.smithi121.stdout:1/851: creat d0/f121 x:0 0 0 2024-02-15T18:35:42.321 INFO:tasks.workunit.client.1.smithi121.stdout:4/689: symlink d5/d9/d19/d4c/d82/ld6 0 2024-02-15T18:35:42.323 INFO:tasks.workunit.client.1.smithi121.stdout:1/852: getdents d0/d39/d4b/dac/d111/d9/d14/d48 0 2024-02-15T18:35:42.324 INFO:tasks.workunit.client.1.smithi121.stdout:1/853: mknod d0/d39/d7d/dc0/d77/d103/c122 0 2024-02-15T18:35:42.325 INFO:tasks.workunit.client.1.smithi121.stdout:4/690: dread d5/d9/d76/d9e/f38 [0,4194304] 0 2024-02-15T18:35:42.326 INFO:tasks.workunit.client.1.smithi121.stdout:1/854: creat d0/d39/d4b/dac/d111/d9/d14/d48/f123 x:0 0 0 2024-02-15T18:35:42.326 INFO:tasks.workunit.client.1.smithi121.stdout:1/855: truncate d0/d39/d4b/dac/d111/d9/d14/d48/f101 499028 0 2024-02-15T18:35:42.326 INFO:tasks.workunit.client.1.smithi121.stdout:1/856: stat d0/d39/d4b/dac/d111/d9/d51 0 2024-02-15T18:35:42.326 INFO:tasks.workunit.client.1.smithi121.stdout:1/857: mkdir d0/d39/d7d/dc0/d77/d124 0 2024-02-15T18:35:42.335 INFO:tasks.workunit.client.1.smithi121.stdout:9/888: dwrite d7/d3a/d8e/f128 [0,4194304] 0 2024-02-15T18:35:42.338 INFO:tasks.workunit.client.1.smithi121.stdout:9/889: mknod d7/d3a/d8e/d9e/da9/d8b/d73/c133 0 2024-02-15T18:35:42.338 INFO:tasks.workunit.client.1.smithi121.stdout:9/890: fdatasync d7/d26/d4f/d5b/d77/fa6 0 2024-02-15T18:35:42.339 INFO:tasks.workunit.client.1.smithi121.stdout:9/891: creat d7/d8/d25/d30/d33/f134 x:0 0 0 2024-02-15T18:35:42.340 INFO:tasks.workunit.client.1.smithi121.stdout:9/892: symlink d7/l135 0 2024-02-15T18:35:42.343 INFO:tasks.workunit.client.1.smithi121.stdout:1/858: fsync d0/d39/d4b/dac/d111/d9/d14/d48/f123 0 2024-02-15T18:35:42.344 INFO:tasks.workunit.client.1.smithi121.stdout:1/859: truncate d0/d39/d4b/dac/d111/d50/f10b 321978 0 2024-02-15T18:35:42.344 INFO:tasks.workunit.client.1.smithi121.stdout:4/691: dwrite d5/d9/d19/d4c/d82/f8e [0,4194304] 0 2024-02-15T18:35:42.345 INFO:tasks.workunit.client.1.smithi121.stdout:1/860: write d0/d39/d4b/dac/d111/d9/d51/f6c [379774,81510] 0 2024-02-15T18:35:42.345 INFO:tasks.workunit.client.1.smithi121.stdout:1/861: readlink d0/d39/l3a 0 2024-02-15T18:35:42.347 INFO:tasks.workunit.client.1.smithi121.stdout:1/862: write d0/d39/d4b/dac/d111/d9/d14/d48/f78 [3637549,52394] 0 2024-02-15T18:35:42.352 INFO:tasks.workunit.client.1.smithi121.stdout:4/692: mknod d5/d9/d19/d2a/d4d/cd7 0 2024-02-15T18:35:42.352 INFO:tasks.workunit.client.1.smithi121.stdout:4/693: fdatasync d5/d31/d3d/d4f/d51/f47 0 2024-02-15T18:35:42.359 INFO:tasks.workunit.client.1.smithi121.stdout:4/694: write d5/d9/d1d/d26/d72/f78 [667334,117986] 0 2024-02-15T18:35:42.361 INFO:tasks.workunit.client.1.smithi121.stdout:4/695: symlink d5/d9/d76/d9e/d14/ld8 0 2024-02-15T18:35:42.361 INFO:tasks.workunit.client.1.smithi121.stdout:9/893: dwrite d7/f1b [0,4194304] 0 2024-02-15T18:35:42.362 INFO:tasks.workunit.client.1.smithi121.stdout:9/894: mknod d7/d8/d40/dc4/c136 0 2024-02-15T18:35:42.362 INFO:tasks.workunit.client.1.smithi121.stdout:4/696: mknod d5/d31/d3d/d4f/cd9 0 2024-02-15T18:35:42.363 INFO:tasks.workunit.client.1.smithi121.stdout:9/895: mkdir d7/d8/d40/dc4/d137 0 2024-02-15T18:35:42.367 INFO:tasks.workunit.client.1.smithi121.stdout:1/863: dread d0/d39/d4b/dac/d111/d9/f23 [0,4194304] 0 2024-02-15T18:35:42.367 INFO:tasks.workunit.client.1.smithi121.stdout:1/864: read - d0/d39/d4b/fde zero size 2024-02-15T18:35:42.367 INFO:tasks.workunit.client.1.smithi121.stdout:1/865: getdents d0/d39/d4b/dac/d111/d50 0 2024-02-15T18:35:42.370 INFO:tasks.workunit.client.1.smithi121.stdout:1/866: rename d0/d39/l3a to d0/d39/d4b/dac/d111/d9/d14/d3e/d88/da6/l125 0 2024-02-15T18:35:42.373 INFO:tasks.workunit.client.1.smithi121.stdout:1/867: dread d0/d39/d4b/dac/d111/d9/fa2 [0,4194304] 0 2024-02-15T18:35:42.373 INFO:tasks.workunit.client.1.smithi121.stdout:1/868: stat d0/d39/d4b/dac/d111/d9/d14/d3e/d88 0 2024-02-15T18:35:42.375 INFO:tasks.workunit.client.1.smithi121.stdout:1/869: mkdir d0/d39/d4b/dac/d111/d9/d14/d48/d126 0 2024-02-15T18:35:42.377 INFO:tasks.workunit.client.1.smithi121.stdout:1/870: mknod d0/d39/d4b/dac/d111/d9/d14/db6/c127 0 2024-02-15T18:35:42.389 INFO:tasks.workunit.client.1.smithi121.stdout:4/697: dwrite d5/d9/d76/d9e/f3c [0,4194304] 0 2024-02-15T18:35:42.392 INFO:tasks.workunit.client.1.smithi121.stdout:9/896: dwrite d7/d26/f6b [0,4194304] 0 2024-02-15T18:35:42.392 INFO:tasks.workunit.client.1.smithi121.stdout:9/897: readlink d7/d8/d25/d30/da5/l85 0 2024-02-15T18:35:42.394 INFO:tasks.workunit.client.1.smithi121.stdout:1/871: dread d0/d39/d4b/dac/d111/d9/fdc [4194304,4194304] 0 2024-02-15T18:35:42.394 INFO:tasks.workunit.client.1.smithi121.stdout:1/872: fdatasync d0/d39/d4b/dac/d111/d9/d51/f100 0 2024-02-15T18:35:42.394 INFO:tasks.workunit.client.1.smithi121.stdout:9/898: link d7/d8/d25/d30/d33/d8f/dd4/c114 d7/dcc/c138 0 2024-02-15T18:35:42.396 INFO:tasks.workunit.client.1.smithi121.stdout:9/899: truncate d7/d8/fa8 27369 0 2024-02-15T18:35:42.396 INFO:tasks.workunit.client.1.smithi121.stdout:9/900: dread - d7/d8/d36/dce/f131 zero size 2024-02-15T18:35:42.399 INFO:tasks.workunit.client.1.smithi121.stdout:9/901: write d7/df6/f11a [727441,47367] 0 2024-02-15T18:35:42.399 INFO:tasks.workunit.client.1.smithi121.stdout:9/902: chown d7/d8/d40/dc4/d137 3330181 1 2024-02-15T18:35:42.400 INFO:tasks.workunit.client.1.smithi121.stdout:9/903: chown d7/d8/d25/d30/d33/d79/fde 2076512 1 2024-02-15T18:35:42.400 INFO:tasks.workunit.client.1.smithi121.stdout:9/904: mknod d7/d8/d36/df9/c139 0 2024-02-15T18:35:42.405 INFO:tasks.workunit.client.1.smithi121.stdout:9/905: unlink d7/f1b 0 2024-02-15T18:35:42.408 INFO:tasks.workunit.client.1.smithi121.stdout:9/906: creat d7/d3a/d8e/d9e/dc7/f13a x:0 0 0 2024-02-15T18:35:42.416 INFO:tasks.workunit.client.1.smithi121.stdout:4/698: dwrite d5/d9/d19/f4e [0,4194304] 0 2024-02-15T18:35:42.417 INFO:tasks.workunit.client.1.smithi121.stdout:1/873: dwrite d0/d39/d4b/dac/d111/d9/d14/db6/fba [0,4194304] 0 2024-02-15T18:35:42.423 INFO:tasks.workunit.client.1.smithi121.stdout:9/907: dread d7/d8/d40/f6e [0,4194304] 0 2024-02-15T18:35:42.423 INFO:tasks.workunit.client.1.smithi121.stdout:9/908: chown d7/d8/d25/d30/da5/fd8 0 1 2024-02-15T18:35:42.424 INFO:tasks.workunit.client.1.smithi121.stdout:0/833: sync 2024-02-15T18:35:42.424 INFO:tasks.workunit.client.1.smithi121.stdout:0/834: dread - d5/d30/fdd zero size 2024-02-15T18:35:42.424 INFO:tasks.workunit.client.1.smithi121.stdout:0/835: write d5/d30/ff6 [602212,18632] 0 2024-02-15T18:35:42.425 INFO:tasks.workunit.client.1.smithi121.stdout:0/836: chown d5/d6b/dab/db4/dca/c119 0 1 2024-02-15T18:35:42.426 INFO:tasks.workunit.client.1.smithi121.stdout:0/837: symlink d5/d6b/dab/db4/d7d/dd/l11e 0 2024-02-15T18:35:42.427 INFO:tasks.workunit.client.1.smithi121.stdout:0/838: getdents d5/d6b/dab/db4/d7d/d2a/d73/d8d/dc8 0 2024-02-15T18:35:42.429 INFO:tasks.workunit.client.1.smithi121.stdout:0/839: dread d5/f6a [0,4194304] 0 2024-02-15T18:35:42.430 INFO:tasks.workunit.client.1.smithi121.stdout:0/840: dread - d5/d6b/dab/db4/d7d/d2a/d4e/f11d zero size 2024-02-15T18:35:42.436 INFO:tasks.workunit.client.1.smithi121.stdout:0/841: rename d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/c104 to d5/d30/d5d/c11f 0 2024-02-15T18:35:42.438 INFO:tasks.workunit.client.1.smithi121.stdout:0/842: getdents d5/d6b/dab/db4/d7d/d2a/d73 0 2024-02-15T18:35:42.438 INFO:tasks.workunit.client.1.smithi121.stdout:0/843: chown d5/d6b/dab/db4/d7d/d74/la3 106439 1 2024-02-15T18:35:42.449 INFO:tasks.workunit.client.1.smithi121.stdout:4/699: dwrite d5/d9/d76/d9e/d14/f55 [0,4194304] 0 2024-02-15T18:35:42.449 INFO:tasks.workunit.client.1.smithi121.stdout:1/874: dwrite d0/d39/d7d/dc0/d9d/f107 [0,4194304] 0 2024-02-15T18:35:42.449 INFO:tasks.workunit.client.1.smithi121.stdout:1/875: write d0/d39/d7d/dc0/d77/d103/fdf [1616560,104129] 0 2024-02-15T18:35:42.450 INFO:tasks.workunit.client.1.smithi121.stdout:9/909: dwrite d7/d8/d36/da3/f12e [0,4194304] 0 2024-02-15T18:35:42.453 INFO:tasks.workunit.client.1.smithi121.stdout:6/836: sync 2024-02-15T18:35:42.453 INFO:tasks.workunit.client.1.smithi121.stdout:2/487: sync 2024-02-15T18:35:42.454 INFO:tasks.workunit.client.1.smithi121.stdout:5/871: sync 2024-02-15T18:35:42.454 INFO:tasks.workunit.client.1.smithi121.stdout:3/567: sync 2024-02-15T18:35:42.454 INFO:tasks.workunit.client.1.smithi121.stdout:7/507: sync 2024-02-15T18:35:42.454 INFO:tasks.workunit.client.1.smithi121.stdout:8/669: sync 2024-02-15T18:35:42.458 INFO:tasks.workunit.client.1.smithi121.stdout:4/700: creat d5/d31/d25/fda x:0 0 0 2024-02-15T18:35:42.460 INFO:tasks.workunit.client.1.smithi121.stdout:1/876: mkdir d0/d39/d4b/dac/d111/d9/d14/d48/d128 0 2024-02-15T18:35:42.460 INFO:tasks.workunit.client.1.smithi121.stdout:1/877: chown d0/f53 1941370310 1 2024-02-15T18:35:42.462 INFO:tasks.workunit.client.1.smithi121.stdout:4/701: rename d5/d9/d76/d9e/d14/f55 to d5/d9/fdb 0 2024-02-15T18:35:42.463 INFO:tasks.workunit.client.1.smithi121.stdout:1/878: rmdir d0/d39/d7d/dc0/d77/d124 0 2024-02-15T18:35:42.464 INFO:tasks.workunit.client.1.smithi121.stdout:1/879: truncate d0/d39/d4b/dac/d111/d9/d14/d3e/d88/da6/df7/f11a 881491 0 2024-02-15T18:35:42.465 INFO:tasks.workunit.client.1.smithi121.stdout:2/488: creat d5/df/d10/f8a x:0 0 0 2024-02-15T18:35:42.465 INFO:tasks.workunit.client.1.smithi121.stdout:7/508: creat d0/d4a/d5b/fa8 x:0 0 0 2024-02-15T18:35:42.466 INFO:tasks.workunit.client.1.smithi121.stdout:8/670: mknod d0/d61/da6/ce8 0 2024-02-15T18:35:42.466 INFO:tasks.workunit.client.1.smithi121.stdout:3/568: rmdir d2/d1a/d27/d63 39 2024-02-15T18:35:42.466 INFO:tasks.workunit.client.1.smithi121.stdout:6/837: mknod d7/dc/d34/d7c/c126 0 2024-02-15T18:35:42.466 INFO:tasks.workunit.client.1.smithi121.stdout:4/702: mknod d5/d9/d1d/cdc 0 2024-02-15T18:35:42.467 INFO:tasks.workunit.client.1.smithi121.stdout:1/880: getdents d0/d39/d4b 0 2024-02-15T18:35:42.468 INFO:tasks.workunit.client.1.smithi121.stdout:2/489: link d5/d49/d4d/l34 d5/df/d10/d4c/l8b 0 2024-02-15T18:35:42.470 INFO:tasks.workunit.client.1.smithi121.stdout:7/509: mkdir d0/d38/d3f/d43/da9 0 2024-02-15T18:35:42.470 INFO:tasks.workunit.client.1.smithi121.stdout:7/510: write d0/d5/d11/d1c/d5f/f7d [1007411,130404] 0 2024-02-15T18:35:42.470 INFO:tasks.workunit.client.1.smithi121.stdout:4/703: getdents d5/d9/d1d/d26/d73/d81 0 2024-02-15T18:35:42.470 INFO:tasks.workunit.client.1.smithi121.stdout:4/704: fdatasync d5/d9/d19/d2a/f96 0 2024-02-15T18:35:42.471 INFO:tasks.workunit.client.1.smithi121.stdout:8/671: write d0/d7/d36/d8c/d45/f74 [2513923,121443] 0 2024-02-15T18:35:42.472 INFO:tasks.workunit.client.1.smithi121.stdout:8/672: write d0/d7/de/f54 [43081,96376] 0 2024-02-15T18:35:42.472 INFO:tasks.workunit.client.1.smithi121.stdout:3/569: rename d2/d1a/d27/d63/da4/d25/d31/d97/daf/fba to d2/d1a/d27/d63/da4/d25/d31/d58/fbd 0 2024-02-15T18:35:42.472 INFO:tasks.workunit.client.1.smithi121.stdout:3/570: truncate d2/d1a/d27/d63/da4/d25/d31/d97/fae 904570 0 2024-02-15T18:35:42.472 INFO:tasks.workunit.client.1.smithi121.stdout:3/571: fsync d2/d1a/d27/d63/da4/f56 0 2024-02-15T18:35:42.472 INFO:tasks.workunit.client.1.smithi121.stdout:3/572: write d2/d1a/d27/d63/d81/fac [676664,60774] 0 2024-02-15T18:35:42.474 INFO:tasks.workunit.client.1.smithi121.stdout:9/910: creat d7/d26/d4f/d5b/f13b x:0 0 0 2024-02-15T18:35:42.474 INFO:tasks.workunit.client.1.smithi121.stdout:2/490: creat d5/df/d60/f8c x:0 0 0 2024-02-15T18:35:42.474 INFO:tasks.workunit.client.1.smithi121.stdout:2/491: chown d5/df/f2b 405259091 1 2024-02-15T18:35:42.475 INFO:tasks.workunit.client.1.smithi121.stdout:4/705: mknod d5/d31/d3d/cdd 0 2024-02-15T18:35:42.475 INFO:tasks.workunit.client.1.smithi121.stdout:1/881: stat d0/d39/d4b/dac/d111/d50/lfa 0 2024-02-15T18:35:42.476 INFO:tasks.workunit.client.1.smithi121.stdout:2/492: mkdir d5/d49/d4d/d22/d81/d8d 0 2024-02-15T18:35:42.477 INFO:tasks.workunit.client.1.smithi121.stdout:4/706: creat d5/d31/d3d/d4a/d7e/fde x:0 0 0 2024-02-15T18:35:42.480 INFO:tasks.workunit.client.1.smithi121.stdout:1/882: rename d0/d39/d4b/dac/d111/d9/leb to d0/d39/d7d/dc0/d9d/l129 0 2024-02-15T18:35:42.483 INFO:tasks.workunit.client.1.smithi121.stdout:9/911: write d7/d26/fd6 [1644099,76500] 0 2024-02-15T18:35:42.483 INFO:tasks.workunit.client.1.smithi121.stdout:9/912: readlink d7/d3a/d8e/d9e/dc7/lea 0 2024-02-15T18:35:42.483 INFO:tasks.workunit.client.1.smithi121.stdout:1/883: rmdir d0/d39/d4b/dac/d111/d9/d51 39 2024-02-15T18:35:42.483 INFO:tasks.workunit.client.1.smithi121.stdout:1/884: chown d0/d39/c96 6796 1 2024-02-15T18:35:42.484 INFO:tasks.workunit.client.1.smithi121.stdout:9/913: creat d7/d26/d4f/f13c x:0 0 0 2024-02-15T18:35:42.485 INFO:tasks.workunit.client.1.smithi121.stdout:1/885: symlink d0/d39/d4b/dac/d111/d9/d14/d3e/d88/l12a 0 2024-02-15T18:35:42.485 INFO:tasks.workunit.client.1.smithi121.stdout:5/872: dwrite d0/d2/d1e/d4e/f91 [0,4194304] 0 2024-02-15T18:35:42.486 INFO:tasks.workunit.client.1.smithi121.stdout:1/886: truncate d0/d39/d4b/dac/d111/d9/d14/d48/f78 3471338 0 2024-02-15T18:35:42.489 INFO:tasks.workunit.client.1.smithi121.stdout:5/873: mknod d0/d2/d1e/d4e/d7d/d9d/d4c/daa/c11b 0 2024-02-15T18:35:42.489 INFO:tasks.workunit.client.1.smithi121.stdout:5/874: fdatasync d0/d2/d18/f101 0 2024-02-15T18:35:42.491 INFO:tasks.workunit.client.1.smithi121.stdout:1/887: truncate d0/d39/d4b/dac/d111/d9/d14/d3e/f4a 3470479 0 2024-02-15T18:35:42.491 INFO:tasks.workunit.client.1.smithi121.stdout:1/888: dread - d0/d39/d7d/dc0/d77/f119 zero size 2024-02-15T18:35:42.492 INFO:tasks.workunit.client.1.smithi121.stdout:0/844: dwrite d5/d6b/dab/db4/d7d/d2a/d4e/fcc [0,4194304] 0 2024-02-15T18:35:42.492 INFO:tasks.workunit.client.1.smithi121.stdout:0/845: truncate d5/d30/f101 236668 0 2024-02-15T18:35:42.492 INFO:tasks.workunit.client.1.smithi121.stdout:0/846: dread - d5/d6b/dab/db4/d7d/d2a/f102 zero size 2024-02-15T18:35:42.499 INFO:tasks.workunit.client.1.smithi121.stdout:9/914: dread d7/d8/d25/d30/da5/fd8 [0,4194304] 0 2024-02-15T18:35:42.503 INFO:tasks.workunit.client.1.smithi121.stdout:5/875: truncate d0/d16/f81 3027361 0 2024-02-15T18:35:42.503 INFO:tasks.workunit.client.1.smithi121.stdout:5/876: chown d0/c1c 46 1 2024-02-15T18:35:42.512 INFO:tasks.workunit.client.1.smithi121.stdout:9/915: dread d7/d8/d36/fc5 [0,4194304] 0 2024-02-15T18:35:42.513 INFO:tasks.workunit.client.1.smithi121.stdout:5/877: creat d0/d2/d1e/d4e/d7d/d47/f11c x:0 0 0 2024-02-15T18:35:42.515 INFO:tasks.workunit.client.1.smithi121.stdout:9/916: symlink d7/d3a/d8e/d9e/dc7/d11d/l13d 0 2024-02-15T18:35:42.515 INFO:tasks.workunit.client.1.smithi121.stdout:9/917: truncate d7/f125 164468 0 2024-02-15T18:35:42.515 INFO:tasks.workunit.client.1.smithi121.stdout:9/918: write d7/d8/f87 [5164705,67118] 0 2024-02-15T18:35:42.515 INFO:tasks.workunit.client.1.smithi121.stdout:6/838: dwrite d7/f11 [0,4194304] 0 2024-02-15T18:35:42.515 INFO:tasks.workunit.client.1.smithi121.stdout:6/839: readlink d7/dc/d2c/d36/l3f 0 2024-02-15T18:35:42.516 INFO:tasks.workunit.client.1.smithi121.stdout:5/878: write d0/d2/d1e/d4e/d7d/d47/f64 [1669098,110685] 0 2024-02-15T18:35:42.518 INFO:tasks.workunit.client.1.smithi121.stdout:7/511: dwrite d0/d38/fa0 [0,4194304] 0 2024-02-15T18:35:42.518 INFO:tasks.workunit.client.1.smithi121.stdout:9/919: rename d7/lc8 to d7/d26/d4f/l13e 0 2024-02-15T18:35:42.518 INFO:tasks.workunit.client.1.smithi121.stdout:5/879: read d0/d2/d1e/d4e/d7d/d9d/f56 [33945,16338] 0 2024-02-15T18:35:42.518 INFO:tasks.workunit.client.1.smithi121.stdout:5/880: dread - d0/d16/d34/da4/f10b zero size 2024-02-15T18:35:42.518 INFO:tasks.workunit.client.1.smithi121.stdout:5/881: fdatasync d0/d16/d34/d8f/fbf 0 2024-02-15T18:35:42.518 INFO:tasks.workunit.client.1.smithi121.stdout:5/882: truncate d0/d16/d34/d8f/fdc 621811 0 2024-02-15T18:35:42.519 INFO:tasks.workunit.client.1.smithi121.stdout:5/883: creat d0/d2/d1e/d4e/d7d/d47/d5a/db3/f11d x:0 0 0 2024-02-15T18:35:42.520 INFO:tasks.workunit.client.1.smithi121.stdout:5/884: dread d0/d16/f98 [4194304,4194304] 0 2024-02-15T18:35:42.523 INFO:tasks.workunit.client.1.smithi121.stdout:2/493: dwrite d5/df/d10/d4c/f77 [0,4194304] 0 2024-02-15T18:35:42.525 INFO:tasks.workunit.client.1.smithi121.stdout:2/494: write d5/d49/d71/f70 [3718671,124815] 0 2024-02-15T18:35:42.527 INFO:tasks.workunit.client.1.smithi121.stdout:5/885: write d0/d16/d7a/db5/fcb [3055790,92439] 0 2024-02-15T18:35:42.527 INFO:tasks.workunit.client.1.smithi121.stdout:9/920: dread d7/d8/d36/df9/f11b [0,4194304] 0 2024-02-15T18:35:42.527 INFO:tasks.workunit.client.1.smithi121.stdout:3/573: dwrite d2/d1a/d27/d63/d81/fb8 [0,4194304] 0 2024-02-15T18:35:42.531 INFO:tasks.workunit.client.1.smithi121.stdout:1/889: dwrite d0/d39/d7d/f86 [0,4194304] 0 2024-02-15T18:35:42.532 INFO:tasks.workunit.client.1.smithi121.stdout:9/921: read d7/d8/d25/d30/d33/f58 [153401,98580] 0 2024-02-15T18:35:42.533 INFO:tasks.workunit.client.1.smithi121.stdout:0/847: dwrite d5/d6b/dab/db4/d7d/dd/f68 [0,4194304] 0 2024-02-15T18:35:42.533 INFO:tasks.workunit.client.1.smithi121.stdout:6/840: dread d7/d4b/f8f [0,4194304] 0 2024-02-15T18:35:42.536 INFO:tasks.workunit.client.1.smithi121.stdout:3/574: creat d2/d1a/d27/da7/d85/d9f/fbe x:0 0 0 2024-02-15T18:35:42.536 INFO:tasks.workunit.client.1.smithi121.stdout:4/707: dwrite d5/d31/d3d/d4f/d51/d63/dc8/fd5 [0,4194304] 0 2024-02-15T18:35:42.536 INFO:tasks.workunit.client.1.smithi121.stdout:3/575: dread - d2/d1a/d27/da7/d85/d9f/fbe zero size 2024-02-15T18:35:42.536 INFO:tasks.workunit.client.1.smithi121.stdout:9/922: readlink d7/d3a/d8e/l92 0 2024-02-15T18:35:42.542 INFO:tasks.workunit.client.1.smithi121.stdout:4/708: write d5/fa [10898589,79143] 0 2024-02-15T18:35:42.542 INFO:tasks.workunit.client.1.smithi121.stdout:7/512: rmdir d0/d4a/d5b 39 2024-02-15T18:35:42.542 INFO:tasks.workunit.client.1.smithi121.stdout:0/848: creat d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/f120 x:0 0 0 2024-02-15T18:35:42.545 INFO:tasks.workunit.client.1.smithi121.stdout:6/841: mknod d7/dc/d34/d42/d62/d63/c127 0 2024-02-15T18:35:42.545 INFO:tasks.workunit.client.1.smithi121.stdout:6/842: stat d7/dc/d2c/d36 0 2024-02-15T18:35:42.545 INFO:tasks.workunit.client.1.smithi121.stdout:6/843: fdatasync d7/dc/d34/f71 0 2024-02-15T18:35:42.545 INFO:tasks.workunit.client.1.smithi121.stdout:6/844: chown d7/dc/d34/d42/d62/fd9 6 1 2024-02-15T18:35:42.546 INFO:tasks.workunit.client.1.smithi121.stdout:6/845: write d7/dc/d34/d42/d62/d63/d6f/d89/f10e [538579,52955] 0 2024-02-15T18:35:42.546 INFO:tasks.workunit.client.1.smithi121.stdout:6/846: dread - d7/dc/d55/f104 zero size 2024-02-15T18:35:42.550 INFO:tasks.workunit.client.1.smithi121.stdout:7/513: rename d0/d33 to d0/d38/d46/daa 0 2024-02-15T18:35:42.550 INFO:tasks.workunit.client.1.smithi121.stdout:9/923: mkdir d7/d3a/d8e/d13f 0 2024-02-15T18:35:42.556 INFO:tasks.workunit.client.1.smithi121.stdout:1/890: getdents d0/d39/d4b/dac/d111/d9/d14/d48 0 2024-02-15T18:35:42.557 INFO:tasks.workunit.client.1.smithi121.stdout:9/924: stat d7/d8/d25/d30/d33/c39 0 2024-02-15T18:35:42.558 INFO:tasks.workunit.client.1.smithi121.stdout:7/514: creat d0/d38/fab x:0 0 0 2024-02-15T18:35:42.558 INFO:tasks.workunit.client.1.smithi121.stdout:7/515: fsync d0/d5/d11/d79/f86 0 2024-02-15T18:35:42.561 INFO:tasks.workunit.client.1.smithi121.stdout:9/925: mkdir d7/d3a/d8e/d9e/d140 0 2024-02-15T18:35:42.562 INFO:tasks.workunit.client.1.smithi121.stdout:0/849: unlink d5/d6b/dab/db4/d7d/l10e 0 2024-02-15T18:35:42.564 INFO:tasks.workunit.client.1.smithi121.stdout:4/709: creat d5/d31/d3d/d4f/d51/d71/fdf x:0 0 0 2024-02-15T18:35:42.566 INFO:tasks.workunit.client.1.smithi121.stdout:9/926: write d7/d8/d36/dce/f103 [3123945,91070] 0 2024-02-15T18:35:42.569 INFO:tasks.workunit.client.1.smithi121.stdout:5/886: dwrite d0/d2/d1e/d4e/d7d/d47/fe9 [0,4194304] 0 2024-02-15T18:35:42.573 INFO:tasks.workunit.client.1.smithi121.stdout:2/495: dwrite d5/d49/d4d/d22/f80 [0,4194304] 0 2024-02-15T18:35:42.573 INFO:tasks.workunit.client.1.smithi121.stdout:3/576: dwrite d2/d1a/d27/da7/d85/fa3 [0,4194304] 0 2024-02-15T18:35:42.574 INFO:tasks.workunit.client.1.smithi121.stdout:0/850: dread d5/d6b/dab/db4/d7d/f11 [4194304,4194304] 0 2024-02-15T18:35:42.575 INFO:tasks.workunit.client.1.smithi121.stdout:0/851: chown d5/d30/fef 10477748 1 2024-02-15T18:35:42.583 INFO:tasks.workunit.client.1.smithi121.stdout:0/852: creat d5/d6b/dab/f121 x:0 0 0 2024-02-15T18:35:42.586 INFO:tasks.workunit.client.1.smithi121.stdout:2/496: dread d5/d49/d4d/d22/f37 [0,4194304] 0 2024-02-15T18:35:42.586 INFO:tasks.workunit.client.1.smithi121.stdout:0/853: mknod d5/d6b/dab/db4/d7d/d74/d96/dc4/c122 0 2024-02-15T18:35:42.588 INFO:tasks.workunit.client.1.smithi121.stdout:2/497: mknod d5/d49/d4d/d22/d81/c8e 0 2024-02-15T18:35:42.588 INFO:tasks.workunit.client.1.smithi121.stdout:2/498: truncate d5/fc 228641 0 2024-02-15T18:35:42.589 INFO:tasks.workunit.client.1.smithi121.stdout:0/854: symlink d5/d6b/dab/db4/dca/l123 0 2024-02-15T18:35:42.591 INFO:tasks.workunit.client.1.smithi121.stdout:6/847: dwrite d7/dc/d34/d42/d62/d63/d6f/d89/f10e [0,4194304] 0 2024-02-15T18:35:42.591 INFO:tasks.workunit.client.1.smithi121.stdout:6/848: readlink d7/dc/d76/le2 0 2024-02-15T18:35:42.599 INFO:tasks.workunit.client.1.smithi121.stdout:1/891: creat d0/d39/d4b/dac/f12b x:0 0 0 2024-02-15T18:35:42.599 INFO:tasks.workunit.client.1.smithi121.stdout:1/892: chown d0/d39/d4b/dac/d111/d9/d14/d3e/f116 12 1 2024-02-15T18:35:42.600 INFO:tasks.workunit.client.1.smithi121.stdout:4/710: mknod d5/d9/d76/d87/db4/ce0 0 2024-02-15T18:35:42.600 INFO:tasks.workunit.client.1.smithi121.stdout:4/711: chown d5/d31/d3d/d4a/d7e 815 1 2024-02-15T18:35:42.600 INFO:tasks.workunit.client.1.smithi121.stdout:4/712: fdatasync d5/d31/d3d/d9b/fa5 0 2024-02-15T18:35:42.600 INFO:tasks.workunit.client.1.smithi121.stdout:4/713: stat d5/d31/d3d/f49 0 2024-02-15T18:35:42.601 INFO:tasks.workunit.client.1.smithi121.stdout:4/714: truncate d5/d9/fdb 4459846 0 2024-02-15T18:35:42.601 INFO:tasks.workunit.client.1.smithi121.stdout:4/715: truncate d5/d9/d19/d2a/f96 135110 0 2024-02-15T18:35:42.602 INFO:tasks.workunit.client.1.smithi121.stdout:5/887: creat d0/d2/d70/de1/f11e x:0 0 0 2024-02-15T18:35:42.604 INFO:tasks.workunit.client.1.smithi121.stdout:1/893: creat d0/d39/d4b/dac/d111/d9/d14/d48/f12c x:0 0 0 2024-02-15T18:35:42.605 INFO:tasks.workunit.client.1.smithi121.stdout:2/499: dread d5/fd [0,4194304] 0 2024-02-15T18:35:42.606 INFO:tasks.workunit.client.1.smithi121.stdout:4/716: truncate d5/d9/d19/d2a/f68 1924993 0 2024-02-15T18:35:42.606 INFO:tasks.workunit.client.1.smithi121.stdout:2/500: getdents d5/d49/d4d/d58 0 2024-02-15T18:35:42.607 INFO:tasks.workunit.client.1.smithi121.stdout:5/888: truncate d0/d2/f7b 1144294 0 2024-02-15T18:35:42.608 INFO:tasks.workunit.client.1.smithi121.stdout:2/501: mknod d5/df/d60/c8f 0 2024-02-15T18:35:42.608 INFO:tasks.workunit.client.1.smithi121.stdout:2/502: fsync d5/df/d27/d3e/f59 0 2024-02-15T18:35:42.608 INFO:tasks.workunit.client.1.smithi121.stdout:9/927: dwrite d7/d8/d40/f70 [0,4194304] 0 2024-02-15T18:35:42.608 INFO:tasks.workunit.client.1.smithi121.stdout:9/928: fsync d7/d3a/d8e/d9e/da9/d8b/d73/ffa 0 2024-02-15T18:35:42.610 INFO:tasks.workunit.client.1.smithi121.stdout:5/889: creat d0/d16/d34/da4/f11f x:0 0 0 2024-02-15T18:35:42.611 INFO:tasks.workunit.client.1.smithi121.stdout:5/890: symlink d0/d2/d1e/d4e/d7d/d39/l120 0 2024-02-15T18:35:42.612 INFO:tasks.workunit.client.1.smithi121.stdout:5/891: mknod d0/d16/d7a/db5/dca/c121 0 2024-02-15T18:35:42.612 INFO:tasks.workunit.client.1.smithi121.stdout:5/892: chown d0/d2/c76 15 1 2024-02-15T18:35:42.615 INFO:tasks.workunit.client.1.smithi121.stdout:4/717: dread d5/d31/d3d/d4f/d51/d71/fbc [0,4194304] 0 2024-02-15T18:35:42.615 INFO:tasks.workunit.client.1.smithi121.stdout:5/893: rename d0/d2/d70/c77 to d0/d16/d7a/c122 0 2024-02-15T18:35:42.618 INFO:tasks.workunit.client.1.smithi121.stdout:0/855: dwrite d5/d6b/fcb [0,4194304] 0 2024-02-15T18:35:42.618 INFO:tasks.workunit.client.1.smithi121.stdout:0/856: fdatasync d5/d6b/dab/db4/d7d/d2a/d73/f7a 0 2024-02-15T18:35:42.618 INFO:tasks.workunit.client.1.smithi121.stdout:0/857: write d5/f92 [4672079,116899] 0 2024-02-15T18:35:42.618 INFO:tasks.workunit.client.1.smithi121.stdout:5/894: link d0/d2/d1e/d4e/d7d/lb6 d0/d16/d7a/db5/d116/l123 0 2024-02-15T18:35:42.618 INFO:tasks.workunit.client.1.smithi121.stdout:5/895: stat d0/d2/d1e/d4e/d7d/d47/d5a/da7/fbd 0 2024-02-15T18:35:42.620 INFO:tasks.workunit.client.1.smithi121.stdout:0/858: mkdir d5/d6b/dab/db4/d7d/d2a/d73/d81/d124 0 2024-02-15T18:35:42.620 INFO:tasks.workunit.client.1.smithi121.stdout:9/929: write d7/d3a/fdc [989251,122413] 0 2024-02-15T18:35:42.620 INFO:tasks.workunit.client.1.smithi121.stdout:0/859: fdatasync d5/d6b/dab/db4/d7d/dd/fb0 0 2024-02-15T18:35:42.621 INFO:tasks.workunit.client.1.smithi121.stdout:1/894: dread d0/d39/d4b/dac/d111/d9/f35 [0,4194304] 0 2024-02-15T18:35:42.623 INFO:tasks.workunit.client.1.smithi121.stdout:0/860: creat d5/d107/f125 x:0 0 0 2024-02-15T18:35:42.624 INFO:tasks.workunit.client.1.smithi121.stdout:5/896: creat d0/d2/d1e/d4e/d7d/d40/d62/db1/f124 x:0 0 0 2024-02-15T18:35:42.624 INFO:tasks.workunit.client.1.smithi121.stdout:5/897: chown d0/d16/d7a/db5/d116/l123 985616 1 2024-02-15T18:35:42.624 INFO:tasks.workunit.client.1.smithi121.stdout:5/898: dread - d0/d2/d10c/ff0 zero size 2024-02-15T18:35:42.624 INFO:tasks.workunit.client.1.smithi121.stdout:6/849: dwrite d7/dc/d34/d42/d62/fdb [0,4194304] 0 2024-02-15T18:35:42.625 INFO:tasks.workunit.client.1.smithi121.stdout:3/577: dwrite d2/d1a/d27/fa2 [4194304,4194304] 0 2024-02-15T18:35:42.625 INFO:tasks.workunit.client.1.smithi121.stdout:4/718: dread d5/d9/d19/f4e [0,4194304] 0 2024-02-15T18:35:42.626 INFO:tasks.workunit.client.1.smithi121.stdout:5/899: creat d0/d16/da1/f125 x:0 0 0 2024-02-15T18:35:42.632 INFO:tasks.workunit.client.1.smithi121.stdout:4/719: unlink d5/d31/d3d/d4f/d8a/fb5 0 2024-02-15T18:35:42.632 INFO:tasks.workunit.client.1.smithi121.stdout:2/503: truncate d5/d49/d4d/d22/f80 784399 0 2024-02-15T18:35:42.632 INFO:tasks.workunit.client.1.smithi121.stdout:2/504: chown d5/df/d10/d4c/l8b 253868687 1 2024-02-15T18:35:42.637 INFO:tasks.workunit.client.1.smithi121.stdout:0/861: mknod d5/d6b/dab/db4/d7d/d2a/d73/d97/c126 0 2024-02-15T18:35:42.645 INFO:tasks.workunit.client.1.smithi121.stdout:4/720: truncate d5/d31/d25/f37 1760878 0 2024-02-15T18:35:42.645 INFO:tasks.workunit.client.1.smithi121.stdout:3/578: rmdir d2/d1a/d27/d63/da4/d25 39 2024-02-15T18:35:42.645 INFO:tasks.workunit.client.1.smithi121.stdout:3/579: chown d2/d1a/f92 1 1 2024-02-15T18:35:42.645 INFO:tasks.workunit.client.1.smithi121.stdout:3/580: chown d2/d1a/d27/d63/da4/d25/d88/d96/l9e 17926871 1 2024-02-15T18:35:42.646 INFO:tasks.workunit.client.1.smithi121.stdout:6/850: getdents d7/d4b/d60/d6c 0 2024-02-15T18:35:42.646 INFO:tasks.workunit.client.1.smithi121.stdout:3/581: mknod d2/d1a/d27/d63/cbf 0 2024-02-15T18:35:42.653 INFO:tasks.workunit.client.1.smithi121.stdout:0/862: dread d5/d6b/dab/db4/d7d/dd/f17 [0,4194304] 0 2024-02-15T18:35:42.653 INFO:tasks.workunit.client.1.smithi121.stdout:0/863: fdatasync d5/d30/f4f 0 2024-02-15T18:35:42.658 INFO:tasks.workunit.client.1.smithi121.stdout:0/864: getdents d5/d6b/dab/db4 0 2024-02-15T18:35:42.662 INFO:tasks.workunit.client.1.smithi121.stdout:6/851: write d7/dc/d2c/f118 [1611185,92578] 0 2024-02-15T18:35:42.662 INFO:tasks.workunit.client.1.smithi121.stdout:6/852: chown d7/d4b/d60/d6c/f8a 170886805 1 2024-02-15T18:35:42.663 INFO:tasks.workunit.client.1.smithi121.stdout:6/853: creat d7/dc/d2c/d100/f128 x:0 0 0 2024-02-15T18:35:42.665 INFO:tasks.workunit.client.1.smithi121.stdout:2/505: dwrite d5/d49/d4d/f75 [0,4194304] 0 2024-02-15T18:35:42.665 INFO:tasks.workunit.client.1.smithi121.stdout:2/506: truncate d5/d49/d4d/d22/f3c 9015923 0 2024-02-15T18:35:42.666 INFO:tasks.workunit.client.1.smithi121.stdout:6/854: mkdir d7/d4b/d129 0 2024-02-15T18:35:42.667 INFO:tasks.workunit.client.1.smithi121.stdout:6/855: dread d7/d4b/d60/d6c/dd5/dfa/fc5 [0,4194304] 0 2024-02-15T18:35:42.670 INFO:tasks.workunit.client.1.smithi121.stdout:9/930: dwrite d7/d26/d4f/f7f [0,4194304] 0 2024-02-15T18:35:42.673 INFO:tasks.workunit.client.1.smithi121.stdout:9/931: mknod d7/d26/d4f/d5b/d77/c141 0 2024-02-15T18:35:42.674 INFO:tasks.workunit.client.1.smithi121.stdout:0/865: dread d5/d6b/dab/db4/d7d/d2a/f2b [0,4194304] 0 2024-02-15T18:35:42.675 INFO:tasks.workunit.client.1.smithi121.stdout:6/856: rmdir d7/dc/d34/d42/d62/d63/d6f/d89/da1/d116 0 2024-02-15T18:35:42.675 INFO:tasks.workunit.client.1.smithi121.stdout:6/857: truncate d7/f11c 615619 0 2024-02-15T18:35:42.676 INFO:tasks.workunit.client.1.smithi121.stdout:6/858: symlink d7/l12a 0 2024-02-15T18:35:42.682 INFO:tasks.workunit.client.1.smithi121.stdout:5/900: dwrite d0/d2/d1e/d4e/d7d/fe2 [0,4194304] 0 2024-02-15T18:35:42.685 INFO:tasks.workunit.client.1.smithi121.stdout:3/582: dwrite d2/d1a/d27/da7/f42 [0,4194304] 0 2024-02-15T18:35:42.685 INFO:tasks.workunit.client.1.smithi121.stdout:0/866: dread d5/d30/fe6 [0,4194304] 0 2024-02-15T18:35:42.689 INFO:tasks.workunit.client.1.smithi121.stdout:0/867: truncate d5/f6a 1136957 0 2024-02-15T18:35:42.691 INFO:tasks.workunit.client.1.smithi121.stdout:4/721: dwrite d5/d9/d1d/d26/d73/d81/fd4 [0,4194304] 0 2024-02-15T18:35:42.691 INFO:tasks.workunit.client.1.smithi121.stdout:0/868: mkdir d5/d6b/dab/db4/d7d/d2a/d4e/d127 0 2024-02-15T18:35:42.694 INFO:tasks.workunit.client.1.smithi121.stdout:0/869: getdents d5/d6b/dab/db4/d7d/d2a 0 2024-02-15T18:35:42.694 INFO:tasks.workunit.client.1.smithi121.stdout:0/870: chown d5/d6b/dab/db4/d7d/fff 12959 1 2024-02-15T18:35:42.698 INFO:tasks.workunit.client.1.smithi121.stdout:5/901: mknod d0/d2/d1e/d4e/d7d/c126 0 2024-02-15T18:35:42.699 INFO:tasks.workunit.client.1.smithi121.stdout:4/722: read d5/d9/d1d/f41 [474600,93003] 0 2024-02-15T18:35:42.702 INFO:tasks.workunit.client.1.smithi121.stdout:5/902: mkdir d0/d2/d1e/d4e/d127 0 2024-02-15T18:35:42.703 INFO:tasks.workunit.client.1.smithi121.stdout:4/723: mknod d5/d31/d3d/d4f/d51/d71/dca/ce1 0 2024-02-15T18:35:42.711 INFO:tasks.workunit.client.1.smithi121.stdout:5/903: creat d0/d2/d1e/d4e/d7d/d9d/d4c/daa/f128 x:0 0 0 2024-02-15T18:35:42.713 INFO:tasks.workunit.client.1.smithi121.stdout:2/507: dwrite d5/df/f6c [0,4194304] 0 2024-02-15T18:35:42.714 INFO:tasks.workunit.client.1.smithi121.stdout:2/508: chown d5/df/d10/l67 165171764 1 2024-02-15T18:35:42.715 INFO:tasks.workunit.client.1.smithi121.stdout:2/509: creat d5/d49/d4d/d22/d81/d8d/f90 x:0 0 0 2024-02-15T18:35:42.717 INFO:tasks.workunit.client.1.smithi121.stdout:2/510: symlink d5/d49/d4d/d22/d81/d8d/l91 0 2024-02-15T18:35:42.717 INFO:tasks.workunit.client.1.smithi121.stdout:2/511: readlink d5/df/d3f/l57 0 2024-02-15T18:35:42.717 INFO:tasks.workunit.client.1.smithi121.stdout:2/512: creat d5/df/d54/f92 x:0 0 0 2024-02-15T18:35:42.718 INFO:tasks.workunit.client.1.smithi121.stdout:2/513: dread d5/df/f2b [0,4194304] 0 2024-02-15T18:35:42.721 INFO:tasks.workunit.client.1.smithi121.stdout:5/904: dread d0/d2/d18/f20 [0,4194304] 0 2024-02-15T18:35:42.722 INFO:tasks.workunit.client.1.smithi121.stdout:5/905: creat d0/f129 x:0 0 0 2024-02-15T18:35:42.724 INFO:tasks.workunit.client.1.smithi121.stdout:5/906: creat d0/d2/f12a x:0 0 0 2024-02-15T18:35:42.725 INFO:tasks.workunit.client.1.smithi121.stdout:5/907: creat d0/d2/d70/f12b x:0 0 0 2024-02-15T18:35:42.730 INFO:tasks.workunit.client.1.smithi121.stdout:9/932: dwrite d7/d3a/d8e/d9e/da9/d8b/d73/f100 [0,4194304] 0 2024-02-15T18:35:42.731 INFO:tasks.workunit.client.1.smithi121.stdout:9/933: read - d7/d26/d4f/d5b/d77/fe6 zero size 2024-02-15T18:35:42.752 INFO:tasks.workunit.client.1.smithi121.stdout:0/871: dwrite d5/d6b/fa8 [0,4194304] 0 2024-02-15T18:35:42.754 INFO:tasks.workunit.client.1.smithi121.stdout:0/872: stat d5/d6b/dab/db4/d7d/d2a/c7b 0 2024-02-15T18:35:42.758 INFO:tasks.workunit.client.1.smithi121.stdout:4/724: dwrite d5/d9/d76/d9e/f75 [0,4194304] 0 2024-02-15T18:35:42.759 INFO:tasks.workunit.client.1.smithi121.stdout:4/725: fdatasync d5/d9/d76/d9e/f38 0 2024-02-15T18:35:42.759 INFO:tasks.workunit.client.1.smithi121.stdout:4/726: chown c3 889 1 2024-02-15T18:35:42.761 INFO:tasks.workunit.client.1.smithi121.stdout:2/514: write d5/d49/d4d/d22/f80 [583198,70100] 0 2024-02-15T18:35:42.761 INFO:tasks.workunit.client.1.smithi121.stdout:2/515: readlink d5/d49/d4d/d22/l83 0 2024-02-15T18:35:42.761 INFO:tasks.workunit.client.1.smithi121.stdout:2/516: read - d5/d49/d4d/d22/d81/d8d/f90 zero size 2024-02-15T18:35:42.761 INFO:tasks.workunit.client.1.smithi121.stdout:3/583: dwrite d2/d1a/d27/f4b [0,4194304] 0 2024-02-15T18:35:42.762 INFO:tasks.workunit.client.1.smithi121.stdout:4/727: symlink d5/d9/d19/da0/le2 0 2024-02-15T18:35:42.764 INFO:tasks.workunit.client.1.smithi121.stdout:5/908: dwrite d0/d16/d34/da4/f117 [0,4194304] 0 2024-02-15T18:35:42.766 INFO:tasks.workunit.client.1.smithi121.stdout:4/728: creat d5/d9/d19/d2a/fe3 x:0 0 0 2024-02-15T18:35:42.766 INFO:tasks.workunit.client.1.smithi121.stdout:4/729: dread - d5/d9/d76/d87/fab zero size 2024-02-15T18:35:42.771 INFO:tasks.workunit.client.1.smithi121.stdout:4/730: dread d5/d9/f2e [0,4194304] 0 2024-02-15T18:35:42.771 INFO:tasks.workunit.client.1.smithi121.stdout:4/731: chown d5/d31/d3d/d4f/cc1 2 1 2024-02-15T18:35:42.782 INFO:tasks.workunit.client.1.smithi121.stdout:6/859: write d7/dc/f90 [4021980,10923] 0 2024-02-15T18:35:42.784 INFO:tasks.workunit.client.1.smithi121.stdout:0/873: dwrite d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/f120 [0,4194304] 0 2024-02-15T18:35:42.788 INFO:tasks.workunit.client.1.smithi121.stdout:2/517: dread d5/d49/d4d/d22/f80 [0,4194304] 0 2024-02-15T18:35:42.789 INFO:tasks.workunit.client.1.smithi121.stdout:5/909: dwrite d0/d2/d1e/d4e/d7d/d40/d62/db1/fc3 [0,4194304] 0 2024-02-15T18:35:42.792 INFO:tasks.workunit.client.1.smithi121.stdout:0/874: getdents d5/d6b/dab/db4/d7d 0 2024-02-15T18:35:42.793 INFO:tasks.workunit.client.1.smithi121.stdout:0/875: getdents d5/d9b 0 2024-02-15T18:35:42.797 INFO:tasks.workunit.client.1.smithi121.stdout:4/732: dwrite d5/d9/d1d/f2f [0,4194304] 0 2024-02-15T18:35:42.805 INFO:tasks.workunit.client.1.smithi121.stdout:0/876: write d5/f6a [1316982,68520] 0 2024-02-15T18:35:42.806 INFO:tasks.workunit.client.1.smithi121.stdout:5/910: mknod d0/d16/d7a/c12c 0 2024-02-15T18:35:42.807 INFO:tasks.workunit.client.1.smithi121.stdout:5/911: fsync d0/d2/d18/f29 0 2024-02-15T18:35:42.807 INFO:tasks.workunit.client.1.smithi121.stdout:0/877: write d5/d6b/dab/db4/d7d/d2a/d4e/fcc [880564,45110] 0 2024-02-15T18:35:42.807 INFO:tasks.workunit.client.1.smithi121.stdout:0/878: chown d5/d6b/dab/db4/d7d/d2a/d36/d44/db7/f108 0 1 2024-02-15T18:35:42.807 INFO:tasks.workunit.client.1.smithi121.stdout:0/879: chown d5/d30/fdd 311787 1 2024-02-15T18:35:42.807 INFO:tasks.workunit.client.1.smithi121.stdout:0/880: chown d5/d30/d5d/f111 598439 1 2024-02-15T18:35:42.808 INFO:tasks.workunit.client.1.smithi121.stdout:4/733: creat d5/d31/d25/fe4 x:0 0 0 2024-02-15T18:35:42.808 INFO:tasks.workunit.client.1.smithi121.stdout:4/734: getdents d5/d9/d19/d57 0 2024-02-15T18:35:42.811 INFO:tasks.workunit.client.1.smithi121.stdout:0/881: dread d5/d30/f4f [0,4194304] 0 2024-02-15T18:35:42.812 INFO:tasks.workunit.client.1.smithi121.stdout:0/882: readlink d5/d6b/dab/db4/d7d/dd/l11e 0 2024-02-15T18:35:42.812 INFO:tasks.workunit.client.1.smithi121.stdout:0/883: fsync d5/d6b/dab/db4/d7d/d2a/d73/d97/ffb 0 2024-02-15T18:35:42.812 INFO:tasks.workunit.client.1.smithi121.stdout:2/518: unlink d5/d49/d4d/c1e 0 2024-02-15T18:35:42.812 INFO:tasks.workunit.client.1.smithi121.stdout:4/735: creat d5/d9/d1d/d26/d73/d81/fe5 x:0 0 0 2024-02-15T18:35:42.813 INFO:tasks.workunit.client.1.smithi121.stdout:5/912: dread d0/d2/d1e/d4e/d7d/d9d/d4c/daa/ffc [0,4194304] 0 2024-02-15T18:35:42.813 INFO:tasks.workunit.client.1.smithi121.stdout:5/913: stat d0/d2/d1e/d4e/d7d/d40/cf1 0 2024-02-15T18:35:42.813 INFO:tasks.workunit.client.1.smithi121.stdout:0/884: mkdir d5/d6b/dab/db4/dca/d128 0 2024-02-15T18:35:42.815 INFO:tasks.workunit.client.1.smithi121.stdout:2/519: truncate d5/df/f1b 86285 0 2024-02-15T18:35:42.817 INFO:tasks.workunit.client.1.smithi121.stdout:4/736: mkdir d5/d9/d19/d4c/dc3/de6 0 2024-02-15T18:35:42.818 INFO:tasks.workunit.client.1.smithi121.stdout:5/914: symlink d0/d2/d18/l12d 0 2024-02-15T18:35:42.818 INFO:tasks.workunit.client.1.smithi121.stdout:5/915: chown d0/d2/d70 125 1 2024-02-15T18:35:42.818 INFO:tasks.workunit.client.1.smithi121.stdout:5/916: fsync d0/d16/d34/da4/f11f 0 2024-02-15T18:35:42.819 INFO:tasks.workunit.client.1.smithi121.stdout:2/520: dread d5/f3b [0,4194304] 0 2024-02-15T18:35:42.819 INFO:tasks.workunit.client.1.smithi121.stdout:2/521: readlink d5/df/l12 0 2024-02-15T18:35:42.821 INFO:tasks.workunit.client.1.smithi121.stdout:5/917: creat d0/d2/d1e/d4e/d7d/d47/d5a/dbc/f12e x:0 0 0 2024-02-15T18:35:42.821 INFO:tasks.workunit.client.1.smithi121.stdout:4/737: dread d5/d9/d19/d2a/f96 [0,4194304] 0 2024-02-15T18:35:42.821 INFO:tasks.workunit.client.1.smithi121.stdout:4/738: chown d5/d31/d3d/d4f/d51/c7d 2430 1 2024-02-15T18:35:42.822 INFO:tasks.workunit.client.1.smithi121.stdout:5/918: symlink d0/d16/d34/da4/l12f 0 2024-02-15T18:35:42.823 INFO:tasks.workunit.client.1.smithi121.stdout:5/919: write d0/d2/fd [8647015,75592] 0 2024-02-15T18:35:42.846 INFO:tasks.workunit.client.1.smithi121.stdout:4/739: rename d5/d31/d3d/d4f/d51/f5d to d5/d9/d19/d4c/dc3/de6/fe7 0 2024-02-15T18:35:42.846 INFO:tasks.workunit.client.1.smithi121.stdout:4/740: stat d5/d9/d19/d2a/fe3 0 2024-02-15T18:35:42.851 INFO:tasks.workunit.client.1.smithi121.stdout:5/920: rename d0/d2/d1e/d4e/d7d/d47/d5a/db3/f11d to d0/d2/d70/f130 0 2024-02-15T18:35:42.851 INFO:tasks.workunit.client.1.smithi121.stdout:4/741: rename d5/d9/d76/d87 to d5/d9/d76/d87/de8 22 2024-02-15T18:35:42.854 INFO:tasks.workunit.client.1.smithi121.stdout:5/921: symlink d0/d2/d1e/d4e/d7d/d47/d5a/db3/l131 0 2024-02-15T18:35:42.855 INFO:tasks.workunit.client.1.smithi121.stdout:4/742: read d5/d31/d3d/d9b/fa5 [2814011,109108] 0 2024-02-15T18:35:42.855 INFO:tasks.workunit.client.1.smithi121.stdout:4/743: stat d5/d9/d1d/d26 0 2024-02-15T18:35:42.857 INFO:tasks.workunit.client.1.smithi121.stdout:0/885: dwrite d5/d6b/dab/db4/d7d/d74/d96/de4/ff1 [0,4194304] 0 2024-02-15T18:35:42.857 INFO:tasks.workunit.client.1.smithi121.stdout:0/886: chown d5/d6b/dab/db4/d7d/d2a/d73/d8d/dc8/de0 541335 1 2024-02-15T18:35:42.858 INFO:tasks.workunit.client.1.smithi121.stdout:5/922: rename d0/d2/d1e/d4e/d7d/d40/lfb to d0/d2/d1e/d4e/d7d/d47/d5a/l132 0 2024-02-15T18:35:42.858 INFO:tasks.workunit.client.1.smithi121.stdout:2/522: dwrite d5/df/d27/d61/f5c [0,4194304] 0 2024-02-15T18:35:42.859 INFO:tasks.workunit.client.1.smithi121.stdout:2/523: fsync d5/df/f6c 0 2024-02-15T18:35:42.861 INFO:tasks.workunit.client.1.smithi121.stdout:2/524: read d5/df/d27/f79 [214251,52362] 0 2024-02-15T18:35:42.862 INFO:tasks.workunit.client.1.smithi121.stdout:4/744: creat d5/d31/d25/db6/fe9 x:0 0 0 2024-02-15T18:35:42.866 INFO:tasks.workunit.client.1.smithi121.stdout:4/745: dread d5/d9/d76/d9e/f38 [0,4194304] 0 2024-02-15T18:35:42.868 INFO:tasks.workunit.client.1.smithi121.stdout:4/746: creat d5/d9/d19/d4c/d8d/fea x:0 0 0 2024-02-15T18:35:42.872 INFO:tasks.workunit.client.1.smithi121.stdout:4/747: write d5/d31/f23 [3720710,63400] 0 2024-02-15T18:35:42.872 INFO:tasks.workunit.client.1.smithi121.stdout:4/748: truncate d5/d31/d25/fd2 939556 0 2024-02-15T18:35:42.872 INFO:tasks.workunit.client.1.smithi121.stdout:4/749: dread - d5/d31/d25/fe4 zero size 2024-02-15T18:35:42.875 INFO:tasks.workunit.client.1.smithi121.stdout:4/750: mknod d5/d31/d3d/d4f/ceb 0 2024-02-15T18:35:42.876 INFO:tasks.workunit.client.1.smithi121.stdout:4/751: unlink d5/d9/d19/d28/f21 0 2024-02-15T18:35:42.878 INFO:tasks.workunit.client.1.smithi121.stdout:4/752: unlink c3 0 2024-02-15T18:35:42.880 INFO:tasks.workunit.client.1.smithi121.stdout:4/753: mknod d5/d9/d1d/cec 0 2024-02-15T18:35:42.880 INFO:tasks.workunit.client.1.smithi121.stdout:4/754: fsync d5/fa 0 2024-02-15T18:35:42.882 INFO:tasks.workunit.client.1.smithi121.stdout:4/755: creat d5/d9/d76/d9e/fed x:0 0 0 2024-02-15T18:35:42.884 INFO:tasks.workunit.client.1.smithi121.stdout:4/756: dread d5/d9/d19/d2a/f68 [0,4194304] 0 2024-02-15T18:35:42.895 INFO:tasks.workunit.client.1.smithi121.stdout:5/923: dwrite d0/d2/d1e/d4e/d7d/d47/d5a/db3/ffd [0,4194304] 0 2024-02-15T18:35:42.896 INFO:tasks.workunit.client.1.smithi121.stdout:0/887: dwrite d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/f120 [4194304,4194304] 0 2024-02-15T18:35:42.900 INFO:tasks.workunit.client.1.smithi121.stdout:2/525: dwrite d5/d49/d4d/d22/f44 [0,4194304] 0 2024-02-15T18:35:42.902 INFO:tasks.workunit.client.1.smithi121.stdout:4/757: dread d5/fa [4194304,4194304] 0 2024-02-15T18:35:42.903 INFO:tasks.workunit.client.1.smithi121.stdout:2/526: creat d5/df/d27/d3e/f93 x:0 0 0 2024-02-15T18:35:42.903 INFO:tasks.workunit.client.1.smithi121.stdout:0/888: unlink d5/d6b/dab/db4/d7d/dd/c5c 0 2024-02-15T18:35:42.905 INFO:tasks.workunit.client.1.smithi121.stdout:2/527: mkdir d5/d49/d4d/d58/d63/d94 0 2024-02-15T18:35:42.905 INFO:tasks.workunit.client.1.smithi121.stdout:5/924: unlink d0/d2/d1e/d4e/d7d/l24 0 2024-02-15T18:35:42.906 INFO:tasks.workunit.client.1.smithi121.stdout:0/889: symlink d5/d6b/dab/db4/d7d/d2a/d73/d8d/dc8/de0/l129 0 2024-02-15T18:35:42.907 INFO:tasks.workunit.client.1.smithi121.stdout:2/528: mknod d5/df/d27/c95 0 2024-02-15T18:35:42.907 INFO:tasks.workunit.client.1.smithi121.stdout:2/529: chown d5/d49/d71/l74 25904 1 2024-02-15T18:35:42.909 INFO:tasks.workunit.client.1.smithi121.stdout:2/530: symlink d5/d49/d4d/d58/d63/d94/l96 0 2024-02-15T18:35:42.909 INFO:tasks.workunit.client.1.smithi121.stdout:2/531: dread - d5/d49/d4d/d22/d81/d8d/f90 zero size 2024-02-15T18:35:42.909 INFO:tasks.workunit.client.1.smithi121.stdout:2/532: fsync d5/d49/d4d/d58/f87 0 2024-02-15T18:35:42.909 INFO:tasks.workunit.client.1.smithi121.stdout:2/533: write d5/d49/f84 [427499,9808] 0 2024-02-15T18:35:42.909 INFO:tasks.workunit.client.1.smithi121.stdout:5/925: rename d0/d2/d1e/d4e/d7d/d47/d5a/da7/fbd to d0/d2/d18/f133 0 2024-02-15T18:35:42.909 INFO:tasks.workunit.client.1.smithi121.stdout:5/926: chown d0/d2/d1e/d4e/d7d/d9d/d4c/daa/ld0 131 1 2024-02-15T18:35:42.909 INFO:tasks.workunit.client.1.smithi121.stdout:5/927: chown d0/d2/d1e/d4e/d7d/d40/l85 38994 1 2024-02-15T18:35:42.911 INFO:tasks.workunit.client.1.smithi121.stdout:5/928: creat d0/d2/d1e/d4e/d7d/d9d/d4c/daa/f134 x:0 0 0 2024-02-15T18:35:42.916 INFO:tasks.workunit.client.1.smithi121.stdout:0/890: write d5/d6b/dab/db1/dbf/fd9 [3307156,58582] 0 2024-02-15T18:35:42.916 INFO:tasks.workunit.client.1.smithi121.stdout:0/891: write d5/d30/fef [639499,15489] 0 2024-02-15T18:35:42.918 INFO:tasks.workunit.client.1.smithi121.stdout:0/892: link d5/d6b/dab/db4/d7d/d2a/d73/d97/cc2 d5/d6b/dab/db4/dca/c12a 0 2024-02-15T18:35:42.922 INFO:tasks.workunit.client.1.smithi121.stdout:0/893: mkdir d5/d6b/dab/db4/d7d/d2a/d4e/d127/d12b 0 2024-02-15T18:35:42.924 INFO:tasks.workunit.client.1.smithi121.stdout:2/534: rmdir d5/df/d27/d3e 39 2024-02-15T18:35:42.925 INFO:tasks.workunit.client.1.smithi121.stdout:8/673: sync 2024-02-15T18:35:42.926 INFO:tasks.workunit.client.1.smithi121.stdout:0/894: creat d5/d6b/dab/db4/d7d/d2a/d73/d81/ddb/deb/f12c x:0 0 0 2024-02-15T18:35:42.928 INFO:tasks.workunit.client.1.smithi121.stdout:0/895: mkdir d5/d6b/dab/db4/d7d/d74/d96/de4/d12d 0 2024-02-15T18:35:42.929 INFO:tasks.workunit.client.1.smithi121.stdout:4/758: dwrite d5/d31/d25/f37 [0,4194304] 0 2024-02-15T18:35:42.934 INFO:tasks.workunit.client.1.smithi121.stdout:8/674: dread d0/d7/db1/dbb/f78 [0,4194304] 0 2024-02-15T18:35:42.935 INFO:tasks.workunit.client.1.smithi121.stdout:8/675: write d0/d7/d36/d8c/f4f [1367113,75581] 0 2024-02-15T18:35:42.935 INFO:tasks.workunit.client.1.smithi121.stdout:8/676: write d0/d7/d36/fba [561846,108860] 0 2024-02-15T18:35:42.935 INFO:tasks.workunit.client.1.smithi121.stdout:8/677: fdatasync d0/d7/fa4 0 2024-02-15T18:35:42.935 INFO:tasks.workunit.client.1.smithi121.stdout:5/929: dwrite d0/d2/d1e/d4e/d7d/d9d/d4c/fd9 [0,4194304] 0 2024-02-15T18:35:42.937 INFO:tasks.workunit.client.1.smithi121.stdout:8/678: rename d0/d48/dd4 to d0/d7/d36/d8c/de9 0 2024-02-15T18:35:42.943 INFO:tasks.workunit.client.1.smithi121.stdout:4/759: unlink d5/d31/c34 0 2024-02-15T18:35:42.943 INFO:tasks.workunit.client.1.smithi121.stdout:4/760: fsync d5/d9/d19/f4e 0 2024-02-15T18:35:42.943 INFO:tasks.workunit.client.1.smithi121.stdout:4/761: getdents d5/d9/d19/d57 0 2024-02-15T18:35:42.943 INFO:tasks.workunit.client.1.smithi121.stdout:0/896: unlink d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/f120 0 2024-02-15T18:35:42.943 INFO:tasks.workunit.client.1.smithi121.stdout:0/897: write d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/fea [448541,8915] 0 2024-02-15T18:35:42.945 INFO:tasks.workunit.client.1.smithi121.stdout:0/898: getdents d5/d6b/dab/db4/d7d/d2a/d73/d97/db8 0 2024-02-15T18:35:42.945 INFO:tasks.workunit.client.1.smithi121.stdout:0/899: write d5/d6b/dab/db4/d7d/dd/f68 [2798830,2135] 0 2024-02-15T18:35:42.946 INFO:tasks.workunit.client.1.smithi121.stdout:2/535: dwrite d5/df/d27/d61/f5c [0,4194304] 0 2024-02-15T18:35:42.950 INFO:tasks.workunit.client.1.smithi121.stdout:4/762: rename d5/l3a to d5/d9/d76/d87/d95/lee 0 2024-02-15T18:35:42.951 INFO:tasks.workunit.client.1.smithi121.stdout:0/900: mkdir d5/d6b/dab/db4/d7d/d2a/d73/d12e 0 2024-02-15T18:35:42.951 INFO:tasks.workunit.client.1.smithi121.stdout:8/679: dread d0/d2a/d51/f6b [0,4194304] 0 2024-02-15T18:35:42.951 INFO:tasks.workunit.client.1.smithi121.stdout:8/680: fdatasync d0/d2a/fd5 0 2024-02-15T18:35:42.952 INFO:tasks.workunit.client.1.smithi121.stdout:2/536: symlink d5/df/d54/l97 0 2024-02-15T18:35:42.954 INFO:tasks.workunit.client.1.smithi121.stdout:0/901: write d5/d6b/dab/db4/d7d/d74/f82 [1010170,38183] 0 2024-02-15T18:35:42.954 INFO:tasks.workunit.client.1.smithi121.stdout:2/537: read d5/df/d27/f79 [4952,120733] 0 2024-02-15T18:35:42.954 INFO:tasks.workunit.client.1.smithi121.stdout:2/538: readlink d5/d49/d4d/d22/l5b 0 2024-02-15T18:35:42.954 INFO:tasks.workunit.client.1.smithi121.stdout:2/539: dread - d5/df/d10/d4c/f50 zero size 2024-02-15T18:35:42.954 INFO:tasks.workunit.client.1.smithi121.stdout:4/763: write d5/d9/d76/d9e/d14/f29 [2957036,58418] 0 2024-02-15T18:35:42.959 INFO:tasks.workunit.client.1.smithi121.stdout:8/681: creat d0/d73/fea x:0 0 0 2024-02-15T18:35:42.961 INFO:tasks.workunit.client.1.smithi121.stdout:0/902: symlink d5/d6b/dab/db4/d7d/d2a/d73/d81/l12f 0 2024-02-15T18:35:42.961 INFO:tasks.workunit.client.1.smithi121.stdout:0/903: readlink d5/d30/l116 0 2024-02-15T18:35:42.962 INFO:tasks.workunit.client.1.smithi121.stdout:0/904: write d5/d9b/fbc [911308,50299] 0 2024-02-15T18:35:42.962 INFO:tasks.workunit.client.1.smithi121.stdout:8/682: write d0/f8 [3533795,121123] 0 2024-02-15T18:35:42.962 INFO:tasks.workunit.client.1.smithi121.stdout:2/540: getdents d5/df/d27/d3e/d6a 0 2024-02-15T18:35:42.962 INFO:tasks.workunit.client.1.smithi121.stdout:4/764: unlink d5/d31/d3d/d4f/d51/d63/dc8/lc9 0 2024-02-15T18:35:42.965 INFO:tasks.workunit.client.1.smithi121.stdout:4/765: read d5/d9/d76/d9e/f75 [1991141,117876] 0 2024-02-15T18:35:42.966 INFO:tasks.workunit.client.1.smithi121.stdout:8/683: creat d0/d7/da5/dcb/feb x:0 0 0 2024-02-15T18:35:42.966 INFO:tasks.workunit.client.1.smithi121.stdout:8/684: fdatasync d0/d61/d66/f80 0 2024-02-15T18:35:42.966 INFO:tasks.workunit.client.1.smithi121.stdout:8/685: readlink d0/l4 0 2024-02-15T18:35:42.968 INFO:tasks.workunit.client.1.smithi121.stdout:8/686: creat d0/d7/de/d67/fec x:0 0 0 2024-02-15T18:35:42.969 INFO:tasks.workunit.client.1.smithi121.stdout:8/687: mknod d0/d7/d36/d52/ced 0 2024-02-15T18:35:42.970 INFO:tasks.workunit.client.1.smithi121.stdout:4/766: stat d5/d9/d76/d87/cc7 0 2024-02-15T18:35:42.971 INFO:tasks.workunit.client.1.smithi121.stdout:8/688: creat d0/d61/d7b/fee x:0 0 0 2024-02-15T18:35:42.973 INFO:tasks.workunit.client.1.smithi121.stdout:8/689: creat d0/d7/d36/fef x:0 0 0 2024-02-15T18:35:42.973 INFO:tasks.workunit.client.1.smithi121.stdout:8/690: readlink d0/d7/d36/l60 0 2024-02-15T18:35:42.974 INFO:tasks.workunit.client.1.smithi121.stdout:8/691: mknod d0/d7/d36/d8c/d45/d53/cf0 0 2024-02-15T18:35:42.975 INFO:tasks.workunit.client.1.smithi121.stdout:8/692: symlink d0/d7/d36/d8c/d45/lf1 0 2024-02-15T18:35:42.982 INFO:tasks.workunit.client.1.smithi121.stdout:2/541: dwrite d5/df/d27/d3a/f43 [0,4194304] 0 2024-02-15T18:35:42.983 INFO:tasks.workunit.client.1.smithi121.stdout:5/930: dwrite d0/d2/d1e/d4e/d7d/d9d/f56 [0,4194304] 0 2024-02-15T18:35:42.984 INFO:tasks.workunit.client.1.smithi121.stdout:4/767: dread d5/d31/d25/f37 [0,4194304] 0 2024-02-15T18:35:42.996 INFO:tasks.workunit.client.1.smithi121.stdout:4/768: dread d5/d31/d3d/d4f/d51/d71/fbc [0,4194304] 0 2024-02-15T18:35:42.998 INFO:tasks.workunit.client.1.smithi121.stdout:4/769: creat d5/d9/d76/d87/d95/fef x:0 0 0 2024-02-15T18:35:43.002 INFO:tasks.workunit.client.1.smithi121.stdout:4/770: symlink d5/d9/d76/d87/lf0 0 2024-02-15T18:35:43.005 INFO:tasks.workunit.client.1.smithi121.stdout:0/905: dwrite d5/fa4 [0,4194304] 0 2024-02-15T18:35:43.005 INFO:tasks.workunit.client.1.smithi121.stdout:0/906: write d5/d6b/dab/db4/d7d/d74/d96/faa [216933,119106] 0 2024-02-15T18:35:43.006 INFO:tasks.workunit.client.1.smithi121.stdout:4/771: read d5/d31/d3d/d4a/d7e/fcd [775957,18816] 0 2024-02-15T18:35:43.007 INFO:tasks.workunit.client.1.smithi121.stdout:4/772: creat d5/d9/d1d/d26/d73/d81/ff1 x:0 0 0 2024-02-15T18:35:43.007 INFO:tasks.workunit.client.1.smithi121.stdout:4/773: fdatasync d5/f6 0 2024-02-15T18:35:43.008 INFO:tasks.workunit.client.1.smithi121.stdout:8/693: dwrite d0/d7/db1/dbb/fa2 [0,4194304] 0 2024-02-15T18:35:43.008 INFO:tasks.workunit.client.1.smithi121.stdout:8/694: chown d0/d7/db1/le1 1 1 2024-02-15T18:35:43.008 INFO:tasks.workunit.client.1.smithi121.stdout:4/774: rename d5/d31/d3d/d4f/d51/f47 to d5/d31/d3d/d4f/d51/ff2 0 2024-02-15T18:35:43.008 INFO:tasks.workunit.client.1.smithi121.stdout:4/775: stat d5/d9/d1d/d26/d73/d81/fd4 0 2024-02-15T18:35:43.010 INFO:tasks.workunit.client.1.smithi121.stdout:5/931: dwrite d0/d2/d1e/d4e/d7d/fe2 [0,4194304] 0 2024-02-15T18:35:43.010 INFO:tasks.workunit.client.1.smithi121.stdout:8/695: creat d0/d3/ff2 x:0 0 0 2024-02-15T18:35:43.011 INFO:tasks.workunit.client.1.smithi121.stdout:4/776: getdents d5/d31/d3d/d4f/d51/d63/dc8 0 2024-02-15T18:35:43.011 INFO:tasks.workunit.client.1.smithi121.stdout:4/777: fdatasync d5/d9/d76/d9e/f6a 0 2024-02-15T18:35:43.013 INFO:tasks.workunit.client.1.smithi121.stdout:8/696: getdents d0/d7/db1/dbb/de2 0 2024-02-15T18:35:43.015 INFO:tasks.workunit.client.1.smithi121.stdout:0/907: dread d5/d6b/dab/db4/d7d/dd/fb0 [0,4194304] 0 2024-02-15T18:35:43.017 INFO:tasks.workunit.client.1.smithi121.stdout:5/932: rename d0/d2/d1e/d4e/d7d/fb8 to d0/d16/d7a/db5/dca/f135 0 2024-02-15T18:35:43.017 INFO:tasks.workunit.client.1.smithi121.stdout:0/908: getdents d5/d6b/dab/db4/d7d/d2a/d36/d44 0 2024-02-15T18:35:43.021 INFO:tasks.workunit.client.1.smithi121.stdout:8/697: unlink d0/d7/de/c8d 0 2024-02-15T18:35:43.021 INFO:tasks.workunit.client.1.smithi121.stdout:0/909: chown d5/d6b/dab/db4/d7d/d2a/d4e/ff2 48 1 2024-02-15T18:35:43.021 INFO:tasks.workunit.client.1.smithi121.stdout:4/778: getdents d5/d31/d3d/d4f/d51/d80 0 2024-02-15T18:35:43.022 INFO:tasks.workunit.client.1.smithi121.stdout:4/779: read d5/d9/fdb [3860519,42929] 0 2024-02-15T18:35:43.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:42.596736+0000 mgr.smithi121.ckiryi (mgr.24385) 15 : cluster [DBG] pgmap v15: 65 pgs: 65 active+clean; 1.6 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 39 MiB/s rd, 87 MiB/s wr, 328 op/s 2024-02-15T18:35:43.025 INFO:tasks.workunit.client.1.smithi121.stdout:8/698: link d0/d61/d7b/fd8 d0/d7/d36/d8c/d45/dcd/ff3 0 2024-02-15T18:35:43.025 INFO:tasks.workunit.client.1.smithi121.stdout:8/699: chown d0/d7/de/d67 7160193 1 2024-02-15T18:35:43.026 INFO:tasks.workunit.client.1.smithi121.stdout:0/910: creat d5/d6b/dab/db4/dca/f130 x:0 0 0 2024-02-15T18:35:43.027 INFO:tasks.workunit.client.1.smithi121.stdout:8/700: write d0/d7/d36/d8c/d45/f3e [2101472,125192] 0 2024-02-15T18:35:43.027 INFO:tasks.workunit.client.1.smithi121.stdout:0/911: creat d5/d6b/dab/db4/d7d/d2a/d73/d81/ddb/deb/f131 x:0 0 0 2024-02-15T18:35:43.028 INFO:tasks.workunit.client.1.smithi121.stdout:0/912: truncate d5/d6b/dab/db4/d7d/d74/d96/de3/ffa 569141 0 2024-02-15T18:35:43.029 INFO:tasks.workunit.client.1.smithi121.stdout:4/780: creat d5/d9/d76/ff3 x:0 0 0 2024-02-15T18:35:43.029 INFO:tasks.workunit.client.1.smithi121.stdout:4/781: readlink d5/d31/d3d/d4f/d51/d80/ld0 0 2024-02-15T18:35:43.030 INFO:tasks.workunit.client.1.smithi121.stdout:4/782: truncate d5/d31/d3d/d4f/d51/d71/f48 4787713 0 2024-02-15T18:35:43.030 INFO:tasks.workunit.client.1.smithi121.stdout:8/701: rename d0/d7/db1/dbb/de2/fe5 to d0/d7/ff4 0 2024-02-15T18:35:43.030 INFO:tasks.workunit.client.1.smithi121.stdout:4/783: write d5/d9/d19/d2a/fe3 [51346,46058] 0 2024-02-15T18:35:43.031 INFO:tasks.workunit.client.1.smithi121.stdout:8/702: mknod d0/d61/d7b/cf5 0 2024-02-15T18:35:43.031 INFO:tasks.workunit.client.1.smithi121.stdout:8/703: read d0/d7/de/d67/fbe [1536306,25577] 0 2024-02-15T18:35:43.032 INFO:tasks.workunit.client.1.smithi121.stdout:8/704: chown d0/d7/d36/d52 582873673 1 2024-02-15T18:35:43.032 INFO:tasks.workunit.client.1.smithi121.stdout:8/705: fdatasync d0/d7/da5/dcb/f9f 0 2024-02-15T18:35:43.034 INFO:tasks.workunit.client.1.smithi121.stdout:4/784: rename d5/d9/d19/d28/f74 to d5/d9/d1d/d26/d72/ff4 0 2024-02-15T18:35:43.035 INFO:tasks.workunit.client.1.smithi121.stdout:5/933: dwrite d0/d2/d1e/d4e/d7d/d47/d5a/db3/ffd [0,4194304] 0 2024-02-15T18:35:43.038 INFO:tasks.workunit.client.1.smithi121.stdout:5/934: creat d0/d16/d7a/f136 x:0 0 0 2024-02-15T18:35:43.039 INFO:tasks.workunit.client.1.smithi121.stdout:5/935: creat d0/d2/d1e/d4e/d7d/d39/f137 x:0 0 0 2024-02-15T18:35:43.054 INFO:tasks.workunit.client.1.smithi121.stdout:5/936: write d0/d16/d34/d8f/ff6 [2196951,55911] 0 2024-02-15T18:35:43.055 INFO:tasks.workunit.client.1.smithi121.stdout:5/937: dread d0/d2/d18/f9e [0,4194304] 0 2024-02-15T18:35:43.065 INFO:tasks.workunit.client.1.smithi121.stdout:0/913: dwrite d5/d30/fdd [0,4194304] 0 2024-02-15T18:35:43.067 INFO:tasks.workunit.client.1.smithi121.stdout:8/706: dwrite d0/d7/d36/f81 [0,4194304] 0 2024-02-15T18:35:43.067 INFO:tasks.workunit.client.1.smithi121.stdout:0/914: mkdir d5/d6b/dab/db4/d7d/d2a/d73/d8d/dc8/de0/d132 0 2024-02-15T18:35:43.070 INFO:tasks.workunit.client.1.smithi121.stdout:4/785: dwrite d5/d9/d1d/d26/d73/d81/ff1 [0,4194304] 0 2024-02-15T18:35:43.070 INFO:tasks.workunit.client.1.smithi121.stdout:5/938: dread d0/d2/d1e/d4e/d7d/d40/f78 [0,4194304] 0 2024-02-15T18:35:43.070 INFO:tasks.workunit.client.1.smithi121.stdout:4/786: chown d5/d31/lc2 377318593 1 2024-02-15T18:35:43.070 INFO:tasks.workunit.client.1.smithi121.stdout:8/707: link d0/d7/db1/le1 d0/d61/d7b/lf6 0 2024-02-15T18:35:43.070 INFO:tasks.workunit.client.1.smithi121.stdout:5/939: read d0/d2/d1e/d4e/d7d/d40/d62/db1/fdd [316886,84468] 0 2024-02-15T18:35:43.070 INFO:tasks.workunit.client.1.smithi121.stdout:0/915: truncate d5/d6b/dab/db4/d7d/d2a/d73/d97/db8/fee 469624 0 2024-02-15T18:35:43.071 INFO:tasks.workunit.client.1.smithi121.stdout:0/916: read d5/f6a [632678,69739] 0 2024-02-15T18:35:43.071 INFO:tasks.workunit.client.1.smithi121.stdout:0/917: getdents d5/d6b/dab/db4/d7d/d2a/d73/d8d/dc8 0 2024-02-15T18:35:43.072 INFO:tasks.workunit.client.1.smithi121.stdout:0/918: write d5/d6b/dab/db4/d7d/d2a/d36/d44/f115 [642000,75454] 0 2024-02-15T18:35:43.072 INFO:tasks.workunit.client.1.smithi121.stdout:0/919: fdatasync d5/d6b/dab/db4/d7d/d2a/d73/d97/db8/fee 0 2024-02-15T18:35:43.073 INFO:tasks.workunit.client.1.smithi121.stdout:5/940: mkdir d0/d16/d138 0 2024-02-15T18:35:43.075 INFO:tasks.workunit.client.1.smithi121.stdout:0/920: mknod d5/d6b/dab/db4/d7d/d74/d96/de4/c133 0 2024-02-15T18:35:43.075 INFO:tasks.workunit.client.1.smithi121.stdout:8/708: rename d0/d7/de/f2d to d0/d7/d36/d8c/d45/dcd/ff7 0 2024-02-15T18:35:43.075 INFO:tasks.workunit.client.1.smithi121.stdout:8/709: fdatasync d0/d7/de/d67/fec 0 2024-02-15T18:35:43.076 INFO:tasks.workunit.client.1.smithi121.stdout:8/710: dread - d0/d2a/fae zero size 2024-02-15T18:35:43.076 INFO:tasks.workunit.client.1.smithi121.stdout:5/941: creat d0/d2/d1e/d4e/f139 x:0 0 0 2024-02-15T18:35:43.077 INFO:tasks.workunit.client.1.smithi121.stdout:0/921: write d5/f3b [1895745,92949] 0 2024-02-15T18:35:43.078 INFO:tasks.workunit.client.1.smithi121.stdout:8/711: rename d0/d61/da6 to d0/dce/df8 0 2024-02-15T18:35:43.078 INFO:tasks.workunit.client.1.smithi121.stdout:0/922: rmdir d5/d6b/dab/db4/d7d/d2a/d4e/d100 39 2024-02-15T18:35:43.079 INFO:tasks.workunit.client.1.smithi121.stdout:0/923: creat d5/d6b/dab/db1/f134 x:0 0 0 2024-02-15T18:35:43.083 INFO:tasks.workunit.client.1.smithi121.stdout:7/516: sync 2024-02-15T18:35:43.083 INFO:tasks.workunit.client.1.smithi121.stdout:1/895: sync 2024-02-15T18:35:43.086 INFO:tasks.workunit.client.1.smithi121.stdout:4/787: dwrite d5/d9/d19/d2a/f2c [8388608,4194304] 0 2024-02-15T18:35:43.086 INFO:tasks.workunit.client.1.smithi121.stdout:5/942: dread d0/d2/d1e/d4e/f91 [0,4194304] 0 2024-02-15T18:35:43.087 INFO:tasks.workunit.client.1.smithi121.stdout:5/943: fsync d0/f4b 0 2024-02-15T18:35:43.088 INFO:tasks.workunit.client.1.smithi121.stdout:8/712: dread d0/d7/da5/dcb/f91 [0,4194304] 0 2024-02-15T18:35:43.088 INFO:tasks.workunit.client.1.smithi121.stdout:1/896: creat d0/d39/d4b/dac/d111/d9/d14/db6/d11e/f12d x:0 0 0 2024-02-15T18:35:43.092 INFO:tasks.workunit.client.1.smithi121.stdout:8/713: link d0/d48/cc5 d0/d7/db1/cf9 0 2024-02-15T18:35:43.097 INFO:tasks.workunit.client.1.smithi121.stdout:1/897: dread d0/d39/d4b/dac/d111/d9/d14/d3e/f116 [0,4194304] 0 2024-02-15T18:35:43.101 INFO:tasks.workunit.client.1.smithi121.stdout:4/788: mknod d5/d9/d19/d5c/cf5 0 2024-02-15T18:35:43.103 INFO:tasks.workunit.client.1.smithi121.stdout:5/944: mkdir d0/d2/d1e/d4e/d7d/d47/d5a/dbc/d13a 0 2024-02-15T18:35:43.104 INFO:tasks.workunit.client.1.smithi121.stdout:5/945: fdatasync d0/d2/d1e/d4e/f55 0 2024-02-15T18:35:43.104 INFO:tasks.workunit.client.1.smithi121.stdout:5/946: readlink d0/l28 0 2024-02-15T18:35:43.104 INFO:tasks.workunit.client.1.smithi121.stdout:0/924: dwrite d5/d6b/dab/db4/dca/f130 [0,4194304] 0 2024-02-15T18:35:43.107 INFO:tasks.workunit.client.1.smithi121.stdout:1/898: mkdir d0/d39/d7d/dc0/dcf/d12e 0 2024-02-15T18:35:43.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:42.596736+0000 mgr.smithi121.ckiryi (mgr.24385) 15 : cluster [DBG] pgmap v15: 65 pgs: 65 active+clean; 1.6 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 39 MiB/s rd, 87 MiB/s wr, 328 op/s 2024-02-15T18:35:43.109 INFO:tasks.workunit.client.1.smithi121.stdout:5/947: rename d0/d16/d7a/db5/d116/l123 to d0/d2/d1e/d4e/d7d/d9d/d4c/daa/l13b 0 2024-02-15T18:35:43.109 INFO:tasks.workunit.client.1.smithi121.stdout:5/948: stat d0/d2/d1e/d4e/d7d/d9d/d4c/daa/c11b 0 2024-02-15T18:35:43.109 INFO:tasks.workunit.client.1.smithi121.stdout:1/899: getdents d0/d39/d7d 0 2024-02-15T18:35:43.111 INFO:tasks.workunit.client.1.smithi121.stdout:0/925: creat d5/d6b/dab/db4/d7d/f135 x:0 0 0 2024-02-15T18:35:43.113 INFO:tasks.workunit.client.1.smithi121.stdout:0/926: mkdir d5/d6b/dab/db4/d7d/d2a/d73/d97/df4/d136 0 2024-02-15T18:35:43.114 INFO:tasks.workunit.client.1.smithi121.stdout:0/927: mkdir d5/d6b/dab/db4/d7d/d2a/d73/d81/d124/d137 0 2024-02-15T18:35:43.114 INFO:tasks.workunit.client.1.smithi121.stdout:1/900: rename d0/d39/d4b/dac/d111/d9/d14/d3e/d88/daf to d0/d39/d4b/dac/d111/d72/dd9/dfd/d12f 0 2024-02-15T18:35:43.116 INFO:tasks.workunit.client.1.smithi121.stdout:0/928: truncate d5/f92 3427726 0 2024-02-15T18:35:43.117 INFO:tasks.workunit.client.1.smithi121.stdout:5/949: link d0/d2/d70/l104 d0/d2/d1e/d4e/d7d/l13c 0 2024-02-15T18:35:43.118 INFO:tasks.workunit.client.1.smithi121.stdout:1/901: creat d0/d39/d4b/dac/d111/d72/f130 x:0 0 0 2024-02-15T18:35:43.120 INFO:tasks.workunit.client.1.smithi121.stdout:8/714: dwrite d0/d7/d36/fef [0,4194304] 0 2024-02-15T18:35:43.120 INFO:tasks.workunit.client.1.smithi121.stdout:1/902: rename d0/d39/d4b/dac/d111/d50/c8b to d0/d39/d4b/dac/d111/d50/d7e/dec/c131 0 2024-02-15T18:35:43.120 INFO:tasks.workunit.client.1.smithi121.stdout:1/903: readlink d0/d39/d4b/dac/d111/d50/le4 0 2024-02-15T18:35:43.121 INFO:tasks.workunit.client.1.smithi121.stdout:4/789: dwrite d5/d31/d25/db6/fe9 [0,4194304] 0 2024-02-15T18:35:43.123 INFO:tasks.workunit.client.1.smithi121.stdout:1/904: symlink d0/d39/d4b/dac/d111/d9/d51/d5f/l132 0 2024-02-15T18:35:43.129 INFO:tasks.workunit.client.1.smithi121.stdout:7/517: dwrite d0/d38/d46/daa/f3b [0,4194304] 0 2024-02-15T18:35:43.129 INFO:tasks.workunit.client.1.smithi121.stdout:7/518: chown d0/d38/d46/daa/f6e 89152 1 2024-02-15T18:35:43.129 INFO:tasks.workunit.client.1.smithi121.stdout:7/519: dread - d0/d4a/d5b/da6/f7f zero size 2024-02-15T18:35:43.129 INFO:tasks.workunit.client.1.smithi121.stdout:7/520: fsync d0/f52 0 2024-02-15T18:35:43.129 INFO:tasks.workunit.client.1.smithi121.stdout:0/929: dread d5/d6b/dab/db4/d7d/f24 [0,4194304] 0 2024-02-15T18:35:43.130 INFO:tasks.workunit.client.1.smithi121.stdout:0/930: write d5/d6b/dab/db4/d7d/d2a/d36/d44/f115 [299696,130088] 0 2024-02-15T18:35:43.130 INFO:tasks.workunit.client.1.smithi121.stdout:1/905: stat d0/d39/d4b/dac/d111/d9/d14/db6/cf3 0 2024-02-15T18:35:43.130 INFO:tasks.workunit.client.1.smithi121.stdout:1/906: readlink d0/d39/d4b/dac/d111/d9/d14/d3e/d88/da6/l125 0 2024-02-15T18:35:43.132 INFO:tasks.workunit.client.1.smithi121.stdout:0/931: write d5/d6b/dab/db4/d7d/d2a/d73/d97/db8/fee [316578,12969] 0 2024-02-15T18:35:43.132 INFO:tasks.workunit.client.1.smithi121.stdout:7/521: rename d0/l2a to d0/d5/d11/d1c/d5f/lac 0 2024-02-15T18:35:43.134 INFO:tasks.workunit.client.1.smithi121.stdout:7/522: mknod d0/d38/d61/cad 0 2024-02-15T18:35:43.134 INFO:tasks.workunit.client.1.smithi121.stdout:7/523: write d0/f52 [123460,123324] 0 2024-02-15T18:35:43.135 INFO:tasks.workunit.client.1.smithi121.stdout:7/524: write d0/d5/d4c/f59 [54319,117416] 0 2024-02-15T18:35:43.135 INFO:tasks.workunit.client.1.smithi121.stdout:4/790: rmdir d5/d9/d19/d5c 39 2024-02-15T18:35:43.136 INFO:tasks.workunit.client.1.smithi121.stdout:7/525: rename d0/d5/d11/c4f to d0/d4a/d5b/cae 0 2024-02-15T18:35:43.136 INFO:tasks.workunit.client.1.smithi121.stdout:7/526: stat d0/d4a/d5b/f89 0 2024-02-15T18:35:43.136 INFO:tasks.workunit.client.1.smithi121.stdout:7/527: stat d0/f77 0 2024-02-15T18:35:43.136 INFO:tasks.workunit.client.1.smithi121.stdout:0/932: write d5/d6b/dab/db4/d7d/dd/f16 [1978237,103872] 0 2024-02-15T18:35:43.136 INFO:tasks.workunit.client.1.smithi121.stdout:1/907: dread d0/d39/d4b/dac/d111/d9/d14/d3e/d88/da6/fbd [0,4194304] 0 2024-02-15T18:35:43.136 INFO:tasks.workunit.client.1.smithi121.stdout:1/908: dread - d0/d39/d7d/dc0/dcf/f106 zero size 2024-02-15T18:35:43.137 INFO:tasks.workunit.client.1.smithi121.stdout:1/909: fdatasync d0/d39/d4b/dac/f12b 0 2024-02-15T18:35:43.137 INFO:tasks.workunit.client.1.smithi121.stdout:1/910: write d0/d39/d4b/f61 [2657326,22069] 0 2024-02-15T18:35:43.137 INFO:tasks.workunit.client.1.smithi121.stdout:7/528: symlink d0/laf 0 2024-02-15T18:35:43.141 INFO:tasks.workunit.client.1.smithi121.stdout:7/529: truncate d0/f27 5428037 0 2024-02-15T18:35:43.142 INFO:tasks.workunit.client.1.smithi121.stdout:7/530: getdents d0/d5/d4c/d6f 0 2024-02-15T18:35:43.143 INFO:tasks.workunit.client.1.smithi121.stdout:7/531: write d0/d5/d11/d79/f86 [3902621,50285] 0 2024-02-15T18:35:43.145 INFO:tasks.workunit.client.1.smithi121.stdout:7/532: mknod d0/d4a/cb0 0 2024-02-15T18:35:43.145 INFO:tasks.workunit.client.1.smithi121.stdout:7/533: dread - d0/d4a/f7e zero size 2024-02-15T18:35:43.146 INFO:tasks.workunit.client.1.smithi121.stdout:7/534: mknod d0/d38/d3f/cb1 0 2024-02-15T18:35:43.146 INFO:tasks.workunit.client.1.smithi121.stdout:7/535: stat d0/d38/d3f/cb1 0 2024-02-15T18:35:43.147 INFO:tasks.workunit.client.1.smithi121.stdout:7/536: write d0/d38/d3c/f4b [5132559,24245] 0 2024-02-15T18:35:43.147 INFO:tasks.workunit.client.1.smithi121.stdout:7/537: dread - d0/d38/d61/f8a zero size 2024-02-15T18:35:43.148 INFO:tasks.workunit.client.1.smithi121.stdout:1/911: symlink d0/d39/d7d/dc0/dcf/l133 0 2024-02-15T18:35:43.148 INFO:tasks.workunit.client.1.smithi121.stdout:1/912: read d0/d39/d4b/dac/d111/d9/fa2 [372701,71760] 0 2024-02-15T18:35:43.150 INFO:tasks.workunit.client.1.smithi121.stdout:7/538: link d0/d5/d11/f40 d0/d5/d11/d1c/fb2 0 2024-02-15T18:35:43.150 INFO:tasks.workunit.client.1.smithi121.stdout:7/539: write d0/d4a/d5b/f92 [4735,87147] 0 2024-02-15T18:35:43.150 INFO:tasks.workunit.client.1.smithi121.stdout:7/540: fdatasync d0/d38/d46/f96 0 2024-02-15T18:35:43.151 INFO:tasks.workunit.client.1.smithi121.stdout:1/913: mkdir d0/d39/d134 0 2024-02-15T18:35:43.155 INFO:tasks.workunit.client.1.smithi121.stdout:1/914: link d0/d39/d4b/dac/d111/d9/d14/c114 d0/d39/d7d/dc0/d9d/c135 0 2024-02-15T18:35:43.157 INFO:tasks.workunit.client.1.smithi121.stdout:0/933: dwrite d5/d6b/dab/db4/d7d/d2a/d36/f54 [0,4194304] 0 2024-02-15T18:35:43.163 INFO:tasks.workunit.client.1.smithi121.stdout:0/934: dread d5/f6a [0,4194304] 0 2024-02-15T18:35:43.165 INFO:tasks.workunit.client.1.smithi121.stdout:0/935: write d5/d6b/dab/db4/d7d/f56 [351112,113986] 0 2024-02-15T18:35:43.169 INFO:tasks.workunit.client.1.smithi121.stdout:1/915: dread d0/d39/d7d/dc0/fd4 [0,4194304] 0 2024-02-15T18:35:43.170 INFO:tasks.workunit.client.1.smithi121.stdout:1/916: truncate d0/d39/d4b/dac/d111/d9/f35 1521910 0 2024-02-15T18:35:43.170 INFO:tasks.workunit.client.1.smithi121.stdout:1/917: stat d0/d39/d4b/dac/d111/d9/d14/cb7 0 2024-02-15T18:35:43.171 INFO:tasks.workunit.client.1.smithi121.stdout:1/918: creat d0/d39/d4b/dac/d111/d72/dd9/dfd/f136 x:0 0 0 2024-02-15T18:35:43.174 INFO:tasks.workunit.client.1.smithi121.stdout:1/919: mknod d0/d39/d4b/dac/d111/d9/d14/d48/d128/c137 0 2024-02-15T18:35:43.177 INFO:tasks.workunit.client.1.smithi121.stdout:1/920: dread d0/d39/d7d/dc0/fd4 [0,4194304] 0 2024-02-15T18:35:43.178 INFO:tasks.workunit.client.1.smithi121.stdout:1/921: write d0/d39/d7d/dc0/d77/d103/fb1 [3699183,28192] 0 2024-02-15T18:35:43.185 INFO:tasks.workunit.client.1.smithi121.stdout:7/541: dwrite d0/d38/d3f/d43/f8f [0,4194304] 0 2024-02-15T18:35:43.186 INFO:tasks.workunit.client.1.smithi121.stdout:7/542: mkdir d0/d5/d4c/d6f/db3 0 2024-02-15T18:35:43.187 INFO:tasks.workunit.client.1.smithi121.stdout:7/543: symlink d0/d5/d11/d79/lb4 0 2024-02-15T18:35:43.189 INFO:tasks.workunit.client.1.smithi121.stdout:0/936: dwrite d5/d6b/dab/db4/d7d/f43 [0,4194304] 0 2024-02-15T18:35:43.189 INFO:tasks.workunit.client.1.smithi121.stdout:0/937: write d5/d6b/dab/db4/d7d/f11 [6101005,87942] 0 2024-02-15T18:35:43.195 INFO:tasks.workunit.client.1.smithi121.stdout:0/938: mkdir d5/d6b/dab/db4/d7d/d138 0 2024-02-15T18:35:43.196 INFO:tasks.workunit.client.1.smithi121.stdout:0/939: write d5/d6b/dab/db4/d7d/d2a/d73/d81/fd6 [314760,96858] 0 2024-02-15T18:35:43.196 INFO:tasks.workunit.client.1.smithi121.stdout:0/940: chown d5/d6b/dab/db4/d7d/dd/c1b 783978 1 2024-02-15T18:35:43.196 INFO:tasks.workunit.client.1.smithi121.stdout:0/941: write f2 [8670744,28101] 0 2024-02-15T18:35:43.200 INFO:tasks.workunit.client.1.smithi121.stdout:1/922: dwrite d0/d39/d7d/dc0/d9d/f107 [4194304,4194304] 0 2024-02-15T18:35:43.215 INFO:tasks.workunit.client.1.smithi121.stdout:7/544: dwrite d0/f52 [0,4194304] 0 2024-02-15T18:35:43.218 INFO:tasks.workunit.client.1.smithi121.stdout:0/942: dwrite d5/d6b/dab/db4/d7d/d2a/f2b [0,4194304] 0 2024-02-15T18:35:43.218 INFO:tasks.workunit.client.1.smithi121.stdout:0/943: stat d5/d107 0 2024-02-15T18:35:43.220 INFO:tasks.workunit.client.1.smithi121.stdout:7/545: creat d0/d4a/da2/fb5 x:0 0 0 2024-02-15T18:35:43.221 INFO:tasks.workunit.client.1.smithi121.stdout:7/546: mkdir d0/d5/d4c/d6f/db6 0 2024-02-15T18:35:43.221 INFO:tasks.workunit.client.1.smithi121.stdout:7/547: truncate d0/d38/d3c/f4d 4831343 0 2024-02-15T18:35:43.222 INFO:tasks.workunit.client.1.smithi121.stdout:7/548: creat d0/d5/d4c/d6f/db3/fb7 x:0 0 0 2024-02-15T18:35:43.224 INFO:tasks.workunit.client.1.smithi121.stdout:7/549: creat d0/d5/d76/fb8 x:0 0 0 2024-02-15T18:35:43.224 INFO:tasks.workunit.client.1.smithi121.stdout:7/550: stat d0/d4a/d5b/l6b 0 2024-02-15T18:35:43.224 INFO:tasks.workunit.client.1.smithi121.stdout:7/551: readlink d0/d4a/d5b/da6/l6c 0 2024-02-15T18:35:43.224 INFO:tasks.workunit.client.1.smithi121.stdout:7/552: getdents d0/d5/d11 0 2024-02-15T18:35:43.225 INFO:tasks.workunit.client.1.smithi121.stdout:7/553: creat d0/d38/fb9 x:0 0 0 2024-02-15T18:35:43.229 INFO:tasks.workunit.client.1.smithi121.stdout:3/584: sync 2024-02-15T18:35:43.230 INFO:tasks.workunit.client.1.smithi121.stdout:9/934: sync 2024-02-15T18:35:43.230 INFO:tasks.workunit.client.1.smithi121.stdout:3/585: mknod d2/d1a/d27/d63/da4/d25/d31/d97/daf/cc0 0 2024-02-15T18:35:43.231 INFO:tasks.workunit.client.1.smithi121.stdout:9/935: symlink d7/d26/d4f/d5b/d77/l142 0 2024-02-15T18:35:43.232 INFO:tasks.workunit.client.1.smithi121.stdout:1/923: dwrite d0/d39/d7d/dc0/d9d/f107 [0,4194304] 0 2024-02-15T18:35:43.233 INFO:tasks.workunit.client.1.smithi121.stdout:3/586: dread d2/d1a/d27/f7f [0,4194304] 0 2024-02-15T18:35:43.233 INFO:tasks.workunit.client.1.smithi121.stdout:9/936: symlink d7/d8/d25/d30/d33/d79/l143 0 2024-02-15T18:35:43.233 INFO:tasks.workunit.client.1.smithi121.stdout:3/587: write d2/d1a/d27/d63/da4/d25/f52 [3365,39044] 0 2024-02-15T18:35:43.234 INFO:tasks.workunit.client.1.smithi121.stdout:1/924: mknod d0/d39/d7d/dc0/dcf/d12e/c138 0 2024-02-15T18:35:43.234 INFO:tasks.workunit.client.1.smithi121.stdout:3/588: rmdir d2/d1a/d27/d63/d81 39 2024-02-15T18:35:43.235 INFO:tasks.workunit.client.1.smithi121.stdout:3/589: chown d2/d1a/d27/da7/d85/d9f 180841743 1 2024-02-15T18:35:43.236 INFO:tasks.workunit.client.1.smithi121.stdout:3/590: mkdir d2/d1a/d54/dc1 0 2024-02-15T18:35:43.236 INFO:tasks.workunit.client.1.smithi121.stdout:1/925: creat d0/d39/d4b/f139 x:0 0 0 2024-02-15T18:35:43.236 INFO:tasks.workunit.client.1.smithi121.stdout:1/926: write d0/d39/d7d/dc0/fd4 [3394834,96210] 0 2024-02-15T18:35:43.238 INFO:tasks.workunit.client.1.smithi121.stdout:7/554: dread d0/d5/d11/d1c/f3e [4194304,4194304] 0 2024-02-15T18:35:43.239 INFO:tasks.workunit.client.1.smithi121.stdout:0/944: dwrite d5/d6b/dab/db4/d7d/d2a/d73/f7a [0,4194304] 0 2024-02-15T18:35:43.239 INFO:tasks.workunit.client.1.smithi121.stdout:0/945: chown d5/d6b/dab/db4/d7d 83 1 2024-02-15T18:35:43.241 INFO:tasks.workunit.client.1.smithi121.stdout:7/555: write d0/d5/f26 [6350327,75341] 0 2024-02-15T18:35:43.243 INFO:tasks.workunit.client.1.smithi121.stdout:9/937: write d7/d8/d25/d30/d33/d79/ff1 [946381,33252] 0 2024-02-15T18:35:43.244 INFO:tasks.workunit.client.1.smithi121.stdout:3/591: dread d2/f7 [4194304,4194304] 0 2024-02-15T18:35:43.245 INFO:tasks.workunit.client.1.smithi121.stdout:0/946: symlink d5/d6b/dab/db4/d7d/d74/d96/dc4/l139 0 2024-02-15T18:35:43.249 INFO:tasks.workunit.client.1.smithi121.stdout:3/592: mknod d2/d1a/d54/dc1/cc2 0 2024-02-15T18:35:43.250 INFO:tasks.workunit.client.1.smithi121.stdout:0/947: creat d5/d6b/dab/db1/dbf/f13a x:0 0 0 2024-02-15T18:35:43.251 INFO:tasks.workunit.client.1.smithi121.stdout:3/593: creat d2/d1a/d27/da7/fc3 x:0 0 0 2024-02-15T18:35:43.255 INFO:tasks.workunit.client.1.smithi121.stdout:3/594: link d2/d1a/d27/d63/da4/d25/d31/d34/d4a/f6d d2/d5a/fc4 0 2024-02-15T18:35:43.255 INFO:tasks.workunit.client.1.smithi121.stdout:0/948: unlink d5/d6b/dab/db4/d7d/d2a/d36/ld7 0 2024-02-15T18:35:43.258 INFO:tasks.workunit.client.1.smithi121.stdout:3/595: rename d2/d1a/d27/d63/da4/d25/d31/d34/d4a/f99 to d2/d1a/d6b/d7d/fc5 0 2024-02-15T18:35:43.258 INFO:tasks.workunit.client.1.smithi121.stdout:0/949: stat d5/d6b/dab/db4/d7d/d74/c94 0 2024-02-15T18:35:43.259 INFO:tasks.workunit.client.1.smithi121.stdout:3/596: mknod d2/d1a/d27/d63/da4/d25/d88/d9a/cc6 0 2024-02-15T18:35:43.261 INFO:tasks.workunit.client.1.smithi121.stdout:3/597: rename d2/c14 to d2/d1a/d27/d63/da4/d25/d30/db3/cc7 0 2024-02-15T18:35:43.261 INFO:tasks.workunit.client.1.smithi121.stdout:3/598: write d2/d1a/d27/f77 [371264,33398] 0 2024-02-15T18:35:43.262 INFO:tasks.workunit.client.1.smithi121.stdout:3/599: rmdir d2/d1a/da8 0 2024-02-15T18:35:43.262 INFO:tasks.workunit.client.1.smithi121.stdout:3/600: write d2/d1a/d27/d63/da4/f2f [3294891,47619] 0 2024-02-15T18:35:43.275 INFO:tasks.workunit.client.1.smithi121.stdout:0/950: dread d5/d6b/dab/db1/dbf/f117 [0,4194304] 0 2024-02-15T18:35:43.275 INFO:tasks.workunit.client.1.smithi121.stdout:7/556: dwrite d0/d38/d46/daa/f34 [0,4194304] 0 2024-02-15T18:35:43.275 INFO:tasks.workunit.client.1.smithi121.stdout:0/951: chown d5/d30/ff6 6745445 1 2024-02-15T18:35:43.275 INFO:tasks.workunit.client.1.smithi121.stdout:1/927: dwrite d0/d39/d4b/dac/d111/d9/d14/db6/ff8 [0,4194304] 0 2024-02-15T18:35:43.276 INFO:tasks.workunit.client.1.smithi121.stdout:1/928: fsync d0/d39/d4b/dac/d111/d9/d14/db6/d11e/f12d 0 2024-02-15T18:35:43.276 INFO:tasks.workunit.client.1.smithi121.stdout:1/929: read - d0/d39/d7d/dc0/dcf/f106 zero size 2024-02-15T18:35:43.276 INFO:tasks.workunit.client.1.smithi121.stdout:1/930: write d0/d39/d4b/f61 [3216422,2095] 0 2024-02-15T18:35:43.279 INFO:tasks.workunit.client.1.smithi121.stdout:7/557: getdents d0/d5/d4c/d6f/db3 0 2024-02-15T18:35:43.279 INFO:tasks.workunit.client.1.smithi121.stdout:9/938: dwrite d7/d3a/d8e/f118 [0,4194304] 0 2024-02-15T18:35:43.279 INFO:tasks.workunit.client.1.smithi121.stdout:0/952: creat d5/d107/d109/f13b x:0 0 0 2024-02-15T18:35:43.279 INFO:tasks.workunit.client.1.smithi121.stdout:9/939: read d7/d3a/d8e/d9e/dc7/fe8 [377765,32728] 0 2024-02-15T18:35:43.282 INFO:tasks.workunit.client.1.smithi121.stdout:7/558: truncate d0/d38/d46/daa/f5d 161564 0 2024-02-15T18:35:43.282 INFO:tasks.workunit.client.1.smithi121.stdout:7/559: dread - d0/d4a/d5b/d9a/f9d zero size 2024-02-15T18:35:43.282 INFO:tasks.workunit.client.1.smithi121.stdout:7/560: read - d0/d5/d4c/d6f/f94 zero size 2024-02-15T18:35:43.284 INFO:tasks.workunit.client.1.smithi121.stdout:9/940: read d7/d26/f6b [2553993,72161] 0 2024-02-15T18:35:43.284 INFO:tasks.workunit.client.1.smithi121.stdout:0/953: dread d5/d6b/dab/db4/d7d/d2a/f64 [0,4194304] 0 2024-02-15T18:35:43.284 INFO:tasks.workunit.client.1.smithi121.stdout:0/954: chown d5/d6b/dab/db1/dbf 12017 1 2024-02-15T18:35:43.296 INFO:tasks.workunit.client.1.smithi121.stdout:7/561: mkdir d0/d5/d11/d5c/dba 0 2024-02-15T18:35:43.296 INFO:tasks.workunit.client.1.smithi121.stdout:7/562: truncate d0/d4a/da2/fb5 431391 0 2024-02-15T18:35:43.296 INFO:tasks.workunit.client.1.smithi121.stdout:9/941: mkdir d7/d8/d36/d144 0 2024-02-15T18:35:43.297 INFO:tasks.workunit.client.1.smithi121.stdout:1/931: dwrite d0/d39/d4b/dac/d111/d9/d14/db6/fe1 [4194304,4194304] 0 2024-02-15T18:35:43.297 INFO:tasks.workunit.client.1.smithi121.stdout:1/932: fsync d0/d39/d4b/dac/d111/d9/d14/db6/fba 0 2024-02-15T18:35:43.298 INFO:tasks.workunit.client.1.smithi121.stdout:0/955: symlink d5/d6b/dab/db4/d7d/d2a/d73/d81/l13c 0 2024-02-15T18:35:43.300 INFO:tasks.workunit.client.1.smithi121.stdout:7/563: getdents d0/d5/d11/d1c 0 2024-02-15T18:35:43.300 INFO:tasks.workunit.client.1.smithi121.stdout:7/564: write d0/d5/d11/d5c/fa7 [1017542,34397] 0 2024-02-15T18:35:43.300 INFO:tasks.workunit.client.1.smithi121.stdout:7/565: chown d0/d5/f47 1192 1 2024-02-15T18:35:43.303 INFO:tasks.workunit.client.1.smithi121.stdout:0/956: creat d5/d6b/dab/db4/d7d/d2a/d73/d81/d124/f13d x:0 0 0 2024-02-15T18:35:43.310 INFO:tasks.workunit.client.1.smithi121.stdout:0/957: creat d5/d6b/dab/db4/d7d/d2a/d73/d81/ddb/f13e x:0 0 0 2024-02-15T18:35:43.310 INFO:tasks.workunit.client.1.smithi121.stdout:0/958: chown d5/d6b/dab/db4/d7d/d2a/d73/l89 715741340 1 2024-02-15T18:35:43.318 INFO:tasks.workunit.client.1.smithi121.stdout:0/959: dread d5/d30/f33 [0,4194304] 0 2024-02-15T18:35:43.378 INFO:tasks.workunit.client.1.smithi121.stdout:1/933: write d0/d39/d4b/dac/d111/d9/d14/d48/f78 [1254973,77157] 0 2024-02-15T18:35:43.378 INFO:tasks.workunit.client.1.smithi121.stdout:1/934: fsync d0/d39/d4b/dac/d111/d9/fa4 0 2024-02-15T18:35:43.378 INFO:tasks.workunit.client.1.smithi121.stdout:1/935: readlink d0/d39/d4b/dac/d111/d9/d14/d3e/d88/da6/l125 0 2024-02-15T18:35:43.378 INFO:tasks.workunit.client.1.smithi121.stdout:1/936: write d0/d39/d4b/dac/d111/d9/d14/db6/d11e/f12d [825073,104923] 0 2024-02-15T18:35:43.378 INFO:tasks.workunit.client.1.smithi121.stdout:1/937: fsync d0/d39/d4b/dac/d111/d9/d14/d3e/d88/da6/fbd 0 2024-02-15T18:35:43.386 INFO:tasks.workunit.client.1.smithi121.stdout:9/942: rename d7/d8 to d7/d26/d4f/d145 0 2024-02-15T18:35:43.390 INFO:tasks.workunit.client.1.smithi121.stdout:9/943: creat d7/d3a/d8e/d9e/da9/f146 x:0 0 0 2024-02-15T18:35:43.391 INFO:tasks.workunit.client.1.smithi121.stdout:9/944: write d7/d26/d4f/d145/d25/d30/d33/d79/ff1 [907422,101489] 0 2024-02-15T18:35:43.393 INFO:tasks.workunit.client.1.smithi121.stdout:9/945: unlink d7/d3a/d8e/d9e/dc7/f13a 0 2024-02-15T18:35:43.394 INFO:tasks.workunit.client.1.smithi121.stdout:9/946: stat d7/f57 0 2024-02-15T18:35:43.395 INFO:tasks.workunit.client.1.smithi121.stdout:9/947: link d7/f125 d7/d3a/f147 0 2024-02-15T18:35:43.395 INFO:tasks.workunit.client.1.smithi121.stdout:9/948: chown d7/d3a/d8e/f118 7021 1 2024-02-15T18:35:43.395 INFO:tasks.workunit.client.1.smithi121.stdout:1/938: dwrite d0/d39/d7d/dc0/fd4 [0,4194304] 0 2024-02-15T18:35:43.396 INFO:tasks.workunit.client.1.smithi121.stdout:1/939: readlink d0/d39/d7d/dc0/la1 0 2024-02-15T18:35:43.399 INFO:tasks.workunit.client.1.smithi121.stdout:1/940: symlink d0/d39/d4b/dac/d111/d9/d14/d3e/d88/da6/l13a 0 2024-02-15T18:35:43.400 INFO:tasks.workunit.client.1.smithi121.stdout:1/941: creat d0/d39/d4b/dac/d111/d9/d14/db6/f13b x:0 0 0 2024-02-15T18:35:43.401 INFO:tasks.workunit.client.1.smithi121.stdout:1/942: chown d0/d39/d4b/dac/d111/f94 23281 1 2024-02-15T18:35:43.405 INFO:tasks.workunit.client.1.smithi121.stdout:9/949: dread d7/d26/d4f/d145/d25/d30/d33/d3c/f84 [0,4194304] 0 2024-02-15T18:35:43.407 INFO:tasks.workunit.client.1.smithi121.stdout:9/950: rmdir d7/d3a/d8e/d9e/dc7/d101 39 2024-02-15T18:35:43.408 INFO:tasks.workunit.client.1.smithi121.stdout:9/951: symlink d7/d26/def/l148 0 2024-02-15T18:35:43.409 INFO:tasks.workunit.client.1.smithi121.stdout:1/943: dread d0/d39/d4b/dac/d111/d9/fa4 [0,4194304] 0 2024-02-15T18:35:43.409 INFO:tasks.workunit.client.1.smithi121.stdout:1/944: chown d0/d39/d4b/dac/d111/d9/d14/l49 48 1 2024-02-15T18:35:43.409 INFO:tasks.workunit.client.1.smithi121.stdout:1/945: fdatasync d0/d39/d7d/dc0/f9a 0 2024-02-15T18:35:43.410 INFO:tasks.workunit.client.1.smithi121.stdout:9/952: creat d7/d26/d4f/d145/d25/d30/d33/d3c/f149 x:0 0 0 2024-02-15T18:35:43.415 INFO:tasks.workunit.client.1.smithi121.stdout:9/953: creat d7/d26/d4f/d145/f14a x:0 0 0 2024-02-15T18:35:43.415 INFO:tasks.workunit.client.1.smithi121.stdout:9/954: write d7/d26/d4f/d145/d36/df9/f112 [506424,67779] 0 2024-02-15T18:35:43.415 INFO:tasks.workunit.client.1.smithi121.stdout:9/955: chown d7/d3a/d8e/d9e/da9/d8b 381510303 1 2024-02-15T18:35:43.416 INFO:tasks.workunit.client.1.smithi121.stdout:9/956: chown d7/d26/d4f/d145/d25/d30/d33/f58 93 1 2024-02-15T18:35:43.418 INFO:tasks.workunit.client.1.smithi121.stdout:9/957: rename d7/d3a/d8e/d9e/dc7/lea to d7/d26/d4f/d145/d25/d30/d33/d3c/dd0/dd9/l14b 0 2024-02-15T18:35:43.418 INFO:tasks.workunit.client.1.smithi121.stdout:9/958: write d7/d26/d4f/d145/d25/d30/d33/d3c/dd0/ff2 [5017588,43417] 0 2024-02-15T18:35:43.424 INFO:tasks.workunit.client.1.smithi121.stdout:1/946: dwrite d0/d39/d4b/dac/d111/d9/d14/d3e/d88/da6/df7/f11a [0,4194304] 0 2024-02-15T18:35:43.426 INFO:tasks.workunit.client.1.smithi121.stdout:9/959: dread d7/d26/fd2 [0,4194304] 0 2024-02-15T18:35:43.426 INFO:tasks.workunit.client.1.smithi121.stdout:9/960: chown d7/d26/d4f/d145/d40/daf 0 1 2024-02-15T18:35:43.427 INFO:tasks.workunit.client.1.smithi121.stdout:9/961: creat d7/d26/d4f/d145/d25/d30/da5/f14c x:0 0 0 2024-02-15T18:35:43.482 INFO:tasks.workunit.client.1.smithi121.stdout:9/962: unlink d7/d26/d4f/d5b/d77/cc1 0 2024-02-15T18:35:43.483 INFO:tasks.workunit.client.1.smithi121.stdout:9/963: rename d7/d26/d4f/d145/d40/dc4/l51 to d7/d3a/d8e/d9e/da9/l14d 0 2024-02-15T18:35:43.484 INFO:tasks.workunit.client.1.smithi121.stdout:9/964: symlink d7/d26/d4f/d145/d36/df9/l14e 0 2024-02-15T18:35:43.487 INFO:tasks.workunit.client.1.smithi121.stdout:9/965: symlink d7/d26/dba/l14f 0 2024-02-15T18:35:43.487 INFO:tasks.workunit.client.1.smithi121.stdout:9/966: rename d7/d3a/d8e/d9e/da9/d8b/d122/lb6 to d7/d26/d4f/d145/d40/dc4/d137/l150 0 2024-02-15T18:35:43.488 INFO:tasks.workunit.client.1.smithi121.stdout:1/947: symlink d0/d39/d4b/dac/d111/d72/dd9/dfd/l13c 0 2024-02-15T18:35:43.488 INFO:tasks.workunit.client.1.smithi121.stdout:9/967: mknod d7/d26/def/c151 0 2024-02-15T18:35:43.489 INFO:tasks.workunit.client.1.smithi121.stdout:9/968: write d7/d3a/d8e/d9e/da9/d8b/f7b [700509,66902] 0 2024-02-15T18:35:43.515 INFO:tasks.workunit.client.1.smithi121.stdout:1/948: dwrite d0/d39/d7d/f86 [0,4194304] 0 2024-02-15T18:35:43.516 INFO:tasks.workunit.client.1.smithi121.stdout:9/969: dwrite d7/d26/d4f/d145/d25/d30/da5/f41 [0,4194304] 0 2024-02-15T18:35:43.528 INFO:tasks.workunit.client.1.smithi121.stdout:1/949: rmdir d0/d39/d7d/dc0/d9d 39 2024-02-15T18:35:43.528 INFO:tasks.workunit.client.1.smithi121.stdout:1/950: dread - d0/d39/d7d/dc0/d77/fad zero size 2024-02-15T18:35:43.542 INFO:tasks.workunit.client.1.smithi121.stdout:6/860: sync 2024-02-15T18:35:43.542 INFO:tasks.workunit.client.1.smithi121.stdout:1/951: creat d0/d39/d7d/dc0/d77/d103/f13d x:0 0 0 2024-02-15T18:35:43.544 INFO:tasks.workunit.client.1.smithi121.stdout:1/952: mkdir d0/d39/d7d/dc0/dcf/d13e 0 2024-02-15T18:35:43.544 INFO:tasks.workunit.client.1.smithi121.stdout:6/861: creat d7/dc/d34/d42/d62/d63/d66/dce/f12b x:0 0 0 2024-02-15T18:35:43.546 INFO:tasks.workunit.client.1.smithi121.stdout:1/953: mkdir d0/d39/d13f 0 2024-02-15T18:35:43.546 INFO:tasks.workunit.client.1.smithi121.stdout:1/954: dread - d0/d39/d4b/dac/d111/d9/d14/db6/f13b zero size 2024-02-15T18:35:43.549 INFO:tasks.workunit.client.1.smithi121.stdout:1/955: symlink d0/d39/d7d/dc0/l140 0 2024-02-15T18:35:43.552 INFO:tasks.workunit.client.1.smithi121.stdout:1/956: mkdir d0/d141 0 2024-02-15T18:35:43.555 INFO:tasks.workunit.client.1.smithi121.stdout:1/957: creat d0/d39/d4b/dac/d111/d72/dd9/f142 x:0 0 0 2024-02-15T18:35:43.557 INFO:tasks.workunit.client.1.smithi121.stdout:1/958: creat d0/d39/d4b/dac/d111/d9/d14/db6/d11e/f143 x:0 0 0 2024-02-15T18:35:43.557 INFO:tasks.workunit.client.1.smithi121.stdout:1/959: readlink d0/le 0 2024-02-15T18:35:43.557 INFO:tasks.workunit.client.1.smithi121.stdout:6/862: dread d7/dc/d34/fbd [0,4194304] 0 2024-02-15T18:35:43.560 INFO:tasks.workunit.client.1.smithi121.stdout:1/960: rename d0/d39/d4b/dac/d111/f1b to d0/d39/d4b/dac/d111/d72/dd9/f144 0 2024-02-15T18:35:43.560 INFO:tasks.workunit.client.1.smithi121.stdout:1/961: chown d0/d39/d4b/dac/d111/d9/d14/f3b 8253885 1 2024-02-15T18:35:43.560 INFO:tasks.workunit.client.1.smithi121.stdout:1/962: chown d0/d39/d4b/dac/d111/d9/d14/la0 368543 1 2024-02-15T18:35:43.560 INFO:tasks.workunit.client.1.smithi121.stdout:6/863: rmdir d7/dc/d34/d42/d62/d63/d66 39 2024-02-15T18:35:43.561 INFO:tasks.workunit.client.1.smithi121.stdout:6/864: truncate d7/d4b/f53 1544005 0 2024-02-15T18:35:43.562 INFO:tasks.workunit.client.1.smithi121.stdout:1/963: getdents d0/d39/d4b/dac/d111/d50/d7e/dec 0 2024-02-15T18:35:43.562 INFO:tasks.workunit.client.1.smithi121.stdout:6/865: mkdir d7/d4b/d59/d12c 0 2024-02-15T18:35:43.582 INFO:tasks.workunit.client.1.smithi121.stdout:1/964: dwrite d0/d39/d4b/dac/d111/d50/f10b [0,4194304] 0 2024-02-15T18:35:43.582 INFO:tasks.workunit.client.1.smithi121.stdout:1/965: read - d0/d39/d4b/dac/d111/d9/d14/db6/fe7 zero size 2024-02-15T18:35:43.582 INFO:tasks.workunit.client.1.smithi121.stdout:1/966: getdents d0/d39/d134 0 2024-02-15T18:35:43.582 INFO:tasks.workunit.client.1.smithi121.stdout:1/967: truncate d0/d39/d4b/dac/d111/d9/d14/db6/fe7 169241 0 2024-02-15T18:35:43.585 INFO:tasks.workunit.client.1.smithi121.stdout:1/968: link d0/d39/d7d/dc0/f5d d0/d39/d4b/dac/d111/d9/d14/d3e/d88/f145 0 2024-02-15T18:35:43.652 INFO:tasks.workunit.client.1.smithi121.stdout:2/542: sync 2024-02-15T18:35:43.652 INFO:tasks.workunit.client.1.smithi121.stdout:2/543: fdatasync d5/d49/f7f 0 2024-02-15T18:35:43.652 INFO:tasks.workunit.client.1.smithi121.stdout:2/544: fsync d5/fd 0 2024-02-15T18:35:43.652 INFO:tasks.workunit.client.1.smithi121.stdout:2/545: write d5/d49/f7f [51568,118229] 0 2024-02-15T18:35:43.777 INFO:tasks.workunit.client.1.smithi121.stdout:5/950: sync 2024-02-15T18:35:43.779 INFO:tasks.workunit.client.1.smithi121.stdout:5/951: getdents d0/d2/d1e/d4e/d7d/d40/d62/db1 0 2024-02-15T18:35:43.779 INFO:tasks.workunit.client.1.smithi121.stdout:5/952: chown d0/d2/d1e/d4e/d7d/d9d/d4c/daa/f134 213980 1 2024-02-15T18:35:43.782 INFO:tasks.workunit.client.1.smithi121.stdout:5/953: creat d0/d2/d1e/f13d x:0 0 0 2024-02-15T18:35:43.782 INFO:tasks.workunit.client.1.smithi121.stdout:5/954: chown d0/d2/d1e/d4e/l6f 24434 1 2024-02-15T18:35:43.782 INFO:tasks.workunit.client.1.smithi121.stdout:5/955: truncate d0/d2/d1e/d4e/d7d/d40/d62/fae 589802 0 2024-02-15T18:35:43.782 INFO:tasks.workunit.client.1.smithi121.stdout:5/956: chown d0/d2/d1e/d4e/d7d/d47/d5a/dbc/l110 236398253 1 2024-02-15T18:35:43.791 INFO:tasks.workunit.client.1.smithi121.stdout:5/957: dread d0/f4b [0,4194304] 0 2024-02-15T18:35:43.793 INFO:tasks.workunit.client.1.smithi121.stdout:5/958: dread d0/d2/d1e/d4e/d7d/d9d/d4c/f5e [0,4194304] 0 2024-02-15T18:35:43.794 INFO:tasks.workunit.client.1.smithi121.stdout:5/959: mknod d0/d2/c13e 0 2024-02-15T18:35:43.799 INFO:tasks.workunit.client.1.smithi121.stdout:5/960: link d0/d2/d1e/d4e/d7d/c5d d0/d2/d1e/d4e/d7d/d47/d5a/db3/c13f 0 2024-02-15T18:35:43.802 INFO:tasks.workunit.client.1.smithi121.stdout:5/961: link d0/d2/d1e/d4e/d7d/c2f d0/d2/d70/c140 0 2024-02-15T18:35:43.803 INFO:tasks.workunit.client.1.smithi121.stdout:5/962: creat d0/d16/d7a/f141 x:0 0 0 2024-02-15T18:35:43.804 INFO:tasks.workunit.client.1.smithi121.stdout:5/963: write d0/d2/d1e/d4e/d7d/d47/d5a/fa0 [2265,41823] 0 2024-02-15T18:35:43.806 INFO:tasks.workunit.client.1.smithi121.stdout:5/964: symlink d0/d2/d1e/d4e/d7d/d40/d62/db1/l142 0 2024-02-15T18:35:43.807 INFO:tasks.workunit.client.1.smithi121.stdout:5/965: symlink d0/d16/d34/da4/dcc/l143 0 2024-02-15T18:35:43.807 INFO:tasks.workunit.client.1.smithi121.stdout:5/966: chown d0/d2/l2e 139973 1 2024-02-15T18:35:43.807 INFO:tasks.workunit.client.1.smithi121.stdout:5/967: chown d0/d2/d1e/d4e/d7d/d39/df2/l10f 1991780 1 2024-02-15T18:35:43.811 INFO:tasks.workunit.client.1.smithi121.stdout:5/968: mkdir d0/d16/d34/d144 0 2024-02-15T18:35:43.813 INFO:tasks.workunit.client.1.smithi121.stdout:5/969: creat d0/d2/d70/f145 x:0 0 0 2024-02-15T18:35:43.816 INFO:tasks.workunit.client.1.smithi121.stdout:5/970: rename d0/d2/d1e/d4e/d7d/d47/l65 to d0/d2/l146 0 2024-02-15T18:35:43.818 INFO:tasks.workunit.client.1.smithi121.stdout:5/971: creat d0/d16/da1/f147 x:0 0 0 2024-02-15T18:35:43.821 INFO:tasks.workunit.client.1.smithi121.stdout:5/972: mkdir d0/d16/d34/d144/d148 0 2024-02-15T18:35:43.822 INFO:tasks.workunit.client.1.smithi121.stdout:5/973: symlink d0/d2/d1e/d4e/d7d/d47/d5a/l149 0 2024-02-15T18:35:43.824 INFO:tasks.workunit.client.1.smithi121.stdout:5/974: rename d0/d2/d1e/d4e/d7d/d40/d62/db1/fc1 to d0/d2/d1e/f14a 0 2024-02-15T18:35:43.825 INFO:tasks.workunit.client.1.smithi121.stdout:5/975: creat d0/d16/f14b x:0 0 0 2024-02-15T18:35:43.826 INFO:tasks.workunit.client.1.smithi121.stdout:5/976: write d0/f82 [2540877,26947] 0 2024-02-15T18:35:43.828 INFO:tasks.workunit.client.1.smithi121.stdout:5/977: rmdir d0/d2/d1e/d4e/d7d/d47/d5a/dbc 39 2024-02-15T18:35:43.830 INFO:tasks.workunit.client.1.smithi121.stdout:5/978: mkdir d0/d16/d34/d14c 0 2024-02-15T18:35:43.883 INFO:tasks.workunit.client.1.smithi121.stdout:5/979: dwrite d0/d2/d1e/d4e/d7d/d47/d5a/f5c [0,4194304] 0 2024-02-15T18:35:43.885 INFO:tasks.workunit.client.1.smithi121.stdout:5/980: creat d0/d2/d1e/d4e/f14d x:0 0 0 2024-02-15T18:35:43.888 INFO:tasks.workunit.client.1.smithi121.stdout:5/981: creat d0/d16/d34/d144/d148/f14e x:0 0 0 2024-02-15T18:35:43.890 INFO:tasks.workunit.client.1.smithi121.stdout:5/982: symlink d0/d2/l14f 0 2024-02-15T18:35:43.906 INFO:tasks.workunit.client.1.smithi121.stdout:5/983: dwrite d0/d2/d1e/d4e/d7d/d39/fe7 [0,4194304] 0 2024-02-15T18:35:43.907 INFO:tasks.workunit.client.1.smithi121.stdout:5/984: creat d0/d2/d1e/d4e/d7d/f150 x:0 0 0 2024-02-15T18:35:43.922 INFO:tasks.workunit.client.1.smithi121.stdout:5/985: dwrite d0/d2/d1e/d4e/d7d/d40/f79 [0,4194304] 0 2024-02-15T18:35:43.927 INFO:tasks.workunit.client.1.smithi121.stdout:5/986: creat d0/f151 x:0 0 0 2024-02-15T18:35:43.927 INFO:tasks.workunit.client.1.smithi121.stdout:5/987: write d0/d2/faf [3926340,69453] 0 2024-02-15T18:35:43.954 INFO:tasks.workunit.client.1.smithi121.stdout:5/988: dwrite d0/d2/d18/f20 [0,4194304] 0 2024-02-15T18:35:43.970 INFO:tasks.workunit.client.1.smithi121.stdout:5/989: dwrite d0/d2/d18/f10d [0,4194304] 0 2024-02-15T18:35:43.971 INFO:tasks.workunit.client.1.smithi121.stdout:5/990: fsync d0/d2/d70/f145 0 2024-02-15T18:35:43.986 INFO:tasks.workunit.client.1.smithi121.stdout:5/991: mkdir d0/d16/d7a/d152 0 2024-02-15T18:35:43.986 INFO:tasks.workunit.client.1.smithi121.stdout:5/992: readlink d0/d2/d70/ldb 0 2024-02-15T18:35:43.986 INFO:tasks.workunit.client.1.smithi121.stdout:5/993: dread - d0/d2/d1e/d4e/d7d/f150 zero size 2024-02-15T18:35:43.986 INFO:tasks.workunit.client.1.smithi121.stdout:5/994: fsync d0/d2/d70/fc0 0 2024-02-15T18:35:43.991 INFO:tasks.workunit.client.1.smithi121.stdout:5/995: creat d0/d2/d1e/d4e/d7d/d47/d5a/db3/f153 x:0 0 0 2024-02-15T18:35:43.993 INFO:tasks.workunit.client.1.smithi121.stdout:5/996: mknod d0/d16/d7a/c154 0 2024-02-15T18:35:43.997 INFO:tasks.workunit.client.1.smithi121.stdout:5/997: unlink d0/d2/fd 0 2024-02-15T18:35:43.997 INFO:tasks.workunit.client.1.smithi121.stdout:5/998: fdatasync d0/f129 0 2024-02-15T18:35:43.998 INFO:tasks.workunit.client.1.smithi121.stdout:5/999: mknod d0/d2/d70/de1/c155 0 2024-02-15T18:35:44.033 INFO:tasks.workunit.client.1.smithi121.stdout:3/601: sync 2024-02-15T18:35:44.033 INFO:tasks.workunit.client.1.smithi121.stdout:8/715: sync 2024-02-15T18:35:44.033 INFO:tasks.workunit.client.1.smithi121.stdout:4/791: sync 2024-02-15T18:35:44.034 INFO:tasks.workunit.client.1.smithi121.stdout:4/792: write d5/d9/d19/d4c/dc3/de6/fe7 [1366667,94701] 0 2024-02-15T18:35:44.037 INFO:tasks.workunit.client.1.smithi121.stdout:4/793: creat d5/d9/d19/d4c/d8d/ff6 x:0 0 0 2024-02-15T18:35:44.042 INFO:tasks.workunit.client.1.smithi121.stdout:3/602: dread d2/d1a/d6b/f82 [0,4194304] 0 2024-02-15T18:35:44.043 INFO:tasks.workunit.client.1.smithi121.stdout:8/716: dread d0/d2a/f97 [0,4194304] 0 2024-02-15T18:35:44.043 INFO:tasks.workunit.client.1.smithi121.stdout:3/603: write d2/d1a/d6b/f82 [2116037,25582] 0 2024-02-15T18:35:44.045 INFO:tasks.workunit.client.1.smithi121.stdout:3/604: creat d2/d1a/d6b/fc8 x:0 0 0 2024-02-15T18:35:44.045 INFO:tasks.workunit.client.1.smithi121.stdout:8/717: write d0/d7/d36/d8c/f49 [2287628,53443] 0 2024-02-15T18:35:44.046 INFO:tasks.workunit.client.1.smithi121.stdout:3/605: write d2/d1a/d27/f4b [676372,53034] 0 2024-02-15T18:35:44.047 INFO:tasks.workunit.client.1.smithi121.stdout:8/718: symlink d0/d61/d66/lfa 0 2024-02-15T18:35:44.047 INFO:tasks.workunit.client.1.smithi121.stdout:8/719: fsync d0/d61/d8f/fdd 0 2024-02-15T18:35:44.047 INFO:tasks.workunit.client.1.smithi121.stdout:8/720: readlink d0/d7/db1/lb5 0 2024-02-15T18:35:44.050 INFO:tasks.workunit.client.1.smithi121.stdout:8/721: symlink d0/d2a/d51/lfb 0 2024-02-15T18:35:44.051 INFO:tasks.workunit.client.1.smithi121.stdout:3/606: write d2/d1a/d27/fa2 [3180589,70826] 0 2024-02-15T18:35:44.053 INFO:tasks.workunit.client.1.smithi121.stdout:3/607: write d2/d1a/d6b/d7d/fc5 [3827272,127700] 0 2024-02-15T18:35:44.055 INFO:tasks.workunit.client.1.smithi121.stdout:4/794: dwrite d5/d9/d76/d87/fab [0,4194304] 0 2024-02-15T18:35:44.067 INFO:tasks.workunit.client.1.smithi121.stdout:4/795: dread d5/d9/d1d/f24 [0,4194304] 0 2024-02-15T18:35:44.067 INFO:tasks.workunit.client.1.smithi121.stdout:4/796: chown d5/d31/d25/db6 491309149 1 2024-02-15T18:35:44.067 INFO:tasks.workunit.client.1.smithi121.stdout:4/797: write d5/d9/d76/d87/d95/fef [229177,27037] 0 2024-02-15T18:35:44.069 INFO:tasks.workunit.client.1.smithi121.stdout:8/722: dwrite d0/d7/f10 [0,4194304] 0 2024-02-15T18:35:44.073 INFO:tasks.workunit.client.1.smithi121.stdout:3/608: dread d2/d1a/d27/fa2 [0,4194304] 0 2024-02-15T18:35:44.075 INFO:tasks.workunit.client.1.smithi121.stdout:3/609: creat d2/d1a/d27/d63/da4/d25/d31/d97/daf/fc9 x:0 0 0 2024-02-15T18:35:44.076 INFO:tasks.workunit.client.1.smithi121.stdout:3/610: symlink d2/d1a/d6b/d7d/lca 0 2024-02-15T18:35:44.077 INFO:tasks.workunit.client.1.smithi121.stdout:3/611: creat d2/d1a/fcb x:0 0 0 2024-02-15T18:35:44.079 INFO:tasks.workunit.client.1.smithi121.stdout:3/612: creat d2/d1a/d27/d63/fcc x:0 0 0 2024-02-15T18:35:44.081 INFO:tasks.workunit.client.1.smithi121.stdout:3/613: creat d2/d1a/d27/d63/da4/d25/d31/d97/fcd x:0 0 0 2024-02-15T18:35:44.081 INFO:tasks.workunit.client.1.smithi121.stdout:3/614: dread - d2/d1a/d27/d63/fcc zero size 2024-02-15T18:35:44.081 INFO:tasks.workunit.client.1.smithi121.stdout:3/615: rmdir d2/d1a/d27/d63/da4/d25/d88 39 2024-02-15T18:35:44.082 INFO:tasks.workunit.client.1.smithi121.stdout:3/616: truncate d2/d1a/d27/f7f 1301091 0 2024-02-15T18:35:44.083 INFO:tasks.workunit.client.1.smithi121.stdout:8/723: dwrite d0/d7/d36/d8c/d45/fcc [0,4194304] 0 2024-02-15T18:35:44.084 INFO:tasks.workunit.client.1.smithi121.stdout:3/617: rename d2/fe to d2/fce 0 2024-02-15T18:35:44.088 INFO:tasks.workunit.client.1.smithi121.stdout:8/724: rmdir d0/d7/da5/dcb 39 2024-02-15T18:35:44.090 INFO:tasks.workunit.client.1.smithi121.stdout:8/725: dread d0/d7/de/f54 [0,4194304] 0 2024-02-15T18:35:44.090 INFO:tasks.workunit.client.1.smithi121.stdout:3/618: truncate d2/d1a/d27/da7/d85/fa3 489603 0 2024-02-15T18:35:44.090 INFO:tasks.workunit.client.1.smithi121.stdout:3/619: write d2/d1a/d27/d63/da4/d25/d31/f73 [709267,13486] 0 2024-02-15T18:35:44.091 INFO:tasks.workunit.client.1.smithi121.stdout:8/726: write d0/d7/d36/d8c/d45/f3a [4086259,60975] 0 2024-02-15T18:35:44.091 INFO:tasks.workunit.client.1.smithi121.stdout:3/620: mknod d2/d1a/d27/d63/da4/d25/d31/d58/ccf 0 2024-02-15T18:35:44.091 INFO:tasks.workunit.client.1.smithi121.stdout:3/621: dread - d2/d5a/fc4 zero size 2024-02-15T18:35:44.091 INFO:tasks.workunit.client.1.smithi121.stdout:3/622: chown d2/d1a/d27/d63/da4/d25/d30/db3/cc7 79490508 1 2024-02-15T18:35:44.091 INFO:tasks.workunit.client.1.smithi121.stdout:3/623: fdatasync d2/fce 0 2024-02-15T18:35:44.092 INFO:tasks.workunit.client.1.smithi121.stdout:8/727: rmdir d0/dce/df8 39 2024-02-15T18:35:44.092 INFO:tasks.workunit.client.1.smithi121.stdout:3/624: mknod d2/d1a/d27/da7/d85/d9f/cd0 0 2024-02-15T18:35:44.094 INFO:tasks.workunit.client.1.smithi121.stdout:3/625: write d2/d1a/d6b/d7d/f8e [387557,112893] 0 2024-02-15T18:35:44.094 INFO:tasks.workunit.client.1.smithi121.stdout:8/728: rmdir d0/d3 39 2024-02-15T18:35:44.094 INFO:tasks.workunit.client.1.smithi121.stdout:4/798: creat d5/d9/ff7 x:0 0 0 2024-02-15T18:35:44.094 INFO:tasks.workunit.client.1.smithi121.stdout:4/799: stat d5/d31/d25/f37 0 2024-02-15T18:35:44.094 INFO:tasks.workunit.client.1.smithi121.stdout:4/800: write d5/d9/d76/d9e/f38 [2939065,129990] 0 2024-02-15T18:35:44.098 INFO:tasks.workunit.client.1.smithi121.stdout:4/801: mknod d5/d9/d1d/cf8 0 2024-02-15T18:35:44.101 INFO:tasks.workunit.client.1.smithi121.stdout:3/626: link d2/d1a/d27/d63/da4/d25/d31/d34/c86 d2/d1a/d27/d63/da4/d25/d30/db3/cd1 0 2024-02-15T18:35:44.102 INFO:tasks.workunit.client.1.smithi121.stdout:8/729: rename d0/d7/c9d to d0/d7/da5/dcb/cfc 0 2024-02-15T18:35:44.102 INFO:tasks.workunit.client.1.smithi121.stdout:8/730: read - d0/d7/da5/dcb/feb zero size 2024-02-15T18:35:44.102 INFO:tasks.workunit.client.1.smithi121.stdout:4/802: creat d5/d31/d3d/ff9 x:0 0 0 2024-02-15T18:35:44.103 INFO:tasks.workunit.client.1.smithi121.stdout:4/803: stat d5/d9/d19/d2a/d4d/l6d 0 2024-02-15T18:35:44.103 INFO:tasks.workunit.client.1.smithi121.stdout:4/804: read - d5/d31/d3d/d4f/d51/d71/fc6 zero size 2024-02-15T18:35:44.103 INFO:tasks.workunit.client.1.smithi121.stdout:4/805: readlink d5/d9/d1d/d26/l67 0 2024-02-15T18:35:44.105 INFO:tasks.workunit.client.1.smithi121.stdout:8/731: creat d0/d7/da7/ffd x:0 0 0 2024-02-15T18:35:44.105 INFO:tasks.workunit.client.1.smithi121.stdout:8/732: truncate d0/d3/ff2 1035061 0 2024-02-15T18:35:44.105 INFO:tasks.workunit.client.1.smithi121.stdout:4/806: symlink d5/d31/d3d/d4f/d51/d71/dca/lfa 0 2024-02-15T18:35:44.107 INFO:tasks.workunit.client.1.smithi121.stdout:8/733: creat d0/d7/d36/d8c/d45/ffe x:0 0 0 2024-02-15T18:35:44.111 INFO:tasks.workunit.client.1.smithi121.stdout:8/734: rename d0/d7/db1/dbb/f78 to d0/d7/d36/d8c/d45/fff 0 2024-02-15T18:35:44.117 INFO:tasks.workunit.client.1.smithi121.stdout:8/735: symlink d0/d7/db1/dbb/de2/l100 0 2024-02-15T18:35:44.117 INFO:tasks.workunit.client.1.smithi121.stdout:4/807: mkdir d5/d9/d1d/d26/d72/dfb 0 2024-02-15T18:35:44.119 INFO:tasks.workunit.client.1.smithi121.stdout:4/808: write d5/d9/d1d/f2f [1205188,100266] 0 2024-02-15T18:35:44.123 INFO:tasks.workunit.client.1.smithi121.stdout:4/809: unlink d5/d9/d76/d9e/c4b 0 2024-02-15T18:35:44.124 INFO:tasks.workunit.client.1.smithi121.stdout:4/810: rmdir d5/d9/d19/d4c/dc3 39 2024-02-15T18:35:44.133 INFO:tasks.workunit.client.1.smithi121.stdout:8/736: dwrite d0/d61/d66/f6d [0,4194304] 0 2024-02-15T18:35:44.135 INFO:tasks.workunit.client.1.smithi121.stdout:8/737: creat d0/d48/f101 x:0 0 0 2024-02-15T18:35:44.136 INFO:tasks.workunit.client.1.smithi121.stdout:8/738: mkdir d0/d7/da5/dcb/d102 0 2024-02-15T18:35:44.138 INFO:tasks.workunit.client.1.smithi121.stdout:8/739: link d0/c34 d0/d7/d36/d8c/d45/dcd/c103 0 2024-02-15T18:35:44.138 INFO:tasks.workunit.client.1.smithi121.stdout:8/740: truncate d0/d7/d36/d8c/d45/d53/fa0 991183 0 2024-02-15T18:35:44.139 INFO:tasks.workunit.client.1.smithi121.stdout:8/741: mkdir d0/d7/d36/d52/d104 0 2024-02-15T18:35:44.140 INFO:tasks.workunit.client.1.smithi121.stdout:8/742: write d0/d61/d7b/dac/fe6 [1361360,24386] 0 2024-02-15T18:35:44.148 INFO:tasks.workunit.client.1.smithi121.stdout:8/743: dread d0/d7/d36/d8c/f49 [0,4194304] 0 2024-02-15T18:35:44.163 INFO:tasks.workunit.client.1.smithi121.stdout:8/744: dwrite d0/dce/df8/f99 [0,4194304] 0 2024-02-15T18:35:44.167 INFO:tasks.workunit.client.1.smithi121.stdout:8/745: rename d0/d3/c16 to d0/d48/c105 0 2024-02-15T18:35:44.168 INFO:tasks.workunit.client.1.smithi121.stdout:8/746: mkdir d0/d7/de/d106 0 2024-02-15T18:35:44.168 INFO:tasks.workunit.client.1.smithi121.stdout:8/747: chown d0/dce/df8/c88 115942799 1 2024-02-15T18:35:44.168 INFO:tasks.workunit.client.1.smithi121.stdout:8/748: stat d0/d7/d36/d8c/l40 0 2024-02-15T18:35:44.168 INFO:tasks.workunit.client.1.smithi121.stdout:8/749: fsync d0/d7/d36/d8c/d45/f89 0 2024-02-15T18:35:44.180 INFO:tasks.workunit.client.1.smithi121.stdout:0/960: sync 2024-02-15T18:35:44.180 INFO:tasks.workunit.client.1.smithi121.stdout:6/866: sync 2024-02-15T18:35:44.180 INFO:tasks.workunit.client.1.smithi121.stdout:2/546: sync 2024-02-15T18:35:44.180 INFO:tasks.workunit.client.1.smithi121.stdout:1/969: sync 2024-02-15T18:35:44.180 INFO:tasks.workunit.client.1.smithi121.stdout:7/566: sync 2024-02-15T18:35:44.180 INFO:tasks.workunit.client.1.smithi121.stdout:6/867: fdatasync d7/d4b/f8f 0 2024-02-15T18:35:44.181 INFO:tasks.workunit.client.1.smithi121.stdout:7/567: dread - d0/d4a/d5b/d9a/f9d zero size 2024-02-15T18:35:44.181 INFO:tasks.workunit.client.1.smithi121.stdout:9/970: sync 2024-02-15T18:35:44.183 INFO:tasks.workunit.client.1.smithi121.stdout:0/961: link d5/d6b/dab/db1/dbf/f13a d5/d6b/dab/db4/d7d/d2a/d4e/d127/f13f 0 2024-02-15T18:35:44.184 INFO:tasks.workunit.client.1.smithi121.stdout:7/568: mkdir d0/d5/d11/dbb 0 2024-02-15T18:35:44.185 INFO:tasks.workunit.client.1.smithi121.stdout:7/569: fsync d0/f69 0 2024-02-15T18:35:44.186 INFO:tasks.workunit.client.1.smithi121.stdout:9/971: symlink d7/d26/d4f/d145/d25/d30/d33/d8f/l152 0 2024-02-15T18:35:44.189 INFO:tasks.workunit.client.1.smithi121.stdout:7/570: rename d0/d5/d4c/f59 to d0/d5/d4c/d6f/db6/fbc 0 2024-02-15T18:35:44.191 INFO:tasks.workunit.client.1.smithi121.stdout:7/571: mknod d0/d5/cbd 0 2024-02-15T18:35:44.192 INFO:tasks.workunit.client.1.smithi121.stdout:7/572: write d0/d5/d11/f48 [507009,105238] 0 2024-02-15T18:35:44.192 INFO:tasks.workunit.client.1.smithi121.stdout:9/972: rename d7/d26/d4f/d145/d40/l55 to d7/d3a/d8e/de5/l153 0 2024-02-15T18:35:44.195 INFO:tasks.workunit.client.1.smithi121.stdout:6/868: dread d7/dc/d34/d7c/f85 [0,4194304] 0 2024-02-15T18:35:44.195 INFO:tasks.workunit.client.1.smithi121.stdout:8/750: dwrite d0/d7/d36/d8c/d45/fcc [4194304,4194304] 0 2024-02-15T18:35:44.196 INFO:tasks.workunit.client.1.smithi121.stdout:9/973: truncate d7/d3a/f10a 3177062 0 2024-02-15T18:35:44.196 INFO:tasks.workunit.client.1.smithi121.stdout:9/974: chown d7/d3a/d8e/d9e/da9/d8b/d73/fad 4119005 1 2024-02-15T18:35:44.203 INFO:tasks.workunit.client.1.smithi121.stdout:8/751: link d0/d3/c9 d0/d7/db1/c107 0 2024-02-15T18:35:44.203 INFO:tasks.workunit.client.1.smithi121.stdout:9/975: rename d7/d26/d4f/d145/d40/daf/c102 to d7/d26/d4f/d145/d25/d30/da5/d108/c154 0 2024-02-15T18:35:44.205 INFO:tasks.workunit.client.1.smithi121.stdout:9/976: write d7/f1c [3743138,17788] 0 2024-02-15T18:35:44.212 INFO:tasks.workunit.client.1.smithi121.stdout:0/962: dread d5/d6b/dab/db4/d7d/d2a/f91 [0,4194304] 0 2024-02-15T18:35:44.214 INFO:tasks.workunit.client.1.smithi121.stdout:0/963: link d5/d6b/dab/db1/dbf/f117 d5/d6b/dab/db4/d7d/d74/f140 0 2024-02-15T18:35:44.216 INFO:tasks.workunit.client.1.smithi121.stdout:0/964: creat d5/d6b/dab/db4/f141 x:0 0 0 2024-02-15T18:35:44.217 INFO:tasks.workunit.client.1.smithi121.stdout:0/965: mkdir d5/d6b/dab/db4/d142 0 2024-02-15T18:35:44.217 INFO:tasks.workunit.client.1.smithi121.stdout:0/966: chown d5/d6b/dab/db4/cb3 98036938 1 2024-02-15T18:35:44.218 INFO:tasks.workunit.client.1.smithi121.stdout:0/967: write d5/d30/ff6 [166037,83190] 0 2024-02-15T18:35:44.218 INFO:tasks.workunit.client.1.smithi121.stdout:0/968: truncate d5/d6b/fcb 2826487 0 2024-02-15T18:35:44.220 INFO:tasks.workunit.client.1.smithi121.stdout:0/969: link d5/d6b/dab/db4/d7d/d2a/d36/d44/c67 d5/d11c/c143 0 2024-02-15T18:35:44.222 INFO:tasks.workunit.client.1.smithi121.stdout:0/970: rename d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/fea to d5/d107/d109/f144 0 2024-02-15T18:35:44.222 INFO:tasks.workunit.client.1.smithi121.stdout:0/971: write d5/d107/f125 [515946,40967] 0 2024-02-15T18:35:44.223 INFO:tasks.workunit.client.1.smithi121.stdout:0/972: mkdir d5/d6b/dab/db4/d7d/d2a/d73/d97/db8/d145 0 2024-02-15T18:35:44.223 INFO:tasks.workunit.client.1.smithi121.stdout:0/973: truncate d5/d6b/dab/db4/d7d/dd/f22 549864 0 2024-02-15T18:35:44.223 INFO:tasks.workunit.client.1.smithi121.stdout:0/974: mkdir d5/d6b/d146 0 2024-02-15T18:35:44.247 INFO:tasks.workunit.client.1.smithi121.stdout:1/970: dwrite d0/d39/d7d/dc0/fff [0,4194304] 0 2024-02-15T18:35:44.249 INFO:tasks.workunit.client.1.smithi121.stdout:1/971: rename d0/d39/d4b/dac/d111/d9/d14/d48/f70 to d0/d39/d4b/dac/d111/d72/dd9/dfd/f146 0 2024-02-15T18:35:44.251 INFO:tasks.workunit.client.1.smithi121.stdout:2/547: dwrite f4 [0,4194304] 0 2024-02-15T18:35:44.251 INFO:tasks.workunit.client.1.smithi121.stdout:8/752: dwrite d0/d7/ff4 [0,4194304] 0 2024-02-15T18:35:44.251 INFO:tasks.workunit.client.1.smithi121.stdout:8/753: readlink d0/d61/l70 0 2024-02-15T18:35:44.251 INFO:tasks.workunit.client.1.smithi121.stdout:1/972: rename d0/d39/d4b/dac/d111/d9/d14/d48/f123 to d0/d39/d4b/dac/d111/d9/d14/f147 0 2024-02-15T18:35:44.252 INFO:tasks.workunit.client.1.smithi121.stdout:6/869: dwrite d7/dc/f37 [0,4194304] 0 2024-02-15T18:35:44.253 INFO:tasks.workunit.client.1.smithi121.stdout:2/548: unlink d5/d49/d4d/l2e 0 2024-02-15T18:35:44.253 INFO:tasks.workunit.client.1.smithi121.stdout:6/870: stat d7/dc/f21 0 2024-02-15T18:35:44.253 INFO:tasks.workunit.client.1.smithi121.stdout:2/549: write d5/f78 [1299444,69463] 0 2024-02-15T18:35:44.254 INFO:tasks.workunit.client.1.smithi121.stdout:8/754: read d0/d61/d95/f9c [872578,57042] 0 2024-02-15T18:35:44.254 INFO:tasks.workunit.client.1.smithi121.stdout:9/977: dwrite d7/f125 [0,4194304] 0 2024-02-15T18:35:44.254 INFO:tasks.workunit.client.1.smithi121.stdout:8/755: chown d0/d7/ff4 804872779 1 2024-02-15T18:35:44.254 INFO:tasks.workunit.client.1.smithi121.stdout:1/973: creat d0/d39/d4b/dac/d111/d9/d14/db6/f148 x:0 0 0 2024-02-15T18:35:44.257 INFO:tasks.workunit.client.1.smithi121.stdout:0/975: dwrite d5/d30/d5d/f111 [0,4194304] 0 2024-02-15T18:35:44.257 INFO:tasks.workunit.client.1.smithi121.stdout:0/976: chown d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/l87 0 1 2024-02-15T18:35:44.266 INFO:tasks.workunit.client.1.smithi121.stdout:1/974: dread d0/d39/d4b/dac/d111/d50/d7e/f76 [0,4194304] 0 2024-02-15T18:35:44.276 INFO:tasks.workunit.client.1.smithi121.stdout:6/871: mkdir d7/dc/d2c/d36/d12d 0 2024-02-15T18:35:44.279 INFO:tasks.workunit.client.1.smithi121.stdout:2/550: rename d5/d49/d4d to d5/df/d60/d98 0 2024-02-15T18:35:44.279 INFO:tasks.workunit.client.1.smithi121.stdout:2/551: truncate d5/df/d60/f8c 268377 0 2024-02-15T18:35:44.280 INFO:tasks.workunit.client.1.smithi121.stdout:9/978: dwrite d7/d26/def/f12f [0,4194304] 0 2024-02-15T18:35:44.282 INFO:tasks.workunit.client.1.smithi121.stdout:0/977: symlink d5/d6b/dab/db4/d7d/d2a/d73/d8d/l147 0 2024-02-15T18:35:44.284 INFO:tasks.workunit.client.1.smithi121.stdout:8/756: dwrite d0/d3/ff2 [0,4194304] 0 2024-02-15T18:35:44.284 INFO:tasks.workunit.client.1.smithi121.stdout:8/757: truncate d0/d7/da7/ffd 385587 0 2024-02-15T18:35:44.286 INFO:tasks.workunit.client.1.smithi121.stdout:1/975: dwrite d0/d39/d4b/dac/f12b [0,4194304] 0 2024-02-15T18:35:44.288 INFO:tasks.workunit.client.1.smithi121.stdout:1/976: write d0/d39/d7d/dc0/f54 [2192427,56014] 0 2024-02-15T18:35:44.288 INFO:tasks.workunit.client.1.smithi121.stdout:1/977: chown d0/d39/d4b/dac/d111/d9/d14/d48/f60 2 1 2024-02-15T18:35:44.288 INFO:tasks.workunit.client.1.smithi121.stdout:1/978: read - d0/d39/d4b/dac/d111/d72/dd9/dfd/f136 zero size 2024-02-15T18:35:44.291 INFO:tasks.workunit.client.1.smithi121.stdout:1/979: creat d0/d39/d4b/dac/d111/d9/f149 x:0 0 0 2024-02-15T18:35:44.292 INFO:tasks.workunit.client.1.smithi121.stdout:1/980: stat d0/d39/d7d/dc0/la1 0 2024-02-15T18:35:44.293 INFO:tasks.workunit.client.1.smithi121.stdout:9/979: dread d7/f113 [0,4194304] 0 2024-02-15T18:35:44.293 INFO:tasks.workunit.client.1.smithi121.stdout:0/978: dread d5/d6b/dab/db4/d7d/d2a/d36/d44/fc7 [0,4194304] 0 2024-02-15T18:35:44.294 INFO:tasks.workunit.client.1.smithi121.stdout:0/979: read - d5/d6b/dab/db4/d7d/d2a/d73/d81/ddb/deb/f12c zero size 2024-02-15T18:35:44.294 INFO:tasks.workunit.client.1.smithi121.stdout:0/980: fsync d5/d6b/dab/db1/fc1 0 2024-02-15T18:35:44.294 INFO:tasks.workunit.client.1.smithi121.stdout:0/981: dread - d5/d6b/dab/db4/f141 zero size 2024-02-15T18:35:44.295 INFO:tasks.workunit.client.1.smithi121.stdout:1/981: creat d0/d39/d7d/f14a x:0 0 0 2024-02-15T18:35:44.295 INFO:tasks.workunit.client.1.smithi121.stdout:1/982: write d0/d39/d4b/dac/d111/d9/f23 [4352694,24403] 0 2024-02-15T18:35:44.299 INFO:tasks.workunit.client.1.smithi121.stdout:6/872: mknod d7/dc/d34/d42/d62/dac/d10a/c12e 0 2024-02-15T18:35:44.301 INFO:tasks.workunit.client.1.smithi121.stdout:2/552: creat d5/d89/f99 x:0 0 0 2024-02-15T18:35:44.301 INFO:tasks.workunit.client.1.smithi121.stdout:2/553: chown d5/df/d27/f79 1079 1 2024-02-15T18:35:44.306 INFO:tasks.workunit.client.1.smithi121.stdout:9/980: rename d7/d3a/d8e/f128 to d7/d26/d4f/f155 0 2024-02-15T18:35:44.307 INFO:tasks.workunit.client.1.smithi121.stdout:0/982: unlink d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/lba 0 2024-02-15T18:35:44.308 INFO:tasks.workunit.client.1.smithi121.stdout:9/981: mknod d7/d26/d4f/d145/d40/c156 0 2024-02-15T18:35:44.308 INFO:tasks.workunit.client.1.smithi121.stdout:9/982: read - d7/d3a/d8e/d9e/da9/d8b/f106 zero size 2024-02-15T18:35:44.309 INFO:tasks.workunit.client.1.smithi121.stdout:9/983: dread - d7/d26/d4f/d145/d25/d30/f127 zero size 2024-02-15T18:35:44.310 INFO:tasks.workunit.client.1.smithi121.stdout:0/983: creat d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/d10a/f148 x:0 0 0 2024-02-15T18:35:44.310 INFO:tasks.workunit.client.1.smithi121.stdout:0/984: chown d5/d6b/dab/db4/d7d/dd/f22 5173985 1 2024-02-15T18:35:44.311 INFO:tasks.workunit.client.1.smithi121.stdout:1/983: rename d0/d39/d4b/dac/d111/d9/d14/d48 to d0/d39/d4b/dac/d111/d72/dd9/dfd/d12f/db4/de8/d14b 0 2024-02-15T18:35:44.311 INFO:tasks.workunit.client.1.smithi121.stdout:0/985: mkdir d5/d6b/dab/db4/d7d/d2a/d73/d97/db8/d149 0 2024-02-15T18:35:44.313 INFO:tasks.workunit.client.1.smithi121.stdout:1/984: mknod d0/d39/d4b/dac/d111/d9/d14/db6/c14c 0 2024-02-15T18:35:44.313 INFO:tasks.workunit.client.1.smithi121.stdout:1/985: dread - d0/d39/d4b/dac/d111/d9/d14/d3e/d88/da6/f109 zero size 2024-02-15T18:35:44.314 INFO:tasks.workunit.client.1.smithi121.stdout:0/986: link d5/d30/f118 d5/d6b/dab/db4/d7d/d2a/d73/d97/df4/f14a 0 2024-02-15T18:35:44.314 INFO:tasks.workunit.client.1.smithi121.stdout:0/987: dread - d5/d6b/dab/db4/d7d/dd/fb5 zero size 2024-02-15T18:35:44.314 INFO:tasks.workunit.client.1.smithi121.stdout:0/988: chown d5/d6b/dab/db4/d7d/d2a/d36/d44/d71/l90 133 1 2024-02-15T18:35:44.314 INFO:tasks.workunit.client.1.smithi121.stdout:0/989: stat d5/d6b/dab/db4/d7d/dd/c59 0 2024-02-15T18:35:44.315 INFO:tasks.workunit.client.1.smithi121.stdout:2/554: dread d5/df/f2a [0,4194304] 0 2024-02-15T18:35:44.315 INFO:tasks.workunit.client.1.smithi121.stdout:0/990: creat d5/d6b/dab/db4/d7d/f14b x:0 0 0 2024-02-15T18:35:44.315 INFO:tasks.workunit.client.1.smithi121.stdout:1/986: unlink d0/d39/d4b/dac/ldd 0 2024-02-15T18:35:44.316 INFO:tasks.workunit.client.1.smithi121.stdout:2/555: mkdir d5/df/d60/d98/d22/d81/d9a 0 2024-02-15T18:35:44.317 INFO:tasks.workunit.client.1.smithi121.stdout:2/556: write d5/df/d60/d98/d22/f4f [751965,98413] 0 2024-02-15T18:35:44.318 INFO:tasks.workunit.client.1.smithi121.stdout:0/991: mkdir d5/d107/d109/d14c 0 2024-02-15T18:35:44.318 INFO:tasks.workunit.client.1.smithi121.stdout:0/992: write d5/d30/f118 [1371940,40221] 0 2024-02-15T18:35:44.319 INFO:tasks.workunit.client.1.smithi121.stdout:2/557: creat d5/df/d27/f9b x:0 0 0 2024-02-15T18:35:44.319 INFO:tasks.workunit.client.1.smithi121.stdout:8/758: dwrite d0/d7/db1/dbb/fa2 [0,4194304] 0 2024-02-15T18:35:44.320 INFO:tasks.workunit.client.1.smithi121.stdout:6/873: dwrite d7/dc/d2c/d36/fb8 [0,4194304] 0 2024-02-15T18:35:44.321 INFO:tasks.workunit.client.1.smithi121.stdout:2/558: creat d5/df/d60/d98/d58/f9c x:0 0 0 2024-02-15T18:35:44.321 INFO:tasks.workunit.client.1.smithi121.stdout:2/559: write d5/df/d60/f86 [134965,85976] 0 2024-02-15T18:35:44.321 INFO:tasks.workunit.client.1.smithi121.stdout:2/560: truncate d5/df/d10/f6d 1269645 0 2024-02-15T18:35:44.324 INFO:tasks.workunit.client.1.smithi121.stdout:9/984: dread d7/d26/d4f/d145/d36/f7d [0,4194304] 0 2024-02-15T18:35:44.324 INFO:tasks.workunit.client.1.smithi121.stdout:8/759: link d0/d7/d36/d52/fc7 d0/d7/d36/d52/d7a/f108 0 2024-02-15T18:35:44.325 INFO:tasks.workunit.client.1.smithi121.stdout:8/760: write d0/d7/d36/d8c/d45/fcc [4545813,24121] 0 2024-02-15T18:35:44.326 INFO:tasks.workunit.client.1.smithi121.stdout:8/761: fdatasync d0/d2a/f4b 0 2024-02-15T18:35:44.326 INFO:tasks.workunit.client.1.smithi121.stdout:6/874: unlink d7/dc/d34/c69 0 2024-02-15T18:35:44.327 INFO:tasks.workunit.client.1.smithi121.stdout:1/987: dread d0/d39/d4b/dac/d111/d9/d51/d5f/f62 [0,4194304] 0 2024-02-15T18:35:44.327 INFO:tasks.workunit.client.1.smithi121.stdout:2/561: rmdir d5/df/d27/d61 39 2024-02-15T18:35:44.328 INFO:tasks.workunit.client.1.smithi121.stdout:1/988: dread d0/d39/d4b/dac/d111/d9/d14/db6/fe7 [0,4194304] 0 2024-02-15T18:35:44.328 INFO:tasks.workunit.client.1.smithi121.stdout:1/989: fdatasync d0/d39/fb3 0 2024-02-15T18:35:44.328 INFO:tasks.workunit.client.1.smithi121.stdout:1/990: dread - d0/f121 zero size 2024-02-15T18:35:44.328 INFO:tasks.workunit.client.1.smithi121.stdout:1/991: write d0/d39/d7d/dc0/d9d/fce [1021901,5827] 0 2024-02-15T18:35:44.328 INFO:tasks.workunit.client.1.smithi121.stdout:1/992: truncate d0/d39/d4b/dac/d111/d9/d14/d3e/d88/f145 8615772 0 2024-02-15T18:35:44.330 INFO:tasks.workunit.client.1.smithi121.stdout:9/985: link d7/d26/d4f/d145/f16 d7/d26/d4f/d145/d40/dc4/f157 0 2024-02-15T18:35:44.330 INFO:tasks.workunit.client.1.smithi121.stdout:8/762: getdents d0/dce/df8 0 2024-02-15T18:35:44.330 INFO:tasks.workunit.client.1.smithi121.stdout:9/986: dread - d7/d26/d4f/d5b/f117 zero size 2024-02-15T18:35:44.336 INFO:tasks.workunit.client.1.smithi121.stdout:2/562: mknod d5/df/d27/d3a/c9d 0 2024-02-15T18:35:44.337 INFO:tasks.workunit.client.1.smithi121.stdout:2/563: chown d5/df/d60/d98/d58/d63/d94/l96 826 1 2024-02-15T18:35:44.337 INFO:tasks.workunit.client.1.smithi121.stdout:6/875: creat d7/dc/d76/f12f x:0 0 0 2024-02-15T18:35:44.337 INFO:tasks.workunit.client.1.smithi121.stdout:2/564: truncate d5/f3b 1234500 0 2024-02-15T18:35:44.337 INFO:tasks.workunit.client.1.smithi121.stdout:2/565: chown d5/df/d10/d4c/l7c 119254 1 2024-02-15T18:35:44.337 INFO:tasks.workunit.client.1.smithi121.stdout:2/566: write d5/df/d27/d61/f6b [176844,49645] 0 2024-02-15T18:35:44.342 INFO:tasks.workunit.client.1.smithi121.stdout:0/993: write d5/d6b/dab/db4/d7d/d2a/f95 [944385,109326] 0 2024-02-15T18:35:44.344 INFO:tasks.workunit.client.1.smithi121.stdout:0/994: read d5/d6b/dab/db4/dca/f130 [609832,118001] 0 2024-02-15T18:35:44.344 INFO:tasks.workunit.client.1.smithi121.stdout:0/995: fdatasync d5/d6b/dab/db4/d7d/d2a/d73/d81/ddb/deb/f12c 0 2024-02-15T18:35:44.344 INFO:tasks.workunit.client.1.smithi121.stdout:2/567: write d5/df/d60/d98/f75 [441552,65142] 0 2024-02-15T18:35:44.344 INFO:tasks.workunit.client.1.smithi121.stdout:2/568: write d5/df/d10/f6d [1735304,98246] 0 2024-02-15T18:35:44.345 INFO:tasks.workunit.client.1.smithi121.stdout:1/993: dwrite d0/d39/d4b/dac/d111/d9/d14/d3e/d88/da6/f109 [0,4194304] 0 2024-02-15T18:35:44.345 INFO:tasks.workunit.client.1.smithi121.stdout:1/994: chown d0/d39/d4b/dac/d111/d72/dd9/lfe 28576583 1 2024-02-15T18:35:44.345 INFO:tasks.workunit.client.1.smithi121.stdout:9/987: link d7/d3a/d8e/d9e/da9/d8b/d122/fe9 d7/d26/d4f/d145/d36/df9/f158 0 2024-02-15T18:35:44.346 INFO:tasks.workunit.client.1.smithi121.stdout:0/996: dread d5/d9b/fbc [0,4194304] 0 2024-02-15T18:35:44.347 INFO:tasks.workunit.client.1.smithi121.stdout:6/876: unlink d7/d4b/d60/d6c/dd5/dfa/fc3 0 2024-02-15T18:35:44.352 INFO:tasks.workunit.client.1.smithi121.stdout:2/569: unlink d5/df/d27/d3e/c6f 0 2024-02-15T18:35:44.352 INFO:tasks.workunit.client.1.smithi121.stdout:2/570: stat d5/df/l68 0 2024-02-15T18:35:44.352 INFO:tasks.workunit.client.1.smithi121.stdout:2/571: write d5/d49/d71/f70 [4469808,126028] 0 2024-02-15T18:35:44.354 INFO:tasks.workunit.client.1.smithi121.stdout:9/988: creat d7/d3a/d8e/d9e/dc7/d101/f159 x:0 0 0 2024-02-15T18:35:44.355 INFO:tasks.workunit.client.1.smithi121.stdout:0/997: symlink d5/d6b/dab/db4/d7d/d2a/d73/d8d/dc8/l14d 0 2024-02-15T18:35:44.356 INFO:tasks.workunit.client.1.smithi121.stdout:6/877: symlink d7/d4b/d60/d6c/dd5/l130 0 2024-02-15T18:35:44.357 INFO:tasks.workunit.client.1.smithi121.stdout:2/572: symlink d5/df/d60/d98/d22/d81/d9a/l9e 0 2024-02-15T18:35:44.359 INFO:tasks.workunit.client.1.smithi121.stdout:9/989: unlink d7/d26/d4f/d145/d36/da3/d110/f12b 0 2024-02-15T18:35:44.360 INFO:tasks.workunit.client.1.smithi121.stdout:9/990: write d7/d3a/fdc [3103601,54037] 0 2024-02-15T18:35:44.362 INFO:tasks.workunit.client.1.smithi121.stdout:2/573: mkdir d5/df/d27/d3e/d9f 0 2024-02-15T18:35:44.363 INFO:tasks.workunit.client.1.smithi121.stdout:2/574: readlink d5/df/d60/d98/d22/l83 0 2024-02-15T18:35:44.364 INFO:tasks.workunit.client.1.smithi121.stdout:0/998: dread d5/d6b/dab/db4/ff9 [0,4194304] 0 2024-02-15T18:35:44.364 INFO:tasks.workunit.client.1.smithi121.stdout:2/575: dread d5/fc [0,4194304] 0 2024-02-15T18:35:44.364 INFO:tasks.workunit.client.1.smithi121.stdout:2/576: chown d5/df/d27/d3e/d6a/l6e 27666157 1 2024-02-15T18:35:44.364 INFO:tasks.workunit.client.1.smithi121.stdout:2/577: chown d5/df/d60/f8c 128410584 1 2024-02-15T18:35:44.366 INFO:tasks.workunit.client.1.smithi121.stdout:0/999: creat d5/d6b/dab/db4/d7d/d74/d96/de4/f14e x:0 0 0 2024-02-15T18:35:44.367 INFO:tasks.workunit.client.1.smithi121.stdout:2/578: unlink d5/df/d3f/l57 0 2024-02-15T18:35:44.367 INFO:tasks.workunit.client.1.smithi121.stdout:1/995: dwrite d0/d39/d4b/dac/d111/d50/d7e/fc9 [0,4194304] 0 2024-02-15T18:35:44.371 INFO:tasks.workunit.client.1.smithi121.stdout:1/996: truncate d0/d39/d4b/dac/d111/d9/d14/f58 2527857 0 2024-02-15T18:35:44.374 INFO:tasks.workunit.client.1.smithi121.stdout:9/991: dread d7/f13 [0,4194304] 0 2024-02-15T18:35:44.376 INFO:tasks.workunit.client.1.smithi121.stdout:1/997: link d0/d39/d4b/dac/f12b d0/d39/d4b/dac/d111/d9/d14/d3e/d88/da6/df7/f14d 0 2024-02-15T18:35:44.379 INFO:tasks.workunit.client.1.smithi121.stdout:6/878: rmdir d7/d4b/d60/d6c/dd5/dfa 39 2024-02-15T18:35:44.382 INFO:tasks.workunit.client.1.smithi121.stdout:9/992: rename d7/d3a/d8e/l92 to d7/d26/d4f/d145/d25/d30/d33/d8f/d93/l15a 0 2024-02-15T18:35:44.386 INFO:tasks.workunit.client.1.smithi121.stdout:9/993: rename d7/d26/d4f/d145/d36/df9/l14e to d7/d26/d4f/d145/d25/d30/da5/l15b 0 2024-02-15T18:35:44.386 INFO:tasks.workunit.client.1.smithi121.stdout:9/994: write d7/d26/d4f/d145/d36/f10d [750473,63012] 0 2024-02-15T18:35:44.400 INFO:tasks.workunit.client.1.smithi121.stdout:1/998: dwrite d0/d39/d4b/dac/d111/d9/d14/db6/d11e/f143 [0,4194304] 0 2024-02-15T18:35:44.403 INFO:tasks.workunit.client.1.smithi121.stdout:1/999: creat d0/d39/d7d/dc0/dcf/d12e/f14e x:0 0 0 2024-02-15T18:35:44.408 INFO:tasks.workunit.client.1.smithi121.stdout:6/879: dwrite d7/dc/d34/f86 [0,4194304] 0 2024-02-15T18:35:44.409 INFO:tasks.workunit.client.1.smithi121.stdout:9/995: dwrite d7/d3a/d8e/d9e/da9/d8b/d73/f100 [0,4194304] 0 2024-02-15T18:35:44.414 INFO:tasks.workunit.client.1.smithi121.stdout:6/880: mknod d7/dc/d34/d42/d62/dac/de8/c131 0 2024-02-15T18:35:44.414 INFO:tasks.workunit.client.1.smithi121.stdout:6/881: read d7/dc/d34/d42/d62/fb0 [258794,104431] 0 2024-02-15T18:35:44.415 INFO:tasks.workunit.client.1.smithi121.stdout:9/996: creat d7/d26/d4f/d145/d36/da3/f15c x:0 0 0 2024-02-15T18:35:44.416 INFO:tasks.workunit.client.1.smithi121.stdout:6/882: creat d7/dc/f132 x:0 0 0 2024-02-15T18:35:44.416 INFO:tasks.workunit.client.1.smithi121.stdout:6/883: chown d7/d4b/d59/cbc 10627731 1 2024-02-15T18:35:44.416 INFO:tasks.workunit.client.1.smithi121.stdout:9/997: mknod d7/d26/d4f/d145/d25/d30/da5/d12c/c15d 0 2024-02-15T18:35:44.418 INFO:tasks.workunit.client.1.smithi121.stdout:6/884: write d7/dae/f11d [451572,101535] 0 2024-02-15T18:35:44.419 INFO:tasks.workunit.client.1.smithi121.stdout:9/998: rename d7/f125 to d7/d26/d4f/d145/d36/dce/f15e 0 2024-02-15T18:35:44.421 INFO:tasks.workunit.client.1.smithi121.stdout:6/885: rename d7/dc/f2d to d7/dc/d55/df8/f133 0 2024-02-15T18:35:44.421 INFO:tasks.workunit.client.1.smithi121.stdout:9/999: mkdir d7/d26/d4f/d145/d36/d144/d15f 0 2024-02-15T18:35:44.422 INFO:tasks.workunit.client.1.smithi121.stdout:6/886: creat d7/d4b/d60/f134 x:0 0 0 2024-02-15T18:35:44.423 INFO:tasks.workunit.client.1.smithi121.stdout:6/887: write d7/dc/d34/d42/d62/dac/fc4 [1038027,41056] 0 2024-02-15T18:35:44.423 INFO:tasks.workunit.client.1.smithi121.stdout:6/888: symlink d7/dc/d34/l135 0 2024-02-15T18:35:44.441 INFO:tasks.workunit.client.1.smithi121.stdout:6/889: dwrite d7/d4b/d60/f134 [0,4194304] 0 2024-02-15T18:35:44.450 INFO:tasks.workunit.client.1.smithi121.stdout:6/890: truncate d7/dae/f11d 237219 0 2024-02-15T18:35:44.451 INFO:tasks.workunit.client.1.smithi121.stdout:6/891: fsync d7/dc/d34/d42/d62/d63/d6f/f101 0 2024-02-15T18:35:44.452 INFO:tasks.workunit.client.1.smithi121.stdout:6/892: creat d7/dc/d34/d42/d62/d63/d6f/d89/f136 x:0 0 0 2024-02-15T18:35:44.452 INFO:tasks.workunit.client.1.smithi121.stdout:6/893: mkdir d7/dc/d55/df8/d137 0 2024-02-15T18:35:44.454 INFO:tasks.workunit.client.1.smithi121.stdout:6/894: creat d7/d4b/d60/d6c/f138 x:0 0 0 2024-02-15T18:35:44.455 INFO:tasks.workunit.client.1.smithi121.stdout:6/895: symlink d7/dc/d55/df8/l139 0 2024-02-15T18:35:44.462 INFO:tasks.workunit.client.1.smithi121.stdout:6/896: rmdir d7/dc/d34 39 2024-02-15T18:35:44.465 INFO:tasks.workunit.client.1.smithi121.stdout:6/897: rmdir d7/d4b/d60/d73/dc9 39 2024-02-15T18:35:44.465 INFO:tasks.workunit.client.1.smithi121.stdout:6/898: dread - d7/dc/d34/d42/fee zero size 2024-02-15T18:35:44.466 INFO:tasks.workunit.client.1.smithi121.stdout:6/899: rename d7/l12a to d7/d4b/d59/l13a 0 2024-02-15T18:35:44.467 INFO:tasks.workunit.client.1.smithi121.stdout:6/900: mknod d7/dc/d34/d42/d62/dac/d10a/c13b 0 2024-02-15T18:35:44.469 INFO:tasks.workunit.client.1.smithi121.stdout:6/901: creat d7/dc/d34/d42/d62/d63/d6f/d89/da1/f13c x:0 0 0 2024-02-15T18:35:44.470 INFO:tasks.workunit.client.1.smithi121.stdout:6/902: rmdir d7/dc/d55 39 2024-02-15T18:35:44.471 INFO:tasks.workunit.client.1.smithi121.stdout:4/811: sync 2024-02-15T18:35:44.471 INFO:tasks.workunit.client.1.smithi121.stdout:3/627: sync 2024-02-15T18:35:44.472 INFO:tasks.workunit.client.1.smithi121.stdout:3/628: mknod d2/d1a/d27/d63/da4/d25/d31/d34/cd2 0 2024-02-15T18:35:44.473 INFO:tasks.workunit.client.1.smithi121.stdout:4/812: creat d5/d31/d3d/d4f/d51/d80/ffc x:0 0 0 2024-02-15T18:35:44.474 INFO:tasks.workunit.client.1.smithi121.stdout:3/629: mkdir d2/d1a/d27/d63/da4/d25/d30/db3/dd3 0 2024-02-15T18:35:44.474 INFO:tasks.workunit.client.1.smithi121.stdout:3/630: dread - d2/d1a/fcb zero size 2024-02-15T18:35:44.474 INFO:tasks.workunit.client.1.smithi121.stdout:3/631: read - d2/d1a/d27/d63/da4/d25/d31/d97/daf/fc9 zero size 2024-02-15T18:35:44.474 INFO:tasks.workunit.client.1.smithi121.stdout:3/632: chown d2/d1a/d27/d63/da4/d25/d31/d97/daf/fc9 771 1 2024-02-15T18:35:44.476 INFO:tasks.workunit.client.1.smithi121.stdout:4/813: mknod d5/d31/d3d/cfd 0 2024-02-15T18:35:44.476 INFO:tasks.workunit.client.1.smithi121.stdout:4/814: fdatasync d5/d31/d25/fda 0 2024-02-15T18:35:44.476 INFO:tasks.workunit.client.1.smithi121.stdout:6/903: unlink d7/dc/d34/d7c/faa 0 2024-02-15T18:35:44.477 INFO:tasks.workunit.client.1.smithi121.stdout:4/815: symlink d5/d31/d25/lfe 0 2024-02-15T18:35:44.478 INFO:tasks.workunit.client.1.smithi121.stdout:3/633: rename d2/d1a/f89 to d2/d1a/fd4 0 2024-02-15T18:35:44.478 INFO:tasks.workunit.client.1.smithi121.stdout:6/904: mknod d7/dc/d76/d103/c13d 0 2024-02-15T18:35:44.479 INFO:tasks.workunit.client.1.smithi121.stdout:3/634: link d2/d1a/d27/d63/da4/d25/d31/d58/f61 d2/d1a/d27/d63/d81/fd5 0 2024-02-15T18:35:44.479 INFO:tasks.workunit.client.1.smithi121.stdout:4/816: mkdir d5/d31/d3d/d4f/d51/dff 0 2024-02-15T18:35:44.480 INFO:tasks.workunit.client.1.smithi121.stdout:3/635: creat d2/d1a/d27/d63/da4/d25/d30/fd6 x:0 0 0 2024-02-15T18:35:44.481 INFO:tasks.workunit.client.1.smithi121.stdout:4/817: symlink d5/d9/d19/d2a/d4d/l100 0 2024-02-15T18:35:44.482 INFO:tasks.workunit.client.1.smithi121.stdout:3/636: read d2/fd [11271,18927] 0 2024-02-15T18:35:44.482 INFO:tasks.workunit.client.1.smithi121.stdout:3/637: readlink d2/d1a/d27/d63/da4/d25/d88/d96/l9e 0 2024-02-15T18:35:44.482 INFO:tasks.workunit.client.1.smithi121.stdout:3/638: truncate d2/d1a/fcb 936785 0 2024-02-15T18:35:44.482 INFO:tasks.workunit.client.1.smithi121.stdout:3/639: mknod d2/d1a/d27/d63/da4/d25/d88/cd7 0 2024-02-15T18:35:44.483 INFO:tasks.workunit.client.1.smithi121.stdout:3/640: creat d2/d1a/d27/d63/da4/fd8 x:0 0 0 2024-02-15T18:35:44.484 INFO:tasks.workunit.client.1.smithi121.stdout:4/818: creat d5/d9/d19/d4c/dc3/f101 x:0 0 0 2024-02-15T18:35:44.484 INFO:tasks.workunit.client.1.smithi121.stdout:4/819: write d5/d31/d25/fda [233786,92372] 0 2024-02-15T18:35:44.484 INFO:tasks.workunit.client.1.smithi121.stdout:4/820: truncate d5/d31/d25/db6/fe9 4511499 0 2024-02-15T18:35:44.484 INFO:tasks.workunit.client.1.smithi121.stdout:4/821: truncate d5/d31/d3d/d4a/d7e/f7f 361731 0 2024-02-15T18:35:44.485 INFO:tasks.workunit.client.1.smithi121.stdout:4/822: write d5/d9/d19/d4c/dc3/de6/fe7 [1613455,1895] 0 2024-02-15T18:35:44.486 INFO:tasks.workunit.client.1.smithi121.stdout:6/905: dread d7/d4b/d60/d6c/feb [0,4194304] 0 2024-02-15T18:35:44.486 INFO:tasks.workunit.client.1.smithi121.stdout:6/906: readlink d7/dc/d2c/d36/l3f 0 2024-02-15T18:35:44.487 INFO:tasks.workunit.client.1.smithi121.stdout:6/907: chown d7/dc/d34/d42/d62/f74 1982703 1 2024-02-15T18:35:44.487 INFO:tasks.workunit.client.1.smithi121.stdout:6/908: chown d7/d4b/d59/l122 26080939 1 2024-02-15T18:35:44.488 INFO:tasks.workunit.client.1.smithi121.stdout:6/909: creat d7/dc/d34/d42/d62/d63/d66/dce/f13e x:0 0 0 2024-02-15T18:35:44.489 INFO:tasks.workunit.client.1.smithi121.stdout:6/910: symlink d7/dc/d34/d42/d62/l13f 0 2024-02-15T18:35:44.490 INFO:tasks.workunit.client.1.smithi121.stdout:6/911: mknod d7/dc/d34/d42/d62/d63/d6f/c140 0 2024-02-15T18:35:44.491 INFO:tasks.workunit.client.1.smithi121.stdout:6/912: write d7/dc/d55/fe1 [412035,125649] 0 2024-02-15T18:35:44.492 INFO:tasks.workunit.client.1.smithi121.stdout:6/913: mkdir d7/dc/d2c/d141 0 2024-02-15T18:35:44.493 INFO:tasks.workunit.client.1.smithi121.stdout:6/914: mknod d7/d4b/d59/c142 0 2024-02-15T18:35:44.504 INFO:tasks.workunit.client.1.smithi121.stdout:4/823: dwrite d5/d31/d3d/f49 [0,4194304] 0 2024-02-15T18:35:44.518 INFO:tasks.workunit.client.1.smithi121.stdout:4/824: symlink d5/d9/d19/l102 0 2024-02-15T18:35:44.522 INFO:tasks.workunit.client.1.smithi121.stdout:4/825: mkdir d5/d103 0 2024-02-15T18:35:44.523 INFO:tasks.workunit.client.1.smithi121.stdout:4/826: read d5/d9/d1d/d26/d72/f78 [322204,127865] 0 2024-02-15T18:35:44.681 INFO:tasks.workunit.client.1.smithi121.stdout:7/573: sync 2024-02-15T18:35:44.682 INFO:tasks.workunit.client.1.smithi121.stdout:8/763: sync 2024-02-15T18:35:44.682 INFO:tasks.workunit.client.1.smithi121.stdout:8/764: dread - d0/d2a/fd5 zero size 2024-02-15T18:35:44.682 INFO:tasks.workunit.client.1.smithi121.stdout:7/574: getdents d0/d5/d11 0 2024-02-15T18:35:44.685 INFO:tasks.workunit.client.1.smithi121.stdout:8/765: unlink d0/d61/l70 0 2024-02-15T18:35:44.685 INFO:tasks.workunit.client.1.smithi121.stdout:8/766: write d0/d7/d36/d8c/d45/fd3 [1030590,22103] 0 2024-02-15T18:35:44.685 INFO:tasks.workunit.client.1.smithi121.stdout:8/767: write d0/d7/d36/fcf [308106,53886] 0 2024-02-15T18:35:44.689 INFO:tasks.workunit.client.1.smithi121.stdout:8/768: unlink d0/d2a/d51/f38 0 2024-02-15T18:35:44.689 INFO:tasks.workunit.client.1.smithi121.stdout:8/769: chown d0/d7/de/fd9 11 1 2024-02-15T18:35:44.689 INFO:tasks.workunit.client.1.smithi121.stdout:8/770: chown d0/d61 76564 1 2024-02-15T18:35:44.690 INFO:tasks.workunit.client.1.smithi121.stdout:8/771: mkdir d0/dce/d109 0 2024-02-15T18:35:44.693 INFO:tasks.workunit.client.1.smithi121.stdout:8/772: creat d0/d7/db1/dbb/de2/f10a x:0 0 0 2024-02-15T18:35:44.715 INFO:tasks.workunit.client.1.smithi121.stdout:7/575: dwrite d0/d38/fa0 [0,4194304] 0 2024-02-15T18:35:44.719 INFO:tasks.workunit.client.1.smithi121.stdout:8/773: dwrite d0/d61/d66/f3b [4194304,4194304] 0 2024-02-15T18:35:44.719 INFO:tasks.workunit.client.1.smithi121.stdout:8/774: fsync d0/d7/d36/d8c/f49 0 2024-02-15T18:35:44.719 INFO:tasks.workunit.client.1.smithi121.stdout:8/775: stat d0/d7/d36/d8c/l40 0 2024-02-15T18:35:44.720 INFO:tasks.workunit.client.1.smithi121.stdout:7/576: write d0/fb [948212,10639] 0 2024-02-15T18:35:44.720 INFO:tasks.workunit.client.1.smithi121.stdout:7/577: read d0/d4a/d5b/f92 [68015,111475] 0 2024-02-15T18:35:44.720 INFO:tasks.workunit.client.1.smithi121.stdout:7/578: readlink d0/d4a/d5b/da6/l75 0 2024-02-15T18:35:44.721 INFO:tasks.workunit.client.1.smithi121.stdout:8/776: rename d0/dce/df8/c2e to d0/d61/d95/c10b 0 2024-02-15T18:35:44.738 INFO:tasks.workunit.client.1.smithi121.stdout:7/579: dwrite d0/d38/d3f/d43/f8f [4194304,4194304] 0 2024-02-15T18:35:44.738 INFO:tasks.workunit.client.1.smithi121.stdout:7/580: readlink d0/d5/d11/l68 0 2024-02-15T18:35:44.740 INFO:tasks.workunit.client.1.smithi121.stdout:7/581: link d0/d5/d11/d5c/fa7 d0/d5/d4c/d6f/fbe 0 2024-02-15T18:35:44.740 INFO:tasks.workunit.client.1.smithi121.stdout:7/582: write d0/f69 [1269834,43040] 0 2024-02-15T18:35:44.744 INFO:tasks.workunit.client.1.smithi121.stdout:7/583: dread d0/d5/d4c/d6f/db6/fbc [0,4194304] 0 2024-02-15T18:35:44.744 INFO:tasks.workunit.client.1.smithi121.stdout:7/584: fsync d0/d38/d61/f8a 0 2024-02-15T18:35:44.744 INFO:tasks.workunit.client.1.smithi121.stdout:8/777: dwrite d0/d61/d66/f80 [4194304,4194304] 0 2024-02-15T18:35:44.746 INFO:tasks.workunit.client.1.smithi121.stdout:7/585: creat d0/d4a/d5b/da6/fbf x:0 0 0 2024-02-15T18:35:44.748 INFO:tasks.workunit.client.1.smithi121.stdout:8/778: mkdir d0/d7/da5/dcb/d10c 0 2024-02-15T18:35:44.750 INFO:tasks.workunit.client.1.smithi121.stdout:7/586: rename d0/d5/d11/d5c/dba to d0/d5/d11/d1c/d5f/dc0 0 2024-02-15T18:35:44.754 INFO:tasks.workunit.client.1.smithi121.stdout:8/779: rename d0/d7/da5/dcb/faf to d0/d7/da5/dcb/f10d 0 2024-02-15T18:35:44.754 INFO:tasks.workunit.client.1.smithi121.stdout:8/780: chown d0/d7/d36/d8c/d45/f2c 0 1 2024-02-15T18:35:44.757 INFO:tasks.workunit.client.1.smithi121.stdout:7/587: dread d0/f18 [0,4194304] 0 2024-02-15T18:35:44.757 INFO:tasks.workunit.client.1.smithi121.stdout:7/588: chown d0/d4a/d5b/fa8 602337 1 2024-02-15T18:35:44.760 INFO:tasks.workunit.client.1.smithi121.stdout:7/589: rmdir d0/d38/d3f/d43/da9 0 2024-02-15T18:35:44.761 INFO:tasks.workunit.client.1.smithi121.stdout:7/590: read d0/f69 [586584,37461] 0 2024-02-15T18:35:44.763 INFO:tasks.workunit.client.1.smithi121.stdout:7/591: rename d0/d5/d11/d1c/d5f/f90 to d0/d38/fc1 0 2024-02-15T18:35:44.763 INFO:tasks.workunit.client.1.smithi121.stdout:7/592: truncate d0/d5/d11/d1c/f35 2544065 0 2024-02-15T18:35:44.765 INFO:tasks.workunit.client.1.smithi121.stdout:7/593: mkdir d0/d38/d46/daa/dc2 0 2024-02-15T18:35:44.766 INFO:tasks.workunit.client.1.smithi121.stdout:7/594: creat d0/d5/fc3 x:0 0 0 2024-02-15T18:35:44.769 INFO:tasks.workunit.client.1.smithi121.stdout:8/781: dwrite d0/d7/da5/dcb/f9f [0,4194304] 0 2024-02-15T18:35:44.782 INFO:tasks.workunit.client.1.smithi121.stdout:7/595: dwrite d0/d5/d11/f40 [4194304,4194304] 0 2024-02-15T18:35:44.783 INFO:tasks.workunit.client.1.smithi121.stdout:7/596: rename d0/d4a/d5b/cae to d0/d4a/da2/cc4 0 2024-02-15T18:35:44.785 INFO:tasks.workunit.client.1.smithi121.stdout:7/597: unlink d0/d5/d11/l3a 0 2024-02-15T18:35:44.787 INFO:tasks.workunit.client.1.smithi121.stdout:7/598: creat d0/d5/d11/d1c/d5f/fc5 x:0 0 0 2024-02-15T18:35:44.789 INFO:tasks.workunit.client.1.smithi121.stdout:7/599: unlink d0/d38/d3c/f80 0 2024-02-15T18:35:44.790 INFO:tasks.workunit.client.1.smithi121.stdout:7/600: mkdir d0/d38/d3c/dc6 0 2024-02-15T18:35:44.790 INFO:tasks.workunit.client.1.smithi121.stdout:7/601: fdatasync d0/d4a/d5b/f65 0 2024-02-15T18:35:44.790 INFO:tasks.workunit.client.1.smithi121.stdout:7/602: chown d0/d38/d61/f8a 482707 1 2024-02-15T18:35:44.792 INFO:tasks.workunit.client.1.smithi121.stdout:7/603: mknod d0/d38/cc7 0 2024-02-15T18:35:44.793 INFO:tasks.workunit.client.1.smithi121.stdout:7/604: write d0/d5/d11/d1c/d5f/f82 [169417,83330] 0 2024-02-15T18:35:44.794 INFO:tasks.workunit.client.1.smithi121.stdout:7/605: getdents d0/d5/d4c/d6f 0 2024-02-15T18:35:44.794 INFO:tasks.workunit.client.1.smithi121.stdout:7/606: fdatasync d0/d38/d3c/f4d 0 2024-02-15T18:35:44.794 INFO:tasks.workunit.client.1.smithi121.stdout:7/607: write d0/d4a/d5b/d9a/f9d [335906,41144] 0 2024-02-15T18:35:44.892 INFO:tasks.workunit.client.1.smithi121.stdout:2/579: sync 2024-02-15T18:35:44.893 INFO:tasks.workunit.client.1.smithi121.stdout:2/580: mknod d5/df/ca0 0 2024-02-15T18:35:44.894 INFO:tasks.workunit.client.1.smithi121.stdout:2/581: write d5/d49/f84 [41502,38195] 0 2024-02-15T18:35:44.894 INFO:tasks.workunit.client.1.smithi121.stdout:2/582: write d5/df/d60/d98/d22/f4f [1402562,128277] 0 2024-02-15T18:35:44.896 INFO:tasks.workunit.client.1.smithi121.stdout:2/583: rmdir d5/df/d27 39 2024-02-15T18:35:44.897 INFO:tasks.workunit.client.1.smithi121.stdout:2/584: mkdir d5/df/d60/d98/d22/d81/d9a/da1 0 2024-02-15T18:35:44.899 INFO:tasks.workunit.client.1.smithi121.stdout:2/585: creat d5/df/d27/d3e/d9f/fa2 x:0 0 0 2024-02-15T18:35:44.899 INFO:tasks.workunit.client.1.smithi121.stdout:2/586: fdatasync d5/df/d27/d3e/f62 0 2024-02-15T18:35:44.899 INFO:tasks.workunit.client.1.smithi121.stdout:2/587: fsync d5/df/d60/d98/d22/f44 0 2024-02-15T18:35:44.900 INFO:tasks.workunit.client.1.smithi121.stdout:2/588: symlink d5/df/d60/d98/d22/d81/la3 0 2024-02-15T18:35:44.902 INFO:tasks.workunit.client.1.smithi121.stdout:2/589: rename d5/df/l20 to d5/df/d60/d98/d58/d63/la4 0 2024-02-15T18:35:44.903 INFO:tasks.workunit.client.1.smithi121.stdout:2/590: mkdir d5/df/d60/d98/d22/d81/d9a/da5 0 2024-02-15T18:35:44.904 INFO:tasks.workunit.client.1.smithi121.stdout:2/591: link d5/df/d60/d98/d22/f37 d5/d49/fa6 0 2024-02-15T18:35:44.905 INFO:tasks.workunit.client.1.smithi121.stdout:2/592: creat d5/df/d3f/fa7 x:0 0 0 2024-02-15T18:35:44.915 INFO:tasks.workunit.client.1.smithi121.stdout:2/593: dread d5/df/d60/d98/d58/f87 [0,4194304] 0 2024-02-15T18:35:44.930 INFO:tasks.workunit.client.1.smithi121.stdout:2/594: dwrite d5/df/f1b [0,4194304] 0 2024-02-15T18:35:44.933 INFO:tasks.workunit.client.1.smithi121.stdout:2/595: truncate d5/df/f2b 103522 0 2024-02-15T18:35:44.933 INFO:tasks.workunit.client.1.smithi121.stdout:2/596: stat d5/df/d27/c95 0 2024-02-15T18:35:44.933 INFO:tasks.workunit.client.1.smithi121.stdout:2/597: chown d5/df/d3f/f69 2128995 1 2024-02-15T18:35:44.934 INFO:tasks.workunit.client.1.smithi121.stdout:2/598: getdents d5/df/d60/d98 0 2024-02-15T18:35:44.934 INFO:tasks.workunit.client.1.smithi121.stdout:2/599: dread - d5/df/d3f/fa7 zero size 2024-02-15T18:35:44.936 INFO:tasks.workunit.client.1.smithi121.stdout:2/600: unlink d5/df/d60/d98/d22/f3c 0 2024-02-15T18:35:44.937 INFO:tasks.workunit.client.1.smithi121.stdout:2/601: chown d5/df/d27/d3e/d6a/l6e 6 1 2024-02-15T18:35:44.937 INFO:tasks.workunit.client.1.smithi121.stdout:2/602: fsync d5/df/f2a 0 2024-02-15T18:35:44.938 INFO:tasks.workunit.client.1.smithi121.stdout:2/603: symlink d5/df/d60/d98/d46/la8 0 2024-02-15T18:35:44.939 INFO:tasks.workunit.client.1.smithi121.stdout:2/604: truncate d5/f78 1301910 0 2024-02-15T18:35:44.940 INFO:tasks.workunit.client.1.smithi121.stdout:2/605: creat d5/df/d10/fa9 x:0 0 0 2024-02-15T18:35:44.941 INFO:tasks.workunit.client.1.smithi121.stdout:2/606: mkdir d5/df/d3f/daa 0 2024-02-15T18:35:44.941 INFO:tasks.workunit.client.1.smithi121.stdout:2/607: truncate d5/d49/f7f 233677 0 2024-02-15T18:35:44.942 INFO:tasks.workunit.client.1.smithi121.stdout:2/608: creat d5/df/d60/d98/d22/d81/d9a/da1/fab x:0 0 0 2024-02-15T18:35:44.950 INFO:tasks.workunit.client.1.smithi121.stdout:2/609: read d5/f78 [339669,70635] 0 2024-02-15T18:35:44.951 INFO:tasks.workunit.client.1.smithi121.stdout:2/610: mkdir d5/df/d60/d98/d22/d81/dac 0 2024-02-15T18:35:44.951 INFO:tasks.workunit.client.1.smithi121.stdout:2/611: chown d5/df/l68 86 1 2024-02-15T18:35:44.954 INFO:tasks.workunit.client.1.smithi121.stdout:2/612: dread d5/d49/f84 [0,4194304] 0 2024-02-15T18:35:44.955 INFO:tasks.workunit.client.1.smithi121.stdout:2/613: dread d5/df/f2b [0,4194304] 0 2024-02-15T18:35:44.956 INFO:tasks.workunit.client.1.smithi121.stdout:2/614: mkdir d5/df/d10/d4c/dad 0 2024-02-15T18:35:44.956 INFO:tasks.workunit.client.1.smithi121.stdout:2/615: readlink d5/df/d27/d3e/l28 0 2024-02-15T18:35:44.956 INFO:tasks.workunit.client.1.smithi121.stdout:2/616: mknod d5/df/d60/d98/d46/cae 0 2024-02-15T18:35:44.957 INFO:tasks.workunit.client.1.smithi121.stdout:2/617: creat d5/df/d27/d3e/faf x:0 0 0 2024-02-15T18:35:45.089 INFO:tasks.workunit.client.1.smithi121.stdout:3/641: sync 2024-02-15T18:35:45.089 INFO:tasks.workunit.client.1.smithi121.stdout:6/915: sync 2024-02-15T18:35:45.089 INFO:tasks.workunit.client.1.smithi121.stdout:3/642: chown d2/d1a/d27/d63/da4/d25/d31/f73 0 1 2024-02-15T18:35:45.089 INFO:tasks.workunit.client.1.smithi121.stdout:6/916: truncate d7/dc/d34/d42/d62/d63/d6f/d89/f136 481817 0 2024-02-15T18:35:45.089 INFO:tasks.workunit.client.1.smithi121.stdout:3/643: write d2/d1a/d27/da7/fad [337853,59187] 0 2024-02-15T18:35:45.089 INFO:tasks.workunit.client.1.smithi121.stdout:3/644: chown d2/d1a/d27/d63/da4/d25 6133 1 2024-02-15T18:35:45.090 INFO:tasks.workunit.client.1.smithi121.stdout:6/917: link d7/dc/d34/d42/d62/d63/f7a d7/d4b/f143 0 2024-02-15T18:35:45.093 INFO:tasks.workunit.client.1.smithi121.stdout:3/645: rename d2/f7 to d2/d1a/d27/d63/da4/d25/d30/db3/dd3/fd9 0 2024-02-15T18:35:45.093 INFO:tasks.workunit.client.1.smithi121.stdout:3/646: write d2/d1a/f1f [3365668,21257] 0 2024-02-15T18:35:45.094 INFO:tasks.workunit.client.1.smithi121.stdout:6/918: mknod d7/d4b/d60/d6c/dd5/dfa/d94/c144 0 2024-02-15T18:35:45.094 INFO:tasks.workunit.client.1.smithi121.stdout:6/919: chown d7/dc/d34/d42/d62/d63/ld2 838 1 2024-02-15T18:35:45.095 INFO:tasks.workunit.client.1.smithi121.stdout:6/920: write d7/dc/d2c/d36/f56 [418521,98391] 0 2024-02-15T18:35:45.096 INFO:tasks.workunit.client.1.smithi121.stdout:6/921: mkdir d7/dc/d34/d42/d62/dac/d10a/d145 0 2024-02-15T18:35:45.096 INFO:tasks.workunit.client.1.smithi121.stdout:6/922: write d7/dc/ff3 [4274314,81574] 0 2024-02-15T18:35:45.114 INFO:tasks.workunit.client.1.smithi121.stdout:3/647: dwrite d2/d1a/d27/da7/f35 [0,4194304] 0 2024-02-15T18:35:45.116 INFO:tasks.workunit.client.1.smithi121.stdout:3/648: read d2/d1a/d27/da7/f35 [1617237,13346] 0 2024-02-15T18:35:45.116 INFO:tasks.workunit.client.1.smithi121.stdout:3/649: fdatasync d2/d1a/d27/d63/da4/d25/d31/d58/fbd 0 2024-02-15T18:35:45.116 INFO:tasks.workunit.client.1.smithi121.stdout:3/650: write d2/d1a/d27/d63/da4/f1b [5484088,106412] 0 2024-02-15T18:35:45.116 INFO:tasks.workunit.client.1.smithi121.stdout:3/651: write d2/d1a/d27/da7/fc3 [498322,82441] 0 2024-02-15T18:35:45.120 INFO:tasks.workunit.client.1.smithi121.stdout:3/652: link d2/d1a/d27/da7/l3c d2/d1a/d27/da7/lda 0 2024-02-15T18:35:45.120 INFO:tasks.workunit.client.1.smithi121.stdout:3/653: stat d2/d1a/d27/da7/d85/d9f 0 2024-02-15T18:35:45.120 INFO:tasks.workunit.client.1.smithi121.stdout:3/654: dread - d2/d1a/d27/d63/da4/d25/d31/d34/d4a/fa0 zero size 2024-02-15T18:35:45.121 INFO:tasks.workunit.client.1.smithi121.stdout:3/655: creat d2/d1a/d27/d63/da4/d25/d31/d34/fdb x:0 0 0 2024-02-15T18:35:45.122 INFO:tasks.workunit.client.1.smithi121.stdout:3/656: dread d2/d1a/d6b/d7d/fa1 [0,4194304] 0 2024-02-15T18:35:45.122 INFO:tasks.workunit.client.1.smithi121.stdout:3/657: chown d2/d1a/d27/d63/da4/d25/d30/fd6 47802 1 2024-02-15T18:35:45.122 INFO:tasks.workunit.client.1.smithi121.stdout:3/658: dread - d2/d1a/d6b/fa5 zero size 2024-02-15T18:35:45.122 INFO:tasks.workunit.client.1.smithi121.stdout:3/659: fsync d2/d1a/d27/d63/da4/d25/d30/f41 0 2024-02-15T18:35:45.123 INFO:tasks.workunit.client.1.smithi121.stdout:3/660: mknod d2/d1a/cdc 0 2024-02-15T18:35:45.124 INFO:tasks.workunit.client.1.smithi121.stdout:6/923: dwrite d7/d4b/d60/d6c/dd5/dfa/f97 [0,4194304] 0 2024-02-15T18:35:45.128 INFO:tasks.workunit.client.1.smithi121.stdout:6/924: symlink d7/dc/d34/d42/d62/d63/d66/dce/l146 0 2024-02-15T18:35:45.128 INFO:tasks.workunit.client.1.smithi121.stdout:6/925: write d7/dc/d2c/d36/f121 [210545,87862] 0 2024-02-15T18:35:45.128 INFO:tasks.workunit.client.1.smithi121.stdout:6/926: chown d7/dc/d2c/d100 5 1 2024-02-15T18:35:45.130 INFO:tasks.workunit.client.1.smithi121.stdout:6/927: link d7/d4b/c7b d7/d4b/d129/c147 0 2024-02-15T18:35:45.140 INFO:tasks.workunit.client.1.smithi121.stdout:3/661: dwrite d2/d5a/f67 [0,4194304] 0 2024-02-15T18:35:45.140 INFO:tasks.workunit.client.1.smithi121.stdout:3/662: fdatasync d2/d5a/f5f 0 2024-02-15T18:35:45.143 INFO:tasks.workunit.client.1.smithi121.stdout:3/663: mkdir d2/d1a/d27/da7/d85/d9f/ddd 0 2024-02-15T18:35:45.143 INFO:tasks.workunit.client.1.smithi121.stdout:3/664: chown d2/d1a/d27/d63/da4/d25/d31/d34/d4a/f5c 62 1 2024-02-15T18:35:45.152 INFO:tasks.workunit.client.1.smithi121.stdout:3/665: dread d2/f9 [0,4194304] 0 2024-02-15T18:35:45.154 INFO:tasks.workunit.client.1.smithi121.stdout:3/666: link d2/d1a/d27/d63/da4/f2f d2/d1a/d27/d63/da4/d25/d30/db3/fde 0 2024-02-15T18:35:45.155 INFO:tasks.workunit.client.1.smithi121.stdout:3/667: mkdir d2/d1a/d27/d63/da4/d25/d30/db3/ddf 0 2024-02-15T18:35:45.159 INFO:tasks.workunit.client.1.smithi121.stdout:3/668: dread d2/d1a/d27/f7f [0,4194304] 0 2024-02-15T18:35:45.160 INFO:tasks.workunit.client.1.smithi121.stdout:3/669: creat d2/d1a/d27/da7/d85/d9f/ddd/fe0 x:0 0 0 2024-02-15T18:35:45.160 INFO:tasks.workunit.client.1.smithi121.stdout:3/670: read - d2/d1a/d27/d63/da4/d25/d31/d58/f6f zero size 2024-02-15T18:35:45.160 INFO:tasks.workunit.client.1.smithi121.stdout:3/671: mknod d2/d1a/d27/da7/d85/d9f/ce1 0 2024-02-15T18:35:45.160 INFO:tasks.workunit.client.1.smithi121.stdout:3/672: mkdir d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2 0 2024-02-15T18:35:45.174 INFO:tasks.workunit.client.1.smithi121.stdout:3/673: dwrite d2/d1a/d27/d63/da4/d25/f52 [0,4194304] 0 2024-02-15T18:35:45.174 INFO:tasks.workunit.client.1.smithi121.stdout:3/674: truncate d2/d1a/d6b/fa5 638034 0 2024-02-15T18:35:45.183 INFO:tasks.workunit.client.1.smithi121.stdout:3/675: dread d2/d1a/d27/da7/f35 [0,4194304] 0 2024-02-15T18:35:45.184 INFO:tasks.workunit.client.1.smithi121.stdout:3/676: symlink d2/le3 0 2024-02-15T18:35:45.185 INFO:tasks.workunit.client.1.smithi121.stdout:3/677: creat d2/d1a/d27/d63/da4/fe4 x:0 0 0 2024-02-15T18:35:45.185 INFO:tasks.workunit.client.1.smithi121.stdout:3/678: write d2/d1a/d27/d63/da4/d25/d31/d34/d4a/fb7 [464037,25342] 0 2024-02-15T18:35:45.186 INFO:tasks.workunit.client.1.smithi121.stdout:3/679: rename d2/d1a/d27/d63/da4/d25/f52 to d2/d1a/d54/fe5 0 2024-02-15T18:35:45.187 INFO:tasks.workunit.client.1.smithi121.stdout:3/680: write d2/d1a/d27/da7/f42 [3633649,120913] 0 2024-02-15T18:35:45.189 INFO:tasks.workunit.client.1.smithi121.stdout:3/681: rename d2/d1a/c8f to d2/d1a/d27/d63/d81/ce6 0 2024-02-15T18:35:45.189 INFO:tasks.workunit.client.1.smithi121.stdout:3/682: truncate d2/d1a/f7a 251399 0 2024-02-15T18:35:45.190 INFO:tasks.workunit.client.1.smithi121.stdout:3/683: write d2/d1a/d27/d63/f80 [2342513,19718] 0 2024-02-15T18:35:45.190 INFO:tasks.workunit.client.1.smithi121.stdout:3/684: chown d2/d1a/d27/c68 33885 1 2024-02-15T18:35:45.190 INFO:tasks.workunit.client.1.smithi121.stdout:3/685: write d2/d1a/d27/d63/da4/d25/d31/d34/d4a/fa0 [818786,7492] 0 2024-02-15T18:35:45.192 INFO:tasks.workunit.client.1.smithi121.stdout:3/686: mknod d2/d1a/d27/d63/da4/d25/d30/db3/ddf/ce7 0 2024-02-15T18:35:45.192 INFO:tasks.workunit.client.1.smithi121.stdout:3/687: mkdir d2/d1a/d27/da7/d85/d9f/ddd/de8 0 2024-02-15T18:35:45.193 INFO:tasks.workunit.client.1.smithi121.stdout:3/688: mknod d2/d1a/d27/ce9 0 2024-02-15T18:35:45.198 INFO:tasks.workunit.client.1.smithi121.stdout:3/689: mknod d2/d1a/d27/d63/da4/d25/d30/db3/cea 0 2024-02-15T18:35:45.199 INFO:tasks.workunit.client.1.smithi121.stdout:3/690: rename d2/d1a/d27/ce9 to d2/d1a/d27/ceb 0 2024-02-15T18:35:45.199 INFO:tasks.workunit.client.1.smithi121.stdout:3/691: getdents d2/d1a/d27/da7/d85/d9f/ddd/de8 0 2024-02-15T18:35:45.219 INFO:tasks.workunit.client.1.smithi121.stdout:3/692: dwrite d2/d1a/d27/f77 [0,4194304] 0 2024-02-15T18:35:45.219 INFO:tasks.workunit.client.1.smithi121.stdout:3/693: chown d2/d1a/d6b/d7d/lca 513914032 1 2024-02-15T18:35:45.222 INFO:tasks.workunit.client.1.smithi121.stdout:3/694: symlink d2/d1a/d27/d63/da4/d25/d31/d58/lec 0 2024-02-15T18:35:45.223 INFO:tasks.workunit.client.1.smithi121.stdout:3/695: creat d2/d1a/d27/d63/da4/d25/d88/d9a/fed x:0 0 0 2024-02-15T18:35:45.231 INFO:tasks.workunit.client.1.smithi121.stdout:3/696: dread d2/d1a/d27/d63/da4/d25/d30/f46 [0,4194304] 0 2024-02-15T18:35:45.232 INFO:tasks.workunit.client.1.smithi121.stdout:3/697: creat d2/d1a/d27/d63/da4/d25/d31/d97/daf/fee x:0 0 0 2024-02-15T18:35:45.233 INFO:tasks.workunit.client.1.smithi121.stdout:3/698: mknod d2/d1a/d54/dc1/cef 0 2024-02-15T18:35:45.233 INFO:tasks.workunit.client.1.smithi121.stdout:3/699: chown d2/d1a/d6b/f79 30226 1 2024-02-15T18:35:45.233 INFO:tasks.workunit.client.1.smithi121.stdout:3/700: truncate d2/d1a/d27/d63/da4/d25/d31/d34/d4a/f5c 218254 0 2024-02-15T18:35:45.235 INFO:tasks.workunit.client.1.smithi121.stdout:3/701: rename d2/d1a/d27/d63/da4/d25/d31/d34/cd2 to d2/d1a/d27/d63/da4/d25/d30/db3/ddf/cf0 0 2024-02-15T18:35:45.235 INFO:tasks.workunit.client.1.smithi121.stdout:3/702: readlink d2/d1a/l21 0 2024-02-15T18:35:45.235 INFO:tasks.workunit.client.1.smithi121.stdout:3/703: fsync d2/d1a/d27/d63/da4/d25/d30/f41 0 2024-02-15T18:35:45.235 INFO:tasks.workunit.client.1.smithi121.stdout:3/704: write d2/d1a/d27/d63/da4/d25/d30/f46 [661825,130094] 0 2024-02-15T18:35:45.235 INFO:tasks.workunit.client.1.smithi121.stdout:3/705: write d2/d1a/d27/da7/d85/d9f/ddd/fe0 [321479,40303] 0 2024-02-15T18:35:45.235 INFO:tasks.workunit.client.1.smithi121.stdout:3/706: write d2/d1a/d27/d63/da4/fe4 [245336,80911] 0 2024-02-15T18:35:45.235 INFO:tasks.workunit.client.1.smithi121.stdout:3/707: fsync d2/d1a/d27/d63/da4/d25/d31/d34/d4a/f9c 0 2024-02-15T18:35:45.236 INFO:tasks.workunit.client.1.smithi121.stdout:3/708: mkdir d2/d1a/d27/da7/d85/d9f/df1 0 2024-02-15T18:35:45.236 INFO:tasks.workunit.client.1.smithi121.stdout:3/709: mknod d2/d1a/d27/d63/da4/d25/d31/d97/daf/cf2 0 2024-02-15T18:35:45.242 INFO:tasks.workunit.client.1.smithi121.stdout:3/710: rmdir d2/d1a/d27/d63/da4 39 2024-02-15T18:35:45.243 INFO:tasks.workunit.client.1.smithi121.stdout:3/711: rmdir d2/d1a/d27 39 2024-02-15T18:35:45.243 INFO:tasks.workunit.client.1.smithi121.stdout:3/712: dread - d2/d1a/d27/da7/f53 zero size 2024-02-15T18:35:45.244 INFO:tasks.workunit.client.1.smithi121.stdout:3/713: stat d2/d1a/d27/d63/f80 0 2024-02-15T18:35:45.244 INFO:tasks.workunit.client.1.smithi121.stdout:3/714: dread - d2/d1a/d27/da7/fb9 zero size 2024-02-15T18:35:45.244 INFO:tasks.workunit.client.1.smithi121.stdout:3/715: symlink d2/d1a/d27/da7/d85/d9f/ddd/lf3 0 2024-02-15T18:35:45.245 INFO:tasks.workunit.client.1.smithi121.stdout:3/716: mknod d2/d1a/d27/d63/da4/d25/d30/db3/ddf/cf4 0 2024-02-15T18:35:45.246 INFO:tasks.workunit.client.1.smithi121.stdout:3/717: creat d2/d1a/d27/ff5 x:0 0 0 2024-02-15T18:35:45.247 INFO:tasks.workunit.client.1.smithi121.stdout:3/718: mknod d2/d1a/d27/da7/d85/d9f/ddd/cf6 0 2024-02-15T18:35:45.247 INFO:tasks.workunit.client.1.smithi121.stdout:3/719: dread - d2/d1a/d27/d63/da4/d25/d30/fd6 zero size 2024-02-15T18:35:45.247 INFO:tasks.workunit.client.1.smithi121.stdout:3/720: mknod d2/d1a/d27/d63/cf7 0 2024-02-15T18:35:45.247 INFO:tasks.workunit.client.1.smithi121.stdout:3/721: readlink d2/d1a/d27/da7/l3c 0 2024-02-15T18:35:45.263 INFO:tasks.workunit.client.1.smithi121.stdout:3/722: dwrite d2/d5a/f95 [0,4194304] 0 2024-02-15T18:35:45.265 INFO:tasks.workunit.client.1.smithi121.stdout:3/723: dread d2/d1a/d27/d63/da4/d25/d31/d34/d4a/fb7 [0,4194304] 0 2024-02-15T18:35:45.271 INFO:tasks.workunit.client.1.smithi121.stdout:3/724: symlink d2/d1a/d6b/d7d/lf8 0 2024-02-15T18:35:45.271 INFO:tasks.workunit.client.1.smithi121.stdout:3/725: read - d2/d1a/d27/d63/da4/d25/fb0 zero size 2024-02-15T18:35:45.272 INFO:tasks.workunit.client.1.smithi121.stdout:3/726: symlink d2/d5a/lf9 0 2024-02-15T18:35:45.273 INFO:tasks.workunit.client.1.smithi121.stdout:8/782: sync 2024-02-15T18:35:45.273 INFO:tasks.workunit.client.1.smithi121.stdout:4/827: sync 2024-02-15T18:35:45.273 INFO:tasks.workunit.client.1.smithi121.stdout:8/783: chown d0/d7/d36/f5c 327 1 2024-02-15T18:35:45.273 INFO:tasks.workunit.client.1.smithi121.stdout:8/784: stat d0/d61/d8f/fdd 0 2024-02-15T18:35:45.274 INFO:tasks.workunit.client.1.smithi121.stdout:3/727: rename d2/d1a/f37 to d2/d1a/d27/d63/da4/d25/d30/db3/ffa 0 2024-02-15T18:35:45.275 INFO:tasks.workunit.client.1.smithi121.stdout:3/728: mknod d2/d1a/d27/d63/d81/cfb 0 2024-02-15T18:35:45.276 INFO:tasks.workunit.client.1.smithi121.stdout:8/785: unlink d0/d2a/fd6 0 2024-02-15T18:35:45.277 INFO:tasks.workunit.client.1.smithi121.stdout:3/729: truncate d2/d1a/d27/d63/f80 262277 0 2024-02-15T18:35:45.278 INFO:tasks.workunit.client.1.smithi121.stdout:8/786: unlink d0/d2a/f6e 0 2024-02-15T18:35:45.279 INFO:tasks.workunit.client.1.smithi121.stdout:3/730: link d2/d1a/d27/d63/da4/d25/d31/d34/d4a/l90 d2/d1a/d27/lfc 0 2024-02-15T18:35:45.280 INFO:tasks.workunit.client.1.smithi121.stdout:8/787: symlink d0/d7/d36/l10e 0 2024-02-15T18:35:45.282 INFO:tasks.workunit.client.1.smithi121.stdout:4/828: dread d5/d9/d76/d87/fab [0,4194304] 0 2024-02-15T18:35:45.283 INFO:tasks.workunit.client.1.smithi121.stdout:4/829: rename d5/d9/d1d/cec to d5/d9/d19/c104 0 2024-02-15T18:35:45.285 INFO:tasks.workunit.client.1.smithi121.stdout:3/731: getdents d2/d1a/d54 0 2024-02-15T18:35:45.288 INFO:tasks.workunit.client.1.smithi121.stdout:8/788: unlink d0/d7/d36/d8c/f4f 0 2024-02-15T18:35:45.290 INFO:tasks.workunit.client.1.smithi121.stdout:3/732: creat d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2/ffd x:0 0 0 2024-02-15T18:35:45.290 INFO:tasks.workunit.client.1.smithi121.stdout:8/789: rmdir d0/d7/d36/d52/d7a 39 2024-02-15T18:35:45.305 INFO:tasks.workunit.client.1.smithi121.stdout:4/830: dwrite d5/d9/d1d/f24 [4194304,4194304] 0 2024-02-15T18:35:45.305 INFO:tasks.workunit.client.1.smithi121.stdout:4/831: stat d5/d31 0 2024-02-15T18:35:45.308 INFO:tasks.workunit.client.1.smithi121.stdout:4/832: mknod d5/d9/d19/d4c/d82/c105 0 2024-02-15T18:35:45.321 INFO:tasks.workunit.client.1.smithi121.stdout:4/833: mknod d5/d9/d1d/d26/d72/c106 0 2024-02-15T18:35:45.321 INFO:tasks.workunit.client.1.smithi121.stdout:8/790: dwrite d0/d7/de/f4c [0,4194304] 0 2024-02-15T18:35:45.321 INFO:tasks.workunit.client.1.smithi121.stdout:8/791: chown d0/d7/d36/d8c/d45/f2c 10735 1 2024-02-15T18:35:45.321 INFO:tasks.workunit.client.1.smithi121.stdout:8/792: write d0/d7/de/f1c [8736065,88493] 0 2024-02-15T18:35:45.322 INFO:tasks.workunit.client.1.smithi121.stdout:3/733: dwrite d2/d1a/d27/d63/d81/fac [0,4194304] 0 2024-02-15T18:35:45.322 INFO:tasks.workunit.client.1.smithi121.stdout:3/734: readlink d2/d1a/d27/d63/da4/d25/d31/l4f 0 2024-02-15T18:35:45.322 INFO:tasks.workunit.client.1.smithi121.stdout:8/793: dread d0/fbd [0,4194304] 0 2024-02-15T18:35:45.326 INFO:tasks.workunit.client.1.smithi121.stdout:8/794: link d0/d7/d36/d8c/d45/dcd/lda d0/d7/d36/d8c/d45/dcd/l10f 0 2024-02-15T18:35:45.332 INFO:tasks.workunit.client.1.smithi121.stdout:3/735: dread d2/d1a/d27/d63/da4/f2f [0,4194304] 0 2024-02-15T18:35:45.333 INFO:tasks.workunit.client.1.smithi121.stdout:3/736: link d2/d1a/d27/d63/fcc d2/d1a/d27/d63/da4/d25/d31/d97/ffe 0 2024-02-15T18:35:45.334 INFO:tasks.workunit.client.1.smithi121.stdout:3/737: creat d2/d1a/d27/d63/da4/d25/d88/d9a/fff x:0 0 0 2024-02-15T18:35:45.341 INFO:tasks.workunit.client.1.smithi121.stdout:4/834: symlink d5/d9/l107 0 2024-02-15T18:35:45.353 INFO:tasks.workunit.client.1.smithi121.stdout:3/738: dwrite d2/d1a/d27/d63/da4/f7b [0,4194304] 0 2024-02-15T18:35:45.353 INFO:tasks.workunit.client.1.smithi121.stdout:3/739: getdents d2/d1a/d27/da7/d85/d9f/df1 0 2024-02-15T18:35:45.353 INFO:tasks.workunit.client.1.smithi121.stdout:8/795: dwrite d0/d7/f10 [4194304,4194304] 0 2024-02-15T18:35:45.356 INFO:tasks.workunit.client.1.smithi121.stdout:3/740: symlink d2/d1a/d54/l100 0 2024-02-15T18:35:45.356 INFO:tasks.workunit.client.1.smithi121.stdout:3/741: write d2/d1a/d27/da7/f47 [4783231,40659] 0 2024-02-15T18:35:45.357 INFO:tasks.workunit.client.1.smithi121.stdout:3/742: creat d2/d1a/d27/d63/da4/d25/d88/d96/f101 x:0 0 0 2024-02-15T18:35:45.358 INFO:tasks.workunit.client.1.smithi121.stdout:3/743: rmdir d2/d1a/d6b/d7d 39 2024-02-15T18:35:45.365 INFO:tasks.workunit.client.1.smithi121.stdout:4/835: dwrite d5/d9/d19/d28/fb3 [0,4194304] 0 2024-02-15T18:35:45.365 INFO:tasks.workunit.client.1.smithi121.stdout:4/836: readlink d5/d31/lc2 0 2024-02-15T18:35:45.368 INFO:tasks.workunit.client.1.smithi121.stdout:4/837: read d5/d31/d25/db6/fe9 [3185420,57539] 0 2024-02-15T18:35:45.374 INFO:tasks.workunit.client.1.smithi121.stdout:3/744: dwrite d2/d1a/d27/d63/da4/d25/d31/d34/d4a/fa0 [0,4194304] 0 2024-02-15T18:35:45.389 INFO:tasks.workunit.client.1.smithi121.stdout:3/745: dwrite d2/d5a/f95 [0,4194304] 0 2024-02-15T18:35:45.391 INFO:tasks.workunit.client.1.smithi121.stdout:3/746: creat d2/d1a/d27/d63/da4/d25/d30/db3/f102 x:0 0 0 2024-02-15T18:35:45.393 INFO:tasks.workunit.client.1.smithi121.stdout:3/747: creat d2/d1a/d27/d63/da4/d25/d30/db3/ddf/f103 x:0 0 0 2024-02-15T18:35:45.455 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:44.597118+0000 mgr.smithi121.ckiryi (mgr.24385) 16 : cluster [DBG] pgmap v16: 65 pgs: 65 active+clean; 1.6 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 39 MiB/s rd, 87 MiB/s wr, 328 op/s 2024-02-15T18:35:45.574 INFO:tasks.workunit.client.1.smithi121.stdout:2/618: sync 2024-02-15T18:35:45.575 INFO:tasks.workunit.client.1.smithi121.stdout:7/608: sync 2024-02-15T18:35:45.575 INFO:tasks.workunit.client.1.smithi121.stdout:6/928: sync 2024-02-15T18:35:45.575 INFO:tasks.workunit.client.1.smithi121.stdout:2/619: write d5/df/d60/d98/d58/f87 [4169115,77937] 0 2024-02-15T18:35:45.576 INFO:tasks.workunit.client.1.smithi121.stdout:2/620: write d5/fd [2824288,11234] 0 2024-02-15T18:35:45.576 INFO:tasks.workunit.client.1.smithi121.stdout:2/621: chown d5/df/d27/l55 20 1 2024-02-15T18:35:45.576 INFO:tasks.workunit.client.1.smithi121.stdout:6/929: creat d7/dc/f148 x:0 0 0 2024-02-15T18:35:45.577 INFO:tasks.workunit.client.1.smithi121.stdout:7/609: rmdir d0/d5/d76 39 2024-02-15T18:35:45.579 INFO:tasks.workunit.client.1.smithi121.stdout:2/622: creat d5/df/d54/fb0 x:0 0 0 2024-02-15T18:35:45.581 INFO:tasks.workunit.client.1.smithi121.stdout:6/930: mkdir d7/dc/d34/d42/d62/dac/d10a/d145/d149 0 2024-02-15T18:35:45.582 INFO:tasks.workunit.client.1.smithi121.stdout:2/623: symlink d5/df/d10/lb1 0 2024-02-15T18:35:45.584 INFO:tasks.workunit.client.1.smithi121.stdout:7/610: dread d0/d4a/d5b/d9a/f9d [0,4194304] 0 2024-02-15T18:35:45.587 INFO:tasks.workunit.client.1.smithi121.stdout:7/611: rename d0/d4a/d5b/da6/c8c to d0/d5/d11/dbb/cc8 0 2024-02-15T18:35:45.588 INFO:tasks.workunit.client.1.smithi121.stdout:7/612: creat d0/d4a/d5b/d9a/fc9 x:0 0 0 2024-02-15T18:35:45.590 INFO:tasks.workunit.client.1.smithi121.stdout:7/613: write d0/d38/fa0 [4186719,116105] 0 2024-02-15T18:35:45.608 INFO:tasks.workunit.client.1.smithi121.stdout:2/624: dwrite d5/df/d60/d98/f2d [4194304,4194304] 0 2024-02-15T18:35:45.609 INFO:tasks.workunit.client.1.smithi121.stdout:7/614: dwrite d0/d38/d46/daa/f6e [0,4194304] 0 2024-02-15T18:35:45.609 INFO:tasks.workunit.client.1.smithi121.stdout:7/615: write d0/d5/f47 [974756,122052] 0 2024-02-15T18:35:45.615 INFO:tasks.workunit.client.1.smithi121.stdout:7/616: mknod d0/d5/d11/d5c/cca 0 2024-02-15T18:35:45.615 INFO:tasks.workunit.client.1.smithi121.stdout:7/617: write d0/d38/fc1 [623935,50852] 0 2024-02-15T18:35:45.617 INFO:tasks.workunit.client.1.smithi121.stdout:7/618: unlink d0/d5/f5e 0 2024-02-15T18:35:45.618 INFO:tasks.workunit.client.1.smithi121.stdout:7/619: getdents d0/d5/d4c 0 2024-02-15T18:35:45.620 INFO:tasks.workunit.client.1.smithi121.stdout:7/620: mknod d0/d38/d46/ccb 0 2024-02-15T18:35:45.623 INFO:tasks.workunit.client.1.smithi121.stdout:7/621: rename d0/d38/d3c to d0/d5/d11/d5c/dcc 0 2024-02-15T18:35:45.624 INFO:tasks.workunit.client.1.smithi121.stdout:7/622: fdatasync d0/f27 0 2024-02-15T18:35:45.626 INFO:tasks.workunit.client.1.smithi121.stdout:7/623: symlink d0/d38/d3f/d43/lcd 0 2024-02-15T18:35:45.637 INFO:tasks.workunit.client.1.smithi121.stdout:2/625: dwrite f4 [0,4194304] 0 2024-02-15T18:35:45.638 INFO:tasks.workunit.client.1.smithi121.stdout:2/626: chown d5/df/d60/d98/d22/d81/d9a/da1/fab 837603 1 2024-02-15T18:35:45.640 INFO:tasks.workunit.client.1.smithi121.stdout:2/627: creat d5/df/d10/d4c/fb2 x:0 0 0 2024-02-15T18:35:45.643 INFO:tasks.workunit.client.1.smithi121.stdout:2/628: symlink d5/lb3 0 2024-02-15T18:35:45.645 INFO:tasks.workunit.client.1.smithi121.stdout:2/629: link d5/df/d10/f6d d5/df/d27/d3a/fb4 0 2024-02-15T18:35:45.647 INFO:tasks.workunit.client.1.smithi121.stdout:2/630: link d5/df/d3f/f88 d5/fb5 0 2024-02-15T18:35:45.647 INFO:tasks.workunit.client.1.smithi121.stdout:2/631: write d5/df/d10/f8a [70596,26611] 0 2024-02-15T18:35:45.648 INFO:tasks.workunit.client.1.smithi121.stdout:2/632: getdents d5/df/d60/d98/d46 0 2024-02-15T18:35:45.649 INFO:tasks.workunit.client.1.smithi121.stdout:2/633: mknod d5/df/d60/d98/d58/d63/cb6 0 2024-02-15T18:35:45.650 INFO:tasks.workunit.client.1.smithi121.stdout:2/634: creat d5/df/d60/d98/d58/d63/d94/fb7 x:0 0 0 2024-02-15T18:35:45.651 INFO:tasks.workunit.client.1.smithi121.stdout:2/635: creat d5/df/d60/d98/fb8 x:0 0 0 2024-02-15T18:35:45.652 INFO:tasks.workunit.client.1.smithi121.stdout:2/636: truncate d5/df/f2a 3542451 0 2024-02-15T18:35:45.652 INFO:tasks.workunit.client.1.smithi121.stdout:2/637: rmdir d5/df/d60/d98/d58/d63 39 2024-02-15T18:35:45.654 INFO:tasks.workunit.client.1.smithi121.stdout:2/638: rename d5/df/d3f/f69 to d5/df/d60/d98/d46/fb9 0 2024-02-15T18:35:45.655 INFO:tasks.workunit.client.1.smithi121.stdout:2/639: fsync d5/df/d27/f79 0 2024-02-15T18:35:45.655 INFO:tasks.workunit.client.1.smithi121.stdout:2/640: chown d5/df/d27/d3e/f62 836602646 1 2024-02-15T18:35:45.655 INFO:tasks.workunit.client.1.smithi121.stdout:2/641: chown d5/df/d27/d3e/d6a/l7a 155166556 1 2024-02-15T18:35:45.656 INFO:tasks.workunit.client.1.smithi121.stdout:2/642: rename d5/df/d27/d3e/faf to d5/d49/d71/fba 0 2024-02-15T18:35:45.657 INFO:tasks.workunit.client.1.smithi121.stdout:2/643: symlink d5/df/d3f/daa/lbb 0 2024-02-15T18:35:45.675 INFO:tasks.workunit.client.1.smithi121.stdout:2/644: dwrite d5/f3b [0,4194304] 0 2024-02-15T18:35:45.678 INFO:tasks.workunit.client.1.smithi121.stdout:2/645: creat d5/df/d60/d98/d22/d81/dac/fbc x:0 0 0 2024-02-15T18:35:45.678 INFO:tasks.workunit.client.1.smithi121.stdout:2/646: fdatasync d5/df/d27/d3e/f62 0 2024-02-15T18:35:45.678 INFO:tasks.workunit.client.1.smithi121.stdout:2/647: truncate d5/f65 511864 0 2024-02-15T18:35:45.680 INFO:tasks.workunit.client.1.smithi121.stdout:2/648: rename d5/d49/d71/l74 to d5/df/d10/d4c/dad/lbd 0 2024-02-15T18:35:45.680 INFO:tasks.workunit.client.1.smithi121.stdout:2/649: truncate d5/df/d27/f79 1322051 0 2024-02-15T18:35:45.680 INFO:tasks.workunit.client.1.smithi121.stdout:2/650: mkdir d5/df/d60/d98/d22/d81/dbe 0 2024-02-15T18:35:45.682 INFO:tasks.workunit.client.1.smithi121.stdout:2/651: rename l1 to d5/df/d60/d98/d22/d81/d9a/lbf 0 2024-02-15T18:35:45.682 INFO:tasks.workunit.client.1.smithi121.stdout:2/652: chown d5/df/d60/f86 306 1 2024-02-15T18:35:45.684 INFO:tasks.workunit.client.1.smithi121.stdout:2/653: getdents d5 0 2024-02-15T18:35:45.685 INFO:tasks.workunit.client.1.smithi121.stdout:2/654: truncate d5/df/d60/d98/d58/d63/d94/fb7 922189 0 2024-02-15T18:35:45.686 INFO:tasks.workunit.client.1.smithi121.stdout:2/655: creat d5/df/d10/fc0 x:0 0 0 2024-02-15T18:35:45.687 INFO:tasks.workunit.client.1.smithi121.stdout:2/656: mkdir d5/df/dc1 0 2024-02-15T18:35:45.704 INFO:tasks.workunit.client.1.smithi121.stdout:2/657: dwrite d5/df/d27/d3e/f62 [0,4194304] 0 2024-02-15T18:35:45.707 INFO:tasks.workunit.client.1.smithi121.stdout:2/658: rename d5/df/d60/d98/d22/d81/d9a/da5 to d5/df/d27/d3e/d9f/dc2 0 2024-02-15T18:35:45.709 INFO:tasks.workunit.client.1.smithi121.stdout:2/659: mkdir d5/df/d60/d98/d58/d63/d94/dc3 0 2024-02-15T18:35:45.711 INFO:tasks.workunit.client.1.smithi121.stdout:2/660: rename d5/df/d27/d3a/c4e to d5/df/cc4 0 2024-02-15T18:35:45.712 INFO:tasks.workunit.client.1.smithi121.stdout:2/661: rmdir d5/df/d27/d3e/d6a 39 2024-02-15T18:35:45.714 INFO:tasks.workunit.client.1.smithi121.stdout:2/662: creat d5/df/d27/d3e/fc5 x:0 0 0 2024-02-15T18:35:45.714 INFO:tasks.workunit.client.1.smithi121.stdout:2/663: write d5/df/d60/d98/d22/d81/dac/fbc [189049,107807] 0 2024-02-15T18:35:45.714 INFO:tasks.workunit.client.1.smithi121.stdout:2/664: truncate d5/df/d60/f86 854583 0 2024-02-15T18:35:45.721 INFO:tasks.workunit.client.1.smithi121.stdout:2/665: unlink d5/df/d60/d98/d46/fb9 0 2024-02-15T18:35:45.742 INFO:tasks.workunit.client.1.smithi121.stdout:2/666: dwrite d5/df/d10/fa9 [0,4194304] 0 2024-02-15T18:35:45.749 INFO:tasks.workunit.client.1.smithi121.stdout:2/667: symlink d5/df/d27/d3e/d9f/lc6 0 2024-02-15T18:35:45.749 INFO:tasks.workunit.client.1.smithi121.stdout:2/668: fsync d5/df/d10/fc0 0 2024-02-15T18:35:45.752 INFO:tasks.workunit.client.1.smithi121.stdout:2/669: dread d5/df/d27/d3e/f59 [0,4194304] 0 2024-02-15T18:35:45.753 INFO:tasks.workunit.client.1.smithi121.stdout:2/670: rmdir d5/d49 39 2024-02-15T18:35:45.763 INFO:tasks.workunit.client.1.smithi121.stdout:2/671: dread d5/df/d10/f7e [0,4194304] 0 2024-02-15T18:35:45.767 INFO:tasks.workunit.client.1.smithi121.stdout:2/672: link d5/df/d27/f21 d5/df/d10/d4c/fc7 0 2024-02-15T18:35:45.767 INFO:tasks.workunit.client.1.smithi121.stdout:2/673: chown d5/df/d3f/daa 516727128 1 2024-02-15T18:35:45.768 INFO:tasks.workunit.client.1.smithi121.stdout:2/674: truncate d5/df/d60/d98/d58/f87 2347128 0 2024-02-15T18:35:45.771 INFO:tasks.workunit.client.1.smithi121.stdout:2/675: mkdir d5/df/d60/d98/d22/dc8 0 2024-02-15T18:35:45.781 INFO:tasks.workunit.client.1.smithi121.stdout:2/676: dread d5/f3b [0,4194304] 0 2024-02-15T18:35:45.781 INFO:tasks.workunit.client.1.smithi121.stdout:2/677: write d5/df/f2b [907907,6930] 0 2024-02-15T18:35:45.799 INFO:tasks.workunit.client.1.smithi121.stdout:2/678: dwrite d5/df/d60/d98/d22/d81/d9a/da1/fab [0,4194304] 0 2024-02-15T18:35:45.803 INFO:tasks.workunit.client.1.smithi121.stdout:2/679: creat d5/df/d60/d98/d58/d63/fc9 x:0 0 0 2024-02-15T18:35:45.804 INFO:tasks.workunit.client.1.smithi121.stdout:2/680: read d5/df/f1b [1994110,113361] 0 2024-02-15T18:35:45.806 INFO:tasks.workunit.client.1.smithi121.stdout:2/681: getdents d5/df/d60 0 2024-02-15T18:35:45.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:44.597118+0000 mgr.smithi121.ckiryi (mgr.24385) 16 : cluster [DBG] pgmap v16: 65 pgs: 65 active+clean; 1.6 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 39 MiB/s rd, 87 MiB/s wr, 328 op/s 2024-02-15T18:35:46.081 INFO:tasks.workunit.client.1.smithi121.stdout:6/931: sync 2024-02-15T18:35:46.082 INFO:tasks.workunit.client.1.smithi121.stdout:7/624: sync 2024-02-15T18:35:46.082 INFO:tasks.workunit.client.1.smithi121.stdout:4/838: sync 2024-02-15T18:35:46.082 INFO:tasks.workunit.client.1.smithi121.stdout:3/748: sync 2024-02-15T18:35:46.082 INFO:tasks.workunit.client.1.smithi121.stdout:6/932: dread - d7/dc/d34/d42/d62/d63/f106 zero size 2024-02-15T18:35:46.082 INFO:tasks.workunit.client.1.smithi121.stdout:8/796: sync 2024-02-15T18:35:46.082 INFO:tasks.workunit.client.1.smithi121.stdout:2/682: sync 2024-02-15T18:35:46.083 INFO:tasks.workunit.client.1.smithi121.stdout:7/625: read d0/d5/f47 [480244,24385] 0 2024-02-15T18:35:46.083 INFO:tasks.workunit.client.1.smithi121.stdout:7/626: stat d0/d38/d3f/d43/f8f 0 2024-02-15T18:35:46.083 INFO:tasks.workunit.client.1.smithi121.stdout:3/749: mknod d2/d1a/d6b/c104 0 2024-02-15T18:35:46.083 INFO:tasks.workunit.client.1.smithi121.stdout:3/750: readlink d2/le3 0 2024-02-15T18:35:46.084 INFO:tasks.workunit.client.1.smithi121.stdout:6/933: symlink d7/dc/d34/d42/d62/d63/d8d/l14a 0 2024-02-15T18:35:46.084 INFO:tasks.workunit.client.1.smithi121.stdout:6/934: dread - d7/d4b/d60/d6c/dd5/dfa/ffd zero size 2024-02-15T18:35:46.085 INFO:tasks.workunit.client.1.smithi121.stdout:8/797: symlink d0/d61/d66/l110 0 2024-02-15T18:35:46.085 INFO:tasks.workunit.client.1.smithi121.stdout:8/798: stat d0/d2a/fae 0 2024-02-15T18:35:46.086 INFO:tasks.workunit.client.1.smithi121.stdout:7/627: write d0/d38/d46/daa/f6e [396251,31003] 0 2024-02-15T18:35:46.087 INFO:tasks.workunit.client.1.smithi121.stdout:2/683: symlink d5/df/d60/d98/d22/d81/d8d/lca 0 2024-02-15T18:35:46.088 INFO:tasks.workunit.client.1.smithi121.stdout:4/839: dread d5/d31/d3d/d4f/d51/d71/fbc [0,4194304] 0 2024-02-15T18:35:46.089 INFO:tasks.workunit.client.1.smithi121.stdout:3/751: rename d2/d1a/d27/d63/da4/d25/d31/d34/d4a/f71 to d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2/f105 0 2024-02-15T18:35:46.089 INFO:tasks.workunit.client.1.smithi121.stdout:3/752: dread - d2/d1a/d27/da7/d85/d9f/fbe zero size 2024-02-15T18:35:46.091 INFO:tasks.workunit.client.1.smithi121.stdout:6/935: symlink d7/d4b/d60/d6c/l14b 0 2024-02-15T18:35:46.091 INFO:tasks.workunit.client.1.smithi121.stdout:6/936: chown d7/d4b/d60/ff7 239769027 1 2024-02-15T18:35:46.092 INFO:tasks.workunit.client.1.smithi121.stdout:4/840: mknod d5/d31/d3d/d4a/c108 0 2024-02-15T18:35:46.093 INFO:tasks.workunit.client.1.smithi121.stdout:4/841: creat d5/d31/d3d/d4f/d51/d80/f109 x:0 0 0 2024-02-15T18:35:46.094 INFO:tasks.workunit.client.1.smithi121.stdout:4/842: read - d5/d9/d76/ff3 zero size 2024-02-15T18:35:46.094 INFO:tasks.workunit.client.1.smithi121.stdout:7/628: mknod d0/d5/d11/d79/cce 0 2024-02-15T18:35:46.094 INFO:tasks.workunit.client.1.smithi121.stdout:7/629: truncate d0/d38/d61/f8a 957742 0 2024-02-15T18:35:46.094 INFO:tasks.workunit.client.1.smithi121.stdout:3/753: mknod d2/d1a/d27/d63/c106 0 2024-02-15T18:35:46.097 INFO:tasks.workunit.client.1.smithi121.stdout:6/937: mknod d7/d4b/c14c 0 2024-02-15T18:35:46.097 INFO:tasks.workunit.client.1.smithi121.stdout:7/630: creat d0/d38/d3f/d43/fcf x:0 0 0 2024-02-15T18:35:46.097 INFO:tasks.workunit.client.1.smithi121.stdout:3/754: mkdir d2/d1a/d27/da7/d85/d9f/d107 0 2024-02-15T18:35:46.099 INFO:tasks.workunit.client.1.smithi121.stdout:7/631: mknod d0/d4a/d5b/cd0 0 2024-02-15T18:35:46.100 INFO:tasks.workunit.client.1.smithi121.stdout:6/938: mkdir d7/d4b/d60/d6c/dd5/dfa/d14d 0 2024-02-15T18:35:46.105 INFO:tasks.workunit.client.1.smithi121.stdout:3/755: dread d2/d1a/d6b/f82 [0,4194304] 0 2024-02-15T18:35:46.106 INFO:tasks.workunit.client.1.smithi121.stdout:3/756: stat d2/d1a/d27/d63/da4/d25/d31/d34/c49 0 2024-02-15T18:35:46.107 INFO:tasks.workunit.client.1.smithi121.stdout:3/757: creat d2/d1a/d27/da7/d85/d9f/ddd/de8/f108 x:0 0 0 2024-02-15T18:35:46.109 INFO:tasks.workunit.client.1.smithi121.stdout:3/758: symlink d2/d1a/d27/d63/da4/d25/d31/l109 0 2024-02-15T18:35:46.109 INFO:tasks.workunit.client.1.smithi121.stdout:3/759: truncate d2/d1a/d27/d63/da4/d25/d31/d34/d4a/f6d 944366 0 2024-02-15T18:35:46.109 INFO:tasks.workunit.client.1.smithi121.stdout:8/799: dwrite d0/d7/de/fd9 [0,4194304] 0 2024-02-15T18:35:46.111 INFO:tasks.workunit.client.1.smithi121.stdout:2/684: dwrite d5/df/d27/d3a/fb4 [0,4194304] 0 2024-02-15T18:35:46.117 INFO:tasks.workunit.client.1.smithi121.stdout:2/685: mknod d5/df/d60/d98/d22/dc8/ccb 0 2024-02-15T18:35:46.118 INFO:tasks.workunit.client.1.smithi121.stdout:2/686: creat d5/df/d60/d98/d22/d81/d9a/da1/fcc x:0 0 0 2024-02-15T18:35:46.119 INFO:tasks.workunit.client.1.smithi121.stdout:2/687: symlink d5/df/d60/d98/d22/dc8/lcd 0 2024-02-15T18:35:46.119 INFO:tasks.workunit.client.1.smithi121.stdout:2/688: chown d5/df/d27 23277 1 2024-02-15T18:35:46.119 INFO:tasks.workunit.client.1.smithi121.stdout:2/689: chown d5/df/d10 2006 1 2024-02-15T18:35:46.121 INFO:tasks.workunit.client.1.smithi121.stdout:2/690: unlink d5/df/d60/d98/d22/f80 0 2024-02-15T18:35:46.122 INFO:tasks.workunit.client.1.smithi121.stdout:2/691: chown d5/df/d60/d98/l34 1 1 2024-02-15T18:35:46.122 INFO:tasks.workunit.client.1.smithi121.stdout:2/692: stat d5/df/d60/d98/l64 0 2024-02-15T18:35:46.122 INFO:tasks.workunit.client.1.smithi121.stdout:8/800: dread d0/d7/f10 [4194304,4194304] 0 2024-02-15T18:35:46.123 INFO:tasks.workunit.client.1.smithi121.stdout:2/693: symlink d5/d49/lce 0 2024-02-15T18:35:46.123 INFO:tasks.workunit.client.1.smithi121.stdout:2/694: chown d5/df/f39 510244693 1 2024-02-15T18:35:46.123 INFO:tasks.workunit.client.1.smithi121.stdout:3/760: dread d2/d5a/f67 [0,4194304] 0 2024-02-15T18:35:46.124 INFO:tasks.workunit.client.1.smithi121.stdout:8/801: creat d0/d3/dc9/f111 x:0 0 0 2024-02-15T18:35:46.124 INFO:tasks.workunit.client.1.smithi121.stdout:8/802: write d0/d7/fa4 [165115,33415] 0 2024-02-15T18:35:46.125 INFO:tasks.workunit.client.1.smithi121.stdout:2/695: creat d5/df/d60/d98/d22/d81/d8d/fcf x:0 0 0 2024-02-15T18:35:46.125 INFO:tasks.workunit.client.1.smithi121.stdout:8/803: write d0/d61/d66/f3b [4845612,1780] 0 2024-02-15T18:35:46.127 INFO:tasks.workunit.client.1.smithi121.stdout:3/761: rename d2/d1a/d27/d63/da4/d25/d31/d34/d4a to d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2/d10a 0 2024-02-15T18:35:46.127 INFO:tasks.workunit.client.1.smithi121.stdout:3/762: write d2/d5a/f78 [6080547,22289] 0 2024-02-15T18:35:46.128 INFO:tasks.workunit.client.1.smithi121.stdout:2/696: symlink d5/df/d60/d98/d22/d81/dac/ld0 0 2024-02-15T18:35:46.129 INFO:tasks.workunit.client.1.smithi121.stdout:4/843: dwrite d5/d9/d19/d2a/d4d/f54 [0,4194304] 0 2024-02-15T18:35:46.129 INFO:tasks.workunit.client.1.smithi121.stdout:4/844: write d5/d31/d3d/d4f/d51/d80/ffc [1016943,22510] 0 2024-02-15T18:35:46.129 INFO:tasks.workunit.client.1.smithi121.stdout:6/939: dwrite d7/dc/d34/d42/d62/d63/d8d/f105 [0,4194304] 0 2024-02-15T18:35:46.129 INFO:tasks.workunit.client.1.smithi121.stdout:6/940: dread - d7/d4b/d59/fec zero size 2024-02-15T18:35:46.129 INFO:tasks.workunit.client.1.smithi121.stdout:8/804: creat d0/dce/df8/f112 x:0 0 0 2024-02-15T18:35:46.130 INFO:tasks.workunit.client.1.smithi121.stdout:2/697: dread d5/df/f2b [0,4194304] 0 2024-02-15T18:35:46.130 INFO:tasks.workunit.client.1.smithi121.stdout:2/698: write d5/df/d60/d98/d22/f44 [5213939,32682] 0 2024-02-15T18:35:46.132 INFO:tasks.workunit.client.1.smithi121.stdout:4/845: getdents d5/d9/d19/d4c/d8d 0 2024-02-15T18:35:46.133 INFO:tasks.workunit.client.1.smithi121.stdout:4/846: fsync d5/d9/d19/f4e 0 2024-02-15T18:35:46.133 INFO:tasks.workunit.client.1.smithi121.stdout:4/847: fsync d5/d9/d1d/d26/d73/d81/fe5 0 2024-02-15T18:35:46.134 INFO:tasks.workunit.client.1.smithi121.stdout:3/763: mknod d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2/c10b 0 2024-02-15T18:35:46.135 INFO:tasks.workunit.client.1.smithi121.stdout:3/764: dread d2/fce [0,4194304] 0 2024-02-15T18:35:46.137 INFO:tasks.workunit.client.1.smithi121.stdout:8/805: mknod d0/d7/db1/dbb/de2/c113 0 2024-02-15T18:35:46.138 INFO:tasks.workunit.client.1.smithi121.stdout:2/699: creat d5/fd1 x:0 0 0 2024-02-15T18:35:46.139 INFO:tasks.workunit.client.1.smithi121.stdout:4/848: rmdir d5/d31/d3d/d4f/dd3 0 2024-02-15T18:35:46.139 INFO:tasks.workunit.client.1.smithi121.stdout:4/849: getdents d5/d9/d1d/d26/d9d 0 2024-02-15T18:35:46.139 INFO:tasks.workunit.client.1.smithi121.stdout:4/850: write d5/d31/d3d/d4f/d51/d71/fc6 [985507,12474] 0 2024-02-15T18:35:46.146 INFO:tasks.workunit.client.1.smithi121.stdout:3/765: getdents d2/d1a/d6b/d7d 0 2024-02-15T18:35:46.148 INFO:tasks.workunit.client.1.smithi121.stdout:2/700: creat d5/df/d3f/fd2 x:0 0 0 2024-02-15T18:35:46.148 INFO:tasks.workunit.client.1.smithi121.stdout:2/701: dread - d5/df/d3f/fa7 zero size 2024-02-15T18:35:46.149 INFO:tasks.workunit.client.1.smithi121.stdout:2/702: creat d5/df/d60/d98/d58/d63/d94/fd3 x:0 0 0 2024-02-15T18:35:46.151 INFO:tasks.workunit.client.1.smithi121.stdout:2/703: write d5/df/d60/d98/d58/f87 [1589087,114262] 0 2024-02-15T18:35:46.151 INFO:tasks.workunit.client.1.smithi121.stdout:2/704: fsync d5/df/d60/d98/d58/d63/fc9 0 2024-02-15T18:35:46.153 INFO:tasks.workunit.client.1.smithi121.stdout:2/705: mkdir d5/df/d60/d98/d22/d81/dac/dd4 0 2024-02-15T18:35:46.154 INFO:tasks.workunit.client.1.smithi121.stdout:2/706: symlink d5/df/ld5 0 2024-02-15T18:35:46.155 INFO:tasks.workunit.client.1.smithi121.stdout:2/707: symlink d5/df/d60/d98/d58/d63/d94/dc3/ld6 0 2024-02-15T18:35:46.156 INFO:tasks.workunit.client.1.smithi121.stdout:2/708: dread - d5/df/d54/f92 zero size 2024-02-15T18:35:46.156 INFO:tasks.workunit.client.1.smithi121.stdout:2/709: write d5/d49/f84 [966189,128805] 0 2024-02-15T18:35:46.156 INFO:tasks.workunit.client.1.smithi121.stdout:2/710: stat d5/df/d27/d3e/f93 0 2024-02-15T18:35:46.156 INFO:tasks.workunit.client.1.smithi121.stdout:2/711: getdents d5/df/d60/d98/d22/d81 0 2024-02-15T18:35:46.158 INFO:tasks.workunit.client.1.smithi121.stdout:2/712: symlink d5/df/d60/d98/d22/d81/d9a/ld7 0 2024-02-15T18:35:46.158 INFO:tasks.workunit.client.1.smithi121.stdout:2/713: readlink d5/d49/d71/l4a 0 2024-02-15T18:35:46.158 INFO:tasks.workunit.client.1.smithi121.stdout:2/714: readlink d5/df/d10/d4c/l7c 0 2024-02-15T18:35:46.159 INFO:tasks.workunit.client.1.smithi121.stdout:2/715: mknod d5/df/d60/d98/d58/d63/d94/cd8 0 2024-02-15T18:35:46.159 INFO:tasks.workunit.client.1.smithi121.stdout:2/716: fsync d5/fb5 0 2024-02-15T18:35:46.161 INFO:tasks.workunit.client.1.smithi121.stdout:4/851: getdents d5/d31/d3d/d4f/d51/d71/dca 0 2024-02-15T18:35:46.161 INFO:tasks.workunit.client.1.smithi121.stdout:2/717: rename d5/df/d10/d4c/l8b to d5/df/d27/d3e/ld9 0 2024-02-15T18:35:46.161 INFO:tasks.workunit.client.1.smithi121.stdout:2/718: fdatasync d5/df/d60/d98/d22/f44 0 2024-02-15T18:35:46.164 INFO:tasks.workunit.client.1.smithi121.stdout:3/766: dwrite d2/d5a/f67 [4194304,4194304] 0 2024-02-15T18:35:46.177 INFO:tasks.workunit.client.1.smithi121.stdout:8/806: rmdir d0/dce/d109 0 2024-02-15T18:35:46.178 INFO:tasks.workunit.client.1.smithi121.stdout:8/807: mkdir d0/d7/de/d114 0 2024-02-15T18:35:46.179 INFO:tasks.workunit.client.1.smithi121.stdout:8/808: rename d0/d7/db1/dbb/de2/c113 to d0/d61/d8f/c115 0 2024-02-15T18:35:46.183 INFO:tasks.workunit.client.1.smithi121.stdout:2/719: dwrite d5/df/d10/f8a [0,4194304] 0 2024-02-15T18:35:46.185 INFO:tasks.workunit.client.1.smithi121.stdout:2/720: write d5/d49/fa6 [3733635,73828] 0 2024-02-15T18:35:46.185 INFO:tasks.workunit.client.1.smithi121.stdout:2/721: truncate d5/d89/f99 940885 0 2024-02-15T18:35:46.185 INFO:tasks.workunit.client.1.smithi121.stdout:2/722: dread d5/df/d60/f8c [0,4194304] 0 2024-02-15T18:35:46.186 INFO:tasks.workunit.client.1.smithi121.stdout:3/767: dwrite d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2/d10a/f9c [0,4194304] 0 2024-02-15T18:35:46.189 INFO:tasks.workunit.client.1.smithi121.stdout:8/809: dread d0/f37 [0,4194304] 0 2024-02-15T18:35:46.190 INFO:tasks.workunit.client.1.smithi121.stdout:2/723: symlink d5/df/d60/d98/d22/d81/d8d/lda 0 2024-02-15T18:35:46.190 INFO:tasks.workunit.client.1.smithi121.stdout:2/724: chown d5/df/d60/d98/d22/d81/d8d/lca 177765 1 2024-02-15T18:35:46.194 INFO:tasks.workunit.client.1.smithi121.stdout:3/768: mkdir d2/d1a/d27/d63/da4/d25/d88/d9a/d10c 0 2024-02-15T18:35:46.194 INFO:tasks.workunit.client.1.smithi121.stdout:8/810: creat d0/d3/dc9/f116 x:0 0 0 2024-02-15T18:35:46.194 INFO:tasks.workunit.client.1.smithi121.stdout:2/725: mkdir d5/df/d10/ddb 0 2024-02-15T18:35:46.223 INFO:tasks.workunit.client.1.smithi121.stdout:8/811: dwrite d0/d2a/fd5 [0,4194304] 0 2024-02-15T18:35:46.228 INFO:tasks.workunit.client.1.smithi121.stdout:2/726: dwrite d5/df/d60/d98/f1c [0,4194304] 0 2024-02-15T18:35:46.228 INFO:tasks.workunit.client.1.smithi121.stdout:2/727: fsync d5/d49/f84 0 2024-02-15T18:35:46.228 INFO:tasks.workunit.client.1.smithi121.stdout:2/728: chown d5/df/d60/d98/d22/dc8/lcd 37486 1 2024-02-15T18:35:46.230 INFO:tasks.workunit.client.1.smithi121.stdout:3/769: dwrite d2/d1a/d27/d63/da4/d25/d31/d58/fbd [0,4194304] 0 2024-02-15T18:35:46.230 INFO:tasks.workunit.client.1.smithi121.stdout:3/770: chown d2/d1a/d27/d63/da4/d25/d30/db3/cc7 22 1 2024-02-15T18:35:46.230 INFO:tasks.workunit.client.1.smithi121.stdout:2/729: rename d5/df/d60/d98/d22/d81/dac/dd4 to d5/d49/ddc 0 2024-02-15T18:35:46.230 INFO:tasks.workunit.client.1.smithi121.stdout:2/730: chown d5/df/d27/d3e/d6a/l6e 385 1 2024-02-15T18:35:46.232 INFO:tasks.workunit.client.1.smithi121.stdout:2/731: write d5/df/d10/fa9 [2734188,119024] 0 2024-02-15T18:35:46.232 INFO:tasks.workunit.client.1.smithi121.stdout:2/732: truncate d5/df/d60/d98/d58/f9c 415961 0 2024-02-15T18:35:46.232 INFO:tasks.workunit.client.1.smithi121.stdout:3/771: creat d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2/d10a/f10d x:0 0 0 2024-02-15T18:35:46.246 INFO:tasks.workunit.client.1.smithi121.stdout:2/733: dread d5/df/d10/fa9 [0,4194304] 0 2024-02-15T18:35:46.249 INFO:tasks.workunit.client.1.smithi121.stdout:3/772: dwrite d2/d5a/f67 [4194304,4194304] 0 2024-02-15T18:35:46.249 INFO:tasks.workunit.client.1.smithi121.stdout:3/773: chown d2/d1a/d54/l94 3820 1 2024-02-15T18:35:46.250 INFO:tasks.workunit.client.1.smithi121.stdout:3/774: read d2/d5a/f5f [171662,45799] 0 2024-02-15T18:35:46.251 INFO:tasks.workunit.client.1.smithi121.stdout:3/775: creat d2/f10e x:0 0 0 2024-02-15T18:35:46.253 INFO:tasks.workunit.client.1.smithi121.stdout:3/776: dread d2/d1a/d27/da7/fc3 [0,4194304] 0 2024-02-15T18:35:46.268 INFO:tasks.workunit.client.1.smithi121.stdout:3/777: dwrite d2/d1a/d6b/d7d/fa1 [0,4194304] 0 2024-02-15T18:35:46.269 INFO:tasks.workunit.client.1.smithi121.stdout:3/778: getdents d2/d1a/d27/d63 0 2024-02-15T18:35:46.270 INFO:tasks.workunit.client.1.smithi121.stdout:3/779: fdatasync d2/d1a/d27/d63/da4/d25/fb0 0 2024-02-15T18:35:46.273 INFO:tasks.workunit.client.1.smithi121.stdout:3/780: truncate d2/d1a/d27/d63/da4/d25/d31/d58/fbd 1007329 0 2024-02-15T18:35:46.273 INFO:tasks.workunit.client.1.smithi121.stdout:3/781: truncate d2/d1a/d27/d63/da4/fe4 1295320 0 2024-02-15T18:35:46.289 INFO:tasks.workunit.client.1.smithi121.stdout:3/782: dwrite d2/d1a/d6b/d7d/f8e [0,4194304] 0 2024-02-15T18:35:46.291 INFO:tasks.workunit.client.1.smithi121.stdout:3/783: rename d2/d1a/d27/d63/da4/d25/d31/c3d to d2/d1a/d6b/d72/c10f 0 2024-02-15T18:35:46.292 INFO:tasks.workunit.client.1.smithi121.stdout:3/784: mkdir d2/d1a/d27/d63/d110 0 2024-02-15T18:35:46.292 INFO:tasks.workunit.client.1.smithi121.stdout:3/785: chown d2/d1a/d27/d63/da4/d25/d30/f5b 28715006 1 2024-02-15T18:35:46.293 INFO:tasks.workunit.client.1.smithi121.stdout:3/786: write d2/d1a/d27/d63/da4/f19 [3401028,78553] 0 2024-02-15T18:35:46.293 INFO:tasks.workunit.client.1.smithi121.stdout:3/787: creat d2/d1a/d27/da7/d85/d9f/d107/f111 x:0 0 0 2024-02-15T18:35:46.315 INFO:tasks.workunit.client.1.smithi121.stdout:4/852: rmdir d5/d31/d3d/d4f/d51/d80 39 2024-02-15T18:35:46.318 INFO:tasks.workunit.client.1.smithi121.stdout:3/788: dwrite d2/d1a/d27/d63/da4/d25/d31/f73 [0,4194304] 0 2024-02-15T18:35:46.322 INFO:tasks.workunit.client.1.smithi121.stdout:3/789: link d2/d1a/d54/l100 d2/d1a/d54/dc1/l112 0 2024-02-15T18:35:46.326 INFO:tasks.workunit.client.1.smithi121.stdout:4/853: dread d5/d9/fdb [0,4194304] 0 2024-02-15T18:35:46.334 INFO:tasks.workunit.client.1.smithi121.stdout:4/854: dread d5/d9/d1d/d26/d73/d81/ff1 [0,4194304] 0 2024-02-15T18:35:46.341 INFO:tasks.workunit.client.1.smithi121.stdout:3/790: dwrite d2/d1a/f1f [0,4194304] 0 2024-02-15T18:35:46.343 INFO:tasks.workunit.client.1.smithi121.stdout:4/855: dread d5/d9/d19/d28/fb3 [0,4194304] 0 2024-02-15T18:35:46.348 INFO:tasks.workunit.client.1.smithi121.stdout:4/856: rmdir d5/d9/d1d/d26/d9d 0 2024-02-15T18:35:46.350 INFO:tasks.workunit.client.1.smithi121.stdout:4/857: creat d5/d9/d19/d2a/d4d/f10a x:0 0 0 2024-02-15T18:35:46.352 INFO:tasks.workunit.client.1.smithi121.stdout:3/791: dread d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2/d10a/fa0 [0,4194304] 0 2024-02-15T18:35:46.353 INFO:tasks.workunit.client.1.smithi121.stdout:3/792: write d2/d1a/d27/da7/f35 [4079266,54907] 0 2024-02-15T18:35:46.353 INFO:tasks.workunit.client.1.smithi121.stdout:3/793: chown d2/d1a/d27/d63/da4/d25/d31/d58/f6f 728800 1 2024-02-15T18:35:46.353 INFO:tasks.workunit.client.1.smithi121.stdout:4/858: dread d5/d9/fdb [0,4194304] 0 2024-02-15T18:35:46.354 INFO:tasks.workunit.client.1.smithi121.stdout:4/859: fdatasync d5/d31/d25/db6/fe9 0 2024-02-15T18:35:46.354 INFO:tasks.workunit.client.1.smithi121.stdout:4/860: write d5/d9/f20 [1328281,52452] 0 2024-02-15T18:35:46.354 INFO:tasks.workunit.client.1.smithi121.stdout:3/794: symlink d2/d1a/d27/d63/da4/d25/d88/d96/l113 0 2024-02-15T18:35:46.354 INFO:tasks.workunit.client.1.smithi121.stdout:3/795: dread - d2/d1a/d27/d63/da4/d25/d31/d34/fdb zero size 2024-02-15T18:35:46.354 INFO:tasks.workunit.client.1.smithi121.stdout:4/861: write d5/d9/d19/d2a/f68 [2614716,45907] 0 2024-02-15T18:35:46.355 INFO:tasks.workunit.client.1.smithi121.stdout:3/796: write d2/d1a/d27/d63/d81/fd5 [1914341,80881] 0 2024-02-15T18:35:46.355 INFO:tasks.workunit.client.1.smithi121.stdout:3/797: stat d2/d1a/d6b/c104 0 2024-02-15T18:35:46.356 INFO:tasks.workunit.client.1.smithi121.stdout:4/862: mknod d5/d31/d3d/d4f/d51/d80/c10b 0 2024-02-15T18:35:46.359 INFO:tasks.workunit.client.1.smithi121.stdout:4/863: write d5/d9/d1d/f2f [128391,115947] 0 2024-02-15T18:35:46.359 INFO:tasks.workunit.client.1.smithi121.stdout:4/864: write d5/d9/d19/d2a/d4d/f10a [47714,42449] 0 2024-02-15T18:35:46.418 INFO:tasks.workunit.client.1.smithi121.stdout:7/632: sync 2024-02-15T18:35:46.419 INFO:tasks.workunit.client.1.smithi121.stdout:7/633: mknod d0/d5/d11/d5c/dcc/dc6/cd1 0 2024-02-15T18:35:46.420 INFO:tasks.workunit.client.1.smithi121.stdout:7/634: symlink d0/d5/d4c/d6f/ld2 0 2024-02-15T18:35:46.428 INFO:tasks.workunit.client.1.smithi121.stdout:7/635: dread d0/d5/d11/d5c/dcc/f4b [0,4194304] 0 2024-02-15T18:35:46.446 INFO:tasks.workunit.client.1.smithi121.stdout:7/636: dwrite d0/d5/d11/d5c/fa7 [0,4194304] 0 2024-02-15T18:35:46.446 INFO:tasks.workunit.client.1.smithi121.stdout:7/637: truncate d0/d4a/d5b/d9a/f9d 384690 0 2024-02-15T18:35:46.589 INFO:tasks.workunit.client.1.smithi121.stdout:6/941: sync 2024-02-15T18:35:46.590 INFO:tasks.workunit.client.1.smithi121.stdout:6/942: dread - d7/dc/d34/d42/d62/fd9 zero size 2024-02-15T18:35:46.590 INFO:tasks.workunit.client.1.smithi121.stdout:6/943: write d7/dc/d34/d42/d62/d63/d6f/d89/f10e [1849848,103123] 0 2024-02-15T18:35:46.591 INFO:tasks.workunit.client.1.smithi121.stdout:6/944: getdents d7/dc/d34/d42/d62/d63/d6f/d89 0 2024-02-15T18:35:46.593 INFO:tasks.workunit.client.1.smithi121.stdout:6/945: creat d7/dc/d34/d42/d62/d63/d6f/d89/f14e x:0 0 0 2024-02-15T18:35:46.593 INFO:tasks.workunit.client.1.smithi121.stdout:6/946: truncate d7/dc/d34/d42/d62/d63/d66/dce/f12b 302567 0 2024-02-15T18:35:46.602 INFO:tasks.workunit.client.1.smithi121.stdout:6/947: dread d7/f18 [0,4194304] 0 2024-02-15T18:35:46.624 INFO:tasks.workunit.client.1.smithi121.stdout:6/948: dwrite d7/f8 [4194304,4194304] 0 2024-02-15T18:35:46.629 INFO:tasks.workunit.client.1.smithi121.stdout:6/949: unlink d7/dc/d34/d42/d62/d63/d6f/f101 0 2024-02-15T18:35:46.629 INFO:tasks.workunit.client.1.smithi121.stdout:6/950: dread - d7/d4b/d60/ff7 zero size 2024-02-15T18:35:46.631 INFO:tasks.workunit.client.1.smithi121.stdout:6/951: creat d7/dc/d76/d103/f14f x:0 0 0 2024-02-15T18:35:46.634 INFO:tasks.workunit.client.1.smithi121.stdout:6/952: unlink d7/d4b/d60/d6c/dd5/dfa/d94/ld4 0 2024-02-15T18:35:46.635 INFO:tasks.workunit.client.1.smithi121.stdout:6/953: rmdir d7/dc/d55/df8 39 2024-02-15T18:35:46.636 INFO:tasks.workunit.client.1.smithi121.stdout:6/954: truncate d7/d4b/d59/fec 92201 0 2024-02-15T18:35:46.646 INFO:tasks.workunit.client.1.smithi121.stdout:6/955: dread d7/dc/d34/d42/d62/d63/d6f/f88 [4194304,4194304] 0 2024-02-15T18:35:46.646 INFO:tasks.workunit.client.1.smithi121.stdout:6/956: fsync d7/dc/f5e 0 2024-02-15T18:35:46.655 INFO:tasks.workunit.client.1.smithi121.stdout:6/957: dread d7/f31 [0,4194304] 0 2024-02-15T18:35:46.658 INFO:tasks.workunit.client.1.smithi121.stdout:6/958: mknod d7/dc/d34/d42/d62/d63/d6f/d89/da1/c150 0 2024-02-15T18:35:46.658 INFO:tasks.workunit.client.1.smithi121.stdout:6/959: truncate d7/dc/d34/d42/d62/dac/f120 727597 0 2024-02-15T18:35:46.658 INFO:tasks.workunit.client.1.smithi121.stdout:6/960: write d7/d4b/d60/ff7 [19413,5743] 0 2024-02-15T18:35:46.659 INFO:tasks.workunit.client.1.smithi121.stdout:6/961: write d7/dc/d34/d42/d62/d63/d6f/d89/f10e [5196533,27014] 0 2024-02-15T18:35:46.690 INFO:tasks.workunit.client.1.smithi121.stdout:6/962: dwrite d7/dc/d34/d42/d62/d63/d6f/d89/f10e [0,4194304] 0 2024-02-15T18:35:46.691 INFO:tasks.workunit.client.1.smithi121.stdout:6/963: write d7/dc/d34/d42/d62/f6b [4167426,126699] 0 2024-02-15T18:35:46.694 INFO:tasks.workunit.client.1.smithi121.stdout:6/964: dread d7/dc/d2c/d36/f56 [0,4194304] 0 2024-02-15T18:35:46.695 INFO:tasks.workunit.client.1.smithi121.stdout:6/965: write d7/dc/d55/fe1 [3276574,19151] 0 2024-02-15T18:35:46.697 INFO:tasks.workunit.client.1.smithi121.stdout:6/966: mknod d7/dc/d76/c151 0 2024-02-15T18:35:46.700 INFO:tasks.workunit.client.1.smithi121.stdout:6/967: unlink d7/dc/d34/f4a 0 2024-02-15T18:35:46.701 INFO:tasks.workunit.client.1.smithi121.stdout:6/968: chown d7/dc/d34/d42/d62/d63/d6f/d89/df1/cf5 94602 1 2024-02-15T18:35:46.701 INFO:tasks.workunit.client.1.smithi121.stdout:6/969: chown d7/d4b/d60/d6c/f8a 7 1 2024-02-15T18:35:46.701 INFO:tasks.workunit.client.1.smithi121.stdout:6/970: chown d7/dc/f5e 14640 1 2024-02-15T18:35:46.705 INFO:tasks.workunit.client.1.smithi121.stdout:6/971: link d7/dc/d34/d42/d62/d63/d6f/l75 d7/d4b/d60/l152 0 2024-02-15T18:35:46.711 INFO:tasks.workunit.client.1.smithi121.stdout:6/972: mkdir d7/dc/d34/d42/d62/d63/d6f/d153 0 2024-02-15T18:35:46.712 INFO:tasks.workunit.client.1.smithi121.stdout:4/865: sync 2024-02-15T18:35:46.712 INFO:tasks.workunit.client.1.smithi121.stdout:3/798: sync 2024-02-15T18:35:46.712 INFO:tasks.workunit.client.1.smithi121.stdout:2/734: sync 2024-02-15T18:35:46.712 INFO:tasks.workunit.client.1.smithi121.stdout:7/638: sync 2024-02-15T18:35:46.712 INFO:tasks.workunit.client.1.smithi121.stdout:8/812: sync 2024-02-15T18:35:46.712 INFO:tasks.workunit.client.1.smithi121.stdout:2/735: fsync d5/fb5 0 2024-02-15T18:35:46.714 INFO:tasks.workunit.client.1.smithi121.stdout:3/799: mknod d2/d1a/d27/d63/da4/d25/d31/d97/c114 0 2024-02-15T18:35:46.717 INFO:tasks.workunit.client.1.smithi121.stdout:6/973: dread d7/f18 [0,4194304] 0 2024-02-15T18:35:46.717 INFO:tasks.workunit.client.1.smithi121.stdout:6/974: readlink d7/d4b/d60/d6c/dd5/lf4 0 2024-02-15T18:35:46.718 INFO:tasks.workunit.client.1.smithi121.stdout:2/736: creat d5/df/d10/d4c/dad/fdd x:0 0 0 2024-02-15T18:35:46.718 INFO:tasks.workunit.client.1.smithi121.stdout:2/737: stat d5/df/d54/f92 0 2024-02-15T18:35:46.718 INFO:tasks.workunit.client.1.smithi121.stdout:3/800: getdents d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2 0 2024-02-15T18:35:46.718 INFO:tasks.workunit.client.1.smithi121.stdout:2/738: write d5/df/d60/d98/d22/d81/d9a/da1/fcc [454944,125315] 0 2024-02-15T18:35:46.721 INFO:tasks.workunit.client.1.smithi121.stdout:6/975: mkdir d7/d4b/d129/d154 0 2024-02-15T18:35:46.721 INFO:tasks.workunit.client.1.smithi121.stdout:6/976: chown d7/d4b 167543700 1 2024-02-15T18:35:46.721 INFO:tasks.workunit.client.1.smithi121.stdout:3/801: symlink d2/d1a/d27/d63/l115 0 2024-02-15T18:35:46.724 INFO:tasks.workunit.client.1.smithi121.stdout:3/802: creat d2/d1a/d27/d63/da4/d25/d30/db3/ddf/f116 x:0 0 0 2024-02-15T18:35:46.724 INFO:tasks.workunit.client.1.smithi121.stdout:8/813: dread d0/d2a/d51/f57 [0,4194304] 0 2024-02-15T18:35:46.724 INFO:tasks.workunit.client.1.smithi121.stdout:8/814: write d0/d61/d95/fd1 [626312,120726] 0 2024-02-15T18:35:46.724 INFO:tasks.workunit.client.1.smithi121.stdout:7/639: dread d0/d5/f26 [4194304,4194304] 0 2024-02-15T18:35:46.725 INFO:tasks.workunit.client.1.smithi121.stdout:7/640: readlink d0/l49 0 2024-02-15T18:35:46.725 INFO:tasks.workunit.client.1.smithi121.stdout:7/641: write d0/d5/f26 [9036804,103574] 0 2024-02-15T18:35:46.726 INFO:tasks.workunit.client.1.smithi121.stdout:8/815: stat d0/d7/c69 0 2024-02-15T18:35:46.726 INFO:tasks.workunit.client.1.smithi121.stdout:8/816: write d0/d7/d36/fb9 [1861509,14595] 0 2024-02-15T18:35:46.726 INFO:tasks.workunit.client.1.smithi121.stdout:8/817: dread - d0/d7/d36/d8c/de9/fe7 zero size 2024-02-15T18:35:46.728 INFO:tasks.workunit.client.1.smithi121.stdout:8/818: mkdir d0/d7/db1/d117 0 2024-02-15T18:35:46.730 INFO:tasks.workunit.client.1.smithi121.stdout:3/803: read d2/d1a/d27/d63/da4/f2f [306773,53682] 0 2024-02-15T18:35:46.730 INFO:tasks.workunit.client.1.smithi121.stdout:6/977: dread d7/dc/d34/d42/d62/fdb [0,4194304] 0 2024-02-15T18:35:46.732 INFO:tasks.workunit.client.1.smithi121.stdout:7/642: write d0/f52 [2164949,27432] 0 2024-02-15T18:35:46.732 INFO:tasks.workunit.client.1.smithi121.stdout:7/643: chown d0/d5/d76 398028 1 2024-02-15T18:35:46.734 INFO:tasks.workunit.client.1.smithi121.stdout:8/819: write d0/d61/d95/fd1 [620443,40622] 0 2024-02-15T18:35:46.734 INFO:tasks.workunit.client.1.smithi121.stdout:6/978: mkdir d7/dc/d34/d42/d62/d63/d6f/db9/d155 0 2024-02-15T18:35:46.736 INFO:tasks.workunit.client.1.smithi121.stdout:7/644: creat d0/d5/d11/d1c/fd3 x:0 0 0 2024-02-15T18:35:46.737 INFO:tasks.workunit.client.1.smithi121.stdout:6/979: mknod d7/dc/d34/d42/d62/d63/c156 0 2024-02-15T18:35:46.738 INFO:tasks.workunit.client.1.smithi121.stdout:7/645: creat d0/d4a/fd4 x:0 0 0 2024-02-15T18:35:46.739 INFO:tasks.workunit.client.1.smithi121.stdout:2/739: dwrite d5/d49/d71/fba [0,4194304] 0 2024-02-15T18:35:46.741 INFO:tasks.workunit.client.1.smithi121.stdout:4/866: mkdir d5/d9/d1d/d10c 0 2024-02-15T18:35:46.742 INFO:tasks.workunit.client.1.smithi121.stdout:6/980: creat d7/dc/d34/d42/d62/dac/d10a/d145/f157 x:0 0 0 2024-02-15T18:35:46.742 INFO:tasks.workunit.client.1.smithi121.stdout:6/981: fsync d7/dc/f132 0 2024-02-15T18:35:46.742 INFO:tasks.workunit.client.1.smithi121.stdout:6/982: chown d7/dc/d34/d7c/c126 97853 1 2024-02-15T18:35:46.743 INFO:tasks.workunit.client.1.smithi121.stdout:7/646: mkdir d0/d5/d4c/d6f/dd5 0 2024-02-15T18:35:46.746 INFO:tasks.workunit.client.1.smithi121.stdout:2/740: truncate d5/df/f2b 743080 0 2024-02-15T18:35:46.748 INFO:tasks.workunit.client.1.smithi121.stdout:7/647: mkdir d0/d38/d46/daa/dd6 0 2024-02-15T18:35:46.749 INFO:tasks.workunit.client.1.smithi121.stdout:2/741: symlink d5/df/d27/d3a/lde 0 2024-02-15T18:35:46.749 INFO:tasks.workunit.client.1.smithi121.stdout:2/742: fsync d5/df/d60/d98/d58/f9c 0 2024-02-15T18:35:46.752 INFO:tasks.workunit.client.1.smithi121.stdout:7/648: read d0/d38/d46/daa/f5d [91294,130328] 0 2024-02-15T18:35:46.756 INFO:tasks.workunit.client.1.smithi121.stdout:7/649: rename d0/d5/d11/d5c/fa3 to d0/d38/d61/fd7 0 2024-02-15T18:35:46.756 INFO:tasks.workunit.client.1.smithi121.stdout:7/650: truncate d0/d5/d11/d1c/d5f/fc5 266843 0 2024-02-15T18:35:46.757 INFO:tasks.workunit.client.1.smithi121.stdout:3/804: dwrite d2/d1a/d6b/f91 [0,4194304] 0 2024-02-15T18:35:46.757 INFO:tasks.workunit.client.1.smithi121.stdout:7/651: write d0/d5/f26 [6038286,110292] 0 2024-02-15T18:35:46.758 INFO:tasks.workunit.client.1.smithi121.stdout:3/805: mkdir d2/d1a/d54/dc1/d117 0 2024-02-15T18:35:46.759 INFO:tasks.workunit.client.1.smithi121.stdout:3/806: read - d2/d1a/d27/d63/da4/fd8 zero size 2024-02-15T18:35:46.764 INFO:tasks.workunit.client.1.smithi121.stdout:4/867: dwrite d5/d31/d25/db6/fe9 [0,4194304] 0 2024-02-15T18:35:46.766 INFO:tasks.workunit.client.1.smithi121.stdout:7/652: dread d0/d38/d3f/d43/f8f [0,4194304] 0 2024-02-15T18:35:46.766 INFO:tasks.workunit.client.1.smithi121.stdout:4/868: creat d5/d9/d19/d4c/f10d x:0 0 0 2024-02-15T18:35:46.767 INFO:tasks.workunit.client.1.smithi121.stdout:7/653: mknod d0/d5/d11/cd8 0 2024-02-15T18:35:46.768 INFO:tasks.workunit.client.1.smithi121.stdout:4/869: creat d5/d9/d19/d2a/f10e x:0 0 0 2024-02-15T18:35:46.769 INFO:tasks.workunit.client.1.smithi121.stdout:4/870: readlink d5/d31/d3d/d4f/d51/d80/ld0 0 2024-02-15T18:35:46.769 INFO:tasks.workunit.client.1.smithi121.stdout:4/871: read - d5/d31/d25/fe4 zero size 2024-02-15T18:35:46.769 INFO:tasks.workunit.client.1.smithi121.stdout:4/872: stat d5/d9/d1d/d26/d73/l7b 0 2024-02-15T18:35:46.769 INFO:tasks.workunit.client.1.smithi121.stdout:4/873: chown d5/d31/d3d/d4f/d51 2217768 1 2024-02-15T18:35:46.769 INFO:tasks.workunit.client.1.smithi121.stdout:4/874: chown d5/d31/d3d/d4f/cc1 4158848 1 2024-02-15T18:35:46.770 INFO:tasks.workunit.client.1.smithi121.stdout:7/654: write d0/d5/d11/f40 [3186906,11144] 0 2024-02-15T18:35:46.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:46.597542+0000 mgr.smithi121.ckiryi (mgr.24385) 17 : cluster [DBG] pgmap v17: 65 pgs: 65 active+clean; 2.1 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 78 MiB/s rd, 172 MiB/s wr, 531 op/s 2024-02-15T18:35:46.777 INFO:tasks.workunit.client.1.smithi121.stdout:3/807: dread d2/d1a/d27/d63/da4/d25/d30/f39 [0,4194304] 0 2024-02-15T18:35:46.777 INFO:tasks.workunit.client.1.smithi121.stdout:3/808: fdatasync d2/d1a/d54/fe5 0 2024-02-15T18:35:46.777 INFO:tasks.workunit.client.1.smithi121.stdout:3/809: chown d2/d1a/d6b/c104 331537 1 2024-02-15T18:35:46.777 INFO:tasks.workunit.client.1.smithi121.stdout:3/810: stat d2/d1a/d27/fa2 0 2024-02-15T18:35:46.777 INFO:tasks.workunit.client.1.smithi121.stdout:8/820: dwrite d0/f37 [0,4194304] 0 2024-02-15T18:35:46.780 INFO:tasks.workunit.client.1.smithi121.stdout:8/821: creat d0/d7/d36/d52/f118 x:0 0 0 2024-02-15T18:35:46.782 INFO:tasks.workunit.client.1.smithi121.stdout:8/822: creat d0/d7/da5/dcb/d102/f119 x:0 0 0 2024-02-15T18:35:46.782 INFO:tasks.workunit.client.1.smithi121.stdout:8/823: fsync d0/d2a/d51/f6b 0 2024-02-15T18:35:46.783 INFO:tasks.workunit.client.1.smithi121.stdout:8/824: symlink d0/d7/da5/l11a 0 2024-02-15T18:35:46.792 INFO:tasks.workunit.client.1.smithi121.stdout:7/655: dwrite d0/f27 [0,4194304] 0 2024-02-15T18:35:46.795 INFO:tasks.workunit.client.1.smithi121.stdout:7/656: creat d0/d5/d11/d5c/dcc/dc6/fd9 x:0 0 0 2024-02-15T18:35:46.795 INFO:tasks.workunit.client.1.smithi121.stdout:7/657: write d0/d4a/d5b/da6/f7f [387530,3206] 0 2024-02-15T18:35:46.798 INFO:tasks.workunit.client.1.smithi121.stdout:8/825: dread d0/d7/d36/d8c/d45/fcc [0,4194304] 0 2024-02-15T18:35:46.798 INFO:tasks.workunit.client.1.smithi121.stdout:8/826: truncate d0/d2a/d51/f57 4478483 0 2024-02-15T18:35:46.798 INFO:tasks.workunit.client.1.smithi121.stdout:8/827: fsync d0/d7/d36/d8c/d45/fff 0 2024-02-15T18:35:46.798 INFO:tasks.workunit.client.1.smithi121.stdout:8/828: chown d0/dce/df8/f112 131168225 1 2024-02-15T18:35:46.801 INFO:tasks.workunit.client.1.smithi121.stdout:7/658: dread d0/f42 [0,4194304] 0 2024-02-15T18:35:46.801 INFO:tasks.workunit.client.1.smithi121.stdout:8/829: write d0/d7/d36/d8c/d45/f89 [1399326,40261] 0 2024-02-15T18:35:46.803 INFO:tasks.workunit.client.1.smithi121.stdout:8/830: symlink d0/d7/da5/dcb/d102/l11b 0 2024-02-15T18:35:46.804 INFO:tasks.workunit.client.1.smithi121.stdout:8/831: truncate d0/d7/d36/fef 1527268 0 2024-02-15T18:35:46.806 INFO:tasks.workunit.client.1.smithi121.stdout:8/832: symlink d0/d7/d36/d52/d104/l11c 0 2024-02-15T18:35:46.806 INFO:tasks.workunit.client.1.smithi121.stdout:8/833: stat d0/d61/d7b/dac 0 2024-02-15T18:35:46.807 INFO:tasks.workunit.client.1.smithi121.stdout:4/875: dwrite d5/d31/d3d/d4f/d51/ff2 [0,4194304] 0 2024-02-15T18:35:46.808 INFO:tasks.workunit.client.1.smithi121.stdout:3/811: dwrite d2/d1a/d27/d63/da4/d25/d31/f57 [0,4194304] 0 2024-02-15T18:35:46.808 INFO:tasks.workunit.client.1.smithi121.stdout:3/812: fsync d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2/d10a/f5c 0 2024-02-15T18:35:46.808 INFO:tasks.workunit.client.1.smithi121.stdout:8/834: rmdir d0/d3 39 2024-02-15T18:35:46.810 INFO:tasks.workunit.client.1.smithi121.stdout:4/876: dread d5/d31/d3d/d4f/d51/d71/fc6 [0,4194304] 0 2024-02-15T18:35:46.813 INFO:tasks.workunit.client.1.smithi121.stdout:7/659: dread d0/d5/d4c/d6f/fbe [0,4194304] 0 2024-02-15T18:35:46.813 INFO:tasks.workunit.client.1.smithi121.stdout:3/813: creat d2/d1a/d27/d63/da4/d25/d30/db3/dd3/f118 x:0 0 0 2024-02-15T18:35:46.814 INFO:tasks.workunit.client.1.smithi121.stdout:3/814: readlink d2/d1a/d27/d63/da4/d25/d30/l65 0 2024-02-15T18:35:46.814 INFO:tasks.workunit.client.1.smithi121.stdout:8/835: symlink d0/d7/da5/l11d 0 2024-02-15T18:35:46.816 INFO:tasks.workunit.client.1.smithi121.stdout:4/877: rename d5/d31/d3d to d5/d9/d1d/d26/d73/d81/d10f 0 2024-02-15T18:35:46.816 INFO:tasks.workunit.client.1.smithi121.stdout:4/878: stat d5/d9/d1d/d26/d73/d81/d10f/d4f 0 2024-02-15T18:35:46.817 INFO:tasks.workunit.client.1.smithi121.stdout:7/660: mkdir d0/d5/d4c/dda 0 2024-02-15T18:35:46.818 INFO:tasks.workunit.client.1.smithi121.stdout:4/879: mknod d5/d9/d19/c110 0 2024-02-15T18:35:46.818 INFO:tasks.workunit.client.1.smithi121.stdout:4/880: dread - d5/d9/d76/d9e/fed zero size 2024-02-15T18:35:46.824 INFO:tasks.workunit.client.1.smithi121.stdout:4/881: link d5/d9/d1d/d26/d72/ff4 d5/d9/d76/d87/f111 0 2024-02-15T18:35:46.827 INFO:tasks.workunit.client.1.smithi121.stdout:3/815: rename d2/d1a/d6b/d72/c93 to d2/d1a/d27/d63/da4/d25/d31/d34/c119 0 2024-02-15T18:35:46.828 INFO:tasks.workunit.client.1.smithi121.stdout:8/836: write d0/d7/d36/d8c/d45/d53/fa0 [291964,13452] 0 2024-02-15T18:35:46.829 INFO:tasks.workunit.client.1.smithi121.stdout:3/816: creat d2/d1a/d54/dc1/f11a x:0 0 0 2024-02-15T18:35:46.833 INFO:tasks.workunit.client.1.smithi121.stdout:7/661: link d0/d5/d4c/c62 d0/cdb 0 2024-02-15T18:35:46.833 INFO:tasks.workunit.client.1.smithi121.stdout:8/837: creat d0/d3/dc9/f11e x:0 0 0 2024-02-15T18:35:46.833 INFO:tasks.workunit.client.1.smithi121.stdout:3/817: truncate d2/d1a/d27/d63/d81/fd5 2052600 0 2024-02-15T18:35:46.833 INFO:tasks.workunit.client.1.smithi121.stdout:3/818: fsync d2/d1a/d27/d63/da4/d25/d31/d34/fdb 0 2024-02-15T18:35:46.834 INFO:tasks.workunit.client.1.smithi121.stdout:4/882: dread d5/d9/d76/d9e/f3c [4194304,4194304] 0 2024-02-15T18:35:46.837 INFO:tasks.workunit.client.1.smithi121.stdout:7/662: read d0/d5/d11/d1c/fb2 [3091026,93632] 0 2024-02-15T18:35:46.837 INFO:tasks.workunit.client.1.smithi121.stdout:8/838: rename d0/f2 to d0/dce/f11f 0 2024-02-15T18:35:46.840 INFO:tasks.workunit.client.1.smithi121.stdout:3/819: unlink d2/d1a/d27/da7/fc3 0 2024-02-15T18:35:46.842 INFO:tasks.workunit.client.1.smithi121.stdout:8/839: dread d0/d7/d36/d52/fc7 [0,4194304] 0 2024-02-15T18:35:46.844 INFO:tasks.workunit.client.1.smithi121.stdout:7/663: dread d0/d5/d11/d1c/f35 [0,4194304] 0 2024-02-15T18:35:46.846 INFO:tasks.workunit.client.1.smithi121.stdout:4/883: link d5/d9/d19/d4c/f10d d5/f112 0 2024-02-15T18:35:46.854 INFO:tasks.workunit.client.1.smithi121.stdout:8/840: dread d0/d7/d36/d52/fb4 [0,4194304] 0 2024-02-15T18:35:46.865 INFO:tasks.workunit.client.1.smithi121.stdout:4/884: dread d5/d9/d19/f4e [0,4194304] 0 2024-02-15T18:35:46.867 INFO:tasks.workunit.client.1.smithi121.stdout:4/885: rename d5/d9/d1d/d26/d73/d81/d10f/d9b to d5/d9/d19/d113 0 2024-02-15T18:35:46.869 INFO:tasks.workunit.client.1.smithi121.stdout:4/886: creat d5/d31/f114 x:0 0 0 2024-02-15T18:35:46.869 INFO:tasks.workunit.client.1.smithi121.stdout:4/887: truncate d5/d9/d76/ff3 613954 0 2024-02-15T18:35:46.869 INFO:tasks.workunit.client.1.smithi121.stdout:4/888: write d5/d31/f114 [501744,21422] 0 2024-02-15T18:35:46.875 INFO:tasks.workunit.client.1.smithi121.stdout:7/664: dwrite d0/f42 [0,4194304] 0 2024-02-15T18:35:46.879 INFO:tasks.workunit.client.1.smithi121.stdout:3/820: dwrite d2/d1a/d27/fa2 [0,4194304] 0 2024-02-15T18:35:46.882 INFO:tasks.workunit.client.1.smithi121.stdout:3/821: dread d2/d5a/f5f [0,4194304] 0 2024-02-15T18:35:46.885 INFO:tasks.workunit.client.1.smithi121.stdout:3/822: creat d2/d1a/d27/da7/d85/f11b x:0 0 0 2024-02-15T18:35:46.893 INFO:tasks.workunit.client.1.smithi121.stdout:8/841: dwrite d0/d7/de/d67/fbe [0,4194304] 0 2024-02-15T18:35:46.894 INFO:tasks.workunit.client.1.smithi121.stdout:3/823: write d2/d1a/d27/d63/da4/d25/d30/f41 [1297553,21460] 0 2024-02-15T18:35:46.894 INFO:tasks.workunit.client.1.smithi121.stdout:3/824: truncate d2/d1a/d6b/fa5 1128102 0 2024-02-15T18:35:46.894 INFO:tasks.workunit.client.1.smithi121.stdout:3/825: truncate d2/d1a/d27/da7/d85/d9f/d107/f111 446251 0 2024-02-15T18:35:46.896 INFO:tasks.workunit.client.1.smithi121.stdout:3/826: creat d2/d1a/d27/d63/da4/d25/d30/db3/f11c x:0 0 0 2024-02-15T18:35:46.899 INFO:tasks.workunit.client.1.smithi121.stdout:7/665: dwrite d0/d4a/d5b/da6/fbf [0,4194304] 0 2024-02-15T18:35:46.905 INFO:tasks.workunit.client.1.smithi121.stdout:8/842: write d0/d7/d36/d52/d7a/f9a [3153851,18194] 0 2024-02-15T18:35:46.907 INFO:tasks.workunit.client.1.smithi121.stdout:8/843: getdents d0/d3/dc9 0 2024-02-15T18:35:46.907 INFO:tasks.workunit.client.1.smithi121.stdout:8/844: fsync d0/f8 0 2024-02-15T18:35:46.913 INFO:tasks.workunit.client.1.smithi121.stdout:7/666: dread d0/d38/fa0 [0,4194304] 0 2024-02-15T18:35:46.914 INFO:tasks.workunit.client.1.smithi121.stdout:7/667: write d0/d38/d46/f96 [849404,31816] 0 2024-02-15T18:35:46.922 INFO:tasks.workunit.client.1.smithi121.stdout:3/827: dwrite d2/d1a/d6b/f82 [0,4194304] 0 2024-02-15T18:35:46.922 INFO:tasks.workunit.client.1.smithi121.stdout:7/668: unlink d0/d5/d11/d5c/f64 0 2024-02-15T18:35:46.922 INFO:tasks.workunit.client.1.smithi121.stdout:8/845: dread d0/d3/f64 [0,4194304] 0 2024-02-15T18:35:46.922 INFO:tasks.workunit.client.1.smithi121.stdout:8/846: chown d0/f8 209 1 2024-02-15T18:35:46.924 INFO:tasks.workunit.client.1.smithi121.stdout:3/828: symlink d2/d1a/d27/d63/da4/d25/d88/d9a/l11d 0 2024-02-15T18:35:46.925 INFO:tasks.workunit.client.1.smithi121.stdout:7/669: rmdir d0/d38/d46/daa 39 2024-02-15T18:35:46.926 INFO:tasks.workunit.client.1.smithi121.stdout:7/670: chown d0/d4a/cb0 1960232 1 2024-02-15T18:35:46.927 INFO:tasks.workunit.client.1.smithi121.stdout:8/847: creat d0/d7/da5/dcb/d102/f120 x:0 0 0 2024-02-15T18:35:46.957 INFO:tasks.workunit.client.1.smithi121.stdout:3/829: dwrite d2/d1a/d27/da7/f42 [0,4194304] 0 2024-02-15T18:35:46.958 INFO:tasks.workunit.client.1.smithi121.stdout:3/830: truncate d2/d1a/d27/f77 1882138 0 2024-02-15T18:35:46.959 INFO:tasks.workunit.client.1.smithi121.stdout:8/848: dwrite d0/d7/da5/dcb/f9f [0,4194304] 0 2024-02-15T18:35:46.961 INFO:tasks.workunit.client.1.smithi121.stdout:3/831: rename d2/d1a/d27/d63/da4/d25/d88/d96/l98 to d2/d1a/d54/dc1/d117/l11e 0 2024-02-15T18:35:46.961 INFO:tasks.workunit.client.1.smithi121.stdout:3/832: stat d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2/d10a/f10d 0 2024-02-15T18:35:46.964 INFO:tasks.workunit.client.1.smithi121.stdout:7/671: dwrite d0/d5/f47 [0,4194304] 0 2024-02-15T18:35:46.964 INFO:tasks.workunit.client.1.smithi121.stdout:3/833: rmdir d2/d1a/d27/d63/da4/d25/d31 39 2024-02-15T18:35:46.964 INFO:tasks.workunit.client.1.smithi121.stdout:3/834: fsync d2/d1a/d27/da7/f47 0 2024-02-15T18:35:46.964 INFO:tasks.workunit.client.1.smithi121.stdout:3/835: fdatasync d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2/ffd 0 2024-02-15T18:35:46.966 INFO:tasks.workunit.client.1.smithi121.stdout:7/672: readlink d0/l37 0 2024-02-15T18:35:46.967 INFO:tasks.workunit.client.1.smithi121.stdout:3/836: getdents d2/d1a/d27/d63/da4 0 2024-02-15T18:35:46.971 INFO:tasks.workunit.client.1.smithi121.stdout:7/673: rename d0/d38/d61/f78 to d0/d4a/d5b/fdc 0 2024-02-15T18:35:46.972 INFO:tasks.workunit.client.1.smithi121.stdout:3/837: mknod d2/d1a/d54/dc1/d117/c11f 0 2024-02-15T18:35:46.978 INFO:tasks.workunit.client.1.smithi121.stdout:3/838: mkdir d2/d1a/d27/da7/d85/d9f/d120 0 2024-02-15T18:35:46.997 INFO:tasks.workunit.client.1.smithi121.stdout:7/674: dwrite d0/d5/d11/d1c/f3e [4194304,4194304] 0 2024-02-15T18:35:46.997 INFO:tasks.workunit.client.1.smithi121.stdout:7/675: write d0/d4a/f53 [482190,55880] 0 2024-02-15T18:35:46.997 INFO:tasks.workunit.client.1.smithi121.stdout:7/676: readlink d0/d5/d11/d1c/d5f/lac 0 2024-02-15T18:35:46.999 INFO:tasks.workunit.client.1.smithi121.stdout:7/677: link d0/d5/d11/d79/f86 d0/d5/d11/d5c/fdd 0 2024-02-15T18:35:47.001 INFO:tasks.workunit.client.1.smithi121.stdout:7/678: creat d0/d38/fde x:0 0 0 2024-02-15T18:35:47.002 INFO:tasks.workunit.client.1.smithi121.stdout:7/679: creat d0/d5/d11/d1c/d5f/dc0/fdf x:0 0 0 2024-02-15T18:35:47.002 INFO:tasks.workunit.client.1.smithi121.stdout:7/680: dread - d0/d5/d11/d1c/fd3 zero size 2024-02-15T18:35:47.002 INFO:tasks.workunit.client.1.smithi121.stdout:7/681: symlink d0/d5/d4c/d6f/db3/le0 0 2024-02-15T18:35:47.003 INFO:tasks.workunit.client.1.smithi121.stdout:7/682: creat d0/d5/d11/d1c/d5f/dc0/fe1 x:0 0 0 2024-02-15T18:35:47.006 INFO:tasks.workunit.client.1.smithi121.stdout:7/683: rename d0/c8d to d0/d38/ce2 0 2024-02-15T18:35:47.006 INFO:tasks.workunit.client.1.smithi121.stdout:7/684: readlink d0/d5/l8 0 2024-02-15T18:35:47.006 INFO:tasks.workunit.client.1.smithi121.stdout:3/839: dwrite d2/d1a/d27/d63/da4/d25/d30/f46 [4194304,4194304] 0 2024-02-15T18:35:47.008 INFO:tasks.workunit.client.1.smithi121.stdout:7/685: getdents d0/d4a 0 2024-02-15T18:35:47.009 INFO:tasks.workunit.client.1.smithi121.stdout:3/840: mkdir d2/d1a/d27/d63/da4/d25/d31/d34/d121 0 2024-02-15T18:35:47.011 INFO:tasks.workunit.client.1.smithi121.stdout:3/841: rename d2/d1a/cdc to d2/d1a/d27/d63/da4/d25/d30/c122 0 2024-02-15T18:35:47.013 INFO:tasks.workunit.client.1.smithi121.stdout:3/842: mkdir d2/d1a/d27/d63/da4/d25/d31/d97/daf/d123 0 2024-02-15T18:35:47.015 INFO:tasks.workunit.client.1.smithi121.stdout:3/843: truncate d2/d1a/d27/d63/da4/f7b 3090594 0 2024-02-15T18:35:47.016 INFO:tasks.workunit.client.1.smithi121.stdout:3/844: stat d2/d1a/d54/dc1/d117/l11e 0 2024-02-15T18:35:47.017 INFO:tasks.workunit.client.1.smithi121.stdout:3/845: symlink d2/d1a/d27/da7/d85/d9f/ddd/l124 0 2024-02-15T18:35:47.017 INFO:tasks.workunit.client.1.smithi121.stdout:3/846: dread - d2/d1a/d27/d63/da4/d25/d31/d58/fb4 zero size 2024-02-15T18:35:47.021 INFO:tasks.workunit.client.1.smithi121.stdout:7/686: dread d0/d38/d46/daa/f34 [0,4194304] 0 2024-02-15T18:35:47.022 INFO:tasks.workunit.client.1.smithi121.stdout:7/687: stat d0/f42 0 2024-02-15T18:35:47.043 INFO:tasks.workunit.client.1.smithi121.stdout:3/847: dwrite d2/d1a/d27/f7f [0,4194304] 0 2024-02-15T18:35:47.046 INFO:tasks.workunit.client.1.smithi121.stdout:7/688: dwrite d0/d5/d76/fb8 [0,4194304] 0 2024-02-15T18:35:47.054 INFO:tasks.workunit.client.1.smithi121.stdout:7/689: fdatasync d0/f18 0 2024-02-15T18:35:47.055 INFO:tasks.workunit.client.1.smithi121.stdout:7/690: dread d0/d4a/d5b/d9a/f9d [0,4194304] 0 2024-02-15T18:35:47.055 INFO:tasks.workunit.client.1.smithi121.stdout:7/691: truncate d0/d4a/da2/fb5 1001356 0 2024-02-15T18:35:47.058 INFO:tasks.workunit.client.1.smithi121.stdout:7/692: mknod d0/d4a/da2/ce3 0 2024-02-15T18:35:47.067 INFO:tasks.workunit.client.1.smithi121.stdout:3/848: dwrite d2/d1a/d27/d63/fcc [0,4194304] 0 2024-02-15T18:35:47.070 INFO:tasks.workunit.client.1.smithi121.stdout:3/849: truncate d2/d1a/fd4 1841026 0 2024-02-15T18:35:47.070 INFO:tasks.workunit.client.1.smithi121.stdout:3/850: write d2/d1a/d27/d63/da4/fd8 [359200,16596] 0 2024-02-15T18:35:47.072 INFO:tasks.workunit.client.1.smithi121.stdout:3/851: creat d2/d1a/d6b/f125 x:0 0 0 2024-02-15T18:35:47.074 INFO:tasks.workunit.client.1.smithi121.stdout:3/852: mknod d2/d1a/d27/d63/da4/d25/d31/d34/d121/c126 0 2024-02-15T18:35:47.076 INFO:tasks.workunit.client.1.smithi121.stdout:3/853: symlink d2/d1a/d27/d63/da4/d25/d31/d34/d121/l127 0 2024-02-15T18:35:47.077 INFO:tasks.workunit.client.1.smithi121.stdout:3/854: mknod d2/d1a/d27/d63/da4/d25/d30/db3/dd3/c128 0 2024-02-15T18:35:47.079 INFO:tasks.workunit.client.1.smithi121.stdout:3/855: creat d2/d1a/f129 x:0 0 0 2024-02-15T18:35:47.106 INFO:tasks.workunit.client.1.smithi121.stdout:6/983: sync 2024-02-15T18:35:47.106 INFO:tasks.workunit.client.1.smithi121.stdout:2/743: sync 2024-02-15T18:35:47.107 INFO:tasks.workunit.client.1.smithi121.stdout:6/984: write d7/dc/d55/f104 [218745,90191] 0 2024-02-15T18:35:47.107 INFO:tasks.workunit.client.1.smithi121.stdout:6/985: fdatasync d7/dc/d34/d42/d62/fde 0 2024-02-15T18:35:47.107 INFO:tasks.workunit.client.1.smithi121.stdout:4/889: sync 2024-02-15T18:35:47.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:46 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:46.597542+0000 mgr.smithi121.ckiryi (mgr.24385) 17 : cluster [DBG] pgmap v17: 65 pgs: 65 active+clean; 2.1 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 78 MiB/s rd, 172 MiB/s wr, 531 op/s 2024-02-15T18:35:47.110 INFO:tasks.workunit.client.1.smithi121.stdout:6/986: mknod d7/d4b/d59/d12c/c158 0 2024-02-15T18:35:47.112 INFO:tasks.workunit.client.1.smithi121.stdout:6/987: creat d7/d4b/d59/f159 x:0 0 0 2024-02-15T18:35:47.112 INFO:tasks.workunit.client.1.smithi121.stdout:4/890: mknod d5/d9/d19/d4c/d9a/dcc/c115 0 2024-02-15T18:35:47.112 INFO:tasks.workunit.client.1.smithi121.stdout:4/891: dread - d5/d9/d1d/d26/d72/ff4 zero size 2024-02-15T18:35:47.114 INFO:tasks.workunit.client.1.smithi121.stdout:6/988: creat d7/dc/d34/d42/d62/d63/d8d/f15a x:0 0 0 2024-02-15T18:35:47.117 INFO:tasks.workunit.client.1.smithi121.stdout:6/989: read d7/f102 [462911,80567] 0 2024-02-15T18:35:47.118 INFO:tasks.workunit.client.1.smithi121.stdout:4/892: rmdir d5/d31/d25 39 2024-02-15T18:35:47.119 INFO:tasks.workunit.client.1.smithi121.stdout:4/893: chown d5/d9/d76/d87/lf0 449852576 1 2024-02-15T18:35:47.119 INFO:tasks.workunit.client.1.smithi121.stdout:6/990: creat d7/dc/d34/d42/f15b x:0 0 0 2024-02-15T18:35:47.120 INFO:tasks.workunit.client.1.smithi121.stdout:6/991: fsync d7/dc/d34/d42/d62/d63/d6f/d89/fea 0 2024-02-15T18:35:47.123 INFO:tasks.workunit.client.1.smithi121.stdout:6/992: mkdir d7/d4b/d60/d6c/d15c 0 2024-02-15T18:35:47.140 INFO:tasks.workunit.client.1.smithi121.stdout:2/744: dwrite d5/df/d60/d98/d58/d63/d94/fb7 [0,4194304] 0 2024-02-15T18:35:47.140 INFO:tasks.workunit.client.1.smithi121.stdout:2/745: fdatasync d5/df/d60/d98/f75 0 2024-02-15T18:35:47.145 INFO:tasks.workunit.client.1.smithi121.stdout:4/894: dwrite d5/d9/d1d/d26/d72/fa8 [0,4194304] 0 2024-02-15T18:35:47.145 INFO:tasks.workunit.client.1.smithi121.stdout:2/746: dread d5/df/d27/d61/f6b [0,4194304] 0 2024-02-15T18:35:47.146 INFO:tasks.workunit.client.1.smithi121.stdout:2/747: symlink d5/d49/ldf 0 2024-02-15T18:35:47.148 INFO:tasks.workunit.client.1.smithi121.stdout:6/993: dwrite d7/dc/d76/f12f [0,4194304] 0 2024-02-15T18:35:47.148 INFO:tasks.workunit.client.1.smithi121.stdout:6/994: write d7/d4b/f53 [2052628,44542] 0 2024-02-15T18:35:47.153 INFO:tasks.workunit.client.1.smithi121.stdout:6/995: mknod d7/dc/d34/d42/d62/d63/d66/dce/c15d 0 2024-02-15T18:35:47.157 INFO:tasks.workunit.client.1.smithi121.stdout:4/895: dread d5/d9/d76/d9e/f75 [0,4194304] 0 2024-02-15T18:35:47.180 INFO:tasks.workunit.client.1.smithi121.stdout:4/896: dwrite d5/d9/d1d/d26/d73/d81/fd4 [0,4194304] 0 2024-02-15T18:35:47.180 INFO:tasks.workunit.client.1.smithi121.stdout:4/897: dread - d5/d9/d76/d9e/fed zero size 2024-02-15T18:35:47.180 INFO:tasks.workunit.client.1.smithi121.stdout:4/898: chown d5/d9/d1d/f2f 27685399 1 2024-02-15T18:35:47.189 INFO:tasks.workunit.client.1.smithi121.stdout:4/899: link d5/d31/d25/fd2 d5/d9/d1d/d26/d72/f116 0 2024-02-15T18:35:47.193 INFO:tasks.workunit.client.1.smithi121.stdout:4/900: rename d5/d9/d19/d4c/d8d/ff6 to d5/d31/d25/f117 0 2024-02-15T18:35:47.194 INFO:tasks.workunit.client.1.smithi121.stdout:4/901: rmdir d5/d9/d1d/d26/d73/d81/d10f/d4f/d51/d63/dc8 39 2024-02-15T18:35:47.198 INFO:tasks.workunit.client.1.smithi121.stdout:4/902: dread d5/d9/d1d/d26/d73/d81/d10f/d4a/f60 [0,4194304] 0 2024-02-15T18:35:47.200 INFO:tasks.workunit.client.1.smithi121.stdout:4/903: dread d5/d9/d76/d9e/f6a [0,4194304] 0 2024-02-15T18:35:47.204 INFO:tasks.workunit.client.1.smithi121.stdout:4/904: creat d5/d31/d25/f118 x:0 0 0 2024-02-15T18:35:47.205 INFO:tasks.workunit.client.1.smithi121.stdout:4/905: fsync d5/d9/d1d/d26/d73/d81/d10f/d4f/d51/d71/fbc 0 2024-02-15T18:35:47.205 INFO:tasks.workunit.client.1.smithi121.stdout:4/906: dread - d5/d9/d1d/d26/d72/ff4 zero size 2024-02-15T18:35:47.230 INFO:tasks.workunit.client.1.smithi121.stdout:4/907: dwrite d5/d31/d25/f37 [0,4194304] 0 2024-02-15T18:35:47.230 INFO:tasks.workunit.client.1.smithi121.stdout:4/908: chown d5/d9/d1d/d26/d73/d81/lbb 1312 1 2024-02-15T18:35:47.230 INFO:tasks.workunit.client.1.smithi121.stdout:4/909: chown d5/d9/d19/l102 4 1 2024-02-15T18:35:47.255 INFO:tasks.workunit.client.1.smithi121.stdout:4/910: dwrite d5/d9/d19/d28/fb3 [4194304,4194304] 0 2024-02-15T18:35:47.255 INFO:tasks.workunit.client.1.smithi121.stdout:4/911: chown d5/d9/d1d/d26/d73/d81/d10f/d4f/d51/c7d 5343892 1 2024-02-15T18:35:47.255 INFO:tasks.workunit.client.1.smithi121.stdout:4/912: fdatasync d5/d9/d19/d2a/f10e 0 2024-02-15T18:35:47.255 INFO:tasks.workunit.client.1.smithi121.stdout:4/913: chown d5/d9/d76/d87/db4/ce0 44252 1 2024-02-15T18:35:47.281 INFO:tasks.workunit.client.1.smithi121.stdout:4/914: dwrite d5/d9/d1d/d26/d72/f116 [0,4194304] 0 2024-02-15T18:35:47.286 INFO:tasks.workunit.client.1.smithi121.stdout:4/915: mkdir d5/d9/d1d/d26/d119 0 2024-02-15T18:35:47.286 INFO:tasks.workunit.client.1.smithi121.stdout:4/916: chown d5/d9/d76/d9e/d14 236349 1 2024-02-15T18:35:47.315 INFO:tasks.workunit.client.1.smithi121.stdout:4/917: dwrite d5/d9/d19/d2a/d4d/f54 [0,4194304] 0 2024-02-15T18:35:47.317 INFO:tasks.workunit.client.1.smithi121.stdout:4/918: creat d5/d9/d1d/d26/d73/d81/d10f/f11a x:0 0 0 2024-02-15T18:35:47.317 INFO:tasks.workunit.client.1.smithi121.stdout:4/919: chown d5/d9/d76/d9e/f3c 44 1 2024-02-15T18:35:47.337 INFO:tasks.workunit.client.1.smithi121.stdout:4/920: dwrite d5/d9/d1d/d26/d73/d81/d10f/d4f/d51/d71/fdf [0,4194304] 0 2024-02-15T18:35:47.337 INFO:tasks.workunit.client.1.smithi121.stdout:4/921: write d5/d31/d25/fe4 [732436,20401] 0 2024-02-15T18:35:47.338 INFO:tasks.workunit.client.1.smithi121.stdout:4/922: write d5/d9/d76/d9e/f75 [2820301,91786] 0 2024-02-15T18:35:47.338 INFO:tasks.workunit.client.1.smithi121.stdout:4/923: chown d5/d9/d19/c7c 5 1 2024-02-15T18:35:47.338 INFO:tasks.workunit.client.1.smithi121.stdout:4/924: chown d5/d31/d25/l52 61 1 2024-02-15T18:35:47.339 INFO:tasks.workunit.client.1.smithi121.stdout:4/925: write d5/d31/d25/f33 [1257091,56837] 0 2024-02-15T18:35:47.341 INFO:tasks.workunit.client.1.smithi121.stdout:4/926: write d5/d9/d1d/d26/d73/d81/d10f/d4f/d51/ff2 [2503248,86385] 0 2024-02-15T18:35:47.342 INFO:tasks.workunit.client.1.smithi121.stdout:4/927: creat d5/d31/d25/f11b x:0 0 0 2024-02-15T18:35:47.342 INFO:tasks.workunit.client.1.smithi121.stdout:4/928: readlink d5/d9/d19/d2a/d4d/l58 0 2024-02-15T18:35:47.385 INFO:tasks.workunit.client.1.smithi121.stdout:8/849: sync 2024-02-15T18:35:47.387 INFO:tasks.workunit.client.1.smithi121.stdout:8/850: write d0/d7/d36/d52/fb4 [1126182,101312] 0 2024-02-15T18:35:47.400 INFO:tasks.workunit.client.1.smithi121.stdout:8/851: dwrite d0/d7/d36/fb9 [0,4194304] 0 2024-02-15T18:35:47.403 INFO:tasks.workunit.client.1.smithi121.stdout:8/852: getdents d0/d48 0 2024-02-15T18:35:47.403 INFO:tasks.workunit.client.1.smithi121.stdout:8/853: read d0/d7/de/f54 [43649,88299] 0 2024-02-15T18:35:47.403 INFO:tasks.workunit.client.1.smithi121.stdout:8/854: fdatasync d0/d48/f101 0 2024-02-15T18:35:47.404 INFO:tasks.workunit.client.1.smithi121.stdout:8/855: creat d0/dce/df8/f121 x:0 0 0 2024-02-15T18:35:47.404 INFO:tasks.workunit.client.1.smithi121.stdout:8/856: chown d0/d61/d7b 218 1 2024-02-15T18:35:47.404 INFO:tasks.workunit.client.1.smithi121.stdout:8/857: stat d0/d2a/fd5 0 2024-02-15T18:35:47.404 INFO:tasks.workunit.client.1.smithi121.stdout:8/858: stat d0/d7/d36/d8c/d45/f2c 0 2024-02-15T18:35:47.405 INFO:tasks.workunit.client.1.smithi121.stdout:8/859: link d0/d7/d36/d8c/d45/c46 d0/d7/d36/d52/c122 0 2024-02-15T18:35:47.405 INFO:tasks.workunit.client.1.smithi121.stdout:8/860: fdatasync d0/f13 0 2024-02-15T18:35:47.406 INFO:tasks.workunit.client.1.smithi121.stdout:8/861: creat d0/d7/db1/d117/f123 x:0 0 0 2024-02-15T18:35:47.406 INFO:tasks.workunit.client.1.smithi121.stdout:8/862: write d0/d7/d36/d8c/de9/fe7 [208298,22355] 0 2024-02-15T18:35:47.416 INFO:tasks.workunit.client.1.smithi121.stdout:8/863: dread d0/d7/d36/d8c/de9/fe7 [0,4194304] 0 2024-02-15T18:35:47.418 INFO:tasks.workunit.client.1.smithi121.stdout:8/864: creat d0/d7/d36/d52/d104/f124 x:0 0 0 2024-02-15T18:35:47.418 INFO:tasks.workunit.client.1.smithi121.stdout:8/865: fdatasync d0/d2a/f4b 0 2024-02-15T18:35:47.418 INFO:tasks.workunit.client.1.smithi121.stdout:8/866: stat d0/d3/dc9 0 2024-02-15T18:35:47.424 INFO:tasks.workunit.client.1.smithi121.stdout:8/867: dread d0/d7/d36/d52/d7a/f9a [0,4194304] 0 2024-02-15T18:35:47.425 INFO:tasks.workunit.client.1.smithi121.stdout:8/868: rmdir d0 39 2024-02-15T18:35:47.426 INFO:tasks.workunit.client.1.smithi121.stdout:8/869: dread d0/d7/d36/d52/fc7 [0,4194304] 0 2024-02-15T18:35:47.426 INFO:tasks.workunit.client.1.smithi121.stdout:8/870: stat d0/l4 0 2024-02-15T18:35:47.428 INFO:tasks.workunit.client.1.smithi121.stdout:8/871: dread d0/d61/d95/fd1 [0,4194304] 0 2024-02-15T18:35:47.431 INFO:tasks.workunit.client.1.smithi121.stdout:8/872: dread d0/d7/da5/dcb/f91 [0,4194304] 0 2024-02-15T18:35:47.444 INFO:tasks.workunit.client.1.smithi121.stdout:8/873: dwrite d0/d7/d36/d52/fc7 [0,4194304] 0 2024-02-15T18:35:47.447 INFO:tasks.workunit.client.1.smithi121.stdout:8/874: symlink d0/d61/d66/l125 0 2024-02-15T18:35:47.456 INFO:tasks.workunit.client.1.smithi121.stdout:8/875: dread d0/d7/d36/d8c/d45/f9e [0,4194304] 0 2024-02-15T18:35:47.458 INFO:tasks.workunit.client.1.smithi121.stdout:8/876: rename d0/d48/d86 to d0/d126 0 2024-02-15T18:35:47.461 INFO:tasks.workunit.client.1.smithi121.stdout:8/877: link d0/d7/da5/dcb/f91 d0/d7/d36/d8c/d45/f127 0 2024-02-15T18:35:47.467 INFO:tasks.workunit.client.1.smithi121.stdout:3/856: sync 2024-02-15T18:35:47.467 INFO:tasks.workunit.client.1.smithi121.stdout:2/748: sync 2024-02-15T18:35:47.467 INFO:tasks.workunit.client.1.smithi121.stdout:6/996: sync 2024-02-15T18:35:47.467 INFO:tasks.workunit.client.1.smithi121.stdout:7/693: sync 2024-02-15T18:35:47.467 INFO:tasks.workunit.client.1.smithi121.stdout:6/997: chown d7/c30 27 1 2024-02-15T18:35:47.467 INFO:tasks.workunit.client.1.smithi121.stdout:2/749: write d5/fb5 [1455158,94211] 0 2024-02-15T18:35:47.468 INFO:tasks.workunit.client.1.smithi121.stdout:7/694: chown d0/f29 4430441 1 2024-02-15T18:35:47.468 INFO:tasks.workunit.client.1.smithi121.stdout:7/695: write d0/d5/f63 [619438,52606] 0 2024-02-15T18:35:47.469 INFO:tasks.workunit.client.1.smithi121.stdout:3/857: creat d2/d1a/d6b/d7d/f12a x:0 0 0 2024-02-15T18:35:47.470 INFO:tasks.workunit.client.1.smithi121.stdout:7/696: dread d0/d38/fc1 [0,4194304] 0 2024-02-15T18:35:47.470 INFO:tasks.workunit.client.1.smithi121.stdout:6/998: symlink d7/d4b/d60/d6c/l15e 0 2024-02-15T18:35:47.470 INFO:tasks.workunit.client.1.smithi121.stdout:8/878: dread d0/f8 [0,4194304] 0 2024-02-15T18:35:47.470 INFO:tasks.workunit.client.1.smithi121.stdout:3/858: read d2/d1a/d54/fe5 [3567721,130160] 0 2024-02-15T18:35:47.471 INFO:tasks.workunit.client.1.smithi121.stdout:8/879: write d0/d7/d36/d8c/d45/f89 [3951948,56038] 0 2024-02-15T18:35:47.471 INFO:tasks.workunit.client.1.smithi121.stdout:8/880: chown d0/d7/d36/d8c/d45/fcc 1280 1 2024-02-15T18:35:47.473 INFO:tasks.workunit.client.1.smithi121.stdout:7/697: link d0/d4a/f53 d0/d38/d61/fe4 0 2024-02-15T18:35:47.473 INFO:tasks.workunit.client.1.smithi121.stdout:8/881: write d0/d61/d66/f3b [7523963,62833] 0 2024-02-15T18:35:47.475 INFO:tasks.workunit.client.1.smithi121.stdout:3/859: rmdir d2/d1a/d27/d63/da4/d25/d30/db3/ddf 39 2024-02-15T18:35:47.475 INFO:tasks.workunit.client.1.smithi121.stdout:3/860: rename d2/d1a/d27 to d2/d1a/d27/da7/d85/d9f/d120/d12b 22 2024-02-15T18:35:47.479 INFO:tasks.workunit.client.1.smithi121.stdout:2/750: unlink d5/df/d27/d3e/d6a/l85 0 2024-02-15T18:35:47.479 INFO:tasks.workunit.client.1.smithi121.stdout:3/861: getdents d2/d1a/d27/d63/d81 0 2024-02-15T18:35:47.479 INFO:tasks.workunit.client.1.smithi121.stdout:8/882: creat d0/d7/da7/f128 x:0 0 0 2024-02-15T18:35:47.479 INFO:tasks.workunit.client.1.smithi121.stdout:3/862: write d2/d1a/d27/d63/da4/d25/d31/d58/f6f [578265,99789] 0 2024-02-15T18:35:47.479 INFO:tasks.workunit.client.1.smithi121.stdout:3/863: write d2/d1a/f129 [609454,69879] 0 2024-02-15T18:35:47.480 INFO:tasks.workunit.client.1.smithi121.stdout:8/883: dread d0/dce/df8/f77 [0,4194304] 0 2024-02-15T18:35:47.481 INFO:tasks.workunit.client.1.smithi121.stdout:2/751: mkdir d5/df/d60/d98/d58/d63/d94/de0 0 2024-02-15T18:35:47.483 INFO:tasks.workunit.client.1.smithi121.stdout:2/752: rename d5/df/d10/lb1 to d5/df/le1 0 2024-02-15T18:35:47.492 INFO:tasks.workunit.client.1.smithi121.stdout:6/999: dwrite d7/dc/f148 [0,4194304] 0 2024-02-15T18:35:47.499 INFO:tasks.workunit.client.1.smithi121.stdout:3/864: dwrite d2/d1a/d27/d63/d81/fd5 [0,4194304] 0 2024-02-15T18:35:47.499 INFO:tasks.workunit.client.1.smithi121.stdout:2/753: getdents d5/df 0 2024-02-15T18:35:47.500 INFO:tasks.workunit.client.1.smithi121.stdout:2/754: chown d5 280288463 1 2024-02-15T18:35:47.501 INFO:tasks.workunit.client.1.smithi121.stdout:4/929: sync 2024-02-15T18:35:47.501 INFO:tasks.workunit.client.1.smithi121.stdout:4/930: write d5/d9/d19/d2a/d4d/f10a [341136,17395] 0 2024-02-15T18:35:47.503 INFO:tasks.workunit.client.1.smithi121.stdout:3/865: creat d2/d1a/d27/da7/d85/d9f/f12c x:0 0 0 2024-02-15T18:35:47.504 INFO:tasks.workunit.client.1.smithi121.stdout:2/755: creat d5/df/d27/d61/fe2 x:0 0 0 2024-02-15T18:35:47.504 INFO:tasks.workunit.client.1.smithi121.stdout:2/756: chown d5/df/d27/f79 6 1 2024-02-15T18:35:47.513 INFO:tasks.workunit.client.1.smithi121.stdout:2/757: dread d5/df/d10/d4c/f77 [0,4194304] 0 2024-02-15T18:35:47.519 INFO:tasks.workunit.client.1.smithi121.stdout:2/758: mkdir d5/df/d27/d3e/d9f/dc2/de3 0 2024-02-15T18:35:47.519 INFO:tasks.workunit.client.1.smithi121.stdout:2/759: dread - d5/df/d60/d98/fb8 zero size 2024-02-15T18:35:47.522 INFO:tasks.workunit.client.1.smithi121.stdout:2/760: unlink d5/df/d27/f76 0 2024-02-15T18:35:47.523 INFO:tasks.workunit.client.1.smithi121.stdout:2/761: creat d5/df/d60/d98/d58/d63/d94/dc3/fe4 x:0 0 0 2024-02-15T18:35:47.523 INFO:tasks.workunit.client.1.smithi121.stdout:2/762: write d5/df/d10/d4c/fc7 [4646652,76345] 0 2024-02-15T18:35:47.524 INFO:tasks.workunit.client.1.smithi121.stdout:2/763: symlink d5/df/d10/ddb/le5 0 2024-02-15T18:35:47.525 INFO:tasks.workunit.client.1.smithi121.stdout:3/866: dwrite d2/d1a/d27/d63/da4/d25/d31/f43 [0,4194304] 0 2024-02-15T18:35:47.526 INFO:tasks.workunit.client.1.smithi121.stdout:2/764: mkdir d5/df/d60/d98/d22/de6 0 2024-02-15T18:35:47.528 INFO:tasks.workunit.client.1.smithi121.stdout:3/867: creat d2/d1a/d27/da7/d85/f12d x:0 0 0 2024-02-15T18:35:47.528 INFO:tasks.workunit.client.1.smithi121.stdout:4/931: dwrite d5/d9/d76/d87/f111 [0,4194304] 0 2024-02-15T18:35:47.530 INFO:tasks.workunit.client.1.smithi121.stdout:4/932: write d5/d9/d1d/d26/d73/d81/fd4 [1660601,41289] 0 2024-02-15T18:35:47.530 INFO:tasks.workunit.client.1.smithi121.stdout:2/765: dread d5/d49/f84 [0,4194304] 0 2024-02-15T18:35:47.531 INFO:tasks.workunit.client.1.smithi121.stdout:2/766: mknod d5/d49/ddc/ce7 0 2024-02-15T18:35:47.542 INFO:tasks.workunit.client.1.smithi121.stdout:3/868: dread d2/d1a/d54/fe5 [0,4194304] 0 2024-02-15T18:35:47.552 INFO:tasks.workunit.client.1.smithi121.stdout:4/933: dwrite d5/d9/d1d/d26/d73/d81/d10f/d4f/d51/d80/ffc [0,4194304] 0 2024-02-15T18:35:47.552 INFO:tasks.workunit.client.1.smithi121.stdout:2/767: dwrite d5/fd1 [0,4194304] 0 2024-02-15T18:35:47.555 INFO:tasks.workunit.client.1.smithi121.stdout:4/934: creat d5/d9/d19/d5c/f11c x:0 0 0 2024-02-15T18:35:47.555 INFO:tasks.workunit.client.1.smithi121.stdout:4/935: readlink d5/d31/l45 0 2024-02-15T18:35:47.557 INFO:tasks.workunit.client.1.smithi121.stdout:2/768: unlink d5/df/d60/d98/d58/d63/d94/l96 0 2024-02-15T18:35:47.559 INFO:tasks.workunit.client.1.smithi121.stdout:3/869: dread d2/d5a/f95 [0,4194304] 0 2024-02-15T18:35:47.559 INFO:tasks.workunit.client.1.smithi121.stdout:3/870: fsync d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2/f105 0 2024-02-15T18:35:47.559 INFO:tasks.workunit.client.1.smithi121.stdout:2/769: creat d5/df/d27/d3a/fe8 x:0 0 0 2024-02-15T18:35:47.559 INFO:tasks.workunit.client.1.smithi121.stdout:2/770: mknod d5/df/d3f/daa/ce9 0 2024-02-15T18:35:47.560 INFO:tasks.workunit.client.1.smithi121.stdout:4/936: symlink d5/d9/d19/d4c/d9a/dcc/l11d 0 2024-02-15T18:35:47.560 INFO:tasks.workunit.client.1.smithi121.stdout:2/771: mknod d5/df/d54/cea 0 2024-02-15T18:35:47.561 INFO:tasks.workunit.client.1.smithi121.stdout:4/937: creat d5/d9/d76/d9e/d14/f11e x:0 0 0 2024-02-15T18:35:47.563 INFO:tasks.workunit.client.1.smithi121.stdout:4/938: creat d5/d9/d19/d4c/dc3/f11f x:0 0 0 2024-02-15T18:35:47.564 INFO:tasks.workunit.client.1.smithi121.stdout:4/939: creat d5/d9/d76/d9e/d14/f120 x:0 0 0 2024-02-15T18:35:47.566 INFO:tasks.workunit.client.1.smithi121.stdout:4/940: write d5/d9/d76/d87/db4/fae [2698428,88480] 0 2024-02-15T18:35:47.568 INFO:tasks.workunit.client.1.smithi121.stdout:2/772: dread d5/df/f6c [0,4194304] 0 2024-02-15T18:35:47.568 INFO:tasks.workunit.client.1.smithi121.stdout:4/941: write d5/d9/d1d/d26/d73/d81/d10f/d4f/d51/d71/f48 [4032795,67116] 0 2024-02-15T18:35:47.570 INFO:tasks.workunit.client.1.smithi121.stdout:2/773: creat d5/df/feb x:0 0 0 2024-02-15T18:35:47.571 INFO:tasks.workunit.client.1.smithi121.stdout:2/774: creat d5/df/d54/fec x:0 0 0 2024-02-15T18:35:47.579 INFO:tasks.workunit.client.1.smithi121.stdout:3/871: dwrite d2/d1a/d27/d63/da4/d25/d30/db3/f11c [0,4194304] 0 2024-02-15T18:35:47.582 INFO:tasks.workunit.client.1.smithi121.stdout:3/872: creat d2/d1a/d27/da7/d85/d9f/ddd/de8/f12e x:0 0 0 2024-02-15T18:35:47.582 INFO:tasks.workunit.client.1.smithi121.stdout:3/873: write d2/d1a/d27/d63/da4/d25/d30/db3/ddf/f116 [848957,121613] 0 2024-02-15T18:35:47.582 INFO:tasks.workunit.client.1.smithi121.stdout:3/874: chown d2/d1a/d27/da7/d85/d9f/d107/f111 1959013255 1 2024-02-15T18:35:47.582 INFO:tasks.workunit.client.1.smithi121.stdout:3/875: chown d2/d1a/d6b/d7d/f83 0 1 2024-02-15T18:35:47.590 INFO:tasks.workunit.client.1.smithi121.stdout:4/942: dwrite d5/fa [0,4194304] 0 2024-02-15T18:35:47.596 INFO:tasks.workunit.client.1.smithi121.stdout:2/775: dwrite d5/df/d27/d3e/f93 [0,4194304] 0 2024-02-15T18:35:47.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.603+0000 7f5a3dd43700 1 -- 172.21.15.38:0/1687007972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a380f8610 msgr2=0x7f5a380f4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:47.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.603+0000 7f5a3dd43700 1 --2- 172.21.15.38:0/1687007972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a380f8610 0x7f5a380f4820 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f5a2c004660 tx=0x7f5a2c009b30).stop 2024-02-15T18:35:47.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.605+0000 7f5a3dd43700 1 -- 172.21.15.38:0/1687007972 shutdown_connections 2024-02-15T18:35:47.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.605+0000 7f5a3dd43700 1 --2- 172.21.15.38:0/1687007972 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a380f8610 0x7f5a380f4820 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:47.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.605+0000 7f5a3dd43700 1 --2- 172.21.15.38:0/1687007972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a380f7d00 0x7f5a380f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:35:47.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.605+0000 7f5a3dd43700 1 -- 172.21.15.38:0/1687007972 >> 172.21.15.38:0/1687007972 conn(0x7f5a380f0540 msgr2=0x7f5a380f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:47.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.606+0000 7f5a3dd43700 1 -- 172.21.15.38:0/1687007972 shutdown_connections 2024-02-15T18:35:47.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.606+0000 7f5a3dd43700 1 -- 172.21.15.38:0/1687007972 wait complete. 2024-02-15T18:35:47.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.608+0000 7f5a3dd43700 1 Processor -- start 2024-02-15T18:35:47.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.608+0000 7f5a3dd43700 1 -- start start 2024-02-15T18:35:47.600 INFO:tasks.workunit.client.1.smithi121.stdout:2/776: rename d5/df/d54 to d5/ded 0 2024-02-15T18:35:47.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.608+0000 7f5a3dd43700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a380f7d00 0x7f5a3810d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:47.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.608+0000 7f5a3dd43700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a380f8610 0x7f5a3810db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:47.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.608+0000 7f5a3dd43700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3810e180 con 0x7f5a380f8610 2024-02-15T18:35:47.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.608+0000 7f5a3dd43700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3810ea30 con 0x7f5a380f7d00 2024-02-15T18:35:47.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.609+0000 7f5a36ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a380f8610 0x7f5a3810db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:47.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.609+0000 7f5a36ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a380f8610 0x7f5a3810db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55000/0 (socket says 172.21.15.38:55000) 2024-02-15T18:35:47.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.609+0000 7f5a36ffd700 1 -- 172.21.15.38:0/1444389714 learned_addr learned my addr 172.21.15.38:0/1444389714 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:35:47.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.609+0000 7f5a377fe700 1 --2- 172.21.15.38:0/1444389714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a380f7d00 0x7f5a3810d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:47.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.609+0000 7f5a36ffd700 1 -- 172.21.15.38:0/1444389714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a380f7d00 msgr2=0x7f5a3810d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:47.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.609+0000 7f5a36ffd700 1 --2- 172.21.15.38:0/1444389714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a380f7d00 0x7f5a3810d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:47.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.609+0000 7f5a36ffd700 1 -- 172.21.15.38:0/1444389714 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2c005040 con 0x7f5a380f8610 2024-02-15T18:35:47.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.609+0000 7f5a36ffd700 1 --2- 172.21.15.38:0/1444389714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a380f8610 0x7f5a3810db40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f5a2c009f50 tx=0x7f5a2c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:47.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.610+0000 7f5a1ffff700 1 -- 172.21.15.38:0/1444389714 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a2c018070 con 0x7f5a380f8610 2024-02-15T18:35:47.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.610+0000 7f5a1ffff700 1 -- 172.21.15.38:0/1444389714 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f5a2c007d70 con 0x7f5a380f8610 2024-02-15T18:35:47.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.610+0000 7f5a1ffff700 1 -- 172.21.15.38:0/1444389714 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a2c01c620 con 0x7f5a380f8610 2024-02-15T18:35:47.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.611+0000 7f5a3dd43700 1 -- 172.21.15.38:0/1444389714 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3810ecb0 con 0x7f5a380f8610 2024-02-15T18:35:47.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.611+0000 7f5a3dd43700 1 -- 172.21.15.38:0/1444389714 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3810f190 con 0x7f5a380f8610 2024-02-15T18:35:47.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.613+0000 7f5a1ffff700 1 -- 172.21.15.38:0/1444389714 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 23) v1 ==== 74714+0+0 (secure 0 0 0) 0x7f5a2c01c7e0 con 0x7f5a380f8610 2024-02-15T18:35:47.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.613+0000 7f5a1ffff700 1 --2- 172.21.15.38:0/1444389714 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7f5a2005e9c0 0x7f5a20060e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:47.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.613+0000 7f5a1ffff700 1 -- 172.21.15.38:0/1444389714 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5a2c07ae80 con 0x7f5a380f8610 2024-02-15T18:35:47.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.614+0000 7f5a377fe700 1 --2- 172.21.15.38:0/1444389714 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7f5a2005e9c0 0x7f5a20060e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:47.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.614+0000 7f5a3dd43700 1 -- 172.21.15.38:0/1444389714 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a380f41d0 con 0x7f5a380f8610 2024-02-15T18:35:47.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.623+0000 7f5a377fe700 1 --2- 172.21.15.38:0/1444389714 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7f5a2005e9c0 0x7f5a20060e80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f5a2800b790 tx=0x7f5a28008040).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:47.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.623+0000 7f5a1ffff700 1 -- 172.21.15.38:0/1444389714 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7f5a2c04ee80 con 0x7f5a380f8610 2024-02-15T18:35:47.622 INFO:tasks.workunit.client.1.smithi121.stdout:4/943: dwrite d5/d9/d19/d4c/d8d/fea [0,4194304] 0 2024-02-15T18:35:47.623 INFO:tasks.workunit.client.1.smithi121.stdout:2/777: dwrite d5/df/d10/f8a [0,4194304] 0 2024-02-15T18:35:47.626 INFO:tasks.workunit.client.1.smithi121.stdout:2/778: getdents d5/d49/d71 0 2024-02-15T18:35:47.632 INFO:tasks.workunit.client.1.smithi121.stdout:2/779: write d5/df/d60/d98/f2d [1657681,4525] 0 2024-02-15T18:35:47.632 INFO:tasks.workunit.client.1.smithi121.stdout:2/780: truncate d5/df/f2b 1396867 0 2024-02-15T18:35:47.634 INFO:tasks.workunit.client.1.smithi121.stdout:2/781: creat d5/df/d27/fee x:0 0 0 2024-02-15T18:35:47.634 INFO:tasks.workunit.client.1.smithi121.stdout:2/782: fsync d5/df/feb 0 2024-02-15T18:35:47.637 INFO:tasks.workunit.client.1.smithi121.stdout:2/783: write d5/d49/f7f [174410,55631] 0 2024-02-15T18:35:47.637 INFO:tasks.workunit.client.1.smithi121.stdout:2/784: write d5/df/f39 [376320,115634] 0 2024-02-15T18:35:47.637 INFO:tasks.workunit.client.1.smithi121.stdout:2/785: dread - d5/df/d10/d4c/dad/fdd zero size 2024-02-15T18:35:47.637 INFO:tasks.workunit.client.1.smithi121.stdout:4/944: mknod d5/d9/d1d/d26/c121 0 2024-02-15T18:35:47.670 INFO:tasks.workunit.client.1.smithi121.stdout:4/945: dwrite d5/d31/d25/db6/fe9 [4194304,4194304] 0 2024-02-15T18:35:47.671 INFO:tasks.workunit.client.1.smithi121.stdout:2/786: dwrite d5/df/d27/f79 [0,4194304] 0 2024-02-15T18:35:47.675 INFO:tasks.workunit.client.1.smithi121.stdout:2/787: rmdir d5/df/d3f/daa 39 2024-02-15T18:35:47.675 INFO:tasks.workunit.client.1.smithi121.stdout:2/788: write d5/df/d3f/fd2 [778132,12353] 0 2024-02-15T18:35:47.686 INFO:tasks.workunit.client.1.smithi121.stdout:2/789: dread d5/df/d27/d3e/f62 [0,4194304] 0 2024-02-15T18:35:47.688 INFO:tasks.workunit.client.1.smithi121.stdout:2/790: rename d5/df/d27/d3e/f93 to d5/df/d3f/daa/fef 0 2024-02-15T18:35:47.691 INFO:tasks.workunit.client.1.smithi121.stdout:2/791: dread d5/df/d60/d98/d22/d81/d9a/da1/fcc [0,4194304] 0 2024-02-15T18:35:47.692 INFO:tasks.workunit.client.1.smithi121.stdout:2/792: mkdir d5/df/d60/d98/d58/d63/d94/de0/df0 0 2024-02-15T18:35:47.692 INFO:tasks.workunit.client.1.smithi121.stdout:2/793: readlink d5/d49/lce 0 2024-02-15T18:35:47.692 INFO:tasks.workunit.client.1.smithi121.stdout:2/794: creat d5/df/d60/d98/d58/d63/ff1 x:0 0 0 2024-02-15T18:35:47.693 INFO:tasks.workunit.client.1.smithi121.stdout:2/795: creat d5/df/d27/d3e/d6a/ff2 x:0 0 0 2024-02-15T18:35:47.695 INFO:tasks.workunit.client.1.smithi121.stdout:2/796: mkdir d5/df/df3 0 2024-02-15T18:35:47.695 INFO:tasks.workunit.client.1.smithi121.stdout:2/797: fdatasync d5/d49/d71/f70 0 2024-02-15T18:35:47.696 INFO:tasks.workunit.client.1.smithi121.stdout:2/798: mknod d5/df/cf4 0 2024-02-15T18:35:47.697 INFO:tasks.workunit.client.1.smithi121.stdout:2/799: write d5/df/d10/f6d [5087382,82234] 0 2024-02-15T18:35:47.705 INFO:tasks.workunit.client.1.smithi121.stdout:4/946: dwrite d5/d31/d25/f118 [0,4194304] 0 2024-02-15T18:35:47.709 INFO:tasks.workunit.client.1.smithi121.stdout:4/947: link d5/f6 d5/d103/f122 0 2024-02-15T18:35:47.712 INFO:tasks.workunit.client.1.smithi121.stdout:4/948: mknod d5/d9/d1d/d26/d73/d81/d10f/d4a/d7e/c123 0 2024-02-15T18:35:47.713 INFO:tasks.workunit.client.1.smithi121.stdout:4/949: getdents d5/d9/d19/d4c/dc3/de6 0 2024-02-15T18:35:47.714 INFO:tasks.workunit.client.1.smithi121.stdout:2/800: dwrite d5/df/d60/d98/d22/f4f [0,4194304] 0 2024-02-15T18:35:47.715 INFO:tasks.workunit.client.1.smithi121.stdout:4/950: write d5/d9/d76/ff3 [371380,80503] 0 2024-02-15T18:35:47.721 INFO:tasks.workunit.client.1.smithi121.stdout:2/801: symlink d5/df/d27/d3e/d9f/lf5 0 2024-02-15T18:35:47.721 INFO:tasks.workunit.client.1.smithi121.stdout:7/698: sync 2024-02-15T18:35:47.724 INFO:tasks.workunit.client.1.smithi121.stdout:7/699: mkdir d0/d4a/de5 0 2024-02-15T18:35:47.724 INFO:tasks.workunit.client.1.smithi121.stdout:2/802: creat d5/df/ff6 x:0 0 0 2024-02-15T18:35:47.727 INFO:tasks.workunit.client.1.smithi121.stdout:7/700: mkdir d0/d5/d4c/d6f/dd5/de6 0 2024-02-15T18:35:47.727 INFO:tasks.workunit.client.1.smithi121.stdout:2/803: creat d5/df/d10/ff7 x:0 0 0 2024-02-15T18:35:47.729 INFO:tasks.workunit.client.1.smithi121.stdout:7/701: symlink d0/d38/d46/le7 0 2024-02-15T18:35:47.730 INFO:tasks.workunit.client.1.smithi121.stdout:7/702: getdents d0/d4a/da2 0 2024-02-15T18:35:47.732 INFO:tasks.workunit.client.1.smithi121.stdout:7/703: write d0/d5/d11/d1c/f21 [3327885,68304] 0 2024-02-15T18:35:47.732 INFO:tasks.workunit.client.1.smithi121.stdout:7/704: dread - d0/d5/d11/d1c/d5f/dc0/fe1 zero size 2024-02-15T18:35:47.734 INFO:tasks.workunit.client.1.smithi121.stdout:7/705: link d0/d5/d4c/d6f/ld2 d0/d38/d46/daa/le8 0 2024-02-15T18:35:47.734 INFO:tasks.workunit.client.1.smithi121.stdout:7/706: chown d0/d5/d4c/d6f/ld2 1022992 1 2024-02-15T18:35:47.734 INFO:tasks.workunit.client.1.smithi121.stdout:7/707: chown d0/d38/cc7 3 1 2024-02-15T18:35:47.740 INFO:tasks.workunit.client.1.smithi121.stdout:2/804: dread d5/fd1 [0,4194304] 0 2024-02-15T18:35:47.740 INFO:tasks.workunit.client.1.smithi121.stdout:2/805: write d5/df/d27/d3e/f59 [965495,78275] 0 2024-02-15T18:35:47.740 INFO:tasks.workunit.client.1.smithi121.stdout:2/806: fsync d5/df/d60/d98/d58/d63/d94/dc3/fe4 0 2024-02-15T18:35:47.743 INFO:tasks.workunit.client.1.smithi121.stdout:2/807: dread d5/df/f39 [0,4194304] 0 2024-02-15T18:35:47.766 INFO:tasks.workunit.client.1.smithi121.stdout:2/808: dwrite d5/df/d27/d3e/f62 [0,4194304] 0 2024-02-15T18:35:47.786 INFO:tasks.workunit.client.1.smithi121.stdout:2/809: dwrite d5/df/d10/d4c/fb2 [0,4194304] 0 2024-02-15T18:35:47.789 INFO:tasks.workunit.client.1.smithi121.stdout:2/810: unlink d5/df/d60/d98/d58/d63/d94/dc3/ld6 0 2024-02-15T18:35:47.805 INFO:tasks.workunit.client.1.smithi121.stdout:2/811: dwrite d5/df/d60/d98/d58/d63/d94/fb7 [0,4194304] 0 2024-02-15T18:35:47.836 INFO:tasks.workunit.client.1.smithi121.stdout:8/884: sync 2024-02-15T18:35:47.836 INFO:tasks.workunit.client.1.smithi121.stdout:3/876: sync 2024-02-15T18:35:47.836 INFO:tasks.workunit.client.1.smithi121.stdout:8/885: read d0/d7/d36/fba [180862,53429] 0 2024-02-15T18:35:47.836 INFO:tasks.workunit.client.1.smithi121.stdout:3/877: read d2/d5a/f67 [2403172,54202] 0 2024-02-15T18:35:47.837 INFO:tasks.workunit.client.1.smithi121.stdout:3/878: creat d2/d1a/d27/da7/d85/d9f/d120/f12f x:0 0 0 2024-02-15T18:35:47.844 INFO:tasks.workunit.client.1.smithi121.stdout:3/879: write d2/d1a/d27/da7/f35 [2416853,41094] 0 2024-02-15T18:35:47.845 INFO:tasks.workunit.client.1.smithi121.stdout:3/880: creat d2/d1a/d27/da7/d85/f130 x:0 0 0 2024-02-15T18:35:47.847 INFO:tasks.workunit.client.1.smithi121.stdout:3/881: read d2/d1a/d27/d63/d81/fac [412335,129115] 0 2024-02-15T18:35:47.848 INFO:tasks.workunit.client.1.smithi121.stdout:3/882: creat d2/d1a/d27/d63/d110/f131 x:0 0 0 2024-02-15T18:35:47.858 INFO:tasks.workunit.client.1.smithi121.stdout:8/886: dwrite d0/d7/d36/d52/fc7 [0,4194304] 0 2024-02-15T18:35:47.858 INFO:tasks.workunit.client.1.smithi121.stdout:8/887: write d0/d7/d36/d52/d7a/f9a [3004925,130273] 0 2024-02-15T18:35:47.861 INFO:tasks.workunit.client.1.smithi121.stdout:8/888: rename d0/d3/dc9 to d0/dce/df8/d129 0 2024-02-15T18:35:47.862 INFO:tasks.workunit.client.1.smithi121.stdout:8/889: creat d0/dce/df8/f12a x:0 0 0 2024-02-15T18:35:47.862 INFO:tasks.workunit.client.1.smithi121.stdout:8/890: mknod d0/d7/d36/d52/d104/c12b 0 2024-02-15T18:35:47.863 INFO:tasks.workunit.client.1.smithi121.stdout:8/891: rmdir d0/d2a 39 2024-02-15T18:35:47.865 INFO:tasks.workunit.client.1.smithi121.stdout:8/892: creat d0/d7/d36/d52/d7a/f12c x:0 0 0 2024-02-15T18:35:47.866 INFO:tasks.workunit.client.1.smithi121.stdout:8/893: rename d0/d2a/c75 to d0/d61/d95/c12d 0 2024-02-15T18:35:47.869 INFO:tasks.workunit.client.1.smithi121.stdout:3/883: dwrite d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2/d10a/f6d [0,4194304] 0 2024-02-15T18:35:47.869 INFO:tasks.workunit.client.1.smithi121.stdout:3/884: fdatasync d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2/d10a/f6d 0 2024-02-15T18:35:47.876 INFO:tasks.workunit.client.1.smithi121.stdout:8/894: dread d0/d7/da5/dcb/f10d [0,4194304] 0 2024-02-15T18:35:47.876 INFO:tasks.workunit.client.1.smithi121.stdout:8/895: fsync d0/d7/d36/d8c/d45/d53/fa0 0 2024-02-15T18:35:47.877 INFO:tasks.workunit.client.1.smithi121.stdout:8/896: rename d0/d2a/d51/c41 to d0/d7/da5/dcb/d10c/c12e 0 2024-02-15T18:35:47.878 INFO:tasks.workunit.client.1.smithi121.stdout:8/897: write d0/d7/d36/d8c/d45/f3e [151145,34106] 0 2024-02-15T18:35:47.879 INFO:tasks.workunit.client.1.smithi121.stdout:8/898: mkdir d0/d7/de/d12f 0 2024-02-15T18:35:47.880 INFO:tasks.workunit.client.1.smithi121.stdout:3/885: dread d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2/d10a/f9c [0,4194304] 0 2024-02-15T18:35:47.881 INFO:tasks.workunit.client.1.smithi121.stdout:3/886: write d2/d1a/d6b/d7d/f83 [168692,104139] 0 2024-02-15T18:35:47.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.896+0000 7f5a3dd43700 1 -- 172.21.15.38:0/1444389714 --> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a380ff040 con 0x7f5a2005e9c0 2024-02-15T18:35:47.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.900+0000 7f5a1ffff700 1 -- 172.21.15.38:0/1444389714 <== mgr.24385 v2:172.21.15.121:6828/330429733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f5a380ff040 con 0x7f5a2005e9c0 2024-02-15T18:35:47.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.902+0000 7f5a3dd43700 1 -- 172.21.15.38:0/1444389714 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7f5a2005e9c0 msgr2=0x7f5a20060e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:47.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.902+0000 7f5a3dd43700 1 --2- 172.21.15.38:0/1444389714 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7f5a2005e9c0 0x7f5a20060e80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f5a2800b790 tx=0x7f5a28008040).stop 2024-02-15T18:35:47.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.902+0000 7f5a3dd43700 1 -- 172.21.15.38:0/1444389714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a380f8610 msgr2=0x7f5a3810db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:47.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.902+0000 7f5a3dd43700 1 --2- 172.21.15.38:0/1444389714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a380f8610 0x7f5a3810db40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f5a2c009f50 tx=0x7f5a2c009b30).stop 2024-02-15T18:35:47.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.902+0000 7f5a3dd43700 1 -- 172.21.15.38:0/1444389714 shutdown_connections 2024-02-15T18:35:47.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.902+0000 7f5a3dd43700 1 --2- 172.21.15.38:0/1444389714 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7f5a2005e9c0 0x7f5a20060e80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:47.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.902+0000 7f5a3dd43700 1 --2- 172.21.15.38:0/1444389714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a380f7d00 0x7f5a3810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:47.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.902+0000 7f5a3dd43700 1 --2- 172.21.15.38:0/1444389714 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a380f8610 0x7f5a3810db40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:47.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.902+0000 7f5a3dd43700 1 -- 172.21.15.38:0/1444389714 >> 172.21.15.38:0/1444389714 conn(0x7f5a380f0540 msgr2=0x7f5a380f6150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:47.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.903+0000 7f5a3dd43700 1 -- 172.21.15.38:0/1444389714 shutdown_connections 2024-02-15T18:35:47.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:47.903+0000 7f5a3dd43700 1 -- 172.21.15.38:0/1444389714 wait complete. 2024-02-15T18:35:47.900 INFO:tasks.workunit.client.1.smithi121.stdout:3/887: dwrite d2/f10e [0,4194304] 0 2024-02-15T18:35:47.901 INFO:tasks.workunit.client.1.smithi121.stdout:3/888: creat d2/d1a/d27/da7/d85/d9f/f132 x:0 0 0 2024-02-15T18:35:47.904 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-02-15T18:35:47.908 INFO:tasks.workunit.client.1.smithi121.stdout:8/899: rename d0/d7/da5 to d0/d7/d36/d8c/de9/d130 0 2024-02-15T18:35:47.908 INFO:tasks.workunit.client.1.smithi121.stdout:8/900: getdents d0/d7/d36/d8c/de9/d130/dcb/d102 0 2024-02-15T18:35:47.910 INFO:tasks.workunit.client.1.smithi121.stdout:8/901: dread d0/f15 [4194304,4194304] 0 2024-02-15T18:35:47.910 INFO:tasks.workunit.client.1.smithi121.stdout:8/902: dread - d0/d2a/fae zero size 2024-02-15T18:35:47.910 INFO:tasks.workunit.client.1.smithi121.stdout:8/903: chown d0/d3/c17 1 1 2024-02-15T18:35:47.911 INFO:tasks.workunit.client.1.smithi121.stdout:8/904: link d0/d7/d36/d8c/d45/f127 d0/d7/d36/d8c/d45/dcd/f131 0 2024-02-15T18:35:47.911 INFO:tasks.workunit.client.1.smithi121.stdout:8/905: readlink d0/d7/d36/d8c/d45/d53/l7e 0 2024-02-15T18:35:47.912 INFO:tasks.workunit.client.1.smithi121.stdout:8/906: dread d0/d7/fa4 [0,4194304] 0 2024-02-15T18:35:47.914 INFO:tasks.workunit.client.1.smithi121.stdout:8/907: read d0/d7/d36/d8c/d45/f89 [957186,3895] 0 2024-02-15T18:35:47.915 INFO:tasks.workunit.client.1.smithi121.stdout:8/908: write d0/dce/df8/d129/f11e [71511,51555] 0 2024-02-15T18:35:47.915 INFO:tasks.workunit.client.1.smithi121.stdout:8/909: symlink d0/d61/l132 0 2024-02-15T18:35:47.915 INFO:tasks.workunit.client.1.smithi121.stdout:8/910: write d0/d7/d36/fb9 [4855432,94958] 0 2024-02-15T18:35:47.919 INFO:tasks.workunit.client.1.smithi121.stdout:3/889: dwrite d2/d1a/d27/d63/da4/d25/d30/db3/ddf/f116 [0,4194304] 0 2024-02-15T18:35:47.920 INFO:tasks.workunit.client.1.smithi121.stdout:3/890: write d2/d1a/d27/da7/f53 [600095,113619] 0 2024-02-15T18:35:47.926 INFO:tasks.workunit.client.1.smithi121.stdout:8/911: dread d0/d7/d36/d8c/d45/dcd/ff3 [0,4194304] 0 2024-02-15T18:35:47.926 INFO:tasks.workunit.client.1.smithi121.stdout:8/912: write d0/d7/d36/fba [186430,111284] 0 2024-02-15T18:35:47.930 INFO:tasks.workunit.client.1.smithi121.stdout:8/913: mkdir d0/d61/d133 0 2024-02-15T18:35:47.930 INFO:tasks.workunit.client.1.smithi121.stdout:8/914: write d0/f8 [1375180,74815] 0 2024-02-15T18:35:47.931 INFO:tasks.workunit.client.1.smithi121.stdout:8/915: write d0/d61/d7b/fd8 [2123996,102547] 0 2024-02-15T18:35:47.931 INFO:tasks.workunit.client.1.smithi121.stdout:8/916: dread - d0/d61/d8f/fdd zero size 2024-02-15T18:35:47.938 INFO:tasks.workunit.client.1.smithi121.stdout:3/891: dwrite d2/d1a/d27/da7/d85/d9f/ddd/fe0 [0,4194304] 0 2024-02-15T18:35:47.943 INFO:tasks.workunit.client.1.smithi121.stdout:3/892: rename d2/d1a/d27/d63/da4/d25/d31/d97/daf/d123 to d2/d1a/d54/dc1/d117/d133 0 2024-02-15T18:35:47.943 INFO:tasks.workunit.client.1.smithi121.stdout:3/893: dread - d2/d1a/d27/da7/fb9 zero size 2024-02-15T18:35:47.943 INFO:tasks.workunit.client.1.smithi121.stdout:3/894: fdatasync d2/d1a/d27/d63/da4/d25/fb0 0 2024-02-15T18:35:47.947 INFO:tasks.workunit.client.1.smithi121.stdout:8/917: dread d0/dce/df8/f99 [0,4194304] 0 2024-02-15T18:35:47.976 INFO:tasks.workunit.client.1.smithi121.stdout:3/895: dwrite d2/d1a/d27/d63/da4/d25/d30/f41 [0,4194304] 0 2024-02-15T18:35:47.976 INFO:tasks.workunit.client.1.smithi121.stdout:8/918: dwrite d0/dce/df8/f112 [0,4194304] 0 2024-02-15T18:35:47.977 INFO:tasks.workunit.client.1.smithi121.stdout:3/896: rename d2/d1a/d27/d63/da4/d25/d30/f5b to d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2/f134 0 2024-02-15T18:35:47.983 INFO:tasks.workunit.client.1.smithi121.stdout:8/919: mknod d0/d7/de/d106/c134 0 2024-02-15T18:35:47.983 INFO:tasks.workunit.client.1.smithi121.stdout:3/897: mkdir d2/d1a/d27/d63/da4/d25/d88/d9a/d10c/d135 0 2024-02-15T18:35:47.983 INFO:tasks.workunit.client.1.smithi121.stdout:3/898: fsync d2/d1a/d27/da7/f47 0 2024-02-15T18:35:47.986 INFO:tasks.workunit.client.1.smithi121.stdout:3/899: creat d2/d1a/d27/d63/da4/f136 x:0 0 0 2024-02-15T18:35:47.987 INFO:tasks.workunit.client.1.smithi121.stdout:3/900: write d2/d1a/d27/f77 [885399,77642] 0 2024-02-15T18:35:47.991 INFO:tasks.workunit.client.1.smithi121.stdout:2/812: sync 2024-02-15T18:35:47.991 INFO:tasks.workunit.client.1.smithi121.stdout:7/708: sync 2024-02-15T18:35:47.991 INFO:tasks.workunit.client.1.smithi121.stdout:4/951: sync 2024-02-15T18:35:47.992 INFO:tasks.workunit.client.1.smithi121.stdout:4/952: fdatasync d5/d31/d25/f33 0 2024-02-15T18:35:47.992 INFO:tasks.workunit.client.1.smithi121.stdout:7/709: chown d0/d4a/d5b/d9a/fc9 7037 1 2024-02-15T18:35:47.993 INFO:tasks.workunit.client.1.smithi121.stdout:2/813: mknod d5/df/d60/d98/d22/d81/dac/cf8 0 2024-02-15T18:35:47.993 INFO:tasks.workunit.client.1.smithi121.stdout:2/814: chown d5/df/d60/d98/d58/d63/d94/de0/df0 25510584 1 2024-02-15T18:35:47.993 INFO:tasks.workunit.client.1.smithi121.stdout:4/953: truncate d5/d9/d1d/d26/d72/ff4 482876 0 2024-02-15T18:35:47.994 INFO:tasks.workunit.client.1.smithi121.stdout:2/815: read d5/fc [132042,52384] 0 2024-02-15T18:35:47.996 INFO:tasks.workunit.client.1.smithi121.stdout:4/954: link d5/d9/d1d/d26/d73/d81/d10f/d4f/d51/d71/dca/lfa d5/d9/d1d/d26/d73/l124 0 2024-02-15T18:35:47.996 INFO:tasks.workunit.client.1.smithi121.stdout:7/710: read d0/d5/d11/d5c/fa7 [3926172,77717] 0 2024-02-15T18:35:47.998 INFO:tasks.workunit.client.1.smithi121.stdout:7/711: mkdir d0/d4a/d5b/d9a/de9 0 2024-02-15T18:35:48.001 INFO:tasks.workunit.client.1.smithi121.stdout:7/712: write d0/d5/d11/d1c/f35 [1407813,56051] 0 2024-02-15T18:35:48.001 INFO:tasks.workunit.client.1.smithi121.stdout:7/713: chown d0/d5/d11/dbb/cc8 51890566 1 2024-02-15T18:35:48.002 INFO:tasks.workunit.client.1.smithi121.stdout:7/714: truncate d0/d5/f81 3273832 0 2024-02-15T18:35:48.003 INFO:tasks.workunit.client.1.smithi121.stdout:4/955: rmdir d5/d9/d19/d5c 39 2024-02-15T18:35:48.003 INFO:tasks.workunit.client.1.smithi121.stdout:4/956: chown d5/d9/d19/d4c/d82 6070045 1 2024-02-15T18:35:48.003 INFO:tasks.workunit.client.1.smithi121.stdout:7/715: symlink d0/d4a/de5/lea 0 2024-02-15T18:35:48.004 INFO:tasks.workunit.client.1.smithi121.stdout:7/716: symlink d0/d5/d11/d5c/dcc/leb 0 2024-02-15T18:35:48.005 INFO:tasks.workunit.client.1.smithi121.stdout:4/957: creat d5/d9/d76/d87/f125 x:0 0 0 2024-02-15T18:35:48.005 INFO:tasks.workunit.client.1.smithi121.stdout:7/717: truncate d0/d5/d4c/d6f/fbe 1130420 0 2024-02-15T18:35:48.007 INFO:tasks.workunit.client.1.smithi121.stdout:7/718: creat d0/d5/d4c/d6f/db3/fec x:0 0 0 2024-02-15T18:35:48.008 INFO:tasks.workunit.client.1.smithi121.stdout:7/719: mkdir d0/d5/d11/d1c/d5f/ded 0 2024-02-15T18:35:48.008 INFO:tasks.workunit.client.1.smithi121.stdout:7/720: write d0/d38/d46/daa/f5d [518343,88898] 0 2024-02-15T18:35:48.008 INFO:tasks.workunit.client.1.smithi121.stdout:7/721: truncate d0/d5/d4c/d6f/db3/fb7 645685 0 2024-02-15T18:35:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.028+0000 7f7db54d7700 1 -- 172.21.15.38:0/2298626600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db00f77e0 msgr2=0x7f7db00f7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.028+0000 7f7db54d7700 1 --2- 172.21.15.38:0/2298626600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db00f77e0 0x7f7db00f7c40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f7da4004660 tx=0x7f7da4009b30).stop 2024-02-15T18:35:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.028+0000 7f7db54d7700 1 -- 172.21.15.38:0/2298626600 shutdown_connections 2024-02-15T18:35:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.028+0000 7f7db54d7700 1 --2- 172.21.15.38:0/2298626600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db00f77e0 0x7f7db00f7c40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.028+0000 7f7db54d7700 1 --2- 172.21.15.38:0/2298626600 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db00f6540 0x7f7db00f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.028+0000 7f7db54d7700 1 -- 172.21.15.38:0/2298626600 >> 172.21.15.38:0/2298626600 conn(0x7f7db00f1ca0 msgr2=0x7f7db00f40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:48.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.029+0000 7f7db54d7700 1 -- 172.21.15.38:0/2298626600 shutdown_connections 2024-02-15T18:35:48.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.029+0000 7f7db54d7700 1 -- 172.21.15.38:0/2298626600 wait complete. 2024-02-15T18:35:48.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.029+0000 7f7db54d7700 1 Processor -- start 2024-02-15T18:35:48.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.029+0000 7f7db54d7700 1 -- start start 2024-02-15T18:35:48.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.029+0000 7f7db54d7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db00f6540 0x7f7db0106e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:48.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.029+0000 7f7db54d7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db00f77e0 0x7f7db0107320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:48.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.029+0000 7f7db54d7700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db0107920 con 0x7f7db00f6540 2024-02-15T18:35:48.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.030+0000 7f7db54d7700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db0108130 con 0x7f7db00f77e0 2024-02-15T18:35:48.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.030+0000 7f7daeffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db00f6540 0x7f7db0106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:48.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.030+0000 7f7daeffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db00f6540 0x7f7db0106e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55016/0 (socket says 172.21.15.38:55016) 2024-02-15T18:35:48.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.030+0000 7f7daeffd700 1 -- 172.21.15.38:0/2550916638 learned_addr learned my addr 172.21.15.38:0/2550916638 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:35:48.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.030+0000 7f7dae7fc700 1 --2- 172.21.15.38:0/2550916638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db00f77e0 0x7f7db0107320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:48.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.030+0000 7f7daeffd700 1 -- 172.21.15.38:0/2550916638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db00f77e0 msgr2=0x7f7db0107320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:48.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.030+0000 7f7daeffd700 1 --2- 172.21.15.38:0/2550916638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db00f77e0 0x7f7db0107320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:48.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.030+0000 7f7daeffd700 1 -- 172.21.15.38:0/2550916638 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da4005040 con 0x7f7db00f6540 2024-02-15T18:35:48.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.030+0000 7f7daeffd700 1 --2- 172.21.15.38:0/2550916638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db00f6540 0x7f7db0106e30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f7da000e7c0 tx=0x7f7da000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:48.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.030+0000 7f7d977fe700 1 -- 172.21.15.38:0/2550916638 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7da00092e0 con 0x7f7db00f6540 2024-02-15T18:35:48.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.030+0000 7f7d977fe700 1 -- 172.21.15.38:0/2550916638 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f7da000f040 con 0x7f7db00f6540 2024-02-15T18:35:48.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.030+0000 7f7db54d7700 1 -- 172.21.15.38:0/2550916638 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db01083b0 con 0x7f7db00f6540 2024-02-15T18:35:48.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.030+0000 7f7d977fe700 1 -- 172.21.15.38:0/2550916638 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7da0014920 con 0x7f7db00f6540 2024-02-15T18:35:48.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.030+0000 7f7db54d7700 1 -- 172.21.15.38:0/2550916638 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db0108970 con 0x7f7db00f6540 2024-02-15T18:35:48.024 INFO:tasks.workunit.client.1.smithi121.stdout:3/901: dwrite d2/d1a/f1f [8388608,4194304] 0 2024-02-15T18:35:48.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.031+0000 7f7db54d7700 1 -- 172.21.15.38:0/2550916638 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db0059100 con 0x7f7db00f6540 2024-02-15T18:35:48.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.032+0000 7f7d977fe700 1 -- 172.21.15.38:0/2550916638 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 23) v1 ==== 74714+0+0 (secure 0 0 0) 0x7f7da001b030 con 0x7f7db00f6540 2024-02-15T18:35:48.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.032+0000 7f7d977fe700 1 --2- 172.21.15.38:0/2550916638 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7f7d9805e9c0 0x7f7d98060e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:48.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.033+0000 7f7d977fe700 1 -- 172.21.15.38:0/2550916638 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7da00790b0 con 0x7f7db00f6540 2024-02-15T18:35:48.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.033+0000 7f7dae7fc700 1 --2- 172.21.15.38:0/2550916638 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7f7d9805e9c0 0x7f7d98060e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:48.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.033+0000 7f7dae7fc700 1 --2- 172.21.15.38:0/2550916638 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7f7d9805e9c0 0x7f7d98060e80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f7da4004630 tx=0x7f7da4016040).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:48.026 INFO:tasks.workunit.client.1.smithi121.stdout:8/920: dwrite d0/d7/de/f1a [4194304,4194304] 0 2024-02-15T18:35:48.028 INFO:tasks.workunit.client.1.smithi121.stdout:2/816: dwrite d5/df/d10/d4c/fb2 [0,4194304] 0 2024-02-15T18:35:48.028 INFO:tasks.workunit.client.1.smithi121.stdout:2/817: chown d5/df/d27/d3e/ld9 8 1 2024-02-15T18:35:48.028 INFO:tasks.workunit.client.1.smithi121.stdout:2/818: chown d5/df/d3f/fa7 65371934 1 2024-02-15T18:35:48.028 INFO:tasks.workunit.client.1.smithi121.stdout:2/819: readlink d5/d49/lce 0 2024-02-15T18:35:48.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.037+0000 7f7d977fe700 1 -- 172.21.15.38:0/2550916638 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7f7da0049540 con 0x7f7db00f6540 2024-02-15T18:35:48.031 INFO:tasks.workunit.client.1.smithi121.stdout:2/820: rename d5/df/ff6 to d5/df/d60/d98/d22/de6/ff9 0 2024-02-15T18:35:48.032 INFO:tasks.workunit.client.1.smithi121.stdout:2/821: dread - d5/df/d27/d3e/d9f/fa2 zero size 2024-02-15T18:35:48.032 INFO:tasks.workunit.client.1.smithi121.stdout:4/958: dread d5/d9/d1d/d26/d73/d81/d10f/d4f/d51/d80/ffc [0,4194304] 0 2024-02-15T18:35:48.032 INFO:tasks.workunit.client.1.smithi121.stdout:4/959: chown d5/d9/d1d/d26/d73/d81/d10f/f11a 34187 1 2024-02-15T18:35:48.032 INFO:tasks.workunit.client.1.smithi121.stdout:3/902: write d2/d1a/d54/f59 [3492642,78215] 0 2024-02-15T18:35:48.038 INFO:tasks.workunit.client.1.smithi121.stdout:2/822: mknod d5/df/d3f/cfa 0 2024-02-15T18:35:48.040 INFO:tasks.workunit.client.1.smithi121.stdout:2/823: unlink d5/df/le1 0 2024-02-15T18:35:48.042 INFO:tasks.workunit.client.1.smithi121.stdout:2/824: link d5/ded/cea d5/df/d10/d4c/cfb 0 2024-02-15T18:35:48.042 INFO:tasks.workunit.client.1.smithi121.stdout:2/825: read - d5/df/d60/d98/d58/d63/d94/dc3/fe4 zero size 2024-02-15T18:35:48.043 INFO:tasks.workunit.client.1.smithi121.stdout:2/826: stat d5/df/cf4 0 2024-02-15T18:35:48.043 INFO:tasks.workunit.client.1.smithi121.stdout:4/960: dread d5/d31/d25/f33 [0,4194304] 0 2024-02-15T18:35:48.044 INFO:tasks.workunit.client.1.smithi121.stdout:2/827: rename d5/df/d10/fa9 to d5/df/d60/d98/d58/d63/d94/de0/df0/ffc 0 2024-02-15T18:35:48.044 INFO:tasks.workunit.client.1.smithi121.stdout:2/828: dread - d5/df/d27/d3e/fc5 zero size 2024-02-15T18:35:48.048 INFO:tasks.workunit.client.1.smithi121.stdout:4/961: creat d5/d9/d1d/d26/d73/d81/d10f/d4a/f126 x:0 0 0 2024-02-15T18:35:48.048 INFO:tasks.workunit.client.1.smithi121.stdout:2/829: read d5/f3b [3029713,108777] 0 2024-02-15T18:35:48.050 INFO:tasks.workunit.client.1.smithi121.stdout:2/830: rename d5/df/feb to d5/df/d27/d3a/ffd 0 2024-02-15T18:35:48.050 INFO:tasks.workunit.client.1.smithi121.stdout:2/831: write d5/df/d60/f86 [1363372,61365] 0 2024-02-15T18:35:48.050 INFO:tasks.workunit.client.1.smithi121.stdout:2/832: fsync d5/df/d27/f21 0 2024-02-15T18:35:48.051 INFO:tasks.workunit.client.1.smithi121.stdout:2/833: mkdir d5/d49/d71/dfe 0 2024-02-15T18:35:48.052 INFO:tasks.workunit.client.1.smithi121.stdout:8/921: dwrite d0/d7/d36/d8c/d45/d53/fa0 [0,4194304] 0 2024-02-15T18:35:48.053 INFO:tasks.workunit.client.1.smithi121.stdout:2/834: getdents d5/df/d27/d3e/d6a 0 2024-02-15T18:35:48.053 INFO:tasks.workunit.client.1.smithi121.stdout:4/962: creat d5/d9/d19/d4c/dc3/de6/f127 x:0 0 0 2024-02-15T18:35:48.056 INFO:tasks.workunit.client.1.smithi121.stdout:8/922: creat d0/d48/f135 x:0 0 0 2024-02-15T18:35:48.057 INFO:tasks.workunit.client.1.smithi121.stdout:2/835: creat d5/ded/fff x:0 0 0 2024-02-15T18:35:48.058 INFO:tasks.workunit.client.1.smithi121.stdout:8/923: symlink d0/d7/db1/dbb/de2/l136 0 2024-02-15T18:35:48.063 INFO:tasks.workunit.client.1.smithi121.stdout:4/963: dread d5/d9/d76/d9e/fb9 [0,4194304] 0 2024-02-15T18:35:48.064 INFO:tasks.workunit.client.1.smithi121.stdout:4/964: symlink d5/d9/d1d/d26/d73/d81/d10f/d4f/d51/d63/l128 0 2024-02-15T18:35:48.066 INFO:tasks.workunit.client.1.smithi121.stdout:4/965: mknod d5/c129 0 2024-02-15T18:35:48.066 INFO:tasks.workunit.client.1.smithi121.stdout:4/966: chown d5/d9/d1d/cdc 45 1 2024-02-15T18:35:48.067 INFO:tasks.workunit.client.1.smithi121.stdout:4/967: truncate d5/f6 2916907 0 2024-02-15T18:35:48.070 INFO:tasks.workunit.client.1.smithi121.stdout:3/903: dwrite d2/d1a/d27/d63/da4/d25/d30/db3/f11c [0,4194304] 0 2024-02-15T18:35:48.072 INFO:tasks.workunit.client.1.smithi121.stdout:4/968: dread d5/d9/d1d/d26/d73/d81/ff1 [0,4194304] 0 2024-02-15T18:35:48.073 INFO:tasks.workunit.client.1.smithi121.stdout:4/969: fsync d5/d9/d19/d4c/dc3/de6/fe7 0 2024-02-15T18:35:48.073 INFO:tasks.workunit.client.1.smithi121.stdout:7/722: dwrite d0/d5/f81 [0,4194304] 0 2024-02-15T18:35:48.073 INFO:tasks.workunit.client.1.smithi121.stdout:4/970: mkdir d5/d9/d76/d12a 0 2024-02-15T18:35:48.077 INFO:tasks.workunit.client.1.smithi121.stdout:2/836: dread d5/df/d60/f86 [0,4194304] 0 2024-02-15T18:35:48.077 INFO:tasks.workunit.client.1.smithi121.stdout:2/837: write d5/df/f2b [1844201,121602] 0 2024-02-15T18:35:48.083 INFO:tasks.workunit.client.1.smithi121.stdout:2/838: creat d5/df/d60/d98/d22/d81/dbe/f100 x:0 0 0 2024-02-15T18:35:48.084 INFO:tasks.workunit.client.1.smithi121.stdout:2/839: mkdir d5/df/dc1/d101 0 2024-02-15T18:35:48.084 INFO:tasks.workunit.client.1.smithi121.stdout:2/840: chown d5/df/d27/d3a 1258 1 2024-02-15T18:35:48.084 INFO:tasks.workunit.client.1.smithi121.stdout:2/841: truncate d5/ded/fec 813519 0 2024-02-15T18:35:48.084 INFO:tasks.workunit.client.1.smithi121.stdout:2/842: creat d5/df/d27/f102 x:0 0 0 2024-02-15T18:35:48.085 INFO:tasks.workunit.client.1.smithi121.stdout:2/843: truncate d5/df/d3f/fd2 729128 0 2024-02-15T18:35:48.087 INFO:tasks.workunit.client.1.smithi121.stdout:2/844: mknod d5/df/d60/d98/d58/d63/d94/dc3/c103 0 2024-02-15T18:35:48.087 INFO:tasks.workunit.client.1.smithi121.stdout:2/845: fsync d5/df/d60/d98/d22/f44 0 2024-02-15T18:35:48.087 INFO:tasks.workunit.client.1.smithi121.stdout:8/924: dwrite d0/d7/d36/d8c/de9/fe7 [0,4194304] 0 2024-02-15T18:35:48.088 INFO:tasks.workunit.client.1.smithi121.stdout:8/925: read d0/d7/d36/fcf [27894,5301] 0 2024-02-15T18:35:48.088 INFO:tasks.workunit.client.1.smithi121.stdout:8/926: stat d0/d61/d8f 0 2024-02-15T18:35:48.088 INFO:tasks.workunit.client.1.smithi121.stdout:2/846: creat d5/ded/f104 x:0 0 0 2024-02-15T18:35:48.090 INFO:tasks.workunit.client.1.smithi121.stdout:2/847: mknod d5/df/d60/d98/d58/d63/d94/de0/c105 0 2024-02-15T18:35:48.091 INFO:tasks.workunit.client.1.smithi121.stdout:2/848: truncate d5/df/f1b 1495373 0 2024-02-15T18:35:48.092 INFO:tasks.workunit.client.1.smithi121.stdout:2/849: mknod d5/df/d27/d3a/c106 0 2024-02-15T18:35:48.093 INFO:tasks.workunit.client.1.smithi121.stdout:2/850: mknod d5/df/d10/d4c/c107 0 2024-02-15T18:35:48.096 INFO:tasks.workunit.client.1.smithi121.stdout:2/851: unlink d5/df/d10/d4c/dad/lbd 0 2024-02-15T18:35:48.098 INFO:tasks.workunit.client.1.smithi121.stdout:2/852: unlink d5/df/d60/d98/d58/d63/d94/de0/c105 0 2024-02-15T18:35:48.099 INFO:tasks.workunit.client.1.smithi121.stdout:3/904: dwrite d2/d1a/d6b/f91 [0,4194304] 0 2024-02-15T18:35:48.099 INFO:tasks.workunit.client.1.smithi121.stdout:3/905: chown d2/d1a/d6b/f125 3820484 1 2024-02-15T18:35:48.099 INFO:tasks.workunit.client.1.smithi121.stdout:3/906: read d2/d1a/d27/d63/da4/f19 [1682894,78988] 0 2024-02-15T18:35:48.099 INFO:tasks.workunit.client.1.smithi121.stdout:3/907: dread - d2/d1a/d27/da7/d85/d9f/ddd/de8/f12e zero size 2024-02-15T18:35:48.101 INFO:tasks.workunit.client.1.smithi121.stdout:4/971: dwrite d5/d9/d1d/d26/d73/d81/d10f/d4f/d51/ff2 [0,4194304] 0 2024-02-15T18:35:48.101 INFO:tasks.workunit.client.1.smithi121.stdout:3/908: getdents d2/d1a/d27/d63/da4 0 2024-02-15T18:35:48.102 INFO:tasks.workunit.client.1.smithi121.stdout:3/909: symlink d2/d1a/d54/l137 0 2024-02-15T18:35:48.102 INFO:tasks.workunit.client.1.smithi121.stdout:3/910: fsync d2/d1a/d27/d63/da4/fe4 0 2024-02-15T18:35:48.103 INFO:tasks.workunit.client.1.smithi121.stdout:4/972: rename d5/d9/d1d/d26/d73/d81/d10f/d4a/d7e/fde to d5/d9/d19/d4c/d9a/dcc/f12b 0 2024-02-15T18:35:48.103 INFO:tasks.workunit.client.1.smithi121.stdout:2/853: dread d5/d49/f7f [0,4194304] 0 2024-02-15T18:35:48.104 INFO:tasks.workunit.client.1.smithi121.stdout:4/973: rmdir d5/d9/d76 39 2024-02-15T18:35:48.105 INFO:tasks.workunit.client.1.smithi121.stdout:7/723: dwrite d0/d5/d11/d5c/fa7 [0,4194304] 0 2024-02-15T18:35:48.105 INFO:tasks.workunit.client.1.smithi121.stdout:4/974: mknod d5/d9/d1d/d26/d73/d81/d10f/d4f/d51/d63/c12c 0 2024-02-15T18:35:48.107 INFO:tasks.workunit.client.1.smithi121.stdout:3/911: creat d2/d1a/d27/da7/f138 x:0 0 0 2024-02-15T18:35:48.108 INFO:tasks.workunit.client.1.smithi121.stdout:4/975: link d5/d31/d25/f117 d5/d9/d19/d5c/f12d 0 2024-02-15T18:35:48.111 INFO:tasks.workunit.client.1.smithi121.stdout:7/724: rename d0/d4a/d5b/f65 to d0/d5/d4c/fee 0 2024-02-15T18:35:48.111 INFO:tasks.workunit.client.1.smithi121.stdout:7/725: dread - d0/d38/fb9 zero size 2024-02-15T18:35:48.114 INFO:tasks.workunit.client.1.smithi121.stdout:3/912: write d2/d1a/d27/da7/f35 [1950774,34373] 0 2024-02-15T18:35:48.114 INFO:tasks.workunit.client.1.smithi121.stdout:3/913: fsync d2/d1a/d27/d63/d81/fb8 0 2024-02-15T18:35:48.115 INFO:tasks.workunit.client.1.smithi121.stdout:3/914: creat d2/d1a/d27/f139 x:0 0 0 2024-02-15T18:35:48.115 INFO:tasks.workunit.client.1.smithi121.stdout:3/915: dread - d2/d1a/d6b/d7d/f12a zero size 2024-02-15T18:35:48.117 INFO:tasks.workunit.client.1.smithi121.stdout:2/854: dread d5/df/d27/d3a/f43 [0,4194304] 0 2024-02-15T18:35:48.118 INFO:tasks.workunit.client.1.smithi121.stdout:2/855: symlink d5/df/d10/l108 0 2024-02-15T18:35:48.119 INFO:tasks.workunit.client.1.smithi121.stdout:2/856: dread - d5/df/d60/d98/d22/d81/dbe/f100 zero size 2024-02-15T18:35:48.122 INFO:tasks.workunit.client.1.smithi121.stdout:8/927: dwrite d0/dce/df8/d129/f111 [0,4194304] 0 2024-02-15T18:35:48.123 INFO:tasks.workunit.client.1.smithi121.stdout:8/928: readlink d0/d7/d36/d8c/l40 0 2024-02-15T18:35:48.124 INFO:tasks.workunit.client.1.smithi121.stdout:7/726: dread d0/d5/f15 [0,4194304] 0 2024-02-15T18:35:48.124 INFO:tasks.workunit.client.1.smithi121.stdout:7/727: chown d0/d5/d4c/d6f/db6/fbc 25 1 2024-02-15T18:35:48.126 INFO:tasks.workunit.client.1.smithi121.stdout:7/728: write d0/f29 [1177815,65146] 0 2024-02-15T18:35:48.127 INFO:tasks.workunit.client.1.smithi121.stdout:7/729: readlink d0/d38/d3f/d43/l57 0 2024-02-15T18:35:48.127 INFO:tasks.workunit.client.1.smithi121.stdout:8/929: symlink d0/d61/d66/l137 0 2024-02-15T18:35:48.127 INFO:tasks.workunit.client.1.smithi121.stdout:8/930: dread - d0/d2a/fae zero size 2024-02-15T18:35:48.127 INFO:tasks.workunit.client.1.smithi121.stdout:8/931: read - d0/d2a/d51/fc8 zero size 2024-02-15T18:35:48.129 INFO:tasks.workunit.client.1.smithi121.stdout:7/730: truncate d0/d4a/d5b/da6/fbf 2960166 0 2024-02-15T18:35:48.130 INFO:tasks.workunit.client.1.smithi121.stdout:8/932: mknod d0/dce/df8/d129/c138 0 2024-02-15T18:35:48.130 INFO:tasks.workunit.client.1.smithi121.stdout:3/916: dread d2/d1a/d6b/f91 [0,4194304] 0 2024-02-15T18:35:48.132 INFO:tasks.workunit.client.1.smithi121.stdout:3/917: write d2/d1a/d27/d63/da4/f7b [1646944,79386] 0 2024-02-15T18:35:48.132 INFO:tasks.workunit.client.1.smithi121.stdout:3/918: fsync d2/d1a/d27/d63/da4/d25/d30/f46 0 2024-02-15T18:35:48.132 INFO:tasks.workunit.client.1.smithi121.stdout:7/731: rename d0/d38/d46/daa/f34 to d0/d5/d11/d5c/dcc/dc6/fef 0 2024-02-15T18:35:48.135 INFO:tasks.workunit.client.1.smithi121.stdout:8/933: mkdir d0/d61/d7b/d139 0 2024-02-15T18:35:48.137 INFO:tasks.workunit.client.1.smithi121.stdout:7/732: link d0/d5/d76/c8b d0/d38/d46/daa/dc2/cf0 0 2024-02-15T18:35:48.139 INFO:tasks.workunit.client.1.smithi121.stdout:8/934: creat d0/d126/f13a x:0 0 0 2024-02-15T18:35:48.142 INFO:tasks.workunit.client.1.smithi121.stdout:7/733: creat d0/d5/ff1 x:0 0 0 2024-02-15T18:35:48.142 INFO:tasks.workunit.client.1.smithi121.stdout:7/734: dread - d0/d38/d61/fd7 zero size 2024-02-15T18:35:48.144 INFO:tasks.workunit.client.1.smithi121.stdout:2/857: dwrite d5/df/d27/d3e/d9f/fa2 [0,4194304] 0 2024-02-15T18:35:48.144 INFO:tasks.workunit.client.1.smithi121.stdout:8/935: mknod d0/d7/d36/d8c/de9/d130/dcb/d10c/c13b 0 2024-02-15T18:35:48.144 INFO:tasks.workunit.client.1.smithi121.stdout:8/936: write d0/d7/d36/d8c/de9/d130/dcb/feb [340292,117220] 0 2024-02-15T18:35:48.147 INFO:tasks.workunit.client.1.smithi121.stdout:2/858: truncate d5/df/d60/d98/f75 2535291 0 2024-02-15T18:35:48.149 INFO:tasks.workunit.client.1.smithi121.stdout:8/937: unlink d0/d7/d36/d8c/de9/d130/dcb/fde 0 2024-02-15T18:35:48.150 INFO:tasks.workunit.client.1.smithi121.stdout:2/859: mknod d5/df/d60/d98/d46/c109 0 2024-02-15T18:35:48.150 INFO:tasks.workunit.client.1.smithi121.stdout:7/735: dread d0/d38/fa0 [0,4194304] 0 2024-02-15T18:35:48.151 INFO:tasks.workunit.client.1.smithi121.stdout:8/938: creat d0/d61/d7b/dac/f13c x:0 0 0 2024-02-15T18:35:48.152 INFO:tasks.workunit.client.1.smithi121.stdout:2/860: creat d5/f10a x:0 0 0 2024-02-15T18:35:48.152 INFO:tasks.workunit.client.1.smithi121.stdout:2/861: chown d5/lb3 2907074 1 2024-02-15T18:35:48.155 INFO:tasks.workunit.client.1.smithi121.stdout:7/736: mkdir d0/d5/d76/df2 0 2024-02-15T18:35:48.156 INFO:tasks.workunit.client.1.smithi121.stdout:3/919: dwrite d2/d1a/d6b/f79 [0,4194304] 0 2024-02-15T18:35:48.156 INFO:tasks.workunit.client.1.smithi121.stdout:8/939: mkdir d0/d61/d13d 0 2024-02-15T18:35:48.157 INFO:tasks.workunit.client.1.smithi121.stdout:2/862: getdents d5/df/d60/d98/d22/dc8 0 2024-02-15T18:35:48.157 INFO:tasks.workunit.client.1.smithi121.stdout:2/863: readlink d5/df/d60/d98/d58/d63/la4 0 2024-02-15T18:35:48.157 INFO:tasks.workunit.client.1.smithi121.stdout:2/864: getdents d5/df/df3 0 2024-02-15T18:35:48.157 INFO:tasks.workunit.client.1.smithi121.stdout:2/865: readlink d5/df/d60/d98/d22/d81/dac/ld0 0 2024-02-15T18:35:48.158 INFO:tasks.workunit.client.1.smithi121.stdout:7/737: getdents d0/d5/d11/d1c/d5f/dc0 0 2024-02-15T18:35:48.164 INFO:tasks.workunit.client.1.smithi121.stdout:2/866: unlink d5/df/d27/d3e/fc5 0 2024-02-15T18:35:48.164 INFO:tasks.workunit.client.1.smithi121.stdout:7/738: truncate d0/d38/d61/fe4 501109 0 2024-02-15T18:35:48.165 INFO:tasks.workunit.client.1.smithi121.stdout:2/867: symlink d5/d49/l10b 0 2024-02-15T18:35:48.165 INFO:tasks.workunit.client.1.smithi121.stdout:8/940: dread d0/d3/ff2 [0,4194304] 0 2024-02-15T18:35:48.166 INFO:tasks.workunit.client.1.smithi121.stdout:3/920: dread d2/d1a/d27/d63/da4/d25/d30/db3/ddf/f116 [0,4194304] 0 2024-02-15T18:35:48.167 INFO:tasks.workunit.client.1.smithi121.stdout:7/739: mknod d0/d4a/d5b/da6/cf3 0 2024-02-15T18:35:48.167 INFO:tasks.workunit.client.1.smithi121.stdout:2/868: creat d5/d89/f10c x:0 0 0 2024-02-15T18:35:48.168 INFO:tasks.workunit.client.1.smithi121.stdout:8/941: write d0/d7/d36/fb9 [3211442,924] 0 2024-02-15T18:35:48.169 INFO:tasks.workunit.client.1.smithi121.stdout:7/740: write d0/d38/d3f/d43/f8e [48027,21684] 0 2024-02-15T18:35:48.174 INFO:tasks.workunit.client.1.smithi121.stdout:2/869: truncate d5/df/f1b 666472 0 2024-02-15T18:35:48.174 INFO:tasks.workunit.client.1.smithi121.stdout:2/870: truncate d5/df/d27/d61/fe2 809086 0 2024-02-15T18:35:48.174 INFO:tasks.workunit.client.1.smithi121.stdout:2/871: stat d5/df/d27 0 2024-02-15T18:35:48.177 INFO:tasks.workunit.client.1.smithi121.stdout:8/942: unlink d0/d2a/d51/f6b 0 2024-02-15T18:35:48.177 INFO:tasks.workunit.client.1.smithi121.stdout:8/943: chown d0/d61/l132 89850 1 2024-02-15T18:35:48.180 INFO:tasks.workunit.client.1.smithi121.stdout:7/741: dread d0/f77 [0,4194304] 0 2024-02-15T18:35:48.185 INFO:tasks.workunit.client.1.smithi121.stdout:2/872: creat d5/ded/f10d x:0 0 0 2024-02-15T18:35:48.187 INFO:tasks.workunit.client.1.smithi121.stdout:7/742: symlink d0/d5/d11/d5c/dcc/lf4 0 2024-02-15T18:35:48.192 INFO:tasks.workunit.client.1.smithi121.stdout:3/921: dwrite d2/d1a/d27/d63/da4/d25/d30/fd6 [0,4194304] 0 2024-02-15T18:35:48.192 INFO:tasks.workunit.client.1.smithi121.stdout:7/743: rename d0/d5/d11/d1c/f35 to d0/d4a/d5b/d9a/ff5 0 2024-02-15T18:35:48.192 INFO:tasks.workunit.client.1.smithi121.stdout:7/744: readlink d0/d5/l54 0 2024-02-15T18:35:48.193 INFO:tasks.workunit.client.1.smithi121.stdout:7/745: creat d0/d38/ff6 x:0 0 0 2024-02-15T18:35:48.194 INFO:tasks.workunit.client.1.smithi121.stdout:7/746: write d0/d4a/f7e [1023060,12335] 0 2024-02-15T18:35:48.195 INFO:tasks.workunit.client.1.smithi121.stdout:8/944: dwrite d0/fbd [0,4194304] 0 2024-02-15T18:35:48.196 INFO:tasks.workunit.client.1.smithi121.stdout:8/945: read d0/d7/d36/d8c/d45/f3e [3043146,32349] 0 2024-02-15T18:35:48.196 INFO:tasks.workunit.client.1.smithi121.stdout:8/946: fsync d0/d7/d36/d52/d7a/f8a 0 2024-02-15T18:35:48.196 INFO:tasks.workunit.client.1.smithi121.stdout:8/947: chown d0/d7/de/ce3 775 1 2024-02-15T18:35:48.196 INFO:tasks.workunit.client.1.smithi121.stdout:8/948: truncate d0/d7/d36/d8c/d45/ffe 894067 0 2024-02-15T18:35:48.196 INFO:tasks.workunit.client.1.smithi121.stdout:8/949: chown d0/d7/d36/d52/d7a/f90 115468679 1 2024-02-15T18:35:48.198 INFO:tasks.workunit.client.1.smithi121.stdout:7/747: mkdir d0/d38/d46/daa/df7 0 2024-02-15T18:35:48.198 INFO:tasks.workunit.client.1.smithi121.stdout:3/922: rename d2/d1a/d6b/d72/c10f to d2/d1a/d27/d63/da4/d25/d88/d96/c13a 0 2024-02-15T18:35:48.201 INFO:tasks.workunit.client.1.smithi121.stdout:7/748: creat d0/d5/d11/d5c/dcc/ff8 x:0 0 0 2024-02-15T18:35:48.201 INFO:tasks.workunit.client.1.smithi121.stdout:3/923: creat d2/d1a/d27/f13b x:0 0 0 2024-02-15T18:35:48.201 INFO:tasks.workunit.client.1.smithi121.stdout:3/924: chown d2/d1a/d27/d63/da4/d25/d31/la9 1 1 2024-02-15T18:35:48.202 INFO:tasks.workunit.client.1.smithi121.stdout:7/749: creat d0/d5/d11/dbb/ff9 x:0 0 0 2024-02-15T18:35:48.203 INFO:tasks.workunit.client.1.smithi121.stdout:3/925: mknod d2/d1a/d27/d63/da4/d25/d31/d34/d121/c13c 0 2024-02-15T18:35:48.207 INFO:tasks.workunit.client.1.smithi121.stdout:7/750: mkdir d0/d5/d11/d79/dfa 0 2024-02-15T18:35:48.207 INFO:tasks.workunit.client.1.smithi121.stdout:7/751: fdatasync d0/d5/d11/d5c/fa7 0 2024-02-15T18:35:48.207 INFO:tasks.workunit.client.1.smithi121.stdout:3/926: rmdir d2/d1a/d27/d63/da4/d25/d30/db3 39 2024-02-15T18:35:48.207 INFO:tasks.workunit.client.1.smithi121.stdout:3/927: write d2/d1a/d27/da7/f42 [4411562,65896] 0 2024-02-15T18:35:48.209 INFO:tasks.workunit.client.1.smithi121.stdout:3/928: symlink d2/d1a/d54/l13d 0 2024-02-15T18:35:48.212 INFO:tasks.workunit.client.1.smithi121.stdout:7/752: write d0/d38/d61/fe4 [211505,92191] 0 2024-02-15T18:35:48.212 INFO:tasks.workunit.client.1.smithi121.stdout:7/753: fdatasync d0/d38/fab 0 2024-02-15T18:35:48.212 INFO:tasks.workunit.client.1.smithi121.stdout:2/873: dwrite d5/df/d27/f9b [0,4194304] 0 2024-02-15T18:35:48.212 INFO:tasks.workunit.client.1.smithi121.stdout:2/874: dread - d5/df/d60/d98/d58/d63/fc9 zero size 2024-02-15T18:35:48.218 INFO:tasks.workunit.client.1.smithi121.stdout:7/754: rename d0/d5/d76/df2 to d0/d38/d3f/dfb 0 2024-02-15T18:35:48.219 INFO:tasks.workunit.client.1.smithi121.stdout:3/929: write d2/d1a/d27/d63/da4/d25/d30/f41 [1377736,57196] 0 2024-02-15T18:35:48.219 INFO:tasks.workunit.client.1.smithi121.stdout:3/930: chown d2/d1a/d27/d63/da4/d25/d88/d9a/l11d 12 1 2024-02-15T18:35:48.220 INFO:tasks.workunit.client.1.smithi121.stdout:7/755: getdents d0 0 2024-02-15T18:35:48.222 INFO:tasks.workunit.client.1.smithi121.stdout:8/950: dwrite d0/d7/d36/d8c/d45/fff [0,4194304] 0 2024-02-15T18:35:48.230 INFO:tasks.workunit.client.1.smithi121.stdout:3/931: dread d2/d1a/d27/da7/f35 [0,4194304] 0 2024-02-15T18:35:48.231 INFO:tasks.workunit.client.1.smithi121.stdout:7/756: unlink d0/d4a/d5b/d9a/f9d 0 2024-02-15T18:35:48.233 INFO:tasks.workunit.client.1.smithi121.stdout:8/951: rename d0/d2a/f4b to d0/d7/da7/f13e 0 2024-02-15T18:35:48.233 INFO:tasks.workunit.client.1.smithi121.stdout:3/932: rename d2/d1a/d27/d63/da4/d25 to d2/d1a/d27/d63/da4/d25/d31/d13e 22 2024-02-15T18:35:48.233 INFO:tasks.workunit.client.1.smithi121.stdout:8/952: stat d0/d3/cc6 0 2024-02-15T18:35:48.233 INFO:tasks.workunit.client.1.smithi121.stdout:3/933: chown d2/d1a/d27/d63/da4/d25/d88/d9a/d10c 271963 1 2024-02-15T18:35:48.234 INFO:tasks.workunit.client.1.smithi121.stdout:3/934: read - d2/d1a/d27/d63/da4/d25/d88/d9a/fed zero size 2024-02-15T18:35:48.235 INFO:tasks.workunit.client.1.smithi121.stdout:7/757: creat d0/d4a/d5b/da6/ffc x:0 0 0 2024-02-15T18:35:48.235 INFO:tasks.workunit.client.1.smithi121.stdout:8/953: mkdir d0/d7/da7/d13f 0 2024-02-15T18:35:48.236 INFO:tasks.workunit.client.1.smithi121.stdout:3/935: symlink d2/d1a/d27/d63/da4/d25/d31/d34/l13f 0 2024-02-15T18:35:48.237 INFO:tasks.workunit.client.1.smithi121.stdout:7/758: symlink d0/d38/d46/lfd 0 2024-02-15T18:35:48.238 INFO:tasks.workunit.client.1.smithi121.stdout:8/954: creat d0/d7/de/f140 x:0 0 0 2024-02-15T18:35:48.239 INFO:tasks.workunit.client.1.smithi121.stdout:8/955: fsync d0/d7/de/d67/fbe 0 2024-02-15T18:35:48.239 INFO:tasks.workunit.client.1.smithi121.stdout:3/936: link d2/d1a/d27/d63/d81/fac d2/d1a/d27/d63/da4/d25/d88/d9a/d10c/f140 0 2024-02-15T18:35:48.240 INFO:tasks.workunit.client.1.smithi121.stdout:7/759: truncate d0/d5/d76/fb8 3865766 0 2024-02-15T18:35:48.240 INFO:tasks.workunit.client.1.smithi121.stdout:7/760: dread - d0/d5/d11/dbb/ff9 zero size 2024-02-15T18:35:48.241 INFO:tasks.workunit.client.1.smithi121.stdout:8/956: link d0/d7/d36/d8c/d45/lf1 d0/d7/de/d12f/l141 0 2024-02-15T18:35:48.243 INFO:tasks.workunit.client.1.smithi121.stdout:3/937: rename d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2/d10a/f9c to d2/d1a/d27/d63/da4/d25/d30/db3/ddf/f141 0 2024-02-15T18:35:48.244 INFO:tasks.workunit.client.1.smithi121.stdout:7/761: mknod d0/d38/cfe 0 2024-02-15T18:35:48.246 INFO:tasks.workunit.client.1.smithi121.stdout:8/957: mknod d0/d7/da7/c142 0 2024-02-15T18:35:48.246 INFO:tasks.workunit.client.1.smithi121.stdout:7/762: rmdir d0/d5/d4c/d6f/db6 39 2024-02-15T18:35:48.247 INFO:tasks.workunit.client.1.smithi121.stdout:7/763: creat d0/d5/d4c/d6f/dd5/fff x:0 0 0 2024-02-15T18:35:48.248 INFO:tasks.workunit.client.1.smithi121.stdout:7/764: creat d0/d5/d4c/d6f/db6/f100 x:0 0 0 2024-02-15T18:35:48.252 INFO:tasks.workunit.client.1.smithi121.stdout:3/938: dread d2/d1a/d27/d63/da4/d25/d31/f73 [0,4194304] 0 2024-02-15T18:35:48.252 INFO:tasks.workunit.client.1.smithi121.stdout:3/939: fdatasync d2/d1a/d54/f59 0 2024-02-15T18:35:48.253 INFO:tasks.workunit.client.1.smithi121.stdout:7/765: symlink d0/d5/d4c/dda/l101 0 2024-02-15T18:35:48.254 INFO:tasks.workunit.client.1.smithi121.stdout:8/958: dread d0/f15 [0,4194304] 0 2024-02-15T18:35:48.258 INFO:tasks.workunit.client.1.smithi121.stdout:7/766: creat d0/d5/d11/d5c/dcc/f102 x:0 0 0 2024-02-15T18:35:48.259 INFO:tasks.workunit.client.1.smithi121.stdout:7/767: write d0/d5/d11/d5c/dcc/f4d [883860,67339] 0 2024-02-15T18:35:48.259 INFO:tasks.workunit.client.1.smithi121.stdout:7/768: chown d0/c98 14201 1 2024-02-15T18:35:48.259 INFO:tasks.workunit.client.1.smithi121.stdout:7/769: chown d0/d5/d11/d5c/dcc/leb 2 1 2024-02-15T18:35:48.259 INFO:tasks.workunit.client.1.smithi121.stdout:7/770: chown d0/d5/d4c/d6f/db3/le0 192715 1 2024-02-15T18:35:48.259 INFO:tasks.workunit.client.1.smithi121.stdout:7/771: chown d0/d38/cc7 4 1 2024-02-15T18:35:48.260 INFO:tasks.workunit.client.1.smithi121.stdout:8/959: rename d0/d3/ff2 to d0/d7/db1/f143 0 2024-02-15T18:35:48.261 INFO:tasks.workunit.client.1.smithi121.stdout:7/772: mknod d0/d38/d46/c103 0 2024-02-15T18:35:48.262 INFO:tasks.workunit.client.1.smithi121.stdout:8/960: mknod d0/d61/d13d/c144 0 2024-02-15T18:35:48.263 INFO:tasks.workunit.client.1.smithi121.stdout:7/773: unlink d0/f18 0 2024-02-15T18:35:48.264 INFO:tasks.workunit.client.1.smithi121.stdout:8/961: symlink d0/d7/de/d106/l145 0 2024-02-15T18:35:48.264 INFO:tasks.workunit.client.1.smithi121.stdout:8/962: truncate d0/d7/fdb 1013231 0 2024-02-15T18:35:48.264 INFO:tasks.workunit.client.1.smithi121.stdout:8/963: stat d0/d7/ff4 0 2024-02-15T18:35:48.266 INFO:tasks.workunit.client.1.smithi121.stdout:7/774: creat d0/d5/d11/d5c/dcc/dc6/f104 x:0 0 0 2024-02-15T18:35:48.270 INFO:tasks.workunit.client.1.smithi121.stdout:7/775: rmdir d0/d5/d11/d1c/d5f 39 2024-02-15T18:35:48.274 INFO:tasks.workunit.client.1.smithi121.stdout:7/776: link d0/d5/d11/d79/lb4 d0/d5/d76/l105 0 2024-02-15T18:35:48.274 INFO:tasks.workunit.client.1.smithi121.stdout:7/777: read - d0/d5/d11/d1c/fd3 zero size 2024-02-15T18:35:48.277 INFO:tasks.workunit.client.1.smithi121.stdout:7/778: getdents d0/d5/d4c/d6f/db6 0 2024-02-15T18:35:48.284 INFO:tasks.workunit.client.1.smithi121.stdout:7/779: creat d0/d38/d46/daa/dd6/f106 x:0 0 0 2024-02-15T18:35:48.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.295+0000 7f7db54d7700 1 -- 172.21.15.38:0/2550916638 --> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7db00008d0 con 0x7f7d9805e9c0 2024-02-15T18:35:48.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.297+0000 7f7d977fe700 1 -- 172.21.15.38:0/2550916638 <== mgr.24385 v2:172.21.15.121:6828/330429733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f7db00008d0 con 0x7f7d9805e9c0 2024-02-15T18:35:48.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.300+0000 7f7db54d7700 1 -- 172.21.15.38:0/2550916638 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7f7d9805e9c0 msgr2=0x7f7d98060e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:48.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.300+0000 7f7db54d7700 1 --2- 172.21.15.38:0/2550916638 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7f7d9805e9c0 0x7f7d98060e80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f7da4004630 tx=0x7f7da4016040).stop 2024-02-15T18:35:48.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.300+0000 7f7db54d7700 1 -- 172.21.15.38:0/2550916638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db00f6540 msgr2=0x7f7db0106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:48.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.300+0000 7f7db54d7700 1 --2- 172.21.15.38:0/2550916638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db00f6540 0x7f7db0106e30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f7da000e7c0 tx=0x7f7da000b040).stop 2024-02-15T18:35:48.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.300+0000 7f7db54d7700 1 -- 172.21.15.38:0/2550916638 shutdown_connections 2024-02-15T18:35:48.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.300+0000 7f7db54d7700 1 --2- 172.21.15.38:0/2550916638 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7f7d9805e9c0 0x7f7d98060e80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:48.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.300+0000 7f7db54d7700 1 --2- 172.21.15.38:0/2550916638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7db00f6540 0x7f7db0106e30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:48.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.300+0000 7f7db54d7700 1 --2- 172.21.15.38:0/2550916638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7db00f77e0 0x7f7db0107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:48.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.300+0000 7f7db54d7700 1 -- 172.21.15.38:0/2550916638 >> 172.21.15.38:0/2550916638 conn(0x7f7db00f1ca0 msgr2=0x7f7db00fb230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:48.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.300+0000 7f7db54d7700 1 -- 172.21.15.38:0/2550916638 shutdown_connections 2024-02-15T18:35:48.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.300+0000 7f7db54d7700 1 -- 172.21.15.38:0/2550916638 wait complete. 2024-02-15T18:35:48.294 INFO:tasks.workunit.client.1.smithi121.stdout:7/780: dread d0/f42 [0,4194304] 0 2024-02-15T18:35:48.295 INFO:tasks.workunit.client.1.smithi121.stdout:7/781: dread d0/d4a/da2/fb5 [0,4194304] 0 2024-02-15T18:35:48.295 INFO:tasks.workunit.client.1.smithi121.stdout:3/940: dwrite d2/d1a/f92 [0,4194304] 0 2024-02-15T18:35:48.295 INFO:tasks.workunit.client.1.smithi121.stdout:3/941: readlink d2/d1a/l5d 0 2024-02-15T18:35:48.295 INFO:tasks.workunit.client.1.smithi121.stdout:3/942: fdatasync d2/d1a/d6b/f75 0 2024-02-15T18:35:48.298 INFO:tasks.workunit.client.1.smithi121.stdout:7/782: symlink d0/d5/d4c/d6f/db3/l107 0 2024-02-15T18:35:48.298 INFO:tasks.workunit.client.1.smithi121.stdout:3/943: symlink d2/d1a/d27/da7/d85/d9f/ddd/de8/l142 0 2024-02-15T18:35:48.298 INFO:tasks.workunit.client.1.smithi121.stdout:3/944: mknod d2/d1a/d27/d63/da4/d25/d31/d34/d121/c143 0 2024-02-15T18:35:48.300 INFO:tasks.workunit.client.1.smithi121.stdout:3/945: creat d2/d1a/d27/d63/f144 x:0 0 0 2024-02-15T18:35:48.300 INFO:tasks.workunit.client.1.smithi121.stdout:3/946: symlink d2/d1a/d27/d63/d81/l145 0 2024-02-15T18:35:48.301 INFO:tasks.workunit.client.1.smithi121.stdout:3/947: dread - d2/d1a/d27/d63/da4/d25/d30/db3/dd3/f118 zero size 2024-02-15T18:35:48.301 INFO:tasks.workunit.client.1.smithi121.stdout:3/948: read d2/d1a/d27/d63/da4/d25/d31/d58/f6f [642986,88962] 0 2024-02-15T18:35:48.301 INFO:tasks.workunit.client.1.smithi121.stdout:3/949: getdents d2/d1a/d27/d63/da4/d25/d31 0 2024-02-15T18:35:48.307 INFO:tasks.workunit.client.1.smithi121.stdout:7/783: dread d0/d5/d11/d5c/fdd [4194304,4194304] 0 2024-02-15T18:35:48.307 INFO:tasks.workunit.client.1.smithi121.stdout:7/784: read - d0/d4a/d5b/fa8 zero size 2024-02-15T18:35:48.307 INFO:tasks.workunit.client.1.smithi121.stdout:7/785: stat d0/d4a/de5/lea 0 2024-02-15T18:35:48.307 INFO:tasks.workunit.client.1.smithi121.stdout:7/786: mknod d0/d5/d11/d5c/c108 0 2024-02-15T18:35:48.308 INFO:tasks.workunit.client.1.smithi121.stdout:7/787: symlink d0/d38/d3f/dfb/l109 0 2024-02-15T18:35:48.310 INFO:tasks.workunit.client.1.smithi121.stdout:7/788: write d0/d4a/d5b/da6/fbf [144554,51371] 0 2024-02-15T18:35:48.315 INFO:tasks.workunit.client.1.smithi121.stdout:7/789: dread d0/f42 [0,4194304] 0 2024-02-15T18:35:48.317 INFO:tasks.workunit.client.1.smithi121.stdout:7/790: link d0/d5/d11/d5c/dcc/f9f d0/d4a/f10a 0 2024-02-15T18:35:48.317 INFO:tasks.workunit.client.1.smithi121.stdout:7/791: chown d0/d5/d11/d5c/dcc/leb 402 1 2024-02-15T18:35:48.317 INFO:tasks.workunit.client.1.smithi121.stdout:7/792: fsync d0/d5/d4c/d6f/db3/fec 0 2024-02-15T18:35:48.317 INFO:tasks.workunit.client.1.smithi121.stdout:7/793: stat d0/d5/d4c/d6f/f94 0 2024-02-15T18:35:48.317 INFO:tasks.workunit.client.1.smithi121.stdout:7/794: readlink d0/d5/d11/l30 0 2024-02-15T18:35:48.317 INFO:tasks.workunit.client.1.smithi121.stdout:7/795: read d0/d5/d11/d1c/f1f [4799879,116445] 0 2024-02-15T18:35:48.317 INFO:tasks.workunit.client.1.smithi121.stdout:7/796: mkdir d0/d5/d4c/d6f/d10b 0 2024-02-15T18:35:48.320 INFO:tasks.workunit.client.1.smithi121.stdout:3/950: dwrite d2/d1a/d27/d63/d81/fb8 [0,4194304] 0 2024-02-15T18:35:48.321 INFO:tasks.workunit.client.1.smithi121.stdout:3/951: mknod d2/d1a/d27/da7/d85/d9f/ddd/c146 0 2024-02-15T18:35:48.322 INFO:tasks.workunit.client.1.smithi121.stdout:3/952: write d2/d1a/d27/d63/da4/d25/d30/db3/ddf/f116 [1742261,69136] 0 2024-02-15T18:35:48.322 INFO:tasks.workunit.client.1.smithi121.stdout:3/953: readlink d2/d1a/d54/l137 0 2024-02-15T18:35:48.323 INFO:tasks.workunit.client.1.smithi121.stdout:3/954: rmdir d2/d1a/d54 39 2024-02-15T18:35:48.324 INFO:tasks.workunit.client.1.smithi121.stdout:3/955: rename d2/d1a/d27/d63/da4/d25/d30/db3/cd1 to d2/d1a/d27/d63/da4/d25/d31/d97/daf/c147 0 2024-02-15T18:35:48.324 INFO:tasks.workunit.client.1.smithi121.stdout:3/956: fsync d2/d1a/d27/fa2 0 2024-02-15T18:35:48.325 INFO:tasks.workunit.client.1.smithi121.stdout:3/957: mknod d2/d5a/c148 0 2024-02-15T18:35:48.327 INFO:tasks.workunit.client.1.smithi121.stdout:3/958: chown d2/d1a/d27/da7/c23 13492 1 2024-02-15T18:35:48.330 INFO:tasks.workunit.client.1.smithi121.stdout:3/959: rename d2/d1a/d27/d63/da4/d25/d31/f43 to d2/d1a/d54/f149 0 2024-02-15T18:35:48.330 INFO:tasks.workunit.client.1.smithi121.stdout:3/960: chown d2/d1a/d27/d63/da4/d25/d31/d34/cb6 1 1 2024-02-15T18:35:48.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.412+0000 7faaa7622700 1 -- 172.21.15.38:0/835357252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa00f6860 msgr2=0x7faaa00f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:48.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.412+0000 7faaa7622700 1 --2- 172.21.15.38:0/835357252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa00f6860 0x7faaa00f6cc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7faa90004660 tx=0x7faa90009b30).stop 2024-02-15T18:35:48.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.412+0000 7faaa7622700 1 -- 172.21.15.38:0/835357252 shutdown_connections 2024-02-15T18:35:48.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.412+0000 7faaa7622700 1 --2- 172.21.15.38:0/835357252 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa00f6860 0x7faaa00f6cc0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:48.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.412+0000 7faaa7622700 1 --2- 172.21.15.38:0/835357252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faaa00f55c0 0x7faaa00f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:48.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.412+0000 7faaa7622700 1 -- 172.21.15.38:0/835357252 >> 172.21.15.38:0/835357252 conn(0x7faaa00f0d40 msgr2=0x7faaa00f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:48.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.412+0000 7faaa7622700 1 -- 172.21.15.38:0/835357252 shutdown_connections 2024-02-15T18:35:48.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.412+0000 7faaa7622700 1 -- 172.21.15.38:0/835357252 wait complete. 2024-02-15T18:35:48.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.412+0000 7faaa7622700 1 Processor -- start 2024-02-15T18:35:48.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.412+0000 7faaa7622700 1 -- start start 2024-02-15T18:35:48.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.413+0000 7faaa7622700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faaa00f55c0 0x7faaa0106fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:48.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.413+0000 7faaa7622700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa00f6860 0x7faaa01074c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:48.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.413+0000 7faaa7622700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa0107a70 con 0x7faaa00f6860 2024-02-15T18:35:48.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.413+0000 7faaa7622700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa0107bb0 con 0x7faaa00f55c0 2024-02-15T18:35:48.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.413+0000 7faaa4bbd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa00f6860 0x7faaa01074c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:48.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.413+0000 7faaa4bbd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa00f6860 0x7faaa01074c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55038/0 (socket says 172.21.15.38:55038) 2024-02-15T18:35:48.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.413+0000 7faaa4bbd700 1 -- 172.21.15.38:0/2076494403 learned_addr learned my addr 172.21.15.38:0/2076494403 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:35:48.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.413+0000 7faaa53be700 1 --2- 172.21.15.38:0/2076494403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faaa00f55c0 0x7faaa0106fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:48.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.413+0000 7faaa4bbd700 1 -- 172.21.15.38:0/2076494403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faaa00f55c0 msgr2=0x7faaa0106fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:48.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.413+0000 7faaa4bbd700 1 --2- 172.21.15.38:0/2076494403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faaa00f55c0 0x7faaa0106fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.413+0000 7faaa4bbd700 1 -- 172.21.15.38:0/2076494403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa90005040 con 0x7faaa00f6860 2024-02-15T18:35:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.413+0000 7faaa4bbd700 1 --2- 172.21.15.38:0/2076494403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa00f6860 0x7faaa01074c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7faa90000c00 tx=0x7faa9000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.413+0000 7faa95ffb700 1 -- 172.21.15.38:0/2076494403 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa90019070 con 0x7faaa00f6860 2024-02-15T18:35:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.413+0000 7faa95ffb700 1 -- 172.21.15.38:0/2076494403 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7faa90007e80 con 0x7faaa00f6860 2024-02-15T18:35:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.413+0000 7faaa7622700 1 -- 172.21.15.38:0/2076494403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa01084f0 con 0x7faaa00f6860 2024-02-15T18:35:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.413+0000 7faa95ffb700 1 -- 172.21.15.38:0/2076494403 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa90014db0 con 0x7faaa00f6860 2024-02-15T18:35:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.414+0000 7faaa7622700 1 -- 172.21.15.38:0/2076494403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa0108a50 con 0x7faaa00f6860 2024-02-15T18:35:48.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.415+0000 7faaa7622700 1 -- 172.21.15.38:0/2076494403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa0045ad0 con 0x7faaa00f6860 2024-02-15T18:35:48.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.415+0000 7faa95ffb700 1 -- 172.21.15.38:0/2076494403 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 23) v1 ==== 74714+0+0 (secure 0 0 0) 0x7faa90025070 con 0x7faaa00f6860 2024-02-15T18:35:48.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.416+0000 7faa95ffb700 1 --2- 172.21.15.38:0/2076494403 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7faa8c05e900 0x7faa8c060dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:48.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.416+0000 7faa95ffb700 1 -- 172.21.15.38:0/2076494403 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5554+0+0 (secure 0 0 0) 0x7faa90079f50 con 0x7faaa00f6860 2024-02-15T18:35:48.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.416+0000 7faaa53be700 1 --2- 172.21.15.38:0/2076494403 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7faa8c05e900 0x7faa8c060dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:48.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.416+0000 7faaa53be700 1 --2- 172.21.15.38:0/2076494403 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7faa8c05e900 0x7faa8c060dc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7faa9c009fd0 tx=0x7faa9c00b040).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:48.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.420+0000 7faa95ffb700 1 -- 172.21.15.38:0/2076494403 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7faa900a8ad0 con 0x7faaa00f6860 2024-02-15T18:35:48.454 INFO:tasks.workunit.client.1.smithi121.stdout:4/976: sync 2024-02-15T18:35:48.456 INFO:tasks.workunit.client.1.smithi121.stdout:4/977: mkdir d5/d31/d25/db6/d12e 0 2024-02-15T18:35:48.457 INFO:tasks.workunit.client.1.smithi121.stdout:4/978: truncate d5/d9/d1d/d26/d73/d81/d10f/d4f/d51/ff2 429298 0 2024-02-15T18:35:48.458 INFO:tasks.workunit.client.1.smithi121.stdout:4/979: mkdir d5/d9/d1d/d26/d73/d81/d10f/d4a/d12f 0 2024-02-15T18:35:48.458 INFO:tasks.workunit.client.1.smithi121.stdout:4/980: readlink d5/d9/d19/da0/le2 0 2024-02-15T18:35:48.458 INFO:tasks.workunit.client.1.smithi121.stdout:4/981: fdatasync d5/d31/d25/db6/fd1 0 2024-02-15T18:35:48.458 INFO:tasks.workunit.client.1.smithi121.stdout:4/982: dread d5/d9/d19/d2a/f96 [0,4194304] 0 2024-02-15T18:35:48.479 INFO:tasks.workunit.client.1.smithi121.stdout:4/983: dwrite d5/d9/d19/d5c/f12d [0,4194304] 0 2024-02-15T18:35:48.487 INFO:tasks.workunit.client.1.smithi121.stdout:4/984: unlink d5/d31/d25/c90 0 2024-02-15T18:35:48.487 INFO:tasks.workunit.client.1.smithi121.stdout:4/985: creat d5/d103/f130 x:0 0 0 2024-02-15T18:35:48.490 INFO:tasks.workunit.client.1.smithi121.stdout:4/986: truncate d5/d9/d76/d9e/fb9 812309 0 2024-02-15T18:35:48.490 INFO:tasks.workunit.client.1.smithi121.stdout:4/987: write d5/d9/d76/ff3 [1152988,93445] 0 2024-02-15T18:35:48.510 INFO:tasks.workunit.client.1.smithi121.stdout:4/988: dwrite d5/d9/d1d/d26/d73/d81/d10f/d4f/d51/d71/f48 [0,4194304] 0 2024-02-15T18:35:48.529 INFO:tasks.workunit.client.1.smithi121.stdout:4/989: dwrite d5/d9/d1d/d26/d72/f116 [0,4194304] 0 2024-02-15T18:35:48.529 INFO:tasks.workunit.client.1.smithi121.stdout:4/990: fsync d5/d9/d19/d4c/d9a/dcc/f12b 0 2024-02-15T18:35:48.531 INFO:tasks.workunit.client.1.smithi121.stdout:4/991: write d5/d9/fdb [3819121,127016] 0 2024-02-15T18:35:48.532 INFO:tasks.workunit.client.1.smithi121.stdout:4/992: symlink d5/d9/d76/l131 0 2024-02-15T18:35:48.534 INFO:tasks.workunit.client.1.smithi121.stdout:4/993: rename d5/c83 to d5/d103/c132 0 2024-02-15T18:35:48.535 INFO:tasks.workunit.client.1.smithi121.stdout:4/994: symlink d5/d9/d1d/d26/d73/d81/d10f/d4f/d51/d71/l133 0 2024-02-15T18:35:48.537 INFO:tasks.workunit.client.1.smithi121.stdout:4/995: link d5/d9/ff7 d5/d9/d19/f134 0 2024-02-15T18:35:48.554 INFO:tasks.workunit.client.1.smithi121.stdout:4/996: dwrite d5/d103/f130 [0,4194304] 0 2024-02-15T18:35:48.558 INFO:tasks.workunit.client.1.smithi121.stdout:4/997: unlink d5/d9/d19/d4c/dc3/de6/fe7 0 2024-02-15T18:35:48.559 INFO:tasks.workunit.client.1.smithi121.stdout:4/998: read - d5/d9/d19/d4c/d9a/dcc/f12b zero size 2024-02-15T18:35:48.559 INFO:tasks.workunit.client.1.smithi121.stdout:4/999: chown d5/d9/d19/d4c/c50 13549202 1 2024-02-15T18:35:48.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.644+0000 7faaa7622700 1 -- 172.21.15.38:0/2076494403 --> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faaa005fd30 con 0x7faa8c05e900 2024-02-15T18:35:48.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.654+0000 7faa95ffb700 1 -- 172.21.15.38:0/2076494403 <== mgr.24385 v2:172.21.15.121:6828/330429733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7faaa005fd30 con 0x7faa8c05e900 2024-02-15T18:35:48.645 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:35:48.645 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (6m) 22s ago 8m 15.4M - 0.20.0 0881eb8f169f c2034c318e7c 2024-02-15T18:35:48.645 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (8m) 22s ago 8m 6395k - 16.2.4 8d91d370c2b8 0fd3f945d009 2024-02-15T18:35:48.645 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi121 smithi121 running (7m) 14s ago 7m 6405k - 16.2.4 8d91d370c2b8 affc02db354e 2024-02-15T18:35:48.645 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (6m) 22s ago 8m 33.4M - 6.7.4 557c83e11646 2dcccfbfbd2a 2024-02-15T18:35:48.645 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.hxoggl smithi038 running (2m) 22s ago 2m 13.1M - 16.2.4 8d91d370c2b8 be2533754b9c 2024-02-15T18:35:48.645 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.qgkzgn smithi038 running (2m) 22s ago 2m 13.0M - 16.2.4 8d91d370c2b8 dd3ffafba9bd 2024-02-15T18:35:48.646 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.mvumlh smithi121 running (2m) 14s ago 2m 165M - 16.2.4 8d91d370c2b8 88b7d876a39b 2024-02-15T18:35:48.646 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.ovuzki smithi121 running (2m) 14s ago 2m 131M - 16.2.4 8d91d370c2b8 21cacf0012f9 2024-02-15T18:35:48.646 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.khesbl smithi038 *:9283 running (10m) 22s ago 10m 60.8M - 16.2.4 8d91d370c2b8 b0d099c255f4 2024-02-15T18:35:48.646 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi121.ckiryi smithi121 *:8443,9283 running (68s) 14s ago 7m 414M - 16.2.14-521-g0e714d9a 1fbc92c859dc b624ee90b80a 2024-02-15T18:35:48.646 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (10m) 22s ago 10m 49.9M 2048M 16.2.4 8d91d370c2b8 01b30eaebcfe 2024-02-15T18:35:48.646 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi121 smithi121 running (7m) 14s ago 7m 44.6M 2048M 16.2.4 8d91d370c2b8 8f588ecacb71 2024-02-15T18:35:48.646 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (8m) 22s ago 8m 17.1M - 0.18.1 e5a616e4b9cf 037f60e38158 2024-02-15T18:35:48.646 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi121 smithi121 *:9100 running (7m) 14s ago 7m 17.2M - 0.18.1 e5a616e4b9cf a867bff85e18 2024-02-15T18:35:48.646 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (6m) 22s ago 6m 276M 4096M 16.2.4 8d91d370c2b8 51512003a2ef 2024-02-15T18:35:48.646 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (6m) 22s ago 6m 310M 4096M 16.2.4 8d91d370c2b8 16dbdf20b170 2024-02-15T18:35:48.646 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (5m) 22s ago 5m 239M 4096M 16.2.4 8d91d370c2b8 6d6ee813e5be 2024-02-15T18:35:48.646 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi121 running (5m) 14s ago 5m 334M 4096M 16.2.4 8d91d370c2b8 cba2d9ed4730 2024-02-15T18:35:48.646 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi121 running (5m) 14s ago 5m 298M 4096M 16.2.4 8d91d370c2b8 ac17ad7dc628 2024-02-15T18:35:48.646 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi121 running (4m) 14s ago 4m 286M 4096M 16.2.4 8d91d370c2b8 cdc8c669c4db 2024-02-15T18:35:48.646 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 starting - - - - 2024-02-15T18:35:48.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.656+0000 7faaa7622700 1 -- 172.21.15.38:0/2076494403 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7faa8c05e900 msgr2=0x7faa8c060dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:48.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.656+0000 7faaa7622700 1 --2- 172.21.15.38:0/2076494403 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7faa8c05e900 0x7faa8c060dc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7faa9c009fd0 tx=0x7faa9c00b040).stop 2024-02-15T18:35:48.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.656+0000 7faaa7622700 1 -- 172.21.15.38:0/2076494403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa00f6860 msgr2=0x7faaa01074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:48.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.656+0000 7faaa7622700 1 --2- 172.21.15.38:0/2076494403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa00f6860 0x7faaa01074c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7faa90000c00 tx=0x7faa9000b040).stop 2024-02-15T18:35:48.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.656+0000 7faaa7622700 1 -- 172.21.15.38:0/2076494403 shutdown_connections 2024-02-15T18:35:48.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.656+0000 7faaa7622700 1 --2- 172.21.15.38:0/2076494403 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7faa8c05e900 0x7faa8c060dc0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:48.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.656+0000 7faaa7622700 1 --2- 172.21.15.38:0/2076494403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faaa00f55c0 0x7faaa0106fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:48.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.656+0000 7faaa7622700 1 --2- 172.21.15.38:0/2076494403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa00f6860 0x7faaa01074c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:48.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.656+0000 7faaa7622700 1 -- 172.21.15.38:0/2076494403 >> 172.21.15.38:0/2076494403 conn(0x7faaa00f0d40 msgr2=0x7faaa005f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:48.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.656+0000 7faaa7622700 1 -- 172.21.15.38:0/2076494403 shutdown_connections 2024-02-15T18:35:48.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.656+0000 7faaa7622700 1 -- 172.21.15.38:0/2076494403 wait complete. 2024-02-15T18:35:48.766 INFO:tasks.workunit.client.1.smithi121.stdout:2/875: sync 2024-02-15T18:35:48.767 INFO:tasks.workunit.client.1.smithi121.stdout:2/876: creat d5/df/d27/f10e x:0 0 0 2024-02-15T18:35:48.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.787+0000 7fdfc031b700 1 -- 172.21.15.38:0/3148752814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfb80f6860 msgr2=0x7fdfb80f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:48.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.787+0000 7fdfc031b700 1 --2- 172.21.15.38:0/3148752814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfb80f6860 0x7fdfb80f6cc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fdfa8004660 tx=0x7fdfa8009b30).stop 2024-02-15T18:35:48.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.787+0000 7fdfc031b700 1 -- 172.21.15.38:0/3148752814 shutdown_connections 2024-02-15T18:35:48.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.787+0000 7fdfc031b700 1 --2- 172.21.15.38:0/3148752814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfb80f6860 0x7fdfb80f6cc0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:48.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.787+0000 7fdfc031b700 1 --2- 172.21.15.38:0/3148752814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfb80f55c0 0x7fdfb80f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:48.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.787+0000 7fdfc031b700 1 -- 172.21.15.38:0/3148752814 >> 172.21.15.38:0/3148752814 conn(0x7fdfb80f0d40 msgr2=0x7fdfb80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:48.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.787+0000 7fdfc031b700 1 -- 172.21.15.38:0/3148752814 shutdown_connections 2024-02-15T18:35:48.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.787+0000 7fdfc031b700 1 -- 172.21.15.38:0/3148752814 wait complete. 2024-02-15T18:35:48.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.788+0000 7fdfc031b700 1 Processor -- start 2024-02-15T18:35:48.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.788+0000 7fdfc031b700 1 -- start start 2024-02-15T18:35:48.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.788+0000 7fdfc031b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfb80f55c0 0x7fdfb8105eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:48.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.788+0000 7fdfc031b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfb80f6860 0x7fdfb81063a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:48.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.788+0000 7fdfc031b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb8106950 con 0x7fdfb80f6860 2024-02-15T18:35:48.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.788+0000 7fdfc031b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb8106a90 con 0x7fdfb80f55c0 2024-02-15T18:35:48.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.788+0000 7fdfbd8b6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfb80f6860 0x7fdfb81063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:48.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.788+0000 7fdfbe0b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfb80f55c0 0x7fdfb8105eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:48.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.788+0000 7fdfbd8b6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfb80f6860 0x7fdfb81063a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55046/0 (socket says 172.21.15.38:55046) 2024-02-15T18:35:48.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.788+0000 7fdfbd8b6700 1 -- 172.21.15.38:0/3103763228 learned_addr learned my addr 172.21.15.38:0/3103763228 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:35:48.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.789+0000 7fdfbd8b6700 1 -- 172.21.15.38:0/3103763228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfb80f55c0 msgr2=0x7fdfb8105eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:48.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.789+0000 7fdfbd8b6700 1 --2- 172.21.15.38:0/3103763228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfb80f55c0 0x7fdfb8105eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:48.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.789+0000 7fdfbd8b6700 1 -- 172.21.15.38:0/3103763228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa8005040 con 0x7fdfb80f6860 2024-02-15T18:35:48.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.789+0000 7fdfbe0b7700 1 --2- 172.21.15.38:0/3103763228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfb80f55c0 0x7fdfb8105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:35:48.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.789+0000 7fdfbd8b6700 1 --2- 172.21.15.38:0/3103763228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfb80f6860 0x7fdfb81063a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fdfa8000c00 tx=0x7fdfa800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:48.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.789+0000 7fdfaeffd700 1 -- 172.21.15.38:0/3103763228 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfa801b070 con 0x7fdfb80f6860 2024-02-15T18:35:48.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.789+0000 7fdfaeffd700 1 -- 172.21.15.38:0/3103763228 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fdfa8007e80 con 0x7fdfb80f6860 2024-02-15T18:35:48.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.789+0000 7fdfc031b700 1 -- 172.21.15.38:0/3103763228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb81073d0 con 0x7fdfb80f6860 2024-02-15T18:35:48.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.789+0000 7fdfaeffd700 1 -- 172.21.15.38:0/3103763228 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfa8014db0 con 0x7fdfb80f6860 2024-02-15T18:35:48.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.789+0000 7fdfc031b700 1 -- 172.21.15.38:0/3103763228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb8107930 con 0x7fdfb80f6860 2024-02-15T18:35:48.782 INFO:tasks.workunit.client.1.smithi121.stdout:2/877: dwrite d5/d49/f84 [0,4194304] 0 2024-02-15T18:35:48.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.790+0000 7fdfc031b700 1 -- 172.21.15.38:0/3103763228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb8045ad0 con 0x7fdfb80f6860 2024-02-15T18:35:48.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.790+0000 7fdfaeffd700 1 -- 172.21.15.38:0/3103763228 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 23) v1 ==== 74714+0+0 (secure 0 0 0) 0x7fdfa8027070 con 0x7fdfb80f6860 2024-02-15T18:35:48.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.791+0000 7fdfaeffd700 1 --2- 172.21.15.38:0/3103763228 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7fdfa405e960 0x7fdfa4060e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:48.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.791+0000 7fdfaeffd700 1 -- 172.21.15.38:0/3103763228 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fdfa807bc00 con 0x7fdfb80f6860 2024-02-15T18:35:48.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.791+0000 7fdfbe0b7700 1 --2- 172.21.15.38:0/3103763228 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7fdfa405e960 0x7fdfa4060e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:48.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.791+0000 7fdfbe0b7700 1 --2- 172.21.15.38:0/3103763228 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7fdfa405e960 0x7fdfa4060e20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fdfb4009fd0 tx=0x7fdfb400b040).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:48.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:48.795+0000 7fdfaeffd700 1 -- 172.21.15.38:0/3103763228 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7fdfa8081020 con 0x7fdfb80f6860 2024-02-15T18:35:48.789 INFO:tasks.workunit.client.1.smithi121.stdout:2/878: dread d5/df/d27/d3e/f62 [0,4194304] 0 2024-02-15T18:35:48.790 INFO:tasks.workunit.client.1.smithi121.stdout:2/879: unlink d5/d49/f7f 0 2024-02-15T18:35:48.795 INFO:tasks.workunit.client.1.smithi121.stdout:2/880: dread d5/f3b [0,4194304] 0 2024-02-15T18:35:48.797 INFO:tasks.workunit.client.1.smithi121.stdout:2/881: unlink d5/df/d10/d4c/c107 0 2024-02-15T18:35:48.797 INFO:tasks.workunit.client.1.smithi121.stdout:2/882: readlink d5/df/d60/d98/d22/d81/d9a/lbf 0 2024-02-15T18:35:48.797 INFO:tasks.workunit.client.1.smithi121.stdout:2/883: mkdir d5/df/dc1/d101/d10f 0 2024-02-15T18:35:48.816 INFO:tasks.workunit.client.1.smithi121.stdout:2/884: dwrite d5/df/d10/d4c/dad/fdd [0,4194304] 0 2024-02-15T18:35:48.825 INFO:tasks.workunit.client.1.smithi121.stdout:2/885: dread d5/df/d60/d98/d22/d81/d9a/da1/fab [0,4194304] 0 2024-02-15T18:35:48.836 INFO:tasks.workunit.client.1.smithi121.stdout:8/964: sync 2024-02-15T18:35:48.857 INFO:tasks.workunit.client.1.smithi121.stdout:8/965: dwrite d0/d61/d66/fab [4194304,4194304] 0 2024-02-15T18:35:48.858 INFO:tasks.workunit.client.1.smithi121.stdout:8/966: getdents d0/d126 0 2024-02-15T18:35:48.860 INFO:tasks.workunit.client.1.smithi121.stdout:8/967: truncate d0/d7/d36/d8c/d45/fff 1585565 0 2024-02-15T18:35:48.860 INFO:tasks.workunit.client.1.smithi121.stdout:8/968: write d0/d7/de/f1a [8773206,292] 0 2024-02-15T18:35:48.861 INFO:tasks.workunit.client.1.smithi121.stdout:8/969: mkdir d0/d7/d36/d8c/d45/dcd/d146 0 2024-02-15T18:35:48.862 INFO:tasks.workunit.client.1.smithi121.stdout:8/970: truncate d0/d7/d36/d8c/d45/dcd/ff3 2917370 0 2024-02-15T18:35:48.885 INFO:tasks.workunit.client.1.smithi121.stdout:8/971: dwrite d0/d3/f56 [0,4194304] 0 2024-02-15T18:35:48.885 INFO:tasks.workunit.client.1.smithi121.stdout:8/972: chown d0/d2a/c62 266947187 1 2024-02-15T18:35:48.885 INFO:tasks.workunit.client.1.smithi121.stdout:8/973: write d0/d7/d36/d8c/d45/fcc [8777533,77826] 0 2024-02-15T18:35:48.886 INFO:tasks.workunit.client.1.smithi121.stdout:8/974: truncate d0/d7/d36/d8c/d45/f127 1887810 0 2024-02-15T18:35:48.887 INFO:tasks.workunit.client.1.smithi121.stdout:8/975: truncate d0/d7/d36/d8c/de9/d130/dcb/f10d 3275108 0 2024-02-15T18:35:48.887 INFO:tasks.workunit.client.1.smithi121.stdout:8/976: dread - d0/dce/df8/d129/f116 zero size 2024-02-15T18:35:48.887 INFO:tasks.workunit.client.1.smithi121.stdout:8/977: chown d0/d7/de 72944 1 2024-02-15T18:35:48.888 INFO:tasks.workunit.client.1.smithi121.stdout:8/978: mkdir d0/d7/d36/d8c/d147 0 2024-02-15T18:35:48.888 INFO:tasks.workunit.client.1.smithi121.stdout:8/979: truncate d0/d7/de/f54 567719 0 2024-02-15T18:35:48.889 INFO:tasks.workunit.client.1.smithi121.stdout:8/980: getdents d0/d61/d8f 0 2024-02-15T18:35:48.891 INFO:tasks.workunit.client.1.smithi121.stdout:8/981: write d0/d7/f10 [1641327,45292] 0 2024-02-15T18:35:48.891 INFO:tasks.workunit.client.1.smithi121.stdout:8/982: fdatasync d0/d61/d66/fab 0 2024-02-15T18:35:48.893 INFO:tasks.workunit.client.1.smithi121.stdout:8/983: mknod d0/d7/d36/d8c/de9/d130/dcb/d102/c148 0 2024-02-15T18:35:48.894 INFO:tasks.workunit.client.1.smithi121.stdout:8/984: write d0/d61/d7b/fd8 [1770578,82840] 0 2024-02-15T18:35:48.899 INFO:tasks.workunit.client.1.smithi121.stdout:3/961: sync 2024-02-15T18:35:48.899 INFO:tasks.workunit.client.1.smithi121.stdout:7/797: sync 2024-02-15T18:35:48.899 INFO:tasks.workunit.client.1.smithi121.stdout:3/962: fsync d2/d1a/d54/f149 0 2024-02-15T18:35:48.899 INFO:tasks.workunit.client.1.smithi121.stdout:3/963: truncate d2/d1a/d27/f6c 4963708 0 2024-02-15T18:35:48.900 INFO:tasks.workunit.client.1.smithi121.stdout:7/798: rename d0/d38/ff6 to d0/d5/d76/f10c 0 2024-02-15T18:35:48.900 INFO:tasks.workunit.client.1.smithi121.stdout:7/799: fdatasync d0/d5/f81 0 2024-02-15T18:35:48.911 INFO:tasks.workunit.client.1.smithi121.stdout:8/985: truncate d0/d7/d36/d8c/d45/dcd/ff3 1454435 0 2024-02-15T18:35:48.912 INFO:tasks.workunit.client.1.smithi121.stdout:8/986: mknod d0/d7/d36/d8c/c149 0 2024-02-15T18:35:48.912 INFO:tasks.workunit.client.1.smithi121.stdout:8/987: stat d0/d7/d36/d8c/d45/f9e 0 2024-02-15T18:35:48.913 INFO:tasks.workunit.client.1.smithi121.stdout:7/800: dread d0/d4a/d5b/d9a/ff5 [0,4194304] 0 2024-02-15T18:35:48.916 INFO:tasks.workunit.client.1.smithi121.stdout:8/988: symlink d0/d126/l14a 0 2024-02-15T18:35:48.917 INFO:tasks.workunit.client.1.smithi121.stdout:7/801: creat d0/d5/d4c/d6f/db6/f10d x:0 0 0 2024-02-15T18:35:48.918 INFO:tasks.workunit.client.1.smithi121.stdout:3/964: dwrite d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2/d10a/fa0 [0,4194304] 0 2024-02-15T18:35:48.919 INFO:tasks.workunit.client.1.smithi121.stdout:7/802: link d0/d5/d4c/c62 d0/d5/d11/d1c/d5f/c10e 0 2024-02-15T18:35:48.919 INFO:tasks.workunit.client.1.smithi121.stdout:7/803: fsync d0/d5/d11/d5c/dcc/f4d 0 2024-02-15T18:35:48.919 INFO:tasks.workunit.client.1.smithi121.stdout:7/804: write d0/d5/d76/f10c [797656,75403] 0 2024-02-15T18:35:48.919 INFO:tasks.workunit.client.1.smithi121.stdout:7/805: chown d0/d38/d46/l5a 0 1 2024-02-15T18:35:48.921 INFO:tasks.workunit.client.1.smithi121.stdout:3/965: mknod d2/d1a/d27/d63/da4/d25/d88/c14a 0 2024-02-15T18:35:48.921 INFO:tasks.workunit.client.1.smithi121.stdout:3/966: readlink d2/d1a/d27/d63/da4/d25/d31/l4f 0 2024-02-15T18:35:48.924 INFO:tasks.workunit.client.1.smithi121.stdout:3/967: dread d2/d1a/d27/d63/da4/d25/d31/d97/fae [0,4194304] 0 2024-02-15T18:35:48.933 INFO:tasks.workunit.client.1.smithi121.stdout:3/968: dread d2/d1a/d6b/f82 [0,4194304] 0 2024-02-15T18:35:48.935 INFO:tasks.workunit.client.1.smithi121.stdout:3/969: link d2/d1a/d54/c9d d2/d1a/d27/d63/da4/d25/d30/db3/c14b 0 2024-02-15T18:35:48.935 INFO:tasks.workunit.client.1.smithi121.stdout:8/989: dwrite d0/d7/db1/f143 [4194304,4194304] 0 2024-02-15T18:35:48.938 INFO:tasks.workunit.client.1.smithi121.stdout:8/990: write d0/d7/d36/d8c/d45/f2c [3922755,64530] 0 2024-02-15T18:35:48.939 INFO:tasks.workunit.client.1.smithi121.stdout:8/991: mknod d0/d7/d36/d52/d7a/c14b 0 2024-02-15T18:35:48.940 INFO:tasks.workunit.client.1.smithi121.stdout:8/992: mknod d0/d7/de/c14c 0 2024-02-15T18:35:48.957 INFO:tasks.workunit.client.1.smithi121.stdout:8/993: dwrite d0/d7/d36/d52/d104/f124 [0,4194304] 0 2024-02-15T18:35:48.957 INFO:tasks.workunit.client.1.smithi121.stdout:8/994: getdents d0/d7/de/d114 0 2024-02-15T18:35:48.965 INFO:tasks.workunit.client.1.smithi121.stdout:3/970: dwrite d2/d1a/d6b/fa5 [0,4194304] 0 2024-02-15T18:35:48.965 INFO:tasks.workunit.client.1.smithi121.stdout:3/971: truncate d2/d1a/d27/d63/da4/d25/d31/d97/fcd 686613 0 2024-02-15T18:35:48.965 INFO:tasks.workunit.client.1.smithi121.stdout:3/972: dread - d2/d1a/d27/da7/d85/d9f/ddd/de8/f12e zero size 2024-02-15T18:35:48.965 INFO:tasks.workunit.client.1.smithi121.stdout:3/973: read d2/d5a/f5f [521672,91920] 0 2024-02-15T18:35:48.967 INFO:tasks.workunit.client.1.smithi121.stdout:3/974: rename d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2/d10a/fa0 to d2/d1a/d27/d63/da4/d25/d30/db3/ddf/f14c 0 2024-02-15T18:35:48.968 INFO:tasks.workunit.client.1.smithi121.stdout:3/975: mknod d2/d1a/d27/d63/da4/d25/d30/db3/c14d 0 2024-02-15T18:35:48.968 INFO:tasks.workunit.client.1.smithi121.stdout:3/976: fsync d2/d1a/d27/d63/da4/d25/d31/d97/daf/fbc 0 2024-02-15T18:35:48.968 INFO:tasks.workunit.client.1.smithi121.stdout:3/977: fdatasync d2/d1a/d27/d63/da4/d25/d31/d97/fae 0 2024-02-15T18:35:48.969 INFO:tasks.workunit.client.1.smithi121.stdout:3/978: truncate d2/d1a/f1f 174089 0 2024-02-15T18:35:48.969 INFO:tasks.workunit.client.1.smithi121.stdout:3/979: fsync d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2/d10a/f10d 0 2024-02-15T18:35:48.971 INFO:tasks.workunit.client.1.smithi121.stdout:8/995: dwrite d0/dce/df8/f112 [0,4194304] 0 2024-02-15T18:35:48.971 INFO:tasks.workunit.client.1.smithi121.stdout:3/980: creat d2/d1a/d27/da7/d85/d9f/df1/f14e x:0 0 0 2024-02-15T18:35:48.972 INFO:tasks.workunit.client.1.smithi121.stdout:8/996: creat d0/d61/d66/f14d x:0 0 0 2024-02-15T18:35:48.973 INFO:tasks.workunit.client.1.smithi121.stdout:3/981: mkdir d2/d1a/d27/d63/da4/d25/d88/d9a/d10c/d14f 0 2024-02-15T18:35:48.976 INFO:tasks.workunit.client.1.smithi121.stdout:8/997: symlink d0/d7/d36/d8c/l14e 0 2024-02-15T18:35:48.977 INFO:tasks.workunit.client.1.smithi121.stdout:3/982: creat d2/d1a/d27/d63/da4/d25/d31/f150 x:0 0 0 2024-02-15T18:35:48.978 INFO:tasks.workunit.client.1.smithi121.stdout:8/998: creat d0/d61/d95/f14f x:0 0 0 2024-02-15T18:35:48.980 INFO:tasks.workunit.client.1.smithi121.stdout:3/983: rename d2/d1a/d27/d63/da4/d25/d31/f50 to d2/d1a/f151 0 2024-02-15T18:35:48.982 INFO:tasks.workunit.client.1.smithi121.stdout:8/999: rename d0/d7/de/f65 to d0/d7/d36/f150 0 2024-02-15T18:35:48.983 INFO:tasks.workunit.client.1.smithi121.stdout:3/984: truncate d2/d1a/d54/f59 592793 0 2024-02-15T18:35:48.985 INFO:tasks.workunit.client.1.smithi121.stdout:3/985: mknod d2/d1a/d27/d63/da4/d25/d31/d58/c152 0 2024-02-15T18:35:48.999 INFO:tasks.workunit.client.1.smithi121.stdout:3/986: dwrite d2/d5a/f78 [0,4194304] 0 2024-02-15T18:35:49.001 INFO:tasks.workunit.client.1.smithi121.stdout:3/987: truncate d2/d1a/d27/d63/da4/d25/d30/f41 502614 0 2024-02-15T18:35:49.001 INFO:tasks.workunit.client.1.smithi121.stdout:3/988: chown d2/d5a/lf9 8078 1 2024-02-15T18:35:49.019 INFO:tasks.workunit.client.1.smithi121.stdout:3/989: dwrite d2/d1a/d27/d63/da4/d25/d30/db3/ddf/de2/f134 [0,4194304] 0 2024-02-15T18:35:49.022 INFO:tasks.workunit.client.1.smithi121.stdout:3/990: mkdir d2/d1a/d27/da7/d85/d9f/d107/d153 0 2024-02-15T18:35:49.022 INFO:tasks.workunit.client.1.smithi121.stdout:3/991: dread - d2/d1a/d6b/fc8 zero size 2024-02-15T18:35:49.022 INFO:tasks.workunit.client.1.smithi121.stdout:3/992: fdatasync d2/d1a/d27/d63/da4/d25/d31/d97/daf/fc9 0 2024-02-15T18:35:49.039 INFO:tasks.workunit.client.1.smithi121.stdout:3/993: dwrite d2/d1a/d27/d63/d81/fb8 [0,4194304] 0 2024-02-15T18:35:49.040 INFO:tasks.workunit.client.1.smithi121.stdout:3/994: write d2/d1a/d27/da7/d85/d9f/ddd/de8/f12e [253708,79899] 0 2024-02-15T18:35:49.062 INFO:tasks.workunit.client.1.smithi121.stdout:3/995: dwrite d2/d1a/d27/d63/da4/f136 [0,4194304] 0 2024-02-15T18:35:49.062 INFO:tasks.workunit.client.1.smithi121.stdout:3/996: fsync d2/d1a/d27/d63/da4/d25/d31/d34/fdb 0 2024-02-15T18:35:49.066 INFO:tasks.workunit.client.1.smithi121.stdout:3/997: rename d2/d1a/d27/da7/d85/d9f/ddd/de8 to d2/d1a/d27/d63/da4/d25/d30/db3/dd3/d154 0 2024-02-15T18:35:49.067 INFO:tasks.workunit.client.1.smithi121.stdout:3/998: stat d2/d1a/d27/d63/da4/d25/d31/f57 0 2024-02-15T18:35:49.067 INFO:tasks.workunit.client.1.smithi121.stdout:3/999: dread - d2/d1a/d27/ff5 zero size 2024-02-15T18:35:49.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.084+0000 7fdfc031b700 1 -- 172.21.15.38:0/3103763228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdfb8107fc0 con 0x7fdfb80f6860 2024-02-15T18:35:49.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.085+0000 7fdfaeffd700 1 -- 172.21.15.38:0/3103763228 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fdfa804fc00 con 0x7fdfb80f6860 2024-02-15T18:35:49.076 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:35:49.076 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-02-15T18:35:49.077 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2024-02-15T18:35:49.077 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:35:49.077 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-02-15T18:35:49.077 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 1, 2024-02-15T18:35:49.077 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 1 2024-02-15T18:35:49.077 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:35:49.077 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-02-15T18:35:49.077 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2024-02-15T18:35:49.077 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:35:49.077 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-02-15T18:35:49.077 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:35:49.077 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:35:49.077 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-02-15T18:35:49.077 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 1, 2024-02-15T18:35:49.078 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 13 2024-02-15T18:35:49.078 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-02-15T18:35:49.078 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:35:49.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.087+0000 7fdfc031b700 1 -- 172.21.15.38:0/3103763228 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7fdfa405e960 msgr2=0x7fdfa4060e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:49.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.087+0000 7fdfc031b700 1 --2- 172.21.15.38:0/3103763228 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7fdfa405e960 0x7fdfa4060e20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fdfb4009fd0 tx=0x7fdfb400b040).stop 2024-02-15T18:35:49.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.087+0000 7fdfc031b700 1 -- 172.21.15.38:0/3103763228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfb80f6860 msgr2=0x7fdfb81063a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:49.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.087+0000 7fdfc031b700 1 --2- 172.21.15.38:0/3103763228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfb80f6860 0x7fdfb81063a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fdfa8000c00 tx=0x7fdfa800b040).stop 2024-02-15T18:35:49.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.087+0000 7fdfc031b700 1 -- 172.21.15.38:0/3103763228 shutdown_connections 2024-02-15T18:35:49.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.087+0000 7fdfc031b700 1 --2- 172.21.15.38:0/3103763228 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7fdfa405e960 0x7fdfa4060e20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:49.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.087+0000 7fdfc031b700 1 --2- 172.21.15.38:0/3103763228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfb80f55c0 0x7fdfb8105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:49.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.087+0000 7fdfc031b700 1 --2- 172.21.15.38:0/3103763228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdfb80f6860 0x7fdfb81063a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:49.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.087+0000 7fdfc031b700 1 -- 172.21.15.38:0/3103763228 >> 172.21.15.38:0/3103763228 conn(0x7fdfb80f0d40 msgr2=0x7fdfb80fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:49.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.087+0000 7fdfc031b700 1 -- 172.21.15.38:0/3103763228 shutdown_connections 2024-02-15T18:35:49.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.087+0000 7fdfc031b700 1 -- 172.21.15.38:0/3103763228 wait complete. 2024-02-15T18:35:49.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.197+0000 7fa6dfd52700 1 -- 172.21.15.38:0/2767986932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6d80f6860 msgr2=0x7fa6d80f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:49.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.197+0000 7fa6dfd52700 1 --2- 172.21.15.38:0/2767986932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6d80f6860 0x7fa6d80f6cc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fa6d4004660 tx=0x7fa6d4009b30).stop 2024-02-15T18:35:49.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.197+0000 7fa6dfd52700 1 -- 172.21.15.38:0/2767986932 shutdown_connections 2024-02-15T18:35:49.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.197+0000 7fa6dfd52700 1 --2- 172.21.15.38:0/2767986932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6d80f6860 0x7fa6d80f6cc0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:49.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.197+0000 7fa6dfd52700 1 --2- 172.21.15.38:0/2767986932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6d80f55c0 0x7fa6d80f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:49.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.197+0000 7fa6dfd52700 1 -- 172.21.15.38:0/2767986932 >> 172.21.15.38:0/2767986932 conn(0x7fa6d80f0d40 msgr2=0x7fa6d80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:49.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.197+0000 7fa6dfd52700 1 -- 172.21.15.38:0/2767986932 shutdown_connections 2024-02-15T18:35:49.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.197+0000 7fa6dfd52700 1 -- 172.21.15.38:0/2767986932 wait complete. 2024-02-15T18:35:49.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.198+0000 7fa6dfd52700 1 Processor -- start 2024-02-15T18:35:49.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.198+0000 7fa6dfd52700 1 -- start start 2024-02-15T18:35:49.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.198+0000 7fa6dfd52700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6d80f55c0 0x7fa6d81091c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:49.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.198+0000 7fa6dfd52700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6d80f6860 0x7fa6d81096b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:49.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.198+0000 7fa6dfd52700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d8109c60 con 0x7fa6d80f6860 2024-02-15T18:35:49.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.198+0000 7fa6dfd52700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d8109da0 con 0x7fa6d80f55c0 2024-02-15T18:35:49.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.198+0000 7fa6dd2ed700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6d80f6860 0x7fa6d81096b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:49.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.198+0000 7fa6dd2ed700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6d80f6860 0x7fa6d81096b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55062/0 (socket says 172.21.15.38:55062) 2024-02-15T18:35:49.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.198+0000 7fa6dd2ed700 1 -- 172.21.15.38:0/2162574063 learned_addr learned my addr 172.21.15.38:0/2162574063 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:35:49.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.198+0000 7fa6ddaee700 1 --2- 172.21.15.38:0/2162574063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6d80f55c0 0x7fa6d81091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:49.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.198+0000 7fa6dd2ed700 1 -- 172.21.15.38:0/2162574063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6d80f55c0 msgr2=0x7fa6d81091c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:49.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.198+0000 7fa6dd2ed700 1 --2- 172.21.15.38:0/2162574063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6d80f55c0 0x7fa6d81091c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:49.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.198+0000 7fa6dd2ed700 1 -- 172.21.15.38:0/2162574063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d4005040 con 0x7fa6d80f6860 2024-02-15T18:35:49.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.198+0000 7fa6dd2ed700 1 --2- 172.21.15.38:0/2162574063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6d80f6860 0x7fa6d81096b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7fa6d4000c00 tx=0x7fa6d400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:49.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.199+0000 7fa6ce7fc700 1 -- 172.21.15.38:0/2162574063 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6d4019070 con 0x7fa6d80f6860 2024-02-15T18:35:49.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.199+0000 7fa6ce7fc700 1 -- 172.21.15.38:0/2162574063 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fa6d4007e80 con 0x7fa6d80f6860 2024-02-15T18:35:49.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.199+0000 7fa6dfd52700 1 -- 172.21.15.38:0/2162574063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d810a6e0 con 0x7fa6d80f6860 2024-02-15T18:35:49.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.199+0000 7fa6ce7fc700 1 -- 172.21.15.38:0/2162574063 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6d4014db0 con 0x7fa6d80f6860 2024-02-15T18:35:49.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.199+0000 7fa6dfd52700 1 -- 172.21.15.38:0/2162574063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d810ac40 con 0x7fa6d80f6860 2024-02-15T18:35:49.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.200+0000 7fa6dfd52700 1 -- 172.21.15.38:0/2162574063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d8045ad0 con 0x7fa6d80f6860 2024-02-15T18:35:49.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.200+0000 7fa6ce7fc700 1 -- 172.21.15.38:0/2162574063 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 23) v1 ==== 74714+0+0 (secure 0 0 0) 0x7fa6d4025070 con 0x7fa6d80f6860 2024-02-15T18:35:49.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.200+0000 7fa6ce7fc700 1 --2- 172.21.15.38:0/2162574063 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7fa6c405e900 0x7fa6c4060dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:49.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.200+0000 7fa6ce7fc700 1 -- 172.21.15.38:0/2162574063 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fa6d4079f40 con 0x7fa6d80f6860 2024-02-15T18:35:49.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.201+0000 7fa6ddaee700 1 --2- 172.21.15.38:0/2162574063 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7fa6c405e900 0x7fa6c4060dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:49.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.201+0000 7fa6ddaee700 1 --2- 172.21.15.38:0/2162574063 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7fa6c405e900 0x7fa6c4060dc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fa6c8009fd0 tx=0x7fa6c800b040).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:49.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.205+0000 7fa6ce7fc700 1 -- 172.21.15.38:0/2162574063 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7fa6d40a8ad0 con 0x7fa6d80f6860 2024-02-15T18:35:49.197 INFO:tasks.workunit.client.1.smithi121.stdout:2/886: sync 2024-02-15T18:35:49.198 INFO:tasks.workunit.client.1.smithi121.stdout:2/887: creat d5/df/d27/d3e/d6a/f110 x:0 0 0 2024-02-15T18:35:49.200 INFO:tasks.workunit.client.1.smithi121.stdout:2/888: mkdir d5/df/d3f/daa/d111 0 2024-02-15T18:35:49.200 INFO:tasks.workunit.client.1.smithi121.stdout:2/889: dread - d5/ded/fff zero size 2024-02-15T18:35:49.200 INFO:tasks.workunit.client.1.smithi121.stdout:2/890: chown d5/df/d10/d4c/fb2 49 1 2024-02-15T18:35:49.218 INFO:tasks.workunit.client.1.smithi121.stdout:2/891: dwrite d5/df/d60/d98/d58/d63/d94/de0/df0/ffc [4194304,4194304] 0 2024-02-15T18:35:49.222 INFO:tasks.workunit.client.1.smithi121.stdout:2/892: mkdir d5/df/dc1/d101/d10f/d112 0 2024-02-15T18:35:49.222 INFO:tasks.workunit.client.1.smithi121.stdout:2/893: chown d5/df/d10/d4c/fc7 19177 1 2024-02-15T18:35:49.222 INFO:tasks.workunit.client.1.smithi121.stdout:2/894: write d5/df/d60/d98/d58/d63/d94/fb7 [4522726,104051] 0 2024-02-15T18:35:49.222 INFO:tasks.workunit.client.1.smithi121.stdout:2/895: dread - d5/df/d27/d3a/fe8 zero size 2024-02-15T18:35:49.223 INFO:tasks.workunit.client.1.smithi121.stdout:2/896: mknod d5/df/d3f/daa/c113 0 2024-02-15T18:35:49.223 INFO:tasks.workunit.client.1.smithi121.stdout:2/897: write d5/df/d27/d3e/d6a/ff2 [880708,67642] 0 2024-02-15T18:35:49.224 INFO:tasks.workunit.client.1.smithi121.stdout:2/898: write d5/d49/fa6 [2659077,101751] 0 2024-02-15T18:35:49.226 INFO:tasks.workunit.client.1.smithi121.stdout:2/899: mkdir d5/df/d60/d98/d22/d81/dbe/d114 0 2024-02-15T18:35:49.235 INFO:tasks.workunit.client.1.smithi121.stdout:2/900: dread d5/df/d27/d3e/d6a/ff2 [0,4194304] 0 2024-02-15T18:35:49.236 INFO:tasks.workunit.client.1.smithi121.stdout:2/901: creat d5/df/d60/d98/f115 x:0 0 0 2024-02-15T18:35:49.236 INFO:tasks.workunit.client.1.smithi121.stdout:2/902: getdents d5/df/d60/d98/d58 0 2024-02-15T18:35:49.238 INFO:tasks.workunit.client.1.smithi121.stdout:2/903: truncate d5/df/f6c 1682060 0 2024-02-15T18:35:49.239 INFO:tasks.workunit.client.1.smithi121.stdout:2/904: truncate d5/df/d60/d98/f1c 1272714 0 2024-02-15T18:35:49.260 INFO:tasks.workunit.client.1.smithi121.stdout:2/905: dwrite d5/df/d3f/f88 [0,4194304] 0 2024-02-15T18:35:49.261 INFO:tasks.workunit.client.1.smithi121.stdout:2/906: write d5/df/f6c [928503,33700] 0 2024-02-15T18:35:49.270 INFO:tasks.workunit.client.1.smithi121.stdout:7/806: sync 2024-02-15T18:35:49.270 INFO:tasks.workunit.client.1.smithi121.stdout:7/807: write d0/d5/d4c/d6f/db6/f10d [571434,49436] 0 2024-02-15T18:35:49.272 INFO:tasks.workunit.client.1.smithi121.stdout:7/808: link d0/l14 d0/d38/d3f/dfb/l10f 0 2024-02-15T18:35:49.272 INFO:tasks.workunit.client.1.smithi121.stdout:7/809: fdatasync d0/d5/f15 0 2024-02-15T18:35:49.273 INFO:tasks.workunit.client.1.smithi121.stdout:7/810: chown d0/d5/d11/d5c/dcc/f4b 54367 1 2024-02-15T18:35:49.274 INFO:tasks.workunit.client.1.smithi121.stdout:7/811: rename d0/laf to d0/d38/d3f/d43/l110 0 2024-02-15T18:35:49.274 INFO:tasks.workunit.client.1.smithi121.stdout:7/812: chown d0/d5/d11/d5c/dcc/dc6/cd1 4 1 2024-02-15T18:35:49.276 INFO:tasks.workunit.client.1.smithi121.stdout:7/813: rename d0/d5/d11/d1c/d5f/c10e to d0/d5/d11/d1c/c111 0 2024-02-15T18:35:49.276 INFO:tasks.workunit.client.1.smithi121.stdout:2/907: dwrite d5/df/d27/d3a/fb4 [0,4194304] 0 2024-02-15T18:35:49.281 INFO:tasks.workunit.client.1.smithi121.stdout:2/908: rename d5/df/d10 to d5/df/d27/d61/d116 0 2024-02-15T18:35:49.281 INFO:tasks.workunit.client.1.smithi121.stdout:2/909: chown d5/df/d27/f102 2658 1 2024-02-15T18:35:49.282 INFO:tasks.workunit.client.1.smithi121.stdout:2/910: dread d5/df/d60/f8c [0,4194304] 0 2024-02-15T18:35:49.282 INFO:tasks.workunit.client.1.smithi121.stdout:2/911: chown d5/df/d27 749109609 1 2024-02-15T18:35:49.294 INFO:tasks.workunit.client.1.smithi121.stdout:2/912: dread d5/df/d27/d61/f5c [0,4194304] 0 2024-02-15T18:35:49.294 INFO:tasks.workunit.client.1.smithi121.stdout:2/913: mknod d5/df/d27/d61/d116/c117 0 2024-02-15T18:35:49.295 INFO:tasks.workunit.client.1.smithi121.stdout:2/914: symlink d5/df/d60/d98/d58/d63/d94/de0/l118 0 2024-02-15T18:35:49.296 INFO:tasks.workunit.client.1.smithi121.stdout:2/915: chown d5/df/d60/d98/d22/l5b 161 1 2024-02-15T18:35:49.296 INFO:tasks.workunit.client.1.smithi121.stdout:2/916: chown d5/df/d27/d3e/f53 355 1 2024-02-15T18:35:49.296 INFO:tasks.workunit.client.1.smithi121.stdout:2/917: mkdir d5/d89/d119 0 2024-02-15T18:35:49.296 INFO:tasks.workunit.client.1.smithi121.stdout:2/918: dread - d5/df/d60/d98/d58/d63/ff1 zero size 2024-02-15T18:35:49.296 INFO:tasks.workunit.client.1.smithi121.stdout:2/919: chown d5/df/d27/d3e/d9f/dc2 656567309 1 2024-02-15T18:35:49.299 INFO:tasks.workunit.client.1.smithi121.stdout:2/920: dread d5/df/f2b [0,4194304] 0 2024-02-15T18:35:49.300 INFO:tasks.workunit.client.1.smithi121.stdout:2/921: mkdir d5/df/d27/d11a 0 2024-02-15T18:35:49.300 INFO:tasks.workunit.client.1.smithi121.stdout:2/922: write d5/df/d27/d3e/f59 [495112,66991] 0 2024-02-15T18:35:49.300 INFO:tasks.workunit.client.1.smithi121.stdout:2/923: fdatasync d5/df/d27/d3a/f43 0 2024-02-15T18:35:49.301 INFO:tasks.workunit.client.1.smithi121.stdout:2/924: rename d5/d49/d71 to d5/df/d60/d98/d22/d81/dbe/d11b 0 2024-02-15T18:35:49.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.450+0000 7fa6dfd52700 1 -- 172.21.15.38:0/2162574063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa6d810b2d0 con 0x7fa6d80f6860 2024-02-15T18:35:49.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.451+0000 7fa6ce7fc700 1 -- 172.21.15.38:0/2162574063 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1660 (secure 0 0 0) 0x7fa6d4023070 con 0x7fa6d80f6860 2024-02-15T18:35:49.442 INFO:teuthology.orchestra.run.smithi038.stdout:e13 2024-02-15T18:35:49.443 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:35:49.443 INFO:teuthology.orchestra.run.smithi038.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:35:49.443 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-02-15T18:35:49.443 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:35:49.443 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:35:49.443 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-02-15T18:35:49.443 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 13 2024-02-15T18:35:49.443 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 2024-02-15T18:35:49.443 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:35:49.443 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-02-15T18:35:07.485909+0000 2024-02-15T18:35:49.443 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-02-15T18:35:49.443 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-02-15T18:35:49.443 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-02-15T18:35:49.443 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-02-15T18:35:49.443 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-02-15T18:35:49.443 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-02-15T18:35:49.444 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-02-15T18:35:49.444 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-02-15T18:35:49.444 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:35:49.444 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-02-15T18:35:49.444 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-02-15T18:35:49.444 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=24261,1=24275} 2024-02-15T18:35:49.444 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-02-15T18:35:49.444 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-02-15T18:35:49.444 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-02-15T18:35:49.444 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-02-15T18:35:49.444 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-02-15T18:35:49.444 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-02-15T18:35:49.445 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-02-15T18:35:49.445 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-02-15T18:35:49.445 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.mvumlh{0:24261} state up:active seq 3 export targets 1 join_fscid=1 addr [v2:172.21.15.121:6824/1422707571,v1:172.21.15.121:6825/1422707571]] 2024-02-15T18:35:49.445 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.ovuzki{1:24275} state up:active seq 2 export targets 0 join_fscid=1 addr [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845]] 2024-02-15T18:35:49.445 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:35:49.445 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:35:49.445 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-02-15T18:35:49.445 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:35:49.445 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.hxoggl{-1:14446} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.38:6826/3822880235,v1:172.21.15.38:6827/3822880235]] 2024-02-15T18:35:49.445 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.qgkzgn{-1:14456} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.38:6828/703030231,v1:172.21.15.38:6829/703030231]] 2024-02-15T18:35:49.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.453+0000 7fa6dfd52700 1 -- 172.21.15.38:0/2162574063 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7fa6c405e900 msgr2=0x7fa6c4060dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:49.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.453+0000 7fa6dfd52700 1 --2- 172.21.15.38:0/2162574063 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7fa6c405e900 0x7fa6c4060dc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fa6c8009fd0 tx=0x7fa6c800b040).stop 2024-02-15T18:35:49.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.453+0000 7fa6dfd52700 1 -- 172.21.15.38:0/2162574063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6d80f6860 msgr2=0x7fa6d81096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:49.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.453+0000 7fa6dfd52700 1 --2- 172.21.15.38:0/2162574063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6d80f6860 0x7fa6d81096b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7fa6d4000c00 tx=0x7fa6d400b040).stop 2024-02-15T18:35:49.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.453+0000 7fa6dfd52700 1 -- 172.21.15.38:0/2162574063 shutdown_connections 2024-02-15T18:35:49.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.453+0000 7fa6dfd52700 1 --2- 172.21.15.38:0/2162574063 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7fa6c405e900 0x7fa6c4060dc0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:49.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.453+0000 7fa6dfd52700 1 --2- 172.21.15.38:0/2162574063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6d80f55c0 0x7fa6d81091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:49.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.453+0000 7fa6dfd52700 1 --2- 172.21.15.38:0/2162574063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6d80f6860 0x7fa6d81096b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:49.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.453+0000 7fa6dfd52700 1 -- 172.21.15.38:0/2162574063 >> 172.21.15.38:0/2162574063 conn(0x7fa6d80f0d40 msgr2=0x7fa6d805fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:49.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.453+0000 7fa6dfd52700 1 -- 172.21.15.38:0/2162574063 shutdown_connections 2024-02-15T18:35:49.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.453+0000 7fa6dfd52700 1 -- 172.21.15.38:0/2162574063 wait complete. 2024-02-15T18:35:49.447 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 13 2024-02-15T18:35:49.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:47.901545+0000 mgr.smithi121.ckiryi (mgr.24385) 18 : audit [DBG] from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:35:49.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:48.184197+0000 mon.smithi038 (mon.0) 601 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:49.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:48.192641+0000 mon.smithi121 (mon.1) 45 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:35:49.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:48.193098+0000 mgr.smithi121.ckiryi (mgr.24385) 19 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:35:49.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:48.195001+0000 mon.smithi121 (mon.1) 46 : audit [INF] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi038.front.sepia.ceph.com:9095"}]: dispatch 2024-02-15T18:35:49.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:48.195781+0000 mgr.smithi121.ckiryi (mgr.24385) 20 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi038.front.sepia.ceph.com:9095"}]: dispatch 2024-02-15T18:35:49.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:48.207557+0000 mon.smithi038 (mon.0) 602 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:49.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:48.224911+0000 mon.smithi121 (mon.1) 47 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:35:49.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:48.239417+0000 mon.smithi038 (mon.0) 603 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:49.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:35:48.241207+0000 mgr.smithi121.ckiryi (mgr.24385) 21 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi121.ckiryi) 2024-02-15T18:35:49.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:35:48.241713+0000 mgr.smithi121.ckiryi (mgr.24385) 22 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi121.ckiryi) 2024-02-15T18:35:49.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:48.298146+0000 mgr.smithi121.ckiryi (mgr.24385) 23 : audit [DBG] from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:35:49.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:48.597886+0000 mgr.smithi121.ckiryi (mgr.24385) 24 : cluster [DBG] pgmap v18: 65 pgs: 65 active+clean; 2.1 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 60 MiB/s rd, 139 MiB/s wr, 403 op/s 2024-02-15T18:35:49.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:49.086470+0000 mon.smithi038 (mon.0) 604 : audit [DBG] from='client.? 172.21.15.38:0/3103763228' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:35:49.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.559+0000 7f7b8a215700 1 -- 172.21.15.38:0/1666778028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b840f55c0 msgr2=0x7f7b840f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:49.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.559+0000 7f7b8a215700 1 --2- 172.21.15.38:0/1666778028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b840f55c0 0x7f7b840f59e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f7b74004660 tx=0x7f7b74009b30).stop 2024-02-15T18:35:49.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.559+0000 7f7b8a215700 1 -- 172.21.15.38:0/1666778028 shutdown_connections 2024-02-15T18:35:49.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.559+0000 7f7b8a215700 1 --2- 172.21.15.38:0/1666778028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b840f6860 0x7f7b840f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:49.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.559+0000 7f7b8a215700 1 --2- 172.21.15.38:0/1666778028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b840f55c0 0x7f7b840f59e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:49.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.559+0000 7f7b8a215700 1 -- 172.21.15.38:0/1666778028 >> 172.21.15.38:0/1666778028 conn(0x7f7b840f0d40 msgr2=0x7f7b840f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:49.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.559+0000 7f7b8a215700 1 -- 172.21.15.38:0/1666778028 shutdown_connections 2024-02-15T18:35:49.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.559+0000 7f7b8a215700 1 -- 172.21.15.38:0/1666778028 wait complete. 2024-02-15T18:35:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.560+0000 7f7b8a215700 1 Processor -- start 2024-02-15T18:35:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.560+0000 7f7b8a215700 1 -- start start 2024-02-15T18:35:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.561+0000 7f7b8a215700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b840f55c0 0x7f7b84105f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.561+0000 7f7b8a215700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b840f6860 0x7f7b841063f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.561+0000 7f7b8a215700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b841069a0 con 0x7f7b840f55c0 2024-02-15T18:35:49.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.561+0000 7f7b8a215700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b84106ae0 con 0x7f7b840f6860 2024-02-15T18:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.561+0000 7f7b837fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b840f55c0 0x7f7b84105f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.561+0000 7f7b837fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b840f55c0 0x7f7b84105f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55072/0 (socket says 172.21.15.38:55072) 2024-02-15T18:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.561+0000 7f7b837fe700 1 -- 172.21.15.38:0/1256932622 learned_addr learned my addr 172.21.15.38:0/1256932622 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.561+0000 7f7b82ffd700 1 --2- 172.21.15.38:0/1256932622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b840f6860 0x7f7b841063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.561+0000 7f7b837fe700 1 -- 172.21.15.38:0/1256932622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b840f6860 msgr2=0x7f7b841063f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.561+0000 7f7b837fe700 1 --2- 172.21.15.38:0/1256932622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b840f6860 0x7f7b841063f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.561+0000 7f7b837fe700 1 -- 172.21.15.38:0/1256932622 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b74005040 con 0x7f7b840f55c0 2024-02-15T18:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.561+0000 7f7b837fe700 1 --2- 172.21.15.38:0/1256932622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b840f55c0 0x7f7b84105f00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f7b74004e30 tx=0x7f7b74009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.561+0000 7f7b6bfff700 1 -- 172.21.15.38:0/1256932622 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b7401a070 con 0x7f7b840f55c0 2024-02-15T18:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.561+0000 7f7b8a215700 1 -- 172.21.15.38:0/1256932622 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b84107420 con 0x7f7b840f55c0 2024-02-15T18:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.561+0000 7f7b6bfff700 1 -- 172.21.15.38:0/1256932622 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f7b7401faf0 con 0x7f7b840f55c0 2024-02-15T18:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.561+0000 7f7b6bfff700 1 -- 172.21.15.38:0/1256932622 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b7401e8a0 con 0x7f7b840f55c0 2024-02-15T18:35:49.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.561+0000 7f7b8a215700 1 -- 172.21.15.38:0/1256932622 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b84107900 con 0x7f7b840f55c0 2024-02-15T18:35:49.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.562+0000 7f7b8a215700 1 -- 172.21.15.38:0/1256932622 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b84045ad0 con 0x7f7b840f55c0 2024-02-15T18:35:49.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.563+0000 7f7b6bfff700 1 -- 172.21.15.38:0/1256932622 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 23) v1 ==== 74714+0+0 (secure 0 0 0) 0x7f7b7401fc60 con 0x7f7b840f55c0 2024-02-15T18:35:49.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.563+0000 7f7b6bfff700 1 --2- 172.21.15.38:0/1256932622 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7f7b6c05e960 0x7f7b6c060e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:49.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.563+0000 7f7b6bfff700 1 -- 172.21.15.38:0/1256932622 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7b7407d0e0 con 0x7f7b840f55c0 2024-02-15T18:35:49.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.563+0000 7f7b82ffd700 1 --2- 172.21.15.38:0/1256932622 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7f7b6c05e960 0x7f7b6c060e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:49.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.564+0000 7f7b82ffd700 1 --2- 172.21.15.38:0/1256932622 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7f7b6c05e960 0x7f7b6c060e20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f7b78004ce0 tx=0x7f7b7800b040).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:49.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.567+0000 7f7b6bfff700 1 -- 172.21.15.38:0/1256932622 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7f7b74051060 con 0x7f7b840f55c0 2024-02-15T18:35:49.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:47.901545+0000 mgr.smithi121.ckiryi (mgr.24385) 18 : audit [DBG] from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:35:49.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:48.184197+0000 mon.smithi038 (mon.0) 601 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:49.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:48.192641+0000 mon.smithi121 (mon.1) 45 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:35:49.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:48.193098+0000 mgr.smithi121.ckiryi (mgr.24385) 19 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:35:49.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:48.195001+0000 mon.smithi121 (mon.1) 46 : audit [INF] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi038.front.sepia.ceph.com:9095"}]: dispatch 2024-02-15T18:35:49.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:48.195781+0000 mgr.smithi121.ckiryi (mgr.24385) 20 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi038.front.sepia.ceph.com:9095"}]: dispatch 2024-02-15T18:35:49.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:48.207557+0000 mon.smithi038 (mon.0) 602 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:49.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:48.224911+0000 mon.smithi121 (mon.1) 47 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:35:49.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:48.239417+0000 mon.smithi038 (mon.0) 603 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:49.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:35:48.241207+0000 mgr.smithi121.ckiryi (mgr.24385) 21 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi121.ckiryi) 2024-02-15T18:35:49.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:35:48.241713+0000 mgr.smithi121.ckiryi (mgr.24385) 22 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi121.ckiryi) 2024-02-15T18:35:49.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:48.298146+0000 mgr.smithi121.ckiryi (mgr.24385) 23 : audit [DBG] from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:35:49.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:48.597886+0000 mgr.smithi121.ckiryi (mgr.24385) 24 : cluster [DBG] pgmap v18: 65 pgs: 65 active+clean; 2.1 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 60 MiB/s rd, 139 MiB/s wr, 403 op/s 2024-02-15T18:35:49.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:49.086470+0000 mon.smithi038 (mon.0) 604 : audit [DBG] from='client.? 172.21.15.38:0/3103763228' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:35:49.633 INFO:tasks.workunit.client.1.smithi121.stdout:7/814: sync 2024-02-15T18:35:49.635 INFO:tasks.workunit.client.1.smithi121.stdout:7/815: link d0/d5/d76/l105 d0/d38/d46/daa/dd6/l112 0 2024-02-15T18:35:49.644 INFO:tasks.workunit.client.1.smithi121.stdout:7/816: dread d0/d5/d11/d5c/dcc/f4b [0,4194304] 0 2024-02-15T18:35:49.664 INFO:tasks.workunit.client.1.smithi121.stdout:7/817: dwrite d0/d4a/f53 [0,4194304] 0 2024-02-15T18:35:49.664 INFO:tasks.workunit.client.1.smithi121.stdout:7/818: chown d0/d5/d11/d5c/dcc/f102 984 1 2024-02-15T18:35:49.664 INFO:tasks.workunit.client.1.smithi121.stdout:7/819: chown d0/d5/d4c 68 1 2024-02-15T18:35:49.667 INFO:tasks.workunit.client.1.smithi121.stdout:7/820: symlink d0/d4a/de5/l113 0 2024-02-15T18:35:49.668 INFO:tasks.workunit.client.1.smithi121.stdout:7/821: creat d0/d4a/f114 x:0 0 0 2024-02-15T18:35:49.670 INFO:tasks.workunit.client.1.smithi121.stdout:7/822: mknod d0/d5/d11/d5c/dcc/c115 0 2024-02-15T18:35:49.670 INFO:tasks.workunit.client.1.smithi121.stdout:7/823: chown d0/d5/d4c/fee 3502527 1 2024-02-15T18:35:49.673 INFO:tasks.workunit.client.1.smithi121.stdout:7/824: mkdir d0/d5/d4c/d116 0 2024-02-15T18:35:49.683 INFO:tasks.workunit.client.1.smithi121.stdout:7/825: dread d0/f27 [0,4194304] 0 2024-02-15T18:35:49.683 INFO:tasks.workunit.client.1.smithi121.stdout:7/826: chown d0/d5/d11/c25 4023 1 2024-02-15T18:35:49.684 INFO:tasks.workunit.client.1.smithi121.stdout:7/827: symlink d0/d38/d46/daa/df7/l117 0 2024-02-15T18:35:49.686 INFO:tasks.workunit.client.1.smithi121.stdout:7/828: link d0/d4a/d5b/f92 d0/d5/d11/d1c/d5f/dc0/f118 0 2024-02-15T18:35:49.686 INFO:tasks.workunit.client.1.smithi121.stdout:7/829: chown d0/d4a/d5b/cd0 597478 1 2024-02-15T18:35:49.688 INFO:tasks.workunit.client.1.smithi121.stdout:7/830: creat d0/d5/d11/f119 x:0 0 0 2024-02-15T18:35:49.690 INFO:tasks.workunit.client.1.smithi121.stdout:7/831: rename d0/d38/d46/daa/le8 to d0/d38/l11a 0 2024-02-15T18:35:49.712 INFO:tasks.workunit.client.1.smithi121.stdout:7/832: dwrite d0/d5/d4c/fee [0,4194304] 0 2024-02-15T18:35:49.784 INFO:tasks.workunit.client.1.smithi121.stdout:7/833: creat d0/d5/d11/d79/f11b x:0 0 0 2024-02-15T18:35:49.784 INFO:tasks.workunit.client.1.smithi121.stdout:7/834: readlink d0/d4a/d5b/da6/l6c 0 2024-02-15T18:35:49.784 INFO:tasks.workunit.client.1.smithi121.stdout:7/835: readlink d0/d38/d46/daa/dd6/l112 0 2024-02-15T18:35:49.784 INFO:tasks.workunit.client.1.smithi121.stdout:7/836: truncate d0/d5/d11/d1c/fd3 533412 0 2024-02-15T18:35:49.784 INFO:tasks.workunit.client.1.smithi121.stdout:7/837: symlink d0/d5/d11/d79/dfa/l11c 0 2024-02-15T18:35:49.784 INFO:tasks.workunit.client.1.smithi121.stdout:7/838: write d0/d5/d4c/d6f/db6/fbc [629555,62729] 0 2024-02-15T18:35:49.784 INFO:tasks.workunit.client.1.smithi121.stdout:7/839: dread d0/d4a/f10a [0,4194304] 0 2024-02-15T18:35:49.784 INFO:tasks.workunit.client.1.smithi121.stdout:7/840: truncate d0/d4a/f53 4715808 0 2024-02-15T18:35:49.784 INFO:tasks.workunit.client.1.smithi121.stdout:7/841: getdents d0/d38/d3f/dfb 0 2024-02-15T18:35:49.784 INFO:tasks.workunit.client.1.smithi121.stdout:7/842: mkdir d0/d5/d4c/d6f/dd5/de6/d11d 0 2024-02-15T18:35:49.785 INFO:tasks.workunit.client.1.smithi121.stdout:7/843: getdents d0/d4a/d5b/da6 0 2024-02-15T18:35:49.785 INFO:tasks.workunit.client.1.smithi121.stdout:7/844: dread d0/d5/d11/d5c/fa7 [0,4194304] 0 2024-02-15T18:35:49.785 INFO:tasks.workunit.client.1.smithi121.stdout:7/845: fsync d0/d38/fc1 0 2024-02-15T18:35:49.785 INFO:tasks.workunit.client.1.smithi121.stdout:7/846: symlink d0/d5/d4c/d6f/db3/l11e 0 2024-02-15T18:35:49.785 INFO:tasks.workunit.client.1.smithi121.stdout:2/925: sync 2024-02-15T18:35:49.785 INFO:tasks.workunit.client.1.smithi121.stdout:2/926: dread d5/df/d27/d61/f6b [0,4194304] 0 2024-02-15T18:35:49.785 INFO:tasks.workunit.client.1.smithi121.stdout:7/847: read d0/d5/d11/d1c/f3e [4549538,79526] 0 2024-02-15T18:35:49.785 INFO:tasks.workunit.client.1.smithi121.stdout:7/848: dread - d0/d38/d61/fd7 zero size 2024-02-15T18:35:49.785 INFO:tasks.workunit.client.1.smithi121.stdout:2/927: rename d5/df/d27/d61 to d5/df/d60/d98/d58/d63/d11c 0 2024-02-15T18:35:49.785 INFO:tasks.workunit.client.1.smithi121.stdout:7/849: rename d0/d38/fc1 to d0/d5/d11/dbb/f11f 0 2024-02-15T18:35:49.785 INFO:tasks.workunit.client.1.smithi121.stdout:2/928: rmdir d5/df/d60/d98/d22/d81/dac 39 2024-02-15T18:35:49.785 INFO:tasks.workunit.client.1.smithi121.stdout:7/850: truncate d0/d4a/d5b/da6/f7f 78281 0 2024-02-15T18:35:49.785 INFO:tasks.workunit.client.1.smithi121.stdout:2/929: rmdir d5/df/d60/d98/d58/d63/d94/dc3 39 2024-02-15T18:35:49.785 INFO:tasks.workunit.client.1.smithi121.stdout:2/930: creat d5/df/d60/d98/d58/d63/d94/de0/f11d x:0 0 0 2024-02-15T18:35:49.785 INFO:tasks.workunit.client.1.smithi121.stdout:2/931: mkdir d5/df/d27/d3a/d11e 0 2024-02-15T18:35:49.786 INFO:tasks.workunit.client.1.smithi121.stdout:2/932: rename d5/df/d27/d3e/d9f to d5/df/dc1/d11f 0 2024-02-15T18:35:49.786 INFO:tasks.workunit.client.1.smithi121.stdout:2/933: mknod d5/df/d27/d11a/c120 0 2024-02-15T18:35:49.786 INFO:tasks.workunit.client.1.smithi121.stdout:2/934: rename d5/df/d27/d11a/c120 to d5/df/d60/d98/d22/d81/dac/c121 0 2024-02-15T18:35:49.786 INFO:tasks.workunit.client.1.smithi121.stdout:2/935: symlink d5/df/d60/l122 0 2024-02-15T18:35:49.786 INFO:tasks.workunit.client.1.smithi121.stdout:2/936: rename d5/df/f2b to d5/df/d3f/f123 0 2024-02-15T18:35:49.786 INFO:tasks.workunit.client.1.smithi121.stdout:2/937: dread d5/df/d60/d98/f2d [4194304,4194304] 0 2024-02-15T18:35:49.786 INFO:tasks.workunit.client.1.smithi121.stdout:2/938: fsync d5/fc 0 2024-02-15T18:35:49.798 INFO:tasks.workunit.client.1.smithi121.stdout:2/939: dwrite d5/d49/f84 [0,4194304] 0 2024-02-15T18:35:49.798 INFO:tasks.workunit.client.1.smithi121.stdout:2/940: chown d5/df/d60/d98/d58/d63/d94/fd3 106390023 1 2024-02-15T18:35:49.799 INFO:tasks.workunit.client.1.smithi121.stdout:2/941: mknod d5/df/dc1/d11f/c124 0 2024-02-15T18:35:49.801 INFO:tasks.workunit.client.1.smithi121.stdout:2/942: mkdir d5/df/d27/d125 0 2024-02-15T18:35:49.811 INFO:tasks.workunit.client.1.smithi121.stdout:2/943: rename d5/df/dc1/d11f to d5/df/dc1/d126 0 2024-02-15T18:35:49.811 INFO:tasks.workunit.client.1.smithi121.stdout:2/944: fsync d5/df/d60/d98/d22/f4f 0 2024-02-15T18:35:49.811 INFO:tasks.workunit.client.1.smithi121.stdout:2/945: chown d5/df/d27/d3e/ld9 53 1 2024-02-15T18:35:49.815 INFO:tasks.workunit.client.1.smithi121.stdout:2/946: mknod d5/df/d60/d98/d58/d63/d11c/d116/d4c/dad/c127 0 2024-02-15T18:35:49.817 INFO:tasks.workunit.client.1.smithi121.stdout:2/947: write d5/df/d60/d98/d58/d63/d11c/d116/d4c/dad/fdd [3750364,16603] 0 2024-02-15T18:35:49.830 INFO:tasks.workunit.client.1.smithi121.stdout:2/948: dread d5/df/d60/d98/d22/f44 [0,4194304] 0 2024-02-15T18:35:49.832 INFO:tasks.workunit.client.1.smithi121.stdout:2/949: unlink d5/df/d60/d98/d58/d63/d11c/d116/f8a 0 2024-02-15T18:35:49.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.851+0000 7f7b8a215700 1 -- 172.21.15.38:0/1256932622 --> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b840fff00 con 0x7f7b6c05e960 2024-02-15T18:35:49.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.853+0000 7f7b6bfff700 1 -- 172.21.15.38:0/1256932622 <== mgr.24385 v2:172.21.15.121:6828/330429733 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f7b840fff00 con 0x7f7b6c05e960 2024-02-15T18:35:49.845 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:35:49.845 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:35:49.845 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-02-15T18:35:49.845 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:35:49.845 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [], 2024-02-15T18:35:49.845 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "1/21 daemons upgraded", 2024-02-15T18:35:49.845 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading mgr daemons", 2024-02-15T18:35:49.845 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-02-15T18:35:49.845 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:35:49.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.856+0000 7f7b8a215700 1 -- 172.21.15.38:0/1256932622 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7f7b6c05e960 msgr2=0x7f7b6c060e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:49.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.856+0000 7f7b8a215700 1 --2- 172.21.15.38:0/1256932622 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7f7b6c05e960 0x7f7b6c060e20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f7b78004ce0 tx=0x7f7b7800b040).stop 2024-02-15T18:35:49.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.856+0000 7f7b8a215700 1 -- 172.21.15.38:0/1256932622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b840f55c0 msgr2=0x7f7b84105f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:49.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.856+0000 7f7b8a215700 1 --2- 172.21.15.38:0/1256932622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b840f55c0 0x7f7b84105f00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f7b74004e30 tx=0x7f7b74009b30).stop 2024-02-15T18:35:49.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.856+0000 7f7b8a215700 1 -- 172.21.15.38:0/1256932622 shutdown_connections 2024-02-15T18:35:49.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.856+0000 7f7b8a215700 1 --2- 172.21.15.38:0/1256932622 >> [v2:172.21.15.121:6828/330429733,v1:172.21.15.121:6829/330429733] conn(0x7f7b6c05e960 0x7f7b6c060e20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:49.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.856+0000 7f7b8a215700 1 --2- 172.21.15.38:0/1256932622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b840f55c0 0x7f7b84105f00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:49.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.856+0000 7f7b8a215700 1 --2- 172.21.15.38:0/1256932622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b840f6860 0x7f7b841063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:49.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.856+0000 7f7b8a215700 1 -- 172.21.15.38:0/1256932622 >> 172.21.15.38:0/1256932622 conn(0x7f7b840f0d40 msgr2=0x7f7b840fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:49.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.856+0000 7f7b8a215700 1 -- 172.21.15.38:0/1256932622 shutdown_connections 2024-02-15T18:35:49.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:35:49.856+0000 7f7b8a215700 1 -- 172.21.15.38:0/1256932622 wait complete. 2024-02-15T18:35:49.856 INFO:tasks.workunit.client.1.smithi121.stdout:2/950: dwrite d5/df/d60/d98/d58/d63/d11c/f6b [0,4194304] 0 2024-02-15T18:35:49.856 INFO:tasks.workunit.client.1.smithi121.stdout:2/951: write d5/df/d27/fee [298049,96809] 0 2024-02-15T18:35:49.856 INFO:tasks.workunit.client.1.smithi121.stdout:2/952: readlink d5/df/d60/d98/d46/la8 0 2024-02-15T18:35:49.856 INFO:tasks.workunit.client.1.smithi121.stdout:2/953: read - d5/df/d27/f102 zero size 2024-02-15T18:35:49.856 INFO:tasks.workunit.client.1.smithi121.stdout:2/954: chown d5/df/d3f/f123 226 1 2024-02-15T18:35:49.856 INFO:tasks.workunit.client.1.smithi121.stdout:2/955: creat d5/df/d60/d98/d22/d81/d9a/f128 x:0 0 0 2024-02-15T18:35:49.857 INFO:tasks.workunit.client.1.smithi121.stdout:2/956: chown d5/df/d3f/f88 31 1 2024-02-15T18:35:49.857 INFO:tasks.workunit.client.1.smithi121.stdout:2/957: write d5/ded/fec [1687749,92454] 0 2024-02-15T18:35:49.857 INFO:tasks.workunit.client.1.smithi121.stdout:2/958: dread - d5/f10a zero size 2024-02-15T18:35:49.859 INFO:tasks.workunit.client.1.smithi121.stdout:2/959: rmdir d5/d89/d119 0 2024-02-15T18:35:49.859 INFO:tasks.workunit.client.1.smithi121.stdout:2/960: readlink d5/df/d60/d98/d22/d81/dbe/d11b/l5e 0 2024-02-15T18:35:49.859 INFO:tasks.workunit.client.1.smithi121.stdout:2/961: rename d5 to d5/df/df3/d129 22 2024-02-15T18:35:49.860 INFO:tasks.workunit.client.1.smithi121.stdout:2/962: symlink d5/df/d60/d98/d58/d63/d94/de0/df0/l12a 0 2024-02-15T18:35:49.864 INFO:tasks.workunit.client.1.smithi121.stdout:2/963: creat d5/df/d60/d98/d22/de6/f12b x:0 0 0 2024-02-15T18:35:49.867 INFO:tasks.workunit.client.1.smithi121.stdout:2/964: rename d5/df/d27/d3e/f59 to d5/df/d60/f12c 0 2024-02-15T18:35:49.868 INFO:tasks.workunit.client.1.smithi121.stdout:2/965: mknod d5/df/d60/d98/d22/de6/c12d 0 2024-02-15T18:35:49.869 INFO:tasks.workunit.client.1.smithi121.stdout:2/966: mkdir d5/df/d27/d3e/d12e 0 2024-02-15T18:35:49.871 INFO:tasks.workunit.client.1.smithi121.stdout:2/967: symlink d5/l12f 0 2024-02-15T18:35:49.871 INFO:tasks.workunit.client.1.smithi121.stdout:2/968: chown d5/df/d60/d98/d58/d63/d11c/f5c 9 1 2024-02-15T18:35:49.872 INFO:tasks.workunit.client.1.smithi121.stdout:2/969: readlink d5/df/d27/d3e/d6a/l7b 0 2024-02-15T18:35:49.872 INFO:tasks.workunit.client.1.smithi121.stdout:2/970: write d5/df/d27/f102 [650518,71129] 0 2024-02-15T18:35:49.872 INFO:tasks.workunit.client.1.smithi121.stdout:2/971: fdatasync d5/fd 0 2024-02-15T18:35:49.873 INFO:tasks.workunit.client.1.smithi121.stdout:2/972: chown d5/df/d60/d98/d58/d63/d11c/c73 101 1 2024-02-15T18:35:49.873 INFO:tasks.workunit.client.1.smithi121.stdout:2/973: fdatasync d5/df/d60/d98/f1c 0 2024-02-15T18:35:49.894 INFO:tasks.workunit.client.1.smithi121.stdout:2/974: dwrite d5/df/d60/d98/d58/d63/d11c/d116/d4c/fb2 [4194304,4194304] 0 2024-02-15T18:35:49.897 INFO:tasks.workunit.client.1.smithi121.stdout:2/975: dread d5/df/d60/d98/d58/d63/d11c/d116/f7e [0,4194304] 0 2024-02-15T18:35:49.899 INFO:tasks.workunit.client.1.smithi121.stdout:2/976: write d5/df/d60/f12c [282637,40214] 0 2024-02-15T18:35:49.899 INFO:tasks.workunit.client.1.smithi121.stdout:2/977: rename d5/df/d27 to d5/df/d27/d3a/d130 22 2024-02-15T18:35:49.917 INFO:tasks.workunit.client.1.smithi121.stdout:2/978: dwrite d5/ded/f10d [0,4194304] 0 2024-02-15T18:35:49.919 INFO:tasks.workunit.client.1.smithi121.stdout:2/979: creat d5/df/d3f/f131 x:0 0 0 2024-02-15T18:35:49.919 INFO:tasks.workunit.client.1.smithi121.stdout:2/980: chown d5/df/d3f/daa/lbb 143839 1 2024-02-15T18:35:49.921 INFO:tasks.workunit.client.1.smithi121.stdout:2/981: write d5/df/d60/d98/d22/d81/dbe/d11b/fba [925178,114907] 0 2024-02-15T18:35:49.924 INFO:tasks.workunit.client.1.smithi121.stdout:2/982: creat d5/df/d60/f132 x:0 0 0 2024-02-15T18:35:49.925 INFO:tasks.workunit.client.1.smithi121.stdout:2/983: rename d5/df/d60/d98/d22/f44 to d5/f133 0 2024-02-15T18:35:49.941 INFO:tasks.workunit.client.1.smithi121.stdout:2/984: dread d5/df/d60/f12c [0,4194304] 0 2024-02-15T18:35:49.943 INFO:tasks.workunit.client.1.smithi121.stdout:2/985: symlink d5/df/d27/d11a/l134 0 2024-02-15T18:35:49.944 INFO:tasks.workunit.client.1.smithi121.stdout:2/986: readlink d5/df/d60/d98/d22/d81/d9a/ld7 0 2024-02-15T18:35:49.944 INFO:tasks.workunit.client.1.smithi121.stdout:2/987: chown d5/df/d60/d98/d58/d63/d94/cd8 2499 1 2024-02-15T18:35:49.944 INFO:tasks.workunit.client.1.smithi121.stdout:2/988: chown d5/df/d60/d98/d22/dc8/lcd 209300743 1 2024-02-15T18:35:49.944 INFO:tasks.workunit.client.1.smithi121.stdout:2/989: creat d5/df/d60/d98/d58/d63/d94/dc3/f135 x:0 0 0 2024-02-15T18:35:49.946 INFO:tasks.workunit.client.1.smithi121.stdout:2/990: symlink d5/df/d3f/daa/d111/l136 0 2024-02-15T18:35:49.946 INFO:tasks.workunit.client.1.smithi121.stdout:2/991: fsync d5/df/d60/d98/d22/d81/dbe/d11b/fba 0 2024-02-15T18:35:49.946 INFO:tasks.workunit.client.1.smithi121.stdout:2/992: fdatasync d5/df/d3f/f88 0 2024-02-15T18:35:49.946 INFO:tasks.workunit.client.1.smithi121.stdout:2/993: chown d5/df/d3f/daa 3498337 1 2024-02-15T18:35:49.948 INFO:tasks.workunit.client.1.smithi121.stdout:2/994: write d5/df/f6c [1385400,23589] 0 2024-02-15T18:35:49.948 INFO:tasks.workunit.client.1.smithi121.stdout:2/995: chown d5/ded/f10d 33 1 2024-02-15T18:35:49.948 INFO:tasks.workunit.client.1.smithi121.stdout:2/996: truncate d5/df/d60/d98/d22/de6/ff9 910937 0 2024-02-15T18:35:49.948 INFO:tasks.workunit.client.1.smithi121.stdout:2/997: fsync d5/ded/fec 0 2024-02-15T18:35:50.330 INFO:tasks.workunit.client.1.smithi121.stdout:7/851: sync 2024-02-15T18:35:50.331 INFO:tasks.workunit.client.1.smithi121.stdout:7/852: chown d0/f77 6181231 1 2024-02-15T18:35:50.332 INFO:tasks.workunit.client.1.smithi121.stdout:7/853: creat d0/d4a/d5b/d9a/de9/f120 x:0 0 0 2024-02-15T18:35:50.335 INFO:tasks.workunit.client.1.smithi121.stdout:7/854: dread d0/d5/d11/d5c/dcc/dc6/fef [0,4194304] 0 2024-02-15T18:35:50.353 INFO:tasks.workunit.client.1.smithi121.stdout:7/855: dwrite d0/d5/f26 [8388608,4194304] 0 2024-02-15T18:35:50.354 INFO:tasks.workunit.client.1.smithi121.stdout:7/856: mknod d0/d5/d11/d5c/dcc/c121 0 2024-02-15T18:35:50.355 INFO:tasks.workunit.client.1.smithi121.stdout:7/857: unlink d0/d5/d11/d1c/d5f/dc0/fdf 0 2024-02-15T18:35:50.356 INFO:tasks.workunit.client.1.smithi121.stdout:7/858: chown d0/d4a/d5b/da6/fbf 0 1 2024-02-15T18:35:50.358 INFO:tasks.workunit.client.1.smithi121.stdout:7/859: rename d0/d38/d46/daa/dc2/cf0 to d0/d5/d4c/dda/c122 0 2024-02-15T18:35:50.358 INFO:tasks.workunit.client.1.smithi121.stdout:7/860: creat d0/d5/d11/f123 x:0 0 0 2024-02-15T18:35:50.360 INFO:tasks.workunit.client.1.smithi121.stdout:7/861: mkdir d0/d124 0 2024-02-15T18:35:50.362 INFO:tasks.workunit.client.1.smithi121.stdout:7/862: mkdir d0/d4a/d5b/d125 0 2024-02-15T18:35:50.363 INFO:tasks.workunit.client.1.smithi121.stdout:7/863: creat d0/d5/d4c/dda/f126 x:0 0 0 2024-02-15T18:35:50.364 INFO:tasks.workunit.client.1.smithi121.stdout:7/864: fsync d0/d4a/d5b/d9a/fc9 0 2024-02-15T18:35:50.364 INFO:tasks.workunit.client.1.smithi121.stdout:7/865: rmdir d0/d4a/d5b/da6 39 2024-02-15T18:35:50.375 INFO:tasks.workunit.client.1.smithi121.stdout:7/866: rename d0/d5/d11 to d0/d5/d4c/d6f/d127 0 2024-02-15T18:35:50.377 INFO:tasks.workunit.client.1.smithi121.stdout:7/867: rmdir d0/d5/d76 39 2024-02-15T18:35:50.377 INFO:tasks.workunit.client.1.smithi121.stdout:7/868: fsync d0/f52 0 2024-02-15T18:35:50.377 INFO:tasks.workunit.client.1.smithi121.stdout:7/869: write d0/d5/d4c/d6f/d127/f123 [1036281,66917] 0 2024-02-15T18:35:50.377 INFO:tasks.workunit.client.1.smithi121.stdout:7/870: getdents d0/d5/d4c/d6f/dd5/de6 0 2024-02-15T18:35:50.377 INFO:tasks.workunit.client.1.smithi121.stdout:7/871: truncate d0/d5/d4c/d6f/d127/d1c/d5f/dc0/f118 965741 0 2024-02-15T18:35:50.380 INFO:tasks.workunit.client.1.smithi121.stdout:7/872: dread d0/d38/d46/daa/f5d [0,4194304] 0 2024-02-15T18:35:50.381 INFO:tasks.workunit.client.1.smithi121.stdout:2/998: sync 2024-02-15T18:35:50.382 INFO:tasks.workunit.client.1.smithi121.stdout:7/873: mkdir d0/d5/d4c/d6f/d127/d1c/d128 0 2024-02-15T18:35:50.384 INFO:tasks.workunit.client.1.smithi121.stdout:2/999: mknod d5/df/dc1/d126/dc2/de3/c137 0 2024-02-15T18:35:50.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:48.647077+0000 mgr.smithi121.ckiryi (mgr.24385) 25 : audit [DBG] from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:35:50.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:49.452555+0000 mon.smithi038 (mon.0) 605 : audit [DBG] from='client.? 172.21.15.38:0/2162574063' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:35:50.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:50.018043+0000 mon.smithi038 (mon.0) 606 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:50.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:50.020133+0000 mon.smithi121 (mon.1) 48 : audit [INF] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi038.khesbl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:35:50.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:50.020564+0000 mon.smithi038 (mon.0) 607 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi038.khesbl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:35:50.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:50.021625+0000 mon.smithi121 (mon.1) 49 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:35:50.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:50.022619+0000 mon.smithi121 (mon.1) 50 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:35:50.662 INFO:tasks.workunit.client.1.smithi121.stdout:7/874: sync 2024-02-15T18:35:50.663 INFO:tasks.workunit.client.1.smithi121.stdout:7/875: creat d0/d5/d4c/d6f/db3/f129 x:0 0 0 2024-02-15T18:35:50.666 INFO:tasks.workunit.client.1.smithi121.stdout:7/876: rename d0/d5/d4c/d6f/c93 to d0/d5/d4c/d6f/d127/c12a 0 2024-02-15T18:35:50.666 INFO:tasks.workunit.client.1.smithi121.stdout:7/877: readlink d0/l49 0 2024-02-15T18:35:50.694 INFO:tasks.workunit.client.1.smithi121.stdout:7/878: dwrite d0/d5/d4c/d6f/d127/d5c/dcc/ff8 [0,4194304] 0 2024-02-15T18:35:50.694 INFO:tasks.workunit.client.1.smithi121.stdout:7/879: chown d0/d4a/fd4 1 1 2024-02-15T18:35:50.699 INFO:tasks.workunit.client.1.smithi121.stdout:7/880: write d0/d5/d4c/d6f/d127/f40 [1653311,62948] 0 2024-02-15T18:35:50.702 INFO:tasks.workunit.client.1.smithi121.stdout:7/881: mknod d0/c12b 0 2024-02-15T18:35:50.727 INFO:tasks.workunit.client.1.smithi121.stdout:7/882: dwrite d0/d5/d4c/d6f/d127/d1c/fb2 [4194304,4194304] 0 2024-02-15T18:35:50.727 INFO:tasks.workunit.client.1.smithi121.stdout:7/883: stat d0/d5/d4c/d6f/d127/f40 0 2024-02-15T18:35:50.727 INFO:tasks.workunit.client.1.smithi121.stdout:7/884: write d0/f69 [1644462,102352] 0 2024-02-15T18:35:50.755 INFO:tasks.workunit.client.1.smithi121.stdout:7/885: dwrite d0/d38/fa0 [0,4194304] 0 2024-02-15T18:35:50.756 INFO:tasks.workunit.client.1.smithi121.stdout:7/886: mknod d0/d4a/c12c 0 2024-02-15T18:35:50.756 INFO:tasks.workunit.client.1.smithi121.stdout:7/887: write d0/d5/d4c/d6f/db3/fec [235166,98664] 0 2024-02-15T18:35:50.757 INFO:tasks.workunit.client.1.smithi121.stdout:7/888: truncate d0/d5/d4c/d6f/d127/d1c/f44 1001378 0 2024-02-15T18:35:50.766 INFO:tasks.workunit.client.1.smithi121.stdout:7/889: dread d0/d38/d46/daa/f3b [0,4194304] 0 2024-02-15T18:35:50.766 INFO:tasks.workunit.client.1.smithi121.stdout:7/890: read d0/d5/d4c/d6f/d127/d5c/fa7 [138361,79536] 0 2024-02-15T18:35:50.768 INFO:tasks.workunit.client.1.smithi121.stdout:7/891: dread d0/d38/d46/daa/f5d [0,4194304] 0 2024-02-15T18:35:50.769 INFO:tasks.workunit.client.1.smithi121.stdout:7/892: creat d0/d5/d4c/d6f/d127/d1c/d5f/f12d x:0 0 0 2024-02-15T18:35:50.770 INFO:tasks.workunit.client.1.smithi121.stdout:7/893: write d0/d5/d4c/d6f/db6/fbc [585627,52177] 0 2024-02-15T18:35:50.772 INFO:tasks.workunit.client.1.smithi121.stdout:7/894: dread d0/d5/d76/f10c [0,4194304] 0 2024-02-15T18:35:50.772 INFO:tasks.workunit.client.1.smithi121.stdout:7/895: fdatasync d0/d5/d4c/d6f/d127/d5c/dcc/f4b 0 2024-02-15T18:35:50.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:48.647077+0000 mgr.smithi121.ckiryi (mgr.24385) 25 : audit [DBG] from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:35:50.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:49.452555+0000 mon.smithi038 (mon.0) 605 : audit [DBG] from='client.? 172.21.15.38:0/2162574063' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:35:50.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:50.018043+0000 mon.smithi038 (mon.0) 606 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:50.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:50.020133+0000 mon.smithi121 (mon.1) 48 : audit [INF] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi038.khesbl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:35:50.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:50.020564+0000 mon.smithi038 (mon.0) 607 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi038.khesbl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:35:50.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:50.021625+0000 mon.smithi121 (mon.1) 49 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:35:50.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:50.022619+0000 mon.smithi121 (mon.1) 50 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:35:50.774 INFO:tasks.workunit.client.1.smithi121.stdout:7/896: dread d0/d38/d46/f96 [0,4194304] 0 2024-02-15T18:35:50.774 INFO:tasks.workunit.client.1.smithi121.stdout:7/897: truncate d0/d5/d4c/d6f/fbe 3221237 0 2024-02-15T18:35:50.775 INFO:tasks.workunit.client.1.smithi121.stdout:7/898: symlink d0/d4a/d5b/d9a/de9/l12e 0 2024-02-15T18:35:50.777 INFO:tasks.workunit.client.1.smithi121.stdout:7/899: rename d0/d5/d4c/dda/l101 to d0/d5/d4c/d6f/d127/l12f 0 2024-02-15T18:35:50.777 INFO:tasks.workunit.client.1.smithi121.stdout:7/900: write d0/d5/d4c/d6f/d127/d1c/f23 [5652336,74603] 0 2024-02-15T18:35:50.777 INFO:tasks.workunit.client.1.smithi121.stdout:7/901: write d0/d5/d4c/d6f/d127/d1c/fd3 [799762,15980] 0 2024-02-15T18:35:50.798 INFO:tasks.workunit.client.1.smithi121.stdout:7/902: dwrite d0/d5/d4c/d6f/d127/d1c/d5f/f82 [0,4194304] 0 2024-02-15T18:35:50.798 INFO:tasks.workunit.client.1.smithi121.stdout:7/903: chown d0/d38/d3f/d43/fcf 2071248 1 2024-02-15T18:35:50.800 INFO:tasks.workunit.client.1.smithi121.stdout:7/904: creat d0/d38/f130 x:0 0 0 2024-02-15T18:35:50.802 INFO:tasks.workunit.client.1.smithi121.stdout:7/905: dread d0/d5/f63 [0,4194304] 0 2024-02-15T18:35:50.804 INFO:tasks.workunit.client.1.smithi121.stdout:7/906: dread d0/d5/d4c/d6f/d127/d5c/dcc/f9f [0,4194304] 0 2024-02-15T18:35:50.806 INFO:tasks.workunit.client.1.smithi121.stdout:7/907: mkdir d0/d38/d61/d131 0 2024-02-15T18:35:50.827 INFO:tasks.workunit.client.1.smithi121.stdout:7/908: dwrite d0/d5/d4c/d6f/f94 [0,4194304] 0 2024-02-15T18:35:50.842 INFO:tasks.workunit.client.1.smithi121.stdout:7/909: dwrite d0/d5/d4c/d6f/d127/d1c/d5f/fc5 [0,4194304] 0 2024-02-15T18:35:50.846 INFO:tasks.workunit.client.1.smithi121.stdout:7/910: rename d0/d38/l11a to d0/d4a/da2/l132 0 2024-02-15T18:35:50.846 INFO:tasks.workunit.client.1.smithi121.stdout:7/911: fsync d0/d38/fa0 0 2024-02-15T18:35:50.849 INFO:tasks.workunit.client.1.smithi121.stdout:7/912: rename d0/d5/d4c/d6f/d127/d5c/dcc/c121 to d0/d5/d4c/d6f/d127/d79/c133 0 2024-02-15T18:35:50.869 INFO:tasks.workunit.client.1.smithi121.stdout:7/913: dwrite d0/d38/d61/fe4 [0,4194304] 0 2024-02-15T18:35:50.869 INFO:tasks.workunit.client.1.smithi121.stdout:7/914: dread - d0/d5/fc3 zero size 2024-02-15T18:35:50.869 INFO:tasks.workunit.client.1.smithi121.stdout:7/915: chown d0/d5/d4c/d6f/d127/d5c/dcc 1676004 1 2024-02-15T18:35:50.888 INFO:tasks.workunit.client.1.smithi121.stdout:7/916: dwrite d0/d38/d3f/d43/f8e [0,4194304] 0 2024-02-15T18:35:50.904 INFO:tasks.workunit.client.1.smithi121.stdout:7/917: dwrite d0/d38/d61/f8a [0,4194304] 0 2024-02-15T18:35:50.910 INFO:tasks.workunit.client.1.smithi121.stdout:7/918: dread d0/d4a/d5b/da6/fbf [0,4194304] 0 2024-02-15T18:35:50.939 INFO:tasks.workunit.client.1.smithi121.stdout:7/919: dwrite d0/d4a/d5b/f92 [0,4194304] 0 2024-02-15T18:35:50.939 INFO:tasks.workunit.client.1.smithi121.stdout:7/920: chown d0/d38/fa0 5405 1 2024-02-15T18:35:50.944 INFO:tasks.workunit.client.1.smithi121.stdout:7/921: dread d0/d5/d4c/d6f/d127/f123 [0,4194304] 0 2024-02-15T18:35:50.945 INFO:tasks.workunit.client.1.smithi121.stdout:7/922: write d0/d5/f15 [6833651,72695] 0 2024-02-15T18:35:50.945 INFO:tasks.workunit.client.1.smithi121.stdout:7/923: truncate d0/d4a/da2/fb5 1023470 0 2024-02-15T18:35:50.946 INFO:tasks.workunit.client.1.smithi121.stdout:7/924: mkdir d0/d38/d61/d131/d134 0 2024-02-15T18:35:50.948 INFO:tasks.workunit.client.1.smithi121.stdout:7/925: rmdir d0/d5/d4c/d6f/db3 39 2024-02-15T18:35:50.948 INFO:tasks.workunit.client.1.smithi121.stdout:7/926: fsync d0/d5/d4c/d6f/d127/d5c/dcc/f4b 0 2024-02-15T18:35:50.948 INFO:tasks.workunit.client.1.smithi121.stdout:7/927: dread - d0/d4a/f114 zero size 2024-02-15T18:35:50.965 INFO:tasks.workunit.client.1.smithi121.stdout:7/928: dwrite d0/d4a/d5b/d9a/ff5 [4194304,4194304] 0 2024-02-15T18:35:50.966 INFO:tasks.workunit.client.1.smithi121.stdout:7/929: creat d0/d38/f135 x:0 0 0 2024-02-15T18:35:50.983 INFO:tasks.workunit.client.1.smithi121.stdout:7/930: dwrite d0/d38/d3f/d43/fcf [0,4194304] 0 2024-02-15T18:35:50.985 INFO:tasks.workunit.client.1.smithi121.stdout:7/931: creat d0/d4a/d5b/da6/f136 x:0 0 0 2024-02-15T18:35:50.985 INFO:tasks.workunit.client.1.smithi121.stdout:7/932: write d0/d4a/fd4 [597128,53069] 0 2024-02-15T18:35:50.986 INFO:tasks.workunit.client.1.smithi121.stdout:7/933: chown d0/d5/l41 917999 1 2024-02-15T18:35:50.986 INFO:tasks.workunit.client.1.smithi121.stdout:7/934: mkdir d0/d5/d4c/d116/d137 0 2024-02-15T18:35:50.986 INFO:tasks.workunit.client.1.smithi121.stdout:7/935: readlink d0/d4a/d5b/l6b 0 2024-02-15T18:35:50.987 INFO:tasks.workunit.client.1.smithi121.stdout:7/936: truncate d0/d4a/d5b/f92 3839134 0 2024-02-15T18:35:50.992 INFO:tasks.workunit.client.1.smithi121.stdout:7/937: dread d0/f27 [0,4194304] 0 2024-02-15T18:35:51.007 INFO:tasks.workunit.client.1.smithi121.stdout:7/938: dwrite d0/d38/d3f/d43/fcf [4194304,4194304] 0 2024-02-15T18:35:51.023 INFO:tasks.workunit.client.1.smithi121.stdout:7/939: dwrite d0/d5/d4c/d6f/d127/d5c/dcc/f9f [0,4194304] 0 2024-02-15T18:35:51.026 INFO:tasks.workunit.client.1.smithi121.stdout:7/940: fdatasync d0/d5/d4c/d6f/db3/fec 0 2024-02-15T18:35:51.044 INFO:tasks.workunit.client.1.smithi121.stdout:7/941: dwrite d0/d5/f26 [8388608,4194304] 0 2024-02-15T18:35:51.045 INFO:tasks.workunit.client.1.smithi121.stdout:7/942: link d0/d38/f130 d0/d38/d46/f138 0 2024-02-15T18:35:51.423 INFO:tasks.workunit.client.1.smithi121.stdout:7/943: sync 2024-02-15T18:35:51.424 INFO:tasks.workunit.client.1.smithi121.stdout:7/944: write d0/d5/d4c/d6f/d127/f48 [2331735,3145] 0 2024-02-15T18:35:51.424 INFO:tasks.workunit.client.1.smithi121.stdout:7/945: mknod d0/d5/d4c/d6f/dd5/de6/d11d/c139 0 2024-02-15T18:35:51.425 INFO:tasks.workunit.client.1.smithi121.stdout:7/946: creat d0/d5/d4c/d6f/d127/f13a x:0 0 0 2024-02-15T18:35:51.425 INFO:tasks.workunit.client.1.smithi121.stdout:7/947: fsync d0/d5/f26 0 2024-02-15T18:35:51.426 INFO:tasks.workunit.client.1.smithi121.stdout:7/948: mknod d0/d5/d4c/d6f/dd5/de6/d11d/c13b 0 2024-02-15T18:35:51.431 INFO:tasks.workunit.client.1.smithi121.stdout:7/949: dread d0/f29 [0,4194304] 0 2024-02-15T18:35:51.432 INFO:tasks.workunit.client.1.smithi121.stdout:7/950: creat d0/d38/f13c x:0 0 0 2024-02-15T18:35:51.433 INFO:tasks.workunit.client.1.smithi121.stdout:7/951: mkdir d0/d38/d3f/d43/d13d 0 2024-02-15T18:35:51.434 INFO:tasks.workunit.client.1.smithi121.stdout:7/952: link d0/d5/d4c/d6f/d127/d1c/d5f/fc5 d0/d5/d4c/d6f/d127/f13e 0 2024-02-15T18:35:51.436 INFO:tasks.workunit.client.1.smithi121.stdout:7/953: unlink d0/d38/f130 0 2024-02-15T18:35:51.436 INFO:tasks.workunit.client.1.smithi121.stdout:7/954: readlink d0/d4a/d5b/da6/l6c 0 2024-02-15T18:35:51.437 INFO:tasks.workunit.client.1.smithi121.stdout:7/955: mkdir d0/d13f 0 2024-02-15T18:35:51.437 INFO:tasks.workunit.client.1.smithi121.stdout:7/956: write d0/d4a/f7e [1369485,121120] 0 2024-02-15T18:35:51.437 INFO:tasks.workunit.client.1.smithi121.stdout:7/957: chown d0/d13f 1670279 1 2024-02-15T18:35:51.445 INFO:tasks.workunit.client.1.smithi121.stdout:7/958: dread d0/d5/d4c/d6f/d127/d1c/d5f/f73 [0,4194304] 0 2024-02-15T18:35:51.447 INFO:tasks.workunit.client.1.smithi121.stdout:7/959: getdents d0/d5/d4c/d6f/db6 0 2024-02-15T18:35:51.447 INFO:tasks.workunit.client.1.smithi121.stdout:7/960: truncate d0/d5/d4c/d6f/dd5/fff 724107 0 2024-02-15T18:35:51.447 INFO:tasks.workunit.client.1.smithi121.stdout:7/961: dread - d0/d4a/d5b/d9a/de9/f120 zero size 2024-02-15T18:35:51.447 INFO:tasks.workunit.client.1.smithi121.stdout:7/962: write d0/d4a/d5b/da6/ffc [178900,33690] 0 2024-02-15T18:35:51.457 INFO:tasks.workunit.client.1.smithi121.stdout:7/963: dread d0/d5/d4c/d6f/d127/d5c/dcc/f4d [0,4194304] 0 2024-02-15T18:35:51.457 INFO:tasks.workunit.client.1.smithi121.stdout:7/964: write d0/d38/f13c [603180,39508] 0 2024-02-15T18:35:51.457 INFO:tasks.workunit.client.1.smithi121.stdout:7/965: read - d0/d38/fab zero size 2024-02-15T18:35:51.457 INFO:tasks.workunit.client.1.smithi121.stdout:7/966: stat d0/d4a/f7e 0 2024-02-15T18:35:51.462 INFO:tasks.workunit.client.1.smithi121.stdout:7/967: dread d0/d4a/d5b/da6/ffc [0,4194304] 0 2024-02-15T18:35:51.462 INFO:tasks.workunit.client.1.smithi121.stdout:7/968: write d0/d5/d76/f10c [1648857,35078] 0 2024-02-15T18:35:51.463 INFO:tasks.workunit.client.1.smithi121.stdout:7/969: symlink d0/d5/d4c/d6f/d127/d1c/d5f/dc0/l140 0 2024-02-15T18:35:51.465 INFO:tasks.workunit.client.1.smithi121.stdout:7/970: dread d0/d4a/d5b/da6/fbf [0,4194304] 0 2024-02-15T18:35:51.467 INFO:tasks.workunit.client.1.smithi121.stdout:7/971: symlink d0/d4a/d5b/da6/l141 0 2024-02-15T18:35:51.467 INFO:tasks.workunit.client.1.smithi121.stdout:7/972: dread - d0/d5/fc3 zero size 2024-02-15T18:35:51.467 INFO:tasks.workunit.client.1.smithi121.stdout:7/973: chown d0/d38/cfe 28 1 2024-02-15T18:35:51.468 INFO:tasks.workunit.client.1.smithi121.stdout:7/974: rename d0/d5/d4c/d116 to d0/d38/d61/d131/d134/d142 0 2024-02-15T18:35:51.470 INFO:tasks.workunit.client.1.smithi121.stdout:7/975: creat d0/d5/d4c/d6f/db6/f143 x:0 0 0 2024-02-15T18:35:51.470 INFO:tasks.workunit.client.1.smithi121.stdout:7/976: write d0/d4a/d5b/fa8 [443290,89] 0 2024-02-15T18:35:51.472 INFO:tasks.workunit.client.1.smithi121.stdout:7/977: rename d0/d5/d4c/d6f/d127/l30 to d0/d5/d4c/dda/l144 0 2024-02-15T18:35:51.473 INFO:tasks.workunit.client.1.smithi121.stdout:7/978: write d0/d5/d4c/d6f/d127/d1c/d5f/fc5 [2567128,85392] 0 2024-02-15T18:35:51.481 INFO:tasks.workunit.client.1.smithi121.stdout:7/979: truncate d0/d5/d4c/d6f/d127/f48 2218291 0 2024-02-15T18:35:51.482 INFO:tasks.workunit.client.1.smithi121.stdout:7/980: creat d0/d38/d3f/d43/f145 x:0 0 0 2024-02-15T18:35:51.483 INFO:tasks.workunit.client.1.smithi121.stdout:7/981: mknod d0/d5/d4c/d6f/d127/d79/c146 0 2024-02-15T18:35:51.488 INFO:tasks.workunit.client.1.smithi121.stdout:7/982: link d0/d5/d4c/d6f/db6/fbc d0/d13f/f147 0 2024-02-15T18:35:51.488 INFO:tasks.workunit.client.1.smithi121.stdout:7/983: mknod d0/d5/c148 0 2024-02-15T18:35:51.488 INFO:tasks.workunit.client.1.smithi121.stdout:7/984: creat d0/d4a/f149 x:0 0 0 2024-02-15T18:35:51.488 INFO:tasks.workunit.client.1.smithi121.stdout:7/985: fdatasync d0/d5/d4c/d6f/d127/f91 0 2024-02-15T18:35:51.489 INFO:tasks.workunit.client.1.smithi121.stdout:7/986: creat d0/d4a/f14a x:0 0 0 2024-02-15T18:35:51.489 INFO:tasks.workunit.client.1.smithi121.stdout:7/987: mkdir d0/d5/d4c/d6f/dd5/de6/d11d/d14b 0 2024-02-15T18:35:51.490 INFO:tasks.workunit.client.1.smithi121.stdout:7/988: write d0/fb [330684,49534] 0 2024-02-15T18:35:51.491 INFO:tasks.workunit.client.1.smithi121.stdout:7/989: write d0/d5/d4c/d6f/d127/d5c/fdd [2980934,127598] 0 2024-02-15T18:35:51.492 INFO:tasks.workunit.client.1.smithi121.stdout:7/990: stat d0/d38/d3f 0 2024-02-15T18:35:51.492 INFO:tasks.workunit.client.1.smithi121.stdout:7/991: stat d0/d5/d4c/d6f/d127/d79/cce 0 2024-02-15T18:35:51.493 INFO:tasks.workunit.client.1.smithi121.stdout:7/992: link d0/d5/d4c/d6f/d127/f123 d0/d124/f14c 0 2024-02-15T18:35:51.497 INFO:tasks.workunit.client.1.smithi121.stdout:7/993: mkdir d0/d38/d14d 0 2024-02-15T18:35:51.499 INFO:tasks.workunit.client.1.smithi121.stdout:7/994: rename d0/d38/l88 to d0/d38/d46/daa/df7/l14e 0 2024-02-15T18:35:51.500 INFO:tasks.workunit.client.1.smithi121.stdout:7/995: symlink d0/d4a/d5b/d9a/de9/l14f 0 2024-02-15T18:35:51.500 INFO:tasks.workunit.client.1.smithi121.stdout:7/996: truncate d0/d4a/f14a 32862 0 2024-02-15T18:35:51.500 INFO:tasks.workunit.client.1.smithi121.stdout:7/997: stat d0/d5/d4c/d6f/d127/d5c/dcc 0 2024-02-15T18:35:51.500 INFO:tasks.workunit.client.1.smithi121.stdout:7/998: truncate d0/d5/d4c/d6f/d127/f13a 842425 0 2024-02-15T18:35:51.523 INFO:tasks.workunit.client.1.smithi121.stdout:7/999: dwrite d0/d5/d4c/d6f/d127/d5c/fdd [4194304,4194304] 0 2024-02-15T18:35:51.527 INFO:tasks.workunit.client.1.smithi121.stderr:+ rm -rf -- ./tmp.qtnpNMw7fT 2024-02-15T18:35:51.712 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:49.854475+0000 mgr.smithi121.ckiryi (mgr.24385) 26 : audit [DBG] from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:35:51.712 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:35:50.013327+0000 mgr.smithi121.ckiryi (mgr.24385) 27 : cephadm [INF] Upgrade: Updating mgr.smithi038.khesbl 2024-02-15T18:35:51.713 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:35:50.023380+0000 mgr.smithi121.ckiryi (mgr.24385) 28 : cephadm [INF] Deploying daemon mgr.smithi038.khesbl on smithi038 2024-02-15T18:35:51.713 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:50.598447+0000 mgr.smithi121.ckiryi (mgr.24385) 29 : cluster [DBG] pgmap v19: 65 pgs: 65 active+clean; 2.3 GiB data, 6.9 GiB used, 529 GiB / 536 GiB avail; 87 MiB/s rd, 187 MiB/s wr, 600 op/s 2024-02-15T18:35:51.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:49.854475+0000 mgr.smithi121.ckiryi (mgr.24385) 26 : audit [DBG] from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:35:51.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:35:50.013327+0000 mgr.smithi121.ckiryi (mgr.24385) 27 : cephadm [INF] Upgrade: Updating mgr.smithi038.khesbl 2024-02-15T18:35:51.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:35:50.023380+0000 mgr.smithi121.ckiryi (mgr.24385) 28 : cephadm [INF] Deploying daemon mgr.smithi038.khesbl on smithi038 2024-02-15T18:35:51.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:50.598447+0000 mgr.smithi121.ckiryi (mgr.24385) 29 : cluster [DBG] pgmap v19: 65 pgs: 65 active+clean; 2.3 GiB data, 6.9 GiB used, 529 GiB / 536 GiB avail; 87 MiB/s rd, 187 MiB/s wr, 600 op/s 2024-02-15T18:35:52.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:52 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:52.598890+0000 mgr.smithi121.ckiryi (mgr.24385) 30 : cluster [DBG] pgmap v20: 65 pgs: 65 active+clean; 2.3 GiB data, 6.9 GiB used, 529 GiB / 536 GiB avail; 65 MiB/s rd, 133 MiB/s wr, 400 op/s 2024-02-15T18:35:53.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:52.598890+0000 mgr.smithi121.ckiryi (mgr.24385) 30 : cluster [DBG] pgmap v20: 65 pgs: 65 active+clean; 2.3 GiB data, 6.9 GiB used, 529 GiB / 536 GiB avail; 65 MiB/s rd, 133 MiB/s wr, 400 op/s 2024-02-15T18:35:55.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:54.147409+0000 mon.smithi038 (mon.0) 608 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:55.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:35:54.148861+0000 mon.smithi121 (mon.1) 51 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:35:55.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:54.599273+0000 mgr.smithi121.ckiryi (mgr.24385) 31 : cluster [DBG] pgmap v21: 65 pgs: 65 active+clean; 2.3 GiB data, 6.9 GiB used, 529 GiB / 536 GiB avail; 65 MiB/s rd, 133 MiB/s wr, 400 op/s 2024-02-15T18:35:55.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:54.147409+0000 mon.smithi038 (mon.0) 608 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:35:55.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:35:54.148861+0000 mon.smithi121 (mon.1) 51 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:35:55.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:54.599273+0000 mgr.smithi121.ckiryi (mgr.24385) 31 : cluster [DBG] pgmap v21: 65 pgs: 65 active+clean; 2.3 GiB data, 6.9 GiB used, 529 GiB / 536 GiB avail; 65 MiB/s rd, 133 MiB/s wr, 400 op/s 2024-02-15T18:35:57.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:56.599696+0000 mgr.smithi121.ckiryi (mgr.24385) 32 : cluster [DBG] pgmap v22: 65 pgs: 65 active+clean; 2.0 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 70 MiB/s rd, 142 MiB/s wr, 561 op/s 2024-02-15T18:35:57.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:56.599696+0000 mgr.smithi121.ckiryi (mgr.24385) 32 : cluster [DBG] pgmap v22: 65 pgs: 65 active+clean; 2.0 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 70 MiB/s rd, 142 MiB/s wr, 561 op/s 2024-02-15T18:35:59.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:35:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:35:58.600000+0000 mgr.smithi121.ckiryi (mgr.24385) 33 : cluster [DBG] pgmap v23: 65 pgs: 65 active+clean; 2.0 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 32 MiB/s rd, 57 MiB/s wr, 358 op/s 2024-02-15T18:35:59.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:35:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:35:58.600000+0000 mgr.smithi121.ckiryi (mgr.24385) 33 : cluster [DBG] pgmap v23: 65 pgs: 65 active+clean; 2.0 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 32 MiB/s rd, 57 MiB/s wr, 358 op/s 2024-02-15T18:36:00.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:00.600453+0000 mgr.smithi121.ckiryi (mgr.24385) 34 : cluster [DBG] pgmap v24: 65 pgs: 65 active+clean; 1.3 GiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 32 MiB/s rd, 58 MiB/s wr, 487 op/s 2024-02-15T18:36:00.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:00.607865+0000 mon.smithi038 (mon.0) 609 : cluster [DBG] Standby manager daemon smithi038.khesbl restarted 2024-02-15T18:36:00.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:00.607945+0000 mon.smithi038 (mon.0) 610 : cluster [DBG] Standby manager daemon smithi038.khesbl started 2024-02-15T18:36:00.861 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:00.610236+0000 mon.smithi038 (mon.0) 611 : audit [DBG] from='mgr.? 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi038.khesbl/crt"}]: dispatch 2024-02-15T18:36:00.861 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:00.610647+0000 mon.smithi038 (mon.0) 612 : audit [DBG] from='mgr.? 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:36:00.861 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:00.613187+0000 mon.smithi038 (mon.0) 613 : audit [DBG] from='mgr.? 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi038.khesbl/key"}]: dispatch 2024-02-15T18:36:00.861 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:00.614091+0000 mon.smithi038 (mon.0) 614 : audit [DBG] from='mgr.? 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:36:01.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:00.600453+0000 mgr.smithi121.ckiryi (mgr.24385) 34 : cluster [DBG] pgmap v24: 65 pgs: 65 active+clean; 1.3 GiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 32 MiB/s rd, 58 MiB/s wr, 487 op/s 2024-02-15T18:36:01.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:00.607865+0000 mon.smithi038 (mon.0) 609 : cluster [DBG] Standby manager daemon smithi038.khesbl restarted 2024-02-15T18:36:01.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:00.607945+0000 mon.smithi038 (mon.0) 610 : cluster [DBG] Standby manager daemon smithi038.khesbl started 2024-02-15T18:36:01.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:00.610236+0000 mon.smithi038 (mon.0) 611 : audit [DBG] from='mgr.? 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi038.khesbl/crt"}]: dispatch 2024-02-15T18:36:01.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:00.610647+0000 mon.smithi038 (mon.0) 612 : audit [DBG] from='mgr.? 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:36:01.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:00.613187+0000 mon.smithi038 (mon.0) 613 : audit [DBG] from='mgr.? 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi038.khesbl/key"}]: dispatch 2024-02-15T18:36:01.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:00.614091+0000 mon.smithi038 (mon.0) 614 : audit [DBG] from='mgr.? 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:36:01.950 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:00.669605+0000 mon.smithi038 (mon.0) 615 : cluster [DBG] mgrmap e24: smithi121.ckiryi(active, since 43s), standbys: smithi038.khesbl 2024-02-15T18:36:02.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:00.669605+0000 mon.smithi038 (mon.0) 615 : cluster [DBG] mgrmap e24: smithi121.ckiryi(active, since 43s), standbys: smithi038.khesbl 2024-02-15T18:36:03.145 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:01.890333+0000 mon.smithi038 (mon.0) 616 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:36:03.145 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:01.894572+0000 mon.smithi121 (mon.1) 52 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:36:03.146 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:01.895559+0000 mon.smithi121 (mon.1) 53 : audit [INF] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:36:03.146 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:36:01.896581+0000 mgr.smithi121.ckiryi (mgr.24385) 35 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:36:03.146 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:01.902187+0000 mon.smithi038 (mon.0) 617 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:36:03.146 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:01.914648+0000 mon.smithi121 (mon.1) 54 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:03.146 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:36:01.919066+0000 mgr.smithi121.ckiryi (mgr.24385) 36 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi121.ckiryi) 2024-02-15T18:36:03.146 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:36:01.919625+0000 mgr.smithi121.ckiryi (mgr.24385) 37 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi121.ckiryi) 2024-02-15T18:36:03.146 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:36:01.921790+0000 mgr.smithi121.ckiryi (mgr.24385) 38 : cephadm [INF] Failing over to other MGR 2024-02-15T18:36:03.146 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:01.922101+0000 mon.smithi121 (mon.1) 55 : audit [INF] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mgr fail", "who": "smithi121.ckiryi"}]: dispatch 2024-02-15T18:36:03.146 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:01.922856+0000 mon.smithi038 (mon.0) 618 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mgr fail", "who": "smithi121.ckiryi"}]: dispatch 2024-02-15T18:36:03.146 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:01.928166+0000 mon.smithi038 (mon.0) 619 : cluster [DBG] osdmap e45: 6 total, 6 up, 6 in 2024-02-15T18:36:03.146 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:01.937118+0000 mon.smithi121 (mon.1) 56 : audit [INF] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-15T18:36:03.146 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:01.938477+0000 mon.smithi038 (mon.0) 620 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-15T18:36:03.146 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:01.942023+0000 mon.smithi121 (mon.1) 57 : audit [INF] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-15T18:36:03.146 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:01.942604+0000 mon.smithi038 (mon.0) 621 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-15T18:36:03.146 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:02.010271+0000 mon.smithi038 (mon.0) 622 : cluster [DBG] Standby manager daemon smithi121.ckiryi started 2024-02-15T18:36:03.146 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:02.600777+0000 mgr.smithi121.ckiryi (mgr.24385) 39 : cluster [DBG] pgmap v26: 65 pgs: 65 active+clean; 1.3 GiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 348 op/s 2024-02-15T18:36:03.157 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:01.890333+0000 mon.smithi038 (mon.0) 616 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:36:03.157 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:01.894572+0000 mon.smithi121 (mon.1) 52 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:36:03.157 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:01.895559+0000 mon.smithi121 (mon.1) 53 : audit [INF] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:36:03.157 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:36:01.896581+0000 mgr.smithi121.ckiryi (mgr.24385) 35 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:36:03.157 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:01.902187+0000 mon.smithi038 (mon.0) 617 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' 2024-02-15T18:36:03.157 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:01.914648+0000 mon.smithi121 (mon.1) 54 : audit [DBG] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:03.158 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:36:01.919066+0000 mgr.smithi121.ckiryi (mgr.24385) 36 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi121.ckiryi) 2024-02-15T18:36:03.158 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:36:01.919625+0000 mgr.smithi121.ckiryi (mgr.24385) 37 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi121.ckiryi) 2024-02-15T18:36:03.158 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:36:01.921790+0000 mgr.smithi121.ckiryi (mgr.24385) 38 : cephadm [INF] Failing over to other MGR 2024-02-15T18:36:03.158 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:01.922101+0000 mon.smithi121 (mon.1) 55 : audit [INF] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mgr fail", "who": "smithi121.ckiryi"}]: dispatch 2024-02-15T18:36:03.158 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:01.922856+0000 mon.smithi038 (mon.0) 618 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "mgr fail", "who": "smithi121.ckiryi"}]: dispatch 2024-02-15T18:36:03.158 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:01.928166+0000 mon.smithi038 (mon.0) 619 : cluster [DBG] osdmap e45: 6 total, 6 up, 6 in 2024-02-15T18:36:03.158 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:01.937118+0000 mon.smithi121 (mon.1) 56 : audit [INF] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-15T18:36:03.158 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:01.938477+0000 mon.smithi038 (mon.0) 620 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-15T18:36:03.158 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:01.942023+0000 mon.smithi121 (mon.1) 57 : audit [INF] from='mgr.24385 172.21.15.121:0/117536316' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-15T18:36:03.158 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:01.942604+0000 mon.smithi038 (mon.0) 621 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-15T18:36:03.158 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:02.010271+0000 mon.smithi038 (mon.0) 622 : cluster [DBG] Standby manager daemon smithi121.ckiryi started 2024-02-15T18:36:03.158 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:02.600777+0000 mgr.smithi121.ckiryi (mgr.24385) 39 : cluster [DBG] pgmap v26: 65 pgs: 65 active+clean; 1.3 GiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 348 op/s 2024-02-15T18:36:04.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:02.910614+0000 mon.smithi038 (mon.0) 623 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' cmd='[{"prefix": "mgr fail", "who": "smithi121.ckiryi"}]': finished 2024-02-15T18:36:04.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:02.912123+0000 mon.smithi038 (mon.0) 624 : cluster [DBG] mgrmap e25: smithi038.khesbl(active, starting, since 0.988346s), standbys: smithi121.ckiryi 2024-02-15T18:36:04.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:02.915331+0000 mon.smithi038 (mon.0) 625 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-02-15T18:36:04.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:02.915560+0000 mon.smithi038 (mon.0) 626 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:36:04.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:02.918528+0000 mon.smithi038 (mon.0) 627 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.hxoggl"}]: dispatch 2024-02-15T18:36:04.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:02.918809+0000 mon.smithi038 (mon.0) 628 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.qgkzgn"}]: dispatch 2024-02-15T18:36:04.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:02.919050+0000 mon.smithi038 (mon.0) 629 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.mvumlh"}]: dispatch 2024-02-15T18:36:04.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:02.919288+0000 mon.smithi038 (mon.0) 630 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.ovuzki"}]: dispatch 2024-02-15T18:36:04.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:02.919566+0000 mon.smithi038 (mon.0) 631 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mgr metadata", "who": "smithi038.khesbl", "id": "smithi038.khesbl"}]: dispatch 2024-02-15T18:36:04.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:02.919764+0000 mon.smithi038 (mon.0) 632 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mgr metadata", "who": "smithi121.ckiryi", "id": "smithi121.ckiryi"}]: dispatch 2024-02-15T18:36:04.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:02.919948+0000 mon.smithi038 (mon.0) 633 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:36:04.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:02.920178+0000 mon.smithi038 (mon.0) 634 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:36:04.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:02.920422+0000 mon.smithi038 (mon.0) 635 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:36:04.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:02.920673+0000 mon.smithi038 (mon.0) 636 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:36:04.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:02.920900+0000 mon.smithi038 (mon.0) 637 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:36:04.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:02.921098+0000 mon.smithi038 (mon.0) 638 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:36:04.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:02.921296+0000 mon.smithi038 (mon.0) 639 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-15T18:36:04.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:02.921640+0000 mon.smithi038 (mon.0) 640 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-15T18:36:04.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:02.921994+0000 mon.smithi038 (mon.0) 641 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-15T18:36:04.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:02.937974+0000 mon.smithi038 (mon.0) 642 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-15T18:36:04.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:02.938022+0000 mon.smithi038 (mon.0) 643 : cluster [DBG] osdmap e46: 6 total, 6 up, 6 in 2024-02-15T18:36:04.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:02.952204+0000 mon.smithi038 (mon.0) 644 : cluster [INF] Manager daemon smithi038.khesbl is now available 2024-02-15T18:36:04.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:02.975856+0000 mon.smithi038 (mon.0) 645 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:04.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:03.108861+0000 mon.smithi038 (mon.0) 646 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:04.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:03.111427+0000 mon.smithi038 (mon.0) 647 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:04.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:03.170198+0000 mon.smithi038 (mon.0) 648 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:36:04.109 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:03.186132+0000 mon.smithi038 (mon.0) 649 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:36:04.189 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:02.910614+0000 mon.smithi038 (mon.0) 623 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' cmd='[{"prefix": "mgr fail", "who": "smithi121.ckiryi"}]': finished 2024-02-15T18:36:04.189 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:02.912123+0000 mon.smithi038 (mon.0) 624 : cluster [DBG] mgrmap e25: smithi038.khesbl(active, starting, since 0.988346s), standbys: smithi121.ckiryi 2024-02-15T18:36:04.189 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:02.915331+0000 mon.smithi038 (mon.0) 625 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-02-15T18:36:04.189 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:02.915560+0000 mon.smithi038 (mon.0) 626 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:36:04.189 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:02.918528+0000 mon.smithi038 (mon.0) 627 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.hxoggl"}]: dispatch 2024-02-15T18:36:04.190 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:02.918809+0000 mon.smithi038 (mon.0) 628 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.qgkzgn"}]: dispatch 2024-02-15T18:36:04.190 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:02.919050+0000 mon.smithi038 (mon.0) 629 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.mvumlh"}]: dispatch 2024-02-15T18:36:04.190 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:02.919288+0000 mon.smithi038 (mon.0) 630 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.ovuzki"}]: dispatch 2024-02-15T18:36:04.190 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:02.919566+0000 mon.smithi038 (mon.0) 631 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mgr metadata", "who": "smithi038.khesbl", "id": "smithi038.khesbl"}]: dispatch 2024-02-15T18:36:04.190 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:02.919764+0000 mon.smithi038 (mon.0) 632 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mgr metadata", "who": "smithi121.ckiryi", "id": "smithi121.ckiryi"}]: dispatch 2024-02-15T18:36:04.190 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:02.919948+0000 mon.smithi038 (mon.0) 633 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:36:04.190 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:02.920178+0000 mon.smithi038 (mon.0) 634 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:36:04.190 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:02.920422+0000 mon.smithi038 (mon.0) 635 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:36:04.190 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:02.920673+0000 mon.smithi038 (mon.0) 636 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:36:04.190 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:02.920900+0000 mon.smithi038 (mon.0) 637 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:36:04.190 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:02.921098+0000 mon.smithi038 (mon.0) 638 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:36:04.190 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:02.921296+0000 mon.smithi038 (mon.0) 639 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-15T18:36:04.190 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:02.921640+0000 mon.smithi038 (mon.0) 640 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-15T18:36:04.190 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:02.921994+0000 mon.smithi038 (mon.0) 641 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-15T18:36:04.191 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:02.937974+0000 mon.smithi038 (mon.0) 642 : audit [INF] from='mgr.24385 ' entity='mgr.smithi121.ckiryi' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-15T18:36:04.191 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:02.938022+0000 mon.smithi038 (mon.0) 643 : cluster [DBG] osdmap e46: 6 total, 6 up, 6 in 2024-02-15T18:36:04.191 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:02.952204+0000 mon.smithi038 (mon.0) 644 : cluster [INF] Manager daemon smithi038.khesbl is now available 2024-02-15T18:36:04.191 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:02.975856+0000 mon.smithi038 (mon.0) 645 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:04.191 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:03.108861+0000 mon.smithi038 (mon.0) 646 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:04.191 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:03.111427+0000 mon.smithi038 (mon.0) 647 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:04.191 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:03.170198+0000 mon.smithi038 (mon.0) 648 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:36:04.191 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:03.186132+0000 mon.smithi038 (mon.0) 649 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi038.khesbl/trash_purge_schedule"}]: dispatch 2024-02-15T18:36:05.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:04 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:36:02.968389+0000 mgr.smithi038.khesbl (mgr.14664) 1 : cephadm [INF] inventory: adjusted host smithi121 addr 'smithi121' -> '172.21.15.121' 2024-02-15T18:36:05.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:04 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:03.950595+0000 mon.smithi038 (mon.0) 650 : cluster [DBG] mgrmap e26: smithi038.khesbl(active, since 2s), standbys: smithi121.ckiryi 2024-02-15T18:36:05.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:04 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:03.963505+0000 mgr.smithi038.khesbl (mgr.14664) 2 : cluster [DBG] pgmap v4: 65 pgs: 65 active+clean; 757 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:36:05.227 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:04 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:36:02.968389+0000 mgr.smithi038.khesbl (mgr.14664) 1 : cephadm [INF] inventory: adjusted host smithi121 addr 'smithi121' -> '172.21.15.121' 2024-02-15T18:36:05.228 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:04 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:03.950595+0000 mon.smithi038 (mon.0) 650 : cluster [DBG] mgrmap e26: smithi038.khesbl(active, since 2s), standbys: smithi121.ckiryi 2024-02-15T18:36:05.228 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:04 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:03.963505+0000 mgr.smithi038.khesbl (mgr.14664) 2 : cluster [DBG] pgmap v4: 65 pgs: 65 active+clean; 757 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:36:06.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:04.919822+0000 mgr.smithi038.khesbl (mgr.14664) 3 : cluster [DBG] pgmap v5: 65 pgs: 65 active+clean; 757 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:36:06.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:05.378355+0000 mon.smithi038 (mon.0) 651 : cluster [DBG] mgrmap e27: smithi038.khesbl(active, since 3s), standbys: smithi121.ckiryi 2024-02-15T18:36:06.321 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-15T18:36:06.321 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-02-15T18:36:06.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:04.919822+0000 mgr.smithi038.khesbl (mgr.14664) 3 : cluster [DBG] pgmap v5: 65 pgs: 65 active+clean; 757 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:36:06.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:05.378355+0000 mon.smithi038 (mon.0) 651 : cluster [DBG] mgrmap e27: smithi038.khesbl(active, since 3s), standbys: smithi121.ckiryi 2024-02-15T18:36:08.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:06.920180+0000 mgr.smithi038.khesbl (mgr.14664) 4 : cluster [DBG] pgmap v6: 65 pgs: 65 active+clean; 757 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:36:08.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:06.920180+0000 mgr.smithi038.khesbl (mgr.14664) 4 : cluster [DBG] pgmap v6: 65 pgs: 65 active+clean; 757 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:36:09.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:08.246189+0000 mon.smithi038 (mon.0) 652 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:09.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:08.922113+0000 mgr.smithi038.khesbl (mgr.14664) 5 : cluster [DBG] pgmap v7: 65 pgs: 65 active+clean; 757 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:36:09.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:08.246189+0000 mon.smithi038 (mon.0) 652 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:09.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:08.922113+0000 mgr.smithi038.khesbl (mgr.14664) 5 : cluster [DBG] pgmap v7: 65 pgs: 65 active+clean; 757 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:36:10.555 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:10.402893+0000 mon.smithi038 (mon.0) 653 : cluster [DBG] Standby manager daemon smithi121.ckiryi restarted 2024-02-15T18:36:10.555 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:10.402979+0000 mon.smithi038 (mon.0) 654 : cluster [DBG] Standby manager daemon smithi121.ckiryi started 2024-02-15T18:36:10.555 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:10.404453+0000 mon.smithi121 (mon.1) 58 : audit [DBG] from='mgr.? 172.21.15.121:0/3731930017' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi121.ckiryi/crt"}]: dispatch 2024-02-15T18:36:10.556 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:10.405711+0000 mon.smithi121 (mon.1) 59 : audit [DBG] from='mgr.? 172.21.15.121:0/3731930017' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:36:10.556 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:10.408637+0000 mon.smithi121 (mon.1) 60 : audit [DBG] from='mgr.? 172.21.15.121:0/3731930017' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi121.ckiryi/key"}]: dispatch 2024-02-15T18:36:10.556 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:10.409084+0000 mon.smithi121 (mon.1) 61 : audit [DBG] from='mgr.? 172.21.15.121:0/3731930017' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:36:10.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:10.402893+0000 mon.smithi038 (mon.0) 653 : cluster [DBG] Standby manager daemon smithi121.ckiryi restarted 2024-02-15T18:36:10.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:10.402979+0000 mon.smithi038 (mon.0) 654 : cluster [DBG] Standby manager daemon smithi121.ckiryi started 2024-02-15T18:36:10.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:10.404453+0000 mon.smithi121 (mon.1) 58 : audit [DBG] from='mgr.? 172.21.15.121:0/3731930017' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi121.ckiryi/crt"}]: dispatch 2024-02-15T18:36:10.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:10.405711+0000 mon.smithi121 (mon.1) 59 : audit [DBG] from='mgr.? 172.21.15.121:0/3731930017' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:36:10.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:10.408637+0000 mon.smithi121 (mon.1) 60 : audit [DBG] from='mgr.? 172.21.15.121:0/3731930017' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi121.ckiryi/key"}]: dispatch 2024-02-15T18:36:10.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:10.409084+0000 mon.smithi121 (mon.1) 61 : audit [DBG] from='mgr.? 172.21.15.121:0/3731930017' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:36:11.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:10.460276+0000 mon.smithi038 (mon.0) 655 : cluster [DBG] mgrmap e28: smithi038.khesbl(active, since 8s), standbys: smithi121.ckiryi 2024-02-15T18:36:11.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:10.922635+0000 mgr.smithi038.khesbl (mgr.14664) 6 : cluster [DBG] pgmap v8: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 928 KiB/s wr, 147 op/s 2024-02-15T18:36:11.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:10.460276+0000 mon.smithi038 (mon.0) 655 : cluster [DBG] mgrmap e28: smithi038.khesbl(active, since 8s), standbys: smithi121.ckiryi 2024-02-15T18:36:11.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:10.922635+0000 mgr.smithi038.khesbl (mgr.14664) 6 : cluster [DBG] pgmap v8: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 928 KiB/s wr, 147 op/s 2024-02-15T18:36:14.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:12.805411+0000 mon.smithi038 (mon.0) 656 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:14.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:12.922951+0000 mgr.smithi038.khesbl (mgr.14664) 7 : cluster [DBG] pgmap v9: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 721 KiB/s wr, 114 op/s 2024-02-15T18:36:14.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:13.190038+0000 mon.smithi038 (mon.0) 657 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:36:14.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:13.191387+0000 mon.smithi038 (mon.0) 658 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:36:14.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:13.198306+0000 mon.smithi038 (mon.0) 659 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:14.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:13.205186+0000 mon.smithi038 (mon.0) 660 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:14.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:12.805411+0000 mon.smithi038 (mon.0) 656 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:14.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:12.922951+0000 mgr.smithi038.khesbl (mgr.14664) 7 : cluster [DBG] pgmap v9: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 721 KiB/s wr, 114 op/s 2024-02-15T18:36:14.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:13.190038+0000 mon.smithi038 (mon.0) 657 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:36:14.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:13.191387+0000 mon.smithi038 (mon.0) 658 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:36:14.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:13.198306+0000 mon.smithi038 (mon.0) 659 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:14.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:13.205186+0000 mon.smithi038 (mon.0) 660 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:15.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:36:13.192272+0000 mgr.smithi038.khesbl (mgr.14664) 8 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:36:15.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:36:13.192272+0000 mgr.smithi038.khesbl (mgr.14664) 8 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:36:16.084 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:14.923426+0000 mgr.smithi038.khesbl (mgr.14664) 9 : cluster [DBG] pgmap v10: 65 pgs: 65 active+clean; 300 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.2 MiB/s wr, 241 op/s 2024-02-15T18:36:16.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:14.923426+0000 mgr.smithi038.khesbl (mgr.14664) 9 : cluster [DBG] pgmap v10: 65 pgs: 65 active+clean; 300 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.2 MiB/s wr, 241 op/s 2024-02-15T18:36:17.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:16.923850+0000 mgr.smithi038.khesbl (mgr.14664) 10 : cluster [DBG] pgmap v11: 65 pgs: 65 active+clean; 300 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 1.1 MiB/s wr, 220 op/s 2024-02-15T18:36:17.342 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:16.923850+0000 mgr.smithi038.khesbl (mgr.14664) 10 : cluster [DBG] pgmap v11: 65 pgs: 65 active+clean; 300 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 1.1 MiB/s wr, 220 op/s 2024-02-15T18:36:18.240 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:36:17.376739+0000 mgr.smithi038.khesbl (mgr.14664) 11 : cephadm [INF] Upgrade: Updating mgr.smithi121.ckiryi 2024-02-15T18:36:18.241 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:17.377235+0000 mon.smithi038 (mon.0) 661 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi121.ckiryi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:36:18.241 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:17.378280+0000 mon.smithi038 (mon.0) 662 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:36:18.241 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:17.379584+0000 mon.smithi038 (mon.0) 663 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:36:18.241 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:36:17.380503+0000 mgr.smithi038.khesbl (mgr.14664) 12 : cephadm [INF] Deploying daemon mgr.smithi121.ckiryi on smithi121 2024-02-15T18:36:18.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:36:17.376739+0000 mgr.smithi038.khesbl (mgr.14664) 11 : cephadm [INF] Upgrade: Updating mgr.smithi121.ckiryi 2024-02-15T18:36:18.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:17.377235+0000 mon.smithi038 (mon.0) 661 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi121.ckiryi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:36:18.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:17.378280+0000 mon.smithi038 (mon.0) 662 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:36:18.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:17.379584+0000 mon.smithi038 (mon.0) 663 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:36:18.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:36:17.380503+0000 mgr.smithi038.khesbl (mgr.14664) 12 : cephadm [INF] Deploying daemon mgr.smithi121.ckiryi on smithi121 2024-02-15T18:36:19.026 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:18.924263+0000 mgr.smithi038.khesbl (mgr.14664) 13 : cluster [DBG] pgmap v12: 65 pgs: 65 active+clean; 300 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 1.1 MiB/s wr, 220 op/s 2024-02-15T18:36:19.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:18.924263+0000 mgr.smithi038.khesbl (mgr.14664) 13 : cluster [DBG] pgmap v12: 65 pgs: 65 active+clean; 300 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 1.1 MiB/s wr, 220 op/s 2024-02-15T18:36:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.994+0000 7fb94489e700 1 -- 172.21.15.38:0/438412470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9400f55c0 msgr2=0x7fb9400f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.994+0000 7fb94489e700 1 --2- 172.21.15.38:0/438412470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9400f55c0 0x7fb9400f59e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fb930004660 tx=0x7fb930009b30).stop 2024-02-15T18:36:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.995+0000 7fb94489e700 1 -- 172.21.15.38:0/438412470 shutdown_connections 2024-02-15T18:36:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.995+0000 7fb94489e700 1 --2- 172.21.15.38:0/438412470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9400f6860 0x7fb9400f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.995+0000 7fb94489e700 1 --2- 172.21.15.38:0/438412470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9400f55c0 0x7fb9400f59e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.995+0000 7fb94489e700 1 -- 172.21.15.38:0/438412470 >> 172.21.15.38:0/438412470 conn(0x7fb9400f0d40 msgr2=0x7fb9400f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.995+0000 7fb94489e700 1 -- 172.21.15.38:0/438412470 shutdown_connections 2024-02-15T18:36:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.995+0000 7fb94489e700 1 -- 172.21.15.38:0/438412470 wait complete. 2024-02-15T18:36:19.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.995+0000 7fb94489e700 1 Processor -- start 2024-02-15T18:36:19.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.995+0000 7fb94489e700 1 -- start start 2024-02-15T18:36:19.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.995+0000 7fb94489e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9400f55c0 0x7fb9401091f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:19.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.995+0000 7fb94489e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9400f6860 0x7fb9401096e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:19.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.995+0000 7fb94489e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb940109c90 con 0x7fb9400f55c0 2024-02-15T18:36:19.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.995+0000 7fb94489e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb940109dd0 con 0x7fb9400f6860 2024-02-15T18:36:19.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.995+0000 7fb93e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9400f55c0 0x7fb9401091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:19.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.995+0000 7fb93e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9400f55c0 0x7fb9401091f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57852/0 (socket says 172.21.15.38:57852) 2024-02-15T18:36:19.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.996+0000 7fb93e59c700 1 -- 172.21.15.38:0/2298228047 learned_addr learned my addr 172.21.15.38:0/2298228047 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:36:19.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.996+0000 7fb93dd9b700 1 --2- 172.21.15.38:0/2298228047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9400f6860 0x7fb9401096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:19.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.996+0000 7fb93e59c700 1 -- 172.21.15.38:0/2298228047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9400f6860 msgr2=0x7fb9401096e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:19.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.996+0000 7fb93e59c700 1 --2- 172.21.15.38:0/2298228047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9400f6860 0x7fb9401096e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:19.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.996+0000 7fb93e59c700 1 -- 172.21.15.38:0/2298228047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb930005040 con 0x7fb9400f55c0 2024-02-15T18:36:19.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.996+0000 7fb93e59c700 1 --2- 172.21.15.38:0/2298228047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9400f55c0 0x7fb9401091f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fb930002040 tx=0x7fb93000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:19.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.996+0000 7fb92effd700 1 -- 172.21.15.38:0/2298228047 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb930019070 con 0x7fb9400f55c0 2024-02-15T18:36:19.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.996+0000 7fb92effd700 1 -- 172.21.15.38:0/2298228047 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fb930007e80 con 0x7fb9400f55c0 2024-02-15T18:36:19.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.996+0000 7fb94489e700 1 -- 172.21.15.38:0/2298228047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94010a710 con 0x7fb9400f55c0 2024-02-15T18:36:19.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.996+0000 7fb92effd700 1 -- 172.21.15.38:0/2298228047 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb930014db0 con 0x7fb9400f55c0 2024-02-15T18:36:19.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.996+0000 7fb94489e700 1 -- 172.21.15.38:0/2298228047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94010ac70 con 0x7fb9400f55c0 2024-02-15T18:36:19.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.997+0000 7fb94489e700 1 -- 172.21.15.38:0/2298228047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb940045ad0 con 0x7fb9400f55c0 2024-02-15T18:36:19.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.998+0000 7fb92effd700 1 -- 172.21.15.38:0/2298228047 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 28) v1 ==== 77194+0+0 (secure 0 0 0) 0x7fb930025070 con 0x7fb9400f55c0 2024-02-15T18:36:19.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.998+0000 7fb92effd700 1 --2- 172.21.15.38:0/2298228047 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fb928060f50 0x7fb928063410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:19.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.998+0000 7fb92effd700 1 -- 172.21.15.38:0/2298228047 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fb93007d4b0 con 0x7fb9400f55c0 2024-02-15T18:36:19.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.998+0000 7fb93dd9b700 1 --2- 172.21.15.38:0/2298228047 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fb928060f50 0x7fb928063410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:19.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:19.999+0000 7fb93dd9b700 1 --2- 172.21.15.38:0/2298228047 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fb928060f50 0x7fb928063410 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fb934009fd0 tx=0x7fb93400b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:19.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.002+0000 7fb92effd700 1 -- 172.21.15.38:0/2298228047 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7fb9300512f0 con 0x7fb9400f55c0 2024-02-15T18:36:20.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.265+0000 7fb94489e700 1 -- 172.21.15.38:0/2298228047 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9400faaa0 con 0x7fb928060f50 2024-02-15T18:36:20.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.268+0000 7fb92effd700 1 -- 172.21.15.38:0/2298228047 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fb9400faaa0 con 0x7fb928060f50 2024-02-15T18:36:20.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.271+0000 7fb94489e700 1 -- 172.21.15.38:0/2298228047 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fb928060f50 msgr2=0x7fb928063410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:20.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.271+0000 7fb94489e700 1 --2- 172.21.15.38:0/2298228047 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fb928060f50 0x7fb928063410 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fb934009fd0 tx=0x7fb93400b040).stop 2024-02-15T18:36:20.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.271+0000 7fb94489e700 1 -- 172.21.15.38:0/2298228047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9400f55c0 msgr2=0x7fb9401091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:20.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.271+0000 7fb94489e700 1 --2- 172.21.15.38:0/2298228047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9400f55c0 0x7fb9401091f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fb930002040 tx=0x7fb93000b040).stop 2024-02-15T18:36:20.264 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-02-15T18:36:20.265 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-15T18:36:20.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.273+0000 7fb94489e700 1 -- 172.21.15.38:0/2298228047 shutdown_connections 2024-02-15T18:36:20.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.273+0000 7fb94489e700 1 --2- 172.21.15.38:0/2298228047 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fb928060f50 0x7fb928063410 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:20.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.273+0000 7fb94489e700 1 --2- 172.21.15.38:0/2298228047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9400f55c0 0x7fb9401091f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:20.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.273+0000 7fb94489e700 1 --2- 172.21.15.38:0/2298228047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9400f6860 0x7fb9401096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:20.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.273+0000 7fb94489e700 1 -- 172.21.15.38:0/2298228047 >> 172.21.15.38:0/2298228047 conn(0x7fb9400f0d40 msgr2=0x7fb94005fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:20.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.273+0000 7fb94489e700 1 -- 172.21.15.38:0/2298228047 shutdown_connections 2024-02-15T18:36:20.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.273+0000 7fb94489e700 1 -- 172.21.15.38:0/2298228047 wait complete. 2024-02-15T18:36:20.284 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-02-15T18:36:20.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.420+0000 7fbbb65c5700 1 -- 172.21.15.38:0/2704548597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbb00f6860 msgr2=0x7fbbb00f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:20.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.420+0000 7fbbb65c5700 1 --2- 172.21.15.38:0/2704548597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbb00f6860 0x7fbbb00f6cc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fbba4004660 tx=0x7fbba4009b30).stop 2024-02-15T18:36:20.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.420+0000 7fbbb65c5700 1 -- 172.21.15.38:0/2704548597 shutdown_connections 2024-02-15T18:36:20.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.420+0000 7fbbb65c5700 1 --2- 172.21.15.38:0/2704548597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbb00f6860 0x7fbbb00f6cc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:20.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.420+0000 7fbbb65c5700 1 --2- 172.21.15.38:0/2704548597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbb00f55c0 0x7fbbb00f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:36:20.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.420+0000 7fbbb65c5700 1 -- 172.21.15.38:0/2704548597 >> 172.21.15.38:0/2704548597 conn(0x7fbbb00f0d40 msgr2=0x7fbbb00f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:20.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.420+0000 7fbbb65c5700 1 -- 172.21.15.38:0/2704548597 shutdown_connections 2024-02-15T18:36:20.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.420+0000 7fbbb65c5700 1 -- 172.21.15.38:0/2704548597 wait complete. 2024-02-15T18:36:20.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.421+0000 7fbbb65c5700 1 Processor -- start 2024-02-15T18:36:20.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.421+0000 7fbbb65c5700 1 -- start start 2024-02-15T18:36:20.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.421+0000 7fbbb65c5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbb00f55c0 0x7fbbb0105f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:20.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.421+0000 7fbbb65c5700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbb00f6860 0x7fbbb0106460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:20.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.421+0000 7fbbb65c5700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb0106aa0 con 0x7fbbb00f55c0 2024-02-15T18:36:20.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.421+0000 7fbbb65c5700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb0107350 con 0x7fbbb00f6860 2024-02-15T18:36:20.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.421+0000 7fbbaf7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbb00f6860 0x7fbbb0106460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:20.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.421+0000 7fbbaffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbb00f55c0 0x7fbbb0105f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:20.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.422+0000 7fbbaf7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbb00f6860 0x7fbbb0106460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.38:59148/0 (socket says 172.21.15.38:59148) 2024-02-15T18:36:20.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.422+0000 7fbbaf7fe700 1 -- 172.21.15.38:0/3113498181 learned_addr learned my addr 172.21.15.38:0/3113498181 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:36:20.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.422+0000 7fbbaffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbb00f55c0 0x7fbbb0105f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57870/0 (socket says 172.21.15.38:57870) 2024-02-15T18:36:20.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.422+0000 7fbbaf7fe700 1 -- 172.21.15.38:0/3113498181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbb00f55c0 msgr2=0x7fbbb0105f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:20.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.422+0000 7fbbaf7fe700 1 --2- 172.21.15.38:0/3113498181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbb00f55c0 0x7fbbb0105f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:20.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.422+0000 7fbbaf7fe700 1 -- 172.21.15.38:0/3113498181 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba4005040 con 0x7fbbb00f6860 2024-02-15T18:36:20.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.422+0000 7fbbaffff700 1 --2- 172.21.15.38:0/3113498181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbb00f55c0 0x7fbbb0105f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:36:20.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.422+0000 7fbbaf7fe700 1 --2- 172.21.15.38:0/3113498181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbb00f6860 0x7fbbb0106460 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fbba4013fb0 tx=0x7fbba4009b30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:20.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.422+0000 7fbbacff9700 1 -- 172.21.15.38:0/3113498181 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbba4018070 con 0x7fbbb00f6860 2024-02-15T18:36:20.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.422+0000 7fbbb65c5700 1 -- 172.21.15.38:0/3113498181 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb0107510 con 0x7fbbb00f6860 2024-02-15T18:36:20.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.422+0000 7fbbacff9700 1 -- 172.21.15.38:0/3113498181 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fbba4009780 con 0x7fbbb00f6860 2024-02-15T18:36:20.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.422+0000 7fbbacff9700 1 -- 172.21.15.38:0/3113498181 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbba401c6f0 con 0x7fbbb00f6860 2024-02-15T18:36:20.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.422+0000 7fbbb65c5700 1 -- 172.21.15.38:0/3113498181 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb01079f0 con 0x7fbbb00f6860 2024-02-15T18:36:20.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.424+0000 7fbbb65c5700 1 -- 172.21.15.38:0/3113498181 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb00f49b0 con 0x7fbbb00f6860 2024-02-15T18:36:20.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.424+0000 7fbbacff9700 1 -- 172.21.15.38:0/3113498181 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 28) v1 ==== 77194+0+0 (secure 0 0 0) 0x7fbba4023020 con 0x7fbbb00f6860 2024-02-15T18:36:20.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.425+0000 7fbbacff9700 1 --2- 172.21.15.38:0/3113498181 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fbb98060fb0 0x7fbb98063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:20.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.425+0000 7fbbacff9700 1 -- 172.21.15.38:0/3113498181 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fbba407e0e0 con 0x7fbbb00f6860 2024-02-15T18:36:20.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.425+0000 7fbbaffff700 1 --2- 172.21.15.38:0/3113498181 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fbb98060fb0 0x7fbb98063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:20.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.426+0000 7fbbaffff700 1 --2- 172.21.15.38:0/3113498181 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fbb98060fb0 0x7fbb98063470 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fbba000dfd0 tx=0x7fbba0008040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:20.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.431+0000 7fbbacff9700 1 -- 172.21.15.38:0/3113498181 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7fbba4051e60 con 0x7fbbb00f6860 2024-02-15T18:36:20.619 DEBUG:teuthology.parallel:result is None 2024-02-15T18:36:20.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.634+0000 7fbbb65c5700 1 -- 172.21.15.38:0/3113498181 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbb005f620 con 0x7fbb98060fb0 2024-02-15T18:36:20.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.635+0000 7fbbacff9700 1 -- 172.21.15.38:0/3113498181 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fbbb005f620 con 0x7fbb98060fb0 2024-02-15T18:36:20.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.637+0000 7fbbb65c5700 1 -- 172.21.15.38:0/3113498181 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fbb98060fb0 msgr2=0x7fbb98063470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:20.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.637+0000 7fbbb65c5700 1 --2- 172.21.15.38:0/3113498181 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fbb98060fb0 0x7fbb98063470 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fbba000dfd0 tx=0x7fbba0008040).stop 2024-02-15T18:36:20.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.637+0000 7fbbb65c5700 1 -- 172.21.15.38:0/3113498181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbb00f6860 msgr2=0x7fbbb0106460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:20.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.637+0000 7fbbb65c5700 1 --2- 172.21.15.38:0/3113498181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbb00f6860 0x7fbbb0106460 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fbba4013fb0 tx=0x7fbba4009b30).stop 2024-02-15T18:36:20.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.637+0000 7fbbb65c5700 1 -- 172.21.15.38:0/3113498181 shutdown_connections 2024-02-15T18:36:20.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.637+0000 7fbbb65c5700 1 --2- 172.21.15.38:0/3113498181 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fbb98060fb0 0x7fbb98063470 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:20.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.637+0000 7fbbb65c5700 1 --2- 172.21.15.38:0/3113498181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbbb00f55c0 0x7fbbb0105f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:20.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.637+0000 7fbbb65c5700 1 --2- 172.21.15.38:0/3113498181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbb00f6860 0x7fbbb0106460 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:20.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.637+0000 7fbbb65c5700 1 -- 172.21.15.38:0/3113498181 >> 172.21.15.38:0/3113498181 conn(0x7fbbb00f0d40 msgr2=0x7fbbb00fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:20.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.638+0000 7fbbb65c5700 1 -- 172.21.15.38:0/3113498181 shutdown_connections 2024-02-15T18:36:20.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.638+0000 7fbbb65c5700 1 -- 172.21.15.38:0/3113498181 wait complete. 2024-02-15T18:36:20.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.741+0000 7f4343fff700 1 -- 172.21.15.38:0/4179253992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43440f77d0 msgr2=0x7f43440f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:20.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.741+0000 7f4343fff700 1 --2- 172.21.15.38:0/4179253992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43440f77d0 0x7f43440f7c30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f4338004660 tx=0x7f4338009b30).stop 2024-02-15T18:36:20.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.741+0000 7f4343fff700 1 -- 172.21.15.38:0/4179253992 shutdown_connections 2024-02-15T18:36:20.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.741+0000 7f4343fff700 1 --2- 172.21.15.38:0/4179253992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43440f77d0 0x7f43440f7c30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:20.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.741+0000 7f4343fff700 1 --2- 172.21.15.38:0/4179253992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43440f6530 0x7f43440f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:20.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.741+0000 7f4343fff700 1 -- 172.21.15.38:0/4179253992 >> 172.21.15.38:0/4179253992 conn(0x7f43440f1c90 msgr2=0x7f43440f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:20.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.741+0000 7f4343fff700 1 -- 172.21.15.38:0/4179253992 shutdown_connections 2024-02-15T18:36:20.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.741+0000 7f4343fff700 1 -- 172.21.15.38:0/4179253992 wait complete. 2024-02-15T18:36:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.742+0000 7f4343fff700 1 Processor -- start 2024-02-15T18:36:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.742+0000 7f4343fff700 1 -- start start 2024-02-15T18:36:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.742+0000 7f4343fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43440f6530 0x7f4344106e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.742+0000 7f4343fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43440f77d0 0x7f4344107310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.742+0000 7f4343fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4344107910 con 0x7f43440f6530 2024-02-15T18:36:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.742+0000 7f4343fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4344108120 con 0x7f43440f77d0 2024-02-15T18:36:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.742+0000 7f4342ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43440f6530 0x7f4344106e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:20.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.742+0000 7f4342ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43440f6530 0x7f4344106e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57900/0 (socket says 172.21.15.38:57900) 2024-02-15T18:36:20.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.742+0000 7f4342ffd700 1 -- 172.21.15.38:0/4017515765 learned_addr learned my addr 172.21.15.38:0/4017515765 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:36:20.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.742+0000 7f43427fc700 1 --2- 172.21.15.38:0/4017515765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43440f77d0 0x7f4344107310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:20.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.743+0000 7f4342ffd700 1 -- 172.21.15.38:0/4017515765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43440f77d0 msgr2=0x7f4344107310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:20.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.743+0000 7f4342ffd700 1 --2- 172.21.15.38:0/4017515765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43440f77d0 0x7f4344107310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:20.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.743+0000 7f4342ffd700 1 -- 172.21.15.38:0/4017515765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4338005040 con 0x7f43440f6530 2024-02-15T18:36:20.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.743+0000 7f4342ffd700 1 --2- 172.21.15.38:0/4017515765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43440f6530 0x7f4344106e20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f4334002700 tx=0x7f4334009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:20.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.743+0000 7f432b7fe700 1 -- 172.21.15.38:0/4017515765 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43340079c0 con 0x7f43440f6530 2024-02-15T18:36:20.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.743+0000 7f4343fff700 1 -- 172.21.15.38:0/4017515765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43441083a0 con 0x7f43440f6530 2024-02-15T18:36:20.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.743+0000 7f432b7fe700 1 -- 172.21.15.38:0/4017515765 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f433400f040 con 0x7f43440f6530 2024-02-15T18:36:20.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.743+0000 7f432b7fe700 1 -- 172.21.15.38:0/4017515765 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4334014940 con 0x7f43440f6530 2024-02-15T18:36:20.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.743+0000 7f4343fff700 1 -- 172.21.15.38:0/4017515765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4344108960 con 0x7f43440f6530 2024-02-15T18:36:20.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.744+0000 7f4343fff700 1 -- 172.21.15.38:0/4017515765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4344059100 con 0x7f43440f6530 2024-02-15T18:36:20.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.744+0000 7f432b7fe700 1 -- 172.21.15.38:0/4017515765 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 28) v1 ==== 77194+0+0 (secure 0 0 0) 0x7f433401b060 con 0x7f43440f6530 2024-02-15T18:36:20.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.745+0000 7f432b7fe700 1 --2- 172.21.15.38:0/4017515765 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f432c061000 0x7f432c0634c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:20.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.745+0000 7f432b7fe700 1 -- 172.21.15.38:0/4017515765 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f433407bf60 con 0x7f43440f6530 2024-02-15T18:36:20.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.745+0000 7f43427fc700 1 --2- 172.21.15.38:0/4017515765 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f432c061000 0x7f432c0634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:20.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.745+0000 7f43427fc700 1 --2- 172.21.15.38:0/4017515765 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f432c061000 0x7f432c0634c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f4338004630 tx=0x7f4338009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:20.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:20.749+0000 7f432b7fe700 1 -- 172.21.15.38:0/4017515765 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7f433404c3d0 con 0x7f43440f6530 2024-02-15T18:36:21.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.011+0000 7f4343fff700 1 -- 172.21.15.38:0/4017515765 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f43440feee0 con 0x7f432c061000 2024-02-15T18:36:21.014 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:20.269135+0000 mgr.smithi038.khesbl (mgr.14664) 14 : audit [DBG] from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:36:21.014 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:20.636811+0000 mgr.smithi038.khesbl (mgr.14664) 15 : audit [DBG] from='client.24495 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:36:21.014 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:20.925753+0000 mgr.smithi038.khesbl (mgr.14664) 16 : cluster [DBG] pgmap v13: 65 pgs: 65 active+clean; 308 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 2.0 MiB/s wr, 314 op/s 2024-02-15T18:36:21.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.025+0000 7f432b7fe700 1 -- 172.21.15.38:0/4017515765 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f43440feee0 con 0x7f432c061000 2024-02-15T18:36:21.017 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:36:21.017 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (7m) 8s ago 9m 17.6M - 0.20.0 0881eb8f169f c2034c318e7c 2024-02-15T18:36:21.017 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (9m) 8s ago 9m 6395k - 16.2.4 8d91d370c2b8 0fd3f945d009 2024-02-15T18:36:21.017 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi121 smithi121 running (8m) 12s ago 8m 6405k - 16.2.4 8d91d370c2b8 affc02db354e 2024-02-15T18:36:21.017 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (7m) 8s ago 8m 33.4M - 6.7.4 557c83e11646 2dcccfbfbd2a 2024-02-15T18:36:21.017 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.hxoggl smithi038 running (3m) 8s ago 3m 13.6M - 16.2.4 8d91d370c2b8 be2533754b9c 2024-02-15T18:36:21.017 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.qgkzgn smithi038 running (3m) 8s ago 3m 13.5M - 16.2.4 8d91d370c2b8 dd3ffafba9bd 2024-02-15T18:36:21.017 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.mvumlh smithi121 running (3m) 12s ago 3m 130M - 16.2.4 8d91d370c2b8 88b7d876a39b 2024-02-15T18:36:21.017 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.ovuzki smithi121 running (3m) 12s ago 3m 187M - 16.2.4 8d91d370c2b8 21cacf0012f9 2024-02-15T18:36:21.018 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.khesbl smithi038 *:8443,9283 running (27s) 8s ago 10m 410M - 16.2.14-521-g0e714d9a 1fbc92c859dc e642f3c2cfc9 2024-02-15T18:36:21.018 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi121.ckiryi smithi121 *:8443,9283 running (100s) 12s ago 7m 48.0M - 16.2.14-521-g0e714d9a 1fbc92c859dc b624ee90b80a 2024-02-15T18:36:21.018 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (10m) 8s ago 10m 50.7M 2048M 16.2.4 8d91d370c2b8 01b30eaebcfe 2024-02-15T18:36:21.018 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi121 smithi121 running (7m) 12s ago 7m 40.0M 2048M 16.2.4 8d91d370c2b8 8f588ecacb71 2024-02-15T18:36:21.018 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (8m) 8s ago 8m 17.1M - 0.18.1 e5a616e4b9cf 037f60e38158 2024-02-15T18:36:21.018 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi121 smithi121 *:9100 running (7m) 12s ago 7m 17.2M - 0.18.1 e5a616e4b9cf a867bff85e18 2024-02-15T18:36:21.018 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (7m) 8s ago 7m 372M 4096M 16.2.4 8d91d370c2b8 51512003a2ef 2024-02-15T18:36:21.018 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (6m) 8s ago 6m 373M 4096M 16.2.4 8d91d370c2b8 16dbdf20b170 2024-02-15T18:36:21.018 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (6m) 8s ago 6m 346M 4096M 16.2.4 8d91d370c2b8 6d6ee813e5be 2024-02-15T18:36:21.018 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi121 running (5m) 12s ago 5m 433M 4096M 16.2.4 8d91d370c2b8 cba2d9ed4730 2024-02-15T18:36:21.018 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi121 running (5m) 12s ago 5m 434M 4096M 16.2.4 8d91d370c2b8 ac17ad7dc628 2024-02-15T18:36:21.018 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi121 running (5m) 12s ago 5m 387M 4096M 16.2.4 8d91d370c2b8 cdc8c669c4db 2024-02-15T18:36:21.018 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (33s) 8s ago 8m 50.5M - 2.18.1 de242295e225 67e33f9de4e6 2024-02-15T18:36:21.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.028+0000 7f4343fff700 1 -- 172.21.15.38:0/4017515765 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f432c061000 msgr2=0x7f432c0634c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:21.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.028+0000 7f4343fff700 1 --2- 172.21.15.38:0/4017515765 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f432c061000 0x7f432c0634c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f4338004630 tx=0x7f4338009b30).stop 2024-02-15T18:36:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.028+0000 7f4343fff700 1 -- 172.21.15.38:0/4017515765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43440f6530 msgr2=0x7f4344106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.028+0000 7f4343fff700 1 --2- 172.21.15.38:0/4017515765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43440f6530 0x7f4344106e20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f4334002700 tx=0x7f4334009d80).stop 2024-02-15T18:36:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.028+0000 7f4343fff700 1 -- 172.21.15.38:0/4017515765 shutdown_connections 2024-02-15T18:36:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.028+0000 7f4343fff700 1 --2- 172.21.15.38:0/4017515765 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f432c061000 0x7f432c0634c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.028+0000 7f4343fff700 1 --2- 172.21.15.38:0/4017515765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43440f6530 0x7f4344106e20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.028+0000 7f4343fff700 1 --2- 172.21.15.38:0/4017515765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43440f77d0 0x7f4344107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.028+0000 7f4343fff700 1 -- 172.21.15.38:0/4017515765 >> 172.21.15.38:0/4017515765 conn(0x7f43440f1c90 msgr2=0x7f43440fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.028+0000 7f4343fff700 1 -- 172.21.15.38:0/4017515765 shutdown_connections 2024-02-15T18:36:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.028+0000 7f4343fff700 1 -- 172.21.15.38:0/4017515765 wait complete. 2024-02-15T18:36:21.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.175+0000 7f5f662b7700 1 -- 172.21.15.38:0/2308683292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f6006a3b0 msgr2=0x7f5f6006a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:21.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.175+0000 7f5f662b7700 1 --2- 172.21.15.38:0/2308683292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f6006a3b0 0x7f5f6006a830 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f5f48004660 tx=0x7f5f48009b30).stop 2024-02-15T18:36:21.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.175+0000 7f5f662b7700 1 -- 172.21.15.38:0/2308683292 shutdown_connections 2024-02-15T18:36:21.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.175+0000 7f5f662b7700 1 --2- 172.21.15.38:0/2308683292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f6006a3b0 0x7f5f6006a830 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:21.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.175+0000 7f5f662b7700 1 --2- 172.21.15.38:0/2308683292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f6006b9c0 0x7f5f60069ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:21.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.175+0000 7f5f662b7700 1 -- 172.21.15.38:0/2308683292 >> 172.21.15.38:0/2308683292 conn(0x7f5f600f1e40 msgr2=0x7f5f600f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:21.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.175+0000 7f5f662b7700 1 -- 172.21.15.38:0/2308683292 shutdown_connections 2024-02-15T18:36:21.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.175+0000 7f5f662b7700 1 -- 172.21.15.38:0/2308683292 wait complete. 2024-02-15T18:36:21.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.176+0000 7f5f662b7700 1 Processor -- start 2024-02-15T18:36:21.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.176+0000 7f5f662b7700 1 -- start start 2024-02-15T18:36:21.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.176+0000 7f5f662b7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f6006a3b0 0x7f5f60068ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:21.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.177+0000 7f5f662b7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f6006b9c0 0x7f5f60069190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:21.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.177+0000 7f5f662b7700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f60069790 con 0x7f5f6006b9c0 2024-02-15T18:36:21.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.177+0000 7f5f662b7700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f60117130 con 0x7f5f6006a3b0 2024-02-15T18:36:21.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.177+0000 7f5f5f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f6006b9c0 0x7f5f60069190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:21.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.177+0000 7f5f5f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f6006b9c0 0x7f5f60069190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57924/0 (socket says 172.21.15.38:57924) 2024-02-15T18:36:21.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.177+0000 7f5f5f7fe700 1 -- 172.21.15.38:0/3885016408 learned_addr learned my addr 172.21.15.38:0/3885016408 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:36:21.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.177+0000 7f5f5ffff700 1 --2- 172.21.15.38:0/3885016408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f6006a3b0 0x7f5f60068ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:21.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.177+0000 7f5f5f7fe700 1 -- 172.21.15.38:0/3885016408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f6006a3b0 msgr2=0x7f5f60068ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:21.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.177+0000 7f5f5f7fe700 1 --2- 172.21.15.38:0/3885016408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f6006a3b0 0x7f5f60068ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:21.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.177+0000 7f5f5f7fe700 1 -- 172.21.15.38:0/3885016408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f48005040 con 0x7f5f6006b9c0 2024-02-15T18:36:21.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.177+0000 7f5f5f7fe700 1 --2- 172.21.15.38:0/3885016408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f6006b9c0 0x7f5f60069190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f5f48000c00 tx=0x7f5f4800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:21.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.177+0000 7f5f5cff9700 1 -- 172.21.15.38:0/3885016408 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f48019070 con 0x7f5f6006b9c0 2024-02-15T18:36:21.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.177+0000 7f5f5cff9700 1 -- 172.21.15.38:0/3885016408 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f5f48007e80 con 0x7f5f6006b9c0 2024-02-15T18:36:21.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.177+0000 7f5f662b7700 1 -- 172.21.15.38:0/3885016408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f60117350 con 0x7f5f6006b9c0 2024-02-15T18:36:21.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.177+0000 7f5f5cff9700 1 -- 172.21.15.38:0/3885016408 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f48014db0 con 0x7f5f6006b9c0 2024-02-15T18:36:21.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.177+0000 7f5f662b7700 1 -- 172.21.15.38:0/3885016408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f601178b0 con 0x7f5f6006b9c0 2024-02-15T18:36:21.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.178+0000 7f5f662b7700 1 -- 172.21.15.38:0/3885016408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f600f5ad0 con 0x7f5f6006b9c0 2024-02-15T18:36:21.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.180+0000 7f5f5cff9700 1 -- 172.21.15.38:0/3885016408 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 28) v1 ==== 77194+0+0 (secure 0 0 0) 0x7f5f48025070 con 0x7f5f6006b9c0 2024-02-15T18:36:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.180+0000 7f5f5cff9700 1 --2- 172.21.15.38:0/3885016408 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5f4c060f50 0x7f5f4c063410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.180+0000 7f5f5cff9700 1 -- 172.21.15.38:0/3885016408 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f5f4807d560 con 0x7f5f6006b9c0 2024-02-15T18:36:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.180+0000 7f5f5ffff700 1 --2- 172.21.15.38:0/3885016408 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5f4c060f50 0x7f5f4c063410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:21.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.180+0000 7f5f5ffff700 1 --2- 172.21.15.38:0/3885016408 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5f4c060f50 0x7f5f4c063410 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f5f60068a10 tx=0x7f5f5000b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:21.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.183+0000 7f5f5cff9700 1 -- 172.21.15.38:0/3885016408 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7f5f4804d910 con 0x7f5f6006b9c0 2024-02-15T18:36:21.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:20.269135+0000 mgr.smithi038.khesbl (mgr.14664) 14 : audit [DBG] from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:36:21.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:20.636811+0000 mgr.smithi038.khesbl (mgr.14664) 15 : audit [DBG] from='client.24495 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:36:21.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:20.925753+0000 mgr.smithi038.khesbl (mgr.14664) 16 : cluster [DBG] pgmap v13: 65 pgs: 65 active+clean; 308 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 2.0 MiB/s wr, 314 op/s 2024-02-15T18:36:21.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.438+0000 7f5f662b7700 1 -- 172.21.15.38:0/3885016408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5f60045ad0 con 0x7f5f6006b9c0 2024-02-15T18:36:21.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.439+0000 7f5f5cff9700 1 -- 172.21.15.38:0/3885016408 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+697 (secure 0 0 0) 0x7f5f48023070 con 0x7f5f6006b9c0 2024-02-15T18:36:21.430 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:36:21.430 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-02-15T18:36:21.430 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2024-02-15T18:36:21.430 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:36:21.430 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-02-15T18:36:21.430 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:36:21.431 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:36:21.431 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-02-15T18:36:21.431 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2024-02-15T18:36:21.431 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:36:21.431 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-02-15T18:36:21.431 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:36:21.431 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:36:21.431 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-02-15T18:36:21.431 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2, 2024-02-15T18:36:21.431 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 12 2024-02-15T18:36:21.431 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-02-15T18:36:21.431 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:36:21.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.441+0000 7f5f662b7700 1 -- 172.21.15.38:0/3885016408 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5f4c060f50 msgr2=0x7f5f4c063410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:21.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.441+0000 7f5f662b7700 1 --2- 172.21.15.38:0/3885016408 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5f4c060f50 0x7f5f4c063410 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f5f60068a10 tx=0x7f5f5000b040).stop 2024-02-15T18:36:21.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.441+0000 7f5f662b7700 1 -- 172.21.15.38:0/3885016408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f6006b9c0 msgr2=0x7f5f60069190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:21.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.441+0000 7f5f662b7700 1 --2- 172.21.15.38:0/3885016408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f6006b9c0 0x7f5f60069190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f5f48000c00 tx=0x7f5f4800b040).stop 2024-02-15T18:36:21.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.441+0000 7f5f662b7700 1 -- 172.21.15.38:0/3885016408 shutdown_connections 2024-02-15T18:36:21.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.441+0000 7f5f662b7700 1 --2- 172.21.15.38:0/3885016408 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5f4c060f50 0x7f5f4c063410 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:21.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.441+0000 7f5f662b7700 1 --2- 172.21.15.38:0/3885016408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f6006a3b0 0x7f5f60068ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:21.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.441+0000 7f5f662b7700 1 --2- 172.21.15.38:0/3885016408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5f6006b9c0 0x7f5f60069190 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:21.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.441+0000 7f5f662b7700 1 -- 172.21.15.38:0/3885016408 >> 172.21.15.38:0/3885016408 conn(0x7f5f600f1e40 msgr2=0x7f5f600f88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:21.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.441+0000 7f5f662b7700 1 -- 172.21.15.38:0/3885016408 shutdown_connections 2024-02-15T18:36:21.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.441+0000 7f5f662b7700 1 -- 172.21.15.38:0/3885016408 wait complete. 2024-02-15T18:36:21.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.543+0000 7fa3d7fa6700 1 -- 172.21.15.38:0/3741013791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3d00f7d00 msgr2=0x7fa3d00f8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:21.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.543+0000 7fa3d7fa6700 1 --2- 172.21.15.38:0/3741013791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3d00f7d00 0x7fa3d00f8120 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fa3cc004660 tx=0x7fa3cc009b30).stop 2024-02-15T18:36:21.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.543+0000 7fa3d7fa6700 1 -- 172.21.15.38:0/3741013791 shutdown_connections 2024-02-15T18:36:21.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.543+0000 7fa3d7fa6700 1 --2- 172.21.15.38:0/3741013791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3d00f8610 0x7fa3d00f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:21.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.543+0000 7fa3d7fa6700 1 --2- 172.21.15.38:0/3741013791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3d00f7d00 0x7fa3d00f8120 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:21.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.543+0000 7fa3d7fa6700 1 -- 172.21.15.38:0/3741013791 >> 172.21.15.38:0/3741013791 conn(0x7fa3d00f0540 msgr2=0x7fa3d00f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:21.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.543+0000 7fa3d7fa6700 1 -- 172.21.15.38:0/3741013791 shutdown_connections 2024-02-15T18:36:21.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.543+0000 7fa3d7fa6700 1 -- 172.21.15.38:0/3741013791 wait complete. 2024-02-15T18:36:21.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.544+0000 7fa3d7fa6700 1 Processor -- start 2024-02-15T18:36:21.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.544+0000 7fa3d7fa6700 1 -- start start 2024-02-15T18:36:21.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.544+0000 7fa3d7fa6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3d00f7d00 0x7fa3d0107000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:21.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.544+0000 7fa3d7fa6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3d00f8610 0x7fa3d01074f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:21.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.544+0000 7fa3d7fa6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d0107aa0 con 0x7fa3d00f7d00 2024-02-15T18:36:21.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.544+0000 7fa3d7fa6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d0107be0 con 0x7fa3d00f8610 2024-02-15T18:36:21.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.544+0000 7fa3d5d42700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3d00f7d00 0x7fa3d0107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:21.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.544+0000 7fa3d5d42700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3d00f7d00 0x7fa3d0107000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57950/0 (socket says 172.21.15.38:57950) 2024-02-15T18:36:21.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.544+0000 7fa3d5d42700 1 -- 172.21.15.38:0/1707104634 learned_addr learned my addr 172.21.15.38:0/1707104634 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:36:21.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.544+0000 7fa3d5541700 1 --2- 172.21.15.38:0/1707104634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3d00f8610 0x7fa3d01074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:21.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.544+0000 7fa3d5d42700 1 -- 172.21.15.38:0/1707104634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3d00f8610 msgr2=0x7fa3d01074f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:21.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.544+0000 7fa3d5d42700 1 --2- 172.21.15.38:0/1707104634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3d00f8610 0x7fa3d01074f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:21.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.544+0000 7fa3d5d42700 1 -- 172.21.15.38:0/1707104634 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc005040 con 0x7fa3d00f7d00 2024-02-15T18:36:21.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.545+0000 7fa3d5d42700 1 --2- 172.21.15.38:0/1707104634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3d00f7d00 0x7fa3d0107000 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fa3cc000c00 tx=0x7fa3cc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:21.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.545+0000 7fa3c67fc700 1 -- 172.21.15.38:0/1707104634 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3cc019070 con 0x7fa3d00f7d00 2024-02-15T18:36:21.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.545+0000 7fa3d7fa6700 1 -- 172.21.15.38:0/1707104634 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d0108520 con 0x7fa3d00f7d00 2024-02-15T18:36:21.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.545+0000 7fa3c67fc700 1 -- 172.21.15.38:0/1707104634 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fa3cc007e80 con 0x7fa3d00f7d00 2024-02-15T18:36:21.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.545+0000 7fa3c67fc700 1 -- 172.21.15.38:0/1707104634 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3cc014db0 con 0x7fa3d00f7d00 2024-02-15T18:36:21.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.545+0000 7fa3d7fa6700 1 -- 172.21.15.38:0/1707104634 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d0108a80 con 0x7fa3d00f7d00 2024-02-15T18:36:21.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.546+0000 7fa3d7fa6700 1 -- 172.21.15.38:0/1707104634 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d0045ad0 con 0x7fa3d00f7d00 2024-02-15T18:36:21.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.546+0000 7fa3c67fc700 1 -- 172.21.15.38:0/1707104634 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 28) v1 ==== 77194+0+0 (secure 0 0 0) 0x7fa3cc025070 con 0x7fa3d00f7d00 2024-02-15T18:36:21.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.547+0000 7fa3c67fc700 1 --2- 172.21.15.38:0/1707104634 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fa3bc060f50 0x7fa3bc063410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:21.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.547+0000 7fa3c67fc700 1 -- 172.21.15.38:0/1707104634 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fa3cc07d100 con 0x7fa3d00f7d00 2024-02-15T18:36:21.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.547+0000 7fa3d5541700 1 --2- 172.21.15.38:0/1707104634 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fa3bc060f50 0x7fa3bc063410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:21.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.547+0000 7fa3d5541700 1 --2- 172.21.15.38:0/1707104634 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fa3bc060f50 0x7fa3bc063410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fa3c0009fd0 tx=0x7fa3c000b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:21.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.551+0000 7fa3c67fc700 1 -- 172.21.15.38:0/1707104634 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7fa3cc050f30 con 0x7fa3d00f7d00 2024-02-15T18:36:21.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.767+0000 7fa3d7fa6700 1 -- 172.21.15.38:0/1707104634 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa3d00ff090 con 0x7fa3d00f7d00 2024-02-15T18:36:21.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.768+0000 7fa3c67fc700 1 -- 172.21.15.38:0/1707104634 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1660 (secure 0 0 0) 0x7fa3cc023020 con 0x7fa3d00f7d00 2024-02-15T18:36:21.760 INFO:teuthology.orchestra.run.smithi038.stdout:e13 2024-02-15T18:36:21.760 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:36:21.760 INFO:teuthology.orchestra.run.smithi038.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:36:21.760 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-02-15T18:36:21.760 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:36:21.760 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:36:21.760 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-02-15T18:36:21.760 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 13 2024-02-15T18:36:21.760 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 2024-02-15T18:36:21.760 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:36:21.760 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-02-15T18:35:07.485909+0000 2024-02-15T18:36:21.761 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-02-15T18:36:21.761 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-02-15T18:36:21.761 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-02-15T18:36:21.761 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-02-15T18:36:21.761 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-02-15T18:36:21.761 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-02-15T18:36:21.761 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-02-15T18:36:21.761 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-02-15T18:36:21.761 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:36:21.761 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-02-15T18:36:21.761 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-02-15T18:36:21.761 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=24261,1=24275} 2024-02-15T18:36:21.762 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-02-15T18:36:21.762 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-02-15T18:36:21.762 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-02-15T18:36:21.762 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-02-15T18:36:21.762 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-02-15T18:36:21.762 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-02-15T18:36:21.762 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-02-15T18:36:21.762 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-02-15T18:36:21.762 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.mvumlh{0:24261} state up:active seq 3 export targets 1 join_fscid=1 addr [v2:172.21.15.121:6824/1422707571,v1:172.21.15.121:6825/1422707571]] 2024-02-15T18:36:21.762 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.ovuzki{1:24275} state up:active seq 2 export targets 0 join_fscid=1 addr [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845]] 2024-02-15T18:36:21.762 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:36:21.762 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:36:21.762 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-02-15T18:36:21.762 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:36:21.763 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.hxoggl{-1:14446} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.38:6826/3822880235,v1:172.21.15.38:6827/3822880235]] 2024-02-15T18:36:21.763 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.qgkzgn{-1:14456} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.38:6828/703030231,v1:172.21.15.38:6829/703030231]] 2024-02-15T18:36:21.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.770+0000 7fa3d7fa6700 1 -- 172.21.15.38:0/1707104634 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fa3bc060f50 msgr2=0x7fa3bc063410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.770+0000 7fa3d7fa6700 1 --2- 172.21.15.38:0/1707104634 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fa3bc060f50 0x7fa3bc063410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fa3c0009fd0 tx=0x7fa3c000b040).stop 2024-02-15T18:36:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.770+0000 7fa3d7fa6700 1 -- 172.21.15.38:0/1707104634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3d00f7d00 msgr2=0x7fa3d0107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.770+0000 7fa3d7fa6700 1 --2- 172.21.15.38:0/1707104634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3d00f7d00 0x7fa3d0107000 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fa3cc000c00 tx=0x7fa3cc00b040).stop 2024-02-15T18:36:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.770+0000 7fa3d7fa6700 1 -- 172.21.15.38:0/1707104634 shutdown_connections 2024-02-15T18:36:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.770+0000 7fa3d7fa6700 1 --2- 172.21.15.38:0/1707104634 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fa3bc060f50 0x7fa3bc063410 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.770+0000 7fa3d7fa6700 1 --2- 172.21.15.38:0/1707104634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3d00f7d00 0x7fa3d0107000 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.770+0000 7fa3d7fa6700 1 --2- 172.21.15.38:0/1707104634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3d00f8610 0x7fa3d01074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.770+0000 7fa3d7fa6700 1 -- 172.21.15.38:0/1707104634 >> 172.21.15.38:0/1707104634 conn(0x7fa3d00f0540 msgr2=0x7fa3d00fb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.770+0000 7fa3d7fa6700 1 -- 172.21.15.38:0/1707104634 shutdown_connections 2024-02-15T18:36:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.770+0000 7fa3d7fa6700 1 -- 172.21.15.38:0/1707104634 wait complete. 2024-02-15T18:36:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 13 2024-02-15T18:36:21.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.902+0000 7f15e5460700 1 -- 172.21.15.38:0/857000597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15e00f68e0 msgr2=0x7f15e00f6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:21.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.902+0000 7f15e5460700 1 --2- 172.21.15.38:0/857000597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15e00f68e0 0x7f15e00f6d00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f15d0004660 tx=0x7f15d0009b30).stop 2024-02-15T18:36:21.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.903+0000 7f15e5460700 1 -- 172.21.15.38:0/857000597 shutdown_connections 2024-02-15T18:36:21.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.903+0000 7f15e5460700 1 --2- 172.21.15.38:0/857000597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15e00f7b80 0x7f15e00f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:36:21.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.903+0000 7f15e5460700 1 --2- 172.21.15.38:0/857000597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15e00f68e0 0x7f15e00f6d00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:21.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.903+0000 7f15e5460700 1 -- 172.21.15.38:0/857000597 >> 172.21.15.38:0/857000597 conn(0x7f15e00f2020 msgr2=0x7f15e00f4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:21.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.903+0000 7f15e5460700 1 -- 172.21.15.38:0/857000597 shutdown_connections 2024-02-15T18:36:21.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.903+0000 7f15e5460700 1 -- 172.21.15.38:0/857000597 wait complete. 2024-02-15T18:36:21.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.903+0000 7f15e5460700 1 Processor -- start 2024-02-15T18:36:21.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.904+0000 7f15e5460700 1 -- start start 2024-02-15T18:36:21.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.904+0000 7f15e5460700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15e00f68e0 0x7f15e0068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:21.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.904+0000 7f15e5460700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15e00f7b80 0x7f15e0069160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:21.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.904+0000 7f15e5460700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e00697a0 con 0x7f15e00f7b80 2024-02-15T18:36:21.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.904+0000 7f15e5460700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e0110e40 con 0x7f15e00f68e0 2024-02-15T18:36:21.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.904+0000 7f15de7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15e00f7b80 0x7f15e0069160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:21.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.904+0000 7f15de7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15e00f7b80 0x7f15e0069160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57970/0 (socket says 172.21.15.38:57970) 2024-02-15T18:36:21.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.904+0000 7f15de7fc700 1 -- 172.21.15.38:0/3616123864 learned_addr learned my addr 172.21.15.38:0/3616123864 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:36:21.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.904+0000 7f15deffd700 1 --2- 172.21.15.38:0/3616123864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15e00f68e0 0x7f15e0068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:21.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.904+0000 7f15de7fc700 1 -- 172.21.15.38:0/3616123864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15e00f68e0 msgr2=0x7f15e0068c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:21.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.904+0000 7f15de7fc700 1 --2- 172.21.15.38:0/3616123864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15e00f68e0 0x7f15e0068c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:21.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.904+0000 7f15de7fc700 1 -- 172.21.15.38:0/3616123864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d0005040 con 0x7f15e00f7b80 2024-02-15T18:36:21.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.904+0000 7f15de7fc700 1 --2- 172.21.15.38:0/3616123864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15e00f7b80 0x7f15e0069160 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f15c800b410 tx=0x7f15c8006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:21.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.905+0000 7f15d77fe700 1 -- 172.21.15.38:0/3616123864 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15c80125f0 con 0x7f15e00f7b80 2024-02-15T18:36:21.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.905+0000 7f15d77fe700 1 -- 172.21.15.38:0/3616123864 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f15c8012c40 con 0x7f15e00f7b80 2024-02-15T18:36:21.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.905+0000 7f15e5460700 1 -- 172.21.15.38:0/3616123864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e0111120 con 0x7f15e00f7b80 2024-02-15T18:36:21.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.905+0000 7f15e5460700 1 -- 172.21.15.38:0/3616123864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e0111660 con 0x7f15e00f7b80 2024-02-15T18:36:21.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.905+0000 7f15d77fe700 1 -- 172.21.15.38:0/3616123864 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15c8002b80 con 0x7f15e00f7b80 2024-02-15T18:36:21.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.906+0000 7f15e5460700 1 -- 172.21.15.38:0/3616123864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c4004fa0 con 0x7f15e00f7b80 2024-02-15T18:36:21.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.906+0000 7f15d77fe700 1 -- 172.21.15.38:0/3616123864 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 28) v1 ==== 77194+0+0 (secure 0 0 0) 0x7f15c800f030 con 0x7f15e00f7b80 2024-02-15T18:36:21.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.907+0000 7f15d77fe700 1 --2- 172.21.15.38:0/3616123864 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f15cc060fb0 0x7f15cc063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:21.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.907+0000 7f15d77fe700 1 -- 172.21.15.38:0/3616123864 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f15c807c620 con 0x7f15e00f7b80 2024-02-15T18:36:21.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.907+0000 7f15deffd700 1 --2- 172.21.15.38:0/3616123864 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f15cc060fb0 0x7f15cc063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:21.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.908+0000 7f15deffd700 1 --2- 172.21.15.38:0/3616123864 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f15cc060fb0 0x7f15cc063470 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f15d0004e30 tx=0x7f15d0009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:21.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:21.913+0000 7f15d77fe700 1 -- 172.21.15.38:0/3616123864 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7f15c8050440 con 0x7f15e00f7b80 2024-02-15T18:36:22.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:22.131+0000 7f15e5460700 1 -- 172.21.15.38:0/3616123864 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f15c4000bc0 con 0x7f15cc060fb0 2024-02-15T18:36:22.124 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:36:22.125 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:36:22.125 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-02-15T18:36:22.125 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:36:22.125 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [], 2024-02-15T18:36:22.125 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "", 2024-02-15T18:36:22.125 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading mgr daemons", 2024-02-15T18:36:22.125 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-02-15T18:36:22.125 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:36:22.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:22.132+0000 7f15d77fe700 1 -- 172.21.15.38:0/3616123864 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f15c4000bc0 con 0x7f15cc060fb0 2024-02-15T18:36:22.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:22.134+0000 7f15e5460700 1 -- 172.21.15.38:0/3616123864 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f15cc060fb0 msgr2=0x7f15cc063470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:22.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:22.134+0000 7f15e5460700 1 --2- 172.21.15.38:0/3616123864 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f15cc060fb0 0x7f15cc063470 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f15d0004e30 tx=0x7f15d0009b30).stop 2024-02-15T18:36:22.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:22.134+0000 7f15e5460700 1 -- 172.21.15.38:0/3616123864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15e00f7b80 msgr2=0x7f15e0069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:22.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:22.134+0000 7f15e5460700 1 --2- 172.21.15.38:0/3616123864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15e00f7b80 0x7f15e0069160 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f15c800b410 tx=0x7f15c8006ef0).stop 2024-02-15T18:36:22.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:22.134+0000 7f15e5460700 1 -- 172.21.15.38:0/3616123864 shutdown_connections 2024-02-15T18:36:22.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:22.134+0000 7f15e5460700 1 --2- 172.21.15.38:0/3616123864 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f15cc060fb0 0x7f15cc063470 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:22.134+0000 7f15e5460700 1 --2- 172.21.15.38:0/3616123864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15e00f68e0 0x7f15e0068c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:22.134+0000 7f15e5460700 1 --2- 172.21.15.38:0/3616123864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15e00f7b80 0x7f15e0069160 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:22.134+0000 7f15e5460700 1 -- 172.21.15.38:0/3616123864 >> 172.21.15.38:0/3616123864 conn(0x7f15e00f2020 msgr2=0x7f15e00fb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:22.134+0000 7f15e5460700 1 -- 172.21.15.38:0/3616123864 shutdown_connections 2024-02-15T18:36:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:22.134+0000 7f15e5460700 1 -- 172.21.15.38:0/3616123864 wait complete. 2024-02-15T18:36:22.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:21.013416+0000 mgr.smithi038.khesbl (mgr.14664) 17 : audit [DBG] from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:36:22.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:21.440380+0000 mon.smithi038 (mon.0) 664 : audit [DBG] from='client.? 172.21.15.38:0/3885016408' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:36:22.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:21.769643+0000 mon.smithi038 (mon.0) 665 : audit [DBG] from='client.? 172.21.15.38:0/1707104634' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:36:22.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:21.777855+0000 mon.smithi038 (mon.0) 666 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:22.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:21.778701+0000 mon.smithi038 (mon.0) 667 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:22.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:21.013416+0000 mgr.smithi038.khesbl (mgr.14664) 17 : audit [DBG] from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:36:22.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:21.440380+0000 mon.smithi038 (mon.0) 664 : audit [DBG] from='client.? 172.21.15.38:0/3885016408' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:36:22.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:21.769643+0000 mon.smithi038 (mon.0) 665 : audit [DBG] from='client.? 172.21.15.38:0/1707104634' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:36:22.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:21.777855+0000 mon.smithi038 (mon.0) 666 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:22.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:21.778701+0000 mon.smithi038 (mon.0) 667 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:23.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:22.133229+0000 mgr.smithi038.khesbl (mgr.14664) 18 : audit [DBG] from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:36:23.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:22.926128+0000 mgr.smithi038.khesbl (mgr.14664) 19 : cluster [DBG] pgmap v14: 65 pgs: 65 active+clean; 308 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 1.5 MiB/s wr, 229 op/s 2024-02-15T18:36:23.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:22.133229+0000 mgr.smithi038.khesbl (mgr.14664) 18 : audit [DBG] from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:36:23.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:22.926128+0000 mgr.smithi038.khesbl (mgr.14664) 19 : cluster [DBG] pgmap v14: 65 pgs: 65 active+clean; 308 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 1.5 MiB/s wr, 229 op/s 2024-02-15T18:36:25.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:24.926564+0000 mgr.smithi038.khesbl (mgr.14664) 20 : cluster [DBG] pgmap v15: 65 pgs: 65 active+clean; 308 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 2.2 MiB/s wr, 369 op/s 2024-02-15T18:36:25.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:24.926564+0000 mgr.smithi038.khesbl (mgr.14664) 20 : cluster [DBG] pgmap v15: 65 pgs: 65 active+clean; 308 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 2.2 MiB/s wr, 369 op/s 2024-02-15T18:36:27.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:26.233585+0000 mon.smithi038 (mon.0) 668 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:27.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:26.234327+0000 mon.smithi038 (mon.0) 669 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:36:27.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:26.234935+0000 mon.smithi038 (mon.0) 670 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:36:27.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:36:26.235436+0000 mgr.smithi038.khesbl (mgr.14664) 21 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:36:27.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:26.242594+0000 mon.smithi038 (mon.0) 671 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:27.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:26.247275+0000 mon.smithi038 (mon.0) 672 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:27.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:26.249251+0000 mon.smithi038 (mon.0) 673 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:36:27.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:36:26.249762+0000 mgr.smithi038.khesbl (mgr.14664) 22 : cephadm [INF] Upgrade: Setting container_image for all mgr 2024-02-15T18:36:27.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:26.253162+0000 mon.smithi038 (mon.0) 674 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:27.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:26.253840+0000 mon.smithi038 (mon.0) 675 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi038.khesbl"}]: dispatch 2024-02-15T18:36:27.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:26.262583+0000 mon.smithi038 (mon.0) 676 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi038.khesbl"}]': finished 2024-02-15T18:36:27.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:26.263149+0000 mon.smithi038 (mon.0) 677 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi121.ckiryi"}]: dispatch 2024-02-15T18:36:27.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:26.266135+0000 mon.smithi038 (mon.0) 678 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi121.ckiryi"}]': finished 2024-02-15T18:36:27.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:26.267052+0000 mon.smithi038 (mon.0) 679 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-15T18:36:27.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:26.926884+0000 mgr.smithi038.khesbl (mgr.14664) 23 : cluster [DBG] pgmap v16: 65 pgs: 65 active+clean; 308 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.6 MiB/s wr, 234 op/s 2024-02-15T18:36:27.530 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:26.233585+0000 mon.smithi038 (mon.0) 668 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:27.530 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:26.234327+0000 mon.smithi038 (mon.0) 669 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:36:27.530 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:26.234935+0000 mon.smithi038 (mon.0) 670 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:36:27.531 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:36:26.235436+0000 mgr.smithi038.khesbl (mgr.14664) 21 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:36:27.531 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:26.242594+0000 mon.smithi038 (mon.0) 671 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:27.531 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:26.247275+0000 mon.smithi038 (mon.0) 672 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:27.531 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:26.249251+0000 mon.smithi038 (mon.0) 673 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:36:27.531 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:36:26.249762+0000 mgr.smithi038.khesbl (mgr.14664) 22 : cephadm [INF] Upgrade: Setting container_image for all mgr 2024-02-15T18:36:27.531 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:26.253162+0000 mon.smithi038 (mon.0) 674 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:27.531 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:26.253840+0000 mon.smithi038 (mon.0) 675 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi038.khesbl"}]: dispatch 2024-02-15T18:36:27.531 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:26.262583+0000 mon.smithi038 (mon.0) 676 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi038.khesbl"}]': finished 2024-02-15T18:36:27.531 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:26.263149+0000 mon.smithi038 (mon.0) 677 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi121.ckiryi"}]: dispatch 2024-02-15T18:36:27.531 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:26.266135+0000 mon.smithi038 (mon.0) 678 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi121.ckiryi"}]': finished 2024-02-15T18:36:27.531 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:26.267052+0000 mon.smithi038 (mon.0) 679 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-15T18:36:27.531 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:26.926884+0000 mgr.smithi038.khesbl (mgr.14664) 23 : cluster [DBG] pgmap v16: 65 pgs: 65 active+clean; 308 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.6 MiB/s wr, 234 op/s 2024-02-15T18:36:29.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:36:27.984983+0000 mgr.smithi038.khesbl (mgr.14664) 24 : cephadm [INF] Upgrade: Updating mon.smithi038 2024-02-15T18:36:29.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:27.990164+0000 mon.smithi038 (mon.0) 680 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:29.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:27.991094+0000 mon.smithi038 (mon.0) 681 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-15T18:36:29.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:27.991922+0000 mon.smithi038 (mon.0) 682 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-15T18:36:29.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:27.992630+0000 mon.smithi038 (mon.0) 683 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:36:29.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cephadm 2024-02-15T18:36:27.993344+0000 mgr.smithi038.khesbl (mgr.14664) 25 : cephadm [INF] Deploying daemon mon.smithi038 on smithi038 2024-02-15T18:36:29.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:28.552928+0000 mon.smithi038 (mon.0) 684 : cluster [DBG] Standby manager daemon smithi121.ckiryi restarted 2024-02-15T18:36:29.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:28.553030+0000 mon.smithi038 (mon.0) 685 : cluster [DBG] Standby manager daemon smithi121.ckiryi started 2024-02-15T18:36:29.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:28.554760+0000 mon.smithi121 (mon.1) 62 : audit [DBG] from='mgr.? 172.21.15.121:0/1393558274' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi121.ckiryi/crt"}]: dispatch 2024-02-15T18:36:29.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:28.555310+0000 mon.smithi121 (mon.1) 63 : audit [DBG] from='mgr.? 172.21.15.121:0/1393558274' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:36:29.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:28.557678+0000 mon.smithi121 (mon.1) 64 : audit [DBG] from='mgr.? 172.21.15.121:0/1393558274' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi121.ckiryi/key"}]: dispatch 2024-02-15T18:36:29.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: audit 2024-02-15T18:36:28.559108+0000 mon.smithi121 (mon.1) 65 : audit [DBG] from='mgr.? 172.21.15.121:0/1393558274' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:36:29.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:28.927277+0000 mgr.smithi038.khesbl (mgr.14664) 26 : cluster [DBG] pgmap v17: 65 pgs: 65 active+clean; 308 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.6 MiB/s wr, 234 op/s 2024-02-15T18:36:29.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:28 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:36:27.984983+0000 mgr.smithi038.khesbl (mgr.14664) 24 : cephadm [INF] Upgrade: Updating mon.smithi038 2024-02-15T18:36:29.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:28 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:27.990164+0000 mon.smithi038 (mon.0) 680 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:29.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:28 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:27.991094+0000 mon.smithi038 (mon.0) 681 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-15T18:36:29.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:27.991922+0000 mon.smithi038 (mon.0) 682 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-15T18:36:29.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:27.992630+0000 mon.smithi038 (mon.0) 683 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:36:29.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:36:27.993344+0000 mgr.smithi038.khesbl (mgr.14664) 25 : cephadm [INF] Deploying daemon mon.smithi038 on smithi038 2024-02-15T18:36:29.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:28.552928+0000 mon.smithi038 (mon.0) 684 : cluster [DBG] Standby manager daemon smithi121.ckiryi restarted 2024-02-15T18:36:29.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:28.553030+0000 mon.smithi038 (mon.0) 685 : cluster [DBG] Standby manager daemon smithi121.ckiryi started 2024-02-15T18:36:29.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:28.554760+0000 mon.smithi121 (mon.1) 62 : audit [DBG] from='mgr.? 172.21.15.121:0/1393558274' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi121.ckiryi/crt"}]: dispatch 2024-02-15T18:36:29.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:28.555310+0000 mon.smithi121 (mon.1) 63 : audit [DBG] from='mgr.? 172.21.15.121:0/1393558274' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:36:29.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:28.557678+0000 mon.smithi121 (mon.1) 64 : audit [DBG] from='mgr.? 172.21.15.121:0/1393558274' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi121.ckiryi/key"}]: dispatch 2024-02-15T18:36:29.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:28.559108+0000 mon.smithi121 (mon.1) 65 : audit [DBG] from='mgr.? 172.21.15.121:0/1393558274' entity='mgr.smithi121.ckiryi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:36:29.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:28.927277+0000 mgr.smithi038.khesbl (mgr.14664) 26 : cluster [DBG] pgmap v17: 65 pgs: 65 active+clean; 308 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.6 MiB/s wr, 234 op/s 2024-02-15T18:36:30.127 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:29 smithi038 systemd[1]: Stopping Ceph mon.smithi038 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:36:30.127 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: cluster 2024-02-15T18:36:29.002301+0000 mon.smithi038 (mon.0) 686 : cluster [DBG] mgrmap e29: smithi038.khesbl(active, since 27s), standbys: smithi121.ckiryi 2024-02-15T18:36:30.127 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:36:30.134+0000 7f5ad7bbe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi038 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-15T18:36:30.127 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038[38138]: debug 2024-02-15T18:36:30.134+0000 7f5ad7bbe700 -1 mon.smithi038@0(leader) e2 *** Got Signal Terminated *** 2024-02-15T18:36:30.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:29.002301+0000 mon.smithi038 (mon.0) 686 : cluster [DBG] mgrmap e29: smithi038.khesbl(active, since 27s), standbys: smithi121.ckiryi 2024-02-15T18:36:30.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:30 smithi038 bash[77712]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038 2024-02-15T18:36:31.263 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:30 smithi038 bash[77772]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038 2024-02-15T18:36:31.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:31 smithi038 systemd[1]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mon.smithi038.service: Succeeded. 2024-02-15T18:36:31.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:31 smithi038 systemd[1]: Stopped Ceph mon.smithi038 for 7e05753e-cc2f-11ee-95ba-87774f69a715. 2024-02-15T18:36:32.020 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:31 smithi038 systemd[1]: Starting Ceph mon.smithi038 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:36:32.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.185+0000 7f3d865cb880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-15T18:36:32.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.185+0000 7f3d865cb880 0 ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable), process ceph-mon, pid 7 2024-02-15T18:36:32.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.185+0000 7f3d865cb880 0 pidfile_write: ignore empty --pid-file 2024-02-15T18:36:32.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.189+0000 7f3d865cb880 0 load: jerasure load: lrc load: isa 2024-02-15T18:36:32.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: RocksDB version: 6.8.1 2024-02-15T18:36:32.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-15T18:36:32.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Compile date Feb 8 2024 2024-02-15T18:36:32.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: DB SUMMARY 2024-02-15T18:36:32.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: CURRENT file: CURRENT 2024-02-15T18:36:32.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: IDENTITY file: IDENTITY 2024-02-15T18:36:32.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: MANIFEST file: MANIFEST-000008 size: 771 Bytes 2024-02-15T18:36:32.525 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.525 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi038/store.db dir, Total Num: 1, files: 000020.sst 2024-02-15T18:36:32.525 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.525 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi038/store.db: 000018.log size: 4049709 ; 2024-02-15T18:36:32.525 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.525 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.error_if_exists: 0 2024-02-15T18:36:32.525 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.create_if_missing: 0 2024-02-15T18:36:32.525 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.paranoid_checks: 1 2024-02-15T18:36:32.525 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.env: 0x55da145ac080 2024-02-15T18:36:32.525 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.fs: Posix File System 2024-02-15T18:36:32.525 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.info_log: 0x55da166b8560 2024-02-15T18:36:32.525 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-15T18:36:32.525 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.statistics: (nil) 2024-02-15T18:36:32.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.use_fsync: 0 2024-02-15T18:36:32.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.max_log_file_size: 0 2024-02-15T18:36:32.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-15T18:36:32.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-15T18:36:32.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-15T18:36:32.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-15T18:36:32.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.allow_fallocate: 1 2024-02-15T18:36:32.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-15T18:36:32.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-15T18:36:32.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.use_direct_reads: 0 2024-02-15T18:36:32.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-15T18:36:32.526 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-15T18:36:32.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.db_log_dir: 2024-02-15T18:36:32.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi038/store.db 2024-02-15T18:36:32.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-15T18:36:32.527 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.max_subcompactions: 1 2024-02-15T18:36:32.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.max_background_flushes: -1 2024-02-15T18:36:32.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-15T18:36:32.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-15T18:36:32.529 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-15T18:36:32.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-15T18:36:32.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-15T18:36:32.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-15T18:36:32.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-15T18:36:32.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.write_buffer_manager: 0x55da173aa420 2024-02-15T18:36:32.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-15T18:36:32.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-15T18:36:32.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-15T18:36:32.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-15T18:36:32.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-15T18:36:32.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-15T18:36:32.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-15T18:36:32.530 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-15T18:36:32.531 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-15T18:36:32.531 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.unordered_write: 0 2024-02-15T18:36:32.531 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-15T18:36:32.531 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-15T18:36:32.531 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-15T18:36:32.531 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-15T18:36:32.531 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.row_cache: None 2024-02-15T18:36:32.531 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.wal_filter: None 2024-02-15T18:36:32.531 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-15T18:36:32.531 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-15T18:36:32.531 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.preserve_deletes: 0 2024-02-15T18:36:32.531 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.two_write_queues: 0 2024-02-15T18:36:32.531 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-15T18:36:32.532 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.atomic_flush: 0 2024-02-15T18:36:32.532 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-15T18:36:32.532 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-15T18:36:32.532 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-15T18:36:32.532 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.log_readahead_size: 0 2024-02-15T18:36:32.532 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-15T18:36:32.532 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.max_background_jobs: 2 2024-02-15T18:36:32.532 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.max_background_compactions: -1 2024-02-15T18:36:32.532 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-15T18:36:32.532 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-15T18:36:32.532 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-15T18:36:32.532 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-15T18:36:32.532 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-15T18:36:32.533 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-15T18:36:32.533 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-15T18:36:32.533 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-15T18:36:32.533 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.max_open_files: -1 2024-02-15T18:36:32.533 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-15T18:36:32.533 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-15T18:36:32.533 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-15T18:36:32.533 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-15T18:36:32.533 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Compression algorithms supported: 2024-02-15T18:36:32.533 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-15T18:36:32.533 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: kZSTD supported: 0 2024-02-15T18:36:32.533 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: kXpressCompression supported: 0 2024-02-15T18:36:32.533 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-15T18:36:32.533 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: kLZ4Compression supported: 1 2024-02-15T18:36:32.534 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: kBZip2Compression supported: 0 2024-02-15T18:36:32.534 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: kZlibCompression supported: 1 2024-02-15T18:36:32.534 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: kSnappyCompression supported: 1 2024-02-15T18:36:32.534 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-15T18:36:32.534 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi038/store.db/MANIFEST-000008 2024-02-15T18:36:32.534 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.534 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-15T18:36:32.534 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.534 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-15T18:36:32.534 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.merge_operator: 2024-02-15T18:36:32.534 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.compaction_filter: None 2024-02-15T18:36:32.534 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-15T18:36:32.534 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-15T18:36:32.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-15T18:36:32.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55da1665dd28) 2024-02-15T18:36:32.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cache_index_and_filter_blocks: 1 2024-02-15T18:36:32.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-15T18:36:32.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-15T18:36:32.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: pin_top_level_index_and_filter: 1 2024-02-15T18:36:32.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: index_type: 0 2024-02-15T18:36:32.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: data_block_index_type: 0 2024-02-15T18:36:32.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: index_shortening: 1 2024-02-15T18:36:32.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: data_block_hash_table_util_ratio: 0.750000 2024-02-15T18:36:32.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: hash_index_allow_collision: 1 2024-02-15T18:36:32.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: checksum: 1 2024-02-15T18:36:32.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: no_block_cache: 0 2024-02-15T18:36:32.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: block_cache: 0x55da16692f10 2024-02-15T18:36:32.535 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: block_cache_name: BinnedLRUCache 2024-02-15T18:36:32.536 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: block_cache_options: 2024-02-15T18:36:32.536 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: capacity : 536870912 2024-02-15T18:36:32.536 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: num_shard_bits : 4 2024-02-15T18:36:32.536 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: strict_capacity_limit : 0 2024-02-15T18:36:32.536 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: high_pri_pool_ratio: 0.000 2024-02-15T18:36:32.536 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: block_cache_compressed: (nil) 2024-02-15T18:36:32.536 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: persistent_cache: (nil) 2024-02-15T18:36:32.536 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: block_size: 4096 2024-02-15T18:36:32.536 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: block_size_deviation: 10 2024-02-15T18:36:32.536 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: block_restart_interval: 16 2024-02-15T18:36:32.537 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: index_block_restart_interval: 1 2024-02-15T18:36:32.537 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: metadata_block_size: 4096 2024-02-15T18:36:32.537 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: partition_filters: 0 2024-02-15T18:36:32.537 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: use_delta_encoding: 1 2024-02-15T18:36:32.537 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-15T18:36:32.537 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: whole_key_filtering: 1 2024-02-15T18:36:32.537 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: verify_compression: 0 2024-02-15T18:36:32.537 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: read_amp_bytes_per_bit: 0 2024-02-15T18:36:32.537 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: format_version: 2 2024-02-15T18:36:32.537 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: enable_index_compression: 1 2024-02-15T18:36:32.537 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: block_align: 0 2024-02-15T18:36:32.537 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.538 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-15T18:36:32.538 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-15T18:36:32.538 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.compression: NoCompression 2024-02-15T18:36:32.538 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-15T18:36:32.538 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-15T18:36:32.538 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-15T18:36:32.538 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.num_levels: 7 2024-02-15T18:36:32.538 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-15T18:36:32.538 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-15T18:36:32.538 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-15T18:36:32.538 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-15T18:36:32.538 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-15T18:36:32.538 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-15T18:36:32.538 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-15T18:36:32.539 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-15T18:36:32.539 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-15T18:36:32.539 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-15T18:36:32.539 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-15T18:36:32.539 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-15T18:36:32.539 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-15T18:36:32.539 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-15T18:36:32.539 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-15T18:36:32.539 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-15T18:36:32.539 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.190+0000 7f3d865cb880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-15T18:36:32.539 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-15T18:36:32.539 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-15T18:36:32.539 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-15T18:36:32.540 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-15T18:36:32.540 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-15T18:36:32.540 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-15T18:36:32.540 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-15T18:36:32.540 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-15T18:36:32.540 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-15T18:36:32.540 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-15T18:36:32.540 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-15T18:36:32.540 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-15T18:36:32.540 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-15T18:36:32.540 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-15T18:36:32.540 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-15T18:36:32.540 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-15T18:36:32.541 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-15T18:36:32.541 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-15T18:36:32.541 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-15T18:36:32.541 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-15T18:36:32.541 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-15T18:36:32.541 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-15T18:36:32.541 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-15T18:36:32.541 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-15T18:36:32.541 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-15T18:36:32.541 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-15T18:36:32.541 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-15T18:36:32.541 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-15T18:36:32.541 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-15T18:36:32.541 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-15T18:36:32.541 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.table_properties_collectors: 2024-02-15T18:36:32.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.inplace_update_support: 0 2024-02-15T18:36:32.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-15T18:36:32.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-15T18:36:32.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-15T18:36:32.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-15T18:36:32.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.bloom_locality: 0 2024-02-15T18:36:32.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.max_successive_merges: 0 2024-02-15T18:36:32.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-15T18:36:32.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-15T18:36:32.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-15T18:36:32.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-15T18:36:32.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.ttl: 2592000 2024-02-15T18:36:32.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-15T18:36:32.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: [version_set.cc:3291] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-15T18:36:32.542 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi038/store.db/MANIFEST-000008 succeeded,manifest_file_number is 8, next_file_number is 22, last_sequence is 9040, log_number is 18,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-15T18:36:32.543 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.543 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 18 2024-02-15T18:36:32.543 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.543 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708022192193225, "job": 1, "event": "recovery_started", "log_files": [18]} 2024-02-15T18:36:32.543 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.191+0000 7f3d865cb880 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #18 mode 2 2024-02-15T18:36:32.543 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.195+0000 7f3d865cb880 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-15T18:36:32.543 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.199+0000 7f3d865cb880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708022192201226, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 22, "file_size": 3978454, "table_properties": {"data_size": 3972126, "index_size": 4800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 709, "raw_key_size": 5801, "raw_average_key_size": 22, "raw_value_size": 3964452, "raw_average_value_size": 15366, "num_data_blocks": 152, "num_entries": 258, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708022192, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-15T18:36:32.543 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.199+0000 7f3d865cb880 4 rocksdb: [version_set.cc:3826] Creating manifest 23 2024-02-15T18:36:32.543 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.543 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.199+0000 7f3d865cb880 4 rocksdb: [version_set.cc:3291] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-15T18:36:32.543 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.200+0000 7f3d865cb880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708022192201980, "job": 1, "event": "recovery_finished"} 2024-02-15T18:36:32.543 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.202+0000 7f3d865cb880 4 rocksdb: DB pointer 0x55da1675b800 2024-02-15T18:36:32.543 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.203+0000 7f3d6d3bc700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-15T18:36:32.544 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.203+0000 7f3d6d3bc700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-15T18:36:32.544 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: ** DB Stats ** 2024-02-15T18:36:32.544 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:36:32.544 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-15T18:36:32.544 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-15T18:36:32.544 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-15T18:36:32.544 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-15T18:36:32.544 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-15T18:36:32.544 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-15T18:36:32.544 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.544 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: ** Compaction Stats [default] ** 2024-02-15T18:36:32.544 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:36:32.544 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:36:32.545 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: L0 1/0 3.79 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 846.2 0.00 0.00 1 0.004 0 0 2024-02-15T18:36:32.545 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: L6 1/0 20.80 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:36:32.545 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Sum 2/0 24.59 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 846.2 0.00 0.00 1 0.004 0 0 2024-02-15T18:36:32.545 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 846.2 0.00 0.00 1 0.004 0 0 2024-02-15T18:36:32.545 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.545 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: ** Compaction Stats [default] ** 2024-02-15T18:36:32.545 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:36:32.545 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:36:32.545 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 846.2 0.00 0.00 1 0.004 0 0 2024-02-15T18:36:32.545 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:36:32.545 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Flush(GB): cumulative 0.004, interval 0.004 2024-02-15T18:36:32.545 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-15T18:36:32.545 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: AddFile(Total Files): cumulative 0, interval 0 2024-02-15T18:36:32.545 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-15T18:36:32.545 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: AddFile(Keys): cumulative 0, interval 0 2024-02-15T18:36:32.546 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Cumulative compaction: 0.00 GB write, 310.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:36:32.546 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Interval compaction: 0.00 GB write, 310.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:36:32.546 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-15T18:36:32.546 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.546 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: ** File Read Latency Histogram By Level [default] ** 2024-02-15T18:36:32.546 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.546 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: ** Compaction Stats [default] ** 2024-02-15T18:36:32.546 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:36:32.546 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:36:32.546 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: L0 1/0 3.79 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 846.2 0.00 0.00 1 0.004 0 0 2024-02-15T18:36:32.546 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: L6 1/0 20.80 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:36:32.546 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Sum 2/0 24.59 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 846.2 0.00 0.00 1 0.004 0 0 2024-02-15T18:36:32.546 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:36:32.546 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.547 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: ** Compaction Stats [default] ** 2024-02-15T18:36:32.547 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:36:32.547 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:36:32.547 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 846.2 0.00 0.00 1 0.004 0 0 2024-02-15T18:36:32.547 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:36:32.547 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Flush(GB): cumulative 0.004, interval 0.000 2024-02-15T18:36:32.547 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-15T18:36:32.547 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: AddFile(Total Files): cumulative 0, interval 0 2024-02-15T18:36:32.547 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-15T18:36:32.547 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: AddFile(Keys): cumulative 0, interval 0 2024-02-15T18:36:32.547 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Cumulative compaction: 0.00 GB write, 309.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:36:32.547 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:36:32.547 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-15T18:36:32.547 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.548 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: ** File Read Latency Histogram By Level [default] ** 2024-02-15T18:36:32.548 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.548 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.203+0000 7f3d865cb880 0 starting mon.smithi038 rank 0 at public addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] at bind addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi038 fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:36:32.548 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.204+0000 7f3d865cb880 1 mon.smithi038@-1(???) e2 preinit fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:36:32.548 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.204+0000 7f3d865cb880 0 mon.smithi038@-1(???).mds e13 new map 2024-02-15T18:36:32.548 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.204+0000 7f3d865cb880 0 mon.smithi038@-1(???).mds e13 print_map 2024-02-15T18:36:32.548 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: e13 2024-02-15T18:36:32.548 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:36:32.548 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:36:32.548 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: legacy client fscid: 1 2024-02-15T18:36:32.548 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.548 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Filesystem 'cephfs' (1) 2024-02-15T18:36:32.548 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: fs_name cephfs 2024-02-15T18:36:32.548 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: epoch 13 2024-02-15T18:36:32.549 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: flags 12 2024-02-15T18:36:32.549 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:36:32.549 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: modified 2024-02-15T18:35:07.485909+0000 2024-02-15T18:36:32.549 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: tableserver 0 2024-02-15T18:36:32.549 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: root 0 2024-02-15T18:36:32.549 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: session_timeout 60 2024-02-15T18:36:32.549 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: session_autoclose 300 2024-02-15T18:36:32.549 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: max_file_size 1099511627776 2024-02-15T18:36:32.549 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: required_client_features {} 2024-02-15T18:36:32.549 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: last_failure 0 2024-02-15T18:36:32.549 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: last_failure_osd_epoch 0 2024-02-15T18:36:32.549 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:36:32.549 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: max_mds 2 2024-02-15T18:36:32.550 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: in 0,1 2024-02-15T18:36:32.550 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: up {0=24261,1=24275} 2024-02-15T18:36:32.550 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: failed 2024-02-15T18:36:32.550 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: damaged 2024-02-15T18:36:32.550 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: stopped 2024-02-15T18:36:32.550 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: data_pools [3] 2024-02-15T18:36:32.550 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: metadata_pool 2 2024-02-15T18:36:32.550 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: inline_data disabled 2024-02-15T18:36:32.550 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: balancer 2024-02-15T18:36:32.550 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: standby_count_wanted 1 2024-02-15T18:36:32.550 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: [mds.cephfs.smithi121.mvumlh{0:24261} state up:active seq 3 export targets 1 join_fscid=1 addr [v2:172.21.15.121:6824/1422707571,v1:172.21.15.121:6825/1422707571] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:36:32.550 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: [mds.cephfs.smithi121.ovuzki{1:24275} state up:active seq 2 export targets 0 join_fscid=1 addr [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:36:32.550 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.550 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.551 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: Standby daemons: 2024-02-15T18:36:32.551 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.551 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: [mds.cephfs.smithi038.hxoggl{-1:14446} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.38:6826/3822880235,v1:172.21.15.38:6827/3822880235] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:36:32.551 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: [mds.cephfs.smithi038.qgkzgn{-1:14456} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.38:6828/703030231,v1:172.21.15.38:6829/703030231] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:36:32.551 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: 2024-02-15T18:36:32.551 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.205+0000 7f3d865cb880 0 mon.smithi038@-1(???).osd e46 crush map has features 3314933000852226048, adjusting msgr requires 2024-02-15T18:36:32.551 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.205+0000 7f3d865cb880 0 mon.smithi038@-1(???).osd e46 crush map has features 288514051259236352, adjusting msgr requires 2024-02-15T18:36:32.551 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.205+0000 7f3d865cb880 0 mon.smithi038@-1(???).osd e46 crush map has features 288514051259236352, adjusting msgr requires 2024-02-15T18:36:32.551 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.205+0000 7f3d865cb880 0 mon.smithi038@-1(???).osd e46 crush map has features 288514051259236352, adjusting msgr requires 2024-02-15T18:36:32.551 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:29.002301+0000 mon.smithi038 (mon.0) 686 : cluster [DBG] mgrmap e29: smithi038.khesbl(active, since 27s), standbys: smithi121.ckiryi 2024-02-15T18:36:32.551 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:36:32.205+0000 7f3d865cb880 1 mon.smithi038@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-02-15T18:36:32.551 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 bash[77911]: d3e8317a30cad30d568bf619d105b97e89e498292a7b35b0541afc480c34cc81 2024-02-15T18:36:32.551 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:32 smithi038 systemd[1]: Started Ceph mon.smithi038 for 7e05753e-cc2f-11ee-95ba-87774f69a715. 2024-02-15T18:36:33.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:30.139563+0000 mon.smithi121 (mon.1) 66 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-02-15T18:36:33.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:30.139694+0000 mon.smithi121 (mon.1) 67 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:36:33.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:30.927728+0000 mgr.smithi038.khesbl (mgr.14664) 27 : cluster [DBG] pgmap v18: 65 pgs: 65 active+clean; 306 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 38 KiB/s rd, 2.1 MiB/s wr, 369 op/s 2024-02-15T18:36:33.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:32.928230+0000 mgr.smithi038.khesbl (mgr.14664) 28 : cluster [DBG] pgmap v19: 65 pgs: 65 active+clean; 306 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.1 MiB/s wr, 275 op/s 2024-02-15T18:36:33.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:33.616405+0000 mon.smithi038 (mon.0) 1 : cluster [INF] mon.smithi038 calling monitor election 2024-02-15T18:36:33.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:33.631733+0000 mon.smithi038 (mon.0) 2 : cluster [INF] mon.smithi038 is new leader, mons smithi038,smithi121 in quorum (ranks 0,1) 2024-02-15T18:36:33.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:33.638463+0000 mon.smithi038 (mon.0) 3 : cluster [DBG] monmap e2: 2 mons at {smithi038=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0],smithi121=[v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0]} removed_ranks: {} 2024-02-15T18:36:33.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:33.639085+0000 mon.smithi038 (mon.0) 4 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:active} 2 up:standby 2024-02-15T18:36:33.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:33.639129+0000 mon.smithi038 (mon.0) 5 : cluster [DBG] osdmap e46: 6 total, 6 up, 6 in 2024-02-15T18:36:33.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:33.640481+0000 mon.smithi038 (mon.0) 6 : cluster [DBG] mgrmap e29: smithi038.khesbl(active, since 31s), standbys: smithi121.ckiryi 2024-02-15T18:36:33.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:33.653614+0000 mon.smithi038 (mon.0) 7 : cluster [INF] overall HEALTH_OK 2024-02-15T18:36:33.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:33.653792+0000 mon.smithi038 (mon.0) 8 : audit [INF] from='mgr.14664 ' entity='' 2024-02-15T18:36:33.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:33.656981+0000 mon.smithi121 (mon.1) 68 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:30.139563+0000 mon.smithi121 (mon.1) 66 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-02-15T18:36:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:30.139694+0000 mon.smithi121 (mon.1) 67 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:36:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:30.927728+0000 mgr.smithi038.khesbl (mgr.14664) 27 : cluster [DBG] pgmap v18: 65 pgs: 65 active+clean; 306 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 38 KiB/s rd, 2.1 MiB/s wr, 369 op/s 2024-02-15T18:36:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:32.928230+0000 mgr.smithi038.khesbl (mgr.14664) 28 : cluster [DBG] pgmap v19: 65 pgs: 65 active+clean; 306 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.1 MiB/s wr, 275 op/s 2024-02-15T18:36:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:33.616405+0000 mon.smithi038 (mon.0) 1 : cluster [INF] mon.smithi038 calling monitor election 2024-02-15T18:36:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:33.631733+0000 mon.smithi038 (mon.0) 2 : cluster [INF] mon.smithi038 is new leader, mons smithi038,smithi121 in quorum (ranks 0,1) 2024-02-15T18:36:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:33.638463+0000 mon.smithi038 (mon.0) 3 : cluster [DBG] monmap e2: 2 mons at {smithi038=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0],smithi121=[v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0]} removed_ranks: {} 2024-02-15T18:36:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:33.639085+0000 mon.smithi038 (mon.0) 4 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:active} 2 up:standby 2024-02-15T18:36:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:33.639129+0000 mon.smithi038 (mon.0) 5 : cluster [DBG] osdmap e46: 6 total, 6 up, 6 in 2024-02-15T18:36:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:33.640481+0000 mon.smithi038 (mon.0) 6 : cluster [DBG] mgrmap e29: smithi038.khesbl(active, since 31s), standbys: smithi121.ckiryi 2024-02-15T18:36:34.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:33.653614+0000 mon.smithi038 (mon.0) 7 : cluster [INF] overall HEALTH_OK 2024-02-15T18:36:34.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:33.653792+0000 mon.smithi038 (mon.0) 8 : audit [INF] from='mgr.14664 ' entity='' 2024-02-15T18:36:34.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:33.656981+0000 mon.smithi121 (mon.1) 68 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:35.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:33.706097+0000 mon.smithi038 (mon.0) 9 : cluster [DBG] mgrmap e30: smithi038.khesbl(active, since 31s), standbys: smithi121.ckiryi 2024-02-15T18:36:35.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:33.940857+0000 mon.smithi038 (mon.0) 10 : audit [INF] from='mgr.14664 ' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:35.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:33.706097+0000 mon.smithi038 (mon.0) 9 : cluster [DBG] mgrmap e30: smithi038.khesbl(active, since 31s), standbys: smithi121.ckiryi 2024-02-15T18:36:35.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:33.940857+0000 mon.smithi038 (mon.0) 10 : audit [INF] from='mgr.14664 ' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:36.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:34.928938+0000 mgr.smithi038.khesbl (mgr.14664) 29 : cluster [DBG] pgmap v20: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 1.7 MiB/s wr, 437 op/s 2024-02-15T18:36:36.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:34.928938+0000 mgr.smithi038.khesbl (mgr.14664) 29 : cluster [DBG] pgmap v20: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 1.7 MiB/s wr, 437 op/s 2024-02-15T18:36:37.202 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:36.929310+0000 mgr.smithi038.khesbl (mgr.14664) 30 : cluster [DBG] pgmap v21: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 297 op/s 2024-02-15T18:36:37.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:36.929310+0000 mgr.smithi038.khesbl (mgr.14664) 30 : cluster [DBG] pgmap v21: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 297 op/s 2024-02-15T18:36:39.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:38.929721+0000 mgr.smithi038.khesbl (mgr.14664) 31 : cluster [DBG] pgmap v22: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 297 op/s 2024-02-15T18:36:39.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:38.929721+0000 mgr.smithi038.khesbl (mgr.14664) 31 : cluster [DBG] pgmap v22: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 297 op/s 2024-02-15T18:36:41.261 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:40.930418+0000 mgr.smithi038.khesbl (mgr.14664) 32 : cluster [DBG] pgmap v23: 65 pgs: 65 active+clean; 300 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 38 KiB/s rd, 1.6 MiB/s wr, 417 op/s 2024-02-15T18:36:41.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:40.930418+0000 mgr.smithi038.khesbl (mgr.14664) 32 : cluster [DBG] pgmap v23: 65 pgs: 65 active+clean; 300 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 38 KiB/s rd, 1.6 MiB/s wr, 417 op/s 2024-02-15T18:36:43.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:42.930731+0000 mgr.smithi038.khesbl (mgr.14664) 33 : cluster [DBG] pgmap v24: 65 pgs: 65 active+clean; 300 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 283 op/s 2024-02-15T18:36:43.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:42.930731+0000 mgr.smithi038.khesbl (mgr.14664) 33 : cluster [DBG] pgmap v24: 65 pgs: 65 active+clean; 300 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 283 op/s 2024-02-15T18:36:45.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:44.070143+0000 mon.smithi038 (mon.0) 11 : audit [INF] from='mgr.14664 ' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:45.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:44.071574+0000 mon.smithi121 (mon.1) 69 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:36:45.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:44.072241+0000 mon.smithi121 (mon.1) 70 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:36:45.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:36:44.072843+0000 mgr.smithi038.khesbl (mgr.14664) 34 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:36:45.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:44.076089+0000 mon.smithi038 (mon.0) 12 : audit [INF] from='mgr.14664 ' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:45.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:44.083660+0000 mon.smithi121 (mon.1) 71 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:45.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:44.086216+0000 mon.smithi121 (mon.1) 72 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:36:45.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:44.087305+0000 mon.smithi121 (mon.1) 73 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-15T18:36:45.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:44.931215+0000 mgr.smithi038.khesbl (mgr.14664) 35 : cluster [DBG] pgmap v25: 65 pgs: 65 active+clean; 301 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.6 MiB/s wr, 412 op/s 2024-02-15T18:36:45.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:44.070143+0000 mon.smithi038 (mon.0) 11 : audit [INF] from='mgr.14664 ' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:45.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:44.071574+0000 mon.smithi121 (mon.1) 69 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:36:45.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:44.072241+0000 mon.smithi121 (mon.1) 70 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:36:45.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:36:44.072843+0000 mgr.smithi038.khesbl (mgr.14664) 34 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:36:45.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:44.076089+0000 mon.smithi038 (mon.0) 12 : audit [INF] from='mgr.14664 ' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:45.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:44.083660+0000 mon.smithi121 (mon.1) 71 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:45.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:44.086216+0000 mon.smithi121 (mon.1) 72 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:36:45.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:44.087305+0000 mon.smithi121 (mon.1) 73 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-15T18:36:45.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:44.931215+0000 mgr.smithi038.khesbl (mgr.14664) 35 : cluster [DBG] pgmap v25: 65 pgs: 65 active+clean; 301 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.6 MiB/s wr, 412 op/s 2024-02-15T18:36:47.169 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:46.931744+0000 mgr.smithi038.khesbl (mgr.14664) 36 : cluster [DBG] pgmap v26: 65 pgs: 65 active+clean; 301 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 998 KiB/s wr, 249 op/s 2024-02-15T18:36:47.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:46.931744+0000 mgr.smithi038.khesbl (mgr.14664) 36 : cluster [DBG] pgmap v26: 65 pgs: 65 active+clean; 301 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 998 KiB/s wr, 249 op/s 2024-02-15T18:36:49.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:36:48.482069+0000 mgr.smithi038.khesbl (mgr.14664) 37 : cephadm [INF] Upgrade: Updating mon.smithi121 2024-02-15T18:36:49.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:48.486483+0000 mon.smithi038 (mon.0) 13 : audit [INF] from='mgr.14664 ' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:49.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:48.490236+0000 mon.smithi121 (mon.1) 74 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-15T18:36:49.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:48.490890+0000 mon.smithi121 (mon.1) 75 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-15T18:36:49.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: audit 2024-02-15T18:36:48.491544+0000 mon.smithi121 (mon.1) 76 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:36:49.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cephadm 2024-02-15T18:36:48.492099+0000 mgr.smithi038.khesbl (mgr.14664) 38 : cephadm [INF] Deploying daemon mon.smithi121 on smithi121 2024-02-15T18:36:49.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: cluster 2024-02-15T18:36:48.932137+0000 mgr.smithi038.khesbl (mgr.14664) 39 : cluster [DBG] pgmap v27: 65 pgs: 65 active+clean; 301 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 998 KiB/s wr, 249 op/s 2024-02-15T18:36:49.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:36:48.482069+0000 mgr.smithi038.khesbl (mgr.14664) 37 : cephadm [INF] Upgrade: Updating mon.smithi121 2024-02-15T18:36:49.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:48.486483+0000 mon.smithi038 (mon.0) 13 : audit [INF] from='mgr.14664 ' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:49.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:48.490236+0000 mon.smithi121 (mon.1) 74 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-15T18:36:49.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:48.490890+0000 mon.smithi121 (mon.1) 75 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-15T18:36:49.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:48.491544+0000 mon.smithi121 (mon.1) 76 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:36:49.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:36:48.492099+0000 mgr.smithi038.khesbl (mgr.14664) 38 : cephadm [INF] Deploying daemon mon.smithi121 on smithi121 2024-02-15T18:36:49.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:48.932137+0000 mgr.smithi038.khesbl (mgr.14664) 39 : cluster [DBG] pgmap v27: 65 pgs: 65 active+clean; 301 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 998 KiB/s wr, 249 op/s 2024-02-15T18:36:50.486 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:50 smithi121 systemd[1]: Stopping Ceph mon.smithi121 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:36:50.486 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:36:50.493+0000 7f0a47fae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi121 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-15T18:36:50.486 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121[40989]: debug 2024-02-15T18:36:50.493+0000 7f0a47fae700 -1 mon.smithi121@1(peon) e2 *** Got Signal Terminated *** 2024-02-15T18:36:51.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:51 smithi121 bash[59547]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121 2024-02-15T18:36:52.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:51 smithi121 systemd[1]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mon.smithi121.service: Succeeded. 2024-02-15T18:36:52.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:51 smithi121 systemd[1]: Stopped Ceph mon.smithi121 for 7e05753e-cc2f-11ee-95ba-87774f69a715. 2024-02-15T18:36:52.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:51 smithi121 systemd[1]: Starting Ceph mon.smithi121 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:36:52.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.249+0000 7fe08fec1700 1 -- 172.21.15.38:0/2288237049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe08806c340 msgr2=0x7fe08806c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:52.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.249+0000 7fe08fec1700 1 --2- 172.21.15.38:0/2288237049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe08806c340 0x7fe08806c760 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fe084004660 tx=0x7fe084009b30).stop 2024-02-15T18:36:52.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.249+0000 7fe08fec1700 1 -- 172.21.15.38:0/2288237049 shutdown_connections 2024-02-15T18:36:52.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.249+0000 7fe08fec1700 1 --2- 172.21.15.38:0/2288237049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe08806d630 0x7fe08806dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:36:52.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.249+0000 7fe08fec1700 1 --2- 172.21.15.38:0/2288237049 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe08806c340 0x7fe08806c760 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0x7fe084004660 tx=0x7fe084009b30).stop 2024-02-15T18:36:52.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.249+0000 7fe08fec1700 1 -- 172.21.15.38:0/2288237049 >> 172.21.15.38:0/2288237049 conn(0x7fe0880f40e0 msgr2=0x7fe0880f64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:52.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.249+0000 7fe08fec1700 1 -- 172.21.15.38:0/2288237049 shutdown_connections 2024-02-15T18:36:52.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.250+0000 7fe08fec1700 1 -- 172.21.15.38:0/2288237049 wait complete. 2024-02-15T18:36:52.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.251+0000 7fe08fec1700 1 Processor -- start 2024-02-15T18:36:52.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.251+0000 7fe08fec1700 1 -- start start 2024-02-15T18:36:52.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.251+0000 7fe08fec1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe08806d630 0x7fe0880ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:52.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.251+0000 7fe08fec1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe088100200 0x7fe0881173b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:52.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.251+0000 7fe08fec1700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe088100760 con 0x7fe08806d630 2024-02-15T18:36:52.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.251+0000 7fe08fec1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0881008a0 con 0x7fe088100200 2024-02-15T18:36:52.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.251+0000 7fe08d45c700 1 -- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe088100200 msgr2=0x7fe0881173b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.121:3300/0 2024-02-15T18:36:52.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.251+0000 7fe08d45c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe088100200 0x7fe0881173b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-15T18:36:52.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.253+0000 7fe08dc5d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe08806d630 0x7fe0880ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:52.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.253+0000 7fe08dc5d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe08806d630 0x7fe0880ffd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48688/0 (socket says 172.21.15.38:48688) 2024-02-15T18:36:52.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.253+0000 7fe08dc5d700 1 -- 172.21.15.38:0/1163083131 learned_addr learned my addr 172.21.15.38:0/1163083131 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:36:52.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.253+0000 7fe08dc5d700 1 -- 172.21.15.38:0/1163083131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe088100200 msgr2=0x7fe0881173b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-15T18:36:52.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.253+0000 7fe08dc5d700 1 --2- 172.21.15.38:0/1163083131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe088100200 0x7fe0881173b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:36:52.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.253+0000 7fe08dc5d700 1 -- 172.21.15.38:0/1163083131 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe084005040 con 0x7fe08806d630 2024-02-15T18:36:52.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.253+0000 7fe08dc5d700 1 --2- 172.21.15.38:0/1163083131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe08806d630 0x7fe0880ffd10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fe084000c00 tx=0x7fe084009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:52.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.254+0000 7fe07e7fc700 1 -- 172.21.15.38:0/1163083131 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe084019070 con 0x7fe08806d630 2024-02-15T18:36:52.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.254+0000 7fe07e7fc700 1 -- 172.21.15.38:0/1163083131 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fe084007c20 con 0x7fe08806d630 2024-02-15T18:36:52.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.254+0000 7fe07e7fc700 1 -- 172.21.15.38:0/1163083131 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe084012620 con 0x7fe08806d630 2024-02-15T18:36:52.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.254+0000 7fe08fec1700 1 -- 172.21.15.38:0/1163083131 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe088117960 con 0x7fe08806d630 2024-02-15T18:36:52.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.255+0000 7fe08fec1700 1 -- 172.21.15.38:0/1163083131 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe088117ec0 con 0x7fe08806d630 2024-02-15T18:36:52.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.256+0000 7fe08fec1700 1 -- 172.21.15.38:0/1163083131 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe088045ad0 con 0x7fe08806d630 2024-02-15T18:36:52.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.257+0000 7fe07e7fc700 1 -- 172.21.15.38:0/1163083131 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 30) v1 ==== 77246+0+0 (secure 0 0 0) 0x7fe084024030 con 0x7fe08806d630 2024-02-15T18:36:52.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.258+0000 7fe07e7fc700 1 --2- 172.21.15.38:0/1163083131 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe074060fb0 0x7fe074063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:52.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.258+0000 7fe07e7fc700 1 -- 172.21.15.38:0/1163083131 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fe08407d480 con 0x7fe08806d630 2024-02-15T18:36:52.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.258+0000 7fe08d45c700 1 --2- 172.21.15.38:0/1163083131 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe074060fb0 0x7fe074063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:52.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.258+0000 7fe08d45c700 1 --2- 172.21.15.38:0/1163083131 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe074060fb0 0x7fe074063470 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fe078001050 tx=0x7fe078006af0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:52.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.266+0000 7fe07e7fc700 1 -- 172.21.15.38:0/1163083131 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe084051370 con 0x7fe08806d630 2024-02-15T18:36:52.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.557+0000 7fe08fec1700 1 -- 172.21.15.38:0/1163083131 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe088105640 con 0x7fe074060fb0 2024-02-15T18:36:52.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.559+0000 7fe07e7fc700 1 -- 172.21.15.38:0/1163083131 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fe088105640 con 0x7fe074060fb0 2024-02-15T18:36:52.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.561+0000 7fe08fec1700 1 -- 172.21.15.38:0/1163083131 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe074060fb0 msgr2=0x7fe074063470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:52.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.561+0000 7fe08fec1700 1 --2- 172.21.15.38:0/1163083131 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe074060fb0 0x7fe074063470 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fe078001050 tx=0x7fe078006af0).stop 2024-02-15T18:36:52.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.561+0000 7fe08fec1700 1 -- 172.21.15.38:0/1163083131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe08806d630 msgr2=0x7fe0880ffd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:52.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.561+0000 7fe08fec1700 1 --2- 172.21.15.38:0/1163083131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe08806d630 0x7fe0880ffd10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fe084000c00 tx=0x7fe084009b30).stop 2024-02-15T18:36:52.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.561+0000 7fe08fec1700 1 -- 172.21.15.38:0/1163083131 shutdown_connections 2024-02-15T18:36:52.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.561+0000 7fe08fec1700 1 --2- 172.21.15.38:0/1163083131 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe074060fb0 0x7fe074063470 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:52.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.561+0000 7fe08fec1700 1 --2- 172.21.15.38:0/1163083131 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe08806d630 0x7fe0880ffd10 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0x7fe084000c00 tx=0x7fe084009b30).stop 2024-02-15T18:36:52.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.561+0000 7fe08fec1700 1 --2- 172.21.15.38:0/1163083131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe088100200 0x7fe0881173b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:36:52.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.561+0000 7fe08fec1700 1 -- 172.21.15.38:0/1163083131 >> 172.21.15.38:0/1163083131 conn(0x7fe0880f40e0 msgr2=0x7fe0881034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:52.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.563+0000 7fe08fec1700 1 -- 172.21.15.38:0/1163083131 shutdown_connections 2024-02-15T18:36:52.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.563+0000 7fe08fec1700 1 -- 172.21.15.38:0/1163083131 wait complete. 2024-02-15T18:36:52.567 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-02-15T18:36:52.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.674+0000 7f5ccb251700 1 -- 172.21.15.38:0/1977968797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cc40f96f0 msgr2=0x7f5cc40fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:52.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.674+0000 7f5ccb251700 1 --2- 172.21.15.38:0/1977968797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cc40f96f0 0x7f5cc40fbae0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f5cb4004660 tx=0x7f5cb4009b30).stop 2024-02-15T18:36:52.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.674+0000 7f5ccb251700 1 -- 172.21.15.38:0/1977968797 shutdown_connections 2024-02-15T18:36:52.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.674+0000 7f5ccb251700 1 --2- 172.21.15.38:0/1977968797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cc40f96f0 0x7f5cc40fbae0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:52.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.674+0000 7f5ccb251700 1 --2- 172.21.15.38:0/1977968797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5cc40602b0 0x7f5cc40f9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:52.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.674+0000 7f5ccb251700 1 -- 172.21.15.38:0/1977968797 >> 172.21.15.38:0/1977968797 conn(0x7f5cc40f0b90 msgr2=0x7f5cc40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:52.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.674+0000 7f5ccb251700 1 -- 172.21.15.38:0/1977968797 shutdown_connections 2024-02-15T18:36:52.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.674+0000 7f5ccb251700 1 -- 172.21.15.38:0/1977968797 wait complete. 2024-02-15T18:36:52.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.675+0000 7f5ccb251700 1 Processor -- start 2024-02-15T18:36:52.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.675+0000 7f5ccb251700 1 -- start start 2024-02-15T18:36:52.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.675+0000 7f5ccb251700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5cc40602b0 0x7f5cc4106e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:52.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.675+0000 7f5ccb251700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cc40f96f0 0x7f5cc4107330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:52.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.675+0000 7f5ccb251700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc4107930 con 0x7f5cc40f96f0 2024-02-15T18:36:52.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.675+0000 7f5ccb251700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc4108140 con 0x7f5cc40602b0 2024-02-15T18:36:52.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.675+0000 7f5cc9a4e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cc40f96f0 0x7f5cc4107330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:52.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.675+0000 7f5cc9a4e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cc40f96f0 0x7f5cc4107330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48700/0 (socket says 172.21.15.38:48700) 2024-02-15T18:36:52.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.675+0000 7f5cc9a4e700 1 -- 172.21.15.38:0/3468777382 learned_addr learned my addr 172.21.15.38:0/3468777382 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:36:52.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.675+0000 7f5cca24f700 1 --2- 172.21.15.38:0/3468777382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5cc40602b0 0x7f5cc4106e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:52.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.675+0000 7f5cca24f700 1 -- 172.21.15.38:0/3468777382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5cc40602b0 msgr2=0x7f5cc4106e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-15T18:36:52.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.675+0000 7f5cca24f700 1 -- 172.21.15.38:0/3468777382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5cc40602b0 msgr2=0x7f5cc4106e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-15T18:36:52.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.675+0000 7f5cca24f700 1 --2- 172.21.15.38:0/3468777382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5cc40602b0 0x7f5cc4106e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-15T18:36:52.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.676+0000 7f5cca24f700 1 --2- 172.21.15.38:0/3468777382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5cc40602b0 0x7f5cc4106e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-02-15T18:36:52.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.676+0000 7f5cc9a4e700 1 -- 172.21.15.38:0/3468777382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5cc40602b0 msgr2=0x7f5cc4106e40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-15T18:36:52.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.676+0000 7f5cc9a4e700 1 --2- 172.21.15.38:0/3468777382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5cc40602b0 0x7f5cc4106e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:52.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.676+0000 7f5cc9a4e700 1 -- 172.21.15.38:0/3468777382 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb4005040 con 0x7f5cc40f96f0 2024-02-15T18:36:52.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.676+0000 7f5cc9a4e700 1 --2- 172.21.15.38:0/3468777382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cc40f96f0 0x7f5cc4107330 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f5cb4004e30 tx=0x7f5cb4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:52.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.677+0000 7f5cbaffd700 1 -- 172.21.15.38:0/3468777382 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cb4018070 con 0x7f5cc40f96f0 2024-02-15T18:36:52.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.677+0000 7f5cbaffd700 1 -- 172.21.15.38:0/3468777382 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f5cb4007ba0 con 0x7f5cc40f96f0 2024-02-15T18:36:52.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.677+0000 7f5ccb251700 1 -- 172.21.15.38:0/3468777382 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc4108360 con 0x7f5cc40f96f0 2024-02-15T18:36:52.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.677+0000 7f5cbaffd700 1 -- 172.21.15.38:0/3468777382 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cb401c620 con 0x7f5cc40f96f0 2024-02-15T18:36:52.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.677+0000 7f5ccb251700 1 -- 172.21.15.38:0/3468777382 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc41088c0 con 0x7f5cc40f96f0 2024-02-15T18:36:52.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.678+0000 7f5ccb251700 1 -- 172.21.15.38:0/3468777382 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc4059100 con 0x7f5cc40f96f0 2024-02-15T18:36:52.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.678+0000 7f5cbaffd700 1 -- 172.21.15.38:0/3468777382 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 30) v1 ==== 77246+0+0 (secure 0 0 0) 0x7f5cb401d9a0 con 0x7f5cc40f96f0 2024-02-15T18:36:52.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.679+0000 7f5cbaffd700 1 --2- 172.21.15.38:0/3468777382 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5cb0061000 0x7f5cb00634c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:52.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.679+0000 7f5cbaffd700 1 -- 172.21.15.38:0/3468777382 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f5cb407d970 con 0x7f5cc40f96f0 2024-02-15T18:36:52.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.679+0000 7f5cca24f700 1 --2- 172.21.15.38:0/3468777382 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5cb0061000 0x7f5cb00634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:52.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.679+0000 7f5cca24f700 1 --2- 172.21.15.38:0/3468777382 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5cb0061000 0x7f5cb00634c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f5cc4106bb0 tx=0x7f5cc0009c70).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:52.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.683+0000 7f5cbaffd700 1 -- 172.21.15.38:0/3468777382 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f5cb404dde0 con 0x7f5cc40f96f0 2024-02-15T18:36:52.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.526+0000 7fca33108880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-15T18:36:52.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.526+0000 7fca33108880 0 ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable), process ceph-mon, pid 6 2024-02-15T18:36:52.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.526+0000 7fca33108880 0 pidfile_write: ignore empty --pid-file 2024-02-15T18:36:52.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.532+0000 7fca33108880 0 load: jerasure load: lrc load: isa 2024-02-15T18:36:52.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: RocksDB version: 6.8.1 2024-02-15T18:36:52.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: 2024-02-15T18:36:52.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-15T18:36:52.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Compile date Feb 8 2024 2024-02-15T18:36:52.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: DB SUMMARY 2024-02-15T18:36:52.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: 2024-02-15T18:36:52.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: CURRENT file: CURRENT 2024-02-15T18:36:52.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: 2024-02-15T18:36:52.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: IDENTITY file: IDENTITY 2024-02-15T18:36:52.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: 2024-02-15T18:36:52.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 695 Bytes 2024-02-15T18:36:52.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: 2024-02-15T18:36:52.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi121/store.db dir, Total Num: 1, files: 000017.sst 2024-02-15T18:36:52.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: 2024-02-15T18:36:52.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi121/store.db: 000015.log size: 7758801 ; 2024-02-15T18:36:52.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: 2024-02-15T18:36:52.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.error_if_exists: 0 2024-02-15T18:36:52.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.create_if_missing: 0 2024-02-15T18:36:52.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.paranoid_checks: 1 2024-02-15T18:36:52.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.env: 0x55dcf8560080 2024-02-15T18:36:52.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.fs: Posix File System 2024-02-15T18:36:52.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.info_log: 0x55dcfa6f6560 2024-02-15T18:36:52.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-15T18:36:52.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.statistics: (nil) 2024-02-15T18:36:52.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.use_fsync: 0 2024-02-15T18:36:52.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_log_file_size: 0 2024-02-15T18:36:52.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-15T18:36:52.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-15T18:36:52.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-15T18:36:52.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-15T18:36:52.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.allow_fallocate: 1 2024-02-15T18:36:52.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-15T18:36:52.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-15T18:36:52.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.use_direct_reads: 0 2024-02-15T18:36:52.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-15T18:36:52.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-15T18:36:52.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.db_log_dir: 2024-02-15T18:36:52.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi121/store.db 2024-02-15T18:36:52.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-15T18:36:52.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_subcompactions: 1 2024-02-15T18:36:52.862 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_background_flushes: -1 2024-02-15T18:36:52.862 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-15T18:36:52.862 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-15T18:36:52.863 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-15T18:36:52.863 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-15T18:36:52.863 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-15T18:36:52.863 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-15T18:36:52.863 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-15T18:36:52.863 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.write_buffer_manager: 0x55dcfb3e83f0 2024-02-15T18:36:52.863 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-15T18:36:52.863 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-15T18:36:52.863 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-15T18:36:52.863 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-15T18:36:52.863 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-15T18:36:52.863 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-15T18:36:52.864 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-15T18:36:52.864 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-15T18:36:52.864 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-15T18:36:52.864 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.unordered_write: 0 2024-02-15T18:36:52.864 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-15T18:36:52.864 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-15T18:36:52.864 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-15T18:36:52.864 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-15T18:36:52.864 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.row_cache: None 2024-02-15T18:36:52.864 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.wal_filter: None 2024-02-15T18:36:52.864 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-15T18:36:52.864 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-15T18:36:52.864 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.preserve_deletes: 0 2024-02-15T18:36:52.865 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.two_write_queues: 0 2024-02-15T18:36:52.865 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-15T18:36:52.865 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.atomic_flush: 0 2024-02-15T18:36:52.865 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-15T18:36:52.865 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-15T18:36:52.865 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-15T18:36:52.865 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.log_readahead_size: 0 2024-02-15T18:36:52.865 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-15T18:36:52.865 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_background_jobs: 2 2024-02-15T18:36:52.865 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_background_compactions: -1 2024-02-15T18:36:52.865 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-15T18:36:52.865 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-15T18:36:52.865 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-15T18:36:52.866 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-15T18:36:52.866 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-15T18:36:52.866 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-15T18:36:52.866 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-15T18:36:52.866 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-15T18:36:52.866 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_open_files: -1 2024-02-15T18:36:52.866 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-15T18:36:52.866 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-15T18:36:52.866 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-15T18:36:52.866 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-15T18:36:52.866 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Compression algorithms supported: 2024-02-15T18:36:52.866 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-15T18:36:52.866 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: kZSTD supported: 0 2024-02-15T18:36:52.866 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: kXpressCompression supported: 0 2024-02-15T18:36:52.866 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-15T18:36:52.867 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: kLZ4Compression supported: 1 2024-02-15T18:36:52.867 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: kBZip2Compression supported: 0 2024-02-15T18:36:52.867 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: kZlibCompression supported: 1 2024-02-15T18:36:52.867 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: kSnappyCompression supported: 1 2024-02-15T18:36:52.867 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-15T18:36:52.867 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi121/store.db/MANIFEST-000005 2024-02-15T18:36:52.867 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: 2024-02-15T18:36:52.867 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-15T18:36:52.867 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: 2024-02-15T18:36:52.867 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-15T18:36:52.867 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.merge_operator: 2024-02-15T18:36:52.867 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.compaction_filter: None 2024-02-15T18:36:52.867 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-15T18:36:52.868 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-15T18:36:52.868 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-15T18:36:52.868 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55dcfa69bd28) 2024-02-15T18:36:52.868 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cache_index_and_filter_blocks: 1 2024-02-15T18:36:52.868 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-15T18:36:52.868 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-15T18:36:52.868 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: pin_top_level_index_and_filter: 1 2024-02-15T18:36:52.868 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: index_type: 0 2024-02-15T18:36:52.868 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: data_block_index_type: 0 2024-02-15T18:36:52.868 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: index_shortening: 1 2024-02-15T18:36:52.868 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: data_block_hash_table_util_ratio: 0.750000 2024-02-15T18:36:52.868 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: hash_index_allow_collision: 1 2024-02-15T18:36:52.868 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: checksum: 1 2024-02-15T18:36:52.868 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: no_block_cache: 0 2024-02-15T18:36:52.868 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: block_cache: 0x55dcfa6d0f10 2024-02-15T18:36:52.869 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: block_cache_name: BinnedLRUCache 2024-02-15T18:36:52.869 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: block_cache_options: 2024-02-15T18:36:52.869 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: capacity : 536870912 2024-02-15T18:36:52.869 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: num_shard_bits : 4 2024-02-15T18:36:52.869 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: strict_capacity_limit : 0 2024-02-15T18:36:52.869 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: high_pri_pool_ratio: 0.000 2024-02-15T18:36:52.869 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: block_cache_compressed: (nil) 2024-02-15T18:36:52.869 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: persistent_cache: (nil) 2024-02-15T18:36:52.869 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: block_size: 4096 2024-02-15T18:36:52.869 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: block_size_deviation: 10 2024-02-15T18:36:52.869 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: block_restart_interval: 16 2024-02-15T18:36:52.869 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: index_block_restart_interval: 1 2024-02-15T18:36:52.869 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: metadata_block_size: 4096 2024-02-15T18:36:52.869 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: partition_filters: 0 2024-02-15T18:36:52.869 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: use_delta_encoding: 1 2024-02-15T18:36:52.870 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-15T18:36:52.870 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: whole_key_filtering: 1 2024-02-15T18:36:52.870 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: verify_compression: 0 2024-02-15T18:36:52.870 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: read_amp_bytes_per_bit: 0 2024-02-15T18:36:52.870 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: format_version: 2 2024-02-15T18:36:52.870 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: enable_index_compression: 1 2024-02-15T18:36:52.870 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: block_align: 0 2024-02-15T18:36:52.870 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: 2024-02-15T18:36:52.870 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-15T18:36:52.870 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-15T18:36:52.870 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.compression: NoCompression 2024-02-15T18:36:52.870 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-15T18:36:52.870 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-15T18:36:52.871 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-15T18:36:52.871 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.num_levels: 7 2024-02-15T18:36:52.871 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-15T18:36:52.871 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-15T18:36:52.871 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-15T18:36:52.871 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-15T18:36:52.871 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-15T18:36:52.871 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-15T18:36:52.871 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-15T18:36:52.871 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-15T18:36:52.871 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-15T18:36:52.871 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-15T18:36:52.871 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-15T18:36:52.871 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-15T18:36:52.871 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-15T18:36:52.872 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-15T18:36:52.872 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-15T18:36:52.872 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-15T18:36:52.872 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-15T18:36:52.872 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-15T18:36:52.872 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-15T18:36:52.872 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-15T18:36:52.872 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-15T18:36:52.872 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-15T18:36:52.872 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-15T18:36:52.872 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-15T18:36:52.872 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-15T18:36:52.872 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-15T18:36:52.872 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-15T18:36:52.873 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-15T18:36:52.873 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-15T18:36:52.873 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-15T18:36:52.873 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-15T18:36:52.873 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-15T18:36:52.873 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-15T18:36:52.873 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-15T18:36:52.873 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-15T18:36:52.873 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-15T18:36:52.873 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-15T18:36:52.873 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-15T18:36:52.873 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-15T18:36:52.873 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-15T18:36:52.873 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-15T18:36:52.873 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-15T18:36:52.874 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-15T18:36:52.874 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-15T18:36:52.874 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-15T18:36:52.874 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-15T18:36:52.874 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-15T18:36:52.874 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.table_properties_collectors: 2024-02-15T18:36:52.874 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.inplace_update_support: 0 2024-02-15T18:36:52.874 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-15T18:36:52.874 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-15T18:36:52.874 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-15T18:36:52.874 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-15T18:36:52.874 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.bloom_locality: 0 2024-02-15T18:36:52.874 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.max_successive_merges: 0 2024-02-15T18:36:52.874 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-15T18:36:52.874 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-15T18:36:52.875 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-15T18:36:52.875 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-15T18:36:52.875 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.ttl: 2592000 2024-02-15T18:36:52.875 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.533+0000 7fca33108880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-15T18:36:52.875 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.534+0000 7fca33108880 4 rocksdb: [version_set.cc:3291] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-15T18:36:52.875 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.534+0000 7fca33108880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi121/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 19, last_sequence is 8972, log_number is 15,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-15T18:36:52.875 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: 2024-02-15T18:36:52.875 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.534+0000 7fca33108880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 15 2024-02-15T18:36:52.875 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: 2024-02-15T18:36:52.875 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.534+0000 7fca33108880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708022212535675, "job": 1, "event": "recovery_started", "log_files": [15]} 2024-02-15T18:36:52.875 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.534+0000 7fca33108880 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #15 mode 2 2024-02-15T18:36:52.875 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.542+0000 7fca33108880 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-15T18:36:52.875 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.550+0000 7fca33108880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708022212551637, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 19, "file_size": 5058287, "table_properties": {"data_size": 5050129, "index_size": 6374, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 965, "raw_key_size": 7323, "raw_average_key_size": 21, "raw_value_size": 5040223, "raw_average_value_size": 15135, "num_data_blocks": 201, "num_entries": 333, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708022212, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-15T18:36:52.875 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.550+0000 7fca33108880 4 rocksdb: [version_set.cc:3826] Creating manifest 20 2024-02-15T18:36:52.876 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: 2024-02-15T18:36:52.876 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.550+0000 7fca33108880 4 rocksdb: [version_set.cc:3291] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-15T18:36:52.876 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.559+0000 7fca33108880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708022212560238, "job": 1, "event": "recovery_finished"} 2024-02-15T18:36:52.876 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.563+0000 7fca33108880 4 rocksdb: DB pointer 0x55dcfa799800 2024-02-15T18:36:52.876 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.563+0000 7fca33108880 0 starting mon.smithi121 rank 1 at public addrs [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] at bind addrs [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi121 fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:36:52.876 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.564+0000 7fca19ef9700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-15T18:36:52.876 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.564+0000 7fca19ef9700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-15T18:36:52.876 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: ** DB Stats ** 2024-02-15T18:36:52.876 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:36:52.876 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-15T18:36:52.876 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-15T18:36:52.876 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-15T18:36:52.876 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-15T18:36:52.876 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-15T18:36:52.877 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-15T18:36:52.877 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: 2024-02-15T18:36:52.877 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: ** Compaction Stats [default] ** 2024-02-15T18:36:52.877 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:36:52.877 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:36:52.877 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: L0 1/0 4.82 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 583.9 0.01 0.00 1 0.008 0 0 2024-02-15T18:36:52.877 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: L6 1/0 20.80 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:36:52.877 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Sum 2/0 25.62 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 583.9 0.01 0.00 1 0.008 0 0 2024-02-15T18:36:52.877 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 583.9 0.01 0.00 1 0.008 0 0 2024-02-15T18:36:52.877 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: 2024-02-15T18:36:52.877 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: ** Compaction Stats [default] ** 2024-02-15T18:36:52.877 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:36:52.877 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:36:52.877 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 583.9 0.01 0.00 1 0.008 0 0 2024-02-15T18:36:52.878 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:36:52.878 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Flush(GB): cumulative 0.005, interval 0.005 2024-02-15T18:36:52.878 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-15T18:36:52.878 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: AddFile(Total Files): cumulative 0, interval 0 2024-02-15T18:36:52.878 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-15T18:36:52.878 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: AddFile(Keys): cumulative 0, interval 0 2024-02-15T18:36:52.878 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Cumulative compaction: 0.00 GB write, 155.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:36:52.878 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Interval compaction: 0.00 GB write, 155.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:36:52.878 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-15T18:36:52.878 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: 2024-02-15T18:36:52.878 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: ** File Read Latency Histogram By Level [default] ** 2024-02-15T18:36:52.878 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: 2024-02-15T18:36:52.878 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: ** Compaction Stats [default] ** 2024-02-15T18:36:52.878 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:36:52.879 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:36:52.879 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: L0 1/0 4.82 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 583.9 0.01 0.00 1 0.008 0 0 2024-02-15T18:36:52.879 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: L6 1/0 20.80 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:36:52.879 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Sum 2/0 25.62 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 583.9 0.01 0.00 1 0.008 0 0 2024-02-15T18:36:52.879 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:36:52.879 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: 2024-02-15T18:36:52.879 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: ** Compaction Stats [default] ** 2024-02-15T18:36:52.879 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:36:52.879 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:36:52.879 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 583.9 0.01 0.00 1 0.008 0 0 2024-02-15T18:36:52.879 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:36:52.879 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Flush(GB): cumulative 0.005, interval 0.000 2024-02-15T18:36:52.880 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-15T18:36:52.880 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: AddFile(Total Files): cumulative 0, interval 0 2024-02-15T18:36:52.880 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-15T18:36:52.880 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: AddFile(Keys): cumulative 0, interval 0 2024-02-15T18:36:52.880 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Cumulative compaction: 0.00 GB write, 155.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:36:52.880 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:36:52.880 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-15T18:36:52.880 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: 2024-02-15T18:36:52.880 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: ** File Read Latency Histogram By Level [default] ** 2024-02-15T18:36:52.880 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: 2024-02-15T18:36:52.880 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:36:52.564+0000 7fca33108880 1 mon.smithi121@-1(???) e2 preinit fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 2024-02-15T18:36:52.880 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:36:48.482069+0000 mgr.smithi038.khesbl (mgr.14664) 37 : cephadm [INF] Upgrade: Updating mon.smithi121 2024-02-15T18:36:52.880 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:48.486483+0000 mon.smithi038 (mon.0) 13 : audit [INF] from='mgr.14664 ' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:52.881 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:48.490236+0000 mon.smithi121 (mon.1) 74 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-15T18:36:52.881 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:48.490890+0000 mon.smithi121 (mon.1) 75 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-15T18:36:52.881 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:48.491544+0000 mon.smithi121 (mon.1) 76 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:36:52.881 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:36:48.492099+0000 mgr.smithi038.khesbl (mgr.14664) 38 : cephadm [INF] Deploying daemon mon.smithi121 on smithi121 2024-02-15T18:36:52.881 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:36:48.932137+0000 mgr.smithi038.khesbl (mgr.14664) 39 : cluster [DBG] pgmap v27: 65 pgs: 65 active+clean; 301 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 998 KiB/s wr, 249 op/s 2024-02-15T18:36:52.881 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 bash[59718]: 42e1dda1a39df0518e5e125dd9555ba2da1c41448b290aaabc8a318e395d814b 2024-02-15T18:36:52.881 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:52 smithi121 systemd[1]: Started Ceph mon.smithi121 for 7e05753e-cc2f-11ee-95ba-87774f69a715. 2024-02-15T18:36:52.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.884+0000 7f5ccb251700 1 -- 172.21.15.38:0/3468777382 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cc40fefb0 con 0x7f5cb0061000 2024-02-15T18:36:52.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.885+0000 7f5cbaffd700 1 -- 172.21.15.38:0/3468777382 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f5cc40fefb0 con 0x7f5cb0061000 2024-02-15T18:36:52.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.887+0000 7f5ccb251700 1 -- 172.21.15.38:0/3468777382 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5cb0061000 msgr2=0x7f5cb00634c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:52.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.887+0000 7f5ccb251700 1 --2- 172.21.15.38:0/3468777382 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5cb0061000 0x7f5cb00634c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f5cc4106bb0 tx=0x7f5cc0009c70).stop 2024-02-15T18:36:52.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.888+0000 7f5ccb251700 1 -- 172.21.15.38:0/3468777382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cc40f96f0 msgr2=0x7f5cc4107330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:52.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.888+0000 7f5ccb251700 1 --2- 172.21.15.38:0/3468777382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cc40f96f0 0x7f5cc4107330 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f5cb4004e30 tx=0x7f5cb4009b30).stop 2024-02-15T18:36:52.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.888+0000 7f5ccb251700 1 -- 172.21.15.38:0/3468777382 shutdown_connections 2024-02-15T18:36:52.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.888+0000 7f5ccb251700 1 --2- 172.21.15.38:0/3468777382 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5cb0061000 0x7f5cb00634c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:52.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.888+0000 7f5ccb251700 1 --2- 172.21.15.38:0/3468777382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5cc40602b0 0x7f5cc4106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:52.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.888+0000 7f5ccb251700 1 --2- 172.21.15.38:0/3468777382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cc40f96f0 0x7f5cc4107330 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:52.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.888+0000 7f5ccb251700 1 -- 172.21.15.38:0/3468777382 >> 172.21.15.38:0/3468777382 conn(0x7f5cc40f0b90 msgr2=0x7f5cc40f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:52.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.888+0000 7f5ccb251700 1 -- 172.21.15.38:0/3468777382 shutdown_connections 2024-02-15T18:36:52.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.888+0000 7f5ccb251700 1 -- 172.21.15.38:0/3468777382 wait complete. 2024-02-15T18:36:52.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.990+0000 7fcc01731700 1 -- 172.21.15.38:0/1609139870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbfc0f77d0 msgr2=0x7fcbfc0f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:52.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.990+0000 7fcc01731700 1 --2- 172.21.15.38:0/1609139870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbfc0f77d0 0x7fcbfc0f7c30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fcbf0004660 tx=0x7fcbf0009b30).stop 2024-02-15T18:36:52.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.990+0000 7fcc01731700 1 -- 172.21.15.38:0/1609139870 shutdown_connections 2024-02-15T18:36:52.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.990+0000 7fcc01731700 1 --2- 172.21.15.38:0/1609139870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbfc0f77d0 0x7fcbfc0f7c30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:52.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.990+0000 7fcc01731700 1 --2- 172.21.15.38:0/1609139870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0f6530 0x7fcbfc0f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:52.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.990+0000 7fcc01731700 1 -- 172.21.15.38:0/1609139870 >> 172.21.15.38:0/1609139870 conn(0x7fcbfc0f1c90 msgr2=0x7fcbfc0f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:52.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.991+0000 7fcc01731700 1 -- 172.21.15.38:0/1609139870 shutdown_connections 2024-02-15T18:36:52.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.991+0000 7fcc01731700 1 -- 172.21.15.38:0/1609139870 wait complete. 2024-02-15T18:36:52.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.991+0000 7fcc01731700 1 Processor -- start 2024-02-15T18:36:52.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.991+0000 7fcc01731700 1 -- start start 2024-02-15T18:36:52.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.991+0000 7fcc01731700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0f6530 0x7fcbfc06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:52.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.991+0000 7fcc01731700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbfc0f77d0 0x7fcbfc069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:52.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.991+0000 7fcc01731700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc06a380 con 0x7fcbfc0f77d0 2024-02-15T18:36:52.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.991+0000 7fcc01731700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc06a4c0 con 0x7fcbfc0f6530 2024-02-15T18:36:52.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.992+0000 7fcbfb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbfc0f77d0 0x7fcbfc069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:52.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.992+0000 7fcbfb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbfc0f77d0 0x7fcbfc069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48718/0 (socket says 172.21.15.38:48718) 2024-02-15T18:36:52.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.992+0000 7fcbfbfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0f6530 0x7fcbfc06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:52.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.992+0000 7fcbfb7fe700 1 -- 172.21.15.38:0/2421880489 learned_addr learned my addr 172.21.15.38:0/2421880489 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:36:52.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.992+0000 7fcbfbfff700 1 -- 172.21.15.38:0/2421880489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0f6530 msgr2=0x7fcbfc06b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-15T18:36:52.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.992+0000 7fcbfbfff700 1 -- 172.21.15.38:0/2421880489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0f6530 msgr2=0x7fcbfc06b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-15T18:36:52.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.992+0000 7fcbfbfff700 1 --2- 172.21.15.38:0/2421880489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0f6530 0x7fcbfc06b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-15T18:36:52.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.992+0000 7fcbfbfff700 1 --2- 172.21.15.38:0/2421880489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0f6530 0x7fcbfc06b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-02-15T18:36:52.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.992+0000 7fcbfb7fe700 1 -- 172.21.15.38:0/2421880489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0f6530 msgr2=0x7fcbfc06b9a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-15T18:36:52.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.992+0000 7fcbfb7fe700 1 --2- 172.21.15.38:0/2421880489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0f6530 0x7fcbfc06b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:52.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.992+0000 7fcbfb7fe700 1 -- 172.21.15.38:0/2421880489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf0005040 con 0x7fcbfc0f77d0 2024-02-15T18:36:52.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.992+0000 7fcbfb7fe700 1 --2- 172.21.15.38:0/2421880489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbfc0f77d0 0x7fcbfc069e20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fcbf0009f50 tx=0x7fcbf0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:52.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.992+0000 7fcbf8ff9700 1 -- 172.21.15.38:0/2421880489 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbf0018070 con 0x7fcbfc0f77d0 2024-02-15T18:36:52.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.992+0000 7fcbf8ff9700 1 -- 172.21.15.38:0/2421880489 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fcbf0007ba0 con 0x7fcbfc0f77d0 2024-02-15T18:36:52.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.992+0000 7fcc01731700 1 -- 172.21.15.38:0/2421880489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbfc06a740 con 0x7fcbfc0f77d0 2024-02-15T18:36:52.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.992+0000 7fcbf8ff9700 1 -- 172.21.15.38:0/2421880489 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbf001c620 con 0x7fcbfc0f77d0 2024-02-15T18:36:52.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.992+0000 7fcc01731700 1 -- 172.21.15.38:0/2421880489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbfc06aca0 con 0x7fcbfc0f77d0 2024-02-15T18:36:52.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.993+0000 7fcc01731700 1 -- 172.21.15.38:0/2421880489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbfc045ad0 con 0x7fcbfc0f77d0 2024-02-15T18:36:52.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.993+0000 7fcbf8ff9700 1 -- 172.21.15.38:0/2421880489 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 30) v1 ==== 77246+0+0 (secure 0 0 0) 0x7fcbf001d420 con 0x7fcbfc0f77d0 2024-02-15T18:36:52.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.994+0000 7fcbf8ff9700 1 --2- 172.21.15.38:0/2421880489 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcbe4061000 0x7fcbe40634c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:52.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.994+0000 7fcbf8ff9700 1 -- 172.21.15.38:0/2421880489 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fcbf007dfb0 con 0x7fcbfc0f77d0 2024-02-15T18:36:52.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.994+0000 7fcbfbfff700 1 --2- 172.21.15.38:0/2421880489 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcbe4061000 0x7fcbe40634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:52.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.994+0000 7fcbfbfff700 1 --2- 172.21.15.38:0/2421880489 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcbe4061000 0x7fcbe40634c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fcbfc06b730 tx=0x7fcbec009c40).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:52.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:52.998+0000 7fcbf8ff9700 1 -- 172.21.15.38:0/2421880489 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fcbf004e3c0 con 0x7fcbfc0f77d0 2024-02-15T18:36:53.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.185+0000 7fcc01731700 1 -- 172.21.15.38:0/2421880489 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbfc057f10 con 0x7fcbe4061000 2024-02-15T18:36:53.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.195+0000 7fcbf8ff9700 1 -- 172.21.15.38:0/2421880489 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fcbfc057f10 con 0x7fcbe4061000 2024-02-15T18:36:53.185 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:36:53.185 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (7m) 9s ago 9m 19.3M - 0.20.0 0881eb8f169f c2034c318e7c 2024-02-15T18:36:53.185 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (9m) 9s ago 9m 6395k - 16.2.4 8d91d370c2b8 0fd3f945d009 2024-02-15T18:36:53.185 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi121 smithi121 running (8m) 26s ago 8m 6405k - 16.2.4 8d91d370c2b8 affc02db354e 2024-02-15T18:36:53.185 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (7m) 9s ago 9m 33.4M - 6.7.4 557c83e11646 2dcccfbfbd2a 2024-02-15T18:36:53.185 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.hxoggl smithi038 running (3m) 9s ago 3m 14.3M - 16.2.4 8d91d370c2b8 be2533754b9c 2024-02-15T18:36:53.185 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.qgkzgn smithi038 running (3m) 9s ago 3m 13.9M - 16.2.4 8d91d370c2b8 dd3ffafba9bd 2024-02-15T18:36:53.186 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.mvumlh smithi121 running (4m) 26s ago 4m 184M - 16.2.4 8d91d370c2b8 88b7d876a39b 2024-02-15T18:36:53.186 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.ovuzki smithi121 running (3m) 26s ago 3m 130M - 16.2.4 8d91d370c2b8 21cacf0012f9 2024-02-15T18:36:53.186 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.khesbl smithi038 *:8443,9283 running (59s) 9s ago 11m 425M - 16.2.14-521-g0e714d9a 1fbc92c859dc e642f3c2cfc9 2024-02-15T18:36:53.186 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi121.ckiryi smithi121 *:8443,9283 running (32s) 26s ago 8m 43.7M - 16.2.14-521-g0e714d9a 1fbc92c859dc 82515b025594 2024-02-15T18:36:53.186 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (21s) 9s ago 11m 21.9M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc d3e8317a30ca 2024-02-15T18:36:53.186 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi121 smithi121 starting - - - 2048M 2024-02-15T18:36:53.186 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (9m) 9s ago 9m 17.1M - 0.18.1 e5a616e4b9cf 037f60e38158 2024-02-15T18:36:53.186 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi121 smithi121 *:9100 running (8m) 26s ago 8m 17.2M - 0.18.1 e5a616e4b9cf a867bff85e18 2024-02-15T18:36:53.186 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (7m) 9s ago 7m 368M 4096M 16.2.4 8d91d370c2b8 51512003a2ef 2024-02-15T18:36:53.186 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (7m) 9s ago 7m 427M 4096M 16.2.4 8d91d370c2b8 16dbdf20b170 2024-02-15T18:36:53.186 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (6m) 9s ago 6m 330M 4096M 16.2.4 8d91d370c2b8 6d6ee813e5be 2024-02-15T18:36:53.187 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi121 running (6m) 26s ago 6m 421M 4096M 16.2.4 8d91d370c2b8 cba2d9ed4730 2024-02-15T18:36:53.187 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi121 running (6m) 26s ago 6m 416M 4096M 16.2.4 8d91d370c2b8 ac17ad7dc628 2024-02-15T18:36:53.187 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi121 running (5m) 26s ago 5m 414M 4096M 16.2.4 8d91d370c2b8 cdc8c669c4db 2024-02-15T18:36:53.187 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (65s) 9s ago 8m 52.7M - 2.18.1 de242295e225 67e33f9de4e6 2024-02-15T18:36:53.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.197+0000 7fcc01731700 1 -- 172.21.15.38:0/2421880489 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcbe4061000 msgr2=0x7fcbe40634c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:53.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.197+0000 7fcc01731700 1 --2- 172.21.15.38:0/2421880489 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcbe4061000 0x7fcbe40634c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fcbfc06b730 tx=0x7fcbec009c40).stop 2024-02-15T18:36:53.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.197+0000 7fcc01731700 1 -- 172.21.15.38:0/2421880489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbfc0f77d0 msgr2=0x7fcbfc069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:53.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.197+0000 7fcc01731700 1 --2- 172.21.15.38:0/2421880489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbfc0f77d0 0x7fcbfc069e20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fcbf0009f50 tx=0x7fcbf0009b30).stop 2024-02-15T18:36:53.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.197+0000 7fcc01731700 1 -- 172.21.15.38:0/2421880489 shutdown_connections 2024-02-15T18:36:53.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.197+0000 7fcc01731700 1 --2- 172.21.15.38:0/2421880489 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcbe4061000 0x7fcbe40634c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:53.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.197+0000 7fcc01731700 1 --2- 172.21.15.38:0/2421880489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0f6530 0x7fcbfc06b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:53.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.197+0000 7fcc01731700 1 --2- 172.21.15.38:0/2421880489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbfc0f77d0 0x7fcbfc069e20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:53.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.197+0000 7fcc01731700 1 -- 172.21.15.38:0/2421880489 >> 172.21.15.38:0/2421880489 conn(0x7fcbfc0f1c90 msgr2=0x7fcbfc0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:53.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.197+0000 7fcc01731700 1 -- 172.21.15.38:0/2421880489 shutdown_connections 2024-02-15T18:36:53.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.197+0000 7fcc01731700 1 -- 172.21.15.38:0/2421880489 wait complete. 2024-02-15T18:36:53.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.324+0000 7f4f340de700 1 -- 172.21.15.38:0/3285466639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f2c068250 msgr2=0x7f4f2c0686d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:53.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.324+0000 7f4f340de700 1 --2- 172.21.15.38:0/3285466639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f2c068250 0x7f4f2c0686d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f4f1c0046e0 tx=0x7f4f1c009b30).stop 2024-02-15T18:36:53.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.325+0000 7f4f340de700 1 -- 172.21.15.38:0/3285466639 shutdown_connections 2024-02-15T18:36:53.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.325+0000 7f4f340de700 1 --2- 172.21.15.38:0/3285466639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f2c068250 0x7f4f2c0686d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:53.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.325+0000 7f4f340de700 1 --2- 172.21.15.38:0/3285466639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f2c0696d0 0x7f4f2c067d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:36:53.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.325+0000 7f4f340de700 1 -- 172.21.15.38:0/3285466639 >> 172.21.15.38:0/3285466639 conn(0x7f4f2c0636c0 msgr2=0x7f4f2c065ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:53.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.325+0000 7f4f340de700 1 -- 172.21.15.38:0/3285466639 shutdown_connections 2024-02-15T18:36:53.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.326+0000 7f4f340de700 1 -- 172.21.15.38:0/3285466639 wait complete. 2024-02-15T18:36:53.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.326+0000 7f4f340de700 1 Processor -- start 2024-02-15T18:36:53.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.326+0000 7f4f340de700 1 -- start start 2024-02-15T18:36:53.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.327+0000 7f4f340de700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f2c068250 0x7f4f2c10d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:53.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.327+0000 7f4f340de700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f2c0696d0 0x7f4f2c10db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:53.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.327+0000 7f4f340de700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c10e1a0 con 0x7f4f2c068250 2024-02-15T18:36:53.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.327+0000 7f4f340de700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c10ea40 con 0x7f4f2c0696d0 2024-02-15T18:36:53.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.327+0000 7f4f31e7a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f2c068250 0x7f4f2c10d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:53.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.327+0000 7f4f31e7a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f2c068250 0x7f4f2c10d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48734/0 (socket says 172.21.15.38:48734) 2024-02-15T18:36:53.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.327+0000 7f4f31e7a700 1 -- 172.21.15.38:0/23556568 learned_addr learned my addr 172.21.15.38:0/23556568 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:36:53.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.327+0000 7f4f31e7a700 1 -- 172.21.15.38:0/23556568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f2c0696d0 msgr2=0x7f4f2c10db10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-15T18:36:53.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.327+0000 7f4f31e7a700 1 --2- 172.21.15.38:0/23556568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f2c0696d0 0x7f4f2c10db10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:36:53.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.327+0000 7f4f31e7a700 1 -- 172.21.15.38:0/23556568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c005040 con 0x7f4f2c068250 2024-02-15T18:36:53.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.328+0000 7f4f31e7a700 1 --2- 172.21.15.38:0/23556568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f2c068250 0x7f4f2c10d620 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f4f28009360 tx=0x7f4f28006e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:53.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.328+0000 7f4f227fc700 1 -- 172.21.15.38:0/23556568 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f28013460 con 0x7f4f2c068250 2024-02-15T18:36:53.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.328+0000 7f4f227fc700 1 -- 172.21.15.38:0/23556568 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f4f2800d8c0 con 0x7f4f2c068250 2024-02-15T18:36:53.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.328+0000 7f4f340de700 1 -- 172.21.15.38:0/23556568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2c10ed20 con 0x7f4f2c068250 2024-02-15T18:36:53.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.328+0000 7f4f227fc700 1 -- 172.21.15.38:0/23556568 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f28012920 con 0x7f4f2c068250 2024-02-15T18:36:53.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.328+0000 7f4f340de700 1 -- 172.21.15.38:0/23556568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f2c10f2e0 con 0x7f4f2c068250 2024-02-15T18:36:53.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.329+0000 7f4f340de700 1 -- 172.21.15.38:0/23556568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f2c0672e0 con 0x7f4f2c068250 2024-02-15T18:36:53.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.330+0000 7f4f227fc700 1 -- 172.21.15.38:0/23556568 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 30) v1 ==== 77246+0+0 (secure 0 0 0) 0x7f4f2800f070 con 0x7f4f2c068250 2024-02-15T18:36:53.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.331+0000 7f4f227fc700 1 --2- 172.21.15.38:0/23556568 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4f18061000 0x7f4f180634c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:53.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.331+0000 7f4f227fc700 1 -- 172.21.15.38:0/23556568 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f4f2807c200 con 0x7f4f2c068250 2024-02-15T18:36:53.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.332+0000 7f4f31679700 1 --2- 172.21.15.38:0/23556568 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4f18061000 0x7f4f180634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:53.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.338+0000 7f4f227fc700 1 -- 172.21.15.38:0/23556568 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4f2804c5b0 con 0x7f4f2c068250 2024-02-15T18:36:53.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.338+0000 7f4f31679700 1 --2- 172.21.15.38:0/23556568 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4f18061000 0x7f4f180634c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f4f1c000c00 tx=0x7f4f1c009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:53.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.670+0000 7f4f340de700 1 -- 172.21.15.38:0/23556568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4f2c045ad0 con 0x7f4f2c068250 2024-02-15T18:36:53.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.671+0000 7f4f227fc700 1 -- 172.21.15.38:0/23556568 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+712 (secure 0 0 0) 0x7f4f2804fbd0 con 0x7f4f2c068250 2024-02-15T18:36:53.661 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:36:53.661 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-02-15T18:36:53.661 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:36:53.661 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:36:53.661 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-02-15T18:36:53.661 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:36:53.661 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:36:53.662 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-02-15T18:36:53.662 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2024-02-15T18:36:53.662 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:36:53.662 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-02-15T18:36:53.662 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:36:53.662 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:36:53.662 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-02-15T18:36:53.662 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 4, 2024-02-15T18:36:53.662 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 10 2024-02-15T18:36:53.662 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-02-15T18:36:53.662 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:36:53.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.672+0000 7f4f340de700 1 -- 172.21.15.38:0/23556568 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4f18061000 msgr2=0x7f4f180634c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:53.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.672+0000 7f4f340de700 1 --2- 172.21.15.38:0/23556568 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4f18061000 0x7f4f180634c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f4f1c000c00 tx=0x7f4f1c009b30).stop 2024-02-15T18:36:53.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.672+0000 7f4f340de700 1 -- 172.21.15.38:0/23556568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f2c068250 msgr2=0x7f4f2c10d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:53.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.672+0000 7f4f340de700 1 --2- 172.21.15.38:0/23556568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f2c068250 0x7f4f2c10d620 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f4f28009360 tx=0x7f4f28006e10).stop 2024-02-15T18:36:53.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.673+0000 7f4f340de700 1 -- 172.21.15.38:0/23556568 shutdown_connections 2024-02-15T18:36:53.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.673+0000 7f4f340de700 1 --2- 172.21.15.38:0/23556568 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4f18061000 0x7f4f180634c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:53.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.673+0000 7f4f340de700 1 --2- 172.21.15.38:0/23556568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f2c068250 0x7f4f2c10d620 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:53.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.673+0000 7f4f340de700 1 --2- 172.21.15.38:0/23556568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f2c0696d0 0x7f4f2c10db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:36:53.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.673+0000 7f4f340de700 1 -- 172.21.15.38:0/23556568 >> 172.21.15.38:0/23556568 conn(0x7f4f2c0636c0 msgr2=0x7f4f2c101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:53.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.673+0000 7f4f340de700 1 -- 172.21.15.38:0/23556568 shutdown_connections 2024-02-15T18:36:53.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.673+0000 7f4f340de700 1 -- 172.21.15.38:0/23556568 wait complete. 2024-02-15T18:36:53.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.774+0000 7fe2db286700 1 -- 172.21.15.38:0/2931344954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2d40f66e0 msgr2=0x7fe2d40f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:53.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.774+0000 7fe2db286700 1 --2- 172.21.15.38:0/2931344954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2d40f66e0 0x7fe2d40f6b00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fe2d0004660 tx=0x7fe2d0009b30).stop 2024-02-15T18:36:53.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.774+0000 7fe2db286700 1 -- 172.21.15.38:0/2931344954 shutdown_connections 2024-02-15T18:36:53.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.774+0000 7fe2db286700 1 --2- 172.21.15.38:0/2931344954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2d40f7980 0x7fe2d40f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:53.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.774+0000 7fe2db286700 1 --2- 172.21.15.38:0/2931344954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2d40f66e0 0x7fe2d40f6b00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:53.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.774+0000 7fe2db286700 1 -- 172.21.15.38:0/2931344954 >> 172.21.15.38:0/2931344954 conn(0x7fe2d40f1e40 msgr2=0x7fe2d40f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:53.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.774+0000 7fe2db286700 1 -- 172.21.15.38:0/2931344954 shutdown_connections 2024-02-15T18:36:53.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.774+0000 7fe2db286700 1 -- 172.21.15.38:0/2931344954 wait complete. 2024-02-15T18:36:53.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.775+0000 7fe2db286700 1 Processor -- start 2024-02-15T18:36:53.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.775+0000 7fe2db286700 1 -- start start 2024-02-15T18:36:53.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.775+0000 7fe2db286700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2d40f66e0 0x7fe2d4106ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:53.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.775+0000 7fe2db286700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2d40f7980 0x7fe2d41074e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:53.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.775+0000 7fe2db286700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d4107a90 con 0x7fe2d40f66e0 2024-02-15T18:36:53.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.775+0000 7fe2db286700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d4107bd0 con 0x7fe2d40f7980 2024-02-15T18:36:53.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.775+0000 7fe2d9022700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2d40f66e0 0x7fe2d4106ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:53.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.775+0000 7fe2d9022700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2d40f66e0 0x7fe2d4106ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42366/0 (socket says 172.21.15.38:42366) 2024-02-15T18:36:53.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.775+0000 7fe2d9022700 1 -- 172.21.15.38:0/1977283137 learned_addr learned my addr 172.21.15.38:0/1977283137 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:36:53.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.775+0000 7fe2d8821700 1 --2- 172.21.15.38:0/1977283137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2d40f7980 0x7fe2d41074e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:53.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.776+0000 7fe2d8821700 1 -- 172.21.15.38:0/1977283137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2d40f7980 msgr2=0x7fe2d41074e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-15T18:36:53.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.776+0000 7fe2d8821700 1 -- 172.21.15.38:0/1977283137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2d40f7980 msgr2=0x7fe2d41074e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-15T18:36:53.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.776+0000 7fe2d8821700 1 --2- 172.21.15.38:0/1977283137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2d40f7980 0x7fe2d41074e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-15T18:36:53.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.776+0000 7fe2d8821700 1 --2- 172.21.15.38:0/1977283137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2d40f7980 0x7fe2d41074e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-02-15T18:36:53.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.776+0000 7fe2d9022700 1 -- 172.21.15.38:0/1977283137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2d40f7980 msgr2=0x7fe2d41074e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-15T18:36:53.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.776+0000 7fe2d9022700 1 --2- 172.21.15.38:0/1977283137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2d40f7980 0x7fe2d41074e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:53.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.776+0000 7fe2d9022700 1 -- 172.21.15.38:0/1977283137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d0005040 con 0x7fe2d40f66e0 2024-02-15T18:36:53.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.776+0000 7fe2d9022700 1 --2- 172.21.15.38:0/1977283137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2d40f66e0 0x7fe2d4106ff0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fe2d00043d0 tx=0x7fe2d0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:53.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.776+0000 7fe2c9ffb700 1 -- 172.21.15.38:0/1977283137 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2d0018070 con 0x7fe2d40f66e0 2024-02-15T18:36:53.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.776+0000 7fe2c9ffb700 1 -- 172.21.15.38:0/1977283137 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fe2d0007c70 con 0x7fe2d40f66e0 2024-02-15T18:36:53.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.776+0000 7fe2db286700 1 -- 172.21.15.38:0/1977283137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d4108510 con 0x7fe2d40f66e0 2024-02-15T18:36:53.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.776+0000 7fe2c9ffb700 1 -- 172.21.15.38:0/1977283137 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2d001c620 con 0x7fe2d40f66e0 2024-02-15T18:36:53.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.776+0000 7fe2db286700 1 -- 172.21.15.38:0/1977283137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d4108a70 con 0x7fe2d40f66e0 2024-02-15T18:36:53.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.777+0000 7fe2db286700 1 -- 172.21.15.38:0/1977283137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d4045ad0 con 0x7fe2d40f66e0 2024-02-15T18:36:53.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.778+0000 7fe2c9ffb700 1 -- 172.21.15.38:0/1977283137 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 30) v1 ==== 77246+0+0 (secure 0 0 0) 0x7fe2d001d420 con 0x7fe2d40f66e0 2024-02-15T18:36:53.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.778+0000 7fe2c9ffb700 1 --2- 172.21.15.38:0/1977283137 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe2c0060fb0 0x7fe2c0063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:53.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.779+0000 7fe2c9ffb700 1 -- 172.21.15.38:0/1977283137 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fe2d007deb0 con 0x7fe2d40f66e0 2024-02-15T18:36:53.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.779+0000 7fe2d8821700 1 --2- 172.21.15.38:0/1977283137 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe2c0060fb0 0x7fe2c0063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:53.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.779+0000 7fe2d8821700 1 --2- 172.21.15.38:0/1977283137 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe2c0060fb0 0x7fe2c0063470 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fe2c4009fd0 tx=0x7fe2c4009c70).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:53.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:50.498446+0000 mon.smithi038 (mon.0) 14 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-02-15T18:36:53.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:50.500740+0000 mon.smithi038 (mon.0) 15 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:36:53.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:53.783+0000 7fe2c9ffb700 1 -- 172.21.15.38:0/1977283137 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe2d0051d50 con 0x7fe2d40f66e0 2024-02-15T18:36:53.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:50.498446+0000 mon.smithi038 (mon.0) 14 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi038"}]: dispatch 2024-02-15T18:36:53.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:50.500740+0000 mon.smithi038 (mon.0) 15 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mon metadata", "id": "smithi121"}]: dispatch 2024-02-15T18:36:53.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.006+0000 7fe2db286700 1 -- 172.21.15.38:0/1977283137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe2d40ff110 con 0x7fe2d40f66e0 2024-02-15T18:36:53.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.006+0000 7fe2c9ffb700 1 -- 172.21.15.38:0/1977283137 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1807 (secure 0 0 0) 0x7fe2d0007de0 con 0x7fe2d40f66e0 2024-02-15T18:36:53.996 INFO:teuthology.orchestra.run.smithi038.stdout:e13 2024-02-15T18:36:53.996 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:36:53.996 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:36:53.996 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-02-15T18:36:53.997 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:36:53.997 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:36:53.997 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-02-15T18:36:53.997 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 13 2024-02-15T18:36:53.997 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 2024-02-15T18:36:53.997 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:36:53.997 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-02-15T18:35:07.485909+0000 2024-02-15T18:36:53.997 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-02-15T18:36:53.997 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-02-15T18:36:53.997 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-02-15T18:36:53.997 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-02-15T18:36:53.997 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-02-15T18:36:53.997 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-02-15T18:36:53.998 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-02-15T18:36:53.998 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-02-15T18:36:53.998 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:36:53.998 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-02-15T18:36:53.998 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-02-15T18:36:53.998 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=24261,1=24275} 2024-02-15T18:36:53.998 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-02-15T18:36:53.998 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-02-15T18:36:53.998 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-02-15T18:36:53.998 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-02-15T18:36:53.998 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-02-15T18:36:53.998 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-02-15T18:36:53.998 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-02-15T18:36:53.998 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-02-15T18:36:53.999 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.mvumlh{0:24261} state up:active seq 3 export targets 1 join_fscid=1 addr [v2:172.21.15.121:6824/1422707571,v1:172.21.15.121:6825/1422707571] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:36:53.999 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.ovuzki{1:24275} state up:active seq 2 export targets 0 join_fscid=1 addr [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:36:53.999 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:36:53.999 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:36:53.999 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-02-15T18:36:53.999 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:36:53.999 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.hxoggl{-1:14446} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.38:6826/3822880235,v1:172.21.15.38:6827/3822880235] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:36:53.999 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.qgkzgn{-1:14456} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.38:6828/703030231,v1:172.21.15.38:6829/703030231] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:36:54.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.008+0000 7fe2db286700 1 -- 172.21.15.38:0/1977283137 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe2c0060fb0 msgr2=0x7fe2c0063470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:54.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.008+0000 7fe2db286700 1 --2- 172.21.15.38:0/1977283137 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe2c0060fb0 0x7fe2c0063470 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fe2c4009fd0 tx=0x7fe2c4009c70).stop 2024-02-15T18:36:54.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.008+0000 7fe2db286700 1 -- 172.21.15.38:0/1977283137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2d40f66e0 msgr2=0x7fe2d4106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:54.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.008+0000 7fe2db286700 1 --2- 172.21.15.38:0/1977283137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2d40f66e0 0x7fe2d4106ff0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fe2d00043d0 tx=0x7fe2d0009b30).stop 2024-02-15T18:36:54.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.008+0000 7fe2db286700 1 -- 172.21.15.38:0/1977283137 shutdown_connections 2024-02-15T18:36:54.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.008+0000 7fe2db286700 1 --2- 172.21.15.38:0/1977283137 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe2c0060fb0 0x7fe2c0063470 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:54.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.008+0000 7fe2db286700 1 --2- 172.21.15.38:0/1977283137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2d40f66e0 0x7fe2d4106ff0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:54.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.008+0000 7fe2db286700 1 --2- 172.21.15.38:0/1977283137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2d40f7980 0x7fe2d41074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:54.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.008+0000 7fe2db286700 1 -- 172.21.15.38:0/1977283137 >> 172.21.15.38:0/1977283137 conn(0x7fe2d40f1e40 msgr2=0x7fe2d40fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:54.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.008+0000 7fe2db286700 1 -- 172.21.15.38:0/1977283137 shutdown_connections 2024-02-15T18:36:54.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.008+0000 7fe2db286700 1 -- 172.21.15.38:0/1977283137 wait complete. 2024-02-15T18:36:54.001 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 13 2024-02-15T18:36:54.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.111+0000 7f9cdd18d700 1 -- 172.21.15.38:0/800839414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cd80f6530 msgr2=0x7f9cd80f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:54.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.111+0000 7f9cdd18d700 1 --2- 172.21.15.38:0/800839414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cd80f6530 0x7f9cd80f6950 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f9cc8004660 tx=0x7f9cc8009b30).stop 2024-02-15T18:36:54.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.111+0000 7f9cdd18d700 1 -- 172.21.15.38:0/800839414 shutdown_connections 2024-02-15T18:36:54.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.111+0000 7f9cdd18d700 1 --2- 172.21.15.38:0/800839414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd80f77d0 0x7f9cd80f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:54.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.111+0000 7f9cdd18d700 1 --2- 172.21.15.38:0/800839414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cd80f6530 0x7f9cd80f6950 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:54.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.111+0000 7f9cdd18d700 1 -- 172.21.15.38:0/800839414 >> 172.21.15.38:0/800839414 conn(0x7f9cd80f1c90 msgr2=0x7f9cd80f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:54.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.111+0000 7f9cdd18d700 1 -- 172.21.15.38:0/800839414 shutdown_connections 2024-02-15T18:36:54.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.111+0000 7f9cdd18d700 1 -- 172.21.15.38:0/800839414 wait complete. 2024-02-15T18:36:54.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.112+0000 7f9cdd18d700 1 Processor -- start 2024-02-15T18:36:54.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.112+0000 7f9cdd18d700 1 -- start start 2024-02-15T18:36:54.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.112+0000 7f9cdd18d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cd80f6530 0x7f9cd806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:54.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.112+0000 7f9cdd18d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd80f77d0 0x7f9cd8069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:54.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.112+0000 7f9cdd18d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd806a380 con 0x7f9cd80f6530 2024-02-15T18:36:54.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.112+0000 7f9cdd18d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd806a4c0 con 0x7f9cd80f77d0 2024-02-15T18:36:54.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.112+0000 7f9cd7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cd80f6530 0x7f9cd806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:54.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.112+0000 7f9cd7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cd80f6530 0x7f9cd806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42396/0 (socket says 172.21.15.38:42396) 2024-02-15T18:36:54.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.112+0000 7f9cd7fff700 1 -- 172.21.15.38:0/3525102370 learned_addr learned my addr 172.21.15.38:0/3525102370 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:36:54.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.113+0000 7f9cd77fe700 1 --2- 172.21.15.38:0/3525102370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd80f77d0 0x7f9cd8069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:54.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.113+0000 7f9cd77fe700 1 -- 172.21.15.38:0/3525102370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd80f77d0 msgr2=0x7f9cd8069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-15T18:36:54.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.113+0000 7f9cd77fe700 1 -- 172.21.15.38:0/3525102370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd80f77d0 msgr2=0x7f9cd8069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-15T18:36:54.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.113+0000 7f9cd77fe700 1 --2- 172.21.15.38:0/3525102370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd80f77d0 0x7f9cd8069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-15T18:36:54.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.113+0000 7f9cd77fe700 1 --2- 172.21.15.38:0/3525102370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd80f77d0 0x7f9cd8069e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-02-15T18:36:54.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.113+0000 7f9cd7fff700 1 -- 172.21.15.38:0/3525102370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd80f77d0 msgr2=0x7f9cd8069e20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-15T18:36:54.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.113+0000 7f9cd7fff700 1 --2- 172.21.15.38:0/3525102370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd80f77d0 0x7f9cd8069e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:54.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.113+0000 7f9cd7fff700 1 -- 172.21.15.38:0/3525102370 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc8005040 con 0x7f9cd80f6530 2024-02-15T18:36:54.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.113+0000 7f9cd7fff700 1 --2- 172.21.15.38:0/3525102370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cd80f6530 0x7f9cd806b9a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f9cc80043d0 tx=0x7f9cc8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:54.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.113+0000 7f9cd4ff9700 1 -- 172.21.15.38:0/3525102370 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cc8018070 con 0x7f9cd80f6530 2024-02-15T18:36:54.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.113+0000 7f9cd4ff9700 1 -- 172.21.15.38:0/3525102370 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f9cc8007c70 con 0x7f9cd80f6530 2024-02-15T18:36:54.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.113+0000 7f9cdd18d700 1 -- 172.21.15.38:0/3525102370 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd806a740 con 0x7f9cd80f6530 2024-02-15T18:36:54.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.113+0000 7f9cd4ff9700 1 -- 172.21.15.38:0/3525102370 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cc801c620 con 0x7f9cd80f6530 2024-02-15T18:36:54.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.113+0000 7f9cdd18d700 1 -- 172.21.15.38:0/3525102370 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd806aca0 con 0x7f9cd80f6530 2024-02-15T18:36:54.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.114+0000 7f9cdd18d700 1 -- 172.21.15.38:0/3525102370 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd8045ad0 con 0x7f9cd80f6530 2024-02-15T18:36:54.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.115+0000 7f9cd4ff9700 1 -- 172.21.15.38:0/3525102370 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 30) v1 ==== 77246+0+0 (secure 0 0 0) 0x7f9cc801d420 con 0x7f9cd80f6530 2024-02-15T18:36:54.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.115+0000 7f9cd4ff9700 1 --2- 172.21.15.38:0/3525102370 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9cc0061000 0x7f9cc00634c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:54.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.115+0000 7f9cd4ff9700 1 -- 172.21.15.38:0/3525102370 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f9cc807deb0 con 0x7f9cd80f6530 2024-02-15T18:36:54.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.115+0000 7f9cd77fe700 1 --2- 172.21.15.38:0/3525102370 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9cc0061000 0x7f9cc00634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:54.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.116+0000 7f9cd77fe700 1 --2- 172.21.15.38:0/3525102370 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9cc0061000 0x7f9cc00634c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f9ccc0099e0 tx=0x7f9ccc009d10).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:54.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.120+0000 7f9cd4ff9700 1 -- 172.21.15.38:0/3525102370 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9cc804e320 con 0x7f9cd80f6530 2024-02-15T18:36:54.191 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-02-15T18:36:54.191 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-02-15T18:36:54.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.358+0000 7f9cdd18d700 1 -- 172.21.15.38:0/3525102370 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9cd8057f10 con 0x7f9cc0061000 2024-02-15T18:36:54.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.359+0000 7f9cd4ff9700 1 -- 172.21.15.38:0/3525102370 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f9cd8057f10 con 0x7f9cc0061000 2024-02-15T18:36:54.349 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:36:54.350 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:36:54.350 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-02-15T18:36:54.350 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:36:54.350 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-02-15T18:36:54.350 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr" 2024-02-15T18:36:54.350 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-02-15T18:36:54.350 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "3/21 daemons upgraded", 2024-02-15T18:36:54.350 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading mon daemons", 2024-02-15T18:36:54.350 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-02-15T18:36:54.350 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:36:54.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.362+0000 7f9cdd18d700 1 -- 172.21.15.38:0/3525102370 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9cc0061000 msgr2=0x7f9cc00634c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:54.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.362+0000 7f9cdd18d700 1 --2- 172.21.15.38:0/3525102370 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9cc0061000 0x7f9cc00634c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f9ccc0099e0 tx=0x7f9ccc009d10).stop 2024-02-15T18:36:54.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.362+0000 7f9cdd18d700 1 -- 172.21.15.38:0/3525102370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cd80f6530 msgr2=0x7f9cd806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:54.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.362+0000 7f9cdd18d700 1 --2- 172.21.15.38:0/3525102370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cd80f6530 0x7f9cd806b9a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f9cc80043d0 tx=0x7f9cc8009b30).stop 2024-02-15T18:36:54.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.363+0000 7f9cdd18d700 1 -- 172.21.15.38:0/3525102370 shutdown_connections 2024-02-15T18:36:54.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.363+0000 7f9cdd18d700 1 --2- 172.21.15.38:0/3525102370 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9cc0061000 0x7f9cc00634c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:54.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.363+0000 7f9cdd18d700 1 --2- 172.21.15.38:0/3525102370 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cd80f6530 0x7f9cd806b9a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:54.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.363+0000 7f9cdd18d700 1 --2- 172.21.15.38:0/3525102370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cd80f77d0 0x7f9cd8069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:54.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.363+0000 7f9cdd18d700 1 -- 172.21.15.38:0/3525102370 >> 172.21.15.38:0/3525102370 conn(0x7f9cd80f1c90 msgr2=0x7f9cd80fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:54.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.363+0000 7f9cdd18d700 1 -- 172.21.15.38:0/3525102370 shutdown_connections 2024-02-15T18:36:54.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:36:54.363+0000 7f9cdd18d700 1 -- 172.21.15.38:0/3525102370 wait complete. 2024-02-15T18:36:54.580 DEBUG:teuthology.parallel:result is None 2024-02-15T18:36:54.581 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-15T18:36:54.630 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-15T18:36:54.631 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-02-15T18:36:54.698 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-02-15T18:36:54.698 DEBUG:teuthology.parallel:result is None 2024-02-15T18:36:54.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:36:50.932588+0000 mgr.smithi038.khesbl (mgr.14664) 40 : cluster [DBG] pgmap v28: 65 pgs: 65 active+clean; 297 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 37 KiB/s rd, 1.5 MiB/s wr, 364 op/s 2024-02-15T18:36:54.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:52.559007+0000 mgr.smithi038.khesbl (mgr.14664) 41 : audit [DBG] from='client.34100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:36:54.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:52.885821+0000 mgr.smithi038.khesbl (mgr.14664) 42 : audit [DBG] from='client.34104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:36:54.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:36:52.932927+0000 mgr.smithi038.khesbl (mgr.14664) 43 : cluster [DBG] pgmap v29: 65 pgs: 65 active+clean; 297 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 995 KiB/s wr, 243 op/s 2024-02-15T18:36:54.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:53.187109+0000 mgr.smithi038.khesbl (mgr.14664) 44 : audit [DBG] from='client.34108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:36:54.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:36:53.555599+0000 mon.smithi121 (mon.1) 1 : cluster [INF] mon.smithi121 calling monitor election 2024-02-15T18:36:54.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:36:53.557107+0000 mon.smithi038 (mon.0) 16 : cluster [INF] mon.smithi038 calling monitor election 2024-02-15T18:36:54.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:36:53.572598+0000 mon.smithi038 (mon.0) 17 : cluster [INF] mon.smithi038 is new leader, mons smithi038,smithi121 in quorum (ranks 0,1) 2024-02-15T18:36:54.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:36:53.582246+0000 mon.smithi038 (mon.0) 18 : cluster [DBG] monmap e2: 2 mons at {smithi038=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0],smithi121=[v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0]} removed_ranks: {} 2024-02-15T18:36:54.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:36:53.582950+0000 mon.smithi038 (mon.0) 19 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:active} 2 up:standby 2024-02-15T18:36:54.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:36:53.582990+0000 mon.smithi038 (mon.0) 20 : cluster [DBG] osdmap e46: 6 total, 6 up, 6 in 2024-02-15T18:36:54.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:36:53.583355+0000 mon.smithi038 (mon.0) 21 : cluster [DBG] mgrmap e30: smithi038.khesbl(active, since 51s), standbys: smithi121.ckiryi 2024-02-15T18:36:54.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:36:53.588357+0000 mon.smithi038 (mon.0) 22 : cluster [INF] overall HEALTH_OK 2024-02-15T18:36:54.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:53.593098+0000 mon.smithi038 (mon.0) 23 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:54.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:53.594001+0000 mon.smithi038 (mon.0) 24 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:54.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:53.671353+0000 mon.smithi038 (mon.0) 25 : audit [DBG] from='client.? 172.21.15.38:0/23556568' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:36:54.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:54.006787+0000 mon.smithi038 (mon.0) 26 : audit [DBG] from='client.? 172.21.15.38:0/1977283137' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:36:55.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:50.932588+0000 mgr.smithi038.khesbl (mgr.14664) 40 : cluster [DBG] pgmap v28: 65 pgs: 65 active+clean; 297 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 37 KiB/s rd, 1.5 MiB/s wr, 364 op/s 2024-02-15T18:36:55.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:52.559007+0000 mgr.smithi038.khesbl (mgr.14664) 41 : audit [DBG] from='client.34100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:36:55.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:52.885821+0000 mgr.smithi038.khesbl (mgr.14664) 42 : audit [DBG] from='client.34104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:36:55.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:52.932927+0000 mgr.smithi038.khesbl (mgr.14664) 43 : cluster [DBG] pgmap v29: 65 pgs: 65 active+clean; 297 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 995 KiB/s wr, 243 op/s 2024-02-15T18:36:55.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:53.187109+0000 mgr.smithi038.khesbl (mgr.14664) 44 : audit [DBG] from='client.34108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:36:55.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:53.555599+0000 mon.smithi121 (mon.1) 1 : cluster [INF] mon.smithi121 calling monitor election 2024-02-15T18:36:55.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:53.557107+0000 mon.smithi038 (mon.0) 16 : cluster [INF] mon.smithi038 calling monitor election 2024-02-15T18:36:55.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:53.572598+0000 mon.smithi038 (mon.0) 17 : cluster [INF] mon.smithi038 is new leader, mons smithi038,smithi121 in quorum (ranks 0,1) 2024-02-15T18:36:55.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:53.582246+0000 mon.smithi038 (mon.0) 18 : cluster [DBG] monmap e2: 2 mons at {smithi038=[v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0],smithi121=[v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0]} removed_ranks: {} 2024-02-15T18:36:55.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:53.582950+0000 mon.smithi038 (mon.0) 19 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:active} 2 up:standby 2024-02-15T18:36:55.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:53.582990+0000 mon.smithi038 (mon.0) 20 : cluster [DBG] osdmap e46: 6 total, 6 up, 6 in 2024-02-15T18:36:55.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:53.583355+0000 mon.smithi038 (mon.0) 21 : cluster [DBG] mgrmap e30: smithi038.khesbl(active, since 51s), standbys: smithi121.ckiryi 2024-02-15T18:36:55.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:53.588357+0000 mon.smithi038 (mon.0) 22 : cluster [INF] overall HEALTH_OK 2024-02-15T18:36:55.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:53.593098+0000 mon.smithi038 (mon.0) 23 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:55.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:53.594001+0000 mon.smithi038 (mon.0) 24 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:55.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:53.671353+0000 mon.smithi038 (mon.0) 25 : audit [DBG] from='client.? 172.21.15.38:0/23556568' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:36:55.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:54.006787+0000 mon.smithi038 (mon.0) 26 : audit [DBG] from='client.? 172.21.15.38:0/1977283137' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:36:55.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:54.359756+0000 mgr.smithi038.khesbl (mgr.14664) 45 : audit [DBG] from='client.34120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:36:55.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:36:54.933699+0000 mgr.smithi038.khesbl (mgr.14664) 46 : cluster [DBG] pgmap v30: 65 pgs: 65 active+clean; 295 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 37 KiB/s rd, 1.4 MiB/s wr, 355 op/s 2024-02-15T18:36:56.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:54.359756+0000 mgr.smithi038.khesbl (mgr.14664) 45 : audit [DBG] from='client.34120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:36:56.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:54.933699+0000 mgr.smithi038.khesbl (mgr.14664) 46 : cluster [DBG] pgmap v30: 65 pgs: 65 active+clean; 295 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 37 KiB/s rd, 1.4 MiB/s wr, 355 op/s 2024-02-15T18:36:57.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:56.934166+0000 mgr.smithi038.khesbl (mgr.14664) 47 : cluster [DBG] pgmap v31: 65 pgs: 65 active+clean; 295 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 963 KiB/s wr, 226 op/s 2024-02-15T18:36:57.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:36:56.934166+0000 mgr.smithi038.khesbl (mgr.14664) 47 : cluster [DBG] pgmap v31: 65 pgs: 65 active+clean; 295 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 963 KiB/s wr, 226 op/s 2024-02-15T18:36:58.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:57.336324+0000 mon.smithi038 (mon.0) 27 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:58.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:57.337301+0000 mon.smithi038 (mon.0) 28 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:36:58.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:57.338272+0000 mon.smithi038 (mon.0) 29 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:36:58.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:36:57.339108+0000 mgr.smithi038.khesbl (mgr.14664) 48 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:36:58.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:57.342932+0000 mon.smithi038 (mon.0) 30 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:58.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:57.349038+0000 mon.smithi038 (mon.0) 31 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:58.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:57.351681+0000 mon.smithi038 (mon.0) 32 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:36:58.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:57.353142+0000 mon.smithi038 (mon.0) 33 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:36:58.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:36:57.353838+0000 mgr.smithi038.khesbl (mgr.14664) 49 : cephadm [INF] Upgrade: Setting container_image for all mon 2024-02-15T18:36:58.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:57.358928+0000 mon.smithi038 (mon.0) 34 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:58.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:57.359925+0000 mon.smithi038 (mon.0) 35 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi038"}]: dispatch 2024-02-15T18:36:58.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:57.363870+0000 mon.smithi038 (mon.0) 36 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi038"}]': finished 2024-02-15T18:36:58.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:57.364650+0000 mon.smithi038 (mon.0) 37 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi121"}]: dispatch 2024-02-15T18:36:58.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:57.369661+0000 mon.smithi038 (mon.0) 38 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi121"}]': finished 2024-02-15T18:36:58.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:57.336324+0000 mon.smithi038 (mon.0) 27 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:58.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:57.337301+0000 mon.smithi038 (mon.0) 28 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:36:58.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:57.338272+0000 mon.smithi038 (mon.0) 29 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:36:58.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:36:57.339108+0000 mgr.smithi038.khesbl (mgr.14664) 48 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:36:58.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:57.342932+0000 mon.smithi038 (mon.0) 30 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:58.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:57.349038+0000 mon.smithi038 (mon.0) 31 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:58.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:57.351681+0000 mon.smithi038 (mon.0) 32 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:36:58.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:57.353142+0000 mon.smithi038 (mon.0) 33 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:36:58.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:36:57.353838+0000 mgr.smithi038.khesbl (mgr.14664) 49 : cephadm [INF] Upgrade: Setting container_image for all mon 2024-02-15T18:36:58.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:57.358928+0000 mon.smithi038 (mon.0) 34 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:58.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:57.359925+0000 mon.smithi038 (mon.0) 35 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi038"}]: dispatch 2024-02-15T18:36:58.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:57.363870+0000 mon.smithi038 (mon.0) 36 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi038"}]': finished 2024-02-15T18:36:58.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:57.364650+0000 mon.smithi038 (mon.0) 37 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi121"}]: dispatch 2024-02-15T18:36:58.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:57.369661+0000 mon.smithi038 (mon.0) 38 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi121"}]': finished 2024-02-15T18:36:59.583 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:36:58.934524+0000 mgr.smithi038.khesbl (mgr.14664) 50 : cluster [DBG] pgmap v32: 65 pgs: 65 active+clean; 295 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 963 KiB/s wr, 226 op/s 2024-02-15T18:36:59.583 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:59.117554+0000 mon.smithi038 (mon.0) 39 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:59.583 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:59.118352+0000 mon.smithi038 (mon.0) 40 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi038", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-15T18:36:59.583 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:36:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:36:59.119053+0000 mon.smithi038 (mon.0) 41 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:36:59.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:36:58.934524+0000 mgr.smithi038.khesbl (mgr.14664) 50 : cluster [DBG] pgmap v32: 65 pgs: 65 active+clean; 295 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 963 KiB/s wr, 226 op/s 2024-02-15T18:36:59.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:59.117554+0000 mon.smithi038 (mon.0) 39 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:36:59.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:59.118352+0000 mon.smithi038 (mon.0) 40 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi038", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-15T18:36:59.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:36:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:36:59.119053+0000 mon.smithi038 (mon.0) 41 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:37:00.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:36:59.112871+0000 mgr.smithi038.khesbl (mgr.14664) 51 : cephadm [INF] Upgrade: Updating crash.smithi038 (1/2) 2024-02-15T18:37:00.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:36:59.119640+0000 mgr.smithi038.khesbl (mgr.14664) 52 : cephadm [INF] Deploying daemon crash.smithi038 on smithi038 2024-02-15T18:37:00.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:36:59.112871+0000 mgr.smithi038.khesbl (mgr.14664) 51 : cephadm [INF] Upgrade: Updating crash.smithi038 (1/2) 2024-02-15T18:37:00.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:36:59.119640+0000 mgr.smithi038.khesbl (mgr.14664) 52 : cephadm [INF] Deploying daemon crash.smithi038 on smithi038 2024-02-15T18:37:01.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:00.935016+0000 mgr.smithi038.khesbl (mgr.14664) 53 : cluster [DBG] pgmap v33: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1013 KiB/s wr, 258 op/s 2024-02-15T18:37:01.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:00.935016+0000 mgr.smithi038.khesbl (mgr.14664) 53 : cluster [DBG] pgmap v33: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1013 KiB/s wr, 258 op/s 2024-02-15T18:37:04.055 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:02.862048+0000 mon.smithi038 (mon.0) 42 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:04.055 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:02.935528+0000 mgr.smithi038.khesbl (mgr.14664) 54 : cluster [DBG] pgmap v34: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 516 KiB/s wr, 144 op/s 2024-02-15T18:37:04.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:02.862048+0000 mon.smithi038 (mon.0) 42 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:04.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:02.935528+0000 mgr.smithi038.khesbl (mgr.14664) 54 : cluster [DBG] pgmap v34: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 516 KiB/s wr, 144 op/s 2024-02-15T18:37:05.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:37:04.391727+0000 mgr.smithi038.khesbl (mgr.14664) 55 : cephadm [INF] Upgrade: Updating crash.smithi121 (2/2) 2024-02-15T18:37:05.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:04.401134+0000 mon.smithi038 (mon.0) 43 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:05.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:04.402102+0000 mon.smithi038 (mon.0) 44 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi121", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-15T18:37:05.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:04.402958+0000 mon.smithi038 (mon.0) 45 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:37:05.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:37:04.403714+0000 mgr.smithi038.khesbl (mgr.14664) 56 : cephadm [INF] Deploying daemon crash.smithi121 on smithi121 2024-02-15T18:37:05.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:04.936078+0000 mgr.smithi038.khesbl (mgr.14664) 57 : cluster [DBG] pgmap v35: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 517 KiB/s wr, 144 op/s 2024-02-15T18:37:05.853 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:37:04.391727+0000 mgr.smithi038.khesbl (mgr.14664) 55 : cephadm [INF] Upgrade: Updating crash.smithi121 (2/2) 2024-02-15T18:37:05.853 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:04.401134+0000 mon.smithi038 (mon.0) 43 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:05.853 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:04.402102+0000 mon.smithi038 (mon.0) 44 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi121", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-15T18:37:05.853 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:04.402958+0000 mon.smithi038 (mon.0) 45 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:37:05.853 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:37:04.403714+0000 mgr.smithi038.khesbl (mgr.14664) 56 : cephadm [INF] Deploying daemon crash.smithi121 on smithi121 2024-02-15T18:37:05.853 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:04.936078+0000 mgr.smithi038.khesbl (mgr.14664) 57 : cluster [DBG] pgmap v35: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 517 KiB/s wr, 144 op/s 2024-02-15T18:37:07.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:06.936645+0000 mgr.smithi038.khesbl (mgr.14664) 58 : cluster [DBG] pgmap v36: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 3.7 KiB/s rd, 50 KiB/s wr, 32 op/s 2024-02-15T18:37:07.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:06.936645+0000 mgr.smithi038.khesbl (mgr.14664) 58 : cluster [DBG] pgmap v36: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 3.7 KiB/s rd, 50 KiB/s wr, 32 op/s 2024-02-15T18:37:09.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:07.929459+0000 mon.smithi038 (mon.0) 46 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:09.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:07.930425+0000 mon.smithi038 (mon.0) 47 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:37:09.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:07.929459+0000 mon.smithi038 (mon.0) 46 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:09.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:07.930425+0000 mon.smithi038 (mon.0) 47 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:37:10.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:08.937007+0000 mgr.smithi038.khesbl (mgr.14664) 59 : cluster [DBG] pgmap v37: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 3.7 KiB/s rd, 50 KiB/s wr, 32 op/s 2024-02-15T18:37:10.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:08.937007+0000 mgr.smithi038.khesbl (mgr.14664) 59 : cluster [DBG] pgmap v37: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 3.7 KiB/s rd, 50 KiB/s wr, 32 op/s 2024-02-15T18:37:11.261 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:10.937705+0000 mgr.smithi038.khesbl (mgr.14664) 60 : cluster [DBG] pgmap v38: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 3.7 KiB/s rd, 50 KiB/s wr, 32 op/s 2024-02-15T18:37:11.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:10.937705+0000 mgr.smithi038.khesbl (mgr.14664) 60 : cluster [DBG] pgmap v38: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 3.7 KiB/s rd, 50 KiB/s wr, 32 op/s 2024-02-15T18:37:12.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:11.340345+0000 mon.smithi038 (mon.0) 48 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:12.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:11.340345+0000 mon.smithi038 (mon.0) 48 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:13.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:12.937981+0000 mgr.smithi038.khesbl (mgr.14664) 61 : cluster [DBG] pgmap v39: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-15T18:37:13.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:12.937981+0000 mgr.smithi038.khesbl (mgr.14664) 61 : cluster [DBG] pgmap v39: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-15T18:37:15.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:14.938482+0000 mgr.smithi038.khesbl (mgr.14664) 62 : cluster [DBG] pgmap v40: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-15T18:37:15.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:14.938482+0000 mgr.smithi038.khesbl (mgr.14664) 62 : cluster [DBG] pgmap v40: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-15T18:37:16.717 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:15.498525+0000 mon.smithi038 (mon.0) 49 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:16.718 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:15.798982+0000 mon.smithi038 (mon.0) 50 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:37:16.718 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:15.799678+0000 mon.smithi038 (mon.0) 51 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:37:16.718 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:37:15.800237+0000 mgr.smithi038.khesbl (mgr.14664) 63 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:37:16.718 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:15.807955+0000 mon.smithi038 (mon.0) 52 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:16.718 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:15.812609+0000 mon.smithi038 (mon.0) 53 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:37:16.718 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:15.814888+0000 mon.smithi038 (mon.0) 54 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:37:16.718 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:15.816189+0000 mon.smithi038 (mon.0) 55 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:37:16.718 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:15.817423+0000 mon.smithi038 (mon.0) 56 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:37:16.718 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:37:15.817981+0000 mgr.smithi038.khesbl (mgr.14664) 64 : cephadm [INF] Upgrade: Setting container_image for all crash 2024-02-15T18:37:16.718 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:15.821470+0000 mon.smithi038 (mon.0) 57 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:16.718 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:15.822146+0000 mon.smithi038 (mon.0) 58 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi038"}]: dispatch 2024-02-15T18:37:16.718 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:15.829167+0000 mon.smithi038 (mon.0) 59 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi038"}]': finished 2024-02-15T18:37:16.719 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:15.829777+0000 mon.smithi038 (mon.0) 60 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi121"}]: dispatch 2024-02-15T18:37:16.719 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:15.834744+0000 mon.smithi038 (mon.0) 61 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi121"}]': finished 2024-02-15T18:37:16.719 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:15.835759+0000 mon.smithi038 (mon.0) 62 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-15T18:37:16.719 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:15.835892+0000 mgr.smithi038.khesbl (mgr.14664) 65 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-15T18:37:16.719 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:37:15.836544+0000 mgr.smithi038.khesbl (mgr.14664) 66 : cephadm [INF] Upgrade: osd.0 is safe to restart 2024-02-15T18:37:16.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:15.498525+0000 mon.smithi038 (mon.0) 49 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:16.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:15.798982+0000 mon.smithi038 (mon.0) 50 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:37:16.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:15.799678+0000 mon.smithi038 (mon.0) 51 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:37:16.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:37:15.800237+0000 mgr.smithi038.khesbl (mgr.14664) 63 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:37:16.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:15.807955+0000 mon.smithi038 (mon.0) 52 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:16.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:15.812609+0000 mon.smithi038 (mon.0) 53 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:37:16.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:15.814888+0000 mon.smithi038 (mon.0) 54 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:37:16.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:15.816189+0000 mon.smithi038 (mon.0) 55 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:37:16.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:15.817423+0000 mon.smithi038 (mon.0) 56 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:37:16.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:37:15.817981+0000 mgr.smithi038.khesbl (mgr.14664) 64 : cephadm [INF] Upgrade: Setting container_image for all crash 2024-02-15T18:37:16.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:15.821470+0000 mon.smithi038 (mon.0) 57 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:16.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:15.822146+0000 mon.smithi038 (mon.0) 58 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi038"}]: dispatch 2024-02-15T18:37:16.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:15.829167+0000 mon.smithi038 (mon.0) 59 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi038"}]': finished 2024-02-15T18:37:16.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:15.829777+0000 mon.smithi038 (mon.0) 60 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi121"}]: dispatch 2024-02-15T18:37:16.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:15.834744+0000 mon.smithi038 (mon.0) 61 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi121"}]': finished 2024-02-15T18:37:16.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:15.835759+0000 mon.smithi038 (mon.0) 62 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-15T18:37:16.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:15.835892+0000 mgr.smithi038.khesbl (mgr.14664) 65 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-15T18:37:16.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:37:15.836544+0000 mgr.smithi038.khesbl (mgr.14664) 66 : cephadm [INF] Upgrade: osd.0 is safe to restart 2024-02-15T18:37:17.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:16.938811+0000 mgr.smithi038.khesbl (mgr.14664) 67 : cluster [DBG] pgmap v41: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:37:17.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:16.938811+0000 mgr.smithi038.khesbl (mgr.14664) 67 : cluster [DBG] pgmap v41: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:37:18.647 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:37:17.556250+0000 mgr.smithi038.khesbl (mgr.14664) 68 : cephadm [INF] Upgrade: Updating osd.0 2024-02-15T18:37:18.648 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:17.560925+0000 mon.smithi038 (mon.0) 63 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:18.648 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:17.561878+0000 mon.smithi038 (mon.0) 64 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-15T18:37:18.648 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:17.562518+0000 mon.smithi038 (mon.0) 65 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:37:18.648 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:37:17.563728+0000 mgr.smithi038.khesbl (mgr.14664) 69 : cephadm [INF] Deploying daemon osd.0 on smithi038 2024-02-15T18:37:18.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:37:17.556250+0000 mgr.smithi038.khesbl (mgr.14664) 68 : cephadm [INF] Upgrade: Updating osd.0 2024-02-15T18:37:18.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:17.560925+0000 mon.smithi038 (mon.0) 63 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:18.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:17.561878+0000 mon.smithi038 (mon.0) 64 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-15T18:37:18.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:17.562518+0000 mon.smithi038 (mon.0) 65 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:37:18.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:37:17.563728+0000 mgr.smithi038.khesbl (mgr.14664) 69 : cephadm [INF] Deploying daemon osd.0 on smithi038 2024-02-15T18:37:19.730 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:18.939123+0000 mgr.smithi038.khesbl (mgr.14664) 70 : cluster [DBG] pgmap v42: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:37:19.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:18.939123+0000 mgr.smithi038.khesbl (mgr.14664) 70 : cluster [DBG] pgmap v42: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:37:21.206 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:20.939590+0000 mgr.smithi038.khesbl (mgr.14664) 71 : cluster [DBG] pgmap v43: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:37:21.206 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:21 smithi038 systemd[1]: Stopping Ceph osd.0 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:37:21.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:20.939590+0000 mgr.smithi038.khesbl (mgr.14664) 71 : cluster [DBG] pgmap v43: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:37:21.522 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.0[48414]: debug 2024-02-15T18:37:21.214+0000 7f2ea9439700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:37:21.522 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.0[48414]: debug 2024-02-15T18:37:21.214+0000 7f2ea9439700 -1 osd.0 46 *** Got signal Terminated *** 2024-02-15T18:37:21.522 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.0[48414]: debug 2024-02-15T18:37:21.214+0000 7f2ea9439700 -1 osd.0 46 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:37:21.927 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:21 smithi038 bash[80558]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.0 2024-02-15T18:37:22.272 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:21 smithi038 bash[80610]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.0 2024-02-15T18:37:22.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.468685+0000 mon.smithi038 (mon.0) 66 : cluster [DBG] osd.0 reported immediately failed by osd.5 2024-02-15T18:37:22.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.468719+0000 mon.smithi038 (mon.0) 67 : cluster [INF] osd.0 failed (root=default,host=smithi038) (connection refused reported by osd.5) 2024-02-15T18:37:22.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.468847+0000 mon.smithi038 (mon.0) 68 : cluster [DBG] osd.0 reported immediately failed by osd.5 2024-02-15T18:37:22.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.468936+0000 mon.smithi038 (mon.0) 69 : cluster [DBG] osd.0 reported immediately failed by osd.3 2024-02-15T18:37:22.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.468990+0000 mon.smithi038 (mon.0) 70 : cluster [DBG] osd.0 reported immediately failed by osd.2 2024-02-15T18:37:22.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.469136+0000 mon.smithi038 (mon.0) 71 : cluster [DBG] osd.0 reported immediately failed by osd.4 2024-02-15T18:37:22.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.469190+0000 mon.smithi038 (mon.0) 72 : cluster [DBG] osd.0 reported immediately failed by osd.1 2024-02-15T18:37:22.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.469297+0000 mon.smithi038 (mon.0) 73 : cluster [DBG] osd.0 reported immediately failed by osd.3 2024-02-15T18:37:22.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.469385+0000 mon.smithi038 (mon.0) 74 : cluster [DBG] osd.0 reported immediately failed by osd.2 2024-02-15T18:37:22.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.469435+0000 mon.smithi038 (mon.0) 75 : cluster [DBG] osd.0 reported immediately failed by osd.4 2024-02-15T18:37:22.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.469557+0000 mon.smithi038 (mon.0) 76 : cluster [DBG] osd.0 reported immediately failed by osd.1 2024-02-15T18:37:22.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.869626+0000 mon.smithi038 (mon.0) 77 : cluster [DBG] osd.0 reported immediately failed by osd.1 2024-02-15T18:37:22.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.869728+0000 mon.smithi038 (mon.0) 78 : cluster [DBG] osd.0 reported immediately failed by osd.3 2024-02-15T18:37:22.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.869854+0000 mon.smithi038 (mon.0) 79 : cluster [DBG] osd.0 reported immediately failed by osd.4 2024-02-15T18:37:22.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.869931+0000 mon.smithi038 (mon.0) 80 : cluster [DBG] osd.0 reported immediately failed by osd.1 2024-02-15T18:37:22.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.870019+0000 mon.smithi038 (mon.0) 81 : cluster [DBG] osd.0 reported immediately failed by osd.5 2024-02-15T18:37:22.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.870096+0000 mon.smithi038 (mon.0) 82 : cluster [DBG] osd.0 reported immediately failed by osd.3 2024-02-15T18:37:22.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.870171+0000 mon.smithi038 (mon.0) 83 : cluster [DBG] osd.0 reported immediately failed by osd.2 2024-02-15T18:37:22.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.870242+0000 mon.smithi038 (mon.0) 84 : cluster [DBG] osd.0 reported immediately failed by osd.4 2024-02-15T18:37:22.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.870327+0000 mon.smithi038 (mon.0) 85 : cluster [DBG] osd.0 reported immediately failed by osd.5 2024-02-15T18:37:22.274 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:21.870372+0000 mon.smithi038 (mon.0) 86 : cluster [DBG] osd.0 reported immediately failed by osd.2 2024-02-15T18:37:22.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.468685+0000 mon.smithi038 (mon.0) 66 : cluster [DBG] osd.0 reported immediately failed by osd.5 2024-02-15T18:37:22.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.468719+0000 mon.smithi038 (mon.0) 67 : cluster [INF] osd.0 failed (root=default,host=smithi038) (connection refused reported by osd.5) 2024-02-15T18:37:22.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.468847+0000 mon.smithi038 (mon.0) 68 : cluster [DBG] osd.0 reported immediately failed by osd.5 2024-02-15T18:37:22.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.468936+0000 mon.smithi038 (mon.0) 69 : cluster [DBG] osd.0 reported immediately failed by osd.3 2024-02-15T18:37:22.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.468990+0000 mon.smithi038 (mon.0) 70 : cluster [DBG] osd.0 reported immediately failed by osd.2 2024-02-15T18:37:22.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.469136+0000 mon.smithi038 (mon.0) 71 : cluster [DBG] osd.0 reported immediately failed by osd.4 2024-02-15T18:37:22.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.469190+0000 mon.smithi038 (mon.0) 72 : cluster [DBG] osd.0 reported immediately failed by osd.1 2024-02-15T18:37:22.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.469297+0000 mon.smithi038 (mon.0) 73 : cluster [DBG] osd.0 reported immediately failed by osd.3 2024-02-15T18:37:22.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.469385+0000 mon.smithi038 (mon.0) 74 : cluster [DBG] osd.0 reported immediately failed by osd.2 2024-02-15T18:37:22.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.469435+0000 mon.smithi038 (mon.0) 75 : cluster [DBG] osd.0 reported immediately failed by osd.4 2024-02-15T18:37:22.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.469557+0000 mon.smithi038 (mon.0) 76 : cluster [DBG] osd.0 reported immediately failed by osd.1 2024-02-15T18:37:22.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.869626+0000 mon.smithi038 (mon.0) 77 : cluster [DBG] osd.0 reported immediately failed by osd.1 2024-02-15T18:37:22.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.869728+0000 mon.smithi038 (mon.0) 78 : cluster [DBG] osd.0 reported immediately failed by osd.3 2024-02-15T18:37:22.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.869854+0000 mon.smithi038 (mon.0) 79 : cluster [DBG] osd.0 reported immediately failed by osd.4 2024-02-15T18:37:22.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.869931+0000 mon.smithi038 (mon.0) 80 : cluster [DBG] osd.0 reported immediately failed by osd.1 2024-02-15T18:37:22.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.870019+0000 mon.smithi038 (mon.0) 81 : cluster [DBG] osd.0 reported immediately failed by osd.5 2024-02-15T18:37:22.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.870096+0000 mon.smithi038 (mon.0) 82 : cluster [DBG] osd.0 reported immediately failed by osd.3 2024-02-15T18:37:22.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.870171+0000 mon.smithi038 (mon.0) 83 : cluster [DBG] osd.0 reported immediately failed by osd.2 2024-02-15T18:37:22.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.870242+0000 mon.smithi038 (mon.0) 84 : cluster [DBG] osd.0 reported immediately failed by osd.4 2024-02-15T18:37:22.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.870327+0000 mon.smithi038 (mon.0) 85 : cluster [DBG] osd.0 reported immediately failed by osd.5 2024-02-15T18:37:22.358 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:21.870372+0000 mon.smithi038 (mon.0) 86 : cluster [DBG] osd.0 reported immediately failed by osd.2 2024-02-15T18:37:23.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:22.028237+0000 mon.smithi038 (mon.0) 87 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:37:23.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:22.041327+0000 mon.smithi038 (mon.0) 88 : cluster [DBG] osdmap e47: 6 total, 5 up, 6 in 2024-02-15T18:37:23.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:22.041732+0000 mon.smithi038 (mon.0) 89 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-15T18:37:23.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:22.042531+0000 mon.smithi038 (mon.0) 90 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-15T18:37:23.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:22.940008+0000 mgr.smithi038.khesbl (mgr.14664) 72 : cluster [DBG] pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:37:23.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:22.028237+0000 mon.smithi038 (mon.0) 87 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:37:23.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:22.041327+0000 mon.smithi038 (mon.0) 88 : cluster [DBG] osdmap e47: 6 total, 5 up, 6 in 2024-02-15T18:37:23.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:22.041732+0000 mon.smithi038 (mon.0) 89 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-15T18:37:23.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:22.042531+0000 mon.smithi038 (mon.0) 90 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-15T18:37:23.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:22.940008+0000 mgr.smithi038.khesbl (mgr.14664) 72 : cluster [DBG] pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:37:24.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:24 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:23.048590+0000 mon.smithi038 (mon.0) 91 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-15T18:37:24.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:24 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:23.048701+0000 mon.smithi038 (mon.0) 92 : cluster [DBG] osdmap e48: 6 total, 5 up, 6 in 2024-02-15T18:37:24.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.475+0000 7fecb936d700 1 -- 172.21.15.38:0/711143023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecb40f66d0 msgr2=0x7fecb40f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:24.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.475+0000 7fecb936d700 1 --2- 172.21.15.38:0/711143023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecb40f66d0 0x7fecb40f6b30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7feca8004660 tx=0x7feca8009b30).stop 2024-02-15T18:37:24.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.476+0000 7fecb936d700 1 -- 172.21.15.38:0/711143023 shutdown_connections 2024-02-15T18:37:24.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.476+0000 7fecb936d700 1 --2- 172.21.15.38:0/711143023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecb40f66d0 0x7fecb40f6b30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:24.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.476+0000 7fecb936d700 1 --2- 172.21.15.38:0/711143023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fecb40f5430 0x7fecb40f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:37:24.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.476+0000 7fecb936d700 1 -- 172.21.15.38:0/711143023 >> 172.21.15.38:0/711143023 conn(0x7fecb40f0b90 msgr2=0x7fecb40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:24.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.476+0000 7fecb936d700 1 -- 172.21.15.38:0/711143023 shutdown_connections 2024-02-15T18:37:24.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.476+0000 7fecb936d700 1 -- 172.21.15.38:0/711143023 wait complete. 2024-02-15T18:37:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.476+0000 7fecb936d700 1 Processor -- start 2024-02-15T18:37:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.477+0000 7fecb936d700 1 -- start start 2024-02-15T18:37:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.477+0000 7fecb936d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecb40f5430 0x7fecb4109b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.477+0000 7fecb936d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fecb40f66d0 0x7fecb410c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.477+0000 7fecb936d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb4060470 con 0x7fecb40f5430 2024-02-15T18:37:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.477+0000 7fecb936d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb40605b0 con 0x7fecb40f66d0 2024-02-15T18:37:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.477+0000 7fecb3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecb40f5430 0x7fecb4109b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.477+0000 7fecb3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecb40f5430 0x7fecb4109b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42798/0 (socket says 172.21.15.38:42798) 2024-02-15T18:37:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.477+0000 7fecb3fff700 1 -- 172.21.15.38:0/1570728966 learned_addr learned my addr 172.21.15.38:0/1570728966 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:37:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.477+0000 7fecb37fe700 1 --2- 172.21.15.38:0/1570728966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fecb40f66d0 0x7fecb410c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.477+0000 7fecb3fff700 1 -- 172.21.15.38:0/1570728966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fecb40f66d0 msgr2=0x7fecb410c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.477+0000 7fecb3fff700 1 --2- 172.21.15.38:0/1570728966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fecb40f66d0 0x7fecb410c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:24.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.477+0000 7fecb3fff700 1 -- 172.21.15.38:0/1570728966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca8005040 con 0x7fecb40f5430 2024-02-15T18:37:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.477+0000 7fecb37fe700 1 --2- 172.21.15.38:0/1570728966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fecb40f66d0 0x7fecb410c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:37:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.477+0000 7fecb3fff700 1 --2- 172.21.15.38:0/1570728966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecb40f5430 0x7fecb4109b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7feca40093f0 tx=0x7feca4006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.478+0000 7fecb0ff9700 1 -- 172.21.15.38:0/1570728966 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feca4009ea0 con 0x7fecb40f5430 2024-02-15T18:37:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.478+0000 7fecb936d700 1 -- 172.21.15.38:0/1570728966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb410c6b0 con 0x7fecb40f5430 2024-02-15T18:37:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.478+0000 7fecb0ff9700 1 -- 172.21.15.38:0/1570728966 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7feca4013b30 con 0x7fecb40f5430 2024-02-15T18:37:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.478+0000 7fecb0ff9700 1 -- 172.21.15.38:0/1570728966 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feca400dc60 con 0x7fecb40f5430 2024-02-15T18:37:24.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.478+0000 7fecb936d700 1 -- 172.21.15.38:0/1570728966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb410cbc0 con 0x7fecb40f5430 2024-02-15T18:37:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.479+0000 7fecb936d700 1 -- 172.21.15.38:0/1570728966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb4059100 con 0x7fecb40f5430 2024-02-15T18:37:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.479+0000 7fecb0ff9700 1 -- 172.21.15.38:0/1570728966 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 30) v1 ==== 77246+0+0 (secure 0 0 0) 0x7feca401a060 con 0x7fecb40f5430 2024-02-15T18:37:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.479+0000 7fecb0ff9700 1 --2- 172.21.15.38:0/1570728966 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fec9c061000 0x7fec9c0634c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.479+0000 7fecb0ff9700 1 -- 172.21.15.38:0/1570728966 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7feca407c220 con 0x7fecb40f5430 2024-02-15T18:37:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.479+0000 7fecb37fe700 1 --2- 172.21.15.38:0/1570728966 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fec9c061000 0x7fec9c0634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:24.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.480+0000 7fecb37fe700 1 --2- 172.21.15.38:0/1570728966 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fec9c061000 0x7fec9c0634c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7feca8009d60 tx=0x7feca8009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:24.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.484+0000 7fecb0ff9700 1 -- 172.21.15.38:0/1570728966 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7feca404c5b0 con 0x7fecb40f5430 2024-02-15T18:37:24.522 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:24 smithi038 systemd[1]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.0.service: Succeeded. 2024-02-15T18:37:24.522 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:24 smithi038 systemd[1]: Stopped Ceph osd.0 for 7e05753e-cc2f-11ee-95ba-87774f69a715. 2024-02-15T18:37:24.522 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:24 smithi038 systemd[1]: Starting Ceph osd.0 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:37:24.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:23.048590+0000 mon.smithi038 (mon.0) 91 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-15T18:37:24.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:23.048701+0000 mon.smithi038 (mon.0) 92 : cluster [DBG] osdmap e48: 6 total, 5 up, 6 in 2024-02-15T18:37:24.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.677+0000 7fecb936d700 1 -- 172.21.15.38:0/1570728966 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fecb40fdf10 con 0x7fec9c061000 2024-02-15T18:37:24.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.678+0000 7fecb0ff9700 1 -- 172.21.15.38:0/1570728966 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fecb40fdf10 con 0x7fec9c061000 2024-02-15T18:37:24.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.680+0000 7fecb936d700 1 -- 172.21.15.38:0/1570728966 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fec9c061000 msgr2=0x7fec9c0634c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:24.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.680+0000 7fecb936d700 1 --2- 172.21.15.38:0/1570728966 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fec9c061000 0x7fec9c0634c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7feca8009d60 tx=0x7feca8009b30).stop 2024-02-15T18:37:24.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.680+0000 7fecb936d700 1 -- 172.21.15.38:0/1570728966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecb40f5430 msgr2=0x7fecb4109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:24.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.680+0000 7fecb936d700 1 --2- 172.21.15.38:0/1570728966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecb40f5430 0x7fecb4109b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7feca40093f0 tx=0x7feca4006ec0).stop 2024-02-15T18:37:24.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.680+0000 7fecb936d700 1 -- 172.21.15.38:0/1570728966 shutdown_connections 2024-02-15T18:37:24.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.680+0000 7fecb936d700 1 --2- 172.21.15.38:0/1570728966 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fec9c061000 0x7fec9c0634c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:24.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.680+0000 7fecb936d700 1 --2- 172.21.15.38:0/1570728966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecb40f5430 0x7fecb4109b50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:24.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.680+0000 7fecb936d700 1 --2- 172.21.15.38:0/1570728966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fecb40f66d0 0x7fecb410c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:24.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.680+0000 7fecb936d700 1 -- 172.21.15.38:0/1570728966 >> 172.21.15.38:0/1570728966 conn(0x7fecb40f0b90 msgr2=0x7fecb40fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:24.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.680+0000 7fecb936d700 1 -- 172.21.15.38:0/1570728966 shutdown_connections 2024-02-15T18:37:24.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.680+0000 7fecb936d700 1 -- 172.21.15.38:0/1570728966 wait complete. 2024-02-15T18:37:24.681 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-02-15T18:37:24.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.785+0000 7f62900ba700 1 -- 172.21.15.38:0/2547117521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62880f55c0 msgr2=0x7f62880f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:24.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.785+0000 7f62900ba700 1 --2- 172.21.15.38:0/2547117521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62880f55c0 0x7f62880f59e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f6284004660 tx=0x7f6284009b30).stop 2024-02-15T18:37:24.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.785+0000 7f62900ba700 1 -- 172.21.15.38:0/2547117521 shutdown_connections 2024-02-15T18:37:24.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.785+0000 7f62900ba700 1 --2- 172.21.15.38:0/2547117521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62880f6860 0x7f62880f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:24.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.785+0000 7f62900ba700 1 --2- 172.21.15.38:0/2547117521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62880f55c0 0x7f62880f59e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:24.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.785+0000 7f62900ba700 1 -- 172.21.15.38:0/2547117521 >> 172.21.15.38:0/2547117521 conn(0x7f62880f0d40 msgr2=0x7f62880f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:24.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.785+0000 7f62900ba700 1 -- 172.21.15.38:0/2547117521 shutdown_connections 2024-02-15T18:37:24.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.785+0000 7f62900ba700 1 -- 172.21.15.38:0/2547117521 wait complete. 2024-02-15T18:37:24.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.786+0000 7f62900ba700 1 Processor -- start 2024-02-15T18:37:24.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.786+0000 7f62900ba700 1 -- start start 2024-02-15T18:37:24.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.786+0000 7f62900ba700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62880f55c0 0x7f62881091f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:24.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.786+0000 7f62900ba700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62880f6860 0x7f62881096e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:24.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.786+0000 7f62900ba700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6288109c90 con 0x7f62880f6860 2024-02-15T18:37:24.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.786+0000 7f62900ba700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6288109dd0 con 0x7f62880f55c0 2024-02-15T18:37:24.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.786+0000 7f628d655700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62880f6860 0x7f62881096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:24.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.786+0000 7f628d655700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62880f6860 0x7f62881096e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42820/0 (socket says 172.21.15.38:42820) 2024-02-15T18:37:24.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.786+0000 7f628d655700 1 -- 172.21.15.38:0/3318602705 learned_addr learned my addr 172.21.15.38:0/3318602705 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:37:24.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.786+0000 7f628de56700 1 --2- 172.21.15.38:0/3318602705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62880f55c0 0x7f62881091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:24.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.786+0000 7f628d655700 1 -- 172.21.15.38:0/3318602705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62880f55c0 msgr2=0x7f62881091f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:24.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.786+0000 7f628d655700 1 --2- 172.21.15.38:0/3318602705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62880f55c0 0x7f62881091f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:24.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.786+0000 7f628d655700 1 -- 172.21.15.38:0/3318602705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6284005040 con 0x7f62880f6860 2024-02-15T18:37:24.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.786+0000 7f628de56700 1 --2- 172.21.15.38:0/3318602705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62880f55c0 0x7f62881091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:37:24.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.786+0000 7f628d655700 1 --2- 172.21.15.38:0/3318602705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62880f6860 0x7f62881096e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f627c00e7f0 tx=0x7f627c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:24.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.787+0000 7f627a7fc700 1 -- 172.21.15.38:0/3318602705 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f627c0092e0 con 0x7f62880f6860 2024-02-15T18:37:24.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.787+0000 7f627a7fc700 1 -- 172.21.15.38:0/3318602705 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f627c00f040 con 0x7f62880f6860 2024-02-15T18:37:24.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.787+0000 7f62900ba700 1 -- 172.21.15.38:0/3318602705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628810a770 con 0x7f62880f6860 2024-02-15T18:37:24.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.787+0000 7f627a7fc700 1 -- 172.21.15.38:0/3318602705 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f627c014880 con 0x7f62880f6860 2024-02-15T18:37:24.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.787+0000 7f62900ba700 1 -- 172.21.15.38:0/3318602705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628810ad30 con 0x7f62880f6860 2024-02-15T18:37:24.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.788+0000 7f62900ba700 1 -- 172.21.15.38:0/3318602705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6288045ad0 con 0x7f62880f6860 2024-02-15T18:37:24.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.788+0000 7f627a7fc700 1 -- 172.21.15.38:0/3318602705 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 30) v1 ==== 77246+0+0 (secure 0 0 0) 0x7f627c01b030 con 0x7f62880f6860 2024-02-15T18:37:24.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.789+0000 7f627a7fc700 1 --2- 172.21.15.38:0/3318602705 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f6274060fb0 0x7f6274063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:24.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.789+0000 7f627a7fc700 1 -- 172.21.15.38:0/3318602705 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f627c07c030 con 0x7f62880f6860 2024-02-15T18:37:24.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.789+0000 7f628de56700 1 --2- 172.21.15.38:0/3318602705 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f6274060fb0 0x7f6274063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:24.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.789+0000 7f628de56700 1 --2- 172.21.15.38:0/3318602705 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f6274060fb0 0x7f6274063470 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f6284000c00 tx=0x7f628400b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:24.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.793+0000 7f627a7fc700 1 -- 172.21.15.38:0/3318602705 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f627c04ff00 con 0x7f62880f6860 2024-02-15T18:37:24.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.984+0000 7f62900ba700 1 -- 172.21.15.38:0/3318602705 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f62880faaa0 con 0x7f6274060fb0 2024-02-15T18:37:24.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.985+0000 7f627a7fc700 1 -- 172.21.15.38:0/3318602705 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f62880faaa0 con 0x7f6274060fb0 2024-02-15T18:37:24.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.987+0000 7f62900ba700 1 -- 172.21.15.38:0/3318602705 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f6274060fb0 msgr2=0x7f6274063470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:24.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.987+0000 7f62900ba700 1 --2- 172.21.15.38:0/3318602705 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f6274060fb0 0x7f6274063470 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f6284000c00 tx=0x7f628400b040).stop 2024-02-15T18:37:24.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.987+0000 7f62900ba700 1 -- 172.21.15.38:0/3318602705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62880f6860 msgr2=0x7f62881096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:24.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.987+0000 7f62900ba700 1 --2- 172.21.15.38:0/3318602705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62880f6860 0x7f62881096e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f627c00e7f0 tx=0x7f627c00b040).stop 2024-02-15T18:37:24.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.987+0000 7f62900ba700 1 -- 172.21.15.38:0/3318602705 shutdown_connections 2024-02-15T18:37:24.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.987+0000 7f62900ba700 1 --2- 172.21.15.38:0/3318602705 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f6274060fb0 0x7f6274063470 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:24.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.987+0000 7f62900ba700 1 --2- 172.21.15.38:0/3318602705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62880f55c0 0x7f62881091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:24.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.987+0000 7f62900ba700 1 --2- 172.21.15.38:0/3318602705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62880f6860 0x7f62881096e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:24.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.987+0000 7f62900ba700 1 -- 172.21.15.38:0/3318602705 >> 172.21.15.38:0/3318602705 conn(0x7f62880f0d40 msgr2=0x7f628805fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:24.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.987+0000 7f62900ba700 1 -- 172.21.15.38:0/3318602705 shutdown_connections 2024-02-15T18:37:24.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:24.987+0000 7f62900ba700 1 -- 172.21.15.38:0/3318602705 wait complete. 2024-02-15T18:37:25.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.093+0000 7f10e4587700 1 -- 172.21.15.38:0/3918312648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10dc0f66e0 msgr2=0x7f10dc0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:25.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.093+0000 7f10e4587700 1 --2- 172.21.15.38:0/3918312648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10dc0f66e0 0x7f10dc0f6b00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f10d8004660 tx=0x7f10d8009b30).stop 2024-02-15T18:37:25.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.093+0000 7f10e4587700 1 -- 172.21.15.38:0/3918312648 shutdown_connections 2024-02-15T18:37:25.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.093+0000 7f10e4587700 1 --2- 172.21.15.38:0/3918312648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10dc0f7980 0x7f10dc0f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:25.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.093+0000 7f10e4587700 1 --2- 172.21.15.38:0/3918312648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10dc0f66e0 0x7f10dc0f6b00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:25.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.093+0000 7f10e4587700 1 -- 172.21.15.38:0/3918312648 >> 172.21.15.38:0/3918312648 conn(0x7f10dc0f1e40 msgr2=0x7f10dc0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:25.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.093+0000 7f10e4587700 1 -- 172.21.15.38:0/3918312648 shutdown_connections 2024-02-15T18:37:25.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.093+0000 7f10e4587700 1 -- 172.21.15.38:0/3918312648 wait complete. 2024-02-15T18:37:25.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.094+0000 7f10e4587700 1 Processor -- start 2024-02-15T18:37:25.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.094+0000 7f10e4587700 1 -- start start 2024-02-15T18:37:25.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.094+0000 7f10e4587700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10dc0f66e0 0x7f10dc06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:25.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.094+0000 7f10e4587700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10dc0f7980 0x7f10dc069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:25.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.094+0000 7f10e4587700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10dc06a380 con 0x7f10dc0f66e0 2024-02-15T18:37:25.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.094+0000 7f10e4587700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10dc06a4c0 con 0x7f10dc0f7980 2024-02-15T18:37:25.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.095+0000 7f10e2323700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10dc0f66e0 0x7f10dc06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:25.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.095+0000 7f10e2323700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10dc0f66e0 0x7f10dc06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42844/0 (socket says 172.21.15.38:42844) 2024-02-15T18:37:25.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.095+0000 7f10e2323700 1 -- 172.21.15.38:0/3834654074 learned_addr learned my addr 172.21.15.38:0/3834654074 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:37:25.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.095+0000 7f10e1b22700 1 --2- 172.21.15.38:0/3834654074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10dc0f7980 0x7f10dc069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:25.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.095+0000 7f10e2323700 1 -- 172.21.15.38:0/3834654074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10dc0f7980 msgr2=0x7f10dc069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:25.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.095+0000 7f10e2323700 1 --2- 172.21.15.38:0/3834654074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10dc0f7980 0x7f10dc069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:25.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.095+0000 7f10e2323700 1 -- 172.21.15.38:0/3834654074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d8005040 con 0x7f10dc0f66e0 2024-02-15T18:37:25.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.095+0000 7f10e2323700 1 --2- 172.21.15.38:0/3834654074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10dc0f66e0 0x7f10dc06b9a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f10d8013fd0 tx=0x7f10d8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:25.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.095+0000 7f10d2ffd700 1 -- 172.21.15.38:0/3834654074 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10d8018070 con 0x7f10dc0f66e0 2024-02-15T18:37:25.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.095+0000 7f10e4587700 1 -- 172.21.15.38:0/3834654074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10dc06a740 con 0x7f10dc0f66e0 2024-02-15T18:37:25.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.095+0000 7f10d2ffd700 1 -- 172.21.15.38:0/3834654074 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f10d8007d70 con 0x7f10dc0f66e0 2024-02-15T18:37:25.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.095+0000 7f10d2ffd700 1 -- 172.21.15.38:0/3834654074 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10d801c620 con 0x7f10dc0f66e0 2024-02-15T18:37:25.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.095+0000 7f10e4587700 1 -- 172.21.15.38:0/3834654074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10dc06aca0 con 0x7f10dc0f66e0 2024-02-15T18:37:25.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.096+0000 7f10e4587700 1 -- 172.21.15.38:0/3834654074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10dc045ad0 con 0x7f10dc0f66e0 2024-02-15T18:37:25.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.097+0000 7f10d2ffd700 1 -- 172.21.15.38:0/3834654074 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 30) v1 ==== 77246+0+0 (secure 0 0 0) 0x7f10d801d420 con 0x7f10dc0f66e0 2024-02-15T18:37:25.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.097+0000 7f10d2ffd700 1 --2- 172.21.15.38:0/3834654074 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f10c8060fb0 0x7f10c8063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:25.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.097+0000 7f10d2ffd700 1 -- 172.21.15.38:0/3834654074 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f10d807e0d0 con 0x7f10dc0f66e0 2024-02-15T18:37:25.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.097+0000 7f10e1b22700 1 --2- 172.21.15.38:0/3834654074 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f10c8060fb0 0x7f10c8063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:25.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.098+0000 7f10e1b22700 1 --2- 172.21.15.38:0/3834654074 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f10c8060fb0 0x7f10c8063470 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f10cc0041d0 tx=0x7f10cc009040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:25.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.102+0000 7f10d2ffd700 1 -- 172.21.15.38:0/3834654074 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f10d804e5a0 con 0x7f10dc0f66e0 2024-02-15T18:37:25.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.305+0000 7f10e4587700 1 -- 172.21.15.38:0/3834654074 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f10dc0ff130 con 0x7f10c8060fb0 2024-02-15T18:37:25.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.315+0000 7f10d2ffd700 1 -- 172.21.15.38:0/3834654074 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f10dc0ff130 con 0x7f10c8060fb0 2024-02-15T18:37:25.306 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:37:25.306 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (8m) 9s ago 10m 19.3M - 0.20.0 0881eb8f169f c2034c318e7c 2024-02-15T18:37:25.306 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (23s) 9s ago 10m 6635k - 16.2.14-521-g0e714d9a 1fbc92c859dc c2fee0824d4b 2024-02-15T18:37:25.306 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi121 smithi121 running (17s) 13s ago 9m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 05c1a5501a35 2024-02-15T18:37:25.306 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (8m) 9s ago 9m 33.3M - 6.7.4 557c83e11646 2dcccfbfbd2a 2024-02-15T18:37:25.306 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.hxoggl smithi038 running (4m) 9s ago 4m 14.5M - 16.2.4 8d91d370c2b8 be2533754b9c 2024-02-15T18:37:25.306 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.qgkzgn smithi038 running (4m) 9s ago 4m 14.1M - 16.2.4 8d91d370c2b8 dd3ffafba9bd 2024-02-15T18:37:25.306 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.mvumlh smithi121 running (4m) 13s ago 4m 116M - 16.2.4 8d91d370c2b8 88b7d876a39b 2024-02-15T18:37:25.307 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.ovuzki smithi121 running (4m) 13s ago 4m 124M - 16.2.4 8d91d370c2b8 21cacf0012f9 2024-02-15T18:37:25.307 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.khesbl smithi038 *:8443,9283 running (91s) 9s ago 11m 428M - 16.2.14-521-g0e714d9a 1fbc92c859dc e642f3c2cfc9 2024-02-15T18:37:25.307 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi121.ckiryi smithi121 *:8443,9283 running (64s) 13s ago 9m 385M - 16.2.14-521-g0e714d9a 1fbc92c859dc 82515b025594 2024-02-15T18:37:25.307 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (53s) 9s ago 11m 36.5M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc d3e8317a30ca 2024-02-15T18:37:25.307 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi121 smithi121 running (33s) 13s ago 8m 26.2M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 42e1dda1a39d 2024-02-15T18:37:25.307 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (9m) 9s ago 9m 17.1M - 0.18.1 e5a616e4b9cf 037f60e38158 2024-02-15T18:37:25.307 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi121 smithi121 *:9100 running (8m) 13s ago 8m 17.2M - 0.18.1 e5a616e4b9cf a867bff85e18 2024-02-15T18:37:25.307 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (8m) 9s ago 8m 410M 4096M 16.2.4 8d91d370c2b8 51512003a2ef 2024-02-15T18:37:25.307 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (7m) 9s ago 7m 400M 4096M 16.2.4 8d91d370c2b8 16dbdf20b170 2024-02-15T18:37:25.307 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (7m) 9s ago 7m 357M 4096M 16.2.4 8d91d370c2b8 6d6ee813e5be 2024-02-15T18:37:25.307 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi121 running (7m) 13s ago 7m 449M 4096M 16.2.4 8d91d370c2b8 cba2d9ed4730 2024-02-15T18:37:25.307 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi121 running (6m) 13s ago 6m 429M 4096M 16.2.4 8d91d370c2b8 ac17ad7dc628 2024-02-15T18:37:25.307 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi121 running (6m) 13s ago 6m 378M 4096M 16.2.4 8d91d370c2b8 cdc8c669c4db 2024-02-15T18:37:25.307 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (97s) 9s ago 9m 59.6M - 2.18.1 de242295e225 67e33f9de4e6 2024-02-15T18:37:25.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.317+0000 7f10e4587700 1 -- 172.21.15.38:0/3834654074 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f10c8060fb0 msgr2=0x7f10c8063470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:25.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.317+0000 7f10e4587700 1 --2- 172.21.15.38:0/3834654074 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f10c8060fb0 0x7f10c8063470 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f10cc0041d0 tx=0x7f10cc009040).stop 2024-02-15T18:37:25.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.317+0000 7f10e4587700 1 -- 172.21.15.38:0/3834654074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10dc0f66e0 msgr2=0x7f10dc06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:25.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.317+0000 7f10e4587700 1 --2- 172.21.15.38:0/3834654074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10dc0f66e0 0x7f10dc06b9a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f10d8013fd0 tx=0x7f10d8009b30).stop 2024-02-15T18:37:25.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.317+0000 7f10e4587700 1 -- 172.21.15.38:0/3834654074 shutdown_connections 2024-02-15T18:37:25.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.317+0000 7f10e4587700 1 --2- 172.21.15.38:0/3834654074 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f10c8060fb0 0x7f10c8063470 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:25.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.317+0000 7f10e4587700 1 --2- 172.21.15.38:0/3834654074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10dc0f66e0 0x7f10dc06b9a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:25.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.317+0000 7f10e4587700 1 --2- 172.21.15.38:0/3834654074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10dc0f7980 0x7f10dc069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:25.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.317+0000 7f10e4587700 1 -- 172.21.15.38:0/3834654074 >> 172.21.15.38:0/3834654074 conn(0x7f10dc0f1e40 msgr2=0x7f10dc0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:25.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.317+0000 7f10e4587700 1 -- 172.21.15.38:0/3834654074 shutdown_connections 2024-02-15T18:37:25.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.317+0000 7f10e4587700 1 -- 172.21.15.38:0/3834654074 wait complete. 2024-02-15T18:37:25.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:24.679290+0000 mgr.smithi038.khesbl (mgr.14664) 73 : audit [DBG] from='client.34124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:37:25.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:24.940542+0000 mgr.smithi038.khesbl (mgr.14664) 74 : cluster [DBG] pgmap v47: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 65/342 objects degraded (19.006%) 2024-02-15T18:37:25.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.423+0000 7f2d0cf45700 1 -- 172.21.15.38:0/953415045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d080f66e0 msgr2=0x7f2d080f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:25.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.423+0000 7f2d0cf45700 1 --2- 172.21.15.38:0/953415045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d080f66e0 0x7f2d080f6b00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f2cf8004660 tx=0x7f2cf8009b30).stop 2024-02-15T18:37:25.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.423+0000 7f2d0cf45700 1 -- 172.21.15.38:0/953415045 shutdown_connections 2024-02-15T18:37:25.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.423+0000 7f2d0cf45700 1 --2- 172.21.15.38:0/953415045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d080f7980 0x7f2d080f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:25.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.423+0000 7f2d0cf45700 1 --2- 172.21.15.38:0/953415045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d080f66e0 0x7f2d080f6b00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:25.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.423+0000 7f2d0cf45700 1 -- 172.21.15.38:0/953415045 >> 172.21.15.38:0/953415045 conn(0x7f2d080f1e40 msgr2=0x7f2d080f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:25.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.423+0000 7f2d0cf45700 1 -- 172.21.15.38:0/953415045 shutdown_connections 2024-02-15T18:37:25.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.424+0000 7f2d0cf45700 1 -- 172.21.15.38:0/953415045 wait complete. 2024-02-15T18:37:25.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.424+0000 7f2d0cf45700 1 Processor -- start 2024-02-15T18:37:25.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.424+0000 7f2d0cf45700 1 -- start start 2024-02-15T18:37:25.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.424+0000 7f2d0cf45700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d080f66e0 0x7f2d08107030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:25.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.424+0000 7f2d0cf45700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d080f7980 0x7f2d08107520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:25.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.424+0000 7f2d0cf45700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d08107ad0 con 0x7f2d080f7980 2024-02-15T18:37:25.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.424+0000 7f2d0cf45700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d08107c10 con 0x7f2d080f66e0 2024-02-15T18:37:25.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.424+0000 7f2d05d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d080f7980 0x7f2d08107520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:25.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.424+0000 7f2d05d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d080f7980 0x7f2d08107520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42856/0 (socket says 172.21.15.38:42856) 2024-02-15T18:37:25.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.424+0000 7f2d05d9b700 1 -- 172.21.15.38:0/2042597465 learned_addr learned my addr 172.21.15.38:0/2042597465 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:37:25.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.425+0000 7f2d0659c700 1 --2- 172.21.15.38:0/2042597465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d080f66e0 0x7f2d08107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:25.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.425+0000 7f2d05d9b700 1 -- 172.21.15.38:0/2042597465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d080f66e0 msgr2=0x7f2d08107030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:25.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.425+0000 7f2d05d9b700 1 --2- 172.21.15.38:0/2042597465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d080f66e0 0x7f2d08107030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:25.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.425+0000 7f2d05d9b700 1 -- 172.21.15.38:0/2042597465 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf8005040 con 0x7f2d080f7980 2024-02-15T18:37:25.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.425+0000 7f2d05d9b700 1 --2- 172.21.15.38:0/2042597465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d080f7980 0x7f2d08107520 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f2cf000c530 tx=0x7f2cf0007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:25.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.425+0000 7f2cfeffd700 1 -- 172.21.15.38:0/2042597465 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cf000f650 con 0x7f2d080f7980 2024-02-15T18:37:25.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.425+0000 7f2cfeffd700 1 -- 172.21.15.38:0/2042597465 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f2cf000fca0 con 0x7f2d080f7980 2024-02-15T18:37:25.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.425+0000 7f2d0cf45700 1 -- 172.21.15.38:0/2042597465 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d081085b0 con 0x7f2d080f7980 2024-02-15T18:37:25.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.425+0000 7f2cfeffd700 1 -- 172.21.15.38:0/2042597465 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cf0012d40 con 0x7f2d080f7980 2024-02-15T18:37:25.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.425+0000 7f2d0cf45700 1 -- 172.21.15.38:0/2042597465 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d08108b70 con 0x7f2d080f7980 2024-02-15T18:37:25.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.426+0000 7f2d0cf45700 1 -- 172.21.15.38:0/2042597465 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d08045ad0 con 0x7f2d080f7980 2024-02-15T18:37:25.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.426+0000 7f2cfeffd700 1 -- 172.21.15.38:0/2042597465 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 30) v1 ==== 77246+0+0 (secure 0 0 0) 0x7f2cf0010070 con 0x7f2d080f7980 2024-02-15T18:37:25.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.427+0000 7f2cfeffd700 1 --2- 172.21.15.38:0/2042597465 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f2cf4060fb0 0x7f2cf4063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:25.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.427+0000 7f2cfeffd700 1 -- 172.21.15.38:0/2042597465 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f2cf007be80 con 0x7f2d080f7980 2024-02-15T18:37:25.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.427+0000 7f2d0659c700 1 --2- 172.21.15.38:0/2042597465 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f2cf4060fb0 0x7f2cf4063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:25.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.427+0000 7f2d0659c700 1 --2- 172.21.15.38:0/2042597465 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f2cf4060fb0 0x7f2cf4063470 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f2cf8009f50 tx=0x7f2cf8009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:25.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.431+0000 7f2cfeffd700 1 -- 172.21.15.38:0/2042597465 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2cf0047f60 con 0x7f2d080f7980 2024-02-15T18:37:25.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:24.679290+0000 mgr.smithi038.khesbl (mgr.14664) 73 : audit [DBG] from='client.34124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:37:25.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:24.940542+0000 mgr.smithi038.khesbl (mgr.14664) 74 : cluster [DBG] pgmap v47: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 65/342 objects degraded (19.006%) 2024-02-15T18:37:25.523 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:25 smithi038 bash[80871]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-02-15T18:37:25.523 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:25 smithi038 bash[80871]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-15T18:37:25.523 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:25 smithi038 bash[80871]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-0 --no-mon-config 2024-02-15T18:37:25.523 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:25 smithi038 bash[80871]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0/block 2024-02-15T18:37:25.523 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:25 smithi038 bash[80871]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-0/block 2024-02-15T18:37:25.523 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:25 smithi038 bash[80871]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-15T18:37:25.523 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:25 smithi038 bash[80871]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-15T18:37:25.523 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:25 smithi038 bash[80871]: --> ceph-volume lvm activate successful for osd ID: 0 2024-02-15T18:37:25.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.685+0000 7f2d0cf45700 1 -- 172.21.15.38:0/2042597465 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2d08027a00 con 0x7f2d080f7980 2024-02-15T18:37:25.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.685+0000 7f2cfeffd700 1 -- 172.21.15.38:0/2042597465 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f2cf004fca0 con 0x7f2d080f7980 2024-02-15T18:37:25.676 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:37:25.676 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-02-15T18:37:25.676 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:37:25.677 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:37:25.677 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-02-15T18:37:25.677 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:37:25.677 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:37:25.677 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-02-15T18:37:25.677 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 5 2024-02-15T18:37:25.677 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:37:25.677 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-02-15T18:37:25.677 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:37:25.677 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:37:25.677 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-02-15T18:37:25.677 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 4, 2024-02-15T18:37:25.677 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 9 2024-02-15T18:37:25.677 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-02-15T18:37:25.678 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:37:25.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.687+0000 7f2d0cf45700 1 -- 172.21.15.38:0/2042597465 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f2cf4060fb0 msgr2=0x7f2cf4063470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:25.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.687+0000 7f2d0cf45700 1 --2- 172.21.15.38:0/2042597465 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f2cf4060fb0 0x7f2cf4063470 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f2cf8009f50 tx=0x7f2cf8009b30).stop 2024-02-15T18:37:25.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.687+0000 7f2d0cf45700 1 -- 172.21.15.38:0/2042597465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d080f7980 msgr2=0x7f2d08107520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:25.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.687+0000 7f2d0cf45700 1 --2- 172.21.15.38:0/2042597465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d080f7980 0x7f2d08107520 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f2cf000c530 tx=0x7f2cf0007e20).stop 2024-02-15T18:37:25.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.687+0000 7f2d0cf45700 1 -- 172.21.15.38:0/2042597465 shutdown_connections 2024-02-15T18:37:25.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.687+0000 7f2d0cf45700 1 --2- 172.21.15.38:0/2042597465 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f2cf4060fb0 0x7f2cf4063470 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:25.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.687+0000 7f2d0cf45700 1 --2- 172.21.15.38:0/2042597465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d080f66e0 0x7f2d08107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:25.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.687+0000 7f2d0cf45700 1 --2- 172.21.15.38:0/2042597465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d080f7980 0x7f2d08107520 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:25.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.687+0000 7f2d0cf45700 1 -- 172.21.15.38:0/2042597465 >> 172.21.15.38:0/2042597465 conn(0x7f2d080f1e40 msgr2=0x7f2d080fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:25.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.687+0000 7f2d0cf45700 1 -- 172.21.15.38:0/2042597465 shutdown_connections 2024-02-15T18:37:25.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.687+0000 7f2d0cf45700 1 -- 172.21.15.38:0/2042597465 wait complete. 2024-02-15T18:37:25.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.789+0000 7f3732583700 1 -- 172.21.15.38:0/2648852634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f372c0f4d60 msgr2=0x7f372c0f5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:25.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.789+0000 7f3732583700 1 --2- 172.21.15.38:0/2648852634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f372c0f4d60 0x7f372c0f5180 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f371c004660 tx=0x7f371c009b30).stop 2024-02-15T18:37:25.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.789+0000 7f3732583700 1 -- 172.21.15.38:0/2648852634 shutdown_connections 2024-02-15T18:37:25.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.789+0000 7f3732583700 1 --2- 172.21.15.38:0/2648852634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f372c0f5f60 0x7f372c0f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:37:25.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.789+0000 7f3732583700 1 --2- 172.21.15.38:0/2648852634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f372c0f4d60 0x7f372c0f5180 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:25.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.789+0000 7f3732583700 1 -- 172.21.15.38:0/2648852634 >> 172.21.15.38:0/2648852634 conn(0x7f372c0f04c0 msgr2=0x7f372c0f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:25.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.789+0000 7f3732583700 1 -- 172.21.15.38:0/2648852634 shutdown_connections 2024-02-15T18:37:25.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.789+0000 7f3732583700 1 -- 172.21.15.38:0/2648852634 wait complete. 2024-02-15T18:37:25.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.790+0000 7f3732583700 1 Processor -- start 2024-02-15T18:37:25.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.790+0000 7f3732583700 1 -- start start 2024-02-15T18:37:25.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.790+0000 7f3732583700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f372c0f4d60 0x7f372c107100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:25.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.790+0000 7f3732583700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f372c0f5f60 0x7f372c1075f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:25.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.790+0000 7f3732583700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372c107c30 con 0x7f372c0f5f60 2024-02-15T18:37:25.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.790+0000 7f3732583700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372c1084e0 con 0x7f372c0f4d60 2024-02-15T18:37:25.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.790+0000 7f372b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f372c0f5f60 0x7f372c1075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:25.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.790+0000 7f372b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f372c0f5f60 0x7f372c1075f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42868/0 (socket says 172.21.15.38:42868) 2024-02-15T18:37:25.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.790+0000 7f372b7fe700 1 -- 172.21.15.38:0/847789336 learned_addr learned my addr 172.21.15.38:0/847789336 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:37:25.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.790+0000 7f372bfff700 1 --2- 172.21.15.38:0/847789336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f372c0f4d60 0x7f372c107100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:25.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.790+0000 7f372b7fe700 1 -- 172.21.15.38:0/847789336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f372c0f4d60 msgr2=0x7f372c107100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:25.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.790+0000 7f372b7fe700 1 --2- 172.21.15.38:0/847789336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f372c0f4d60 0x7f372c107100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:25.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.790+0000 7f372b7fe700 1 -- 172.21.15.38:0/847789336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c005040 con 0x7f372c0f5f60 2024-02-15T18:37:25.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.790+0000 7f372bfff700 1 --2- 172.21.15.38:0/847789336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f372c0f4d60 0x7f372c107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:37:25.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.791+0000 7f372b7fe700 1 --2- 172.21.15.38:0/847789336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f372c0f5f60 0x7f372c1075f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f372000b410 tx=0x7f3720006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:25.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.791+0000 7f3728ff9700 1 -- 172.21.15.38:0/847789336 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37200125f0 con 0x7f372c0f5f60 2024-02-15T18:37:25.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.791+0000 7f3728ff9700 1 -- 172.21.15.38:0/847789336 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f3720012c40 con 0x7f372c0f5f60 2024-02-15T18:37:25.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.791+0000 7f3732583700 1 -- 172.21.15.38:0/847789336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f372c1087c0 con 0x7f372c0f5f60 2024-02-15T18:37:25.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.791+0000 7f3728ff9700 1 -- 172.21.15.38:0/847789336 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3720002b80 con 0x7f372c0f5f60 2024-02-15T18:37:25.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.791+0000 7f3732583700 1 -- 172.21.15.38:0/847789336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f372c108d00 con 0x7f372c0f5f60 2024-02-15T18:37:25.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.792+0000 7f3732583700 1 -- 172.21.15.38:0/847789336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f372c0f4150 con 0x7f372c0f5f60 2024-02-15T18:37:25.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.792+0000 7f3728ff9700 1 -- 172.21.15.38:0/847789336 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 30) v1 ==== 77246+0+0 (secure 0 0 0) 0x7f372000f030 con 0x7f372c0f5f60 2024-02-15T18:37:25.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.793+0000 7f3728ff9700 1 --2- 172.21.15.38:0/847789336 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3714060fb0 0x7f3714063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:25.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.793+0000 7f3728ff9700 1 -- 172.21.15.38:0/847789336 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f372007c6b0 con 0x7f372c0f5f60 2024-02-15T18:37:25.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.793+0000 7f372bfff700 1 --2- 172.21.15.38:0/847789336 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3714060fb0 0x7f3714063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:25.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.793+0000 7f372bfff700 1 --2- 172.21.15.38:0/847789336 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3714060fb0 0x7f3714063470 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f371c004630 tx=0x7f371c009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:25.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:25.797+0000 7f3728ff9700 1 -- 172.21.15.38:0/847789336 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f37200504d0 con 0x7f372c0f5f60 2024-02-15T18:37:26.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.014+0000 7f3732583700 1 -- 172.21.15.38:0/847789336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f372c0fdf50 con 0x7f372c0f5f60 2024-02-15T18:37:26.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.015+0000 7f3728ff9700 1 -- 172.21.15.38:0/847789336 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1807 (secure 0 0 0) 0x7f37200500c0 con 0x7f372c0f5f60 2024-02-15T18:37:26.006 INFO:teuthology.orchestra.run.smithi038.stdout:e13 2024-02-15T18:37:26.006 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:37:26.006 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:37:26.006 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-02-15T18:37:26.006 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:37:26.006 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:37:26.006 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-02-15T18:37:26.006 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 13 2024-02-15T18:37:26.006 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 2024-02-15T18:37:26.006 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:37:26.006 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-02-15T18:35:07.485909+0000 2024-02-15T18:37:26.006 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-02-15T18:37:26.006 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-02-15T18:37:26.007 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-02-15T18:37:26.007 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-02-15T18:37:26.007 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-02-15T18:37:26.007 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-02-15T18:37:26.007 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-02-15T18:37:26.007 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-02-15T18:37:26.007 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:37:26.007 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-02-15T18:37:26.007 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-02-15T18:37:26.007 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=24261,1=24275} 2024-02-15T18:37:26.007 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-02-15T18:37:26.007 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-02-15T18:37:26.008 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-02-15T18:37:26.008 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-02-15T18:37:26.008 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-02-15T18:37:26.008 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-02-15T18:37:26.008 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-02-15T18:37:26.008 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-02-15T18:37:26.008 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.mvumlh{0:24261} state up:active seq 3 export targets 1 join_fscid=1 addr [v2:172.21.15.121:6824/1422707571,v1:172.21.15.121:6825/1422707571] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:37:26.008 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.ovuzki{1:24275} state up:active seq 2 export targets 0 join_fscid=1 addr [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:37:26.008 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:37:26.008 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:37:26.008 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-02-15T18:37:26.008 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:37:26.008 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.hxoggl{-1:14446} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.38:6826/3822880235,v1:172.21.15.38:6827/3822880235] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:37:26.008 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.qgkzgn{-1:14456} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.38:6828/703030231,v1:172.21.15.38:6829/703030231] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:37:26.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.016+0000 7f3732583700 1 -- 172.21.15.38:0/847789336 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3714060fb0 msgr2=0x7f3714063470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:26.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.016+0000 7f3732583700 1 --2- 172.21.15.38:0/847789336 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3714060fb0 0x7f3714063470 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f371c004630 tx=0x7f371c009b30).stop 2024-02-15T18:37:26.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.016+0000 7f3732583700 1 -- 172.21.15.38:0/847789336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f372c0f5f60 msgr2=0x7f372c1075f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:26.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.016+0000 7f3732583700 1 --2- 172.21.15.38:0/847789336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f372c0f5f60 0x7f372c1075f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f372000b410 tx=0x7f3720006ef0).stop 2024-02-15T18:37:26.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.016+0000 7f3732583700 1 -- 172.21.15.38:0/847789336 shutdown_connections 2024-02-15T18:37:26.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.016+0000 7f3732583700 1 --2- 172.21.15.38:0/847789336 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3714060fb0 0x7f3714063470 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:26.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.016+0000 7f3732583700 1 --2- 172.21.15.38:0/847789336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f372c0f4d60 0x7f372c107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:26.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.016+0000 7f3732583700 1 --2- 172.21.15.38:0/847789336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f372c0f5f60 0x7f372c1075f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:26.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.016+0000 7f3732583700 1 -- 172.21.15.38:0/847789336 >> 172.21.15.38:0/847789336 conn(0x7f372c0f04c0 msgr2=0x7f372c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:26.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.017+0000 7f3732583700 1 -- 172.21.15.38:0/847789336 shutdown_connections 2024-02-15T18:37:26.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.017+0000 7f3732583700 1 -- 172.21.15.38:0/847789336 wait complete. 2024-02-15T18:37:26.010 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 13 2024-02-15T18:37:26.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.127+0000 7fc6c67e0700 1 -- 172.21.15.38:0/1529688408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c00f66e0 msgr2=0x7fc6c00f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:26.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.127+0000 7fc6c67e0700 1 --2- 172.21.15.38:0/1529688408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c00f66e0 0x7fc6c00f6b00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fc6b0004660 tx=0x7fc6b0009b30).stop 2024-02-15T18:37:26.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.127+0000 7fc6c67e0700 1 -- 172.21.15.38:0/1529688408 shutdown_connections 2024-02-15T18:37:26.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.127+0000 7fc6c67e0700 1 --2- 172.21.15.38:0/1529688408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6c00f7980 0x7fc6c00f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:26.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.127+0000 7fc6c67e0700 1 --2- 172.21.15.38:0/1529688408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c00f66e0 0x7fc6c00f6b00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:26.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.127+0000 7fc6c67e0700 1 -- 172.21.15.38:0/1529688408 >> 172.21.15.38:0/1529688408 conn(0x7fc6c00f1e40 msgr2=0x7fc6c00f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:26.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.127+0000 7fc6c67e0700 1 -- 172.21.15.38:0/1529688408 shutdown_connections 2024-02-15T18:37:26.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.129+0000 7fc6c67e0700 1 -- 172.21.15.38:0/1529688408 wait complete. 2024-02-15T18:37:26.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.129+0000 7fc6c67e0700 1 Processor -- start 2024-02-15T18:37:26.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.129+0000 7fc6c67e0700 1 -- start start 2024-02-15T18:37:26.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.130+0000 7fc6c67e0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c00f66e0 0x7fc6c006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:26.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.130+0000 7fc6c67e0700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6c00f7980 0x7fc6c0069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:26.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.130+0000 7fc6c67e0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c006a380 con 0x7fc6c00f66e0 2024-02-15T18:37:26.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.130+0000 7fc6c67e0700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c006a4c0 con 0x7fc6c00f7980 2024-02-15T18:37:26.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.130+0000 7fc6bffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c00f66e0 0x7fc6c006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:26.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.130+0000 7fc6bffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c00f66e0 0x7fc6c006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42878/0 (socket says 172.21.15.38:42878) 2024-02-15T18:37:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.130+0000 7fc6bffff700 1 -- 172.21.15.38:0/3879156334 learned_addr learned my addr 172.21.15.38:0/3879156334 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:37:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.130+0000 7fc6bf7fe700 1 --2- 172.21.15.38:0/3879156334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6c00f7980 0x7fc6c0069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.130+0000 7fc6bffff700 1 -- 172.21.15.38:0/3879156334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6c00f7980 msgr2=0x7fc6c0069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.130+0000 7fc6bffff700 1 --2- 172.21.15.38:0/3879156334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6c00f7980 0x7fc6c0069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.130+0000 7fc6bffff700 1 -- 172.21.15.38:0/3879156334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b0005040 con 0x7fc6c00f66e0 2024-02-15T18:37:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.130+0000 7fc6bffff700 1 --2- 172.21.15.38:0/3879156334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c00f66e0 0x7fc6c006b9a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fc6b0013fd0 tx=0x7fc6b0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.130+0000 7fc6bcff9700 1 -- 172.21.15.38:0/3879156334 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6b0018070 con 0x7fc6c00f66e0 2024-02-15T18:37:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.131+0000 7fc6c67e0700 1 -- 172.21.15.38:0/3879156334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c006a740 con 0x7fc6c00f66e0 2024-02-15T18:37:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.131+0000 7fc6bcff9700 1 -- 172.21.15.38:0/3879156334 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fc6b0007d70 con 0x7fc6c00f66e0 2024-02-15T18:37:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.131+0000 7fc6bcff9700 1 -- 172.21.15.38:0/3879156334 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6b001c620 con 0x7fc6c00f66e0 2024-02-15T18:37:26.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.131+0000 7fc6c67e0700 1 -- 172.21.15.38:0/3879156334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c006aca0 con 0x7fc6c00f66e0 2024-02-15T18:37:26.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.132+0000 7fc6c67e0700 1 -- 172.21.15.38:0/3879156334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c0045ad0 con 0x7fc6c00f66e0 2024-02-15T18:37:26.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.132+0000 7fc6bcff9700 1 -- 172.21.15.38:0/3879156334 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 30) v1 ==== 77246+0+0 (secure 0 0 0) 0x7fc6b001d420 con 0x7fc6c00f66e0 2024-02-15T18:37:26.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.133+0000 7fc6bcff9700 1 --2- 172.21.15.38:0/3879156334 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc6a8060fb0 0x7fc6a8063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:26.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.133+0000 7fc6bcff9700 1 -- 172.21.15.38:0/3879156334 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fc6b007e060 con 0x7fc6c00f66e0 2024-02-15T18:37:26.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.133+0000 7fc6bf7fe700 1 --2- 172.21.15.38:0/3879156334 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc6a8060fb0 0x7fc6a8063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:26.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.133+0000 7fc6bf7fe700 1 --2- 172.21.15.38:0/3879156334 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc6a8060fb0 0x7fc6a8063470 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fc6b4004a70 tx=0x7fc6b4009040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:26.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.140+0000 7fc6bcff9700 1 -- 172.21.15.38:0/3879156334 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc6b004e450 con 0x7fc6c00f66e0 2024-02-15T18:37:26.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.334+0000 7fc6c67e0700 1 -- 172.21.15.38:0/3879156334 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6c00ff130 con 0x7fc6a8060fb0 2024-02-15T18:37:26.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.335+0000 7fc6bcff9700 1 -- 172.21.15.38:0/3879156334 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc6c00ff130 con 0x7fc6a8060fb0 2024-02-15T18:37:26.326 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:37:26.326 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:37:26.326 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-02-15T18:37:26.326 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:37:26.326 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-02-15T18:37:26.326 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-02-15T18:37:26.326 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-02-15T18:37:26.326 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr" 2024-02-15T18:37:26.326 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-02-15T18:37:26.326 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "6/21 daemons upgraded", 2024-02-15T18:37:26.326 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading osd daemons", 2024-02-15T18:37:26.326 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-02-15T18:37:26.327 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:37:26.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.337+0000 7fc6c67e0700 1 -- 172.21.15.38:0/3879156334 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc6a8060fb0 msgr2=0x7fc6a8063470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:26.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.337+0000 7fc6c67e0700 1 --2- 172.21.15.38:0/3879156334 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc6a8060fb0 0x7fc6a8063470 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fc6b4004a70 tx=0x7fc6b4009040).stop 2024-02-15T18:37:26.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.337+0000 7fc6c67e0700 1 -- 172.21.15.38:0/3879156334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c00f66e0 msgr2=0x7fc6c006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:26.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.337+0000 7fc6c67e0700 1 --2- 172.21.15.38:0/3879156334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c00f66e0 0x7fc6c006b9a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fc6b0013fd0 tx=0x7fc6b0009b30).stop 2024-02-15T18:37:26.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.337+0000 7fc6c67e0700 1 -- 172.21.15.38:0/3879156334 shutdown_connections 2024-02-15T18:37:26.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.337+0000 7fc6c67e0700 1 --2- 172.21.15.38:0/3879156334 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc6a8060fb0 0x7fc6a8063470 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:26.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.337+0000 7fc6c67e0700 1 --2- 172.21.15.38:0/3879156334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6c00f66e0 0x7fc6c006b9a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:26.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.337+0000 7fc6c67e0700 1 --2- 172.21.15.38:0/3879156334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6c00f7980 0x7fc6c0069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:26.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.337+0000 7fc6c67e0700 1 -- 172.21.15.38:0/3879156334 >> 172.21.15.38:0/3879156334 conn(0x7fc6c00f1e40 msgr2=0x7fc6c00fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:26.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.337+0000 7fc6c67e0700 1 -- 172.21.15.38:0/3879156334 shutdown_connections 2024-02-15T18:37:26.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:26.337+0000 7fc6c67e0700 1 -- 172.21.15.38:0/3879156334 wait complete. 2024-02-15T18:37:26.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:24.986777+0000 mgr.smithi038.khesbl (mgr.14664) 75 : audit [DBG] from='client.34128 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:37:26.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:25.044508+0000 mon.smithi038 (mon.0) 93 : cluster [WRN] Health check failed: Degraded data redundancy: 65/342 objects degraded (19.006%), 18 pgs degraded (PG_DEGRADED) 2024-02-15T18:37:26.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:25.307778+0000 mgr.smithi038.khesbl (mgr.14664) 76 : audit [DBG] from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:37:26.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:25.686850+0000 mon.smithi038 (mon.0) 94 : audit [DBG] from='client.? 172.21.15.38:0/2042597465' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:37:26.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:26.016312+0000 mon.smithi038 (mon.0) 95 : audit [DBG] from='client.? 172.21.15.38:0/847789336' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:37:26.439 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:24.986777+0000 mgr.smithi038.khesbl (mgr.14664) 75 : audit [DBG] from='client.34128 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:37:26.439 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:25.044508+0000 mon.smithi038 (mon.0) 93 : cluster [WRN] Health check failed: Degraded data redundancy: 65/342 objects degraded (19.006%), 18 pgs degraded (PG_DEGRADED) 2024-02-15T18:37:26.439 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:25.307778+0000 mgr.smithi038.khesbl (mgr.14664) 76 : audit [DBG] from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:37:26.439 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:25.686850+0000 mon.smithi038 (mon.0) 94 : audit [DBG] from='client.? 172.21.15.38:0/2042597465' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:37:26.440 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:26.016312+0000 mon.smithi038 (mon.0) 95 : audit [DBG] from='client.? 172.21.15.38:0/847789336' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:37:27.022 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:26 smithi038 bash[81278]: 623ac7ecbb0a90978fddbb11079664f26638a7293ac37b6f7840b4bc10c7c5d3 2024-02-15T18:37:27.022 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:26 smithi038 systemd[1]: Started Ceph osd.0 for 7e05753e-cc2f-11ee-95ba-87774f69a715. 2024-02-15T18:37:27.441 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-0[81296]: debug 2024-02-15T18:37:27.118+0000 7f7787cfb380 -1 Falling back to public interface 2024-02-15T18:37:28.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:26.336431+0000 mgr.smithi038.khesbl (mgr.14664) 77 : audit [DBG] from='client.34144 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:37:28.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:26.738327+0000 mon.smithi038 (mon.0) 96 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:28.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:26.739061+0000 mon.smithi038 (mon.0) 97 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:37:28.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:26.940908+0000 mgr.smithi038.khesbl (mgr.14664) 78 : cluster [DBG] pgmap v48: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 65/342 objects degraded (19.006%) 2024-02-15T18:37:28.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:26.336431+0000 mgr.smithi038.khesbl (mgr.14664) 77 : audit [DBG] from='client.34144 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:37:28.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:26.738327+0000 mon.smithi038 (mon.0) 96 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:28.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:26.739061+0000 mon.smithi038 (mon.0) 97 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:37:28.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:26.940908+0000 mgr.smithi038.khesbl (mgr.14664) 78 : cluster [DBG] pgmap v48: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 65/342 objects degraded (19.006%) 2024-02-15T18:37:29.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:28.941371+0000 mgr.smithi038.khesbl (mgr.14664) 79 : cluster [DBG] pgmap v49: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 65/342 objects degraded (19.006%) 2024-02-15T18:37:29.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:28.941371+0000 mgr.smithi038.khesbl (mgr.14664) 79 : cluster [DBG] pgmap v49: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 65/342 objects degraded (19.006%) 2024-02-15T18:37:31.260 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:30.941852+0000 mgr.smithi038.khesbl (mgr.14664) 80 : cluster [DBG] pgmap v50: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 65/342 objects degraded (19.006%) 2024-02-15T18:37:31.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:30.941852+0000 mgr.smithi038.khesbl (mgr.14664) 80 : cluster [DBG] pgmap v50: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 65/342 objects degraded (19.006%) 2024-02-15T18:37:34.022 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-0[81296]: debug 2024-02-15T18:37:33.848+0000 7f7787cfb380 -1 osd.0 46 log_to_monitors {default=true} 2024-02-15T18:37:34.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:32.768642+0000 mon.smithi038 (mon.0) 98 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:34.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:32.769413+0000 mon.smithi038 (mon.0) 99 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:37:34.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:32.770083+0000 mon.smithi038 (mon.0) 100 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:37:34.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:37:32.770639+0000 mgr.smithi038.khesbl (mgr.14664) 81 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:37:34.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:32.773548+0000 mon.smithi038 (mon.0) 101 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:34.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:32.778192+0000 mon.smithi038 (mon.0) 102 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:37:34.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:32.780130+0000 mon.smithi038 (mon.0) 103 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:37:34.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:32.781221+0000 mon.smithi038 (mon.0) 104 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:37:34.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:32.782279+0000 mon.smithi038 (mon.0) 105 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:37:34.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:32.783553+0000 mon.smithi038 (mon.0) 106 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:37:34.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:32.783694+0000 mgr.smithi038.khesbl (mgr.14664) 82 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:37:34.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:37:32.784312+0000 mgr.smithi038.khesbl (mgr.14664) 83 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-02-15T18:37:34.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:32.942276+0000 mgr.smithi038.khesbl (mgr.14664) 84 : cluster [DBG] pgmap v51: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 65/342 objects degraded (19.006%) 2024-02-15T18:37:34.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:33.150813+0000 mon.smithi038 (mon.0) 107 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:34.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:32.768642+0000 mon.smithi038 (mon.0) 98 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:34.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:32.769413+0000 mon.smithi038 (mon.0) 99 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:37:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:32.770083+0000 mon.smithi038 (mon.0) 100 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:37:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:37:32.770639+0000 mgr.smithi038.khesbl (mgr.14664) 81 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:37:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:32.773548+0000 mon.smithi038 (mon.0) 101 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:32.778192+0000 mon.smithi038 (mon.0) 102 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:37:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:32.780130+0000 mon.smithi038 (mon.0) 103 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:37:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:32.781221+0000 mon.smithi038 (mon.0) 104 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:37:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:32.782279+0000 mon.smithi038 (mon.0) 105 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:37:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:32.783553+0000 mon.smithi038 (mon.0) 106 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:37:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:32.783694+0000 mgr.smithi038.khesbl (mgr.14664) 82 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:37:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:37:32.784312+0000 mgr.smithi038.khesbl (mgr.14664) 83 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-02-15T18:37:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:32.942276+0000 mgr.smithi038.khesbl (mgr.14664) 84 : cluster [DBG] pgmap v51: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 65/342 objects degraded (19.006%) 2024-02-15T18:37:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:33.150813+0000 mon.smithi038 (mon.0) 107 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:35.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:33.853723+0000 mon.smithi038 (mon.0) 108 : audit [INF] from='osd.0 [v2:172.21.15.38:6802/4082243527,v1:172.21.15.38:6803/4082243527]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-15T18:37:35.023 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:37:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-0[81296]: debug 2024-02-15T18:37:34.790+0000 7f77799e3700 -1 osd.0 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-15T18:37:35.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:33.853723+0000 mon.smithi038 (mon.0) 108 : audit [INF] from='osd.0 [v2:172.21.15.38:6802/4082243527,v1:172.21.15.38:6803/4082243527]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-15T18:37:36.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:34.785292+0000 mon.smithi038 (mon.0) 109 : audit [INF] from='osd.0 [v2:172.21.15.38:6802/4082243527,v1:172.21.15.38:6803/4082243527]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-15T18:37:36.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:34.785347+0000 mon.smithi038 (mon.0) 110 : cluster [DBG] osdmap e49: 6 total, 5 up, 6 in 2024-02-15T18:37:36.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:34.785931+0000 mon.smithi038 (mon.0) 111 : audit [INF] from='osd.0 [v2:172.21.15.38:6802/4082243527,v1:172.21.15.38:6803/4082243527]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-02-15T18:37:36.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:34.942761+0000 mgr.smithi038.khesbl (mgr.14664) 85 : cluster [DBG] pgmap v53: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 614 B/s wr, 0 op/s; 65/342 objects degraded (19.006%) 2024-02-15T18:37:36.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:34.785292+0000 mon.smithi038 (mon.0) 109 : audit [INF] from='osd.0 [v2:172.21.15.38:6802/4082243527,v1:172.21.15.38:6803/4082243527]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-15T18:37:36.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:34.785347+0000 mon.smithi038 (mon.0) 110 : cluster [DBG] osdmap e49: 6 total, 5 up, 6 in 2024-02-15T18:37:36.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:34.785931+0000 mon.smithi038 (mon.0) 111 : audit [INF] from='osd.0 [v2:172.21.15.38:6802/4082243527,v1:172.21.15.38:6803/4082243527]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-02-15T18:37:36.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:34.942761+0000 mgr.smithi038.khesbl (mgr.14664) 85 : cluster [DBG] pgmap v53: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 614 B/s wr, 0 op/s; 65/342 objects degraded (19.006%) 2024-02-15T18:37:37.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:36 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:35.783975+0000 mon.smithi038 (mon.0) 112 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:37:37.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:36 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:35.799816+0000 mon.smithi038 (mon.0) 113 : cluster [INF] osd.0 [v2:172.21.15.38:6802/4082243527,v1:172.21.15.38:6803/4082243527] boot 2024-02-15T18:37:37.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:36 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:35.799900+0000 mon.smithi038 (mon.0) 114 : cluster [DBG] osdmap e50: 6 total, 6 up, 6 in 2024-02-15T18:37:37.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:36 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:35.800633+0000 mon.smithi038 (mon.0) 115 : cluster [DBG] mgrmap e31: smithi038.khesbl(active, since 93s), standbys: smithi121.ckiryi 2024-02-15T18:37:37.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:36 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:35.801275+0000 mon.smithi038 (mon.0) 116 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:37:37.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:35.783975+0000 mon.smithi038 (mon.0) 112 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:37:37.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:35.799816+0000 mon.smithi038 (mon.0) 113 : cluster [INF] osd.0 [v2:172.21.15.38:6802/4082243527,v1:172.21.15.38:6803/4082243527] boot 2024-02-15T18:37:37.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:35.799900+0000 mon.smithi038 (mon.0) 114 : cluster [DBG] osdmap e50: 6 total, 6 up, 6 in 2024-02-15T18:37:37.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:35.800633+0000 mon.smithi038 (mon.0) 115 : cluster [DBG] mgrmap e31: smithi038.khesbl(active, since 93s), standbys: smithi121.ckiryi 2024-02-15T18:37:37.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:35.801275+0000 mon.smithi038 (mon.0) 116 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:37:38.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:36.794752+0000 mon.smithi038 (mon.0) 117 : cluster [DBG] osdmap e51: 6 total, 6 up, 6 in 2024-02-15T18:37:38.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:36.943128+0000 mgr.smithi038.khesbl (mgr.14664) 86 : cluster [DBG] pgmap v56: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s; 65/342 objects degraded (19.006%) 2024-02-15T18:37:38.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:36.794752+0000 mon.smithi038 (mon.0) 117 : cluster [DBG] osdmap e51: 6 total, 6 up, 6 in 2024-02-15T18:37:38.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:36.943128+0000 mgr.smithi038.khesbl (mgr.14664) 86 : cluster [DBG] pgmap v56: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s; 65/342 objects degraded (19.006%) 2024-02-15T18:37:39.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:38.943668+0000 mgr.smithi038.khesbl (mgr.14664) 87 : cluster [DBG] pgmap v57: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s; 65/342 objects degraded (19.006%) 2024-02-15T18:37:39.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:38.943668+0000 mgr.smithi038.khesbl (mgr.14664) 87 : cluster [DBG] pgmap v57: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s; 65/342 objects degraded (19.006%) 2024-02-15T18:37:41.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:40.944187+0000 mgr.smithi038.khesbl (mgr.14664) 88 : cluster [DBG] pgmap v58: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 91 B/s, 0 objects/s recovering 2024-02-15T18:37:41.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:40.994933+0000 mon.smithi038 (mon.0) 118 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 65/342 objects degraded (19.006%), 18 pgs degraded) 2024-02-15T18:37:41.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:40.994994+0000 mon.smithi038 (mon.0) 119 : cluster [INF] Cluster is now healthy 2024-02-15T18:37:41.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:40.944187+0000 mgr.smithi038.khesbl (mgr.14664) 88 : cluster [DBG] pgmap v58: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 91 B/s, 0 objects/s recovering 2024-02-15T18:37:41.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:40.994933+0000 mon.smithi038 (mon.0) 118 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 65/342 objects degraded (19.006%), 18 pgs degraded) 2024-02-15T18:37:41.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:40.994994+0000 mon.smithi038 (mon.0) 119 : cluster [INF] Cluster is now healthy 2024-02-15T18:37:43.498 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:42.944638+0000 mgr.smithi038.khesbl (mgr.14664) 89 : cluster [DBG] pgmap v59: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 70 B/s, 0 objects/s recovering 2024-02-15T18:37:43.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:42.944638+0000 mgr.smithi038.khesbl (mgr.14664) 89 : cluster [DBG] pgmap v59: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 70 B/s, 0 objects/s recovering 2024-02-15T18:37:45.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:44.945216+0000 mgr.smithi038.khesbl (mgr.14664) 90 : cluster [DBG] pgmap v60: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 61 B/s, 0 objects/s recovering 2024-02-15T18:37:45.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:44.945216+0000 mgr.smithi038.khesbl (mgr.14664) 90 : cluster [DBG] pgmap v60: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 61 B/s, 0 objects/s recovering 2024-02-15T18:37:47.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:46.945646+0000 mgr.smithi038.khesbl (mgr.14664) 91 : cluster [DBG] pgmap v61: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 55 B/s, 0 objects/s recovering 2024-02-15T18:37:47.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:46.945646+0000 mgr.smithi038.khesbl (mgr.14664) 91 : cluster [DBG] pgmap v61: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 55 B/s, 0 objects/s recovering 2024-02-15T18:37:48.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:47.798052+0000 mon.smithi038 (mon.0) 120 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:37:48.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:47.798290+0000 mgr.smithi038.khesbl (mgr.14664) 92 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:37:48.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:37:47.799379+0000 mgr.smithi038.khesbl (mgr.14664) 93 : cephadm [INF] Upgrade: osd.1 is safe to restart 2024-02-15T18:37:48.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:48.147195+0000 mon.smithi038 (mon.0) 121 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:48.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:48.163878+0000 mon.smithi038 (mon.0) 122 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:48.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:47.798052+0000 mon.smithi038 (mon.0) 120 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:37:48.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:47.798290+0000 mgr.smithi038.khesbl (mgr.14664) 92 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:37:48.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:37:47.799379+0000 mgr.smithi038.khesbl (mgr.14664) 93 : cephadm [INF] Upgrade: osd.1 is safe to restart 2024-02-15T18:37:48.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:48.147195+0000 mon.smithi038 (mon.0) 121 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:48.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:48.163878+0000 mon.smithi038 (mon.0) 122 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:49.474 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:48.945944+0000 mgr.smithi038.khesbl (mgr.14664) 94 : cluster [DBG] pgmap v62: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46 B/s, 0 objects/s recovering 2024-02-15T18:37:49.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:48.945944+0000 mgr.smithi038.khesbl (mgr.14664) 94 : cluster [DBG] pgmap v62: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46 B/s, 0 objects/s recovering 2024-02-15T18:37:50.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:37:49.505451+0000 mgr.smithi038.khesbl (mgr.14664) 95 : cephadm [INF] Upgrade: Updating osd.1 2024-02-15T18:37:50.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:49.513799+0000 mon.smithi038 (mon.0) 123 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:50.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:49.514638+0000 mon.smithi038 (mon.0) 124 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-15T18:37:50.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:49.515269+0000 mon.smithi038 (mon.0) 125 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:37:50.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:37:49.516269+0000 mgr.smithi038.khesbl (mgr.14664) 96 : cephadm [INF] Deploying daemon osd.1 on smithi038 2024-02-15T18:37:50.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:37:49.505451+0000 mgr.smithi038.khesbl (mgr.14664) 95 : cephadm [INF] Upgrade: Updating osd.1 2024-02-15T18:37:50.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:49.513799+0000 mon.smithi038 (mon.0) 123 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:50.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:49.514638+0000 mon.smithi038 (mon.0) 124 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-15T18:37:50.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:49.515269+0000 mon.smithi038 (mon.0) 125 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:37:50.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:37:49.516269+0000 mgr.smithi038.khesbl (mgr.14664) 96 : cephadm [INF] Deploying daemon osd.1 on smithi038 2024-02-15T18:37:51.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:50.946707+0000 mgr.smithi038.khesbl (mgr.14664) 97 : cluster [DBG] pgmap v63: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46 B/s, 0 objects/s recovering 2024-02-15T18:37:51.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:50.946707+0000 mgr.smithi038.khesbl (mgr.14664) 97 : cluster [DBG] pgmap v63: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46 B/s, 0 objects/s recovering 2024-02-15T18:37:53.022 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:52 smithi038 systemd[1]: Stopping Ceph osd.1 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:37:53.412 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:52.947202+0000 mgr.smithi038.khesbl (mgr.14664) 98 : cluster [DBG] pgmap v64: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:37:53.412 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.1[52470]: debug 2024-02-15T18:37:53.061+0000 7fb66cea8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:37:53.413 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.1[52470]: debug 2024-02-15T18:37:53.061+0000 7fb66cea8700 -1 osd.1 51 *** Got signal Terminated *** 2024-02-15T18:37:53.413 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.1[52470]: debug 2024-02-15T18:37:53.061+0000 7fb66cea8700 -1 osd.1 51 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:37:53.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:52.947202+0000 mgr.smithi038.khesbl (mgr.14664) 98 : cluster [DBG] pgmap v64: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:37:53.742 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:53 smithi038 bash[83258]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.1 2024-02-15T18:37:54.022 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:53 smithi038 bash[83310]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.1 2024-02-15T18:37:54.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.319990+0000 mon.smithi038 (mon.0) 126 : cluster [DBG] osd.1 reported immediately failed by osd.2 2024-02-15T18:37:54.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.320017+0000 mon.smithi038 (mon.0) 127 : cluster [INF] osd.1 failed (root=default,host=smithi038) (connection refused reported by osd.2) 2024-02-15T18:37:54.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.320137+0000 mon.smithi038 (mon.0) 128 : cluster [DBG] osd.1 reported immediately failed by osd.0 2024-02-15T18:37:54.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.320206+0000 mon.smithi038 (mon.0) 129 : cluster [DBG] osd.1 reported immediately failed by osd.2 2024-02-15T18:37:54.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.320279+0000 mon.smithi038 (mon.0) 130 : cluster [DBG] osd.1 reported immediately failed by osd.0 2024-02-15T18:37:54.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.320553+0000 mon.smithi038 (mon.0) 131 : cluster [DBG] osd.1 reported immediately failed by osd.4 2024-02-15T18:37:54.508 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.320597+0000 mon.smithi038 (mon.0) 132 : cluster [DBG] osd.1 reported immediately failed by osd.4 2024-02-15T18:37:54.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.320705+0000 mon.smithi038 (mon.0) 133 : cluster [DBG] osd.1 reported immediately failed by osd.3 2024-02-15T18:37:54.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.320778+0000 mon.smithi038 (mon.0) 134 : cluster [DBG] osd.1 reported immediately failed by osd.5 2024-02-15T18:37:54.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.320819+0000 mon.smithi038 (mon.0) 135 : cluster [DBG] osd.1 reported immediately failed by osd.3 2024-02-15T18:37:54.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.320916+0000 mon.smithi038 (mon.0) 136 : cluster [DBG] osd.1 reported immediately failed by osd.5 2024-02-15T18:37:54.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.720807+0000 mon.smithi038 (mon.0) 137 : cluster [DBG] osd.1 reported immediately failed by osd.0 2024-02-15T18:37:54.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.720904+0000 mon.smithi038 (mon.0) 138 : cluster [DBG] osd.1 reported immediately failed by osd.2 2024-02-15T18:37:54.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.720976+0000 mon.smithi038 (mon.0) 139 : cluster [DBG] osd.1 reported immediately failed by osd.0 2024-02-15T18:37:54.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.721047+0000 mon.smithi038 (mon.0) 140 : cluster [DBG] osd.1 reported immediately failed by osd.2 2024-02-15T18:37:54.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.721620+0000 mon.smithi038 (mon.0) 141 : cluster [DBG] osd.1 reported immediately failed by osd.5 2024-02-15T18:37:54.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.721695+0000 mon.smithi038 (mon.0) 142 : cluster [DBG] osd.1 reported immediately failed by osd.5 2024-02-15T18:37:54.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.721763+0000 mon.smithi038 (mon.0) 143 : cluster [DBG] osd.1 reported immediately failed by osd.3 2024-02-15T18:37:54.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.721843+0000 mon.smithi038 (mon.0) 144 : cluster [DBG] osd.1 reported immediately failed by osd.4 2024-02-15T18:37:54.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.721913+0000 mon.smithi038 (mon.0) 145 : cluster [DBG] osd.1 reported immediately failed by osd.3 2024-02-15T18:37:54.509 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:53.721989+0000 mon.smithi038 (mon.0) 146 : cluster [DBG] osd.1 reported immediately failed by osd.4 2024-02-15T18:37:54.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.319990+0000 mon.smithi038 (mon.0) 126 : cluster [DBG] osd.1 reported immediately failed by osd.2 2024-02-15T18:37:54.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.320017+0000 mon.smithi038 (mon.0) 127 : cluster [INF] osd.1 failed (root=default,host=smithi038) (connection refused reported by osd.2) 2024-02-15T18:37:54.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.320137+0000 mon.smithi038 (mon.0) 128 : cluster [DBG] osd.1 reported immediately failed by osd.0 2024-02-15T18:37:54.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.320206+0000 mon.smithi038 (mon.0) 129 : cluster [DBG] osd.1 reported immediately failed by osd.2 2024-02-15T18:37:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.320279+0000 mon.smithi038 (mon.0) 130 : cluster [DBG] osd.1 reported immediately failed by osd.0 2024-02-15T18:37:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.320553+0000 mon.smithi038 (mon.0) 131 : cluster [DBG] osd.1 reported immediately failed by osd.4 2024-02-15T18:37:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.320597+0000 mon.smithi038 (mon.0) 132 : cluster [DBG] osd.1 reported immediately failed by osd.4 2024-02-15T18:37:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.320705+0000 mon.smithi038 (mon.0) 133 : cluster [DBG] osd.1 reported immediately failed by osd.3 2024-02-15T18:37:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.320778+0000 mon.smithi038 (mon.0) 134 : cluster [DBG] osd.1 reported immediately failed by osd.5 2024-02-15T18:37:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.320819+0000 mon.smithi038 (mon.0) 135 : cluster [DBG] osd.1 reported immediately failed by osd.3 2024-02-15T18:37:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.320916+0000 mon.smithi038 (mon.0) 136 : cluster [DBG] osd.1 reported immediately failed by osd.5 2024-02-15T18:37:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.720807+0000 mon.smithi038 (mon.0) 137 : cluster [DBG] osd.1 reported immediately failed by osd.0 2024-02-15T18:37:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.720904+0000 mon.smithi038 (mon.0) 138 : cluster [DBG] osd.1 reported immediately failed by osd.2 2024-02-15T18:37:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.720976+0000 mon.smithi038 (mon.0) 139 : cluster [DBG] osd.1 reported immediately failed by osd.0 2024-02-15T18:37:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.721047+0000 mon.smithi038 (mon.0) 140 : cluster [DBG] osd.1 reported immediately failed by osd.2 2024-02-15T18:37:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.721620+0000 mon.smithi038 (mon.0) 141 : cluster [DBG] osd.1 reported immediately failed by osd.5 2024-02-15T18:37:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.721695+0000 mon.smithi038 (mon.0) 142 : cluster [DBG] osd.1 reported immediately failed by osd.5 2024-02-15T18:37:54.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.721763+0000 mon.smithi038 (mon.0) 143 : cluster [DBG] osd.1 reported immediately failed by osd.3 2024-02-15T18:37:54.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.721843+0000 mon.smithi038 (mon.0) 144 : cluster [DBG] osd.1 reported immediately failed by osd.4 2024-02-15T18:37:54.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.721913+0000 mon.smithi038 (mon.0) 145 : cluster [DBG] osd.1 reported immediately failed by osd.3 2024-02-15T18:37:54.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:53.721989+0000 mon.smithi038 (mon.0) 146 : cluster [DBG] osd.1 reported immediately failed by osd.4 2024-02-15T18:37:55.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:54.244521+0000 mon.smithi038 (mon.0) 147 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:37:55.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:54.264398+0000 mon.smithi038 (mon.0) 148 : cluster [DBG] osdmap e52: 6 total, 5 up, 6 in 2024-02-15T18:37:55.277 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:54.947579+0000 mgr.smithi038.khesbl (mgr.14664) 99 : cluster [DBG] pgmap v66: 65 pgs: 5 peering, 9 stale+active+clean, 51 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:37:55.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:54.244521+0000 mon.smithi038 (mon.0) 147 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:37:55.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:54.264398+0000 mon.smithi038 (mon.0) 148 : cluster [DBG] osdmap e52: 6 total, 5 up, 6 in 2024-02-15T18:37:55.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:54.947579+0000 mgr.smithi038.khesbl (mgr.14664) 99 : cluster [DBG] pgmap v66: 65 pgs: 5 peering, 9 stale+active+clean, 51 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:37:56.242 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:55 smithi038 systemd[1]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.1.service: Succeeded. 2024-02-15T18:37:56.243 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:55 smithi038 systemd[1]: Stopped Ceph osd.1 for 7e05753e-cc2f-11ee-95ba-87774f69a715. 2024-02-15T18:37:56.243 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:55 smithi038 systemd[1]: Starting Ceph osd.1 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:37:56.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.447+0000 7f8b4e52a700 1 -- 172.21.15.38:0/388272296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b480f55c0 msgr2=0x7f8b480f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:56.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.447+0000 7f8b4e52a700 1 --2- 172.21.15.38:0/388272296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b480f55c0 0x7f8b480f59e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f8b38004660 tx=0x7f8b38009b30).stop 2024-02-15T18:37:56.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.447+0000 7f8b4e52a700 1 -- 172.21.15.38:0/388272296 shutdown_connections 2024-02-15T18:37:56.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.447+0000 7f8b4e52a700 1 --2- 172.21.15.38:0/388272296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b480f6860 0x7f8b480f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.447+0000 7f8b4e52a700 1 --2- 172.21.15.38:0/388272296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b480f55c0 0x7f8b480f59e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.447+0000 7f8b4e52a700 1 -- 172.21.15.38:0/388272296 >> 172.21.15.38:0/388272296 conn(0x7f8b480f0d40 msgr2=0x7f8b480f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.447+0000 7f8b4e52a700 1 -- 172.21.15.38:0/388272296 shutdown_connections 2024-02-15T18:37:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.447+0000 7f8b4e52a700 1 -- 172.21.15.38:0/388272296 wait complete. 2024-02-15T18:37:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.448+0000 7f8b4e52a700 1 Processor -- start 2024-02-15T18:37:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.448+0000 7f8b4e52a700 1 -- start start 2024-02-15T18:37:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.448+0000 7f8b4e52a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b480f55c0 0x7f8b480f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.448+0000 7f8b4e52a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b480f6860 0x7f8b480f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.448+0000 7f8b4e52a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b480f8100 con 0x7f8b480f6860 2024-02-15T18:37:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.448+0000 7f8b4e52a700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b480f8240 con 0x7f8b480f55c0 2024-02-15T18:37:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.448+0000 7f8b477fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b480f6860 0x7f8b480f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:56.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.448+0000 7f8b477fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b480f6860 0x7f8b480f7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52680/0 (socket says 172.21.15.38:52680) 2024-02-15T18:37:56.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.448+0000 7f8b477fe700 1 -- 172.21.15.38:0/2806753872 learned_addr learned my addr 172.21.15.38:0/2806753872 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:37:56.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.448+0000 7f8b47fff700 1 --2- 172.21.15.38:0/2806753872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b480f55c0 0x7f8b480f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:56.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.449+0000 7f8b477fe700 1 -- 172.21.15.38:0/2806753872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b480f55c0 msgr2=0x7f8b480f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:56.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.449+0000 7f8b477fe700 1 --2- 172.21.15.38:0/2806753872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b480f55c0 0x7f8b480f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:56.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.449+0000 7f8b477fe700 1 -- 172.21.15.38:0/2806753872 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b38005040 con 0x7f8b480f6860 2024-02-15T18:37:56.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.449+0000 7f8b477fe700 1 --2- 172.21.15.38:0/2806753872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b480f6860 0x7f8b480f7ba0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f8b3c0026d0 tx=0x7f8b3c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:56.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.449+0000 7f8b44ff9700 1 -- 172.21.15.38:0/2806753872 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b3c0079c0 con 0x7f8b480f6860 2024-02-15T18:37:56.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.449+0000 7f8b4e52a700 1 -- 172.21.15.38:0/2806753872 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b480f8520 con 0x7f8b480f6860 2024-02-15T18:37:56.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.449+0000 7f8b44ff9700 1 -- 172.21.15.38:0/2806753872 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f8b3c00f040 con 0x7f8b480f6860 2024-02-15T18:37:56.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.449+0000 7f8b44ff9700 1 -- 172.21.15.38:0/2806753872 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b3c0149f0 con 0x7f8b480f6860 2024-02-15T18:37:56.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.449+0000 7f8b4e52a700 1 -- 172.21.15.38:0/2806753872 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4810f020 con 0x7f8b480f6860 2024-02-15T18:37:56.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.450+0000 7f8b4e52a700 1 -- 172.21.15.38:0/2806753872 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b48045ad0 con 0x7f8b480f6860 2024-02-15T18:37:56.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.451+0000 7f8b44ff9700 1 -- 172.21.15.38:0/2806753872 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f8b3c007b20 con 0x7f8b480f6860 2024-02-15T18:37:56.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.451+0000 7f8b44ff9700 1 --2- 172.21.15.38:0/2806753872 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8b30060f20 0x7f8b300633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:56.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.451+0000 7f8b44ff9700 1 -- 172.21.15.38:0/2806753872 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f8b3c07c0e0 con 0x7f8b480f6860 2024-02-15T18:37:56.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.451+0000 7f8b47fff700 1 --2- 172.21.15.38:0/2806753872 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8b30060f20 0x7f8b300633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:56.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.451+0000 7f8b47fff700 1 --2- 172.21.15.38:0/2806753872 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8b30060f20 0x7f8b300633e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f8b480f94b0 tx=0x7f8b38009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:56.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.455+0000 7f8b44ff9700 1 -- 172.21.15.38:0/2806753872 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8b3c04c560 con 0x7f8b480f6860 2024-02-15T18:37:56.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:55.257811+0000 mon.smithi038 (mon.0) 149 : cluster [WRN] Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-02-15T18:37:56.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:55.263399+0000 mon.smithi038 (mon.0) 150 : cluster [DBG] osdmap e53: 6 total, 5 up, 6 in 2024-02-15T18:37:56.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:55.257811+0000 mon.smithi038 (mon.0) 149 : cluster [WRN] Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-02-15T18:37:56.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:55.263399+0000 mon.smithi038 (mon.0) 150 : cluster [DBG] osdmap e53: 6 total, 5 up, 6 in 2024-02-15T18:37:56.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.649+0000 7f8b4e52a700 1 -- 172.21.15.38:0/2806753872 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b480faca0 con 0x7f8b30060f20 2024-02-15T18:37:56.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.650+0000 7f8b44ff9700 1 -- 172.21.15.38:0/2806753872 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8b480faca0 con 0x7f8b30060f20 2024-02-15T18:37:56.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.652+0000 7f8b4e52a700 1 -- 172.21.15.38:0/2806753872 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8b30060f20 msgr2=0x7f8b300633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:56.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.652+0000 7f8b4e52a700 1 --2- 172.21.15.38:0/2806753872 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8b30060f20 0x7f8b300633e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f8b480f94b0 tx=0x7f8b38009b30).stop 2024-02-15T18:37:56.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.652+0000 7f8b4e52a700 1 -- 172.21.15.38:0/2806753872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b480f6860 msgr2=0x7f8b480f7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:56.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.652+0000 7f8b4e52a700 1 --2- 172.21.15.38:0/2806753872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b480f6860 0x7f8b480f7ba0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f8b3c0026d0 tx=0x7f8b3c009de0).stop 2024-02-15T18:37:56.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.652+0000 7f8b4e52a700 1 -- 172.21.15.38:0/2806753872 shutdown_connections 2024-02-15T18:37:56.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.652+0000 7f8b4e52a700 1 --2- 172.21.15.38:0/2806753872 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8b30060f20 0x7f8b300633e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:56.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.652+0000 7f8b4e52a700 1 --2- 172.21.15.38:0/2806753872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b480f55c0 0x7f8b480f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:56.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.652+0000 7f8b4e52a700 1 --2- 172.21.15.38:0/2806753872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8b480f6860 0x7f8b480f7ba0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:56.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.652+0000 7f8b4e52a700 1 -- 172.21.15.38:0/2806753872 >> 172.21.15.38:0/2806753872 conn(0x7f8b480f0d40 msgr2=0x7f8b4805fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:56.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.652+0000 7f8b4e52a700 1 -- 172.21.15.38:0/2806753872 shutdown_connections 2024-02-15T18:37:56.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.652+0000 7f8b4e52a700 1 -- 172.21.15.38:0/2806753872 wait complete. 2024-02-15T18:37:56.652 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-02-15T18:37:56.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.758+0000 7f5024bb4700 1 -- 172.21.15.38:0/767513075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50200f55c0 msgr2=0x7f50200f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:56.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.758+0000 7f5024bb4700 1 --2- 172.21.15.38:0/767513075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50200f55c0 0x7f50200f59e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f5010004660 tx=0x7f5010009b30).stop 2024-02-15T18:37:56.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.758+0000 7f5024bb4700 1 -- 172.21.15.38:0/767513075 shutdown_connections 2024-02-15T18:37:56.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.758+0000 7f5024bb4700 1 --2- 172.21.15.38:0/767513075 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50200f6860 0x7f50200f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:56.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.758+0000 7f5024bb4700 1 --2- 172.21.15.38:0/767513075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50200f55c0 0x7f50200f59e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:56.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.758+0000 7f5024bb4700 1 -- 172.21.15.38:0/767513075 >> 172.21.15.38:0/767513075 conn(0x7f50200f0d40 msgr2=0x7f50200f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:56.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.758+0000 7f5024bb4700 1 -- 172.21.15.38:0/767513075 shutdown_connections 2024-02-15T18:37:56.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.758+0000 7f5024bb4700 1 -- 172.21.15.38:0/767513075 wait complete. 2024-02-15T18:37:56.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.759+0000 7f5024bb4700 1 Processor -- start 2024-02-15T18:37:56.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.759+0000 7f5024bb4700 1 -- start start 2024-02-15T18:37:56.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.759+0000 7f5024bb4700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50200f55c0 0x7f5020106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:56.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.759+0000 7f5024bb4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50200f6860 0x7f50201074a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:56.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.759+0000 7f5024bb4700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5020107a50 con 0x7f50200f55c0 2024-02-15T18:37:56.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.759+0000 7f5024bb4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5020107b90 con 0x7f50200f6860 2024-02-15T18:37:56.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.759+0000 7f501e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50200f55c0 0x7f5020106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:56.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.759+0000 7f501e59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50200f55c0 0x7f5020106fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52704/0 (socket says 172.21.15.38:52704) 2024-02-15T18:37:56.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.759+0000 7f501e59c700 1 -- 172.21.15.38:0/963931573 learned_addr learned my addr 172.21.15.38:0/963931573 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:37:56.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.759+0000 7f501dd9b700 1 --2- 172.21.15.38:0/963931573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50200f6860 0x7f50201074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:56.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.760+0000 7f501e59c700 1 -- 172.21.15.38:0/963931573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50200f6860 msgr2=0x7f50201074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:56.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.760+0000 7f501e59c700 1 --2- 172.21.15.38:0/963931573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50200f6860 0x7f50201074a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:56.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.760+0000 7f501e59c700 1 -- 172.21.15.38:0/963931573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5010005040 con 0x7f50200f55c0 2024-02-15T18:37:56.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.760+0000 7f501e59c700 1 --2- 172.21.15.38:0/963931573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50200f55c0 0x7f5020106fb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f5010002040 tx=0x7f501000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:56.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.760+0000 7f500effd700 1 -- 172.21.15.38:0/963931573 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5010019070 con 0x7f50200f55c0 2024-02-15T18:37:56.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.760+0000 7f5024bb4700 1 -- 172.21.15.38:0/963931573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50201084d0 con 0x7f50200f55c0 2024-02-15T18:37:56.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.760+0000 7f500effd700 1 -- 172.21.15.38:0/963931573 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f5010014420 con 0x7f50200f55c0 2024-02-15T18:37:56.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.760+0000 7f500effd700 1 -- 172.21.15.38:0/963931573 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f501001d5c0 con 0x7f50200f55c0 2024-02-15T18:37:56.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.760+0000 7f5024bb4700 1 -- 172.21.15.38:0/963931573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5020108a30 con 0x7f50200f55c0 2024-02-15T18:37:56.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.761+0000 7f5024bb4700 1 -- 172.21.15.38:0/963931573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5020045ad0 con 0x7f50200f55c0 2024-02-15T18:37:56.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.762+0000 7f500effd700 1 -- 172.21.15.38:0/963931573 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f5010025070 con 0x7f50200f55c0 2024-02-15T18:37:56.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.762+0000 7f500effd700 1 --2- 172.21.15.38:0/963931573 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5008060f20 0x7f50080633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:56.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.762+0000 7f500effd700 1 -- 172.21.15.38:0/963931573 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f501007d4b0 con 0x7f50200f55c0 2024-02-15T18:37:56.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.763+0000 7f501dd9b700 1 --2- 172.21.15.38:0/963931573 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5008060f20 0x7f50080633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:56.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.763+0000 7f501dd9b700 1 --2- 172.21.15.38:0/963931573 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5008060f20 0x7f50080633e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f5014007f80 tx=0x7f501400b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:56.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.766+0000 7f500effd700 1 -- 172.21.15.38:0/963931573 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f50100513c0 con 0x7f50200f55c0 2024-02-15T18:37:56.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.963+0000 7f5024bb4700 1 -- 172.21.15.38:0/963931573 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f50200faca0 con 0x7f5008060f20 2024-02-15T18:37:56.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.965+0000 7f500effd700 1 -- 172.21.15.38:0/963931573 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f50200faca0 con 0x7f5008060f20 2024-02-15T18:37:56.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.967+0000 7f5024bb4700 1 -- 172.21.15.38:0/963931573 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5008060f20 msgr2=0x7f50080633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:56.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.967+0000 7f5024bb4700 1 --2- 172.21.15.38:0/963931573 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5008060f20 0x7f50080633e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f5014007f80 tx=0x7f501400b040).stop 2024-02-15T18:37:56.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.967+0000 7f5024bb4700 1 -- 172.21.15.38:0/963931573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50200f55c0 msgr2=0x7f5020106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:56.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.967+0000 7f5024bb4700 1 --2- 172.21.15.38:0/963931573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50200f55c0 0x7f5020106fb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f5010002040 tx=0x7f501000b040).stop 2024-02-15T18:37:56.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.967+0000 7f5024bb4700 1 -- 172.21.15.38:0/963931573 shutdown_connections 2024-02-15T18:37:56.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.967+0000 7f5024bb4700 1 --2- 172.21.15.38:0/963931573 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5008060f20 0x7f50080633e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:56.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.967+0000 7f5024bb4700 1 --2- 172.21.15.38:0/963931573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50200f55c0 0x7f5020106fb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:56.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.967+0000 7f5024bb4700 1 --2- 172.21.15.38:0/963931573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50200f6860 0x7f50201074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:56.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.967+0000 7f5024bb4700 1 -- 172.21.15.38:0/963931573 >> 172.21.15.38:0/963931573 conn(0x7f50200f0d40 msgr2=0x7f502005fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:56.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.967+0000 7f5024bb4700 1 -- 172.21.15.38:0/963931573 shutdown_connections 2024-02-15T18:37:56.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:56.967+0000 7f5024bb4700 1 -- 172.21.15.38:0/963931573 wait complete. 2024-02-15T18:37:57.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.072+0000 7fcd94291700 1 -- 172.21.15.38:0/1226921075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd8c0f6860 msgr2=0x7fcd8c0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:57.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.072+0000 7fcd94291700 1 --2- 172.21.15.38:0/1226921075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd8c0f6860 0x7fcd8c0f6cc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fcd7c004660 tx=0x7fcd7c009b30).stop 2024-02-15T18:37:57.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.072+0000 7fcd94291700 1 -- 172.21.15.38:0/1226921075 shutdown_connections 2024-02-15T18:37:57.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.072+0000 7fcd94291700 1 --2- 172.21.15.38:0/1226921075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd8c0f6860 0x7fcd8c0f6cc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:57.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.072+0000 7fcd94291700 1 --2- 172.21.15.38:0/1226921075 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd8c0f55c0 0x7fcd8c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:57.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.072+0000 7fcd94291700 1 -- 172.21.15.38:0/1226921075 >> 172.21.15.38:0/1226921075 conn(0x7fcd8c0f0d40 msgr2=0x7fcd8c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:57.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.072+0000 7fcd94291700 1 -- 172.21.15.38:0/1226921075 shutdown_connections 2024-02-15T18:37:57.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.072+0000 7fcd94291700 1 -- 172.21.15.38:0/1226921075 wait complete. 2024-02-15T18:37:57.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.073+0000 7fcd94291700 1 Processor -- start 2024-02-15T18:37:57.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.073+0000 7fcd94291700 1 -- start start 2024-02-15T18:37:57.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.073+0000 7fcd94291700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd8c0f55c0 0x7fcd8c109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:57.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.073+0000 7fcd94291700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd8c0f6860 0x7fcd8c10c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:57.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.073+0000 7fcd94291700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c060420 con 0x7fcd8c0f6860 2024-02-15T18:37:57.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.073+0000 7fcd94291700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c060560 con 0x7fcd8c0f55c0 2024-02-15T18:37:57.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.073+0000 7fcd9182c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd8c0f6860 0x7fcd8c10c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:57.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.073+0000 7fcd9182c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd8c0f6860 0x7fcd8c10c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52730/0 (socket says 172.21.15.38:52730) 2024-02-15T18:37:57.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.073+0000 7fcd9182c700 1 -- 172.21.15.38:0/2921398829 learned_addr learned my addr 172.21.15.38:0/2921398829 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:37:57.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.073+0000 7fcd9182c700 1 -- 172.21.15.38:0/2921398829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd8c0f55c0 msgr2=0x7fcd8c109c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:57.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.073+0000 7fcd9182c700 1 --2- 172.21.15.38:0/2921398829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd8c0f55c0 0x7fcd8c109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:37:57.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.073+0000 7fcd9182c700 1 -- 172.21.15.38:0/2921398829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c005040 con 0x7fcd8c0f6860 2024-02-15T18:37:57.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.073+0000 7fcd9182c700 1 --2- 172.21.15.38:0/2921398829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd8c0f6860 0x7fcd8c10c170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fcd7c013fd0 tx=0x7fcd7c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:57.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.073+0000 7fcd82ffd700 1 -- 172.21.15.38:0/2921398829 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd7c018070 con 0x7fcd8c0f6860 2024-02-15T18:37:57.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.073+0000 7fcd82ffd700 1 -- 172.21.15.38:0/2921398829 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fcd7c007ba0 con 0x7fcd8c0f6860 2024-02-15T18:37:57.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.073+0000 7fcd94291700 1 -- 172.21.15.38:0/2921398829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8c10c720 con 0x7fcd8c0f6860 2024-02-15T18:37:57.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.074+0000 7fcd82ffd700 1 -- 172.21.15.38:0/2921398829 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd7c01c620 con 0x7fcd8c0f6860 2024-02-15T18:37:57.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.074+0000 7fcd94291700 1 -- 172.21.15.38:0/2921398829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c10cc50 con 0x7fcd8c0f6860 2024-02-15T18:37:57.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.074+0000 7fcd94291700 1 -- 172.21.15.38:0/2921398829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8c0f49b0 con 0x7fcd8c0f6860 2024-02-15T18:37:57.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.075+0000 7fcd82ffd700 1 -- 172.21.15.38:0/2921398829 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fcd7c01c8e0 con 0x7fcd8c0f6860 2024-02-15T18:37:57.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.075+0000 7fcd82ffd700 1 --2- 172.21.15.38:0/2921398829 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcd78060f20 0x7fcd780633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:57.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.075+0000 7fcd82ffd700 1 -- 172.21.15.38:0/2921398829 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fcd7c07df40 con 0x7fcd8c0f6860 2024-02-15T18:37:57.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.075+0000 7fcd9202d700 1 --2- 172.21.15.38:0/2921398829 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcd78060f20 0x7fcd780633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:57.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.076+0000 7fcd9202d700 1 --2- 172.21.15.38:0/2921398829 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcd78060f20 0x7fcd780633e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fcd8c0600a0 tx=0x7fcd880099e0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:57.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.079+0000 7fcd82ffd700 1 -- 172.21.15.38:0/2921398829 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fcd7c0acf60 con 0x7fcd8c0f6860 2024-02-15T18:37:57.248 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:56 smithi038 bash[83573]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-02-15T18:37:57.248 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:56 smithi038 bash[83573]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-15T18:37:57.248 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:56 smithi038 bash[83573]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-1 --no-mon-config 2024-02-15T18:37:57.248 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:56 smithi038 bash[83573]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-1/block 2024-02-15T18:37:57.248 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:56 smithi038 bash[83573]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-1/block 2024-02-15T18:37:57.248 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:56 smithi038 bash[83573]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-15T18:37:57.248 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:56 smithi038 bash[83573]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-15T18:37:57.248 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:56 smithi038 bash[83573]: --> ceph-volume lvm activate successful for osd ID: 1 2024-02-15T18:37:57.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.270+0000 7fcd94291700 1 -- 172.21.15.38:0/2921398829 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd8c057f10 con 0x7fcd78060f20 2024-02-15T18:37:57.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.280+0000 7fcd82ffd700 1 -- 172.21.15.38:0/2921398829 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fcd8c057f10 con 0x7fcd78060f20 2024-02-15T18:37:57.270 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:37:57.270 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (8m) 24s ago 10m 19.3M - 0.20.0 0881eb8f169f c2034c318e7c 2024-02-15T18:37:57.270 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (55s) 24s ago 10m 6635k - 16.2.14-521-g0e714d9a 1fbc92c859dc c2fee0824d4b 2024-02-15T18:37:57.270 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi121 smithi121 running (49s) 45s ago 9m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 05c1a5501a35 2024-02-15T18:37:57.270 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (8m) 24s ago 10m 33.3M - 6.7.4 557c83e11646 2dcccfbfbd2a 2024-02-15T18:37:57.270 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.hxoggl smithi038 running (5m) 24s ago 5m 14.6M - 16.2.4 8d91d370c2b8 be2533754b9c 2024-02-15T18:37:57.270 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.qgkzgn smithi038 running (4m) 24s ago 4m 14.2M - 16.2.4 8d91d370c2b8 dd3ffafba9bd 2024-02-15T18:37:57.270 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.mvumlh smithi121 running (5m) 45s ago 5m 116M - 16.2.4 8d91d370c2b8 88b7d876a39b 2024-02-15T18:37:57.270 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.ovuzki smithi121 running (5m) 45s ago 5m 124M - 16.2.4 8d91d370c2b8 21cacf0012f9 2024-02-15T18:37:57.270 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.khesbl smithi038 *:8443,9283 running (2m) 24s ago 12m 430M - 16.2.14-521-g0e714d9a 1fbc92c859dc e642f3c2cfc9 2024-02-15T18:37:57.271 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi121.ckiryi smithi121 *:8443,9283 running (96s) 45s ago 9m 385M - 16.2.14-521-g0e714d9a 1fbc92c859dc 82515b025594 2024-02-15T18:37:57.271 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (85s) 24s ago 12m 39.2M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc d3e8317a30ca 2024-02-15T18:37:57.271 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi121 smithi121 running (65s) 45s ago 9m 26.2M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 42e1dda1a39d 2024-02-15T18:37:57.271 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (10m) 24s ago 10m 17.1M - 0.18.1 e5a616e4b9cf 037f60e38158 2024-02-15T18:37:57.271 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi121 smithi121 *:9100 running (9m) 45s ago 9m 17.2M - 0.18.1 e5a616e4b9cf a867bff85e18 2024-02-15T18:37:57.271 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (31s) 24s ago 8m 15.4M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 623ac7ecbb0a 2024-02-15T18:37:57.271 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (8m) 24s ago 8m 399M 4096M 16.2.4 8d91d370c2b8 16dbdf20b170 2024-02-15T18:37:57.271 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (7m) 24s ago 7m 355M 4096M 16.2.4 8d91d370c2b8 6d6ee813e5be 2024-02-15T18:37:57.271 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi121 running (7m) 45s ago 7m 449M 4096M 16.2.4 8d91d370c2b8 cba2d9ed4730 2024-02-15T18:37:57.271 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi121 running (7m) 45s ago 7m 429M 4096M 16.2.4 8d91d370c2b8 ac17ad7dc628 2024-02-15T18:37:57.271 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi121 running (6m) 45s ago 6m 378M 4096M 16.2.4 8d91d370c2b8 cdc8c669c4db 2024-02-15T18:37:57.271 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (2m) 24s ago 9m 59.6M - 2.18.1 de242295e225 67e33f9de4e6 2024-02-15T18:37:57.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.282+0000 7fcd94291700 1 -- 172.21.15.38:0/2921398829 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcd78060f20 msgr2=0x7fcd780633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:57.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.282+0000 7fcd94291700 1 --2- 172.21.15.38:0/2921398829 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcd78060f20 0x7fcd780633e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fcd8c0600a0 tx=0x7fcd880099e0).stop 2024-02-15T18:37:57.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.282+0000 7fcd94291700 1 -- 172.21.15.38:0/2921398829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd8c0f6860 msgr2=0x7fcd8c10c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:57.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.282+0000 7fcd94291700 1 --2- 172.21.15.38:0/2921398829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd8c0f6860 0x7fcd8c10c170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fcd7c013fd0 tx=0x7fcd7c009b30).stop 2024-02-15T18:37:57.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.282+0000 7fcd94291700 1 -- 172.21.15.38:0/2921398829 shutdown_connections 2024-02-15T18:37:57.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.282+0000 7fcd94291700 1 --2- 172.21.15.38:0/2921398829 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcd78060f20 0x7fcd780633e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:57.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.282+0000 7fcd94291700 1 --2- 172.21.15.38:0/2921398829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd8c0f55c0 0x7fcd8c109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:37:57.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.282+0000 7fcd94291700 1 --2- 172.21.15.38:0/2921398829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd8c0f6860 0x7fcd8c10c170 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:57.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.282+0000 7fcd94291700 1 -- 172.21.15.38:0/2921398829 >> 172.21.15.38:0/2921398829 conn(0x7fcd8c0f0d40 msgr2=0x7fcd8c0fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:57.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.282+0000 7fcd94291700 1 -- 172.21.15.38:0/2921398829 shutdown_connections 2024-02-15T18:37:57.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.282+0000 7fcd94291700 1 -- 172.21.15.38:0/2921398829 wait complete. 2024-02-15T18:37:57.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.382+0000 7f9de5161700 1 -- 172.21.15.38:0/2830456798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9de00f5430 msgr2=0x7f9de00f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:57.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.382+0000 7f9de5161700 1 --2- 172.21.15.38:0/2830456798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9de00f5430 0x7f9de00f5850 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f9dd0004660 tx=0x7f9dd0009b30).stop 2024-02-15T18:37:57.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.382+0000 7f9de5161700 1 -- 172.21.15.38:0/2830456798 shutdown_connections 2024-02-15T18:37:57.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.382+0000 7f9de5161700 1 --2- 172.21.15.38:0/2830456798 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9de00f66d0 0x7f9de00f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:57.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.382+0000 7f9de5161700 1 --2- 172.21.15.38:0/2830456798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9de00f5430 0x7f9de00f5850 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:57.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.382+0000 7f9de5161700 1 -- 172.21.15.38:0/2830456798 >> 172.21.15.38:0/2830456798 conn(0x7f9de00f0b90 msgr2=0x7f9de00f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:57.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.383+0000 7f9de5161700 1 -- 172.21.15.38:0/2830456798 shutdown_connections 2024-02-15T18:37:57.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.383+0000 7f9de5161700 1 -- 172.21.15.38:0/2830456798 wait complete. 2024-02-15T18:37:57.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.383+0000 7f9de5161700 1 Processor -- start 2024-02-15T18:37:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.383+0000 7f9de5161700 1 -- start start 2024-02-15T18:37:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.383+0000 7f9de5161700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9de00f5430 0x7f9de0106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.383+0000 7f9de5161700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9de00f66d0 0x7f9de01073b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.383+0000 7f9de5161700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de0107960 con 0x7f9de00f5430 2024-02-15T18:37:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.383+0000 7f9de5161700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de0108160 con 0x7f9de00f66d0 2024-02-15T18:37:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.384+0000 7f9ddffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9de00f5430 0x7f9de0106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.384+0000 7f9ddffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9de00f5430 0x7f9de0106ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52752/0 (socket says 172.21.15.38:52752) 2024-02-15T18:37:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.384+0000 7f9ddffff700 1 -- 172.21.15.38:0/3062769046 learned_addr learned my addr 172.21.15.38:0/3062769046 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:37:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.384+0000 7f9ddf7fe700 1 --2- 172.21.15.38:0/3062769046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9de00f66d0 0x7f9de01073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.384+0000 7f9ddffff700 1 -- 172.21.15.38:0/3062769046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9de00f66d0 msgr2=0x7f9de01073b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.384+0000 7f9ddffff700 1 --2- 172.21.15.38:0/3062769046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9de00f66d0 0x7f9de01073b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.384+0000 7f9ddffff700 1 -- 172.21.15.38:0/3062769046 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd0005040 con 0x7f9de00f5430 2024-02-15T18:37:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.384+0000 7f9ddffff700 1 --2- 172.21.15.38:0/3062769046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9de00f5430 0x7f9de0106ec0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f9dd0013fd0 tx=0x7f9dd0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:57.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.384+0000 7f9ddcff9700 1 -- 172.21.15.38:0/3062769046 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dd0018070 con 0x7f9de00f5430 2024-02-15T18:37:57.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.384+0000 7f9de5161700 1 -- 172.21.15.38:0/3062769046 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de01083e0 con 0x7f9de00f5430 2024-02-15T18:37:57.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.384+0000 7f9ddcff9700 1 -- 172.21.15.38:0/3062769046 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f9dd0007d70 con 0x7f9de00f5430 2024-02-15T18:37:57.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.384+0000 7f9ddcff9700 1 -- 172.21.15.38:0/3062769046 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dd001c620 con 0x7f9de00f5430 2024-02-15T18:37:57.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.384+0000 7f9de5161700 1 -- 172.21.15.38:0/3062769046 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de0108940 con 0x7f9de00f5430 2024-02-15T18:37:57.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.385+0000 7f9de5161700 1 -- 172.21.15.38:0/3062769046 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de0059100 con 0x7f9de00f5430 2024-02-15T18:37:57.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.386+0000 7f9ddcff9700 1 -- 172.21.15.38:0/3062769046 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f9dd001d420 con 0x7f9de00f5430 2024-02-15T18:37:57.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.386+0000 7f9ddcff9700 1 --2- 172.21.15.38:0/3062769046 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9dc8060f70 0x7f9dc8063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:57.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.386+0000 7f9ddcff9700 1 -- 172.21.15.38:0/3062769046 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f9dd007dfe0 con 0x7f9de00f5430 2024-02-15T18:37:57.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.386+0000 7f9ddf7fe700 1 --2- 172.21.15.38:0/3062769046 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9dc8060f70 0x7f9dc8063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:57.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.386+0000 7f9ddf7fe700 1 --2- 172.21.15.38:0/3062769046 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9dc8060f70 0x7f9dc8063430 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f9dd40049e0 tx=0x7f9dd4009040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:57.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.390+0000 7f9ddcff9700 1 -- 172.21.15.38:0/3062769046 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9dd0051f70 con 0x7f9de00f5430 2024-02-15T18:37:57.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:56.650469+0000 mgr.smithi038.khesbl (mgr.14664) 100 : audit [DBG] from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:37:57.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:56.948116+0000 mgr.smithi038.khesbl (mgr.14664) 101 : cluster [DBG] pgmap v68: 65 pgs: 5 peering, 9 stale+active+clean, 51 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:37:57.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:56.965542+0000 mgr.smithi038.khesbl (mgr.14664) 102 : audit [DBG] from='client.34156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:37:57.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:56.650469+0000 mgr.smithi038.khesbl (mgr.14664) 100 : audit [DBG] from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:37:57.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:56.948116+0000 mgr.smithi038.khesbl (mgr.14664) 101 : cluster [DBG] pgmap v68: 65 pgs: 5 peering, 9 stale+active+clean, 51 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:37:57.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:56.965542+0000 mgr.smithi038.khesbl (mgr.14664) 102 : audit [DBG] from='client.34156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:37:57.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.633+0000 7f9de5161700 1 -- 172.21.15.38:0/3062769046 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9de010d3a0 con 0x7f9de00f5430 2024-02-15T18:37:57.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.634+0000 7f9ddcff9700 1 -- 172.21.15.38:0/3062769046 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+820 (secure 0 0 0) 0x7f9dd0051b60 con 0x7f9de00f5430 2024-02-15T18:37:57.623 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:37:57.624 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-02-15T18:37:57.624 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:37:57.624 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:37:57.624 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-02-15T18:37:57.624 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:37:57.624 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:37:57.624 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-02-15T18:37:57.624 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 1, 2024-02-15T18:37:57.624 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:37:57.624 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:37:57.624 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-02-15T18:37:57.624 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:37:57.625 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:37:57.625 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-02-15T18:37:57.625 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 5, 2024-02-15T18:37:57.625 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 8 2024-02-15T18:37:57.625 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-02-15T18:37:57.625 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:37:57.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.635+0000 7f9de5161700 1 -- 172.21.15.38:0/3062769046 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9dc8060f70 msgr2=0x7f9dc8063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:57.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.635+0000 7f9de5161700 1 --2- 172.21.15.38:0/3062769046 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9dc8060f70 0x7f9dc8063430 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f9dd40049e0 tx=0x7f9dd4009040).stop 2024-02-15T18:37:57.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.635+0000 7f9de5161700 1 -- 172.21.15.38:0/3062769046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9de00f5430 msgr2=0x7f9de0106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:57.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.635+0000 7f9de5161700 1 --2- 172.21.15.38:0/3062769046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9de00f5430 0x7f9de0106ec0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f9dd0013fd0 tx=0x7f9dd0009b30).stop 2024-02-15T18:37:57.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.636+0000 7f9de5161700 1 -- 172.21.15.38:0/3062769046 shutdown_connections 2024-02-15T18:37:57.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.636+0000 7f9de5161700 1 --2- 172.21.15.38:0/3062769046 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9dc8060f70 0x7f9dc8063430 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:57.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.636+0000 7f9de5161700 1 --2- 172.21.15.38:0/3062769046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9de00f5430 0x7f9de0106ec0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:57.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.636+0000 7f9de5161700 1 --2- 172.21.15.38:0/3062769046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9de00f66d0 0x7f9de01073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:57.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.636+0000 7f9de5161700 1 -- 172.21.15.38:0/3062769046 >> 172.21.15.38:0/3062769046 conn(0x7f9de00f0b90 msgr2=0x7f9de005fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:57.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.636+0000 7f9de5161700 1 -- 172.21.15.38:0/3062769046 shutdown_connections 2024-02-15T18:37:57.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.636+0000 7f9de5161700 1 -- 172.21.15.38:0/3062769046 wait complete. 2024-02-15T18:37:57.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.737+0000 7fbb824cb700 1 -- 172.21.15.38:0/2675288020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb7c0f66d0 msgr2=0x7fbb7c0f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:57.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.737+0000 7fbb824cb700 1 --2- 172.21.15.38:0/2675288020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb7c0f66d0 0x7fbb7c0f6b30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fbb78004660 tx=0x7fbb78009b30).stop 2024-02-15T18:37:57.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.737+0000 7fbb824cb700 1 -- 172.21.15.38:0/2675288020 shutdown_connections 2024-02-15T18:37:57.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.737+0000 7fbb824cb700 1 --2- 172.21.15.38:0/2675288020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb7c0f66d0 0x7fbb7c0f6b30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:57.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.737+0000 7fbb824cb700 1 --2- 172.21.15.38:0/2675288020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb7c0f5430 0x7fbb7c0f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:57.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.737+0000 7fbb824cb700 1 -- 172.21.15.38:0/2675288020 >> 172.21.15.38:0/2675288020 conn(0x7fbb7c0f0b90 msgr2=0x7fbb7c0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:57.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.737+0000 7fbb824cb700 1 -- 172.21.15.38:0/2675288020 shutdown_connections 2024-02-15T18:37:57.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.737+0000 7fbb824cb700 1 -- 172.21.15.38:0/2675288020 wait complete. 2024-02-15T18:37:57.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.738+0000 7fbb824cb700 1 Processor -- start 2024-02-15T18:37:57.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.738+0000 7fbb824cb700 1 -- start start 2024-02-15T18:37:57.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.738+0000 7fbb824cb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb7c0f5430 0x7fbb7c106e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:57.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.738+0000 7fbb824cb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb7c0f66d0 0x7fbb7c107320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:57.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.738+0000 7fbb824cb700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c107920 con 0x7fbb7c0f5430 2024-02-15T18:37:57.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.738+0000 7fbb824cb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c108130 con 0x7fbb7c0f66d0 2024-02-15T18:37:57.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.738+0000 7fbb814c9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb7c0f5430 0x7fbb7c106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:57.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.738+0000 7fbb814c9700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb7c0f5430 0x7fbb7c106e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52766/0 (socket says 172.21.15.38:52766) 2024-02-15T18:37:57.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.738+0000 7fbb814c9700 1 -- 172.21.15.38:0/1633288614 learned_addr learned my addr 172.21.15.38:0/1633288614 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:37:57.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.738+0000 7fbb80cc8700 1 --2- 172.21.15.38:0/1633288614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb7c0f66d0 0x7fbb7c107320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:57.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.739+0000 7fbb814c9700 1 -- 172.21.15.38:0/1633288614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb7c0f66d0 msgr2=0x7fbb7c107320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:57.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.739+0000 7fbb814c9700 1 --2- 172.21.15.38:0/1633288614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb7c0f66d0 0x7fbb7c107320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:57.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.739+0000 7fbb814c9700 1 -- 172.21.15.38:0/1633288614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb78005040 con 0x7fbb7c0f5430 2024-02-15T18:37:57.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.739+0000 7fbb814c9700 1 --2- 172.21.15.38:0/1633288614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb7c0f5430 0x7fbb7c106e30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fbb7000c530 tx=0x7fbb70007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:57.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.739+0000 7fbb6dffb700 1 -- 172.21.15.38:0/1633288614 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb7000f650 con 0x7fbb7c0f5430 2024-02-15T18:37:57.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.739+0000 7fbb6dffb700 1 -- 172.21.15.38:0/1633288614 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fbb7000fca0 con 0x7fbb7c0f5430 2024-02-15T18:37:57.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.739+0000 7fbb824cb700 1 -- 172.21.15.38:0/1633288614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7c1083b0 con 0x7fbb7c0f5430 2024-02-15T18:37:57.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.739+0000 7fbb6dffb700 1 -- 172.21.15.38:0/1633288614 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb70012df0 con 0x7fbb7c0f5430 2024-02-15T18:37:57.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.739+0000 7fbb824cb700 1 -- 172.21.15.38:0/1633288614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7c108970 con 0x7fbb7c0f5430 2024-02-15T18:37:57.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.740+0000 7fbb824cb700 1 -- 172.21.15.38:0/1633288614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7c059100 con 0x7fbb7c0f5430 2024-02-15T18:37:57.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.741+0000 7fbb6dffb700 1 -- 172.21.15.38:0/1633288614 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fbb70019030 con 0x7fbb7c0f5430 2024-02-15T18:37:57.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.741+0000 7fbb6dffb700 1 --2- 172.21.15.38:0/1633288614 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fbb68060f70 0x7fbb68063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:57.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.741+0000 7fbb6dffb700 1 -- 172.21.15.38:0/1633288614 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fbb7007bcd0 con 0x7fbb7c0f5430 2024-02-15T18:37:57.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.741+0000 7fbb80cc8700 1 --2- 172.21.15.38:0/1633288614 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fbb68060f70 0x7fbb68063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:57.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.742+0000 7fbb80cc8700 1 --2- 172.21.15.38:0/1633288614 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fbb68060f70 0x7fbb68063430 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fbb78009f50 tx=0x7fbb78009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:57.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.746+0000 7fbb6dffb700 1 -- 172.21.15.38:0/1633288614 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbb70048f60 con 0x7fbb7c0f5430 2024-02-15T18:37:57.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.981+0000 7fbb824cb700 1 -- 172.21.15.38:0/1633288614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbb7c10d370 con 0x7fbb7c0f5430 2024-02-15T18:37:57.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.981+0000 7fbb6dffb700 1 -- 172.21.15.38:0/1633288614 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1807 (secure 0 0 0) 0x7fbb70017020 con 0x7fbb7c0f5430 2024-02-15T18:37:57.971 INFO:teuthology.orchestra.run.smithi038.stdout:e13 2024-02-15T18:37:57.971 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:37:57.971 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:37:57.971 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-02-15T18:37:57.971 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:37:57.971 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:37:57.971 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-02-15T18:37:57.972 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 13 2024-02-15T18:37:57.972 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 2024-02-15T18:37:57.972 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:37:57.972 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-02-15T18:35:07.485909+0000 2024-02-15T18:37:57.972 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-02-15T18:37:57.972 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-02-15T18:37:57.972 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-02-15T18:37:57.972 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-02-15T18:37:57.972 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-02-15T18:37:57.972 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-02-15T18:37:57.972 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-02-15T18:37:57.972 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-02-15T18:37:57.972 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:37:57.973 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-02-15T18:37:57.973 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-02-15T18:37:57.973 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=24261,1=24275} 2024-02-15T18:37:57.973 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-02-15T18:37:57.973 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-02-15T18:37:57.973 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-02-15T18:37:57.973 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-02-15T18:37:57.973 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-02-15T18:37:57.973 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-02-15T18:37:57.973 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-02-15T18:37:57.973 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-02-15T18:37:57.973 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.mvumlh{0:24261} state up:active seq 3 export targets 1 join_fscid=1 addr [v2:172.21.15.121:6824/1422707571,v1:172.21.15.121:6825/1422707571] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:37:57.973 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.ovuzki{1:24275} state up:active seq 2 export targets 0 join_fscid=1 addr [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:37:57.974 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:37:57.974 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:37:57.974 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-02-15T18:37:57.974 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:37:57.974 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.hxoggl{-1:14446} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.38:6826/3822880235,v1:172.21.15.38:6827/3822880235] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:37:57.974 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.qgkzgn{-1:14456} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.38:6828/703030231,v1:172.21.15.38:6829/703030231] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:37:57.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.983+0000 7fbb824cb700 1 -- 172.21.15.38:0/1633288614 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fbb68060f70 msgr2=0x7fbb68063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:57.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.983+0000 7fbb824cb700 1 --2- 172.21.15.38:0/1633288614 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fbb68060f70 0x7fbb68063430 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fbb78009f50 tx=0x7fbb78009b30).stop 2024-02-15T18:37:57.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.983+0000 7fbb824cb700 1 -- 172.21.15.38:0/1633288614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb7c0f5430 msgr2=0x7fbb7c106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:57.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.983+0000 7fbb824cb700 1 --2- 172.21.15.38:0/1633288614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb7c0f5430 0x7fbb7c106e30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fbb7000c530 tx=0x7fbb70007e10).stop 2024-02-15T18:37:57.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.983+0000 7fbb824cb700 1 -- 172.21.15.38:0/1633288614 shutdown_connections 2024-02-15T18:37:57.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.983+0000 7fbb824cb700 1 --2- 172.21.15.38:0/1633288614 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fbb68060f70 0x7fbb68063430 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:57.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.983+0000 7fbb824cb700 1 --2- 172.21.15.38:0/1633288614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb7c0f5430 0x7fbb7c106e30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:57.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.983+0000 7fbb824cb700 1 --2- 172.21.15.38:0/1633288614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb7c0f66d0 0x7fbb7c107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:57.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.983+0000 7fbb824cb700 1 -- 172.21.15.38:0/1633288614 >> 172.21.15.38:0/1633288614 conn(0x7fbb7c0f0b90 msgr2=0x7fbb7c05fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:57.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.983+0000 7fbb824cb700 1 -- 172.21.15.38:0/1633288614 shutdown_connections 2024-02-15T18:37:57.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:57.983+0000 7fbb824cb700 1 -- 172.21.15.38:0/1633288614 wait complete. 2024-02-15T18:37:57.975 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 13 2024-02-15T18:37:58.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.087+0000 7f96bb59e700 1 -- 172.21.15.38:0/1459433481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96bc0f6530 msgr2=0x7f96bc0f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:58.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.087+0000 7f96bb59e700 1 --2- 172.21.15.38:0/1459433481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96bc0f6530 0x7f96bc0f6950 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f96ac0099e0 tx=0x7f96ac004d10).stop 2024-02-15T18:37:58.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.087+0000 7f96bb59e700 1 -- 172.21.15.38:0/1459433481 shutdown_connections 2024-02-15T18:37:58.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.087+0000 7f96bb59e700 1 --2- 172.21.15.38:0/1459433481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96bc0f77d0 0x7f96bc0f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:58.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.087+0000 7f96bb59e700 1 --2- 172.21.15.38:0/1459433481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96bc0f6530 0x7f96bc0f6950 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:58.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.087+0000 7f96bb59e700 1 -- 172.21.15.38:0/1459433481 >> 172.21.15.38:0/1459433481 conn(0x7f96bc0f1c90 msgr2=0x7f96bc0f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:58.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.087+0000 7f96bb59e700 1 -- 172.21.15.38:0/1459433481 shutdown_connections 2024-02-15T18:37:58.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.087+0000 7f96bb59e700 1 -- 172.21.15.38:0/1459433481 wait complete. 2024-02-15T18:37:58.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.088+0000 7f96bb59e700 1 Processor -- start 2024-02-15T18:37:58.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.088+0000 7f96bb59e700 1 -- start start 2024-02-15T18:37:58.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.088+0000 7f96bb59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96bc0f6530 0x7f96bc106f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:58.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.088+0000 7f96bb59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96bc0f77d0 0x7f96bc107480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:58.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.088+0000 7f96bb59e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc107a30 con 0x7f96bc0f77d0 2024-02-15T18:37:58.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.088+0000 7f96bb59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc107b70 con 0x7f96bc0f6530 2024-02-15T18:37:58.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.088+0000 7f96b9d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96bc0f77d0 0x7f96bc107480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:58.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.088+0000 7f96b9d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96bc0f77d0 0x7f96bc107480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52780/0 (socket says 172.21.15.38:52780) 2024-02-15T18:37:58.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.088+0000 7f96b9d9b700 1 -- 172.21.15.38:0/786855066 learned_addr learned my addr 172.21.15.38:0/786855066 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:37:58.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.088+0000 7f96ba59c700 1 --2- 172.21.15.38:0/786855066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96bc0f6530 0x7f96bc106f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:58.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.088+0000 7f96b9d9b700 1 -- 172.21.15.38:0/786855066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96bc0f6530 msgr2=0x7f96bc106f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:58.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.088+0000 7f96b9d9b700 1 --2- 172.21.15.38:0/786855066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96bc0f6530 0x7f96bc106f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:58.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.089+0000 7f96b9d9b700 1 -- 172.21.15.38:0/786855066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ac009640 con 0x7f96bc0f77d0 2024-02-15T18:37:58.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.089+0000 7f96b9d9b700 1 --2- 172.21.15.38:0/786855066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96bc0f77d0 0x7f96bc107480 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f96b000e7f0 tx=0x7f96b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:58.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.089+0000 7f96aaffd700 1 -- 172.21.15.38:0/786855066 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96b00092e0 con 0x7f96bc0f77d0 2024-02-15T18:37:58.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.089+0000 7f96aaffd700 1 -- 172.21.15.38:0/786855066 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f96b000f040 con 0x7f96bc0f77d0 2024-02-15T18:37:58.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.089+0000 7f96bb59e700 1 -- 172.21.15.38:0/786855066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96bc108510 con 0x7f96bc0f77d0 2024-02-15T18:37:58.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.089+0000 7f96aaffd700 1 -- 172.21.15.38:0/786855066 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96b0014850 con 0x7f96bc0f77d0 2024-02-15T18:37:58.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.089+0000 7f96bb59e700 1 -- 172.21.15.38:0/786855066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96bc108ad0 con 0x7f96bc0f77d0 2024-02-15T18:37:58.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.090+0000 7f96bb59e700 1 -- 172.21.15.38:0/786855066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96bc045ad0 con 0x7f96bc0f77d0 2024-02-15T18:37:58.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.090+0000 7f96aaffd700 1 -- 172.21.15.38:0/786855066 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f96b001b030 con 0x7f96bc0f77d0 2024-02-15T18:37:58.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.091+0000 7f96aaffd700 1 --2- 172.21.15.38:0/786855066 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f96a4060f70 0x7f96a4063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:37:58.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.091+0000 7f96aaffd700 1 -- 172.21.15.38:0/786855066 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f96b007be30 con 0x7f96bc0f77d0 2024-02-15T18:37:58.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.091+0000 7f96ba59c700 1 --2- 172.21.15.38:0/786855066 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f96a4060f70 0x7f96a4063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:37:58.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.091+0000 7f96ba59c700 1 --2- 172.21.15.38:0/786855066 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f96a4060f70 0x7f96a4063430 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f96ac003380 tx=0x7f96ac015040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:37:58.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.095+0000 7f96aaffd700 1 -- 172.21.15.38:0/786855066 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f96b004fd90 con 0x7f96bc0f77d0 2024-02-15T18:37:58.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.295+0000 7f96bb59e700 1 -- 172.21.15.38:0/786855066 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f96bc108cf0 con 0x7f96a4060f70 2024-02-15T18:37:58.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.296+0000 7f96aaffd700 1 -- 172.21.15.38:0/786855066 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f96bc108cf0 con 0x7f96a4060f70 2024-02-15T18:37:58.285 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:37:58.286 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:37:58.286 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-02-15T18:37:58.286 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:37:58.286 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-02-15T18:37:58.286 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-02-15T18:37:58.286 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-02-15T18:37:58.286 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr" 2024-02-15T18:37:58.286 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-02-15T18:37:58.286 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "7/21 daemons upgraded", 2024-02-15T18:37:58.286 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading osd daemons", 2024-02-15T18:37:58.286 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-02-15T18:37:58.286 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:37:58.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.297+0000 7f96bb59e700 1 -- 172.21.15.38:0/786855066 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f96a4060f70 msgr2=0x7f96a4063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:58.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.297+0000 7f96bb59e700 1 --2- 172.21.15.38:0/786855066 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f96a4060f70 0x7f96a4063430 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f96ac003380 tx=0x7f96ac015040).stop 2024-02-15T18:37:58.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.297+0000 7f96bb59e700 1 -- 172.21.15.38:0/786855066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96bc0f77d0 msgr2=0x7f96bc107480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:37:58.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.297+0000 7f96bb59e700 1 --2- 172.21.15.38:0/786855066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96bc0f77d0 0x7f96bc107480 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f96b000e7f0 tx=0x7f96b000b040).stop 2024-02-15T18:37:58.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.298+0000 7f96bb59e700 1 -- 172.21.15.38:0/786855066 shutdown_connections 2024-02-15T18:37:58.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.298+0000 7f96bb59e700 1 --2- 172.21.15.38:0/786855066 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f96a4060f70 0x7f96a4063430 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:58.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.298+0000 7f96bb59e700 1 --2- 172.21.15.38:0/786855066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96bc0f6530 0x7f96bc106f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:58.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.298+0000 7f96bb59e700 1 --2- 172.21.15.38:0/786855066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96bc0f77d0 0x7f96bc107480 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:37:58.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.298+0000 7f96bb59e700 1 -- 172.21.15.38:0/786855066 >> 172.21.15.38:0/786855066 conn(0x7f96bc0f1c90 msgr2=0x7f96bc0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:37:58.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.298+0000 7f96bb59e700 1 -- 172.21.15.38:0/786855066 shutdown_connections 2024-02-15T18:37:58.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:37:58.298+0000 7f96bb59e700 1 -- 172.21.15.38:0/786855066 wait complete. 2024-02-15T18:37:58.516 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-1[83993]: debug 2024-02-15T18:37:58.419+0000 7fa0b75ca700 1 -- 172.21.15.38:0/707051592 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5583d8756680 con 0x5583d871d400 2024-02-15T18:37:58.516 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:57.272146+0000 mgr.smithi038.khesbl (mgr.14664) 103 : audit [DBG] from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:37:58.516 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:57.634433+0000 mon.smithi038 (mon.0) 151 : audit [DBG] from='client.? 172.21.15.38:0/3062769046' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:37:58.516 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:57.981814+0000 mon.smithi038 (mon.0) 152 : audit [DBG] from='client.? 172.21.15.38:0/1633288614' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:37:58.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:57.272146+0000 mgr.smithi038.khesbl (mgr.14664) 103 : audit [DBG] from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:37:58.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:57.634433+0000 mon.smithi038 (mon.0) 151 : audit [DBG] from='client.? 172.21.15.38:0/3062769046' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:37:58.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:57.981814+0000 mon.smithi038 (mon.0) 152 : audit [DBG] from='client.? 172.21.15.38:0/1633288614' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:37:58.772 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:58 smithi038 bash[83955]: 0f9e8e1f255a1920a5c2e5214f1ef883de05fcaf55a96c07e9345627c7448c03 2024-02-15T18:37:58.772 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:58 smithi038 systemd[1]: Started Ceph osd.1 for 7e05753e-cc2f-11ee-95ba-87774f69a715. 2024-02-15T18:37:59.272 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:37:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-1[83993]: debug 2024-02-15T18:37:58.944+0000 7fa0bf058380 -1 Falling back to public interface 2024-02-15T18:37:59.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:58.296256+0000 mgr.smithi038.khesbl (mgr.14664) 104 : audit [DBG] from='client.34172 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:37:59.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:58.575574+0000 mon.smithi038 (mon.0) 153 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:37:59.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:37:58.576343+0000 mon.smithi038 (mon.0) 154 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:37:59.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:37:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:37:58.948479+0000 mgr.smithi038.khesbl (mgr.14664) 105 : cluster [DBG] pgmap v69: 65 pgs: 5 peering, 9 stale+active+clean, 51 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:38:00.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:58.296256+0000 mgr.smithi038.khesbl (mgr.14664) 104 : audit [DBG] from='client.34172 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:38:00.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:58.575574+0000 mon.smithi038 (mon.0) 153 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:00.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:37:58.576343+0000 mon.smithi038 (mon.0) 154 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:38:00.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:37:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:37:58.948479+0000 mgr.smithi038.khesbl (mgr.14664) 105 : cluster [DBG] pgmap v69: 65 pgs: 5 peering, 9 stale+active+clean, 51 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:38:01.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:00.949177+0000 mgr.smithi038.khesbl (mgr.14664) 106 : cluster [DBG] pgmap v70: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 52/342 objects degraded (15.205%) 2024-02-15T18:38:01.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:00.999905+0000 mon.smithi038 (mon.0) 155 : cluster [WRN] Health check failed: Degraded data redundancy: 52/342 objects degraded (15.205%), 15 pgs degraded (PG_DEGRADED) 2024-02-15T18:38:01.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:00.999961+0000 mon.smithi038 (mon.0) 156 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-02-15T18:38:01.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:00.949177+0000 mgr.smithi038.khesbl (mgr.14664) 106 : cluster [DBG] pgmap v70: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 52/342 objects degraded (15.205%) 2024-02-15T18:38:01.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:00.999905+0000 mon.smithi038 (mon.0) 155 : cluster [WRN] Health check failed: Degraded data redundancy: 52/342 objects degraded (15.205%), 15 pgs degraded (PG_DEGRADED) 2024-02-15T18:38:01.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:00.999961+0000 mon.smithi038 (mon.0) 156 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-02-15T18:38:03.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:02.949550+0000 mgr.smithi038.khesbl (mgr.14664) 107 : cluster [DBG] pgmap v71: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 52/342 objects degraded (15.205%) 2024-02-15T18:38:03.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:03.152043+0000 mon.smithi038 (mon.0) 157 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:03.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:02.949550+0000 mgr.smithi038.khesbl (mgr.14664) 107 : cluster [DBG] pgmap v71: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 52/342 objects degraded (15.205%) 2024-02-15T18:38:03.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:03.152043+0000 mon.smithi038 (mon.0) 157 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:06.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:04.652401+0000 mon.smithi038 (mon.0) 158 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:06.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:04.653276+0000 mon.smithi038 (mon.0) 159 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:38:06.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:04.653967+0000 mon.smithi038 (mon.0) 160 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:38:06.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:38:04.654556+0000 mgr.smithi038.khesbl (mgr.14664) 108 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:38:06.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:04.658447+0000 mon.smithi038 (mon.0) 161 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:06.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:04.663111+0000 mon.smithi038 (mon.0) 162 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:38:06.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:04.665212+0000 mon.smithi038 (mon.0) 163 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:38:06.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:04.666333+0000 mon.smithi038 (mon.0) 164 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:38:06.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:04.667400+0000 mon.smithi038 (mon.0) 165 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:38:06.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:04.668537+0000 mon.smithi038 (mon.0) 166 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-15T18:38:06.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:04.668691+0000 mgr.smithi038.khesbl (mgr.14664) 109 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-15T18:38:06.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:38:04.669310+0000 mgr.smithi038.khesbl (mgr.14664) 110 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-02-15T18:38:06.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:04.950029+0000 mgr.smithi038.khesbl (mgr.14664) 111 : cluster [DBG] pgmap v72: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 52/342 objects degraded (15.205%) 2024-02-15T18:38:06.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:04.652401+0000 mon.smithi038 (mon.0) 158 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:06.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:04.653276+0000 mon.smithi038 (mon.0) 159 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:38:06.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:04.653967+0000 mon.smithi038 (mon.0) 160 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:38:06.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:38:04.654556+0000 mgr.smithi038.khesbl (mgr.14664) 108 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:38:06.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:04.658447+0000 mon.smithi038 (mon.0) 161 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:06.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:04.663111+0000 mon.smithi038 (mon.0) 162 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:38:06.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:04.665212+0000 mon.smithi038 (mon.0) 163 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:38:06.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:04.666333+0000 mon.smithi038 (mon.0) 164 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:38:06.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:04.667400+0000 mon.smithi038 (mon.0) 165 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:38:06.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:04.668537+0000 mon.smithi038 (mon.0) 166 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-15T18:38:06.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:04.668691+0000 mgr.smithi038.khesbl (mgr.14664) 109 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-15T18:38:06.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:38:04.669310+0000 mgr.smithi038.khesbl (mgr.14664) 110 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-02-15T18:38:06.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:04.950029+0000 mgr.smithi038.khesbl (mgr.14664) 111 : cluster [DBG] pgmap v72: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 52/342 objects degraded (15.205%) 2024-02-15T18:38:07.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:06.950518+0000 mgr.smithi038.khesbl (mgr.14664) 112 : cluster [DBG] pgmap v73: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 52/342 objects degraded (15.205%) 2024-02-15T18:38:07.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:06.950518+0000 mgr.smithi038.khesbl (mgr.14664) 112 : cluster [DBG] pgmap v73: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 52/342 objects degraded (15.205%) 2024-02-15T18:38:07.522 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:38:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-1[83993]: debug 2024-02-15T18:38:07.166+0000 7fa0bf058380 -1 osd.1 51 log_to_monitors {default=true} 2024-02-15T18:38:08.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:07.170747+0000 mon.smithi038 (mon.0) 167 : audit [INF] from='osd.1 [v2:172.21.15.38:6810/341024688,v1:172.21.15.38:6811/341024688]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-15T18:38:08.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:07.170747+0000 mon.smithi038 (mon.0) 167 : audit [INF] from='osd.1 [v2:172.21.15.38:6810/341024688,v1:172.21.15.38:6811/341024688]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-15T18:38:08.522 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:38:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-1[83993]: debug 2024-02-15T18:38:08.058+0000 7fa0b0d40700 -1 osd.1 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-15T18:38:09.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:08.053463+0000 mon.smithi038 (mon.0) 168 : audit [INF] from='osd.1 [v2:172.21.15.38:6810/341024688,v1:172.21.15.38:6811/341024688]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-15T18:38:09.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:08.053576+0000 mon.smithi038 (mon.0) 169 : cluster [DBG] osdmap e54: 6 total, 5 up, 6 in 2024-02-15T18:38:09.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:08.054533+0000 mon.smithi038 (mon.0) 170 : audit [INF] from='osd.1 [v2:172.21.15.38:6810/341024688,v1:172.21.15.38:6811/341024688]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-02-15T18:38:09.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:08.950939+0000 mgr.smithi038.khesbl (mgr.14664) 113 : cluster [DBG] pgmap v75: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 52/342 objects degraded (15.205%) 2024-02-15T18:38:09.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:08.053463+0000 mon.smithi038 (mon.0) 168 : audit [INF] from='osd.1 [v2:172.21.15.38:6810/341024688,v1:172.21.15.38:6811/341024688]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-15T18:38:09.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:08.053576+0000 mon.smithi038 (mon.0) 169 : cluster [DBG] osdmap e54: 6 total, 5 up, 6 in 2024-02-15T18:38:09.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:08.054533+0000 mon.smithi038 (mon.0) 170 : audit [INF] from='osd.1 [v2:172.21.15.38:6810/341024688,v1:172.21.15.38:6811/341024688]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-02-15T18:38:09.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:08.950939+0000 mgr.smithi038.khesbl (mgr.14664) 113 : cluster [DBG] pgmap v75: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 52/342 objects degraded (15.205%) 2024-02-15T18:38:10.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:09.046955+0000 mon.smithi038 (mon.0) 171 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:38:10.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:09.055582+0000 mon.smithi038 (mon.0) 172 : cluster [INF] osd.1 [v2:172.21.15.38:6810/341024688,v1:172.21.15.38:6811/341024688] boot 2024-02-15T18:38:10.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:09.055625+0000 mon.smithi038 (mon.0) 173 : cluster [DBG] osdmap e55: 6 total, 6 up, 6 in 2024-02-15T18:38:10.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:09.057946+0000 mon.smithi038 (mon.0) 174 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:38:10.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:10.052838+0000 mon.smithi038 (mon.0) 175 : cluster [DBG] osdmap e56: 6 total, 6 up, 6 in 2024-02-15T18:38:10.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:09.046955+0000 mon.smithi038 (mon.0) 171 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:38:10.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:09.055582+0000 mon.smithi038 (mon.0) 172 : cluster [INF] osd.1 [v2:172.21.15.38:6810/341024688,v1:172.21.15.38:6811/341024688] boot 2024-02-15T18:38:10.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:09.055625+0000 mon.smithi038 (mon.0) 173 : cluster [DBG] osdmap e55: 6 total, 6 up, 6 in 2024-02-15T18:38:10.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:09.057946+0000 mon.smithi038 (mon.0) 174 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:38:10.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:10.052838+0000 mon.smithi038 (mon.0) 175 : cluster [DBG] osdmap e56: 6 total, 6 up, 6 in 2024-02-15T18:38:11.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:10.951524+0000 mgr.smithi038.khesbl (mgr.14664) 114 : cluster [DBG] pgmap v78: 65 pgs: 2 peering, 15 active+undersized+degraded, 17 active+undersized, 31 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 52/342 objects degraded (15.205%) 2024-02-15T18:38:11.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:10.951524+0000 mgr.smithi038.khesbl (mgr.14664) 114 : cluster [DBG] pgmap v78: 65 pgs: 2 peering, 15 active+undersized+degraded, 17 active+undersized, 31 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 52/342 objects degraded (15.205%) 2024-02-15T18:38:13.499 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:12.952147+0000 mgr.smithi038.khesbl (mgr.14664) 115 : cluster [DBG] pgmap v79: 65 pgs: 2 peering, 15 active+undersized+degraded, 17 active+undersized, 31 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 52/342 objects degraded (15.205%) 2024-02-15T18:38:13.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:12.952147+0000 mgr.smithi038.khesbl (mgr.14664) 115 : cluster [DBG] pgmap v79: 65 pgs: 2 peering, 15 active+undersized+degraded, 17 active+undersized, 31 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 52/342 objects degraded (15.205%) 2024-02-15T18:38:15.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:14.952822+0000 mgr.smithi038.khesbl (mgr.14664) 116 : cluster [DBG] pgmap v80: 65 pgs: 65 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:38:15.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:15.003594+0000 mon.smithi038 (mon.0) 176 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 52/342 objects degraded (15.205%), 15 pgs degraded) 2024-02-15T18:38:15.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:15.003632+0000 mon.smithi038 (mon.0) 177 : cluster [INF] Cluster is now healthy 2024-02-15T18:38:15.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:14.952822+0000 mgr.smithi038.khesbl (mgr.14664) 116 : cluster [DBG] pgmap v80: 65 pgs: 65 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:38:15.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:15.003594+0000 mon.smithi038 (mon.0) 176 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 52/342 objects degraded (15.205%), 15 pgs degraded) 2024-02-15T18:38:15.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:15.003632+0000 mon.smithi038 (mon.0) 177 : cluster [INF] Cluster is now healthy 2024-02-15T18:38:17.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:16.953226+0000 mgr.smithi038.khesbl (mgr.14664) 117 : cluster [DBG] pgmap v81: 65 pgs: 65 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:38:17.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:16.953226+0000 mgr.smithi038.khesbl (mgr.14664) 117 : cluster [DBG] pgmap v81: 65 pgs: 65 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:38:19.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:18.148608+0000 mon.smithi038 (mon.0) 178 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:19.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:18.953809+0000 mgr.smithi038.khesbl (mgr.14664) 118 : cluster [DBG] pgmap v82: 65 pgs: 65 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:38:19.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:18.148608+0000 mon.smithi038 (mon.0) 178 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:19.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:18.953809+0000 mgr.smithi038.khesbl (mgr.14664) 118 : cluster [DBG] pgmap v82: 65 pgs: 65 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:38:20.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:20 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:19.684811+0000 mon.smithi038 (mon.0) 179 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-15T18:38:20.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:20 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:19.685020+0000 mgr.smithi038.khesbl (mgr.14664) 119 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-15T18:38:20.497 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:20 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:38:19.686087+0000 mgr.smithi038.khesbl (mgr.14664) 120 : cephadm [INF] Upgrade: osd.2 is safe to restart 2024-02-15T18:38:20.554 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:20 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:19.684811+0000 mon.smithi038 (mon.0) 179 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-15T18:38:20.554 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:20 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:19.685020+0000 mgr.smithi038.khesbl (mgr.14664) 119 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-15T18:38:20.554 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:20 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:38:19.686087+0000 mgr.smithi038.khesbl (mgr.14664) 120 : cephadm [INF] Upgrade: osd.2 is safe to restart 2024-02-15T18:38:21.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:20.954267+0000 mgr.smithi038.khesbl (mgr.14664) 121 : cluster [DBG] pgmap v83: 65 pgs: 65 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:38:21.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:20.954267+0000 mgr.smithi038.khesbl (mgr.14664) 121 : cluster [DBG] pgmap v83: 65 pgs: 65 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:38:22.550 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:38:21.297993+0000 mgr.smithi038.khesbl (mgr.14664) 122 : cephadm [INF] Upgrade: Updating osd.2 2024-02-15T18:38:22.551 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:21.302610+0000 mon.smithi038 (mon.0) 180 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:22.551 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:21.303427+0000 mon.smithi038 (mon.0) 181 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-15T18:38:22.551 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:21.304008+0000 mon.smithi038 (mon.0) 182 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:38:22.551 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:38:21.304999+0000 mgr.smithi038.khesbl (mgr.14664) 123 : cephadm [INF] Deploying daemon osd.2 on smithi038 2024-02-15T18:38:22.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:38:21.297993+0000 mgr.smithi038.khesbl (mgr.14664) 122 : cephadm [INF] Upgrade: Updating osd.2 2024-02-15T18:38:22.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:21.302610+0000 mon.smithi038 (mon.0) 180 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:22.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:21.303427+0000 mon.smithi038 (mon.0) 181 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-15T18:38:22.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:21.304008+0000 mon.smithi038 (mon.0) 182 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:38:22.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:38:21.304999+0000 mgr.smithi038.khesbl (mgr.14664) 123 : cephadm [INF] Deploying daemon osd.2 on smithi038 2024-02-15T18:38:23.500 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:22.954619+0000 mgr.smithi038.khesbl (mgr.14664) 124 : cluster [DBG] pgmap v84: 65 pgs: 65 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:38:23.500 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:23.185807+0000 mon.smithi038 (mon.0) 183 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:23.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:22.954619+0000 mgr.smithi038.khesbl (mgr.14664) 124 : cluster [DBG] pgmap v84: 65 pgs: 65 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:38:23.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:23.185807+0000 mon.smithi038 (mon.0) 183 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:24.772 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:24 smithi038 systemd[1]: Stopping Ceph osd.2 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:38:25.113 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:24.875653+0000 mon.smithi038 (mon.0) 184 : cluster [DBG] osd.2 reported immediately failed by osd.0 2024-02-15T18:38:25.114 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:24.875679+0000 mon.smithi038 (mon.0) 185 : cluster [INF] osd.2 failed (root=default,host=smithi038) (connection refused reported by osd.0) 2024-02-15T18:38:25.114 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.2[56774]: debug 2024-02-15T18:38:24.782+0000 7fb036070700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:38:25.114 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.2[56774]: debug 2024-02-15T18:38:24.782+0000 7fb036070700 -1 osd.2 56 *** Got signal Terminated *** 2024-02-15T18:38:25.114 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.2[56774]: debug 2024-02-15T18:38:24.782+0000 7fb036070700 -1 osd.2 56 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:38:25.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:24 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:24.875653+0000 mon.smithi038 (mon.0) 184 : cluster [DBG] osd.2 reported immediately failed by osd.0 2024-02-15T18:38:25.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:24 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:24.875679+0000 mon.smithi038 (mon.0) 185 : cluster [INF] osd.2 failed (root=default,host=smithi038) (connection refused reported by osd.0) 2024-02-15T18:38:25.454 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:25 smithi038 bash[85938]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.2 2024-02-15T18:38:25.716 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:25 smithi038 bash[85992]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.2 2024-02-15T18:38:26.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:24.926134+0000 mon.smithi038 (mon.0) 186 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:38:26.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:24.932820+0000 mon.smithi038 (mon.0) 187 : cluster [DBG] osdmap e57: 6 total, 5 up, 6 in 2024-02-15T18:38:26.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:24.955005+0000 mgr.smithi038.khesbl (mgr.14664) 125 : cluster [DBG] pgmap v86: 65 pgs: 9 stale+active+clean, 56 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:38:26.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:25.937531+0000 mon.smithi038 (mon.0) 188 : cluster [DBG] osdmap e58: 6 total, 5 up, 6 in 2024-02-15T18:38:26.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:24.926134+0000 mon.smithi038 (mon.0) 186 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:38:26.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:24.932820+0000 mon.smithi038 (mon.0) 187 : cluster [DBG] osdmap e57: 6 total, 5 up, 6 in 2024-02-15T18:38:26.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:24.955005+0000 mgr.smithi038.khesbl (mgr.14664) 125 : cluster [DBG] pgmap v86: 65 pgs: 9 stale+active+clean, 56 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:38:26.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:25.937531+0000 mon.smithi038 (mon.0) 188 : cluster [DBG] osdmap e58: 6 total, 5 up, 6 in 2024-02-15T18:38:27.956 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:27 smithi038 systemd[1]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.2.service: Succeeded. 2024-02-15T18:38:27.956 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:27 smithi038 systemd[1]: Stopped Ceph osd.2 for 7e05753e-cc2f-11ee-95ba-87774f69a715. 2024-02-15T18:38:27.956 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:27 smithi038 systemd[1]: Starting Ceph osd.2 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:38:28.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:26.955511+0000 mgr.smithi038.khesbl (mgr.14664) 126 : cluster [DBG] pgmap v88: 65 pgs: 9 stale+active+clean, 56 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:38:28.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:26.955511+0000 mgr.smithi038.khesbl (mgr.14664) 126 : cluster [DBG] pgmap v88: 65 pgs: 9 stale+active+clean, 56 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:38:28.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.404+0000 7f14787a7700 1 -- 172.21.15.38:0/3240507222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14700f66e0 msgr2=0x7f14700f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:28.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.404+0000 7f14787a7700 1 --2- 172.21.15.38:0/3240507222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14700f66e0 0x7f14700f6b00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f146c004660 tx=0x7f146c009b30).stop 2024-02-15T18:38:28.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.404+0000 7f14787a7700 1 -- 172.21.15.38:0/3240507222 shutdown_connections 2024-02-15T18:38:28.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.404+0000 7f14787a7700 1 --2- 172.21.15.38:0/3240507222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14700f7980 0x7f14700f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:28.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.404+0000 7f14787a7700 1 --2- 172.21.15.38:0/3240507222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14700f66e0 0x7f14700f6b00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:28.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.404+0000 7f14787a7700 1 -- 172.21.15.38:0/3240507222 >> 172.21.15.38:0/3240507222 conn(0x7f14700f1e40 msgr2=0x7f14700f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:38:28.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.404+0000 7f14787a7700 1 -- 172.21.15.38:0/3240507222 shutdown_connections 2024-02-15T18:38:28.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.404+0000 7f14787a7700 1 -- 172.21.15.38:0/3240507222 wait complete. 2024-02-15T18:38:28.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.405+0000 7f14787a7700 1 Processor -- start 2024-02-15T18:38:28.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.405+0000 7f14787a7700 1 -- start start 2024-02-15T18:38:28.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.405+0000 7f14787a7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14700f66e0 0x7f147006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:38:28.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.405+0000 7f14787a7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14700f7980 0x7f1470069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:38:28.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.405+0000 7f14787a7700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147006a380 con 0x7f14700f7980 2024-02-15T18:38:28.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.405+0000 7f14787a7700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147006a4c0 con 0x7f14700f66e0 2024-02-15T18:38:28.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.405+0000 7f1475d42700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14700f7980 0x7f1470069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:38:28.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.405+0000 7f1475d42700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14700f7980 0x7f1470069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51412/0 (socket says 172.21.15.38:51412) 2024-02-15T18:38:28.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.405+0000 7f1475d42700 1 -- 172.21.15.38:0/45118929 learned_addr learned my addr 172.21.15.38:0/45118929 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:38:28.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.405+0000 7f1476543700 1 --2- 172.21.15.38:0/45118929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14700f66e0 0x7f147006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:38:28.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.405+0000 7f1475d42700 1 -- 172.21.15.38:0/45118929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14700f66e0 msgr2=0x7f147006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:28.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.405+0000 7f1475d42700 1 --2- 172.21.15.38:0/45118929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14700f66e0 0x7f147006b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:28.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.405+0000 7f1475d42700 1 -- 172.21.15.38:0/45118929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c005040 con 0x7f14700f7980 2024-02-15T18:38:28.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.406+0000 7f1475d42700 1 --2- 172.21.15.38:0/45118929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14700f7980 0x7f1470069e20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f14600026d0 tx=0x7f1460009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:38:28.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.406+0000 7f1466ffd700 1 -- 172.21.15.38:0/45118929 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14600079c0 con 0x7f14700f7980 2024-02-15T18:38:28.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.406+0000 7f1466ffd700 1 -- 172.21.15.38:0/45118929 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f146000f040 con 0x7f14700f7980 2024-02-15T18:38:28.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.406+0000 7f14787a7700 1 -- 172.21.15.38:0/45118929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f147006a7a0 con 0x7f14700f7980 2024-02-15T18:38:28.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.406+0000 7f1466ffd700 1 -- 172.21.15.38:0/45118929 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14600149f0 con 0x7f14700f7980 2024-02-15T18:38:28.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.406+0000 7f14787a7700 1 -- 172.21.15.38:0/45118929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147010f060 con 0x7f14700f7980 2024-02-15T18:38:28.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.407+0000 7f14787a7700 1 -- 172.21.15.38:0/45118929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1470045ad0 con 0x7f14700f7980 2024-02-15T18:38:28.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.407+0000 7f1466ffd700 1 -- 172.21.15.38:0/45118929 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f1460014b50 con 0x7f14700f7980 2024-02-15T18:38:28.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.407+0000 7f1466ffd700 1 --2- 172.21.15.38:0/45118929 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f145c060f20 0x7f145c0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:38:28.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.408+0000 7f1466ffd700 1 -- 172.21.15.38:0/45118929 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f146007c0c0 con 0x7f14700f7980 2024-02-15T18:38:28.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.408+0000 7f1476543700 1 --2- 172.21.15.38:0/45118929 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f145c060f20 0x7f145c0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:38:28.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.408+0000 7f1476543700 1 --2- 172.21.15.38:0/45118929 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f145c060f20 0x7f145c0633e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f147006b730 tx=0x7f146c009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:38:28.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.412+0000 7f1466ffd700 1 -- 172.21.15.38:0/45118929 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f146004c540 con 0x7f14700f7980 2024-02-15T18:38:28.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.602+0000 7f14787a7700 1 -- 172.21.15.38:0/45118929 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f14700ff0c0 con 0x7f145c060f20 2024-02-15T18:38:28.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.604+0000 7f1466ffd700 1 -- 172.21.15.38:0/45118929 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f14700ff0c0 con 0x7f145c060f20 2024-02-15T18:38:28.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.606+0000 7f14787a7700 1 -- 172.21.15.38:0/45118929 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f145c060f20 msgr2=0x7f145c0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:28.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.606+0000 7f14787a7700 1 --2- 172.21.15.38:0/45118929 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f145c060f20 0x7f145c0633e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f147006b730 tx=0x7f146c009b30).stop 2024-02-15T18:38:28.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.606+0000 7f14787a7700 1 -- 172.21.15.38:0/45118929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14700f7980 msgr2=0x7f1470069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:28.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.606+0000 7f14787a7700 1 --2- 172.21.15.38:0/45118929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14700f7980 0x7f1470069e20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f14600026d0 tx=0x7f1460009de0).stop 2024-02-15T18:38:28.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.606+0000 7f14787a7700 1 -- 172.21.15.38:0/45118929 shutdown_connections 2024-02-15T18:38:28.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.606+0000 7f14787a7700 1 --2- 172.21.15.38:0/45118929 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f145c060f20 0x7f145c0633e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:28.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.606+0000 7f14787a7700 1 --2- 172.21.15.38:0/45118929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14700f66e0 0x7f147006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:28.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.606+0000 7f14787a7700 1 --2- 172.21.15.38:0/45118929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14700f7980 0x7f1470069e20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:28.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.606+0000 7f14787a7700 1 -- 172.21.15.38:0/45118929 >> 172.21.15.38:0/45118929 conn(0x7f14700f1e40 msgr2=0x7f14700fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:38:28.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.606+0000 7f14787a7700 1 -- 172.21.15.38:0/45118929 shutdown_connections 2024-02-15T18:38:28.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.606+0000 7f14787a7700 1 -- 172.21.15.38:0/45118929 wait complete. 2024-02-15T18:38:28.612 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-02-15T18:38:28.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.719+0000 7f4580cc0700 1 -- 172.21.15.38:0/377305212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f457c0f96f0 msgr2=0x7f457c0fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:28.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.719+0000 7f4580cc0700 1 --2- 172.21.15.38:0/377305212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f457c0f96f0 0x7f457c0fbae0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f4570004660 tx=0x7f4570009b30).stop 2024-02-15T18:38:28.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.719+0000 7f4580cc0700 1 -- 172.21.15.38:0/377305212 shutdown_connections 2024-02-15T18:38:28.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.719+0000 7f4580cc0700 1 --2- 172.21.15.38:0/377305212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f457c0f96f0 0x7f457c0fbae0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:28.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.719+0000 7f4580cc0700 1 --2- 172.21.15.38:0/377305212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f457c0602b0 0x7f457c0f9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:28.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.719+0000 7f4580cc0700 1 -- 172.21.15.38:0/377305212 >> 172.21.15.38:0/377305212 conn(0x7f457c0f0b90 msgr2=0x7f457c0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:38:28.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.719+0000 7f4580cc0700 1 -- 172.21.15.38:0/377305212 shutdown_connections 2024-02-15T18:38:28.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.719+0000 7f4580cc0700 1 -- 172.21.15.38:0/377305212 wait complete. 2024-02-15T18:38:28.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.720+0000 7f4580cc0700 1 Processor -- start 2024-02-15T18:38:28.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.720+0000 7f4580cc0700 1 -- start start 2024-02-15T18:38:28.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.720+0000 7f4580cc0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f457c0602b0 0x7f457c106e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:38:28.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.720+0000 7f4580cc0700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f457c0f96f0 0x7f457c107310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:38:28.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.720+0000 7f4580cc0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457c107910 con 0x7f457c0602b0 2024-02-15T18:38:28.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.720+0000 7f4580cc0700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457c108120 con 0x7f457c0f96f0 2024-02-15T18:38:28.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.720+0000 7f457b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f457c0602b0 0x7f457c106e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:38:28.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.720+0000 7f457b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f457c0602b0 0x7f457c106e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51426/0 (socket says 172.21.15.38:51426) 2024-02-15T18:38:28.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.720+0000 7f457b7fe700 1 -- 172.21.15.38:0/2428332568 learned_addr learned my addr 172.21.15.38:0/2428332568 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:38:28.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.720+0000 7f457affd700 1 --2- 172.21.15.38:0/2428332568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f457c0f96f0 0x7f457c107310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:38:28.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.721+0000 7f457b7fe700 1 -- 172.21.15.38:0/2428332568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f457c0f96f0 msgr2=0x7f457c107310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:28.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.721+0000 7f457b7fe700 1 --2- 172.21.15.38:0/2428332568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f457c0f96f0 0x7f457c107310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:28.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.721+0000 7f457b7fe700 1 -- 172.21.15.38:0/2428332568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4570005040 con 0x7f457c0602b0 2024-02-15T18:38:28.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.721+0000 7f457b7fe700 1 --2- 172.21.15.38:0/2428332568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f457c0602b0 0x7f457c106e20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f456c00c530 tx=0x7f456c007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:38:28.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.721+0000 7f4563fff700 1 -- 172.21.15.38:0/2428332568 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f456c00f650 con 0x7f457c0602b0 2024-02-15T18:38:28.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.721+0000 7f4563fff700 1 -- 172.21.15.38:0/2428332568 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f456c00fca0 con 0x7f457c0602b0 2024-02-15T18:38:28.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.721+0000 7f4580cc0700 1 -- 172.21.15.38:0/2428332568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457c1083a0 con 0x7f457c0602b0 2024-02-15T18:38:28.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.721+0000 7f4563fff700 1 -- 172.21.15.38:0/2428332568 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f456c012d40 con 0x7f457c0602b0 2024-02-15T18:38:28.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.721+0000 7f4580cc0700 1 -- 172.21.15.38:0/2428332568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f457c108960 con 0x7f457c0602b0 2024-02-15T18:38:28.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.722+0000 7f4580cc0700 1 -- 172.21.15.38:0/2428332568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f457c059100 con 0x7f457c0602b0 2024-02-15T18:38:28.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.722+0000 7f4563fff700 1 -- 172.21.15.38:0/2428332568 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f456c010070 con 0x7f457c0602b0 2024-02-15T18:38:28.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.723+0000 7f4563fff700 1 --2- 172.21.15.38:0/2428332568 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4564060f70 0x7f4564063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:38:28.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.723+0000 7f4563fff700 1 -- 172.21.15.38:0/2428332568 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f456c07bd90 con 0x7f457c0602b0 2024-02-15T18:38:28.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.723+0000 7f457affd700 1 --2- 172.21.15.38:0/2428332568 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4564060f70 0x7f4564063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:38:28.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.723+0000 7f457affd700 1 --2- 172.21.15.38:0/2428332568 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4564060f70 0x7f4564063430 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f4570009f50 tx=0x7f4570009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:38:28.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.727+0000 7f4563fff700 1 -- 172.21.15.38:0/2428332568 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f456c047f60 con 0x7f457c0602b0 2024-02-15T18:38:28.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.918+0000 7f4580cc0700 1 -- 172.21.15.38:0/2428332568 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f457c0fef00 con 0x7f4564060f70 2024-02-15T18:38:28.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.919+0000 7f4563fff700 1 -- 172.21.15.38:0/2428332568 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f457c0fef00 con 0x7f4564060f70 2024-02-15T18:38:28.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.921+0000 7f4580cc0700 1 -- 172.21.15.38:0/2428332568 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4564060f70 msgr2=0x7f4564063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:28.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.921+0000 7f4580cc0700 1 --2- 172.21.15.38:0/2428332568 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4564060f70 0x7f4564063430 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f4570009f50 tx=0x7f4570009b30).stop 2024-02-15T18:38:28.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.921+0000 7f4580cc0700 1 -- 172.21.15.38:0/2428332568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f457c0602b0 msgr2=0x7f457c106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:28.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.921+0000 7f4580cc0700 1 --2- 172.21.15.38:0/2428332568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f457c0602b0 0x7f457c106e20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f456c00c530 tx=0x7f456c007e20).stop 2024-02-15T18:38:28.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.921+0000 7f4580cc0700 1 -- 172.21.15.38:0/2428332568 shutdown_connections 2024-02-15T18:38:28.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.921+0000 7f4580cc0700 1 --2- 172.21.15.38:0/2428332568 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4564060f70 0x7f4564063430 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:28.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.921+0000 7f4580cc0700 1 --2- 172.21.15.38:0/2428332568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f457c0602b0 0x7f457c106e20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:28.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.921+0000 7f4580cc0700 1 --2- 172.21.15.38:0/2428332568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f457c0f96f0 0x7f457c107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:28.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.921+0000 7f4580cc0700 1 -- 172.21.15.38:0/2428332568 >> 172.21.15.38:0/2428332568 conn(0x7f457c0f0b90 msgr2=0x7f457c0f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:38:28.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.921+0000 7f4580cc0700 1 -- 172.21.15.38:0/2428332568 shutdown_connections 2024-02-15T18:38:28.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:28.921+0000 7f4580cc0700 1 -- 172.21.15.38:0/2428332568 wait complete. 2024-02-15T18:38:29.007 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:28 smithi038 bash[86250]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-02-15T18:38:29.007 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:28 smithi038 bash[86250]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-15T18:38:29.007 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:28 smithi038 bash[86250]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-2 --no-mon-config 2024-02-15T18:38:29.007 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:28 smithi038 bash[86250]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2/block 2024-02-15T18:38:29.007 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:28 smithi038 bash[86250]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-2/block 2024-02-15T18:38:29.007 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:28 smithi038 bash[86250]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-15T18:38:29.007 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:28 smithi038 bash[86250]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-15T18:38:29.007 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:28 smithi038 bash[86250]: --> ceph-volume lvm activate successful for osd ID: 2 2024-02-15T18:38:29.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.027+0000 7fd0d9f9b700 1 -- 172.21.15.38:0/3690047125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0d40f6860 msgr2=0x7fd0d40f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:29.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.027+0000 7fd0d9f9b700 1 --2- 172.21.15.38:0/3690047125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0d40f6860 0x7fd0d40f6cc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fd0c4004660 tx=0x7fd0c4009b30).stop 2024-02-15T18:38:29.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.027+0000 7fd0d9f9b700 1 -- 172.21.15.38:0/3690047125 shutdown_connections 2024-02-15T18:38:29.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.027+0000 7fd0d9f9b700 1 --2- 172.21.15.38:0/3690047125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0d40f6860 0x7fd0d40f6cc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:29.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.027+0000 7fd0d9f9b700 1 --2- 172.21.15.38:0/3690047125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0d40f55c0 0x7fd0d40f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:29.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.027+0000 7fd0d9f9b700 1 -- 172.21.15.38:0/3690047125 >> 172.21.15.38:0/3690047125 conn(0x7fd0d40f0d40 msgr2=0x7fd0d40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:38:29.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.027+0000 7fd0d9f9b700 1 -- 172.21.15.38:0/3690047125 shutdown_connections 2024-02-15T18:38:29.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.027+0000 7fd0d9f9b700 1 -- 172.21.15.38:0/3690047125 wait complete. 2024-02-15T18:38:29.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.028+0000 7fd0d9f9b700 1 Processor -- start 2024-02-15T18:38:29.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.028+0000 7fd0d9f9b700 1 -- start start 2024-02-15T18:38:29.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.028+0000 7fd0d9f9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0d40f55c0 0x7fd0d4106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:38:29.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.028+0000 7fd0d9f9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0d40f6860 0x7fd0d41074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:38:29.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.028+0000 7fd0d9f9b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d4107a60 con 0x7fd0d40f6860 2024-02-15T18:38:29.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.028+0000 7fd0d9f9b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d4107ba0 con 0x7fd0d40f55c0 2024-02-15T18:38:29.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.028+0000 7fd0d2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0d40f6860 0x7fd0d41074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:38:29.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.028+0000 7fd0d2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0d40f6860 0x7fd0d41074b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51440/0 (socket says 172.21.15.38:51440) 2024-02-15T18:38:29.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.028+0000 7fd0d2ffd700 1 -- 172.21.15.38:0/3818354265 learned_addr learned my addr 172.21.15.38:0/3818354265 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:38:29.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.028+0000 7fd0d37fe700 1 --2- 172.21.15.38:0/3818354265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0d40f55c0 0x7fd0d4106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:38:29.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.028+0000 7fd0d2ffd700 1 -- 172.21.15.38:0/3818354265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0d40f55c0 msgr2=0x7fd0d4106fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:29.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.028+0000 7fd0d2ffd700 1 --2- 172.21.15.38:0/3818354265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0d40f55c0 0x7fd0d4106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:29.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.028+0000 7fd0d2ffd700 1 -- 172.21.15.38:0/3818354265 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c4005040 con 0x7fd0d40f6860 2024-02-15T18:38:29.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.029+0000 7fd0d2ffd700 1 --2- 172.21.15.38:0/3818354265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0d40f6860 0x7fd0d41074b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fd0c4009f50 tx=0x7fd0c4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:38:29.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.029+0000 7fd0bbfff700 1 -- 172.21.15.38:0/3818354265 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0c4018070 con 0x7fd0d40f6860 2024-02-15T18:38:29.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.029+0000 7fd0bbfff700 1 -- 172.21.15.38:0/3818354265 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fd0c4007ba0 con 0x7fd0d40f6860 2024-02-15T18:38:29.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.029+0000 7fd0d9f9b700 1 -- 172.21.15.38:0/3818354265 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d41084e0 con 0x7fd0d40f6860 2024-02-15T18:38:29.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.029+0000 7fd0bbfff700 1 -- 172.21.15.38:0/3818354265 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0c401c620 con 0x7fd0d40f6860 2024-02-15T18:38:29.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.029+0000 7fd0d9f9b700 1 -- 172.21.15.38:0/3818354265 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d4108a40 con 0x7fd0d40f6860 2024-02-15T18:38:29.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.030+0000 7fd0d9f9b700 1 -- 172.21.15.38:0/3818354265 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d4045ad0 con 0x7fd0d40f6860 2024-02-15T18:38:29.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.030+0000 7fd0bbfff700 1 -- 172.21.15.38:0/3818354265 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fd0c401c890 con 0x7fd0d40f6860 2024-02-15T18:38:29.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.030+0000 7fd0bbfff700 1 --2- 172.21.15.38:0/3818354265 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd0bc060f20 0x7fd0bc0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:38:29.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.031+0000 7fd0bbfff700 1 -- 172.21.15.38:0/3818354265 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fd0c407dff0 con 0x7fd0d40f6860 2024-02-15T18:38:29.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.031+0000 7fd0d37fe700 1 --2- 172.21.15.38:0/3818354265 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd0bc060f20 0x7fd0bc0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:38:29.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.031+0000 7fd0d37fe700 1 --2- 172.21.15.38:0/3818354265 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd0bc060f20 0x7fd0bc0633e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fd0d40f66c0 tx=0x7fd0c800b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:38:29.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.035+0000 7fd0bbfff700 1 -- 172.21.15.38:0/3818354265 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd0c4051ff0 con 0x7fd0d40f6860 2024-02-15T18:38:29.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.220+0000 7fd0d9f9b700 1 -- 172.21.15.38:0/3818354265 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0d40faca0 con 0x7fd0bc060f20 2024-02-15T18:38:29.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.229+0000 7fd0bbfff700 1 -- 172.21.15.38:0/3818354265 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7fd0d40faca0 con 0x7fd0bc060f20 2024-02-15T18:38:29.220 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:38:29.220 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (9m) 24s ago 11m 19.3M - 0.20.0 0881eb8f169f c2034c318e7c 2024-02-15T18:38:29.220 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (87s) 24s ago 11m 6635k - 16.2.14-521-g0e714d9a 1fbc92c859dc c2fee0824d4b 2024-02-15T18:38:29.220 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi121 smithi121 running (81s) 77s ago 10m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 05c1a5501a35 2024-02-15T18:38:29.220 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (9m) 24s ago 10m 33.3M - 6.7.4 557c83e11646 2dcccfbfbd2a 2024-02-15T18:38:29.220 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.hxoggl smithi038 running (5m) 24s ago 5m 14.7M - 16.2.4 8d91d370c2b8 be2533754b9c 2024-02-15T18:38:29.220 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.qgkzgn smithi038 running (5m) 24s ago 5m 14.3M - 16.2.4 8d91d370c2b8 dd3ffafba9bd 2024-02-15T18:38:29.221 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.mvumlh smithi121 running (5m) 77s ago 5m 116M - 16.2.4 8d91d370c2b8 88b7d876a39b 2024-02-15T18:38:29.221 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.ovuzki smithi121 running (5m) 77s ago 5m 124M - 16.2.4 8d91d370c2b8 21cacf0012f9 2024-02-15T18:38:29.221 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.khesbl smithi038 *:8443,9283 running (2m) 24s ago 12m 431M - 16.2.14-521-g0e714d9a 1fbc92c859dc e642f3c2cfc9 2024-02-15T18:38:29.221 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi121.ckiryi smithi121 *:8443,9283 running (2m) 77s ago 10m 385M - 16.2.14-521-g0e714d9a 1fbc92c859dc 82515b025594 2024-02-15T18:38:29.221 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (117s) 24s ago 12m 42.2M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc d3e8317a30ca 2024-02-15T18:38:29.221 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi121 smithi121 running (97s) 77s ago 10m 26.2M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 42e1dda1a39d 2024-02-15T18:38:29.221 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (10m) 24s ago 10m 17.1M - 0.18.1 e5a616e4b9cf 037f60e38158 2024-02-15T18:38:29.221 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi121 smithi121 *:9100 running (9m) 77s ago 9m 17.2M - 0.18.1 e5a616e4b9cf a867bff85e18 2024-02-15T18:38:29.221 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (63s) 24s ago 9m 192M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 623ac7ecbb0a 2024-02-15T18:38:29.221 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (31s) 24s ago 8m 14.9M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 0f9e8e1f255a 2024-02-15T18:38:29.221 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (8m) 24s ago 8m 353M 4096M 16.2.4 8d91d370c2b8 6d6ee813e5be 2024-02-15T18:38:29.221 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi121 running (8m) 77s ago 8m 449M 4096M 16.2.4 8d91d370c2b8 cba2d9ed4730 2024-02-15T18:38:29.221 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi121 running (7m) 77s ago 7m 429M 4096M 16.2.4 8d91d370c2b8 ac17ad7dc628 2024-02-15T18:38:29.221 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi121 running (7m) 77s ago 7m 378M 4096M 16.2.4 8d91d370c2b8 cdc8c669c4db 2024-02-15T18:38:29.222 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (2m) 24s ago 10m 59.6M - 2.18.1 de242295e225 67e33f9de4e6 2024-02-15T18:38:29.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.231+0000 7fd0d9f9b700 1 -- 172.21.15.38:0/3818354265 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd0bc060f20 msgr2=0x7fd0bc0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:29.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.231+0000 7fd0d9f9b700 1 --2- 172.21.15.38:0/3818354265 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd0bc060f20 0x7fd0bc0633e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fd0d40f66c0 tx=0x7fd0c800b040).stop 2024-02-15T18:38:29.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.231+0000 7fd0d9f9b700 1 -- 172.21.15.38:0/3818354265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0d40f6860 msgr2=0x7fd0d41074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:29.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.231+0000 7fd0d9f9b700 1 --2- 172.21.15.38:0/3818354265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0d40f6860 0x7fd0d41074b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fd0c4009f50 tx=0x7fd0c4009b30).stop 2024-02-15T18:38:29.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.231+0000 7fd0d9f9b700 1 -- 172.21.15.38:0/3818354265 shutdown_connections 2024-02-15T18:38:29.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.231+0000 7fd0d9f9b700 1 --2- 172.21.15.38:0/3818354265 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd0bc060f20 0x7fd0bc0633e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:29.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.231+0000 7fd0d9f9b700 1 --2- 172.21.15.38:0/3818354265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0d40f55c0 0x7fd0d4106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:29.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.231+0000 7fd0d9f9b700 1 --2- 172.21.15.38:0/3818354265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0d40f6860 0x7fd0d41074b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:29.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.231+0000 7fd0d9f9b700 1 -- 172.21.15.38:0/3818354265 >> 172.21.15.38:0/3818354265 conn(0x7fd0d40f0d40 msgr2=0x7fd0d405fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:38:29.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.232+0000 7fd0d9f9b700 1 -- 172.21.15.38:0/3818354265 shutdown_connections 2024-02-15T18:38:29.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.232+0000 7fd0d9f9b700 1 -- 172.21.15.38:0/3818354265 wait complete. 2024-02-15T18:38:29.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:28.605001+0000 mgr.smithi038.khesbl (mgr.14664) 127 : audit [DBG] from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:38:29.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:28.920783+0000 mgr.smithi038.khesbl (mgr.14664) 128 : audit [DBG] from='client.34184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:38:29.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:28.955961+0000 mgr.smithi038.khesbl (mgr.14664) 129 : cluster [DBG] pgmap v89: 65 pgs: 3 active+undersized+degraded, 5 stale+active+clean, 4 active+undersized, 53 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 13/342 objects degraded (3.801%) 2024-02-15T18:38:29.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:28.966001+0000 mon.smithi038 (mon.0) 189 : cluster [WRN] Health check failed: Degraded data redundancy: 13/342 objects degraded (3.801%), 3 pgs degraded (PG_DEGRADED) 2024-02-15T18:38:29.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.332+0000 7f07e3a2b700 1 -- 172.21.15.38:0/3718797591 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07dc06a3b0 msgr2=0x7f07dc06a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:29.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.332+0000 7f07e3a2b700 1 --2- 172.21.15.38:0/3718797591 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07dc06a3b0 0x7f07dc06a830 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f07cc004660 tx=0x7f07cc009b30).stop 2024-02-15T18:38:29.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.332+0000 7f07e3a2b700 1 -- 172.21.15.38:0/3718797591 shutdown_connections 2024-02-15T18:38:29.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.332+0000 7f07e3a2b700 1 --2- 172.21.15.38:0/3718797591 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07dc06a3b0 0x7f07dc06a830 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:29.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.332+0000 7f07e3a2b700 1 --2- 172.21.15.38:0/3718797591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07dc06b9c0 0x7f07dc069ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:29.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.332+0000 7f07e3a2b700 1 -- 172.21.15.38:0/3718797591 >> 172.21.15.38:0/3718797591 conn(0x7f07dc0f1e40 msgr2=0x7f07dc0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:38:29.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.332+0000 7f07e3a2b700 1 -- 172.21.15.38:0/3718797591 shutdown_connections 2024-02-15T18:38:29.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.333+0000 7f07e3a2b700 1 -- 172.21.15.38:0/3718797591 wait complete. 2024-02-15T18:38:29.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.333+0000 7f07e3a2b700 1 Processor -- start 2024-02-15T18:38:29.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.333+0000 7f07e3a2b700 1 -- start start 2024-02-15T18:38:29.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.333+0000 7f07e3a2b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07dc06a3b0 0x7f07dc1091d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:38:29.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.333+0000 7f07e3a2b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07dc06b9c0 0x7f07dc1096c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:38:29.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.333+0000 7f07e3a2b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc109c70 con 0x7f07dc06b9c0 2024-02-15T18:38:29.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.334+0000 7f07e3a2b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc109db0 con 0x7f07dc06a3b0 2024-02-15T18:38:29.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.334+0000 7f07e0fc6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07dc06b9c0 0x7f07dc1096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:38:29.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.334+0000 7f07e0fc6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07dc06b9c0 0x7f07dc1096c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51468/0 (socket says 172.21.15.38:51468) 2024-02-15T18:38:29.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.334+0000 7f07e0fc6700 1 -- 172.21.15.38:0/1696430641 learned_addr learned my addr 172.21.15.38:0/1696430641 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:38:29.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.334+0000 7f07e17c7700 1 --2- 172.21.15.38:0/1696430641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07dc06a3b0 0x7f07dc1091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:38:29.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.334+0000 7f07e0fc6700 1 -- 172.21.15.38:0/1696430641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07dc06a3b0 msgr2=0x7f07dc1091d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:29.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.334+0000 7f07e0fc6700 1 --2- 172.21.15.38:0/1696430641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07dc06a3b0 0x7f07dc1091d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:29.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.334+0000 7f07e0fc6700 1 -- 172.21.15.38:0/1696430641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc005040 con 0x7f07dc06b9c0 2024-02-15T18:38:29.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.334+0000 7f07e17c7700 1 --2- 172.21.15.38:0/1696430641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07dc06a3b0 0x7f07dc1091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:38:29.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.334+0000 7f07e0fc6700 1 --2- 172.21.15.38:0/1696430641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07dc06b9c0 0x7f07dc1096c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f07cc009f50 tx=0x7f07cc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:38:29.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.334+0000 7f07d1ffb700 1 -- 172.21.15.38:0/1696430641 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07cc018070 con 0x7f07dc06b9c0 2024-02-15T18:38:29.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.334+0000 7f07d1ffb700 1 -- 172.21.15.38:0/1696430641 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f07cc007d70 con 0x7f07dc06b9c0 2024-02-15T18:38:29.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.334+0000 7f07d1ffb700 1 -- 172.21.15.38:0/1696430641 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07cc01c620 con 0x7f07dc06b9c0 2024-02-15T18:38:29.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.334+0000 7f07e3a2b700 1 -- 172.21.15.38:0/1696430641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07dc10a6f0 con 0x7f07dc06b9c0 2024-02-15T18:38:29.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.335+0000 7f07e3a2b700 1 -- 172.21.15.38:0/1696430641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07dc10ac50 con 0x7f07dc06b9c0 2024-02-15T18:38:29.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.336+0000 7f07e3a2b700 1 -- 172.21.15.38:0/1696430641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07dc045ad0 con 0x7f07dc06b9c0 2024-02-15T18:38:29.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.336+0000 7f07d1ffb700 1 -- 172.21.15.38:0/1696430641 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f07cc01c8e0 con 0x7f07dc06b9c0 2024-02-15T18:38:29.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.336+0000 7f07d1ffb700 1 --2- 172.21.15.38:0/1696430641 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f07c8060f20 0x7f07c80633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:38:29.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.336+0000 7f07d1ffb700 1 -- 172.21.15.38:0/1696430641 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f07cc07e080 con 0x7f07dc06b9c0 2024-02-15T18:38:29.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.336+0000 7f07e17c7700 1 --2- 172.21.15.38:0/1696430641 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f07c8060f20 0x7f07c80633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:38:29.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.337+0000 7f07e17c7700 1 --2- 172.21.15.38:0/1696430641 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f07c8060f20 0x7f07c80633e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f07d80041d0 tx=0x7f07d8009040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:38:29.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.341+0000 7f07d1ffb700 1 -- 172.21.15.38:0/1696430641 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f07cc051f30 con 0x7f07dc06b9c0 2024-02-15T18:38:29.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:28.605001+0000 mgr.smithi038.khesbl (mgr.14664) 127 : audit [DBG] from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:38:29.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:28.920783+0000 mgr.smithi038.khesbl (mgr.14664) 128 : audit [DBG] from='client.34184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:38:29.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:28.955961+0000 mgr.smithi038.khesbl (mgr.14664) 129 : cluster [DBG] pgmap v89: 65 pgs: 3 active+undersized+degraded, 5 stale+active+clean, 4 active+undersized, 53 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 13/342 objects degraded (3.801%) 2024-02-15T18:38:29.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:28.966001+0000 mon.smithi038 (mon.0) 189 : cluster [WRN] Health check failed: Degraded data redundancy: 13/342 objects degraded (3.801%), 3 pgs degraded (PG_DEGRADED) 2024-02-15T18:38:29.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.593+0000 7f07e3a2b700 1 -- 172.21.15.38:0/1696430641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f07dc027a00 con 0x7f07dc06b9c0 2024-02-15T18:38:29.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.593+0000 7f07d1ffb700 1 -- 172.21.15.38:0/1696430641 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+820 (secure 0 0 0) 0x7f07cc051b20 con 0x7f07dc06b9c0 2024-02-15T18:38:29.584 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:38:29.584 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-02-15T18:38:29.584 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:38:29.584 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:38:29.584 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-02-15T18:38:29.584 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:38:29.584 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:38:29.584 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-02-15T18:38:29.585 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2, 2024-02-15T18:38:29.585 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 3 2024-02-15T18:38:29.585 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:38:29.585 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-02-15T18:38:29.585 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:38:29.585 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:38:29.585 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-02-15T18:38:29.585 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 6, 2024-02-15T18:38:29.585 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 7 2024-02-15T18:38:29.585 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-02-15T18:38:29.585 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:38:29.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.595+0000 7f07e3a2b700 1 -- 172.21.15.38:0/1696430641 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f07c8060f20 msgr2=0x7f07c80633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:29.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.595+0000 7f07e3a2b700 1 --2- 172.21.15.38:0/1696430641 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f07c8060f20 0x7f07c80633e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f07d80041d0 tx=0x7f07d8009040).stop 2024-02-15T18:38:29.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.595+0000 7f07e3a2b700 1 -- 172.21.15.38:0/1696430641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07dc06b9c0 msgr2=0x7f07dc1096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:29.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.595+0000 7f07e3a2b700 1 --2- 172.21.15.38:0/1696430641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07dc06b9c0 0x7f07dc1096c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f07cc009f50 tx=0x7f07cc009b30).stop 2024-02-15T18:38:29.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.595+0000 7f07e3a2b700 1 -- 172.21.15.38:0/1696430641 shutdown_connections 2024-02-15T18:38:29.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.595+0000 7f07e3a2b700 1 --2- 172.21.15.38:0/1696430641 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f07c8060f20 0x7f07c80633e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:29.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.595+0000 7f07e3a2b700 1 --2- 172.21.15.38:0/1696430641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07dc06a3b0 0x7f07dc1091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:29.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.595+0000 7f07e3a2b700 1 --2- 172.21.15.38:0/1696430641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07dc06b9c0 0x7f07dc1096c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:29.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.595+0000 7f07e3a2b700 1 -- 172.21.15.38:0/1696430641 >> 172.21.15.38:0/1696430641 conn(0x7f07dc0f1e40 msgr2=0x7f07dc0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:38:29.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.596+0000 7f07e3a2b700 1 -- 172.21.15.38:0/1696430641 shutdown_connections 2024-02-15T18:38:29.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.596+0000 7f07e3a2b700 1 -- 172.21.15.38:0/1696430641 wait complete. 2024-02-15T18:38:29.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.703+0000 7facebe83700 1 -- 172.21.15.38:0/536308267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7face40f9880 msgr2=0x7face40fbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:29.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.703+0000 7facebe83700 1 --2- 172.21.15.38:0/536308267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7face40f9880 0x7face40fbc70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7face0004660 tx=0x7face0009b30).stop 2024-02-15T18:38:29.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.703+0000 7facebe83700 1 -- 172.21.15.38:0/536308267 shutdown_connections 2024-02-15T18:38:29.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.703+0000 7facebe83700 1 --2- 172.21.15.38:0/536308267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7face40f9880 0x7face40fbc70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:29.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.703+0000 7facebe83700 1 --2- 172.21.15.38:0/536308267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7face40602b0 0x7face40f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:29.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.703+0000 7facebe83700 1 -- 172.21.15.38:0/536308267 >> 172.21.15.38:0/536308267 conn(0x7face40f0d40 msgr2=0x7face40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:38:29.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.703+0000 7facebe83700 1 -- 172.21.15.38:0/536308267 shutdown_connections 2024-02-15T18:38:29.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.703+0000 7facebe83700 1 -- 172.21.15.38:0/536308267 wait complete. 2024-02-15T18:38:29.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.703+0000 7facebe83700 1 Processor -- start 2024-02-15T18:38:29.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.704+0000 7facebe83700 1 -- start start 2024-02-15T18:38:29.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.704+0000 7facebe83700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7face40602b0 0x7face40f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:38:29.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.704+0000 7facebe83700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7face40f9880 0x7face40f5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:38:29.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.704+0000 7facebe83700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face40f5760 con 0x7face40602b0 2024-02-15T18:38:29.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.704+0000 7facebe83700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face40f58a0 con 0x7face40f9880 2024-02-15T18:38:29.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.704+0000 7face9c1f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7face40602b0 0x7face40f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:38:29.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.704+0000 7face9c1f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7face40602b0 0x7face40f6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51490/0 (socket says 172.21.15.38:51490) 2024-02-15T18:38:29.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.704+0000 7face9c1f700 1 -- 172.21.15.38:0/2848297091 learned_addr learned my addr 172.21.15.38:0/2848297091 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:38:29.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.704+0000 7face9c1f700 1 -- 172.21.15.38:0/2848297091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7face40f9880 msgr2=0x7face40f5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:29.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.704+0000 7face9c1f700 1 --2- 172.21.15.38:0/2848297091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7face40f9880 0x7face40f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:38:29.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.704+0000 7face9c1f700 1 -- 172.21.15.38:0/2848297091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face0005040 con 0x7face40602b0 2024-02-15T18:38:29.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.704+0000 7face9c1f700 1 --2- 172.21.15.38:0/2848297091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7face40602b0 0x7face40f6d80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7facd400c530 tx=0x7facd4007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:38:29.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.704+0000 7facda7fc700 1 -- 172.21.15.38:0/2848297091 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facd400f650 con 0x7face40602b0 2024-02-15T18:38:29.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.704+0000 7facda7fc700 1 -- 172.21.15.38:0/2848297091 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7facd400fca0 con 0x7face40602b0 2024-02-15T18:38:29.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.704+0000 7facda7fc700 1 -- 172.21.15.38:0/2848297091 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facd4012e10 con 0x7face40602b0 2024-02-15T18:38:29.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.704+0000 7facebe83700 1 -- 172.21.15.38:0/2848297091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face40f5b80 con 0x7face40602b0 2024-02-15T18:38:29.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.705+0000 7facebe83700 1 -- 172.21.15.38:0/2848297091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face410efc0 con 0x7face40602b0 2024-02-15T18:38:29.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.705+0000 7facebe83700 1 -- 172.21.15.38:0/2848297091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face4045ad0 con 0x7face40602b0 2024-02-15T18:38:29.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.706+0000 7facda7fc700 1 -- 172.21.15.38:0/2848297091 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7facd4010070 con 0x7face40602b0 2024-02-15T18:38:29.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.706+0000 7facda7fc700 1 --2- 172.21.15.38:0/2848297091 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7facd0060f20 0x7facd00633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:38:29.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.706+0000 7facda7fc700 1 -- 172.21.15.38:0/2848297091 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5812+0+0 (secure 0 0 0) 0x7facd407bde0 con 0x7face40602b0 2024-02-15T18:38:29.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.706+0000 7face941e700 1 --2- 172.21.15.38:0/2848297091 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7facd0060f20 0x7facd00633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:38:29.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.707+0000 7face941e700 1 --2- 172.21.15.38:0/2848297091 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7facd0060f20 0x7facd00633e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7face0009f50 tx=0x7face0009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:38:29.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.711+0000 7facda7fc700 1 -- 172.21.15.38:0/2848297091 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7facd4047f60 con 0x7face40602b0 2024-02-15T18:38:29.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.935+0000 7facebe83700 1 -- 172.21.15.38:0/2848297091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7face4027a00 con 0x7face40602b0 2024-02-15T18:38:29.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.936+0000 7facda7fc700 1 -- 172.21.15.38:0/2848297091 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1807 (secure 0 0 0) 0x7facd404fd10 con 0x7face40602b0 2024-02-15T18:38:29.927 INFO:teuthology.orchestra.run.smithi038.stdout:e13 2024-02-15T18:38:29.927 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:38:29.927 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:38:29.927 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-02-15T18:38:29.927 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:38:29.927 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:38:29.927 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-02-15T18:38:29.927 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 13 2024-02-15T18:38:29.927 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 2024-02-15T18:38:29.927 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:38:29.928 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-02-15T18:35:07.485909+0000 2024-02-15T18:38:29.928 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-02-15T18:38:29.928 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-02-15T18:38:29.928 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-02-15T18:38:29.928 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-02-15T18:38:29.928 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-02-15T18:38:29.928 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-02-15T18:38:29.928 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-02-15T18:38:29.928 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-02-15T18:38:29.928 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:38:29.928 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-02-15T18:38:29.928 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-02-15T18:38:29.928 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=24261,1=24275} 2024-02-15T18:38:29.929 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-02-15T18:38:29.929 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-02-15T18:38:29.929 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-02-15T18:38:29.929 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-02-15T18:38:29.929 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-02-15T18:38:29.929 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-02-15T18:38:29.929 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-02-15T18:38:29.929 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-02-15T18:38:29.929 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.mvumlh{0:24261} state up:active seq 3 export targets 1 join_fscid=1 addr [v2:172.21.15.121:6824/1422707571,v1:172.21.15.121:6825/1422707571] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:38:29.929 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.ovuzki{1:24275} state up:active seq 2 export targets 0 join_fscid=1 addr [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:38:29.929 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:38:29.929 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:38:29.929 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-02-15T18:38:29.930 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:38:29.930 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.hxoggl{-1:14446} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.38:6826/3822880235,v1:172.21.15.38:6827/3822880235] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:38:29.930 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.qgkzgn{-1:14456} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.38:6828/703030231,v1:172.21.15.38:6829/703030231] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:38:29.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.938+0000 7facebe83700 1 -- 172.21.15.38:0/2848297091 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7facd0060f20 msgr2=0x7facd00633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:29.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.938+0000 7facebe83700 1 --2- 172.21.15.38:0/2848297091 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7facd0060f20 0x7facd00633e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7face0009f50 tx=0x7face0009b30).stop 2024-02-15T18:38:29.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.938+0000 7facebe83700 1 -- 172.21.15.38:0/2848297091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7face40602b0 msgr2=0x7face40f6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:29.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.938+0000 7facebe83700 1 --2- 172.21.15.38:0/2848297091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7face40602b0 0x7face40f6d80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7facd400c530 tx=0x7facd4007e20).stop 2024-02-15T18:38:29.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.938+0000 7facebe83700 1 -- 172.21.15.38:0/2848297091 shutdown_connections 2024-02-15T18:38:29.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.938+0000 7facebe83700 1 --2- 172.21.15.38:0/2848297091 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7facd0060f20 0x7facd00633e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:29.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.938+0000 7facebe83700 1 --2- 172.21.15.38:0/2848297091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7face40602b0 0x7face40f6d80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:29.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.938+0000 7facebe83700 1 --2- 172.21.15.38:0/2848297091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7face40f9880 0x7face40f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:38:29.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.938+0000 7facebe83700 1 -- 172.21.15.38:0/2848297091 >> 172.21.15.38:0/2848297091 conn(0x7face40f0d40 msgr2=0x7face40f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:38:29.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.938+0000 7facebe83700 1 -- 172.21.15.38:0/2848297091 shutdown_connections 2024-02-15T18:38:29.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:29.938+0000 7facebe83700 1 -- 172.21.15.38:0/2848297091 wait complete. 2024-02-15T18:38:29.931 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 13 2024-02-15T18:38:30.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.042+0000 7fadbd66e700 1 -- 172.21.15.38:0/1156878932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadb80f66e0 msgr2=0x7fadb80f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:30.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.042+0000 7fadbd66e700 1 --2- 172.21.15.38:0/1156878932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadb80f66e0 0x7fadb80f6b00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fadac004660 tx=0x7fadac009b30).stop 2024-02-15T18:38:30.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.043+0000 7fadbd66e700 1 -- 172.21.15.38:0/1156878932 shutdown_connections 2024-02-15T18:38:30.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.043+0000 7fadbd66e700 1 --2- 172.21.15.38:0/1156878932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb80f7980 0x7fadb80f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:30.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.043+0000 7fadbd66e700 1 --2- 172.21.15.38:0/1156878932 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadb80f66e0 0x7fadb80f6b00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:30.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.043+0000 7fadbd66e700 1 -- 172.21.15.38:0/1156878932 >> 172.21.15.38:0/1156878932 conn(0x7fadb80f1e40 msgr2=0x7fadb80f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:38:30.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.043+0000 7fadbd66e700 1 -- 172.21.15.38:0/1156878932 shutdown_connections 2024-02-15T18:38:30.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.043+0000 7fadbd66e700 1 -- 172.21.15.38:0/1156878932 wait complete. 2024-02-15T18:38:30.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.043+0000 7fadbd66e700 1 Processor -- start 2024-02-15T18:38:30.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.043+0000 7fadbd66e700 1 -- start start 2024-02-15T18:38:30.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.044+0000 7fadbd66e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb80f66e0 0x7fadb806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:38:30.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.044+0000 7fadbd66e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadb80f7980 0x7fadb8069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:38:30.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.044+0000 7fadbd66e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb806a380 con 0x7fadb80f7980 2024-02-15T18:38:30.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.044+0000 7fadbd66e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb806a4c0 con 0x7fadb80f66e0 2024-02-15T18:38:30.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.044+0000 7fadb67fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadb80f7980 0x7fadb8069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:38:30.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.044+0000 7fadb67fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadb80f7980 0x7fadb8069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51498/0 (socket says 172.21.15.38:51498) 2024-02-15T18:38:30.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.044+0000 7fadb67fc700 1 -- 172.21.15.38:0/2955966848 learned_addr learned my addr 172.21.15.38:0/2955966848 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:38:30.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.044+0000 7fadb6ffd700 1 --2- 172.21.15.38:0/2955966848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb80f66e0 0x7fadb806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:38:30.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.044+0000 7fadb67fc700 1 -- 172.21.15.38:0/2955966848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb80f66e0 msgr2=0x7fadb806b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:30.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.044+0000 7fadb67fc700 1 --2- 172.21.15.38:0/2955966848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb80f66e0 0x7fadb806b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:30.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.044+0000 7fadb67fc700 1 -- 172.21.15.38:0/2955966848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadac005040 con 0x7fadb80f7980 2024-02-15T18:38:30.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.044+0000 7fadb67fc700 1 --2- 172.21.15.38:0/2955966848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadb80f7980 0x7fadb8069e20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fada800e7f0 tx=0x7fada800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:38:30.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.044+0000 7fad9f7fe700 1 -- 172.21.15.38:0/2955966848 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fada80092e0 con 0x7fadb80f7980 2024-02-15T18:38:30.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.044+0000 7fad9f7fe700 1 -- 172.21.15.38:0/2955966848 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fada800f040 con 0x7fadb80f7980 2024-02-15T18:38:30.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.044+0000 7fadbd66e700 1 -- 172.21.15.38:0/2955966848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb806a7a0 con 0x7fadb80f7980 2024-02-15T18:38:30.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.044+0000 7fad9f7fe700 1 -- 172.21.15.38:0/2955966848 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fada8014850 con 0x7fadb80f7980 2024-02-15T18:38:30.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.044+0000 7fadbd66e700 1 -- 172.21.15.38:0/2955966848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb810f060 con 0x7fadb80f7980 2024-02-15T18:38:30.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.045+0000 7fadbd66e700 1 -- 172.21.15.38:0/2955966848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb8045ad0 con 0x7fadb80f7980 2024-02-15T18:38:30.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.046+0000 7fad9f7fe700 1 -- 172.21.15.38:0/2955966848 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fada801b030 con 0x7fadb80f7980 2024-02-15T18:38:30.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.046+0000 7fad9f7fe700 1 --2- 172.21.15.38:0/2955966848 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fada0060f20 0x7fada00633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:38:30.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.046+0000 7fad9f7fe700 1 -- 172.21.15.38:0/2955966848 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fada807be30 con 0x7fadb80f7980 2024-02-15T18:38:30.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.046+0000 7fadb6ffd700 1 --2- 172.21.15.38:0/2955966848 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fada0060f20 0x7fada00633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:38:30.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.047+0000 7fadb6ffd700 1 --2- 172.21.15.38:0/2955966848 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fada0060f20 0x7fada00633e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fadac004d80 tx=0x7fadac016040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:38:30.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.050+0000 7fad9f7fe700 1 -- 172.21.15.38:0/2955966848 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fada804c360 con 0x7fadb80f7980 2024-02-15T18:38:30.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.247+0000 7fadbd66e700 1 -- 172.21.15.38:0/2955966848 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fadb80ff120 con 0x7fada0060f20 2024-02-15T18:38:30.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.248+0000 7fad9f7fe700 1 -- 172.21.15.38:0/2955966848 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fadb80ff120 con 0x7fada0060f20 2024-02-15T18:38:30.237 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:38:30.237 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:38:30.237 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-02-15T18:38:30.237 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:38:30.238 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-02-15T18:38:30.238 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-02-15T18:38:30.238 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-02-15T18:38:30.238 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr" 2024-02-15T18:38:30.238 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-02-15T18:38:30.238 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "8/21 daemons upgraded", 2024-02-15T18:38:30.238 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading osd daemons", 2024-02-15T18:38:30.238 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-02-15T18:38:30.238 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:38:30.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.250+0000 7fadbd66e700 1 -- 172.21.15.38:0/2955966848 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fada0060f20 msgr2=0x7fada00633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:30.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.250+0000 7fadbd66e700 1 --2- 172.21.15.38:0/2955966848 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fada0060f20 0x7fada00633e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fadac004d80 tx=0x7fadac016040).stop 2024-02-15T18:38:30.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.250+0000 7fadbd66e700 1 -- 172.21.15.38:0/2955966848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadb80f7980 msgr2=0x7fadb8069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:38:30.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.250+0000 7fadbd66e700 1 --2- 172.21.15.38:0/2955966848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadb80f7980 0x7fadb8069e20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fada800e7f0 tx=0x7fada800b040).stop 2024-02-15T18:38:30.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.250+0000 7fadbd66e700 1 -- 172.21.15.38:0/2955966848 shutdown_connections 2024-02-15T18:38:30.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.250+0000 7fadbd66e700 1 --2- 172.21.15.38:0/2955966848 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fada0060f20 0x7fada00633e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:30.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.250+0000 7fadbd66e700 1 --2- 172.21.15.38:0/2955966848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fadb80f66e0 0x7fadb806b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:30.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.250+0000 7fadbd66e700 1 --2- 172.21.15.38:0/2955966848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fadb80f7980 0x7fadb8069e20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:38:30.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.250+0000 7fadbd66e700 1 -- 172.21.15.38:0/2955966848 >> 172.21.15.38:0/2955966848 conn(0x7fadb80f1e40 msgr2=0x7fadb80fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:38:30.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.250+0000 7fadbd66e700 1 -- 172.21.15.38:0/2955966848 shutdown_connections 2024-02-15T18:38:30.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:38:30.250+0000 7fadbd66e700 1 -- 172.21.15.38:0/2955966848 wait complete. 2024-02-15T18:38:30.272 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-2[86649]: debug 2024-02-15T18:38:30.087+0000 7f26459b0700 1 -- 172.21.15.38:0/4139960061 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55af37406680 con 0x55af373cd000 2024-02-15T18:38:30.272 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:30 smithi038 bash[86614]: 37171264b5bed077a4951ee563c33debb61b3b67c9cfe8a4d57f478e8b92bee5 2024-02-15T18:38:30.272 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:30 smithi038 systemd[1]: Started Ceph osd.2 for 7e05753e-cc2f-11ee-95ba-87774f69a715. 2024-02-15T18:38:30.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:29.222636+0000 mgr.smithi038.khesbl (mgr.14664) 130 : audit [DBG] from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:38:30.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:29.595034+0000 mon.smithi038 (mon.0) 190 : audit [DBG] from='client.? 172.21.15.38:0/1696430641' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:38:30.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:29.937629+0000 mon.smithi038 (mon.0) 191 : audit [DBG] from='client.? 172.21.15.38:0/2848297091' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:38:30.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:30 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:29.222636+0000 mgr.smithi038.khesbl (mgr.14664) 130 : audit [DBG] from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:38:30.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:30 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:29.595034+0000 mon.smithi038 (mon.0) 190 : audit [DBG] from='client.? 172.21.15.38:0/1696430641' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:38:30.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:30 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:29.937629+0000 mon.smithi038 (mon.0) 191 : audit [DBG] from='client.? 172.21.15.38:0/2848297091' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:38:30.910 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-2[86649]: debug 2024-02-15T18:38:30.603+0000 7f264d43e380 -1 Falling back to public interface 2024-02-15T18:38:31.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:30.203898+0000 mon.smithi038 (mon.0) 192 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:31.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:30.204682+0000 mon.smithi038 (mon.0) 193 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:38:31.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:30.248197+0000 mgr.smithi038.khesbl (mgr.14664) 131 : audit [DBG] from='client.34200 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:38:31.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:30.448243+0000 mon.smithi038 (mon.0) 194 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:31.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:30.956546+0000 mgr.smithi038.khesbl (mgr.14664) 132 : cluster [DBG] pgmap v90: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 43/342 objects degraded (12.573%) 2024-02-15T18:38:31.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:30.203898+0000 mon.smithi038 (mon.0) 192 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:31.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:30.204682+0000 mon.smithi038 (mon.0) 193 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:38:31.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:30.248197+0000 mgr.smithi038.khesbl (mgr.14664) 131 : audit [DBG] from='client.34200 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:38:31.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:30.448243+0000 mon.smithi038 (mon.0) 194 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:31.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:30.956546+0000 mgr.smithi038.khesbl (mgr.14664) 132 : cluster [DBG] pgmap v90: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 43/342 objects degraded (12.573%) 2024-02-15T18:38:33.499 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:32.956999+0000 mgr.smithi038.khesbl (mgr.14664) 133 : cluster [DBG] pgmap v91: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 43/342 objects degraded (12.573%) 2024-02-15T18:38:33.500 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:33.152118+0000 mon.smithi038 (mon.0) 195 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:33.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:32.956999+0000 mgr.smithi038.khesbl (mgr.14664) 133 : cluster [DBG] pgmap v91: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 43/342 objects degraded (12.573%) 2024-02-15T18:38:33.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:33.152118+0000 mon.smithi038 (mon.0) 195 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:35.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:34.957615+0000 mgr.smithi038.khesbl (mgr.14664) 134 : cluster [DBG] pgmap v92: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 43/342 objects degraded (12.573%) 2024-02-15T18:38:35.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:34.957615+0000 mgr.smithi038.khesbl (mgr.14664) 134 : cluster [DBG] pgmap v92: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 43/342 objects degraded (12.573%) 2024-02-15T18:38:37.772 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-2[86649]: debug 2024-02-15T18:38:37.455+0000 7f264d43e380 -1 osd.2 56 log_to_monitors {default=true} 2024-02-15T18:38:37.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:36.383400+0000 mon.smithi038 (mon.0) 196 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:37.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:36.707005+0000 mon.smithi038 (mon.0) 197 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:38:37.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:36.707774+0000 mon.smithi038 (mon.0) 198 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:38:37.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:38:36.708332+0000 mgr.smithi038.khesbl (mgr.14664) 135 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:38:37.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:36.712885+0000 mon.smithi038 (mon.0) 199 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:37.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:36.717512+0000 mon.smithi038 (mon.0) 200 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:38:37.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:36.719580+0000 mon.smithi038 (mon.0) 201 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:38:37.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:36.720680+0000 mon.smithi038 (mon.0) 202 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:38:37.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:36.721731+0000 mon.smithi038 (mon.0) 203 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:38:37.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:36.722797+0000 mon.smithi038 (mon.0) 204 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-15T18:38:37.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:36.722929+0000 mgr.smithi038.khesbl (mgr.14664) 136 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-15T18:38:37.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:38:36.723558+0000 mgr.smithi038.khesbl (mgr.14664) 137 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-15T18:38:37.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:36.957992+0000 mgr.smithi038.khesbl (mgr.14664) 138 : cluster [DBG] pgmap v93: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 43/342 objects degraded (12.573%) 2024-02-15T18:38:37.774 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:37.236791+0000 mon.smithi038 (mon.0) 205 : cluster [WRN] Health check update: Degraded data redundancy: 43/342 objects degraded (12.573%), 12 pgs degraded (PG_DEGRADED) 2024-02-15T18:38:37.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:36.383400+0000 mon.smithi038 (mon.0) 196 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:37.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:36.707005+0000 mon.smithi038 (mon.0) 197 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:38:37.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:36.707774+0000 mon.smithi038 (mon.0) 198 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:38:37.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:38:36.708332+0000 mgr.smithi038.khesbl (mgr.14664) 135 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:38:37.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:36.712885+0000 mon.smithi038 (mon.0) 199 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:37.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:36.717512+0000 mon.smithi038 (mon.0) 200 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:38:37.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:36.719580+0000 mon.smithi038 (mon.0) 201 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:38:37.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:36.720680+0000 mon.smithi038 (mon.0) 202 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:38:37.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:36.721731+0000 mon.smithi038 (mon.0) 203 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:38:37.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:36.722797+0000 mon.smithi038 (mon.0) 204 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-15T18:38:37.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:36.722929+0000 mgr.smithi038.khesbl (mgr.14664) 136 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-15T18:38:37.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:38:36.723558+0000 mgr.smithi038.khesbl (mgr.14664) 137 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-15T18:38:37.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:36.957992+0000 mgr.smithi038.khesbl (mgr.14664) 138 : cluster [DBG] pgmap v93: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 293 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 43/342 objects degraded (12.573%) 2024-02-15T18:38:37.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:37.236791+0000 mon.smithi038 (mon.0) 205 : cluster [WRN] Health check update: Degraded data redundancy: 43/342 objects degraded (12.573%), 12 pgs degraded (PG_DEGRADED) 2024-02-15T18:38:38.647 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:38 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:37.459122+0000 mon.smithi038 (mon.0) 206 : audit [INF] from='osd.2 [v2:172.21.15.38:6818/209130240,v1:172.21.15.38:6819/209130240]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-15T18:38:38.648 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:38:38 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-2[86649]: debug 2024-02-15T18:38:38.399+0000 7f263c120700 -1 osd.2 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-15T18:38:38.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:38 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:37.459122+0000 mon.smithi038 (mon.0) 206 : audit [INF] from='osd.2 [v2:172.21.15.38:6818/209130240,v1:172.21.15.38:6819/209130240]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-15T18:38:39.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:38.392172+0000 mon.smithi038 (mon.0) 207 : audit [INF] from='osd.2 [v2:172.21.15.38:6818/209130240,v1:172.21.15.38:6819/209130240]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-15T18:38:39.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:38.392244+0000 mon.smithi038 (mon.0) 208 : cluster [DBG] osdmap e59: 6 total, 5 up, 6 in 2024-02-15T18:38:39.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:38.393246+0000 mon.smithi038 (mon.0) 209 : audit [INF] from='osd.2 [v2:172.21.15.38:6818/209130240,v1:172.21.15.38:6819/209130240]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-02-15T18:38:39.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:38.958612+0000 mgr.smithi038.khesbl (mgr.14664) 139 : cluster [DBG] pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 43/342 objects degraded (12.573%) 2024-02-15T18:38:39.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:38.392172+0000 mon.smithi038 (mon.0) 207 : audit [INF] from='osd.2 [v2:172.21.15.38:6818/209130240,v1:172.21.15.38:6819/209130240]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-15T18:38:39.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:38.392244+0000 mon.smithi038 (mon.0) 208 : cluster [DBG] osdmap e59: 6 total, 5 up, 6 in 2024-02-15T18:38:39.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:38.393246+0000 mon.smithi038 (mon.0) 209 : audit [INF] from='osd.2 [v2:172.21.15.38:6818/209130240,v1:172.21.15.38:6819/209130240]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-02-15T18:38:39.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:38.958612+0000 mgr.smithi038.khesbl (mgr.14664) 139 : cluster [DBG] pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 43/342 objects degraded (12.573%) 2024-02-15T18:38:40.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:39.390148+0000 mon.smithi038 (mon.0) 210 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:38:40.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:39.396460+0000 mon.smithi038 (mon.0) 211 : cluster [INF] osd.2 [v2:172.21.15.38:6818/209130240,v1:172.21.15.38:6819/209130240] boot 2024-02-15T18:38:40.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:39.396556+0000 mon.smithi038 (mon.0) 212 : cluster [DBG] osdmap e60: 6 total, 6 up, 6 in 2024-02-15T18:38:40.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:39.398673+0000 mon.smithi038 (mon.0) 213 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:38:40.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:40.396247+0000 mon.smithi038 (mon.0) 214 : cluster [DBG] osdmap e61: 6 total, 6 up, 6 in 2024-02-15T18:38:40.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:39.390148+0000 mon.smithi038 (mon.0) 210 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:38:40.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:39.396460+0000 mon.smithi038 (mon.0) 211 : cluster [INF] osd.2 [v2:172.21.15.38:6818/209130240,v1:172.21.15.38:6819/209130240] boot 2024-02-15T18:38:40.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:39.396556+0000 mon.smithi038 (mon.0) 212 : cluster [DBG] osdmap e60: 6 total, 6 up, 6 in 2024-02-15T18:38:40.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:39.398673+0000 mon.smithi038 (mon.0) 213 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:38:40.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:40.396247+0000 mon.smithi038 (mon.0) 214 : cluster [DBG] osdmap e61: 6 total, 6 up, 6 in 2024-02-15T18:38:41.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:40.959092+0000 mgr.smithi038.khesbl (mgr.14664) 140 : cluster [DBG] pgmap v98: 65 pgs: 5 peering, 10 active+undersized+degraded, 12 active+undersized, 38 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/342 objects degraded (9.942%) 2024-02-15T18:38:41.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:40.959092+0000 mgr.smithi038.khesbl (mgr.14664) 140 : cluster [DBG] pgmap v98: 65 pgs: 5 peering, 10 active+undersized+degraded, 12 active+undersized, 38 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/342 objects degraded (9.942%) 2024-02-15T18:38:42.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:42.237647+0000 mon.smithi038 (mon.0) 215 : cluster [WRN] Health check update: Degraded data redundancy: 34/342 objects degraded (9.942%), 10 pgs degraded (PG_DEGRADED) 2024-02-15T18:38:42.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:42.237647+0000 mon.smithi038 (mon.0) 215 : cluster [WRN] Health check update: Degraded data redundancy: 34/342 objects degraded (9.942%), 10 pgs degraded (PG_DEGRADED) 2024-02-15T18:38:43.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:42.959567+0000 mgr.smithi038.khesbl (mgr.14664) 141 : cluster [DBG] pgmap v99: 65 pgs: 5 peering, 10 active+undersized+degraded, 12 active+undersized, 38 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/342 objects degraded (9.942%) 2024-02-15T18:38:43.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:42.959567+0000 mgr.smithi038.khesbl (mgr.14664) 141 : cluster [DBG] pgmap v99: 65 pgs: 5 peering, 10 active+undersized+degraded, 12 active+undersized, 38 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/342 objects degraded (9.942%) 2024-02-15T18:38:45.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:44.960025+0000 mgr.smithi038.khesbl (mgr.14664) 142 : cluster [DBG] pgmap v100: 65 pgs: 65 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:38:45.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:45.010868+0000 mon.smithi038 (mon.0) 216 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/342 objects degraded (9.942%), 10 pgs degraded) 2024-02-15T18:38:45.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:45.010905+0000 mon.smithi038 (mon.0) 217 : cluster [INF] Cluster is now healthy 2024-02-15T18:38:45.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:44.960025+0000 mgr.smithi038.khesbl (mgr.14664) 142 : cluster [DBG] pgmap v100: 65 pgs: 65 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:38:45.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:45.010868+0000 mon.smithi038 (mon.0) 216 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/342 objects degraded (9.942%), 10 pgs degraded) 2024-02-15T18:38:45.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:45.010905+0000 mon.smithi038 (mon.0) 217 : cluster [INF] Cluster is now healthy 2024-02-15T18:38:47.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:46.960562+0000 mgr.smithi038.khesbl (mgr.14664) 143 : cluster [DBG] pgmap v101: 65 pgs: 65 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:38:47.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:46.960562+0000 mgr.smithi038.khesbl (mgr.14664) 143 : cluster [DBG] pgmap v101: 65 pgs: 65 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:38:49.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:48.153259+0000 mon.smithi038 (mon.0) 218 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:49.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:48.961028+0000 mgr.smithi038.khesbl (mgr.14664) 144 : cluster [DBG] pgmap v102: 65 pgs: 65 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:38:49.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:48.153259+0000 mon.smithi038 (mon.0) 218 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:49.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:48.961028+0000 mgr.smithi038.khesbl (mgr.14664) 144 : cluster [DBG] pgmap v102: 65 pgs: 65 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:38:51.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:50.961644+0000 mgr.smithi038.khesbl (mgr.14664) 145 : cluster [DBG] pgmap v103: 65 pgs: 65 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:38:51.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:50.961644+0000 mgr.smithi038.khesbl (mgr.14664) 145 : cluster [DBG] pgmap v103: 65 pgs: 65 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:38:52.305 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:52 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:51.736549+0000 mon.smithi038 (mon.0) 219 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-15T18:38:52.306 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:52 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:51.736748+0000 mgr.smithi038.khesbl (mgr.14664) 146 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-15T18:38:52.306 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:52 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:38:51.737813+0000 mgr.smithi038.khesbl (mgr.14664) 147 : cephadm [INF] Upgrade: osd.3 is safe to restart 2024-02-15T18:38:52.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:51.736549+0000 mon.smithi038 (mon.0) 219 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-15T18:38:52.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:51.736748+0000 mgr.smithi038.khesbl (mgr.14664) 146 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-15T18:38:52.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:38:51.737813+0000 mgr.smithi038.khesbl (mgr.14664) 147 : cephadm [INF] Upgrade: osd.3 is safe to restart 2024-02-15T18:38:53.500 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:52.961998+0000 mgr.smithi038.khesbl (mgr.14664) 148 : cluster [DBG] pgmap v104: 65 pgs: 65 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:38:53.500 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:53.200315+0000 mon.smithi038 (mon.0) 220 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:53.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:52.961998+0000 mgr.smithi038.khesbl (mgr.14664) 148 : cluster [DBG] pgmap v104: 65 pgs: 65 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:38:53.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:53.200315+0000 mon.smithi038 (mon.0) 220 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:54.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:38:53.342010+0000 mgr.smithi038.khesbl (mgr.14664) 149 : cephadm [INF] Upgrade: Updating osd.3 2024-02-15T18:38:54.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:53.350628+0000 mon.smithi038 (mon.0) 221 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:54.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:53.351578+0000 mon.smithi038 (mon.0) 222 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-15T18:38:54.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:38:53.352372+0000 mon.smithi038 (mon.0) 223 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:38:54.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:38:53.353562+0000 mgr.smithi038.khesbl (mgr.14664) 150 : cephadm [INF] Deploying daemon osd.3 on smithi121 2024-02-15T18:38:54.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:38:53.342010+0000 mgr.smithi038.khesbl (mgr.14664) 149 : cephadm [INF] Upgrade: Updating osd.3 2024-02-15T18:38:54.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:53.350628+0000 mon.smithi038 (mon.0) 221 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:38:54.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:53.351578+0000 mon.smithi038 (mon.0) 222 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-15T18:38:54.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:38:53.352372+0000 mon.smithi038 (mon.0) 223 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:38:54.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:38:53.353562+0000 mgr.smithi038.khesbl (mgr.14664) 150 : cephadm [INF] Deploying daemon osd.3 on smithi121 2024-02-15T18:38:55.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:54.962793+0000 mgr.smithi038.khesbl (mgr.14664) 151 : cluster [DBG] pgmap v105: 65 pgs: 65 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:38:55.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:54.962793+0000 mgr.smithi038.khesbl (mgr.14664) 151 : cluster [DBG] pgmap v105: 65 pgs: 65 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:38:56.606 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:38:56 smithi121 systemd[1]: Stopping Ceph osd.3 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:38:56.955 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:38:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.3[45185]: debug 2024-02-15T18:38:56.661+0000 7fc222bc0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:38:56.955 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:38:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.3[45185]: debug 2024-02-15T18:38:56.661+0000 7fc222bc0700 -1 osd.3 61 *** Got signal Terminated *** 2024-02-15T18:38:56.955 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:38:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.3[45185]: debug 2024-02-15T18:38:56.661+0000 7fc222bc0700 -1 osd.3 61 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:38:56.955 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:56.903177+0000 mon.smithi038 (mon.0) 224 : cluster [DBG] osd.3 reported immediately failed by osd.5 2024-02-15T18:38:56.955 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:56.903217+0000 mon.smithi038 (mon.0) 225 : cluster [INF] osd.3 failed (root=default,host=smithi121) (connection refused reported by osd.5) 2024-02-15T18:38:56.955 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:56.914330+0000 mon.smithi038 (mon.0) 226 : cluster [DBG] osd.3 reported immediately failed by osd.0 2024-02-15T18:38:56.956 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:56.922612+0000 mon.smithi038 (mon.0) 227 : cluster [DBG] osd.3 reported immediately failed by osd.5 2024-02-15T18:38:56.956 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:56.922746+0000 mon.smithi038 (mon.0) 228 : cluster [DBG] osd.3 reported immediately failed by osd.5 2024-02-15T18:38:56.956 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:56.922936+0000 mon.smithi038 (mon.0) 229 : cluster [DBG] osd.3 reported immediately failed by osd.4 2024-02-15T18:38:56.956 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:56.923046+0000 mon.smithi038 (mon.0) 230 : cluster [DBG] osd.3 reported immediately failed by osd.0 2024-02-15T18:38:56.956 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:56.923130+0000 mon.smithi038 (mon.0) 231 : cluster [DBG] osd.3 reported immediately failed by osd.1 2024-02-15T18:38:56.956 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:56.923218+0000 mon.smithi038 (mon.0) 232 : cluster [DBG] osd.3 reported immediately failed by osd.2 2024-02-15T18:38:56.956 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:56.923315+0000 mon.smithi038 (mon.0) 233 : cluster [DBG] osd.3 reported immediately failed by osd.4 2024-02-15T18:38:56.956 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:56.923414+0000 mon.smithi038 (mon.0) 234 : cluster [DBG] osd.3 reported immediately failed by osd.0 2024-02-15T18:38:56.956 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:56.923533+0000 mon.smithi038 (mon.0) 235 : cluster [DBG] osd.3 reported immediately failed by osd.1 2024-02-15T18:38:56.956 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:56.923620+0000 mon.smithi038 (mon.0) 236 : cluster [DBG] osd.3 reported immediately failed by osd.2 2024-02-15T18:38:56.956 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:56.934095+0000 mon.smithi038 (mon.0) 237 : cluster [DBG] osd.3 reported immediately failed by osd.4 2024-02-15T18:38:57.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:56.903177+0000 mon.smithi038 (mon.0) 224 : cluster [DBG] osd.3 reported immediately failed by osd.5 2024-02-15T18:38:57.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:56.903217+0000 mon.smithi038 (mon.0) 225 : cluster [INF] osd.3 failed (root=default,host=smithi121) (connection refused reported by osd.5) 2024-02-15T18:38:57.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:56.914330+0000 mon.smithi038 (mon.0) 226 : cluster [DBG] osd.3 reported immediately failed by osd.0 2024-02-15T18:38:57.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:56.922612+0000 mon.smithi038 (mon.0) 227 : cluster [DBG] osd.3 reported immediately failed by osd.5 2024-02-15T18:38:57.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:56.922746+0000 mon.smithi038 (mon.0) 228 : cluster [DBG] osd.3 reported immediately failed by osd.5 2024-02-15T18:38:57.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:56.922936+0000 mon.smithi038 (mon.0) 229 : cluster [DBG] osd.3 reported immediately failed by osd.4 2024-02-15T18:38:57.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:56.923046+0000 mon.smithi038 (mon.0) 230 : cluster [DBG] osd.3 reported immediately failed by osd.0 2024-02-15T18:38:57.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:56.923130+0000 mon.smithi038 (mon.0) 231 : cluster [DBG] osd.3 reported immediately failed by osd.1 2024-02-15T18:38:57.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:56.923218+0000 mon.smithi038 (mon.0) 232 : cluster [DBG] osd.3 reported immediately failed by osd.2 2024-02-15T18:38:57.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:56.923315+0000 mon.smithi038 (mon.0) 233 : cluster [DBG] osd.3 reported immediately failed by osd.4 2024-02-15T18:38:57.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:56.923414+0000 mon.smithi038 (mon.0) 234 : cluster [DBG] osd.3 reported immediately failed by osd.0 2024-02-15T18:38:57.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:56.923533+0000 mon.smithi038 (mon.0) 235 : cluster [DBG] osd.3 reported immediately failed by osd.1 2024-02-15T18:38:57.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:56.923620+0000 mon.smithi038 (mon.0) 236 : cluster [DBG] osd.3 reported immediately failed by osd.2 2024-02-15T18:38:57.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:56.934095+0000 mon.smithi038 (mon.0) 237 : cluster [DBG] osd.3 reported immediately failed by osd.4 2024-02-15T18:38:57.356 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:38:57 smithi121 bash[61661]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.3 2024-02-15T18:38:58.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:56.953873+0000 mon.smithi038 (mon.0) 238 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:38:58.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:56.961913+0000 mon.smithi038 (mon.0) 239 : cluster [DBG] osdmap e62: 6 total, 5 up, 6 in 2024-02-15T18:38:58.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:56.963124+0000 mgr.smithi038.khesbl (mgr.14664) 152 : cluster [DBG] pgmap v107: 65 pgs: 16 stale+active+clean, 49 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:38:58.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:56.953873+0000 mon.smithi038 (mon.0) 238 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:38:58.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:56.961913+0000 mon.smithi038 (mon.0) 239 : cluster [DBG] osdmap e62: 6 total, 5 up, 6 in 2024-02-15T18:38:58.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:56.963124+0000 mgr.smithi038.khesbl (mgr.14664) 152 : cluster [DBG] pgmap v107: 65 pgs: 16 stale+active+clean, 49 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:38:59.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:57.973349+0000 mon.smithi038 (mon.0) 240 : cluster [DBG] osdmap e63: 6 total, 5 up, 6 in 2024-02-15T18:38:59.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:57.973349+0000 mon.smithi038 (mon.0) 240 : cluster [DBG] osdmap e63: 6 total, 5 up, 6 in 2024-02-15T18:38:59.589 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:38:59 smithi121 systemd[1]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.3.service: Succeeded. 2024-02-15T18:38:59.589 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:38:59 smithi121 systemd[1]: Stopped Ceph osd.3 for 7e05753e-cc2f-11ee-95ba-87774f69a715. 2024-02-15T18:38:59.589 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:38:59 smithi121 systemd[1]: Starting Ceph osd.3 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:39:00.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:58.963680+0000 mgr.smithi038.khesbl (mgr.14664) 153 : cluster [DBG] pgmap v109: 65 pgs: 7 active+undersized+degraded, 13 stale+active+clean, 6 active+undersized, 39 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 23/342 objects degraded (6.725%) 2024-02-15T18:39:00.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:38:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:38:58.970577+0000 mon.smithi038 (mon.0) 241 : cluster [WRN] Health check failed: Degraded data redundancy: 23/342 objects degraded (6.725%), 7 pgs degraded (PG_DEGRADED) 2024-02-15T18:39:00.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:58.963680+0000 mgr.smithi038.khesbl (mgr.14664) 153 : cluster [DBG] pgmap v109: 65 pgs: 7 active+undersized+degraded, 13 stale+active+clean, 6 active+undersized, 39 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 23/342 objects degraded (6.725%) 2024-02-15T18:39:00.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:38:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:38:58.970577+0000 mon.smithi038 (mon.0) 241 : cluster [WRN] Health check failed: Degraded data redundancy: 23/342 objects degraded (6.725%), 7 pgs degraded (PG_DEGRADED) 2024-02-15T18:39:00.356 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:39:00 smithi121 bash[61964]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-02-15T18:39:00.356 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:39:00 smithi121 bash[61964]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-15T18:39:00.356 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:39:00 smithi121 bash[61964]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-3 --no-mon-config 2024-02-15T18:39:00.357 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:39:00 smithi121 bash[61964]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3/block 2024-02-15T18:39:00.357 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:39:00 smithi121 bash[61964]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-3/block 2024-02-15T18:39:00.357 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:39:00 smithi121 bash[61964]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-15T18:39:00.357 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:39:00 smithi121 bash[61964]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-15T18:39:00.357 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:39:00 smithi121 bash[61964]: --> ceph-volume lvm activate successful for osd ID: 3 2024-02-15T18:39:00.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.367+0000 7fc1c2e4f700 1 -- 172.21.15.38:0/4114708193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1bc0f66e0 msgr2=0x7fc1bc0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:00.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.367+0000 7fc1c2e4f700 1 --2- 172.21.15.38:0/4114708193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1bc0f66e0 0x7fc1bc0f6b00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fc1b0004660 tx=0x7fc1b0009b30).stop 2024-02-15T18:39:00.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.367+0000 7fc1c2e4f700 1 -- 172.21.15.38:0/4114708193 shutdown_connections 2024-02-15T18:39:00.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.367+0000 7fc1c2e4f700 1 --2- 172.21.15.38:0/4114708193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1bc0f7980 0x7fc1bc0f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:39:00.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.367+0000 7fc1c2e4f700 1 --2- 172.21.15.38:0/4114708193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1bc0f66e0 0x7fc1bc0f6b00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:00.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.367+0000 7fc1c2e4f700 1 -- 172.21.15.38:0/4114708193 >> 172.21.15.38:0/4114708193 conn(0x7fc1bc0f1e40 msgr2=0x7fc1bc0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:00.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.367+0000 7fc1c2e4f700 1 -- 172.21.15.38:0/4114708193 shutdown_connections 2024-02-15T18:39:00.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.367+0000 7fc1c2e4f700 1 -- 172.21.15.38:0/4114708193 wait complete. 2024-02-15T18:39:00.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.367+0000 7fc1c2e4f700 1 Processor -- start 2024-02-15T18:39:00.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.367+0000 7fc1c2e4f700 1 -- start start 2024-02-15T18:39:00.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.368+0000 7fc1c2e4f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1bc0f66e0 0x7fc1bc1070b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:00.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.368+0000 7fc1c2e4f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1bc0f7980 0x7fc1bc1075a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:00.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.368+0000 7fc1c2e4f700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc107be0 con 0x7fc1bc0f66e0 2024-02-15T18:39:00.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.368+0000 7fc1c2e4f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc108490 con 0x7fc1bc0f7980 2024-02-15T18:39:00.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.368+0000 7fc1c0beb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1bc0f66e0 0x7fc1bc1070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:00.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.368+0000 7fc1c0beb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1bc0f66e0 0x7fc1bc1070b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46322/0 (socket says 172.21.15.38:46322) 2024-02-15T18:39:00.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.368+0000 7fc1c0beb700 1 -- 172.21.15.38:0/3758784122 learned_addr learned my addr 172.21.15.38:0/3758784122 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:39:00.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.368+0000 7fc1bbfff700 1 --2- 172.21.15.38:0/3758784122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1bc0f7980 0x7fc1bc1075a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:00.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.368+0000 7fc1c0beb700 1 -- 172.21.15.38:0/3758784122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1bc0f7980 msgr2=0x7fc1bc1075a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:00.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.368+0000 7fc1c0beb700 1 --2- 172.21.15.38:0/3758784122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1bc0f7980 0x7fc1bc1075a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:00.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.368+0000 7fc1c0beb700 1 -- 172.21.15.38:0/3758784122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b0005040 con 0x7fc1bc0f66e0 2024-02-15T18:39:00.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.368+0000 7fc1c0beb700 1 --2- 172.21.15.38:0/3758784122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1bc0f66e0 0x7fc1bc1070b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fc1b0013fd0 tx=0x7fc1b0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:00.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.369+0000 7fc1b97fa700 1 -- 172.21.15.38:0/3758784122 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1b0018070 con 0x7fc1bc0f66e0 2024-02-15T18:39:00.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.369+0000 7fc1c2e4f700 1 -- 172.21.15.38:0/3758784122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1bc108710 con 0x7fc1bc0f66e0 2024-02-15T18:39:00.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.369+0000 7fc1b97fa700 1 -- 172.21.15.38:0/3758784122 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fc1b0007c70 con 0x7fc1bc0f66e0 2024-02-15T18:39:00.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.369+0000 7fc1b97fa700 1 -- 172.21.15.38:0/3758784122 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1b001c620 con 0x7fc1bc0f66e0 2024-02-15T18:39:00.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.369+0000 7fc1c2e4f700 1 -- 172.21.15.38:0/3758784122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1bc108bf0 con 0x7fc1bc0f66e0 2024-02-15T18:39:00.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.370+0000 7fc1c2e4f700 1 -- 172.21.15.38:0/3758784122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1bc0f5ad0 con 0x7fc1bc0f66e0 2024-02-15T18:39:00.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.370+0000 7fc1b97fa700 1 -- 172.21.15.38:0/3758784122 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fc1b001d420 con 0x7fc1bc0f66e0 2024-02-15T18:39:00.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.370+0000 7fc1b97fa700 1 --2- 172.21.15.38:0/3758784122 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc1ac060f20 0x7fc1ac0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:00.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.371+0000 7fc1b97fa700 1 -- 172.21.15.38:0/3758784122 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fc1b007dfa0 con 0x7fc1bc0f66e0 2024-02-15T18:39:00.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.371+0000 7fc1bbfff700 1 --2- 172.21.15.38:0/3758784122 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc1ac060f20 0x7fc1ac0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:00.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.371+0000 7fc1bbfff700 1 --2- 172.21.15.38:0/3758784122 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc1ac060f20 0x7fc1ac0633e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fc1a8006fd0 tx=0x7fc1a8008040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:00.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.375+0000 7fc1b97fa700 1 -- 172.21.15.38:0/3758784122 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc1b0051e50 con 0x7fc1bc0f66e0 2024-02-15T18:39:00.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.565+0000 7fc1c2e4f700 1 -- 172.21.15.38:0/3758784122 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1bc0ff030 con 0x7fc1ac060f20 2024-02-15T18:39:00.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.566+0000 7fc1b97fa700 1 -- 172.21.15.38:0/3758784122 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc1bc0ff030 con 0x7fc1ac060f20 2024-02-15T18:39:00.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.567+0000 7fc1c2e4f700 1 -- 172.21.15.38:0/3758784122 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc1ac060f20 msgr2=0x7fc1ac0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:00.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.567+0000 7fc1c2e4f700 1 --2- 172.21.15.38:0/3758784122 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc1ac060f20 0x7fc1ac0633e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fc1a8006fd0 tx=0x7fc1a8008040).stop 2024-02-15T18:39:00.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.568+0000 7fc1c2e4f700 1 -- 172.21.15.38:0/3758784122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1bc0f66e0 msgr2=0x7fc1bc1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:00.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.568+0000 7fc1c2e4f700 1 --2- 172.21.15.38:0/3758784122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1bc0f66e0 0x7fc1bc1070b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fc1b0013fd0 tx=0x7fc1b0009b30).stop 2024-02-15T18:39:00.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.568+0000 7fc1c2e4f700 1 -- 172.21.15.38:0/3758784122 shutdown_connections 2024-02-15T18:39:00.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.568+0000 7fc1c2e4f700 1 --2- 172.21.15.38:0/3758784122 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc1ac060f20 0x7fc1ac0633e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:00.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.568+0000 7fc1c2e4f700 1 --2- 172.21.15.38:0/3758784122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1bc0f66e0 0x7fc1bc1070b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:00.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.568+0000 7fc1c2e4f700 1 --2- 172.21.15.38:0/3758784122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1bc0f7980 0x7fc1bc1075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:00.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.568+0000 7fc1c2e4f700 1 -- 172.21.15.38:0/3758784122 >> 172.21.15.38:0/3758784122 conn(0x7fc1bc0f1e40 msgr2=0x7fc1bc0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:00.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.568+0000 7fc1c2e4f700 1 -- 172.21.15.38:0/3758784122 shutdown_connections 2024-02-15T18:39:00.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.568+0000 7fc1c2e4f700 1 -- 172.21.15.38:0/3758784122 wait complete. 2024-02-15T18:39:00.568 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-02-15T18:39:00.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.670+0000 7f62fd05b700 1 -- 172.21.15.38:0/1389513726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f80f96f0 msgr2=0x7f62f80fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:00.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.670+0000 7f62fd05b700 1 --2- 172.21.15.38:0/1389513726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f80f96f0 0x7f62f80fbae0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f62ec004660 tx=0x7f62ec009b30).stop 2024-02-15T18:39:00.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.670+0000 7f62fd05b700 1 -- 172.21.15.38:0/1389513726 shutdown_connections 2024-02-15T18:39:00.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.670+0000 7f62fd05b700 1 --2- 172.21.15.38:0/1389513726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f80f96f0 0x7f62f80fbae0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:00.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.670+0000 7f62fd05b700 1 --2- 172.21.15.38:0/1389513726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62f80602b0 0x7f62f80f9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:00.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.670+0000 7f62fd05b700 1 -- 172.21.15.38:0/1389513726 >> 172.21.15.38:0/1389513726 conn(0x7f62f80f0b90 msgr2=0x7f62f80f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:00.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.670+0000 7f62fd05b700 1 -- 172.21.15.38:0/1389513726 shutdown_connections 2024-02-15T18:39:00.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.670+0000 7f62fd05b700 1 -- 172.21.15.38:0/1389513726 wait complete. 2024-02-15T18:39:00.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.671+0000 7f62fd05b700 1 Processor -- start 2024-02-15T18:39:00.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.671+0000 7f62fd05b700 1 -- start start 2024-02-15T18:39:00.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.671+0000 7f62fd05b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62f80602b0 0x7f62f80f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:00.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.671+0000 7f62fd05b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f80f96f0 0x7f62f80f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:00.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.671+0000 7f62fd05b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f80f55d0 con 0x7f62f80f96f0 2024-02-15T18:39:00.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.671+0000 7f62fd05b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f80f5710 con 0x7f62f80602b0 2024-02-15T18:39:00.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.672+0000 7f62f77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f80f96f0 0x7f62f80f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:00.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.672+0000 7f62f77fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f80f96f0 0x7f62f80f5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46334/0 (socket says 172.21.15.38:46334) 2024-02-15T18:39:00.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.672+0000 7f62f77fe700 1 -- 172.21.15.38:0/107753450 learned_addr learned my addr 172.21.15.38:0/107753450 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:39:00.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.672+0000 7f62f7fff700 1 --2- 172.21.15.38:0/107753450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62f80602b0 0x7f62f80f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:00.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.672+0000 7f62f77fe700 1 -- 172.21.15.38:0/107753450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62f80602b0 msgr2=0x7f62f80f6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:00.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.672+0000 7f62f77fe700 1 --2- 172.21.15.38:0/107753450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62f80602b0 0x7f62f80f6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:00.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.672+0000 7f62f77fe700 1 -- 172.21.15.38:0/107753450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec005040 con 0x7f62f80f96f0 2024-02-15T18:39:00.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.672+0000 7f62f77fe700 1 --2- 172.21.15.38:0/107753450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f80f96f0 0x7f62f80f5070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f62ec004e30 tx=0x7f62ec009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:00.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.672+0000 7f62f4ff9700 1 -- 172.21.15.38:0/107753450 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62ec018070 con 0x7f62f80f96f0 2024-02-15T18:39:00.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.672+0000 7f62fd05b700 1 -- 172.21.15.38:0/107753450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f80f5990 con 0x7f62f80f96f0 2024-02-15T18:39:00.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.672+0000 7f62f4ff9700 1 -- 172.21.15.38:0/107753450 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f62ec007ba0 con 0x7f62f80f96f0 2024-02-15T18:39:00.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.672+0000 7f62f4ff9700 1 -- 172.21.15.38:0/107753450 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62ec01c3c0 con 0x7f62f80f96f0 2024-02-15T18:39:00.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.672+0000 7f62fd05b700 1 -- 172.21.15.38:0/107753450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f80f5ef0 con 0x7f62f80f96f0 2024-02-15T18:39:00.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.673+0000 7f62fd05b700 1 -- 172.21.15.38:0/107753450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f8045ad0 con 0x7f62f80f96f0 2024-02-15T18:39:00.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.674+0000 7f62f4ff9700 1 -- 172.21.15.38:0/107753450 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f62ec01d420 con 0x7f62f80f96f0 2024-02-15T18:39:00.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.674+0000 7f62f4ff9700 1 --2- 172.21.15.38:0/107753450 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f62e0060f70 0x7f62e0063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:00.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.674+0000 7f62f4ff9700 1 -- 172.21.15.38:0/107753450 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f62ec07dd00 con 0x7f62f80f96f0 2024-02-15T18:39:00.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.674+0000 7f62f7fff700 1 --2- 172.21.15.38:0/107753450 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f62e0060f70 0x7f62e0063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:00.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.674+0000 7f62f7fff700 1 --2- 172.21.15.38:0/107753450 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f62e0060f70 0x7f62e0063430 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f62f80f6980 tx=0x7f62e800b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:00.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.678+0000 7f62f4ff9700 1 -- 172.21.15.38:0/107753450 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f62ec081030 con 0x7f62f80f96f0 2024-02-15T18:39:00.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.866+0000 7f62fd05b700 1 -- 172.21.15.38:0/107753450 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f62f80fef10 con 0x7f62e0060f70 2024-02-15T18:39:00.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.868+0000 7f62f4ff9700 1 -- 172.21.15.38:0/107753450 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f62f80fef10 con 0x7f62e0060f70 2024-02-15T18:39:00.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.869+0000 7f62fd05b700 1 -- 172.21.15.38:0/107753450 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f62e0060f70 msgr2=0x7f62e0063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:00.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.869+0000 7f62fd05b700 1 --2- 172.21.15.38:0/107753450 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f62e0060f70 0x7f62e0063430 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f62f80f6980 tx=0x7f62e800b040).stop 2024-02-15T18:39:00.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.869+0000 7f62fd05b700 1 -- 172.21.15.38:0/107753450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f80f96f0 msgr2=0x7f62f80f5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:00.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.869+0000 7f62fd05b700 1 --2- 172.21.15.38:0/107753450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f80f96f0 0x7f62f80f5070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f62ec004e30 tx=0x7f62ec009b30).stop 2024-02-15T18:39:00.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.869+0000 7f62fd05b700 1 -- 172.21.15.38:0/107753450 shutdown_connections 2024-02-15T18:39:00.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.869+0000 7f62fd05b700 1 --2- 172.21.15.38:0/107753450 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f62e0060f70 0x7f62e0063430 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:00.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.869+0000 7f62fd05b700 1 --2- 172.21.15.38:0/107753450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62f80602b0 0x7f62f80f6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:00.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.869+0000 7f62fd05b700 1 --2- 172.21.15.38:0/107753450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f80f96f0 0x7f62f80f5070 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:00.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.869+0000 7f62fd05b700 1 -- 172.21.15.38:0/107753450 >> 172.21.15.38:0/107753450 conn(0x7f62f80f0b90 msgr2=0x7f62f80f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:00.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.870+0000 7f62fd05b700 1 -- 172.21.15.38:0/107753450 shutdown_connections 2024-02-15T18:39:00.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.870+0000 7f62fd05b700 1 -- 172.21.15.38:0/107753450 wait complete. 2024-02-15T18:39:00.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.971+0000 7efc0cfcb700 1 -- 172.21.15.38:0/1803313086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc080f6860 msgr2=0x7efc080f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:00.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.971+0000 7efc0cfcb700 1 --2- 172.21.15.38:0/1803313086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc080f6860 0x7efc080f6cc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7efbf8004660 tx=0x7efbf8009b30).stop 2024-02-15T18:39:00.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.971+0000 7efc0cfcb700 1 -- 172.21.15.38:0/1803313086 shutdown_connections 2024-02-15T18:39:00.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.971+0000 7efc0cfcb700 1 --2- 172.21.15.38:0/1803313086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc080f6860 0x7efc080f6cc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:00.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.971+0000 7efc0cfcb700 1 --2- 172.21.15.38:0/1803313086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc080f55c0 0x7efc080f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:00.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.971+0000 7efc0cfcb700 1 -- 172.21.15.38:0/1803313086 >> 172.21.15.38:0/1803313086 conn(0x7efc080f0d40 msgr2=0x7efc080f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:00.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.971+0000 7efc0cfcb700 1 -- 172.21.15.38:0/1803313086 shutdown_connections 2024-02-15T18:39:00.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.971+0000 7efc0cfcb700 1 -- 172.21.15.38:0/1803313086 wait complete. 2024-02-15T18:39:00.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.971+0000 7efc0cfcb700 1 Processor -- start 2024-02-15T18:39:00.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.971+0000 7efc0cfcb700 1 -- start start 2024-02-15T18:39:00.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.972+0000 7efc0cfcb700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc080f55c0 0x7efc08106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:00.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.972+0000 7efc0cfcb700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc080f6860 0x7efc081074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:00.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.972+0000 7efc0cfcb700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc08107a60 con 0x7efc080f55c0 2024-02-15T18:39:00.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.972+0000 7efc0cfcb700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc08107ba0 con 0x7efc080f6860 2024-02-15T18:39:00.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.972+0000 7efc0659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc080f55c0 0x7efc08106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:00.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.972+0000 7efc0659c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc080f55c0 0x7efc08106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46342/0 (socket says 172.21.15.38:46342) 2024-02-15T18:39:00.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.972+0000 7efc0659c700 1 -- 172.21.15.38:0/3628734460 learned_addr learned my addr 172.21.15.38:0/3628734460 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:39:00.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.972+0000 7efc05d9b700 1 --2- 172.21.15.38:0/3628734460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc080f6860 0x7efc081074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:00.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.972+0000 7efc0659c700 1 -- 172.21.15.38:0/3628734460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc080f6860 msgr2=0x7efc081074b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:00.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.972+0000 7efc0659c700 1 --2- 172.21.15.38:0/3628734460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc080f6860 0x7efc081074b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:00.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.972+0000 7efc0659c700 1 -- 172.21.15.38:0/3628734460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf8005040 con 0x7efc080f55c0 2024-02-15T18:39:00.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.972+0000 7efc0659c700 1 --2- 172.21.15.38:0/3628734460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc080f55c0 0x7efc08106fc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7efbfc002700 tx=0x7efbfc009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:00.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.972+0000 7efbf6ffd700 1 -- 172.21.15.38:0/3628734460 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbfc0079c0 con 0x7efc080f55c0 2024-02-15T18:39:00.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.972+0000 7efbf6ffd700 1 -- 172.21.15.38:0/3628734460 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7efbfc00f040 con 0x7efc080f55c0 2024-02-15T18:39:00.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.972+0000 7efc0cfcb700 1 -- 172.21.15.38:0/3628734460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc08108540 con 0x7efc080f55c0 2024-02-15T18:39:00.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.972+0000 7efbf6ffd700 1 -- 172.21.15.38:0/3628734460 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbfc014940 con 0x7efc080f55c0 2024-02-15T18:39:00.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.972+0000 7efc0cfcb700 1 -- 172.21.15.38:0/3628734460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc08108b00 con 0x7efc080f55c0 2024-02-15T18:39:00.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.973+0000 7efc0cfcb700 1 -- 172.21.15.38:0/3628734460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc08045ad0 con 0x7efc080f55c0 2024-02-15T18:39:00.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.974+0000 7efbf6ffd700 1 -- 172.21.15.38:0/3628734460 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7efbfc01b060 con 0x7efc080f55c0 2024-02-15T18:39:00.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.974+0000 7efbf6ffd700 1 --2- 172.21.15.38:0/3628734460 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7efbf0060f20 0x7efbf00633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:00.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.974+0000 7efbf6ffd700 1 -- 172.21.15.38:0/3628734460 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5812+0+0 (secure 0 0 0) 0x7efbfc07bea0 con 0x7efc080f55c0 2024-02-15T18:39:00.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.974+0000 7efc05d9b700 1 --2- 172.21.15.38:0/3628734460 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7efbf0060f20 0x7efbf00633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:00.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.974+0000 7efc05d9b700 1 --2- 172.21.15.38:0/3628734460 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7efbf0060f20 0x7efbf00633e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7efbf8004630 tx=0x7efbf8009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:00.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:00.978+0000 7efbf6ffd700 1 -- 172.21.15.38:0/3628734460 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7efbfc04fdd0 con 0x7efc080f55c0 2024-02-15T18:39:01.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.164+0000 7efc0cfcb700 1 -- 172.21.15.38:0/3628734460 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efc080faca0 con 0x7efbf0060f20 2024-02-15T18:39:01.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.174+0000 7efbf6ffd700 1 -- 172.21.15.38:0/3628734460 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7efc080faca0 con 0x7efbf0060f20 2024-02-15T18:39:01.164 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:39:01.164 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (9m) 24s ago 11m 19.3M - 0.20.0 0881eb8f169f c2034c318e7c 2024-02-15T18:39:01.164 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (118s) 24s ago 11m 6635k - 16.2.14-521-g0e714d9a 1fbc92c859dc c2fee0824d4b 2024-02-15T18:39:01.164 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi121 smithi121 running (113s) 109s ago 10m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 05c1a5501a35 2024-02-15T18:39:01.165 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (9m) 24s ago 11m 33.4M - 6.7.4 557c83e11646 2dcccfbfbd2a 2024-02-15T18:39:01.165 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.hxoggl smithi038 running (6m) 24s ago 6m 14.8M - 16.2.4 8d91d370c2b8 be2533754b9c 2024-02-15T18:39:01.165 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.qgkzgn smithi038 running (6m) 24s ago 6m 14.4M - 16.2.4 8d91d370c2b8 dd3ffafba9bd 2024-02-15T18:39:01.165 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.mvumlh smithi121 running (6m) 109s ago 6m 116M - 16.2.4 8d91d370c2b8 88b7d876a39b 2024-02-15T18:39:01.165 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.ovuzki smithi121 running (6m) 109s ago 6m 124M - 16.2.4 8d91d370c2b8 21cacf0012f9 2024-02-15T18:39:01.165 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.khesbl smithi038 *:8443,9283 running (3m) 24s ago 13m 433M - 16.2.14-521-g0e714d9a 1fbc92c859dc e642f3c2cfc9 2024-02-15T18:39:01.165 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi121.ckiryi smithi121 *:8443,9283 running (2m) 109s ago 10m 385M - 16.2.14-521-g0e714d9a 1fbc92c859dc 82515b025594 2024-02-15T18:39:01.165 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (2m) 24s ago 13m 40.8M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc d3e8317a30ca 2024-02-15T18:39:01.165 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi121 smithi121 running (2m) 109s ago 10m 26.2M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 42e1dda1a39d 2024-02-15T18:39:01.165 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (11m) 24s ago 11m 17.1M - 0.18.1 e5a616e4b9cf 037f60e38158 2024-02-15T18:39:01.165 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi121 smithi121 *:9100 running (10m) 109s ago 10m 17.2M - 0.18.1 e5a616e4b9cf a867bff85e18 2024-02-15T18:39:01.165 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (94s) 24s ago 9m 191M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 623ac7ecbb0a 2024-02-15T18:39:01.165 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (63s) 24s ago 9m 217M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 0f9e8e1f255a 2024-02-15T18:39:01.165 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (31s) 24s ago 8m 13.4M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 37171264b5be 2024-02-15T18:39:01.166 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi121 running (8m) 109s ago 8m 449M 4096M 16.2.4 8d91d370c2b8 cba2d9ed4730 2024-02-15T18:39:01.166 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi121 running (8m) 109s ago 8m 429M 4096M 16.2.4 8d91d370c2b8 ac17ad7dc628 2024-02-15T18:39:01.166 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi121 running (7m) 109s ago 7m 378M 4096M 16.2.4 8d91d370c2b8 cdc8c669c4db 2024-02-15T18:39:01.166 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (3m) 24s ago 10m 65.9M - 2.18.1 de242295e225 67e33f9de4e6 2024-02-15T18:39:01.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.175+0000 7efc0cfcb700 1 -- 172.21.15.38:0/3628734460 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7efbf0060f20 msgr2=0x7efbf00633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:01.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.175+0000 7efc0cfcb700 1 --2- 172.21.15.38:0/3628734460 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7efbf0060f20 0x7efbf00633e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7efbf8004630 tx=0x7efbf8009b30).stop 2024-02-15T18:39:01.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.175+0000 7efc0cfcb700 1 -- 172.21.15.38:0/3628734460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc080f55c0 msgr2=0x7efc08106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:01.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.175+0000 7efc0cfcb700 1 --2- 172.21.15.38:0/3628734460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc080f55c0 0x7efc08106fc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7efbfc002700 tx=0x7efbfc009d80).stop 2024-02-15T18:39:01.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.176+0000 7efc0cfcb700 1 -- 172.21.15.38:0/3628734460 shutdown_connections 2024-02-15T18:39:01.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.176+0000 7efc0cfcb700 1 --2- 172.21.15.38:0/3628734460 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7efbf0060f20 0x7efbf00633e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:01.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.176+0000 7efc0cfcb700 1 --2- 172.21.15.38:0/3628734460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc080f55c0 0x7efc08106fc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:01.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.176+0000 7efc0cfcb700 1 --2- 172.21.15.38:0/3628734460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc080f6860 0x7efc081074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:01.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.176+0000 7efc0cfcb700 1 -- 172.21.15.38:0/3628734460 >> 172.21.15.38:0/3628734460 conn(0x7efc080f0d40 msgr2=0x7efc0805fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:01.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.176+0000 7efc0cfcb700 1 -- 172.21.15.38:0/3628734460 shutdown_connections 2024-02-15T18:39:01.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.176+0000 7efc0cfcb700 1 -- 172.21.15.38:0/3628734460 wait complete. 2024-02-15T18:39:01.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.276+0000 7fd639112700 1 -- 172.21.15.38:0/3902198205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6340602f0 msgr2=0x7fd6340f9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:01.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.276+0000 7fd639112700 1 --2- 172.21.15.38:0/3902198205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6340602f0 0x7fd6340f9400 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fd624004660 tx=0x7fd624009b30).stop 2024-02-15T18:39:01.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.276+0000 7fd639112700 1 -- 172.21.15.38:0/3902198205 shutdown_connections 2024-02-15T18:39:01.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.276+0000 7fd639112700 1 --2- 172.21.15.38:0/3902198205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6340f98f0 0x7fd6340fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:01.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.276+0000 7fd639112700 1 --2- 172.21.15.38:0/3902198205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6340602f0 0x7fd6340f9400 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:01.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.276+0000 7fd639112700 1 -- 172.21.15.38:0/3902198205 >> 172.21.15.38:0/3902198205 conn(0x7fd6340f0de0 msgr2=0x7fd6340f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:01.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.276+0000 7fd639112700 1 -- 172.21.15.38:0/3902198205 shutdown_connections 2024-02-15T18:39:01.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.276+0000 7fd639112700 1 -- 172.21.15.38:0/3902198205 wait complete. 2024-02-15T18:39:01.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.277+0000 7fd639112700 1 Processor -- start 2024-02-15T18:39:01.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.277+0000 7fd639112700 1 -- start start 2024-02-15T18:39:01.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.277+0000 7fd639112700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6340602f0 0x7fd6340f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:01.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.277+0000 7fd639112700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6340f98f0 0x7fd6340f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:01.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.277+0000 7fd639112700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6340f5860 con 0x7fd6340f98f0 2024-02-15T18:39:01.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.277+0000 7fd639112700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6340f59a0 con 0x7fd6340602f0 2024-02-15T18:39:01.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.277+0000 7fd63259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6340f98f0 0x7fd6340f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:01.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.277+0000 7fd63259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6340f98f0 0x7fd6340f5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46362/0 (socket says 172.21.15.38:46362) 2024-02-15T18:39:01.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.277+0000 7fd63259c700 1 -- 172.21.15.38:0/2008936671 learned_addr learned my addr 172.21.15.38:0/2008936671 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:39:01.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.277+0000 7fd632d9d700 1 --2- 172.21.15.38:0/2008936671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6340602f0 0x7fd6340f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:01.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.277+0000 7fd63259c700 1 -- 172.21.15.38:0/2008936671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6340602f0 msgr2=0x7fd6340f6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:01.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.277+0000 7fd63259c700 1 --2- 172.21.15.38:0/2008936671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6340602f0 0x7fd6340f6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:01.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.277+0000 7fd63259c700 1 -- 172.21.15.38:0/2008936671 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd624005040 con 0x7fd6340f98f0 2024-02-15T18:39:01.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.278+0000 7fd63259c700 1 --2- 172.21.15.38:0/2008936671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6340f98f0 0x7fd6340f5300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fd6280026d0 tx=0x7fd628009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:01.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.278+0000 7fd61b7fe700 1 -- 172.21.15.38:0/2008936671 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6280079c0 con 0x7fd6340f98f0 2024-02-15T18:39:01.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.278+0000 7fd61b7fe700 1 -- 172.21.15.38:0/2008936671 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fd62800f040 con 0x7fd6340f98f0 2024-02-15T18:39:01.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.278+0000 7fd639112700 1 -- 172.21.15.38:0/2008936671 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6340f5c80 con 0x7fd6340f98f0 2024-02-15T18:39:01.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.278+0000 7fd61b7fe700 1 -- 172.21.15.38:0/2008936671 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6280149f0 con 0x7fd6340f98f0 2024-02-15T18:39:01.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.278+0000 7fd639112700 1 -- 172.21.15.38:0/2008936671 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63410f000 con 0x7fd6340f98f0 2024-02-15T18:39:01.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.279+0000 7fd639112700 1 -- 172.21.15.38:0/2008936671 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd634045ad0 con 0x7fd6340f98f0 2024-02-15T18:39:01.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.279+0000 7fd61b7fe700 1 -- 172.21.15.38:0/2008936671 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fd628007b20 con 0x7fd6340f98f0 2024-02-15T18:39:01.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.279+0000 7fd61b7fe700 1 --2- 172.21.15.38:0/2008936671 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd61c060f20 0x7fd61c0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:01.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.280+0000 7fd61b7fe700 1 -- 172.21.15.38:0/2008936671 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fd62807c1a0 con 0x7fd6340f98f0 2024-02-15T18:39:01.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.280+0000 7fd632d9d700 1 --2- 172.21.15.38:0/2008936671 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd61c060f20 0x7fd61c0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:01.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.280+0000 7fd632d9d700 1 --2- 172.21.15.38:0/2008936671 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd61c060f20 0x7fd61c0633e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fd6340f6c10 tx=0x7fd624009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:01.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.284+0000 7fd61b7fe700 1 -- 172.21.15.38:0/2008936671 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd62804c560 con 0x7fd6340f98f0 2024-02-15T18:39:01.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.529+0000 7fd639112700 1 -- 172.21.15.38:0/2008936671 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd634027a00 con 0x7fd6340f98f0 2024-02-15T18:39:01.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.530+0000 7fd61b7fe700 1 -- 172.21.15.38:0/2008936671 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+820 (secure 0 0 0) 0x7fd62804fb80 con 0x7fd6340f98f0 2024-02-15T18:39:01.520 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:39:01.520 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-02-15T18:39:01.521 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:39:01.521 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:39:01.521 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-02-15T18:39:01.521 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:39:01.521 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:39:01.521 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-02-15T18:39:01.521 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 3, 2024-02-15T18:39:01.521 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2024-02-15T18:39:01.521 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:39:01.521 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-02-15T18:39:01.521 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:39:01.521 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:39:01.521 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-02-15T18:39:01.521 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 7, 2024-02-15T18:39:01.522 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2024-02-15T18:39:01.522 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-02-15T18:39:01.522 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:39:01.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.531+0000 7fd639112700 1 -- 172.21.15.38:0/2008936671 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd61c060f20 msgr2=0x7fd61c0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.531+0000 7fd639112700 1 --2- 172.21.15.38:0/2008936671 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd61c060f20 0x7fd61c0633e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fd6340f6c10 tx=0x7fd624009b30).stop 2024-02-15T18:39:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.531+0000 7fd639112700 1 -- 172.21.15.38:0/2008936671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6340f98f0 msgr2=0x7fd6340f5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.531+0000 7fd639112700 1 --2- 172.21.15.38:0/2008936671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6340f98f0 0x7fd6340f5300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fd6280026d0 tx=0x7fd628009de0).stop 2024-02-15T18:39:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.532+0000 7fd639112700 1 -- 172.21.15.38:0/2008936671 shutdown_connections 2024-02-15T18:39:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.532+0000 7fd639112700 1 --2- 172.21.15.38:0/2008936671 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd61c060f20 0x7fd61c0633e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.532+0000 7fd639112700 1 --2- 172.21.15.38:0/2008936671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6340602f0 0x7fd6340f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.532+0000 7fd639112700 1 --2- 172.21.15.38:0/2008936671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6340f98f0 0x7fd6340f5300 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.532+0000 7fd639112700 1 -- 172.21.15.38:0/2008936671 >> 172.21.15.38:0/2008936671 conn(0x7fd6340f0de0 msgr2=0x7fd6340f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.532+0000 7fd639112700 1 -- 172.21.15.38:0/2008936671 shutdown_connections 2024-02-15T18:39:01.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.532+0000 7fd639112700 1 -- 172.21.15.38:0/2008936671 wait complete. 2024-02-15T18:39:01.606 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:39:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-3[62228]: debug 2024-02-15T18:39:01.185+0000 7f5333ba1700 1 -- 172.21.15.121:0/515330272 <== mon.1 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5568bd82e680 con 0x5568bd7f5400 2024-02-15T18:39:01.606 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:39:01 smithi121 bash[62213]: d505a58b375cc8e4dc9a23ee695f8b23520334c0dedb1b1f67dab9c7cc951e5a 2024-02-15T18:39:01.606 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:39:01 smithi121 systemd[1]: Started Ceph osd.3 for 7e05753e-cc2f-11ee-95ba-87774f69a715. 2024-02-15T18:39:01.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.633+0000 7f37ae6de700 1 -- 172.21.15.38:0/1974054064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a80f6860 msgr2=0x7f37a80f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:01.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.633+0000 7f37ae6de700 1 --2- 172.21.15.38:0/1974054064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a80f6860 0x7f37a80f6cc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f379c004660 tx=0x7f379c009b30).stop 2024-02-15T18:39:01.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.633+0000 7f37ae6de700 1 -- 172.21.15.38:0/1974054064 shutdown_connections 2024-02-15T18:39:01.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.633+0000 7f37ae6de700 1 --2- 172.21.15.38:0/1974054064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a80f6860 0x7f37a80f6cc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:01.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.633+0000 7f37ae6de700 1 --2- 172.21.15.38:0/1974054064 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37a80f55c0 0x7f37a80f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:01.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.633+0000 7f37ae6de700 1 -- 172.21.15.38:0/1974054064 >> 172.21.15.38:0/1974054064 conn(0x7f37a80f0d40 msgr2=0x7f37a80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:01.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.633+0000 7f37ae6de700 1 -- 172.21.15.38:0/1974054064 shutdown_connections 2024-02-15T18:39:01.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.633+0000 7f37ae6de700 1 -- 172.21.15.38:0/1974054064 wait complete. 2024-02-15T18:39:01.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.634+0000 7f37ae6de700 1 Processor -- start 2024-02-15T18:39:01.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.634+0000 7f37ae6de700 1 -- start start 2024-02-15T18:39:01.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.634+0000 7f37ae6de700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a80f55c0 0x7f37a81091c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:01.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.634+0000 7f37ae6de700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37a80f6860 0x7f37a81096b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:01.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.634+0000 7f37ae6de700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a8109c60 con 0x7f37a80f55c0 2024-02-15T18:39:01.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.634+0000 7f37ae6de700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a8109da0 con 0x7f37a80f6860 2024-02-15T18:39:01.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.634+0000 7f37a7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a80f55c0 0x7f37a81091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:01.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.634+0000 7f37a7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a80f55c0 0x7f37a81091c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46376/0 (socket says 172.21.15.38:46376) 2024-02-15T18:39:01.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.634+0000 7f37a7fff700 1 -- 172.21.15.38:0/1653342271 learned_addr learned my addr 172.21.15.38:0/1653342271 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:39:01.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.634+0000 7f37a77fe700 1 --2- 172.21.15.38:0/1653342271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37a80f6860 0x7f37a81096b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:01.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.634+0000 7f37a7fff700 1 -- 172.21.15.38:0/1653342271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37a80f6860 msgr2=0x7f37a81096b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:01.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.634+0000 7f37a7fff700 1 --2- 172.21.15.38:0/1653342271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37a80f6860 0x7f37a81096b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:01.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.634+0000 7f37a7fff700 1 -- 172.21.15.38:0/1653342271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c005040 con 0x7f37a80f55c0 2024-02-15T18:39:01.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.635+0000 7f37a7fff700 1 --2- 172.21.15.38:0/1653342271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a80f55c0 0x7f37a81091c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f379800e7c0 tx=0x7f379800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:01.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.635+0000 7f37a4ff9700 1 -- 172.21.15.38:0/1653342271 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37980092e0 con 0x7f37a80f55c0 2024-02-15T18:39:01.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.635+0000 7f37a4ff9700 1 -- 172.21.15.38:0/1653342271 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f379800f040 con 0x7f37a80f55c0 2024-02-15T18:39:01.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.635+0000 7f37ae6de700 1 -- 172.21.15.38:0/1653342271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a810a740 con 0x7f37a80f55c0 2024-02-15T18:39:01.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.635+0000 7f37a4ff9700 1 -- 172.21.15.38:0/1653342271 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3798014920 con 0x7f37a80f55c0 2024-02-15T18:39:01.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.635+0000 7f37ae6de700 1 -- 172.21.15.38:0/1653342271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a810ad00 con 0x7f37a80f55c0 2024-02-15T18:39:01.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.636+0000 7f37ae6de700 1 -- 172.21.15.38:0/1653342271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a8045ad0 con 0x7f37a80f55c0 2024-02-15T18:39:01.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.636+0000 7f37a4ff9700 1 -- 172.21.15.38:0/1653342271 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f379801b030 con 0x7f37a80f55c0 2024-02-15T18:39:01.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.637+0000 7f37a4ff9700 1 --2- 172.21.15.38:0/1653342271 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3790060f20 0x7f37900633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:01.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.637+0000 7f37a4ff9700 1 -- 172.21.15.38:0/1653342271 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f379807c110 con 0x7f37a80f55c0 2024-02-15T18:39:01.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.637+0000 7f37a77fe700 1 --2- 172.21.15.38:0/1653342271 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3790060f20 0x7f37900633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:01.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.637+0000 7f37a77fe700 1 --2- 172.21.15.38:0/1653342271 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3790060f20 0x7f37900633e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f379c009720 tx=0x7f379c016040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:01.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.641+0000 7f37a4ff9700 1 -- 172.21.15.38:0/1653342271 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f379804ff00 con 0x7f37a80f55c0 2024-02-15T18:39:01.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.855+0000 7f37ae6de700 1 -- 172.21.15.38:0/1653342271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f37a810f7e0 con 0x7f37a80f55c0 2024-02-15T18:39:01.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.856+0000 7f37a4ff9700 1 -- 172.21.15.38:0/1653342271 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1807 (secure 0 0 0) 0x7f3798019070 con 0x7f37a80f55c0 2024-02-15T18:39:01.846 INFO:teuthology.orchestra.run.smithi038.stdout:e13 2024-02-15T18:39:01.846 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:39:01.846 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:39:01.846 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-02-15T18:39:01.847 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:39:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:39:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-02-15T18:39:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 13 2024-02-15T18:39:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 2024-02-15T18:39:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:39:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-02-15T18:35:07.485909+0000 2024-02-15T18:39:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-02-15T18:39:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-02-15T18:39:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-02-15T18:39:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-02-15T18:39:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-02-15T18:39:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-02-15T18:39:01.847 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-02-15T18:39:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-02-15T18:39:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:39:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-02-15T18:39:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-02-15T18:39:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=24261,1=24275} 2024-02-15T18:39:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-02-15T18:39:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-02-15T18:39:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-02-15T18:39:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-02-15T18:39:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-02-15T18:39:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-02-15T18:39:01.848 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-02-15T18:39:01.849 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-02-15T18:39:01.849 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.mvumlh{0:24261} state up:active seq 3 export targets 1 join_fscid=1 addr [v2:172.21.15.121:6824/1422707571,v1:172.21.15.121:6825/1422707571] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:39:01.849 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.ovuzki{1:24275} state up:active seq 2 export targets 0 join_fscid=1 addr [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:39:01.849 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:39:01.849 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:39:01.849 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-02-15T18:39:01.849 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:39:01.849 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.hxoggl{-1:14446} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.38:6826/3822880235,v1:172.21.15.38:6827/3822880235] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:39:01.849 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.qgkzgn{-1:14456} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.38:6828/703030231,v1:172.21.15.38:6829/703030231] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:39:01.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.857+0000 7f37ae6de700 1 -- 172.21.15.38:0/1653342271 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3790060f20 msgr2=0x7f37900633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:01.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.857+0000 7f37ae6de700 1 --2- 172.21.15.38:0/1653342271 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3790060f20 0x7f37900633e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f379c009720 tx=0x7f379c016040).stop 2024-02-15T18:39:01.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.857+0000 7f37ae6de700 1 -- 172.21.15.38:0/1653342271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a80f55c0 msgr2=0x7f37a81091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:01.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.857+0000 7f37ae6de700 1 --2- 172.21.15.38:0/1653342271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a80f55c0 0x7f37a81091c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f379800e7c0 tx=0x7f379800b040).stop 2024-02-15T18:39:01.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.858+0000 7f37ae6de700 1 -- 172.21.15.38:0/1653342271 shutdown_connections 2024-02-15T18:39:01.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.858+0000 7f37ae6de700 1 --2- 172.21.15.38:0/1653342271 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3790060f20 0x7f37900633e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:01.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.858+0000 7f37ae6de700 1 --2- 172.21.15.38:0/1653342271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a80f55c0 0x7f37a81091c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:01.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.858+0000 7f37ae6de700 1 --2- 172.21.15.38:0/1653342271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37a80f6860 0x7f37a81096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:01.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.858+0000 7f37ae6de700 1 -- 172.21.15.38:0/1653342271 >> 172.21.15.38:0/1653342271 conn(0x7f37a80f0d40 msgr2=0x7f37a805fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:01.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.858+0000 7f37ae6de700 1 -- 172.21.15.38:0/1653342271 shutdown_connections 2024-02-15T18:39:01.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.858+0000 7f37ae6de700 1 -- 172.21.15.38:0/1653342271 wait complete. 2024-02-15T18:39:01.851 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 13 2024-02-15T18:39:01.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.960+0000 7ff732655700 1 -- 172.21.15.38:0/861985837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff72c0f5440 msgr2=0x7ff72c0f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:01.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.960+0000 7ff732655700 1 --2- 172.21.15.38:0/861985837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff72c0f5440 0x7ff72c0f5860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7ff728004660 tx=0x7ff728009b30).stop 2024-02-15T18:39:01.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.960+0000 7ff732655700 1 -- 172.21.15.38:0/861985837 shutdown_connections 2024-02-15T18:39:01.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.960+0000 7ff732655700 1 --2- 172.21.15.38:0/861985837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff72c0f66e0 0x7ff72c0f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:01.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.960+0000 7ff732655700 1 --2- 172.21.15.38:0/861985837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff72c0f5440 0x7ff72c0f5860 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:01.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.960+0000 7ff732655700 1 -- 172.21.15.38:0/861985837 >> 172.21.15.38:0/861985837 conn(0x7ff72c0f0ba0 msgr2=0x7ff72c0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:01.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.960+0000 7ff732655700 1 -- 172.21.15.38:0/861985837 shutdown_connections 2024-02-15T18:39:01.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.960+0000 7ff732655700 1 -- 172.21.15.38:0/861985837 wait complete. 2024-02-15T18:39:01.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.961+0000 7ff732655700 1 Processor -- start 2024-02-15T18:39:01.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.961+0000 7ff732655700 1 -- start start 2024-02-15T18:39:01.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.961+0000 7ff732655700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff72c0f5440 0x7ff72c106ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:01.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.961+0000 7ff732655700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff72c0f66e0 0x7ff72c107390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:01.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.961+0000 7ff732655700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c107940 con 0x7ff72c0f5440 2024-02-15T18:39:01.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.961+0000 7ff732655700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c108140 con 0x7ff72c0f66e0 2024-02-15T18:39:01.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.961+0000 7ff731653700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff72c0f5440 0x7ff72c106ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:01.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.961+0000 7ff731653700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff72c0f5440 0x7ff72c106ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46390/0 (socket says 172.21.15.38:46390) 2024-02-15T18:39:01.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.961+0000 7ff731653700 1 -- 172.21.15.38:0/1679937490 learned_addr learned my addr 172.21.15.38:0/1679937490 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:39:01.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.961+0000 7ff730e52700 1 --2- 172.21.15.38:0/1679937490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff72c0f66e0 0x7ff72c107390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:01.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.962+0000 7ff731653700 1 -- 172.21.15.38:0/1679937490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff72c0f66e0 msgr2=0x7ff72c107390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:01.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.962+0000 7ff731653700 1 --2- 172.21.15.38:0/1679937490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff72c0f66e0 0x7ff72c107390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:01.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.962+0000 7ff731653700 1 -- 172.21.15.38:0/1679937490 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff728005040 con 0x7ff72c0f5440 2024-02-15T18:39:01.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.962+0000 7ff731653700 1 --2- 172.21.15.38:0/1679937490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff72c0f5440 0x7ff72c106ea0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7ff728004e30 tx=0x7ff728009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:01.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.962+0000 7ff721ffb700 1 -- 172.21.15.38:0/1679937490 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff728018070 con 0x7ff72c0f5440 2024-02-15T18:39:01.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.962+0000 7ff732655700 1 -- 172.21.15.38:0/1679937490 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff72c1083c0 con 0x7ff72c0f5440 2024-02-15T18:39:01.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.962+0000 7ff721ffb700 1 -- 172.21.15.38:0/1679937490 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7ff728007ba0 con 0x7ff72c0f5440 2024-02-15T18:39:01.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.962+0000 7ff721ffb700 1 -- 172.21.15.38:0/1679937490 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff72801c3c0 con 0x7ff72c0f5440 2024-02-15T18:39:01.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.962+0000 7ff732655700 1 -- 172.21.15.38:0/1679937490 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff72c108920 con 0x7ff72c0f5440 2024-02-15T18:39:01.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.963+0000 7ff732655700 1 -- 172.21.15.38:0/1679937490 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72c059100 con 0x7ff72c0f5440 2024-02-15T18:39:01.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.964+0000 7ff721ffb700 1 -- 172.21.15.38:0/1679937490 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7ff72801d420 con 0x7ff72c0f5440 2024-02-15T18:39:01.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.964+0000 7ff721ffb700 1 --2- 172.21.15.38:0/1679937490 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7ff718060f70 0x7ff718063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:01.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.964+0000 7ff721ffb700 1 -- 172.21.15.38:0/1679937490 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5812+0+0 (secure 0 0 0) 0x7ff72807dd00 con 0x7ff72c0f5440 2024-02-15T18:39:01.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.964+0000 7ff730e52700 1 --2- 172.21.15.38:0/1679937490 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7ff718060f70 0x7ff718063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:01.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.964+0000 7ff730e52700 1 --2- 172.21.15.38:0/1679937490 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7ff718060f70 0x7ff718063430 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7ff71c004c40 tx=0x7ff71c00b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:01.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:01.968+0000 7ff721ffb700 1 -- 172.21.15.38:0/1679937490 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff728081030 con 0x7ff72c0f5440 2024-02-15T18:39:01.972 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:39:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-3[62228]: debug 2024-02-15T18:39:01.701+0000 7f533b62f380 -1 Falling back to public interface 2024-02-15T18:39:02.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:02.158+0000 7ff732655700 1 -- 172.21.15.38:0/1679937490 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff72c0fab20 con 0x7ff718060f70 2024-02-15T18:39:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:02.159+0000 7ff721ffb700 1 -- 172.21.15.38:0/1679937490 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff72c0fab20 con 0x7ff718060f70 2024-02-15T18:39:02.149 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:39:02.149 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:39:02.149 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-02-15T18:39:02.149 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:39:02.149 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-02-15T18:39:02.150 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-02-15T18:39:02.150 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-02-15T18:39:02.150 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr" 2024-02-15T18:39:02.150 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-02-15T18:39:02.150 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "9/21 daemons upgraded", 2024-02-15T18:39:02.150 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading osd daemons", 2024-02-15T18:39:02.150 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-02-15T18:39:02.150 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:39:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:02.161+0000 7ff732655700 1 -- 172.21.15.38:0/1679937490 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7ff718060f70 msgr2=0x7ff718063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:02.161+0000 7ff732655700 1 --2- 172.21.15.38:0/1679937490 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7ff718060f70 0x7ff718063430 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7ff71c004c40 tx=0x7ff71c00b040).stop 2024-02-15T18:39:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:02.161+0000 7ff732655700 1 -- 172.21.15.38:0/1679937490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff72c0f5440 msgr2=0x7ff72c106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:02.161+0000 7ff732655700 1 --2- 172.21.15.38:0/1679937490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff72c0f5440 0x7ff72c106ea0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7ff728004e30 tx=0x7ff728009b30).stop 2024-02-15T18:39:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:02.161+0000 7ff732655700 1 -- 172.21.15.38:0/1679937490 shutdown_connections 2024-02-15T18:39:02.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:02.161+0000 7ff732655700 1 --2- 172.21.15.38:0/1679937490 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7ff718060f70 0x7ff718063430 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:02.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:02.161+0000 7ff732655700 1 --2- 172.21.15.38:0/1679937490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff72c0f5440 0x7ff72c106ea0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:02.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:02.161+0000 7ff732655700 1 --2- 172.21.15.38:0/1679937490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff72c0f66e0 0x7ff72c107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:02.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:02.161+0000 7ff732655700 1 -- 172.21.15.38:0/1679937490 >> 172.21.15.38:0/1679937490 conn(0x7ff72c0f0ba0 msgr2=0x7ff72c05fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:02.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:02.161+0000 7ff732655700 1 -- 172.21.15.38:0/1679937490 shutdown_connections 2024-02-15T18:39:02.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:02.161+0000 7ff732655700 1 -- 172.21.15.38:0/1679937490 wait complete. 2024-02-15T18:39:02.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:00.567258+0000 mgr.smithi038.khesbl (mgr.14664) 154 : audit [DBG] from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:39:02.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:00.869117+0000 mgr.smithi038.khesbl (mgr.14664) 155 : audit [DBG] from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:39:02.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:00.964161+0000 mgr.smithi038.khesbl (mgr.14664) 156 : cluster [DBG] pgmap v110: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 70/342 objects degraded (20.468%) 2024-02-15T18:39:02.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:01.318568+0000 mon.smithi038 (mon.0) 242 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:02.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:01.319466+0000 mon.smithi038 (mon.0) 243 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:39:02.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:01.531465+0000 mon.smithi038 (mon.0) 244 : audit [DBG] from='client.? 172.21.15.38:0/2008936671' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:39:02.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:01.857460+0000 mon.smithi038 (mon.0) 245 : audit [DBG] from='client.? 172.21.15.38:0/1653342271' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:39:02.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:00.567258+0000 mgr.smithi038.khesbl (mgr.14664) 154 : audit [DBG] from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:39:02.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:00.869117+0000 mgr.smithi038.khesbl (mgr.14664) 155 : audit [DBG] from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:39:02.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:00.964161+0000 mgr.smithi038.khesbl (mgr.14664) 156 : cluster [DBG] pgmap v110: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 70/342 objects degraded (20.468%) 2024-02-15T18:39:02.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:01.318568+0000 mon.smithi038 (mon.0) 242 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:02.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:01.319466+0000 mon.smithi038 (mon.0) 243 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:39:02.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:01.531465+0000 mon.smithi038 (mon.0) 244 : audit [DBG] from='client.? 172.21.15.38:0/2008936671' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:39:02.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:01.857460+0000 mon.smithi038 (mon.0) 245 : audit [DBG] from='client.? 172.21.15.38:0/1653342271' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:39:03.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:01.166712+0000 mgr.smithi038.khesbl (mgr.14664) 157 : audit [DBG] from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:39:03.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:01.166712+0000 mgr.smithi038.khesbl (mgr.14664) 157 : audit [DBG] from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:39:04.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:02.160293+0000 mgr.smithi038.khesbl (mgr.14664) 158 : audit [DBG] from='client.34230 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:39:04.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:02.964599+0000 mgr.smithi038.khesbl (mgr.14664) 159 : cluster [DBG] pgmap v111: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 70/342 objects degraded (20.468%) 2024-02-15T18:39:04.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:03.150627+0000 mon.smithi038 (mon.0) 246 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:04.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:02.160293+0000 mgr.smithi038.khesbl (mgr.14664) 158 : audit [DBG] from='client.34230 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:39:04.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:02.964599+0000 mgr.smithi038.khesbl (mgr.14664) 159 : cluster [DBG] pgmap v111: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 70/342 objects degraded (20.468%) 2024-02-15T18:39:04.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:03.150627+0000 mon.smithi038 (mon.0) 246 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:06.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:04.837951+0000 mon.smithi038 (mon.0) 247 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:06.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:04.838865+0000 mon.smithi038 (mon.0) 248 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:39:06.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:04.839856+0000 mon.smithi038 (mon.0) 249 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:39:06.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:39:04.840610+0000 mgr.smithi038.khesbl (mgr.14664) 160 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:39:06.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:04.844413+0000 mon.smithi038 (mon.0) 250 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:06.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:04.850442+0000 mon.smithi038 (mon.0) 251 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:39:06.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:04.853057+0000 mon.smithi038 (mon.0) 252 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:39:06.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:04.854469+0000 mon.smithi038 (mon.0) 253 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:39:06.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:04.855839+0000 mon.smithi038 (mon.0) 254 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:39:06.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:04.857270+0000 mon.smithi038 (mon.0) 255 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-15T18:39:06.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:04.857438+0000 mgr.smithi038.khesbl (mgr.14664) 161 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-15T18:39:06.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:39:04.858204+0000 mgr.smithi038.khesbl (mgr.14664) 162 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-02-15T18:39:06.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:04.965256+0000 mgr.smithi038.khesbl (mgr.14664) 163 : cluster [DBG] pgmap v112: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 70/342 objects degraded (20.468%) 2024-02-15T18:39:06.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:04.837951+0000 mon.smithi038 (mon.0) 247 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:06.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:04.838865+0000 mon.smithi038 (mon.0) 248 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:39:06.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:04.839856+0000 mon.smithi038 (mon.0) 249 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:39:06.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:39:04.840610+0000 mgr.smithi038.khesbl (mgr.14664) 160 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:39:06.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:04.844413+0000 mon.smithi038 (mon.0) 250 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:06.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:04.850442+0000 mon.smithi038 (mon.0) 251 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:39:06.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:04.853057+0000 mon.smithi038 (mon.0) 252 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:39:06.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:04.854469+0000 mon.smithi038 (mon.0) 253 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:39:06.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:04.855839+0000 mon.smithi038 (mon.0) 254 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:39:06.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:04.857270+0000 mon.smithi038 (mon.0) 255 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-15T18:39:06.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:04.857438+0000 mgr.smithi038.khesbl (mgr.14664) 161 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-15T18:39:06.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:39:04.858204+0000 mgr.smithi038.khesbl (mgr.14664) 162 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-02-15T18:39:06.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:04.965256+0000 mgr.smithi038.khesbl (mgr.14664) 163 : cluster [DBG] pgmap v112: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 70/342 objects degraded (20.468%) 2024-02-15T18:39:07.108 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:05.851063+0000 mon.smithi038 (mon.0) 256 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:active} 2 up:standby 2024-02-15T18:39:07.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:05.851063+0000 mon.smithi038 (mon.0) 256 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:active} 2 up:standby 2024-02-15T18:39:08.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:06.965819+0000 mgr.smithi038.khesbl (mgr.14664) 164 : cluster [DBG] pgmap v113: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 70/342 objects degraded (20.468%) 2024-02-15T18:39:08.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:07.242598+0000 mon.smithi038 (mon.0) 257 : cluster [WRN] Health check update: Degraded data redundancy: 70/342 objects degraded (20.468%), 18 pgs degraded (PG_DEGRADED) 2024-02-15T18:39:08.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:06.965819+0000 mgr.smithi038.khesbl (mgr.14664) 164 : cluster [DBG] pgmap v113: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 70/342 objects degraded (20.468%) 2024-02-15T18:39:08.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:07.242598+0000 mon.smithi038 (mon.0) 257 : cluster [WRN] Health check update: Degraded data redundancy: 70/342 objects degraded (20.468%), 18 pgs degraded (PG_DEGRADED) 2024-02-15T18:39:09.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:08.966355+0000 mgr.smithi038.khesbl (mgr.14664) 165 : cluster [DBG] pgmap v114: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 70/342 objects degraded (20.468%) 2024-02-15T18:39:09.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:08.966355+0000 mgr.smithi038.khesbl (mgr.14664) 165 : cluster [DBG] pgmap v114: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 293 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 70/342 objects degraded (20.468%) 2024-02-15T18:39:09.856 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:39:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-3[62228]: debug 2024-02-15T18:39:09.525+0000 7f533b62f380 -1 osd.3 61 log_to_monitors {default=true} 2024-02-15T18:39:10.356 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:39:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-3[62228]: debug 2024-02-15T18:39:10.065+0000 7f532c315700 -1 osd.3 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-15T18:39:10.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:09.530008+0000 mon.smithi121 (mon.1) 2 : audit [INF] from='osd.3 [v2:172.21.15.121:6800/1455526151,v1:172.21.15.121:6801/1455526151]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-15T18:39:10.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:09.530436+0000 mon.smithi038 (mon.0) 258 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-15T18:39:10.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:09.530008+0000 mon.smithi121 (mon.1) 2 : audit [INF] from='osd.3 [v2:172.21.15.121:6800/1455526151,v1:172.21.15.121:6801/1455526151]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-15T18:39:10.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:09.530436+0000 mon.smithi038 (mon.0) 258 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-15T18:39:11.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:10.057705+0000 mon.smithi038 (mon.0) 259 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-15T18:39:11.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:10.057798+0000 mon.smithi038 (mon.0) 260 : cluster [DBG] osdmap e64: 6 total, 5 up, 6 in 2024-02-15T18:39:11.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:10.058377+0000 mon.smithi121 (mon.1) 3 : audit [INF] from='osd.3 [v2:172.21.15.121:6800/1455526151,v1:172.21.15.121:6801/1455526151]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:39:11.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:10.059048+0000 mon.smithi038 (mon.0) 261 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:39:11.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:10.966840+0000 mgr.smithi038.khesbl (mgr.14664) 166 : cluster [DBG] pgmap v116: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 70/342 objects degraded (20.468%) 2024-02-15T18:39:11.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:10.057705+0000 mon.smithi038 (mon.0) 259 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-15T18:39:11.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:10.057798+0000 mon.smithi038 (mon.0) 260 : cluster [DBG] osdmap e64: 6 total, 5 up, 6 in 2024-02-15T18:39:11.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:10.058377+0000 mon.smithi121 (mon.1) 3 : audit [INF] from='osd.3 [v2:172.21.15.121:6800/1455526151,v1:172.21.15.121:6801/1455526151]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:39:11.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:10.059048+0000 mon.smithi038 (mon.0) 261 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:39:11.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:10.966840+0000 mgr.smithi038.khesbl (mgr.14664) 166 : cluster [DBG] pgmap v116: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 70/342 objects degraded (20.468%) 2024-02-15T18:39:12.306 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:11.054622+0000 mon.smithi038 (mon.0) 262 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:39:12.306 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:11.065469+0000 mon.smithi038 (mon.0) 263 : cluster [INF] osd.3 [v2:172.21.15.121:6800/1455526151,v1:172.21.15.121:6801/1455526151] boot 2024-02-15T18:39:12.306 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:11.065550+0000 mon.smithi038 (mon.0) 264 : cluster [DBG] osdmap e65: 6 total, 6 up, 6 in 2024-02-15T18:39:12.306 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:11.065899+0000 mon.smithi038 (mon.0) 265 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:39:12.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:11.054622+0000 mon.smithi038 (mon.0) 262 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:39:12.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:11.065469+0000 mon.smithi038 (mon.0) 263 : cluster [INF] osd.3 [v2:172.21.15.121:6800/1455526151,v1:172.21.15.121:6801/1455526151] boot 2024-02-15T18:39:12.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:11.065550+0000 mon.smithi038 (mon.0) 264 : cluster [DBG] osdmap e65: 6 total, 6 up, 6 in 2024-02-15T18:39:12.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:11.065899+0000 mon.smithi038 (mon.0) 265 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:39:13.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:12.072645+0000 mon.smithi038 (mon.0) 266 : cluster [DBG] osdmap e66: 6 total, 6 up, 6 in 2024-02-15T18:39:13.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:12.967299+0000 mgr.smithi038.khesbl (mgr.14664) 167 : cluster [DBG] pgmap v119: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 70/342 objects degraded (20.468%) 2024-02-15T18:39:13.500 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:12.072645+0000 mon.smithi038 (mon.0) 266 : cluster [DBG] osdmap e66: 6 total, 6 up, 6 in 2024-02-15T18:39:13.500 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:12.967299+0000 mgr.smithi038.khesbl (mgr.14664) 167 : cluster [DBG] pgmap v119: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 70/342 objects degraded (20.468%) 2024-02-15T18:39:15.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:14.967958+0000 mgr.smithi038.khesbl (mgr.14664) 168 : cluster [DBG] pgmap v120: 65 pgs: 9 active+undersized+degraded, 7 active+undersized, 49 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 42/342 objects degraded (12.281%) 2024-02-15T18:39:15.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:15.018816+0000 mon.smithi038 (mon.0) 267 : cluster [WRN] Health check update: Degraded data redundancy: 42/342 objects degraded (12.281%), 9 pgs degraded (PG_DEGRADED) 2024-02-15T18:39:15.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:14.967958+0000 mgr.smithi038.khesbl (mgr.14664) 168 : cluster [DBG] pgmap v120: 65 pgs: 9 active+undersized+degraded, 7 active+undersized, 49 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 42/342 objects degraded (12.281%) 2024-02-15T18:39:15.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:15.018816+0000 mon.smithi038 (mon.0) 267 : cluster [WRN] Health check update: Degraded data redundancy: 42/342 objects degraded (12.281%), 9 pgs degraded (PG_DEGRADED) 2024-02-15T18:39:17.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:16.968589+0000 mgr.smithi038.khesbl (mgr.14664) 169 : cluster [DBG] pgmap v121: 65 pgs: 65 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:17.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:17.019384+0000 mon.smithi038 (mon.0) 268 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 42/342 objects degraded (12.281%), 9 pgs degraded) 2024-02-15T18:39:17.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:17.019462+0000 mon.smithi038 (mon.0) 269 : cluster [INF] Cluster is now healthy 2024-02-15T18:39:17.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:16.968589+0000 mgr.smithi038.khesbl (mgr.14664) 169 : cluster [DBG] pgmap v121: 65 pgs: 65 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:17.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:17.019384+0000 mon.smithi038 (mon.0) 268 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 42/342 objects degraded (12.281%), 9 pgs degraded) 2024-02-15T18:39:17.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:17.019462+0000 mon.smithi038 (mon.0) 269 : cluster [INF] Cluster is now healthy 2024-02-15T18:39:19.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:18.151315+0000 mon.smithi038 (mon.0) 270 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:19.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:18.969188+0000 mgr.smithi038.khesbl (mgr.14664) 170 : cluster [DBG] pgmap v122: 65 pgs: 65 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:19.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:18.151315+0000 mon.smithi038 (mon.0) 270 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:19.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:18.969188+0000 mgr.smithi038.khesbl (mgr.14664) 170 : cluster [DBG] pgmap v122: 65 pgs: 65 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:20.402 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:20 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:19.867541+0000 mon.smithi038 (mon.0) 271 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-15T18:39:20.402 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:20 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:19.867768+0000 mgr.smithi038.khesbl (mgr.14664) 171 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-15T18:39:20.402 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:20 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:39:19.868862+0000 mgr.smithi038.khesbl (mgr.14664) 172 : cephadm [INF] Upgrade: osd.4 is safe to restart 2024-02-15T18:39:20.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:20 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:19.867541+0000 mon.smithi038 (mon.0) 271 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-15T18:39:20.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:20 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:19.867768+0000 mgr.smithi038.khesbl (mgr.14664) 171 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-15T18:39:20.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:20 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:39:19.868862+0000 mgr.smithi038.khesbl (mgr.14664) 172 : cephadm [INF] Upgrade: osd.4 is safe to restart 2024-02-15T18:39:21.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:20.969738+0000 mgr.smithi038.khesbl (mgr.14664) 173 : cluster [DBG] pgmap v123: 65 pgs: 65 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:21.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:20.969738+0000 mgr.smithi038.khesbl (mgr.14664) 173 : cluster [DBG] pgmap v123: 65 pgs: 65 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:23.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:39:21.679672+0000 mgr.smithi038.khesbl (mgr.14664) 174 : cephadm [INF] Upgrade: Updating osd.4 2024-02-15T18:39:23.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:21.684807+0000 mon.smithi038 (mon.0) 272 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:23.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:21.685809+0000 mon.smithi038 (mon.0) 273 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-15T18:39:23.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:21.686575+0000 mon.smithi038 (mon.0) 274 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:39:23.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:39:21.687801+0000 mgr.smithi038.khesbl (mgr.14664) 175 : cephadm [INF] Deploying daemon osd.4 on smithi121 2024-02-15T18:39:23.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:39:21.679672+0000 mgr.smithi038.khesbl (mgr.14664) 174 : cephadm [INF] Upgrade: Updating osd.4 2024-02-15T18:39:23.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:21.684807+0000 mon.smithi038 (mon.0) 272 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:23.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:21.685809+0000 mon.smithi038 (mon.0) 273 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-15T18:39:23.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:21.686575+0000 mon.smithi038 (mon.0) 274 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:39:23.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:39:21.687801+0000 mgr.smithi038.khesbl (mgr.14664) 175 : cephadm [INF] Deploying daemon osd.4 on smithi121 2024-02-15T18:39:23.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:22.970129+0000 mgr.smithi038.khesbl (mgr.14664) 176 : cluster [DBG] pgmap v124: 65 pgs: 65 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:23.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:23.217229+0000 mon.smithi038 (mon.0) 275 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:24.104 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:22.970129+0000 mgr.smithi038.khesbl (mgr.14664) 176 : cluster [DBG] pgmap v124: 65 pgs: 65 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:24.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:23.217229+0000 mon.smithi038 (mon.0) 275 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:24.707 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:24 smithi121 systemd[1]: Stopping Ceph osd.4 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:39:25.019 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:24 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.4[48784]: debug 2024-02-15T18:39:24.716+0000 7f7921872700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:39:25.019 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:24 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.4[48784]: debug 2024-02-15T18:39:24.716+0000 7f7921872700 -1 osd.4 66 *** Got signal Terminated *** 2024-02-15T18:39:25.020 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:24 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.4[48784]: debug 2024-02-15T18:39:24.716+0000 7f7921872700 -1 osd.4 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:39:25.020 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:24.970732+0000 mgr.smithi038.khesbl (mgr.14664) 177 : cluster [DBG] pgmap v125: 65 pgs: 65 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:25.020 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:24.972650+0000 mon.smithi038 (mon.0) 276 : cluster [DBG] osd.4 reported immediately failed by osd.5 2024-02-15T18:39:25.020 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:24.972709+0000 mon.smithi038 (mon.0) 277 : cluster [INF] osd.4 failed (root=default,host=smithi121) (connection refused reported by osd.5) 2024-02-15T18:39:25.020 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:24.972910+0000 mon.smithi038 (mon.0) 278 : cluster [DBG] osd.4 reported immediately failed by osd.5 2024-02-15T18:39:25.020 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:24.973077+0000 mon.smithi038 (mon.0) 279 : cluster [DBG] osd.4 reported immediately failed by osd.1 2024-02-15T18:39:25.020 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:24.973135+0000 mon.smithi038 (mon.0) 280 : cluster [DBG] osd.4 reported immediately failed by osd.2 2024-02-15T18:39:25.020 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:24.973330+0000 mon.smithi038 (mon.0) 281 : cluster [DBG] osd.4 reported immediately failed by osd.3 2024-02-15T18:39:25.021 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:24.973449+0000 mon.smithi038 (mon.0) 282 : cluster [DBG] osd.4 reported immediately failed by osd.0 2024-02-15T18:39:25.021 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:24.973554+0000 mon.smithi038 (mon.0) 283 : cluster [DBG] osd.4 reported immediately failed by osd.1 2024-02-15T18:39:25.021 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:24.973668+0000 mon.smithi038 (mon.0) 284 : cluster [DBG] osd.4 reported immediately failed by osd.2 2024-02-15T18:39:25.021 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:24.973805+0000 mon.smithi038 (mon.0) 285 : cluster [DBG] osd.4 reported immediately failed by osd.3 2024-02-15T18:39:25.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:24.970732+0000 mgr.smithi038.khesbl (mgr.14664) 177 : cluster [DBG] pgmap v125: 65 pgs: 65 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:25.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:24.972650+0000 mon.smithi038 (mon.0) 276 : cluster [DBG] osd.4 reported immediately failed by osd.5 2024-02-15T18:39:25.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:24.972709+0000 mon.smithi038 (mon.0) 277 : cluster [INF] osd.4 failed (root=default,host=smithi121) (connection refused reported by osd.5) 2024-02-15T18:39:25.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:24.972910+0000 mon.smithi038 (mon.0) 278 : cluster [DBG] osd.4 reported immediately failed by osd.5 2024-02-15T18:39:25.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:24.973077+0000 mon.smithi038 (mon.0) 279 : cluster [DBG] osd.4 reported immediately failed by osd.1 2024-02-15T18:39:25.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:24.973135+0000 mon.smithi038 (mon.0) 280 : cluster [DBG] osd.4 reported immediately failed by osd.2 2024-02-15T18:39:25.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:24.973330+0000 mon.smithi038 (mon.0) 281 : cluster [DBG] osd.4 reported immediately failed by osd.3 2024-02-15T18:39:25.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:24.973449+0000 mon.smithi038 (mon.0) 282 : cluster [DBG] osd.4 reported immediately failed by osd.0 2024-02-15T18:39:25.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:24.973554+0000 mon.smithi038 (mon.0) 283 : cluster [DBG] osd.4 reported immediately failed by osd.1 2024-02-15T18:39:25.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:24.973668+0000 mon.smithi038 (mon.0) 284 : cluster [DBG] osd.4 reported immediately failed by osd.2 2024-02-15T18:39:25.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:24.973805+0000 mon.smithi038 (mon.0) 285 : cluster [DBG] osd.4 reported immediately failed by osd.3 2024-02-15T18:39:25.273 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:24.973907+0000 mon.smithi038 (mon.0) 286 : cluster [DBG] osd.4 reported immediately failed by osd.0 2024-02-15T18:39:25.300 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:24.973907+0000 mon.smithi038 (mon.0) 286 : cluster [DBG] osd.4 reported immediately failed by osd.0 2024-02-15T18:39:25.300 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:25 smithi121 bash[63877]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.4 2024-02-15T18:39:25.605 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:25 smithi121 bash[63932]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.4 2024-02-15T18:39:26.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:25.023447+0000 mon.smithi038 (mon.0) 287 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:39:26.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:25.030537+0000 mon.smithi038 (mon.0) 288 : cluster [DBG] osdmap e67: 6 total, 5 up, 6 in 2024-02-15T18:39:26.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:25.023447+0000 mon.smithi038 (mon.0) 287 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:39:26.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:25.030537+0000 mon.smithi038 (mon.0) 288 : cluster [DBG] osdmap e67: 6 total, 5 up, 6 in 2024-02-15T18:39:27.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:26.033611+0000 mon.smithi038 (mon.0) 289 : cluster [DBG] osdmap e68: 6 total, 5 up, 6 in 2024-02-15T18:39:27.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:26.971270+0000 mgr.smithi038.khesbl (mgr.14664) 178 : cluster [DBG] pgmap v128: 65 pgs: 11 peering, 5 stale+active+clean, 49 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:27.356 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:27 smithi121 systemd[1]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.4.service: Succeeded. 2024-02-15T18:39:27.357 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:27 smithi121 systemd[1]: Stopped Ceph osd.4 for 7e05753e-cc2f-11ee-95ba-87774f69a715. 2024-02-15T18:39:27.357 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:27 smithi121 systemd[1]: Starting Ceph osd.4 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:39:27.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:26.033611+0000 mon.smithi038 (mon.0) 289 : cluster [DBG] osdmap e68: 6 total, 5 up, 6 in 2024-02-15T18:39:27.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:26.971270+0000 mgr.smithi038.khesbl (mgr.14664) 178 : cluster [DBG] pgmap v128: 65 pgs: 11 peering, 5 stale+active+clean, 49 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:28.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:28 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:27.039511+0000 mon.smithi038 (mon.0) 290 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-15T18:39:28.356 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:28 smithi121 bash[64190]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-02-15T18:39:28.357 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:28 smithi121 bash[64190]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-15T18:39:28.357 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:28 smithi121 bash[64190]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-4 --no-mon-config 2024-02-15T18:39:28.357 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:28 smithi121 bash[64190]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-4/block 2024-02-15T18:39:28.357 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:28 smithi121 bash[64190]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-4/block 2024-02-15T18:39:28.357 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:28 smithi121 bash[64190]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-15T18:39:28.357 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:28 smithi121 bash[64190]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-15T18:39:28.357 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:28 smithi121 bash[64190]: --> ceph-volume lvm activate successful for osd ID: 4 2024-02-15T18:39:28.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:27.039511+0000 mon.smithi038 (mon.0) 290 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-15T18:39:29.345 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:28.971897+0000 mgr.smithi038.khesbl (mgr.14664) 179 : cluster [DBG] pgmap v129: 65 pgs: 4 active+undersized+degraded, 11 peering, 3 stale+active+clean, 6 active+undersized, 41 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 11/342 objects degraded (3.216%) 2024-02-15T18:39:29.345 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-4[64474]: debug 2024-02-15T18:39:29.264+0000 7f9ccdc4f700 1 -- 172.21.15.121:0/954764062 <== mon.1 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555f7d23c680 con 0x555f7d203400 2024-02-15T18:39:29.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:28.971897+0000 mgr.smithi038.khesbl (mgr.14664) 179 : cluster [DBG] pgmap v129: 65 pgs: 4 active+undersized+degraded, 11 peering, 3 stale+active+clean, 6 active+undersized, 41 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 11/342 objects degraded (3.216%) 2024-02-15T18:39:29.605 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:29 smithi121 bash[64459]: 0ce6b6abb99eb6596076b5cf79b0dcc4754a04f90cb18d7a2f98f341fc5f67d5 2024-02-15T18:39:29.606 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:29 smithi121 systemd[1]: Started Ceph osd.4 for 7e05753e-cc2f-11ee-95ba-87774f69a715. 2024-02-15T18:39:30.063 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-4[64474]: debug 2024-02-15T18:39:29.788+0000 7f9cd56dd380 -1 Falling back to public interface 2024-02-15T18:39:30.360 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:30 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:29.056127+0000 mon.smithi038 (mon.0) 291 : cluster [WRN] Health check failed: Degraded data redundancy: 11/342 objects degraded (3.216%), 4 pgs degraded (PG_DEGRADED) 2024-02-15T18:39:30.360 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:30 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:29.401967+0000 mon.smithi038 (mon.0) 292 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:30.360 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:30 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:29.402852+0000 mon.smithi038 (mon.0) 293 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:39:30.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:29.056127+0000 mon.smithi038 (mon.0) 291 : cluster [WRN] Health check failed: Degraded data redundancy: 11/342 objects degraded (3.216%), 4 pgs degraded (PG_DEGRADED) 2024-02-15T18:39:30.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:29.401967+0000 mon.smithi038 (mon.0) 292 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:30.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:29.402852+0000 mon.smithi038 (mon.0) 293 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:39:31.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:30.972731+0000 mgr.smithi038.khesbl (mgr.14664) 180 : cluster [DBG] pgmap v130: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 58/342 objects degraded (16.959%) 2024-02-15T18:39:31.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:30.972731+0000 mgr.smithi038.khesbl (mgr.14664) 180 : cluster [DBG] pgmap v130: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 58/342 objects degraded (16.959%) 2024-02-15T18:39:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.279+0000 7f40325ed700 1 -- 172.21.15.38:0/3466010563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f402c0f55c0 msgr2=0x7f402c0f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.279+0000 7f40325ed700 1 --2- 172.21.15.38:0/3466010563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f402c0f55c0 0x7f402c0f59e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f401c004660 tx=0x7f401c009b30).stop 2024-02-15T18:39:32.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.279+0000 7f40325ed700 1 -- 172.21.15.38:0/3466010563 shutdown_connections 2024-02-15T18:39:32.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.279+0000 7f40325ed700 1 --2- 172.21.15.38:0/3466010563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f402c0f6860 0x7f402c0f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:32.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.279+0000 7f40325ed700 1 --2- 172.21.15.38:0/3466010563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f402c0f55c0 0x7f402c0f59e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:32.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.279+0000 7f40325ed700 1 -- 172.21.15.38:0/3466010563 >> 172.21.15.38:0/3466010563 conn(0x7f402c0f0d40 msgr2=0x7f402c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:32.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.279+0000 7f40325ed700 1 -- 172.21.15.38:0/3466010563 shutdown_connections 2024-02-15T18:39:32.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.279+0000 7f40325ed700 1 -- 172.21.15.38:0/3466010563 wait complete. 2024-02-15T18:39:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.280+0000 7f40325ed700 1 Processor -- start 2024-02-15T18:39:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.280+0000 7f40325ed700 1 -- start start 2024-02-15T18:39:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.280+0000 7f40325ed700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f402c0f55c0 0x7f402c0f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.280+0000 7f40325ed700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f402c0f6860 0x7f402c0f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.280+0000 7f40325ed700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c0f8100 con 0x7f402c0f6860 2024-02-15T18:39:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.280+0000 7f40325ed700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c0f8240 con 0x7f402c0f55c0 2024-02-15T18:39:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.280+0000 7f402b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f402c0f6860 0x7f402c0f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.280+0000 7f402bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f402c0f55c0 0x7f402c0f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.280+0000 7f402b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f402c0f6860 0x7f402c0f7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57508/0 (socket says 172.21.15.38:57508) 2024-02-15T18:39:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.280+0000 7f402b7fe700 1 -- 172.21.15.38:0/2910555585 learned_addr learned my addr 172.21.15.38:0/2910555585 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:39:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.281+0000 7f402b7fe700 1 -- 172.21.15.38:0/2910555585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f402c0f55c0 msgr2=0x7f402c0f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.281+0000 7f402b7fe700 1 --2- 172.21.15.38:0/2910555585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f402c0f55c0 0x7f402c0f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.281+0000 7f402b7fe700 1 -- 172.21.15.38:0/2910555585 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c005040 con 0x7f402c0f6860 2024-02-15T18:39:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.281+0000 7f402b7fe700 1 --2- 172.21.15.38:0/2910555585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f402c0f6860 0x7f402c0f7ba0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f402000e7f0 tx=0x7f402000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.281+0000 7f4028ff9700 1 -- 172.21.15.38:0/2910555585 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40200092e0 con 0x7f402c0f6860 2024-02-15T18:39:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.281+0000 7f4028ff9700 1 -- 172.21.15.38:0/2910555585 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f402000f040 con 0x7f402c0f6860 2024-02-15T18:39:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.281+0000 7f40325ed700 1 -- 172.21.15.38:0/2910555585 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f402c0f8520 con 0x7f402c0f6860 2024-02-15T18:39:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.281+0000 7f4028ff9700 1 -- 172.21.15.38:0/2910555585 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4020014850 con 0x7f402c0f6860 2024-02-15T18:39:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.281+0000 7f40325ed700 1 -- 172.21.15.38:0/2910555585 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402c10f020 con 0x7f402c0f6860 2024-02-15T18:39:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.282+0000 7f40325ed700 1 -- 172.21.15.38:0/2910555585 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f402c045ad0 con 0x7f402c0f6860 2024-02-15T18:39:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.283+0000 7f4028ff9700 1 -- 172.21.15.38:0/2910555585 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f402001b030 con 0x7f402c0f6860 2024-02-15T18:39:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.283+0000 7f4028ff9700 1 --2- 172.21.15.38:0/2910555585 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4014060f20 0x7f40140633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.283+0000 7f4028ff9700 1 -- 172.21.15.38:0/2910555585 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f402007bdd0 con 0x7f402c0f6860 2024-02-15T18:39:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.283+0000 7f402bfff700 1 --2- 172.21.15.38:0/2910555585 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4014060f20 0x7f40140633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.283+0000 7f402bfff700 1 --2- 172.21.15.38:0/2910555585 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4014060f20 0x7f40140633e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f401c004d80 tx=0x7f401c016040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:32.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.287+0000 7f4028ff9700 1 -- 172.21.15.38:0/2910555585 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f402004c300 con 0x7f402c0f6860 2024-02-15T18:39:32.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:32 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:31.073316+0000 mon.smithi038 (mon.0) 294 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-15T18:39:32.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.479+0000 7f40325ed700 1 -- 172.21.15.38:0/2910555585 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f402c0faca0 con 0x7f4014060f20 2024-02-15T18:39:32.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.480+0000 7f4028ff9700 1 -- 172.21.15.38:0/2910555585 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f402c0faca0 con 0x7f4014060f20 2024-02-15T18:39:32.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.482+0000 7f40325ed700 1 -- 172.21.15.38:0/2910555585 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4014060f20 msgr2=0x7f40140633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:32.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.482+0000 7f40325ed700 1 --2- 172.21.15.38:0/2910555585 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4014060f20 0x7f40140633e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f401c004d80 tx=0x7f401c016040).stop 2024-02-15T18:39:32.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.482+0000 7f40325ed700 1 -- 172.21.15.38:0/2910555585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f402c0f6860 msgr2=0x7f402c0f7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:32.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.482+0000 7f40325ed700 1 --2- 172.21.15.38:0/2910555585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f402c0f6860 0x7f402c0f7ba0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f402000e7f0 tx=0x7f402000b040).stop 2024-02-15T18:39:32.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.482+0000 7f40325ed700 1 -- 172.21.15.38:0/2910555585 shutdown_connections 2024-02-15T18:39:32.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.482+0000 7f40325ed700 1 --2- 172.21.15.38:0/2910555585 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4014060f20 0x7f40140633e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:32.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.482+0000 7f40325ed700 1 --2- 172.21.15.38:0/2910555585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f402c0f55c0 0x7f402c0f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:32.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.482+0000 7f40325ed700 1 --2- 172.21.15.38:0/2910555585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f402c0f6860 0x7f402c0f7ba0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:32.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.482+0000 7f40325ed700 1 -- 172.21.15.38:0/2910555585 >> 172.21.15.38:0/2910555585 conn(0x7f402c0f0d40 msgr2=0x7f402c05fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:32.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.482+0000 7f40325ed700 1 -- 172.21.15.38:0/2910555585 shutdown_connections 2024-02-15T18:39:32.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.482+0000 7f40325ed700 1 -- 172.21.15.38:0/2910555585 wait complete. 2024-02-15T18:39:32.481 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-02-15T18:39:32.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:32 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:31.073316+0000 mon.smithi038 (mon.0) 294 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-15T18:39:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.584+0000 7fa4370db700 1 -- 172.21.15.38:0/541078013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4300f6860 msgr2=0x7fa4300f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.584+0000 7fa4370db700 1 --2- 172.21.15.38:0/541078013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4300f6860 0x7fa4300f6cc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fa41c004660 tx=0x7fa41c009b30).stop 2024-02-15T18:39:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.584+0000 7fa4370db700 1 -- 172.21.15.38:0/541078013 shutdown_connections 2024-02-15T18:39:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.584+0000 7fa4370db700 1 --2- 172.21.15.38:0/541078013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4300f6860 0x7fa4300f6cc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.584+0000 7fa4370db700 1 --2- 172.21.15.38:0/541078013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4300f55c0 0x7fa4300f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.584+0000 7fa4370db700 1 -- 172.21.15.38:0/541078013 >> 172.21.15.38:0/541078013 conn(0x7fa4300f0d40 msgr2=0x7fa4300f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.584+0000 7fa4370db700 1 -- 172.21.15.38:0/541078013 shutdown_connections 2024-02-15T18:39:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.584+0000 7fa4370db700 1 -- 172.21.15.38:0/541078013 wait complete. 2024-02-15T18:39:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.585+0000 7fa4370db700 1 Processor -- start 2024-02-15T18:39:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.585+0000 7fa4370db700 1 -- start start 2024-02-15T18:39:32.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.585+0000 7fa4370db700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4300f55c0 0x7fa430106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:32.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.585+0000 7fa4370db700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4300f6860 0x7fa4301074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:32.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.585+0000 7fa4370db700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa430107a60 con 0x7fa4300f6860 2024-02-15T18:39:32.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.585+0000 7fa4370db700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa430107ba0 con 0x7fa4300f55c0 2024-02-15T18:39:32.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.585+0000 7fa42ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4300f6860 0x7fa4301074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:32.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.585+0000 7fa42ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4300f6860 0x7fa4301074b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57528/0 (socket says 172.21.15.38:57528) 2024-02-15T18:39:32.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.585+0000 7fa42ffff700 1 -- 172.21.15.38:0/3414301555 learned_addr learned my addr 172.21.15.38:0/3414301555 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:39:32.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.585+0000 7fa434e77700 1 --2- 172.21.15.38:0/3414301555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4300f55c0 0x7fa430106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:32.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.586+0000 7fa42ffff700 1 -- 172.21.15.38:0/3414301555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4300f55c0 msgr2=0x7fa430106fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:32.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.586+0000 7fa42ffff700 1 --2- 172.21.15.38:0/3414301555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4300f55c0 0x7fa430106fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:32.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.586+0000 7fa42ffff700 1 -- 172.21.15.38:0/3414301555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41c005040 con 0x7fa4300f6860 2024-02-15T18:39:32.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.586+0000 7fa434e77700 1 --2- 172.21.15.38:0/3414301555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4300f55c0 0x7fa430106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:39:32.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.586+0000 7fa42ffff700 1 --2- 172.21.15.38:0/3414301555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4300f6860 0x7fa4301074b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fa41c000c00 tx=0x7fa41c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:32.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.586+0000 7fa42d7fa700 1 -- 172.21.15.38:0/3414301555 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa41c019070 con 0x7fa4300f6860 2024-02-15T18:39:32.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.586+0000 7fa42d7fa700 1 -- 172.21.15.38:0/3414301555 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fa41c014420 con 0x7fa4300f6860 2024-02-15T18:39:32.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.586+0000 7fa4370db700 1 -- 172.21.15.38:0/3414301555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4301084e0 con 0x7fa4300f6860 2024-02-15T18:39:32.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.586+0000 7fa42d7fa700 1 -- 172.21.15.38:0/3414301555 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa41c01d600 con 0x7fa4300f6860 2024-02-15T18:39:32.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.586+0000 7fa4370db700 1 -- 172.21.15.38:0/3414301555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa430108a40 con 0x7fa4300f6860 2024-02-15T18:39:32.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.587+0000 7fa4370db700 1 -- 172.21.15.38:0/3414301555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa430045ad0 con 0x7fa4300f6860 2024-02-15T18:39:32.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.588+0000 7fa42d7fa700 1 -- 172.21.15.38:0/3414301555 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fa41c025070 con 0x7fa4300f6860 2024-02-15T18:39:32.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.588+0000 7fa42d7fa700 1 --2- 172.21.15.38:0/3414301555 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fa420060f20 0x7fa4200633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:32.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.588+0000 7fa42d7fa700 1 -- 172.21.15.38:0/3414301555 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fa41c07d030 con 0x7fa4300f6860 2024-02-15T18:39:32.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.588+0000 7fa434e77700 1 --2- 172.21.15.38:0/3414301555 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fa420060f20 0x7fa4200633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:32.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.588+0000 7fa434e77700 1 --2- 172.21.15.38:0/3414301555 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fa420060f20 0x7fa4200633e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fa424009da0 tx=0x7fa42400b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:32.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.592+0000 7fa42d7fa700 1 -- 172.21.15.38:0/3414301555 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa41c050f60 con 0x7fa4300f6860 2024-02-15T18:39:32.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.787+0000 7fa4370db700 1 -- 172.21.15.38:0/3414301555 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4300faca0 con 0x7fa420060f20 2024-02-15T18:39:32.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.788+0000 7fa42d7fa700 1 -- 172.21.15.38:0/3414301555 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fa4300faca0 con 0x7fa420060f20 2024-02-15T18:39:32.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.790+0000 7fa4370db700 1 -- 172.21.15.38:0/3414301555 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fa420060f20 msgr2=0x7fa4200633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:32.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.790+0000 7fa4370db700 1 --2- 172.21.15.38:0/3414301555 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fa420060f20 0x7fa4200633e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fa424009da0 tx=0x7fa42400b040).stop 2024-02-15T18:39:32.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.790+0000 7fa4370db700 1 -- 172.21.15.38:0/3414301555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4300f6860 msgr2=0x7fa4301074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:32.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.790+0000 7fa4370db700 1 --2- 172.21.15.38:0/3414301555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4300f6860 0x7fa4301074b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fa41c000c00 tx=0x7fa41c00b040).stop 2024-02-15T18:39:32.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.790+0000 7fa4370db700 1 -- 172.21.15.38:0/3414301555 shutdown_connections 2024-02-15T18:39:32.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.790+0000 7fa4370db700 1 --2- 172.21.15.38:0/3414301555 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fa420060f20 0x7fa4200633e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:32.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.790+0000 7fa4370db700 1 --2- 172.21.15.38:0/3414301555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4300f55c0 0x7fa430106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:32.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.790+0000 7fa4370db700 1 --2- 172.21.15.38:0/3414301555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4300f6860 0x7fa4301074b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:32.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.790+0000 7fa4370db700 1 -- 172.21.15.38:0/3414301555 >> 172.21.15.38:0/3414301555 conn(0x7fa4300f0d40 msgr2=0x7fa43005fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:32.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.790+0000 7fa4370db700 1 -- 172.21.15.38:0/3414301555 shutdown_connections 2024-02-15T18:39:32.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.790+0000 7fa4370db700 1 -- 172.21.15.38:0/3414301555 wait complete. 2024-02-15T18:39:32.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.891+0000 7f3cfb3aa700 1 -- 172.21.15.38:0/192189282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cf40f55c0 msgr2=0x7f3cf40f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:32.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.891+0000 7f3cfb3aa700 1 --2- 172.21.15.38:0/192189282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cf40f55c0 0x7f3cf40f59e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f3cf0004660 tx=0x7f3cf0009b30).stop 2024-02-15T18:39:32.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.891+0000 7f3cfb3aa700 1 -- 172.21.15.38:0/192189282 shutdown_connections 2024-02-15T18:39:32.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.891+0000 7f3cfb3aa700 1 --2- 172.21.15.38:0/192189282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf40f6860 0x7f3cf40f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:32.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.891+0000 7f3cfb3aa700 1 --2- 172.21.15.38:0/192189282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cf40f55c0 0x7f3cf40f59e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:32.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.891+0000 7f3cfb3aa700 1 -- 172.21.15.38:0/192189282 >> 172.21.15.38:0/192189282 conn(0x7f3cf40f0d40 msgr2=0x7f3cf40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:32.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.891+0000 7f3cfb3aa700 1 -- 172.21.15.38:0/192189282 shutdown_connections 2024-02-15T18:39:32.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.891+0000 7f3cfb3aa700 1 -- 172.21.15.38:0/192189282 wait complete. 2024-02-15T18:39:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.892+0000 7f3cfb3aa700 1 Processor -- start 2024-02-15T18:39:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.892+0000 7f3cfb3aa700 1 -- start start 2024-02-15T18:39:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.892+0000 7f3cfb3aa700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf40f55c0 0x7f3cf4106ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.892+0000 7f3cfb3aa700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cf40f6860 0x7f3cf41074e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.892+0000 7f3cfb3aa700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf4107a90 con 0x7f3cf40f6860 2024-02-15T18:39:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.892+0000 7f3cfb3aa700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf4107bd0 con 0x7f3cf40f55c0 2024-02-15T18:39:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.892+0000 7f3cf8945700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cf40f6860 0x7f3cf41074e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.892+0000 7f3cf8945700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cf40f6860 0x7f3cf41074e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57556/0 (socket says 172.21.15.38:57556) 2024-02-15T18:39:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.892+0000 7f3cf8945700 1 -- 172.21.15.38:0/1175949604 learned_addr learned my addr 172.21.15.38:0/1175949604 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:39:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.892+0000 7f3cf9146700 1 --2- 172.21.15.38:0/1175949604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf40f55c0 0x7f3cf4106ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.892+0000 7f3cf8945700 1 -- 172.21.15.38:0/1175949604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf40f55c0 msgr2=0x7f3cf4106ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.892+0000 7f3cf8945700 1 --2- 172.21.15.38:0/1175949604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf40f55c0 0x7f3cf4106ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.892+0000 7f3cf8945700 1 -- 172.21.15.38:0/1175949604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf0005040 con 0x7f3cf40f6860 2024-02-15T18:39:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.892+0000 7f3cf8945700 1 --2- 172.21.15.38:0/1175949604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cf40f6860 0x7f3cf41074e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f3ce8002700 tx=0x7f3ce8009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.893+0000 7f3ce5ffb700 1 -- 172.21.15.38:0/1175949604 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ce8009320 con 0x7f3cf40f6860 2024-02-15T18:39:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.893+0000 7f3cfb3aa700 1 -- 172.21.15.38:0/1175949604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf4108570 con 0x7f3cf40f6860 2024-02-15T18:39:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.893+0000 7f3ce5ffb700 1 -- 172.21.15.38:0/1175949604 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f3ce800f040 con 0x7f3cf40f6860 2024-02-15T18:39:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.893+0000 7f3ce5ffb700 1 -- 172.21.15.38:0/1175949604 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ce80149a0 con 0x7f3cf40f6860 2024-02-15T18:39:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.893+0000 7f3cfb3aa700 1 -- 172.21.15.38:0/1175949604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf4108ab0 con 0x7f3cf40f6860 2024-02-15T18:39:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.894+0000 7f3cfb3aa700 1 -- 172.21.15.38:0/1175949604 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf4045ad0 con 0x7f3cf40f6860 2024-02-15T18:39:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.894+0000 7f3ce5ffb700 1 -- 172.21.15.38:0/1175949604 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f3ce80077b0 con 0x7f3cf40f6860 2024-02-15T18:39:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.895+0000 7f3ce5ffb700 1 --2- 172.21.15.38:0/1175949604 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3ce0060f20 0x7f3ce00633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.895+0000 7f3ce5ffb700 1 -- 172.21.15.38:0/1175949604 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f3ce807c3a0 con 0x7f3cf40f6860 2024-02-15T18:39:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.895+0000 7f3cf9146700 1 --2- 172.21.15.38:0/1175949604 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3ce0060f20 0x7f3ce00633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.895+0000 7f3cf9146700 1 --2- 172.21.15.38:0/1175949604 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3ce0060f20 0x7f3ce00633e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f3cf0004630 tx=0x7f3cf0009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:32.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:32.899+0000 7f3ce5ffb700 1 -- 172.21.15.38:0/1175949604 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3ce8050250 con 0x7f3cf40f6860 2024-02-15T18:39:33.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.088+0000 7f3cfb3aa700 1 -- 172.21.15.38:0/1175949604 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cf40faca0 con 0x7f3ce0060f20 2024-02-15T18:39:33.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.099+0000 7f3ce5ffb700 1 -- 172.21.15.38:0/1175949604 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f3cf40faca0 con 0x7f3ce0060f20 2024-02-15T18:39:33.088 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:39:33.088 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (10m) 56s ago 12m 19.3M - 0.20.0 0881eb8f169f c2034c318e7c 2024-02-15T18:39:33.089 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (2m) 56s ago 12m 6635k - 16.2.14-521-g0e714d9a 1fbc92c859dc c2fee0824d4b 2024-02-15T18:39:33.089 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi121 smithi121 running (2m) 0s ago 11m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 05c1a5501a35 2024-02-15T18:39:33.089 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (10m) 56s ago 12m 33.4M - 6.7.4 557c83e11646 2dcccfbfbd2a 2024-02-15T18:39:33.089 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.hxoggl smithi038 running (6m) 56s ago 6m 14.8M - 16.2.4 8d91d370c2b8 be2533754b9c 2024-02-15T18:39:33.089 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.qgkzgn smithi038 running (6m) 56s ago 6m 14.4M - 16.2.4 8d91d370c2b8 dd3ffafba9bd 2024-02-15T18:39:33.089 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.mvumlh smithi121 running (6m) 0s ago 6m 116M - 16.2.4 8d91d370c2b8 88b7d876a39b 2024-02-15T18:39:33.089 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.ovuzki smithi121 running (6m) 0s ago 6m 124M - 16.2.4 8d91d370c2b8 21cacf0012f9 2024-02-15T18:39:33.089 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.khesbl smithi038 *:8443,9283 running (3m) 56s ago 13m 433M - 16.2.14-521-g0e714d9a 1fbc92c859dc e642f3c2cfc9 2024-02-15T18:39:33.089 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi121.ckiryi smithi121 *:8443,9283 running (3m) 0s ago 11m 387M - 16.2.14-521-g0e714d9a 1fbc92c859dc 82515b025594 2024-02-15T18:39:33.089 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (3m) 56s ago 13m 40.8M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc d3e8317a30ca 2024-02-15T18:39:33.089 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi121 smithi121 running (2m) 0s ago 11m 39.9M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 42e1dda1a39d 2024-02-15T18:39:33.089 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (11m) 56s ago 11m 17.1M - 0.18.1 e5a616e4b9cf 037f60e38158 2024-02-15T18:39:33.089 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi121 smithi121 *:9100 running (10m) 0s ago 10m 17.2M - 0.18.1 e5a616e4b9cf a867bff85e18 2024-02-15T18:39:33.090 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (2m) 56s ago 10m 191M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 623ac7ecbb0a 2024-02-15T18:39:33.090 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (95s) 56s ago 9m 217M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 0f9e8e1f255a 2024-02-15T18:39:33.090 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (63s) 56s ago 9m 13.4M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 37171264b5be 2024-02-15T18:39:33.090 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi121 running (32s) 0s ago 9m 252M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc d505a58b375c 2024-02-15T18:39:33.090 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi121 running (4s) 0s ago 8m 15.4M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 0ce6b6abb99e 2024-02-15T18:39:33.090 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi121 running (8m) 0s ago 8m 371M 4096M 16.2.4 8d91d370c2b8 cdc8c669c4db 2024-02-15T18:39:33.090 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (3m) 56s ago 11m 65.9M - 2.18.1 de242295e225 67e33f9de4e6 2024-02-15T18:39:33.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.101+0000 7f3cfb3aa700 1 -- 172.21.15.38:0/1175949604 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3ce0060f20 msgr2=0x7f3ce00633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:33.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.101+0000 7f3cfb3aa700 1 --2- 172.21.15.38:0/1175949604 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3ce0060f20 0x7f3ce00633e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f3cf0004630 tx=0x7f3cf0009b30).stop 2024-02-15T18:39:33.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.101+0000 7f3cfb3aa700 1 -- 172.21.15.38:0/1175949604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cf40f6860 msgr2=0x7f3cf41074e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:33.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.101+0000 7f3cfb3aa700 1 --2- 172.21.15.38:0/1175949604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cf40f6860 0x7f3cf41074e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f3ce8002700 tx=0x7f3ce8009e00).stop 2024-02-15T18:39:33.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.101+0000 7f3cfb3aa700 1 -- 172.21.15.38:0/1175949604 shutdown_connections 2024-02-15T18:39:33.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.101+0000 7f3cfb3aa700 1 --2- 172.21.15.38:0/1175949604 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3ce0060f20 0x7f3ce00633e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:33.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.101+0000 7f3cfb3aa700 1 --2- 172.21.15.38:0/1175949604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf40f55c0 0x7f3cf4106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:33.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.101+0000 7f3cfb3aa700 1 --2- 172.21.15.38:0/1175949604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cf40f6860 0x7f3cf41074e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:33.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.101+0000 7f3cfb3aa700 1 -- 172.21.15.38:0/1175949604 >> 172.21.15.38:0/1175949604 conn(0x7f3cf40f0d40 msgr2=0x7f3cf405fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:33.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.101+0000 7f3cfb3aa700 1 -- 172.21.15.38:0/1175949604 shutdown_connections 2024-02-15T18:39:33.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.101+0000 7f3cfb3aa700 1 -- 172.21.15.38:0/1175949604 wait complete. 2024-02-15T18:39:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.202+0000 7fe339f77700 1 -- 172.21.15.38:0/1278531013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3340f66d0 msgr2=0x7fe3340f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.202+0000 7fe339f77700 1 --2- 172.21.15.38:0/1278531013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3340f66d0 0x7fe3340f6b30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fe320004660 tx=0x7fe320009b30).stop 2024-02-15T18:39:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.202+0000 7fe339f77700 1 -- 172.21.15.38:0/1278531013 shutdown_connections 2024-02-15T18:39:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.202+0000 7fe339f77700 1 --2- 172.21.15.38:0/1278531013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3340f66d0 0x7fe3340f6b30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.202+0000 7fe339f77700 1 --2- 172.21.15.38:0/1278531013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3340f5430 0x7fe3340f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.202+0000 7fe339f77700 1 -- 172.21.15.38:0/1278531013 >> 172.21.15.38:0/1278531013 conn(0x7fe3340f0b90 msgr2=0x7fe3340f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.202+0000 7fe339f77700 1 -- 172.21.15.38:0/1278531013 shutdown_connections 2024-02-15T18:39:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.203+0000 7fe339f77700 1 -- 172.21.15.38:0/1278531013 wait complete. 2024-02-15T18:39:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.203+0000 7fe339f77700 1 Processor -- start 2024-02-15T18:39:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.203+0000 7fe339f77700 1 -- start start 2024-02-15T18:39:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.203+0000 7fe339f77700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3340f5430 0x7fe3340f9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.203+0000 7fe339f77700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3340f66d0 0x7fe3340f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.203+0000 7fe339f77700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3340f7f70 con 0x7fe3340f5430 2024-02-15T18:39:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.203+0000 7fe339f77700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3340f80b0 con 0x7fe3340f66d0 2024-02-15T18:39:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.203+0000 7fe338f75700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3340f5430 0x7fe3340f9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.204+0000 7fe338f75700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3340f5430 0x7fe3340f9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57578/0 (socket says 172.21.15.38:57578) 2024-02-15T18:39:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.204+0000 7fe338f75700 1 -- 172.21.15.38:0/3227660569 learned_addr learned my addr 172.21.15.38:0/3227660569 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:39:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.204+0000 7fe333fff700 1 --2- 172.21.15.38:0/3227660569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3340f66d0 0x7fe3340f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.204+0000 7fe338f75700 1 -- 172.21.15.38:0/3227660569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3340f66d0 msgr2=0x7fe3340f7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:33.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.204+0000 7fe338f75700 1 --2- 172.21.15.38:0/3227660569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3340f66d0 0x7fe3340f7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:33.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.204+0000 7fe338f75700 1 -- 172.21.15.38:0/3227660569 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe320005040 con 0x7fe3340f5430 2024-02-15T18:39:33.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.204+0000 7fe338f75700 1 --2- 172.21.15.38:0/3227660569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3340f5430 0x7fe3340f9590 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fe328002700 tx=0x7fe328009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:33.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.204+0000 7fe3317fa700 1 -- 172.21.15.38:0/3227660569 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3280079c0 con 0x7fe3340f5430 2024-02-15T18:39:33.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.204+0000 7fe339f77700 1 -- 172.21.15.38:0/3227660569 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3340f8390 con 0x7fe3340f5430 2024-02-15T18:39:33.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.204+0000 7fe3317fa700 1 -- 172.21.15.38:0/3227660569 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fe32800f040 con 0x7fe3340f5430 2024-02-15T18:39:33.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.204+0000 7fe3317fa700 1 -- 172.21.15.38:0/3227660569 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe328014940 con 0x7fe3340f5430 2024-02-15T18:39:33.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.204+0000 7fe339f77700 1 -- 172.21.15.38:0/3227660569 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe33410edf0 con 0x7fe3340f5430 2024-02-15T18:39:33.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.205+0000 7fe339f77700 1 -- 172.21.15.38:0/3227660569 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe334045ad0 con 0x7fe3340f5430 2024-02-15T18:39:33.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.206+0000 7fe3317fa700 1 -- 172.21.15.38:0/3227660569 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fe32801b060 con 0x7fe3340f5430 2024-02-15T18:39:33.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.206+0000 7fe3317fa700 1 --2- 172.21.15.38:0/3227660569 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe324060f70 0x7fe324063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:33.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.206+0000 7fe3317fa700 1 -- 172.21.15.38:0/3227660569 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fe32807bf40 con 0x7fe3340f5430 2024-02-15T18:39:33.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.206+0000 7fe333fff700 1 --2- 172.21.15.38:0/3227660569 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe324060f70 0x7fe324063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:33.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.206+0000 7fe333fff700 1 --2- 172.21.15.38:0/3227660569 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe324060f70 0x7fe324063430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fe320004630 tx=0x7fe320009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:33.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.210+0000 7fe3317fa700 1 -- 172.21.15.38:0/3227660569 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe32804c3c0 con 0x7fe3340f5430 2024-02-15T18:39:33.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.454+0000 7fe339f77700 1 -- 172.21.15.38:0/3227660569 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe33410f570 con 0x7fe3340f5430 2024-02-15T18:39:33.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.455+0000 7fe3317fa700 1 -- 172.21.15.38:0/3227660569 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+820 (secure 0 0 0) 0x7fe32804f9e0 con 0x7fe3340f5430 2024-02-15T18:39:33.444 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:39:33.444 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-02-15T18:39:33.444 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:39:33.444 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:39:33.444 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-02-15T18:39:33.445 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:39:33.445 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:39:33.445 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-02-15T18:39:33.445 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 4, 2024-02-15T18:39:33.445 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 1 2024-02-15T18:39:33.445 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:39:33.445 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-02-15T18:39:33.445 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:39:33.445 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:39:33.445 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-02-15T18:39:33.445 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 8, 2024-02-15T18:39:33.445 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 5 2024-02-15T18:39:33.445 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-02-15T18:39:33.445 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:39:33.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.457+0000 7fe339f77700 1 -- 172.21.15.38:0/3227660569 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe324060f70 msgr2=0x7fe324063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:33.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.457+0000 7fe339f77700 1 --2- 172.21.15.38:0/3227660569 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe324060f70 0x7fe324063430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fe320004630 tx=0x7fe320009b30).stop 2024-02-15T18:39:33.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.457+0000 7fe339f77700 1 -- 172.21.15.38:0/3227660569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3340f5430 msgr2=0x7fe3340f9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:33.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.457+0000 7fe339f77700 1 --2- 172.21.15.38:0/3227660569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3340f5430 0x7fe3340f9590 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fe328002700 tx=0x7fe328009d80).stop 2024-02-15T18:39:33.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.457+0000 7fe339f77700 1 -- 172.21.15.38:0/3227660569 shutdown_connections 2024-02-15T18:39:33.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.457+0000 7fe339f77700 1 --2- 172.21.15.38:0/3227660569 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe324060f70 0x7fe324063430 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:33.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.457+0000 7fe339f77700 1 --2- 172.21.15.38:0/3227660569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3340f5430 0x7fe3340f9590 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:33.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.457+0000 7fe339f77700 1 --2- 172.21.15.38:0/3227660569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3340f66d0 0x7fe3340f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:33.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.457+0000 7fe339f77700 1 -- 172.21.15.38:0/3227660569 >> 172.21.15.38:0/3227660569 conn(0x7fe3340f0b90 msgr2=0x7fe33405fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:33.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.457+0000 7fe339f77700 1 -- 172.21.15.38:0/3227660569 shutdown_connections 2024-02-15T18:39:33.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.457+0000 7fe339f77700 1 -- 172.21.15.38:0/3227660569 wait complete. 2024-02-15T18:39:33.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.561+0000 7fdeebf9d700 1 -- 172.21.15.38:0/280270795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee40f6860 msgr2=0x7fdee40f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:33.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.561+0000 7fdeebf9d700 1 --2- 172.21.15.38:0/280270795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee40f6860 0x7fdee40f6cc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fded4004660 tx=0x7fded4009b30).stop 2024-02-15T18:39:33.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.561+0000 7fdeebf9d700 1 -- 172.21.15.38:0/280270795 shutdown_connections 2024-02-15T18:39:33.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.561+0000 7fdeebf9d700 1 --2- 172.21.15.38:0/280270795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee40f6860 0x7fdee40f6cc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:33.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.561+0000 7fdeebf9d700 1 --2- 172.21.15.38:0/280270795 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee40f55c0 0x7fdee40f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:33.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.561+0000 7fdeebf9d700 1 -- 172.21.15.38:0/280270795 >> 172.21.15.38:0/280270795 conn(0x7fdee40f0d40 msgr2=0x7fdee40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:33.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.561+0000 7fdeebf9d700 1 -- 172.21.15.38:0/280270795 shutdown_connections 2024-02-15T18:39:33.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.561+0000 7fdeebf9d700 1 -- 172.21.15.38:0/280270795 wait complete. 2024-02-15T18:39:33.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.561+0000 7fdeebf9d700 1 Processor -- start 2024-02-15T18:39:33.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.561+0000 7fdeebf9d700 1 -- start start 2024-02-15T18:39:33.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.561+0000 7fdeebf9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee40f55c0 0x7fdee40f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:33.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.561+0000 7fdeebf9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee40f6860 0x7fdee40f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:33.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.562+0000 7fdeebf9d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee40f8100 con 0x7fdee40f55c0 2024-02-15T18:39:33.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.562+0000 7fdeebf9d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee40f8240 con 0x7fdee40f6860 2024-02-15T18:39:33.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.562+0000 7fdee9d39700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee40f55c0 0x7fdee40f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:33.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.562+0000 7fdee9d39700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee40f55c0 0x7fdee40f9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50830/0 (socket says 172.21.15.38:50830) 2024-02-15T18:39:33.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.562+0000 7fdee9d39700 1 -- 172.21.15.38:0/147308485 learned_addr learned my addr 172.21.15.38:0/147308485 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:39:33.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.562+0000 7fdee9538700 1 --2- 172.21.15.38:0/147308485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee40f6860 0x7fdee40f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:33.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.562+0000 7fdee9d39700 1 -- 172.21.15.38:0/147308485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee40f6860 msgr2=0x7fdee40f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:33.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.562+0000 7fdee9d39700 1 --2- 172.21.15.38:0/147308485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee40f6860 0x7fdee40f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:33.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.562+0000 7fdee9d39700 1 -- 172.21.15.38:0/147308485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded4005040 con 0x7fdee40f55c0 2024-02-15T18:39:33.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.562+0000 7fdee9d39700 1 --2- 172.21.15.38:0/147308485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee40f55c0 0x7fdee40f9720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fdee000e7c0 tx=0x7fdee000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:33.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.562+0000 7fdeda7fc700 1 -- 172.21.15.38:0/147308485 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdee00092e0 con 0x7fdee40f55c0 2024-02-15T18:39:33.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.562+0000 7fdeda7fc700 1 -- 172.21.15.38:0/147308485 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fdee000f040 con 0x7fdee40f55c0 2024-02-15T18:39:33.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.562+0000 7fdeebf9d700 1 -- 172.21.15.38:0/147308485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee40f8520 con 0x7fdee40f55c0 2024-02-15T18:39:33.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.562+0000 7fdeda7fc700 1 -- 172.21.15.38:0/147308485 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdee0014920 con 0x7fdee40f55c0 2024-02-15T18:39:33.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.562+0000 7fdeebf9d700 1 -- 172.21.15.38:0/147308485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee410eff0 con 0x7fdee40f55c0 2024-02-15T18:39:33.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.563+0000 7fdeebf9d700 1 -- 172.21.15.38:0/147308485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee4045ad0 con 0x7fdee40f55c0 2024-02-15T18:39:33.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.564+0000 7fdeda7fc700 1 -- 172.21.15.38:0/147308485 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fdee001b030 con 0x7fdee40f55c0 2024-02-15T18:39:33.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.564+0000 7fdeda7fc700 1 --2- 172.21.15.38:0/147308485 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fded0060f20 0x7fded00633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:33.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.564+0000 7fdeda7fc700 1 -- 172.21.15.38:0/147308485 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fdee007c050 con 0x7fdee40f55c0 2024-02-15T18:39:33.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.564+0000 7fdee9538700 1 --2- 172.21.15.38:0/147308485 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fded0060f20 0x7fded00633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:33.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.565+0000 7fdee9538700 1 --2- 172.21.15.38:0/147308485 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fded0060f20 0x7fded00633e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fded4004630 tx=0x7fded4016040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:33.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.568+0000 7fdeda7fc700 1 -- 172.21.15.38:0/147308485 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fdee004c4d0 con 0x7fdee40f55c0 2024-02-15T18:39:33.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.789+0000 7fdeebf9d700 1 -- 172.21.15.38:0/147308485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdee410f740 con 0x7fdee40f55c0 2024-02-15T18:39:33.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.790+0000 7fdeda7fc700 1 -- 172.21.15.38:0/147308485 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1790 (secure 0 0 0) 0x7fdee0019070 con 0x7fdee40f55c0 2024-02-15T18:39:33.779 INFO:teuthology.orchestra.run.smithi038.stdout:e14 2024-02-15T18:39:33.779 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:39:33.779 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:39:33.779 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-02-15T18:39:33.780 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:39:33.780 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:39:33.780 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-02-15T18:39:33.780 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 14 2024-02-15T18:39:33.780 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 2024-02-15T18:39:33.780 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:39:33.780 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-02-15T18:39:05.842870+0000 2024-02-15T18:39:33.780 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-02-15T18:39:33.780 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-02-15T18:39:33.780 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-02-15T18:39:33.780 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-02-15T18:39:33.780 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-02-15T18:39:33.781 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-02-15T18:39:33.781 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-02-15T18:39:33.781 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-02-15T18:39:33.781 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:39:33.781 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-02-15T18:39:33.781 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-02-15T18:39:33.781 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=24261,1=24275} 2024-02-15T18:39:33.781 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-02-15T18:39:33.781 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-02-15T18:39:33.781 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-02-15T18:39:33.781 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-02-15T18:39:33.781 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-02-15T18:39:33.781 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-02-15T18:39:33.782 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-02-15T18:39:33.782 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-02-15T18:39:33.782 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.mvumlh{0:24261} state up:active seq 3 join_fscid=1 addr [v2:172.21.15.121:6824/1422707571,v1:172.21.15.121:6825/1422707571] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:39:33.782 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.ovuzki{1:24275} state up:active seq 2 export targets 0 join_fscid=1 addr [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:39:33.782 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:39:33.782 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:39:33.782 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-02-15T18:39:33.782 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:39:33.782 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.hxoggl{-1:14446} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.38:6826/3822880235,v1:172.21.15.38:6827/3822880235] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:39:33.782 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.qgkzgn{-1:14456} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.38:6828/703030231,v1:172.21.15.38:6829/703030231] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:39:33.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.792+0000 7fdeebf9d700 1 -- 172.21.15.38:0/147308485 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fded0060f20 msgr2=0x7fded00633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:33.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.792+0000 7fdeebf9d700 1 --2- 172.21.15.38:0/147308485 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fded0060f20 0x7fded00633e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fded4004630 tx=0x7fded4016040).stop 2024-02-15T18:39:33.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.792+0000 7fdeebf9d700 1 -- 172.21.15.38:0/147308485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee40f55c0 msgr2=0x7fdee40f9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:33.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.792+0000 7fdeebf9d700 1 --2- 172.21.15.38:0/147308485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee40f55c0 0x7fdee40f9720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fdee000e7c0 tx=0x7fdee000b040).stop 2024-02-15T18:39:33.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.792+0000 7fdeebf9d700 1 -- 172.21.15.38:0/147308485 shutdown_connections 2024-02-15T18:39:33.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.792+0000 7fdeebf9d700 1 --2- 172.21.15.38:0/147308485 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fded0060f20 0x7fded00633e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:33.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.792+0000 7fdeebf9d700 1 --2- 172.21.15.38:0/147308485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee40f55c0 0x7fdee40f9720 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:33.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.792+0000 7fdeebf9d700 1 --2- 172.21.15.38:0/147308485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee40f6860 0x7fdee40f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:33.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.792+0000 7fdeebf9d700 1 -- 172.21.15.38:0/147308485 >> 172.21.15.38:0/147308485 conn(0x7fdee40f0d40 msgr2=0x7fdee405fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:33.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.792+0000 7fdeebf9d700 1 -- 172.21.15.38:0/147308485 shutdown_connections 2024-02-15T18:39:33.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.792+0000 7fdeebf9d700 1 -- 172.21.15.38:0/147308485 wait complete. 2024-02-15T18:39:33.784 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 14 2024-02-15T18:39:33.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.894+0000 7f550a8f0700 1 -- 172.21.15.38:0/43315393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55040f6860 msgr2=0x7f55040f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:33.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.894+0000 7f550a8f0700 1 --2- 172.21.15.38:0/43315393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55040f6860 0x7f55040f6cc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f54f8004660 tx=0x7f54f8009b30).stop 2024-02-15T18:39:33.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.894+0000 7f550a8f0700 1 -- 172.21.15.38:0/43315393 shutdown_connections 2024-02-15T18:39:33.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.894+0000 7f550a8f0700 1 --2- 172.21.15.38:0/43315393 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55040f6860 0x7f55040f6cc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:33.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.894+0000 7f550a8f0700 1 --2- 172.21.15.38:0/43315393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55040f55c0 0x7f55040f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:33.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.894+0000 7f550a8f0700 1 -- 172.21.15.38:0/43315393 >> 172.21.15.38:0/43315393 conn(0x7f55040f0d40 msgr2=0x7f55040f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:33.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.894+0000 7f550a8f0700 1 -- 172.21.15.38:0/43315393 shutdown_connections 2024-02-15T18:39:33.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.894+0000 7f550a8f0700 1 -- 172.21.15.38:0/43315393 wait complete. 2024-02-15T18:39:33.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.894+0000 7f550a8f0700 1 Processor -- start 2024-02-15T18:39:33.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.895+0000 7f550a8f0700 1 -- start start 2024-02-15T18:39:33.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.895+0000 7f550a8f0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55040f55c0 0x7f5504106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:33.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.895+0000 7f550a8f0700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55040f6860 0x7f55041074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:33.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.895+0000 7f550a8f0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5504107a60 con 0x7f55040f55c0 2024-02-15T18:39:33.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.895+0000 7f550a8f0700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5504107ba0 con 0x7f55040f6860 2024-02-15T18:39:33.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.895+0000 7f5503fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55040f55c0 0x7f5504106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:33.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.895+0000 7f5503fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55040f55c0 0x7f5504106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50846/0 (socket says 172.21.15.38:50846) 2024-02-15T18:39:33.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.895+0000 7f5503fff700 1 -- 172.21.15.38:0/3614905583 learned_addr learned my addr 172.21.15.38:0/3614905583 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:39:33.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.895+0000 7f55037fe700 1 --2- 172.21.15.38:0/3614905583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55040f6860 0x7f55041074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:33.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.895+0000 7f5503fff700 1 -- 172.21.15.38:0/3614905583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55040f6860 msgr2=0x7f55041074b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:33.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.895+0000 7f5503fff700 1 --2- 172.21.15.38:0/3614905583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55040f6860 0x7f55041074b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:33.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.895+0000 7f5503fff700 1 -- 172.21.15.38:0/3614905583 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f8005040 con 0x7f55040f55c0 2024-02-15T18:39:33.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.895+0000 7f5503fff700 1 --2- 172.21.15.38:0/3614905583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55040f55c0 0x7f5504106fc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f54f400e7c0 tx=0x7f54f400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:33.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.896+0000 7f5500ff9700 1 -- 172.21.15.38:0/3614905583 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54f40092e0 con 0x7f55040f55c0 2024-02-15T18:39:33.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.896+0000 7f5500ff9700 1 -- 172.21.15.38:0/3614905583 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f54f400f040 con 0x7f55040f55c0 2024-02-15T18:39:33.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.896+0000 7f550a8f0700 1 -- 172.21.15.38:0/3614905583 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5504108540 con 0x7f55040f55c0 2024-02-15T18:39:33.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.896+0000 7f5500ff9700 1 -- 172.21.15.38:0/3614905583 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54f4014920 con 0x7f55040f55c0 2024-02-15T18:39:33.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.896+0000 7f550a8f0700 1 -- 172.21.15.38:0/3614905583 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5504108b00 con 0x7f55040f55c0 2024-02-15T18:39:33.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.897+0000 7f550a8f0700 1 -- 172.21.15.38:0/3614905583 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5504045ad0 con 0x7f55040f55c0 2024-02-15T18:39:33.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.897+0000 7f5500ff9700 1 -- 172.21.15.38:0/3614905583 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f54f401b030 con 0x7f55040f55c0 2024-02-15T18:39:33.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.897+0000 7f5500ff9700 1 --2- 172.21.15.38:0/3614905583 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f54ec060f20 0x7f54ec0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:39:33.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.897+0000 7f5500ff9700 1 -- 172.21.15.38:0/3614905583 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f54f407bf90 con 0x7f55040f55c0 2024-02-15T18:39:33.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.897+0000 7f55037fe700 1 --2- 172.21.15.38:0/3614905583 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f54ec060f20 0x7f54ec0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:39:33.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.898+0000 7f55037fe700 1 --2- 172.21.15.38:0/3614905583 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f54ec060f20 0x7f54ec0633e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f54f8009720 tx=0x7f54f8016040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:39:33.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:33.902+0000 7f5500ff9700 1 -- 172.21.15.38:0/3614905583 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f54f404fe40 con 0x7f55040f55c0 2024-02-15T18:39:34.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:32.480565+0000 mgr.smithi038.khesbl (mgr.14664) 181 : audit [DBG] from='client.34236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:39:34.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:32.755063+0000 mon.smithi038 (mon.0) 295 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:34.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:32.788678+0000 mgr.smithi038.khesbl (mgr.14664) 182 : audit [DBG] from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:39:34.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:32.973104+0000 mgr.smithi038.khesbl (mgr.14664) 183 : cluster [DBG] pgmap v131: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 58/342 objects degraded (16.959%) 2024-02-15T18:39:34.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:33.065680+0000 mon.smithi038 (mon.0) 296 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:39:34.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:33.066294+0000 mon.smithi038 (mon.0) 297 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:39:34.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:33.075908+0000 mon.smithi038 (mon.0) 298 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:34.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:33.083749+0000 mon.smithi038 (mon.0) 299 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:39:34.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:33.086529+0000 mon.smithi038 (mon.0) 300 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:39:34.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:33.088050+0000 mon.smithi038 (mon.0) 301 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:39:34.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:33.089491+0000 mon.smithi038 (mon.0) 302 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:39:34.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:33.090656+0000 mon.smithi038 (mon.0) 303 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-15T18:39:34.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:33.149433+0000 mon.smithi038 (mon.0) 304 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:34.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:33.455459+0000 mon.smithi038 (mon.0) 305 : audit [DBG] from='client.? 172.21.15.38:0/3227660569' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:39:34.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:34.093+0000 7f550a8f0700 1 -- 172.21.15.38:0/3614905583 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f55040faca0 con 0x7f54ec060f20 2024-02-15T18:39:34.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:34.094+0000 7f5500ff9700 1 -- 172.21.15.38:0/3614905583 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f55040faca0 con 0x7f54ec060f20 2024-02-15T18:39:34.084 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:39:34.084 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:39:34.084 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-02-15T18:39:34.084 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:39:34.084 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-02-15T18:39:34.084 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-02-15T18:39:34.084 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-02-15T18:39:34.084 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr" 2024-02-15T18:39:34.084 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-02-15T18:39:34.084 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "11/21 daemons upgraded", 2024-02-15T18:39:34.084 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading osd daemons", 2024-02-15T18:39:34.084 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-02-15T18:39:34.085 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:39:34.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:34.096+0000 7f550a8f0700 1 -- 172.21.15.38:0/3614905583 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f54ec060f20 msgr2=0x7f54ec0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:34.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:34.096+0000 7f550a8f0700 1 --2- 172.21.15.38:0/3614905583 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f54ec060f20 0x7f54ec0633e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f54f8009720 tx=0x7f54f8016040).stop 2024-02-15T18:39:34.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:34.096+0000 7f550a8f0700 1 -- 172.21.15.38:0/3614905583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55040f55c0 msgr2=0x7f5504106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:39:34.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:34.096+0000 7f550a8f0700 1 --2- 172.21.15.38:0/3614905583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55040f55c0 0x7f5504106fc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f54f400e7c0 tx=0x7f54f400b040).stop 2024-02-15T18:39:34.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:34.096+0000 7f550a8f0700 1 -- 172.21.15.38:0/3614905583 shutdown_connections 2024-02-15T18:39:34.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:34.096+0000 7f550a8f0700 1 --2- 172.21.15.38:0/3614905583 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f54ec060f20 0x7f54ec0633e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:34.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:34.096+0000 7f550a8f0700 1 --2- 172.21.15.38:0/3614905583 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55040f55c0 0x7f5504106fc0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:34.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:34.096+0000 7f550a8f0700 1 --2- 172.21.15.38:0/3614905583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55040f6860 0x7f55041074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:39:34.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:34.096+0000 7f550a8f0700 1 -- 172.21.15.38:0/3614905583 >> 172.21.15.38:0/3614905583 conn(0x7f55040f0d40 msgr2=0x7f550405fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:39:34.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:34.096+0000 7f550a8f0700 1 -- 172.21.15.38:0/3614905583 shutdown_connections 2024-02-15T18:39:34.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:39:34.096+0000 7f550a8f0700 1 -- 172.21.15.38:0/3614905583 wait complete. 2024-02-15T18:39:34.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:32.480565+0000 mgr.smithi038.khesbl (mgr.14664) 181 : audit [DBG] from='client.34236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:39:34.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:32.755063+0000 mon.smithi038 (mon.0) 295 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:34.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:32.788678+0000 mgr.smithi038.khesbl (mgr.14664) 182 : audit [DBG] from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:39:34.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:32.973104+0000 mgr.smithi038.khesbl (mgr.14664) 183 : cluster [DBG] pgmap v131: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 58/342 objects degraded (16.959%) 2024-02-15T18:39:34.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:33.065680+0000 mon.smithi038 (mon.0) 296 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:39:34.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:33.066294+0000 mon.smithi038 (mon.0) 297 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:39:34.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:33.075908+0000 mon.smithi038 (mon.0) 298 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:34.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:33.083749+0000 mon.smithi038 (mon.0) 299 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:39:34.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:33.086529+0000 mon.smithi038 (mon.0) 300 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:39:34.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:33.088050+0000 mon.smithi038 (mon.0) 301 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:39:34.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:33.089491+0000 mon.smithi038 (mon.0) 302 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:39:34.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:33.090656+0000 mon.smithi038 (mon.0) 303 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-15T18:39:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:33.149433+0000 mon.smithi038 (mon.0) 304 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:34.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:33.455459+0000 mon.smithi038 (mon.0) 305 : audit [DBG] from='client.? 172.21.15.38:0/3227660569' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:39:35.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:39:33.066812+0000 mgr.smithi038.khesbl (mgr.14664) 184 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:39:35.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:33.090053+0000 mgr.smithi038.khesbl (mgr.14664) 185 : audit [DBG] from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:39:35.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:33.090780+0000 mgr.smithi038.khesbl (mgr.14664) 186 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-15T18:39:35.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:39:33.092745+0000 mgr.smithi038.khesbl (mgr.14664) 187 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-15T18:39:35.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:33.790595+0000 mon.smithi038 (mon.0) 306 : audit [DBG] from='client.? 172.21.15.38:0/147308485' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:39:35.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:39:33.066812+0000 mgr.smithi038.khesbl (mgr.14664) 184 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:39:35.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:33.090053+0000 mgr.smithi038.khesbl (mgr.14664) 185 : audit [DBG] from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:39:35.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:33.090780+0000 mgr.smithi038.khesbl (mgr.14664) 186 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-15T18:39:35.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:39:33.092745+0000 mgr.smithi038.khesbl (mgr.14664) 187 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-15T18:39:35.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:33.790595+0000 mon.smithi038 (mon.0) 306 : audit [DBG] from='client.? 172.21.15.38:0/147308485' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:39:36.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:34.094807+0000 mgr.smithi038.khesbl (mgr.14664) 188 : audit [DBG] from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:39:36.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:34.973640+0000 mgr.smithi038.khesbl (mgr.14664) 189 : cluster [DBG] pgmap v132: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 58/342 objects degraded (16.959%) 2024-02-15T18:39:36.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:34.094807+0000 mgr.smithi038.khesbl (mgr.14664) 188 : audit [DBG] from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:39:36.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:34.973640+0000 mgr.smithi038.khesbl (mgr.14664) 189 : cluster [DBG] pgmap v132: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 58/342 objects degraded (16.959%) 2024-02-15T18:39:37.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:36.974199+0000 mgr.smithi038.khesbl (mgr.14664) 190 : cluster [DBG] pgmap v133: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 58/342 objects degraded (16.959%) 2024-02-15T18:39:37.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:36.974199+0000 mgr.smithi038.khesbl (mgr.14664) 190 : cluster [DBG] pgmap v133: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 58/342 objects degraded (16.959%) 2024-02-15T18:39:38.040 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-4[64474]: debug 2024-02-15T18:39:37.778+0000 7f9cd56dd380 -1 osd.4 66 log_to_monitors {default=true} 2024-02-15T18:39:38.355 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:39:38 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-4[64474]: debug 2024-02-15T18:39:38.265+0000 7f9cc83c7700 -1 osd.4 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-15T18:39:38.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:38 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:37.248550+0000 mon.smithi038 (mon.0) 307 : cluster [WRN] Health check update: Degraded data redundancy: 58/342 objects degraded (16.959%), 16 pgs degraded (PG_DEGRADED) 2024-02-15T18:39:38.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:38 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:37.782231+0000 mon.smithi121 (mon.1) 4 : audit [INF] from='osd.4 [v2:172.21.15.121:6808/1905463927,v1:172.21.15.121:6809/1905463927]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-15T18:39:38.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:38 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:37.782866+0000 mon.smithi038 (mon.0) 308 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-15T18:39:38.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:38 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:37.248550+0000 mon.smithi038 (mon.0) 307 : cluster [WRN] Health check update: Degraded data redundancy: 58/342 objects degraded (16.959%), 16 pgs degraded (PG_DEGRADED) 2024-02-15T18:39:38.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:38 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:37.782231+0000 mon.smithi121 (mon.1) 4 : audit [INF] from='osd.4 [v2:172.21.15.121:6808/1905463927,v1:172.21.15.121:6809/1905463927]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-15T18:39:38.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:38 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:37.782866+0000 mon.smithi038 (mon.0) 308 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-15T18:39:39.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:38.058314+0000 mon.smithi038 (mon.0) 309 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:active} 2 up:standby 2024-02-15T18:39:39.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:38.255647+0000 mon.smithi038 (mon.0) 310 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-15T18:39:39.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:38.255741+0000 mon.smithi038 (mon.0) 311 : cluster [DBG] osdmap e69: 6 total, 5 up, 6 in 2024-02-15T18:39:39.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:38.258694+0000 mon.smithi121 (mon.1) 5 : audit [INF] from='osd.4 [v2:172.21.15.121:6808/1905463927,v1:172.21.15.121:6809/1905463927]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:39:39.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:38.259177+0000 mon.smithi038 (mon.0) 312 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:39:39.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:38.974774+0000 mgr.smithi038.khesbl (mgr.14664) 191 : cluster [DBG] pgmap v135: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 58/342 objects degraded (16.959%) 2024-02-15T18:39:39.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:38.058314+0000 mon.smithi038 (mon.0) 309 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:active} 2 up:standby 2024-02-15T18:39:39.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:38.255647+0000 mon.smithi038 (mon.0) 310 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-15T18:39:39.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:38.255741+0000 mon.smithi038 (mon.0) 311 : cluster [DBG] osdmap e69: 6 total, 5 up, 6 in 2024-02-15T18:39:39.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:38.258694+0000 mon.smithi121 (mon.1) 5 : audit [INF] from='osd.4 [v2:172.21.15.121:6808/1905463927,v1:172.21.15.121:6809/1905463927]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:39:39.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:38.259177+0000 mon.smithi038 (mon.0) 312 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:39:39.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:38.974774+0000 mgr.smithi038.khesbl (mgr.14664) 191 : cluster [DBG] pgmap v135: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 58/342 objects degraded (16.959%) 2024-02-15T18:39:40.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:39.253601+0000 mon.smithi038 (mon.0) 313 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:39:40.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:39.258683+0000 mon.smithi038 (mon.0) 314 : cluster [INF] osd.4 [v2:172.21.15.121:6808/1905463927,v1:172.21.15.121:6809/1905463927] boot 2024-02-15T18:39:40.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:39.258761+0000 mon.smithi038 (mon.0) 315 : cluster [DBG] osdmap e70: 6 total, 6 up, 6 in 2024-02-15T18:39:40.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:40 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:39.259221+0000 mon.smithi038 (mon.0) 316 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:39:40.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:39.253601+0000 mon.smithi038 (mon.0) 313 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:39:40.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:39.258683+0000 mon.smithi038 (mon.0) 314 : cluster [INF] osd.4 [v2:172.21.15.121:6808/1905463927,v1:172.21.15.121:6809/1905463927] boot 2024-02-15T18:39:40.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:39.258761+0000 mon.smithi038 (mon.0) 315 : cluster [DBG] osdmap e70: 6 total, 6 up, 6 in 2024-02-15T18:39:40.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:40 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:39.259221+0000 mon.smithi038 (mon.0) 316 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:39:41.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:40.259858+0000 mon.smithi038 (mon.0) 317 : cluster [DBG] osdmap e71: 6 total, 6 up, 6 in 2024-02-15T18:39:41.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:40.975275+0000 mgr.smithi038.khesbl (mgr.14664) 192 : cluster [DBG] pgmap v138: 65 pgs: 16 peering, 8 active+undersized+degraded, 10 active+undersized, 31 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 21/342 objects degraded (6.140%) 2024-02-15T18:39:41.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:40.259858+0000 mon.smithi038 (mon.0) 317 : cluster [DBG] osdmap e71: 6 total, 6 up, 6 in 2024-02-15T18:39:41.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:40.975275+0000 mgr.smithi038.khesbl (mgr.14664) 192 : cluster [DBG] pgmap v138: 65 pgs: 16 peering, 8 active+undersized+degraded, 10 active+undersized, 31 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 21/342 objects degraded (6.140%) 2024-02-15T18:39:42.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:42.249894+0000 mon.smithi038 (mon.0) 318 : cluster [WRN] Health check update: Degraded data redundancy: 21/342 objects degraded (6.140%), 8 pgs degraded (PG_DEGRADED) 2024-02-15T18:39:42.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:42.249894+0000 mon.smithi038 (mon.0) 318 : cluster [WRN] Health check update: Degraded data redundancy: 21/342 objects degraded (6.140%), 8 pgs degraded (PG_DEGRADED) 2024-02-15T18:39:43.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:42.975811+0000 mgr.smithi038.khesbl (mgr.14664) 193 : cluster [DBG] pgmap v139: 65 pgs: 16 peering, 8 active+undersized+degraded, 10 active+undersized, 31 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 21/342 objects degraded (6.140%) 2024-02-15T18:39:43.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:42.975811+0000 mgr.smithi038.khesbl (mgr.14664) 193 : cluster [DBG] pgmap v139: 65 pgs: 16 peering, 8 active+undersized+degraded, 10 active+undersized, 31 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 21/342 objects degraded (6.140%) 2024-02-15T18:39:45.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:44.976301+0000 mgr.smithi038.khesbl (mgr.14664) 194 : cluster [DBG] pgmap v140: 65 pgs: 10 peering, 55 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:45.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:45.027047+0000 mon.smithi038 (mon.0) 319 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/342 objects degraded (6.140%), 8 pgs degraded) 2024-02-15T18:39:45.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:45.027079+0000 mon.smithi038 (mon.0) 320 : cluster [INF] Cluster is now healthy 2024-02-15T18:39:45.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:44.976301+0000 mgr.smithi038.khesbl (mgr.14664) 194 : cluster [DBG] pgmap v140: 65 pgs: 10 peering, 55 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:45.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:45.027047+0000 mon.smithi038 (mon.0) 319 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/342 objects degraded (6.140%), 8 pgs degraded) 2024-02-15T18:39:45.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:45.027079+0000 mon.smithi038 (mon.0) 320 : cluster [INF] Cluster is now healthy 2024-02-15T18:39:47.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:46.976818+0000 mgr.smithi038.khesbl (mgr.14664) 195 : cluster [DBG] pgmap v141: 65 pgs: 65 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:47.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:46.976818+0000 mgr.smithi038.khesbl (mgr.14664) 195 : cluster [DBG] pgmap v141: 65 pgs: 65 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:48.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:48.108214+0000 mon.smithi038 (mon.0) 321 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-15T18:39:48.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:48.154295+0000 mon.smithi038 (mon.0) 322 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:48.581 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:48.108214+0000 mon.smithi038 (mon.0) 321 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-15T18:39:48.581 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:48.154295+0000 mon.smithi038 (mon.0) 322 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:49.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:48.108431+0000 mgr.smithi038.khesbl (mgr.14664) 196 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-15T18:39:49.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:39:48.109476+0000 mgr.smithi038.khesbl (mgr.14664) 197 : cephadm [INF] Upgrade: osd.5 is safe to restart 2024-02-15T18:39:49.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:48.977261+0000 mgr.smithi038.khesbl (mgr.14664) 198 : cluster [DBG] pgmap v142: 65 pgs: 65 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:49.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:48.108431+0000 mgr.smithi038.khesbl (mgr.14664) 196 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-15T18:39:49.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:39:48.109476+0000 mgr.smithi038.khesbl (mgr.14664) 197 : cephadm [INF] Upgrade: osd.5 is safe to restart 2024-02-15T18:39:49.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:48.977261+0000 mgr.smithi038.khesbl (mgr.14664) 198 : cluster [DBG] pgmap v142: 65 pgs: 65 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:51.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:39:49.829099+0000 mgr.smithi038.khesbl (mgr.14664) 199 : cephadm [INF] Upgrade: Updating osd.5 2024-02-15T18:39:51.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:49.839076+0000 mon.smithi038 (mon.0) 323 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:51.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:49.840184+0000 mon.smithi038 (mon.0) 324 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-15T18:39:51.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:49.840942+0000 mon.smithi038 (mon.0) 325 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:39:51.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:39:49.842174+0000 mgr.smithi038.khesbl (mgr.14664) 200 : cephadm [INF] Deploying daemon osd.5 on smithi121 2024-02-15T18:39:51.260 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:39:49.829099+0000 mgr.smithi038.khesbl (mgr.14664) 199 : cephadm [INF] Upgrade: Updating osd.5 2024-02-15T18:39:51.260 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:49.839076+0000 mon.smithi038 (mon.0) 323 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:51.260 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:49.840184+0000 mon.smithi038 (mon.0) 324 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-15T18:39:51.260 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:49.840942+0000 mon.smithi038 (mon.0) 325 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:39:51.260 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:39:49.842174+0000 mgr.smithi038.khesbl (mgr.14664) 200 : cephadm [INF] Deploying daemon osd.5 on smithi121 2024-02-15T18:39:52.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:50.977892+0000 mgr.smithi038.khesbl (mgr.14664) 201 : cluster [DBG] pgmap v143: 65 pgs: 65 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:52.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:50.977892+0000 mgr.smithi038.khesbl (mgr.14664) 201 : cluster [DBG] pgmap v143: 65 pgs: 65 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:53.187 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:53 smithi121 systemd[1]: Stopping Ceph osd.5 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:39:53.501 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:52.978337+0000 mgr.smithi038.khesbl (mgr.14664) 202 : cluster [DBG] pgmap v144: 65 pgs: 65 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:53.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:53.233023+0000 mon.smithi038 (mon.0) 326 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:53.581 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:52.978337+0000 mgr.smithi038.khesbl (mgr.14664) 202 : cluster [DBG] pgmap v144: 65 pgs: 65 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:53.581 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:53.233023+0000 mon.smithi038 (mon.0) 326 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:53.581 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.5[52426]: debug 2024-02-15T18:39:53.197+0000 7f97f3088700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:39:53.581 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.5[52426]: debug 2024-02-15T18:39:53.197+0000 7f97f3088700 -1 osd.5 71 *** Got signal Terminated *** 2024-02-15T18:39:53.581 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.5[52426]: debug 2024-02-15T18:39:53.197+0000 7f97f3088700 -1 osd.5 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:39:53.856 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:53 smithi121 bash[66157]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.5 2024-02-15T18:39:54.170 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:53 smithi121 bash[66212]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.5 2024-02-15T18:39:54.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.450127+0000 mon.smithi038 (mon.0) 327 : cluster [DBG] osd.5 reported immediately failed by osd.1 2024-02-15T18:39:54.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.450170+0000 mon.smithi038 (mon.0) 328 : cluster [INF] osd.5 failed (root=default,host=smithi121) (connection refused reported by osd.1) 2024-02-15T18:39:54.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.450327+0000 mon.smithi038 (mon.0) 329 : cluster [DBG] osd.5 reported immediately failed by osd.1 2024-02-15T18:39:54.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.450399+0000 mon.smithi038 (mon.0) 330 : cluster [DBG] osd.5 reported immediately failed by osd.2 2024-02-15T18:39:54.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.450645+0000 mon.smithi038 (mon.0) 331 : cluster [DBG] osd.5 reported immediately failed by osd.4 2024-02-15T18:39:54.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.450763+0000 mon.smithi038 (mon.0) 332 : cluster [DBG] osd.5 reported immediately failed by osd.0 2024-02-15T18:39:54.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.450826+0000 mon.smithi038 (mon.0) 333 : cluster [DBG] osd.5 reported immediately failed by osd.2 2024-02-15T18:39:54.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.450995+0000 mon.smithi038 (mon.0) 334 : cluster [DBG] osd.5 reported immediately failed by osd.3 2024-02-15T18:39:54.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.451067+0000 mon.smithi038 (mon.0) 335 : cluster [DBG] osd.5 reported immediately failed by osd.0 2024-02-15T18:39:54.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.451235+0000 mon.smithi038 (mon.0) 336 : cluster [DBG] osd.5 reported immediately failed by osd.4 2024-02-15T18:39:54.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.451371+0000 mon.smithi038 (mon.0) 337 : cluster [DBG] osd.5 reported immediately failed by osd.3 2024-02-15T18:39:54.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.850990+0000 mon.smithi038 (mon.0) 338 : cluster [DBG] osd.5 reported immediately failed by osd.4 2024-02-15T18:39:54.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.851209+0000 mon.smithi038 (mon.0) 339 : cluster [DBG] osd.5 reported immediately failed by osd.3 2024-02-15T18:39:54.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.851289+0000 mon.smithi038 (mon.0) 340 : cluster [DBG] osd.5 reported immediately failed by osd.0 2024-02-15T18:39:54.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.851421+0000 mon.smithi038 (mon.0) 341 : cluster [DBG] osd.5 reported immediately failed by osd.1 2024-02-15T18:39:54.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.851472+0000 mon.smithi038 (mon.0) 342 : cluster [DBG] osd.5 reported immediately failed by osd.2 2024-02-15T18:39:54.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.851647+0000 mon.smithi038 (mon.0) 343 : cluster [DBG] osd.5 reported immediately failed by osd.4 2024-02-15T18:39:54.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.851703+0000 mon.smithi038 (mon.0) 344 : cluster [DBG] osd.5 reported immediately failed by osd.0 2024-02-15T18:39:54.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.851861+0000 mon.smithi038 (mon.0) 345 : cluster [DBG] osd.5 reported immediately failed by osd.1 2024-02-15T18:39:54.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.851913+0000 mon.smithi038 (mon.0) 346 : cluster [DBG] osd.5 reported immediately failed by osd.2 2024-02-15T18:39:54.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:53.852081+0000 mon.smithi038 (mon.0) 347 : cluster [DBG] osd.5 reported immediately failed by osd.3 2024-02-15T18:39:54.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:54 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:54.124220+0000 mon.smithi038 (mon.0) 348 : cluster [DBG] osd.5 reported immediately failed by osd.1 2024-02-15T18:39:54.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.450127+0000 mon.smithi038 (mon.0) 327 : cluster [DBG] osd.5 reported immediately failed by osd.1 2024-02-15T18:39:54.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.450170+0000 mon.smithi038 (mon.0) 328 : cluster [INF] osd.5 failed (root=default,host=smithi121) (connection refused reported by osd.1) 2024-02-15T18:39:54.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.450327+0000 mon.smithi038 (mon.0) 329 : cluster [DBG] osd.5 reported immediately failed by osd.1 2024-02-15T18:39:54.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.450399+0000 mon.smithi038 (mon.0) 330 : cluster [DBG] osd.5 reported immediately failed by osd.2 2024-02-15T18:39:54.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.450645+0000 mon.smithi038 (mon.0) 331 : cluster [DBG] osd.5 reported immediately failed by osd.4 2024-02-15T18:39:54.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.450763+0000 mon.smithi038 (mon.0) 332 : cluster [DBG] osd.5 reported immediately failed by osd.0 2024-02-15T18:39:54.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.450826+0000 mon.smithi038 (mon.0) 333 : cluster [DBG] osd.5 reported immediately failed by osd.2 2024-02-15T18:39:54.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.450995+0000 mon.smithi038 (mon.0) 334 : cluster [DBG] osd.5 reported immediately failed by osd.3 2024-02-15T18:39:54.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.451067+0000 mon.smithi038 (mon.0) 335 : cluster [DBG] osd.5 reported immediately failed by osd.0 2024-02-15T18:39:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.451235+0000 mon.smithi038 (mon.0) 336 : cluster [DBG] osd.5 reported immediately failed by osd.4 2024-02-15T18:39:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.451371+0000 mon.smithi038 (mon.0) 337 : cluster [DBG] osd.5 reported immediately failed by osd.3 2024-02-15T18:39:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.850990+0000 mon.smithi038 (mon.0) 338 : cluster [DBG] osd.5 reported immediately failed by osd.4 2024-02-15T18:39:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.851209+0000 mon.smithi038 (mon.0) 339 : cluster [DBG] osd.5 reported immediately failed by osd.3 2024-02-15T18:39:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.851289+0000 mon.smithi038 (mon.0) 340 : cluster [DBG] osd.5 reported immediately failed by osd.0 2024-02-15T18:39:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.851421+0000 mon.smithi038 (mon.0) 341 : cluster [DBG] osd.5 reported immediately failed by osd.1 2024-02-15T18:39:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.851472+0000 mon.smithi038 (mon.0) 342 : cluster [DBG] osd.5 reported immediately failed by osd.2 2024-02-15T18:39:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.851647+0000 mon.smithi038 (mon.0) 343 : cluster [DBG] osd.5 reported immediately failed by osd.4 2024-02-15T18:39:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.851703+0000 mon.smithi038 (mon.0) 344 : cluster [DBG] osd.5 reported immediately failed by osd.0 2024-02-15T18:39:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.851861+0000 mon.smithi038 (mon.0) 345 : cluster [DBG] osd.5 reported immediately failed by osd.1 2024-02-15T18:39:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.851913+0000 mon.smithi038 (mon.0) 346 : cluster [DBG] osd.5 reported immediately failed by osd.2 2024-02-15T18:39:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:53.852081+0000 mon.smithi038 (mon.0) 347 : cluster [DBG] osd.5 reported immediately failed by osd.3 2024-02-15T18:39:54.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:54.124220+0000 mon.smithi038 (mon.0) 348 : cluster [DBG] osd.5 reported immediately failed by osd.1 2024-02-15T18:39:55.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:54.258763+0000 mon.smithi038 (mon.0) 349 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:39:55.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:54.268862+0000 mon.smithi038 (mon.0) 350 : cluster [DBG] osdmap e72: 6 total, 5 up, 6 in 2024-02-15T18:39:55.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:54.978892+0000 mgr.smithi038.khesbl (mgr.14664) 203 : cluster [DBG] pgmap v146: 65 pgs: 3 peering, 11 stale+active+clean, 51 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:55.547 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:54.258763+0000 mon.smithi038 (mon.0) 349 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:39:55.547 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:54.268862+0000 mon.smithi038 (mon.0) 350 : cluster [DBG] osdmap e72: 6 total, 5 up, 6 in 2024-02-15T18:39:55.547 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:54.978892+0000 mgr.smithi038.khesbl (mgr.14664) 203 : cluster [DBG] pgmap v146: 65 pgs: 3 peering, 11 stale+active+clean, 51 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:56.254 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:55 smithi121 systemd[1]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.5.service: Succeeded. 2024-02-15T18:39:56.255 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:55 smithi121 systemd[1]: Stopped Ceph osd.5 for 7e05753e-cc2f-11ee-95ba-87774f69a715. 2024-02-15T18:39:56.255 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:56 smithi121 systemd[1]: Starting Ceph osd.5 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:39:56.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:55.266075+0000 mon.smithi038 (mon.0) 351 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-02-15T18:39:56.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:56 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:55.277985+0000 mon.smithi038 (mon.0) 352 : cluster [DBG] osdmap e73: 6 total, 5 up, 6 in 2024-02-15T18:39:56.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:55.266075+0000 mon.smithi038 (mon.0) 351 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-02-15T18:39:56.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:55.277985+0000 mon.smithi038 (mon.0) 352 : cluster [DBG] osdmap e73: 6 total, 5 up, 6 in 2024-02-15T18:39:57.106 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:56 smithi121 bash[66469]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-02-15T18:39:57.106 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:56 smithi121 bash[66469]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-15T18:39:57.106 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:56 smithi121 bash[66469]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-5 --no-mon-config 2024-02-15T18:39:57.106 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:56 smithi121 bash[66469]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5/block 2024-02-15T18:39:57.106 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:56 smithi121 bash[66469]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-5/block 2024-02-15T18:39:57.106 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:56 smithi121 bash[66469]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-15T18:39:57.106 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:56 smithi121 bash[66469]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-15T18:39:57.106 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:56 smithi121 bash[66469]: --> ceph-volume lvm activate successful for osd ID: 5 2024-02-15T18:39:57.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:56.979317+0000 mgr.smithi038.khesbl (mgr.14664) 204 : cluster [DBG] pgmap v148: 65 pgs: 13 peering, 8 stale+active+clean, 44 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:57.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:56.979317+0000 mgr.smithi038.khesbl (mgr.14664) 204 : cluster [DBG] pgmap v148: 65 pgs: 13 peering, 8 stale+active+clean, 44 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:39:58.355 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:39:58.078+0000 7f89fe20a700 1 -- 172.21.15.121:0/2877672965 <== mon.1 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c31c796680 con 0x55c31c75d400 2024-02-15T18:39:58.355 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:58 smithi121 bash[66735]: 79bb22243769147da2e6ed78d0e1b9aecae3f9b724ef1ab429b72659ef844ac0 2024-02-15T18:39:58.356 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:58 smithi121 systemd[1]: Started Ceph osd.5 for 7e05753e-cc2f-11ee-95ba-87774f69a715. 2024-02-15T18:39:58.855 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:39:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:39:58.596+0000 7f8a05c98380 -1 Falling back to public interface 2024-02-15T18:39:59.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:58.217773+0000 mon.smithi038 (mon.0) 353 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:59.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:39:58.218666+0000 mon.smithi038 (mon.0) 354 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:39:59.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:39:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:58.979882+0000 mgr.smithi038.khesbl (mgr.14664) 205 : cluster [DBG] pgmap v149: 65 pgs: 8 active+undersized+degraded, 13 peering, 7 active+undersized, 37 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 22/342 objects degraded (6.433%) 2024-02-15T18:39:59.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:58.217773+0000 mon.smithi038 (mon.0) 353 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:39:59.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:39:58.218666+0000 mon.smithi038 (mon.0) 354 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:39:59.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:39:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:58.979882+0000 mgr.smithi038.khesbl (mgr.14664) 205 : cluster [DBG] pgmap v149: 65 pgs: 8 active+undersized+degraded, 13 peering, 7 active+undersized, 37 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 22/342 objects degraded (6.433%) 2024-02-15T18:40:00.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:39:59.215416+0000 mon.smithi038 (mon.0) 355 : cluster [WRN] Health check failed: Degraded data redundancy: 22/342 objects degraded (6.433%), 8 pgs degraded (PG_DEGRADED) 2024-02-15T18:40:00.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:00.000201+0000 mon.smithi038 (mon.0) 356 : cluster [WRN] Health detail: HEALTH_WARN 1 osds down; Reduced data availability: 5 pgs peering; Degraded data redundancy: 22/342 objects degraded (6.433%), 8 pgs degraded 2024-02-15T18:40:00.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:00.000228+0000 mon.smithi038 (mon.0) 357 : cluster [WRN] [WRN] OSD_DOWN: 1 osds down 2024-02-15T18:40:00.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:00.000239+0000 mon.smithi038 (mon.0) 358 : cluster [WRN] osd.5 (root=default,host=smithi121) is down 2024-02-15T18:40:00.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:00.000248+0000 mon.smithi038 (mon.0) 359 : cluster [WRN] [WRN] PG_AVAILABILITY: Reduced data availability: 5 pgs peering 2024-02-15T18:40:00.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:00.000257+0000 mon.smithi038 (mon.0) 360 : cluster [WRN] pg 2.16 is stuck peering for 3m, current state peering, last acting [3,2] 2024-02-15T18:40:00.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:00.000283+0000 mon.smithi038 (mon.0) 361 : cluster [WRN] pg 3.c is stuck peering for 3m, current state peering, last acting [0,3] 2024-02-15T18:40:00.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:00.000309+0000 mon.smithi038 (mon.0) 362 : cluster [WRN] pg 3.f is stuck peering for 3m, current state peering, last acting [3,0] 2024-02-15T18:40:00.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:00.000322+0000 mon.smithi038 (mon.0) 363 : cluster [WRN] pg 3.10 is stuck peering for 3m, current state peering, last acting [0,1] 2024-02-15T18:40:00.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:00.000334+0000 mon.smithi038 (mon.0) 364 : cluster [WRN] pg 3.16 is stuck peering for 3m, current state peering, last acting [3,1] 2024-02-15T18:40:00.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:00.000345+0000 mon.smithi038 (mon.0) 365 : cluster [WRN] [WRN] PG_DEGRADED: Degraded data redundancy: 22/342 objects degraded (6.433%), 8 pgs degraded 2024-02-15T18:40:00.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:00.000354+0000 mon.smithi038 (mon.0) 366 : cluster [WRN] pg 2.2 is active+undersized+degraded, acting [1,0] 2024-02-15T18:40:00.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:00.000363+0000 mon.smithi038 (mon.0) 367 : cluster [WRN] pg 2.3 is active+undersized+degraded, acting [2,1] 2024-02-15T18:40:00.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:00.000372+0000 mon.smithi038 (mon.0) 368 : cluster [WRN] pg 2.f is active+undersized+degraded, acting [4,0] 2024-02-15T18:40:00.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:00.000383+0000 mon.smithi038 (mon.0) 369 : cluster [WRN] pg 2.17 is active+undersized+degraded, acting [2,1] 2024-02-15T18:40:00.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:00.000397+0000 mon.smithi038 (mon.0) 370 : cluster [WRN] pg 2.18 is active+undersized+degraded, acting [4,3] 2024-02-15T18:40:00.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:00.000410+0000 mon.smithi038 (mon.0) 371 : cluster [WRN] pg 2.1b is active+undersized+degraded, acting [1,0] 2024-02-15T18:40:00.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:00.000423+0000 mon.smithi038 (mon.0) 372 : cluster [WRN] pg 2.1c is active+undersized+degraded, acting [4,2] 2024-02-15T18:40:00.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:00.000436+0000 mon.smithi038 (mon.0) 373 : cluster [WRN] pg 2.1e is active+undersized+degraded, acting [2,0] 2024-02-15T18:40:00.554 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:39:59.215416+0000 mon.smithi038 (mon.0) 355 : cluster [WRN] Health check failed: Degraded data redundancy: 22/342 objects degraded (6.433%), 8 pgs degraded (PG_DEGRADED) 2024-02-15T18:40:00.554 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:00.000201+0000 mon.smithi038 (mon.0) 356 : cluster [WRN] Health detail: HEALTH_WARN 1 osds down; Reduced data availability: 5 pgs peering; Degraded data redundancy: 22/342 objects degraded (6.433%), 8 pgs degraded 2024-02-15T18:40:00.554 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:00.000228+0000 mon.smithi038 (mon.0) 357 : cluster [WRN] [WRN] OSD_DOWN: 1 osds down 2024-02-15T18:40:00.554 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:00.000239+0000 mon.smithi038 (mon.0) 358 : cluster [WRN] osd.5 (root=default,host=smithi121) is down 2024-02-15T18:40:00.554 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:00.000248+0000 mon.smithi038 (mon.0) 359 : cluster [WRN] [WRN] PG_AVAILABILITY: Reduced data availability: 5 pgs peering 2024-02-15T18:40:00.554 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:00.000257+0000 mon.smithi038 (mon.0) 360 : cluster [WRN] pg 2.16 is stuck peering for 3m, current state peering, last acting [3,2] 2024-02-15T18:40:00.554 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:00.000283+0000 mon.smithi038 (mon.0) 361 : cluster [WRN] pg 3.c is stuck peering for 3m, current state peering, last acting [0,3] 2024-02-15T18:40:00.554 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:00.000309+0000 mon.smithi038 (mon.0) 362 : cluster [WRN] pg 3.f is stuck peering for 3m, current state peering, last acting [3,0] 2024-02-15T18:40:00.555 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:00.000322+0000 mon.smithi038 (mon.0) 363 : cluster [WRN] pg 3.10 is stuck peering for 3m, current state peering, last acting [0,1] 2024-02-15T18:40:00.555 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:00.000334+0000 mon.smithi038 (mon.0) 364 : cluster [WRN] pg 3.16 is stuck peering for 3m, current state peering, last acting [3,1] 2024-02-15T18:40:00.555 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:00.000345+0000 mon.smithi038 (mon.0) 365 : cluster [WRN] [WRN] PG_DEGRADED: Degraded data redundancy: 22/342 objects degraded (6.433%), 8 pgs degraded 2024-02-15T18:40:00.555 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:00.000354+0000 mon.smithi038 (mon.0) 366 : cluster [WRN] pg 2.2 is active+undersized+degraded, acting [1,0] 2024-02-15T18:40:00.555 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:00.000363+0000 mon.smithi038 (mon.0) 367 : cluster [WRN] pg 2.3 is active+undersized+degraded, acting [2,1] 2024-02-15T18:40:00.555 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:00.000372+0000 mon.smithi038 (mon.0) 368 : cluster [WRN] pg 2.f is active+undersized+degraded, acting [4,0] 2024-02-15T18:40:00.555 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:00.000383+0000 mon.smithi038 (mon.0) 369 : cluster [WRN] pg 2.17 is active+undersized+degraded, acting [2,1] 2024-02-15T18:40:00.555 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:00.000397+0000 mon.smithi038 (mon.0) 370 : cluster [WRN] pg 2.18 is active+undersized+degraded, acting [4,3] 2024-02-15T18:40:00.555 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:00.000410+0000 mon.smithi038 (mon.0) 371 : cluster [WRN] pg 2.1b is active+undersized+degraded, acting [1,0] 2024-02-15T18:40:00.555 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:00.000423+0000 mon.smithi038 (mon.0) 372 : cluster [WRN] pg 2.1c is active+undersized+degraded, acting [4,2] 2024-02-15T18:40:00.555 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:00.000436+0000 mon.smithi038 (mon.0) 373 : cluster [WRN] pg 2.1e is active+undersized+degraded, acting [2,0] 2024-02-15T18:40:01.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:00.980441+0000 mgr.smithi038.khesbl (mgr.14664) 206 : cluster [DBG] pgmap v150: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 54/342 objects degraded (15.789%) 2024-02-15T18:40:01.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:00.980441+0000 mgr.smithi038.khesbl (mgr.14664) 206 : cluster [DBG] pgmap v150: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 54/342 objects degraded (15.789%) 2024-02-15T18:40:02.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:01.225380+0000 mon.smithi038 (mon.0) 374 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs peering) 2024-02-15T18:40:02.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.556113+0000 mon.smithi038 (mon.0) 375 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:02.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.557064+0000 mon.smithi038 (mon.0) 376 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:40:02.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.558004+0000 mon.smithi038 (mon.0) 377 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:40:02.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:40:01.558749+0000 mgr.smithi038.khesbl (mgr.14664) 207 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:40:02.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.564542+0000 mon.smithi038 (mon.0) 378 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:02.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.570602+0000 mon.smithi038 (mon.0) 379 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:02.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.573174+0000 mon.smithi038 (mon.0) 380 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:02.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.574592+0000 mon.smithi038 (mon.0) 381 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:02.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.575920+0000 mon.smithi038 (mon.0) 382 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.577611+0000 mon.smithi038 (mon.0) 383 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:40:01.578286+0000 mgr.smithi038.khesbl (mgr.14664) 208 : cephadm [INF] Upgrade: Setting container_image for all osd 2024-02-15T18:40:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.582413+0000 mon.smithi038 (mon.0) 384 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.583268+0000 mon.smithi038 (mon.0) 385 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-02-15T18:40:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.586978+0000 mon.smithi038 (mon.0) 386 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-02-15T18:40:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.587672+0000 mon.smithi038 (mon.0) 387 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-02-15T18:40:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.591336+0000 mon.smithi038 (mon.0) 388 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-02-15T18:40:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.592014+0000 mon.smithi038 (mon.0) 389 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-02-15T18:40:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.595640+0000 mon.smithi038 (mon.0) 390 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-02-15T18:40:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.596315+0000 mon.smithi038 (mon.0) 391 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-02-15T18:40:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.599858+0000 mon.smithi038 (mon.0) 392 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-02-15T18:40:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.600541+0000 mon.smithi038 (mon.0) 393 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-02-15T18:40:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.604185+0000 mon.smithi038 (mon.0) 394 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-02-15T18:40:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.604856+0000 mon.smithi038 (mon.0) 395 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-02-15T18:40:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.608362+0000 mon.smithi038 (mon.0) 396 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-02-15T18:40:02.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:40:01.610118+0000 mgr.smithi038.khesbl (mgr.14664) 209 : cephadm [INF] Upgrade: Scaling down filesystem cephfs 2024-02-15T18:40:02.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.612871+0000 mon.smithi038 (mon.0) 397 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:02.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:01.613238+0000 mon.smithi038 (mon.0) 398 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-02-15T18:40:02.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:01.225380+0000 mon.smithi038 (mon.0) 374 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs peering) 2024-02-15T18:40:02.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.556113+0000 mon.smithi038 (mon.0) 375 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:02.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.557064+0000 mon.smithi038 (mon.0) 376 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:40:02.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.558004+0000 mon.smithi038 (mon.0) 377 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:40:02.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:40:01.558749+0000 mgr.smithi038.khesbl (mgr.14664) 207 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:40:02.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.564542+0000 mon.smithi038 (mon.0) 378 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:02.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.570602+0000 mon.smithi038 (mon.0) 379 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:02.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.573174+0000 mon.smithi038 (mon.0) 380 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:02.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.574592+0000 mon.smithi038 (mon.0) 381 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:02.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.575920+0000 mon.smithi038 (mon.0) 382 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:02.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.577611+0000 mon.smithi038 (mon.0) 383 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:02.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:40:01.578286+0000 mgr.smithi038.khesbl (mgr.14664) 208 : cephadm [INF] Upgrade: Setting container_image for all osd 2024-02-15T18:40:02.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.582413+0000 mon.smithi038 (mon.0) 384 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:02.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.583268+0000 mon.smithi038 (mon.0) 385 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-02-15T18:40:02.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.586978+0000 mon.smithi038 (mon.0) 386 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-02-15T18:40:02.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.587672+0000 mon.smithi038 (mon.0) 387 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-02-15T18:40:02.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.591336+0000 mon.smithi038 (mon.0) 388 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-02-15T18:40:02.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.592014+0000 mon.smithi038 (mon.0) 389 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-02-15T18:40:02.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.595640+0000 mon.smithi038 (mon.0) 390 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-02-15T18:40:02.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.596315+0000 mon.smithi038 (mon.0) 391 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-02-15T18:40:02.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.599858+0000 mon.smithi038 (mon.0) 392 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-02-15T18:40:02.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.600541+0000 mon.smithi038 (mon.0) 393 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-02-15T18:40:02.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.604185+0000 mon.smithi038 (mon.0) 394 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-02-15T18:40:02.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.604856+0000 mon.smithi038 (mon.0) 395 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-02-15T18:40:02.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.608362+0000 mon.smithi038 (mon.0) 396 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-02-15T18:40:02.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:40:01.610118+0000 mgr.smithi038.khesbl (mgr.14664) 209 : cephadm [INF] Upgrade: Scaling down filesystem cephfs 2024-02-15T18:40:02.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.612871+0000 mon.smithi038 (mon.0) 397 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:02.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:01.613238+0000 mon.smithi038 (mon.0) 398 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-02-15T18:40:03.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:02.254144+0000 mon.smithi038 (mon.0) 399 : cluster [INF] stopping daemon mds.cephfs.smithi121.ovuzki 2024-02-15T18:40:03.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:02.262026+0000 mon.smithi038 (mon.0) 400 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-02-15T18:40:03.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:02.262118+0000 mon.smithi038 (mon.0) 401 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:stopping} 2 up:standby 2024-02-15T18:40:03.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:02.266990+0000 mon.smithi038 (mon.0) 402 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:03.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:02.268794+0000 mon.smithi038 (mon.0) 403 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:40:03.502 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:02.269619+0000 mon.smithi038 (mon.0) 404 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:40:03.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:40:02.270284+0000 mgr.smithi038.khesbl (mgr.14664) 210 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:40:03.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:02.278532+0000 mon.smithi038 (mon.0) 405 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:03.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:02.284351+0000 mon.smithi038 (mon.0) 406 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:03.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:02.286918+0000 mon.smithi038 (mon.0) 407 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:03.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:02.288244+0000 mon.smithi038 (mon.0) 408 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:03.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:02.289582+0000 mon.smithi038 (mon.0) 409 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:03.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:02.291265+0000 mon.smithi038 (mon.0) 410 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:03.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:40:02.292824+0000 mgr.smithi038.khesbl (mgr.14664) 211 : cephadm [INF] Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-02-15T18:40:03.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:02.980988+0000 mgr.smithi038.khesbl (mgr.14664) 212 : cluster [DBG] pgmap v151: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 54/342 objects degraded (15.789%) 2024-02-15T18:40:03.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:03.150294+0000 mon.smithi038 (mon.0) 411 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:03.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:02.254144+0000 mon.smithi038 (mon.0) 399 : cluster [INF] stopping daemon mds.cephfs.smithi121.ovuzki 2024-02-15T18:40:03.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:02.262026+0000 mon.smithi038 (mon.0) 400 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-02-15T18:40:03.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:02.262118+0000 mon.smithi038 (mon.0) 401 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi121.mvumlh=up:active,1=cephfs.smithi121.ovuzki=up:stopping} 2 up:standby 2024-02-15T18:40:03.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:02.266990+0000 mon.smithi038 (mon.0) 402 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:03.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:02.268794+0000 mon.smithi038 (mon.0) 403 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:40:03.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:02.269619+0000 mon.smithi038 (mon.0) 404 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:40:03.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:40:02.270284+0000 mgr.smithi038.khesbl (mgr.14664) 210 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:40:03.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:02.278532+0000 mon.smithi038 (mon.0) 405 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:03.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:02.284351+0000 mon.smithi038 (mon.0) 406 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:03.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:02.286918+0000 mon.smithi038 (mon.0) 407 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:03.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:02.288244+0000 mon.smithi038 (mon.0) 408 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:03.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:02.289582+0000 mon.smithi038 (mon.0) 409 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:03.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:02.291265+0000 mon.smithi038 (mon.0) 410 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:03.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:40:02.292824+0000 mgr.smithi038.khesbl (mgr.14664) 211 : cephadm [INF] Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-02-15T18:40:03.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:02.980988+0000 mgr.smithi038.khesbl (mgr.14664) 212 : cluster [DBG] pgmap v151: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 54/342 objects degraded (15.789%) 2024-02-15T18:40:03.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:03.150294+0000 mon.smithi038 (mon.0) 411 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:04.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.211+0000 7f4bea38c700 1 -- 172.21.15.38:0/2951822386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4be40f66a0 msgr2=0x7f4be40f6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:04.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.211+0000 7f4bea38c700 1 --2- 172.21.15.38:0/2951822386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4be40f66a0 0x7f4be40f6ac0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f4bd4004660 tx=0x7f4bd4009b30).stop 2024-02-15T18:40:04.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.212+0000 7f4bea38c700 1 -- 172.21.15.38:0/2951822386 shutdown_connections 2024-02-15T18:40:04.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.212+0000 7f4bea38c700 1 --2- 172.21.15.38:0/2951822386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4be40f7940 0x7f4be40f7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:04.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.212+0000 7f4bea38c700 1 --2- 172.21.15.38:0/2951822386 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4be40f66a0 0x7f4be40f6ac0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:04.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.212+0000 7f4bea38c700 1 -- 172.21.15.38:0/2951822386 >> 172.21.15.38:0/2951822386 conn(0x7f4be40f1e00 msgr2=0x7f4be40f4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:04.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.212+0000 7f4bea38c700 1 -- 172.21.15.38:0/2951822386 shutdown_connections 2024-02-15T18:40:04.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.212+0000 7f4bea38c700 1 -- 172.21.15.38:0/2951822386 wait complete. 2024-02-15T18:40:04.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.212+0000 7f4bea38c700 1 Processor -- start 2024-02-15T18:40:04.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.213+0000 7f4bea38c700 1 -- start start 2024-02-15T18:40:04.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.213+0000 7f4bea38c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4be40f66a0 0x7f4be4106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:04.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.213+0000 7f4bea38c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4be40f7940 0x7f4be4107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:04.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.213+0000 7f4bea38c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be4107a00 con 0x7f4be40f66a0 2024-02-15T18:40:04.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.213+0000 7f4bea38c700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be4107b40 con 0x7f4be40f7940 2024-02-15T18:40:04.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.213+0000 7f4be3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4be40f66a0 0x7f4be4106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:04.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.213+0000 7f4be3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4be40f66a0 0x7f4be4106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57974/0 (socket says 172.21.15.38:57974) 2024-02-15T18:40:04.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.213+0000 7f4be3fff700 1 -- 172.21.15.38:0/2128583999 learned_addr learned my addr 172.21.15.38:0/2128583999 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:40:04.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.213+0000 7f4be37fe700 1 --2- 172.21.15.38:0/2128583999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4be40f7940 0x7f4be4107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:04.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.213+0000 7f4be3fff700 1 -- 172.21.15.38:0/2128583999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4be40f7940 msgr2=0x7f4be4107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:04.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.213+0000 7f4be3fff700 1 --2- 172.21.15.38:0/2128583999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4be40f7940 0x7f4be4107450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:04.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.213+0000 7f4be3fff700 1 -- 172.21.15.38:0/2128583999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd4005040 con 0x7f4be40f66a0 2024-02-15T18:40:04.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.214+0000 7f4be3fff700 1 --2- 172.21.15.38:0/2128583999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4be40f66a0 0x7f4be4106f60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f4bd4004e30 tx=0x7f4bd4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:04.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.214+0000 7f4be0ff9700 1 -- 172.21.15.38:0/2128583999 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bd4018070 con 0x7f4be40f66a0 2024-02-15T18:40:04.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.214+0000 7f4be0ff9700 1 -- 172.21.15.38:0/2128583999 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f4bd4007ba0 con 0x7f4be40f66a0 2024-02-15T18:40:04.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.214+0000 7f4bea38c700 1 -- 172.21.15.38:0/2128583999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be4108480 con 0x7f4be40f66a0 2024-02-15T18:40:04.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.214+0000 7f4be0ff9700 1 -- 172.21.15.38:0/2128583999 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bd401c3c0 con 0x7f4be40f66a0 2024-02-15T18:40:04.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.214+0000 7f4bea38c700 1 -- 172.21.15.38:0/2128583999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be41089e0 con 0x7f4be40f66a0 2024-02-15T18:40:04.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.215+0000 7f4bea38c700 1 -- 172.21.15.38:0/2128583999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be4045ad0 con 0x7f4be40f66a0 2024-02-15T18:40:04.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.215+0000 7f4be0ff9700 1 -- 172.21.15.38:0/2128583999 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f4bd401d420 con 0x7f4be40f66a0 2024-02-15T18:40:04.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.216+0000 7f4be0ff9700 1 --2- 172.21.15.38:0/2128583999 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4bcc060f70 0x7f4bcc063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:04.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.216+0000 7f4be0ff9700 1 -- 172.21.15.38:0/2128583999 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f4bd407dd10 con 0x7f4be40f66a0 2024-02-15T18:40:04.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.216+0000 7f4be37fe700 1 --2- 172.21.15.38:0/2128583999 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4bcc060f70 0x7f4bcc063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:04.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.216+0000 7f4be37fe700 1 --2- 172.21.15.38:0/2128583999 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4bcc060f70 0x7f4bcc063430 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f4bd8004c40 tx=0x7f4bd800b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:04.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.220+0000 7f4be0ff9700 1 -- 172.21.15.38:0/2128583999 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4bd4051c30 con 0x7f4be40f66a0 2024-02-15T18:40:04.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.411+0000 7f4bea38c700 1 -- 172.21.15.38:0/2128583999 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4be4108c00 con 0x7f4bcc060f70 2024-02-15T18:40:04.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.412+0000 7f4be0ff9700 1 -- 172.21.15.38:0/2128583999 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4be4108c00 con 0x7f4bcc060f70 2024-02-15T18:40:04.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.414+0000 7f4bea38c700 1 -- 172.21.15.38:0/2128583999 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4bcc060f70 msgr2=0x7f4bcc063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:04.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.415+0000 7f4bea38c700 1 --2- 172.21.15.38:0/2128583999 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4bcc060f70 0x7f4bcc063430 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f4bd8004c40 tx=0x7f4bd800b040).stop 2024-02-15T18:40:04.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.415+0000 7f4bea38c700 1 -- 172.21.15.38:0/2128583999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4be40f66a0 msgr2=0x7f4be4106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:04.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.415+0000 7f4bea38c700 1 --2- 172.21.15.38:0/2128583999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4be40f66a0 0x7f4be4106f60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f4bd4004e30 tx=0x7f4bd4009b30).stop 2024-02-15T18:40:04.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.415+0000 7f4bea38c700 1 -- 172.21.15.38:0/2128583999 shutdown_connections 2024-02-15T18:40:04.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.415+0000 7f4bea38c700 1 --2- 172.21.15.38:0/2128583999 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4bcc060f70 0x7f4bcc063430 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:04.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.415+0000 7f4bea38c700 1 --2- 172.21.15.38:0/2128583999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4be40f66a0 0x7f4be4106f60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:04.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.415+0000 7f4bea38c700 1 --2- 172.21.15.38:0/2128583999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4be40f7940 0x7f4be4107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:04.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.415+0000 7f4bea38c700 1 -- 172.21.15.38:0/2128583999 >> 172.21.15.38:0/2128583999 conn(0x7f4be40f1e00 msgr2=0x7f4be40fb390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:04.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.415+0000 7f4bea38c700 1 -- 172.21.15.38:0/2128583999 shutdown_connections 2024-02-15T18:40:04.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.415+0000 7f4bea38c700 1 -- 172.21.15.38:0/2128583999 wait complete. 2024-02-15T18:40:04.415 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-02-15T18:40:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.517+0000 7feac7093700 1 -- 172.21.15.38:0/792402059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feac00f7980 msgr2=0x7feac00f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.517+0000 7feac7093700 1 --2- 172.21.15.38:0/792402059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feac00f7980 0x7feac00f7de0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7feaac004660 tx=0x7feaac009b30).stop 2024-02-15T18:40:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.517+0000 7feac7093700 1 -- 172.21.15.38:0/792402059 shutdown_connections 2024-02-15T18:40:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.517+0000 7feac7093700 1 --2- 172.21.15.38:0/792402059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feac00f7980 0x7feac00f7de0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.517+0000 7feac7093700 1 --2- 172.21.15.38:0/792402059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feac00f66e0 0x7feac00f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.517+0000 7feac7093700 1 -- 172.21.15.38:0/792402059 >> 172.21.15.38:0/792402059 conn(0x7feac00f1e40 msgr2=0x7feac00f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.517+0000 7feac7093700 1 -- 172.21.15.38:0/792402059 shutdown_connections 2024-02-15T18:40:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.517+0000 7feac7093700 1 -- 172.21.15.38:0/792402059 wait complete. 2024-02-15T18:40:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.517+0000 7feac7093700 1 Processor -- start 2024-02-15T18:40:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.518+0000 7feac7093700 1 -- start start 2024-02-15T18:40:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.518+0000 7feac7093700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feac00f66e0 0x7feac0107000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.518+0000 7feac7093700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feac00f7980 0x7feac01074f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.518+0000 7feac7093700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac0107aa0 con 0x7feac00f66e0 2024-02-15T18:40:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.518+0000 7feac7093700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac0107be0 con 0x7feac00f7980 2024-02-15T18:40:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.518+0000 7feac4e2f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feac00f66e0 0x7feac0107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.518+0000 7feac4e2f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feac00f66e0 0x7feac0107000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57984/0 (socket says 172.21.15.38:57984) 2024-02-15T18:40:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.518+0000 7feac4e2f700 1 -- 172.21.15.38:0/1786463224 learned_addr learned my addr 172.21.15.38:0/1786463224 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:40:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.518+0000 7feabffff700 1 --2- 172.21.15.38:0/1786463224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feac00f7980 0x7feac01074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.518+0000 7feac4e2f700 1 -- 172.21.15.38:0/1786463224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feac00f7980 msgr2=0x7feac01074f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.518+0000 7feac4e2f700 1 --2- 172.21.15.38:0/1786463224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feac00f7980 0x7feac01074f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.518+0000 7feac4e2f700 1 -- 172.21.15.38:0/1786463224 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaac005040 con 0x7feac00f66e0 2024-02-15T18:40:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.518+0000 7feac4e2f700 1 --2- 172.21.15.38:0/1786463224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feac00f66e0 0x7feac0107000 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7feab4002700 tx=0x7feab4009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.519+0000 7feabd7fa700 1 -- 172.21.15.38:0/1786463224 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feab40079c0 con 0x7feac00f66e0 2024-02-15T18:40:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.519+0000 7feabd7fa700 1 -- 172.21.15.38:0/1786463224 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7feab400f040 con 0x7feac00f66e0 2024-02-15T18:40:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.519+0000 7feac7093700 1 -- 172.21.15.38:0/1786463224 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac0108580 con 0x7feac00f66e0 2024-02-15T18:40:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.519+0000 7feabd7fa700 1 -- 172.21.15.38:0/1786463224 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feab4014990 con 0x7feac00f66e0 2024-02-15T18:40:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.519+0000 7feac7093700 1 -- 172.21.15.38:0/1786463224 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac0108ac0 con 0x7feac00f66e0 2024-02-15T18:40:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.520+0000 7feac7093700 1 -- 172.21.15.38:0/1786463224 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac0045ad0 con 0x7feac00f66e0 2024-02-15T18:40:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.520+0000 7feabd7fa700 1 -- 172.21.15.38:0/1786463224 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7feab401b060 con 0x7feac00f66e0 2024-02-15T18:40:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.521+0000 7feabd7fa700 1 --2- 172.21.15.38:0/1786463224 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7feab0060f20 0x7feab00633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.521+0000 7feabd7fa700 1 -- 172.21.15.38:0/1786463224 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5812+0+0 (secure 0 0 0) 0x7feab407bfd0 con 0x7feac00f66e0 2024-02-15T18:40:04.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.521+0000 7feabffff700 1 --2- 172.21.15.38:0/1786463224 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7feab0060f20 0x7feab00633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:04.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.521+0000 7feabffff700 1 --2- 172.21.15.38:0/1786463224 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7feab0060f20 0x7feab00633e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7feaac004630 tx=0x7feaac009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:04.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.525+0000 7feabd7fa700 1 -- 172.21.15.38:0/1786463224 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7feab404fe80 con 0x7feac00f66e0 2024-02-15T18:40:04.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.715+0000 7feac7093700 1 -- 172.21.15.38:0/1786463224 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feac0108d40 con 0x7feab0060f20 2024-02-15T18:40:04.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.716+0000 7feabd7fa700 1 -- 172.21.15.38:0/1786463224 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7feac0108d40 con 0x7feab0060f20 2024-02-15T18:40:04.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.718+0000 7feac7093700 1 -- 172.21.15.38:0/1786463224 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7feab0060f20 msgr2=0x7feab00633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:04.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.718+0000 7feac7093700 1 --2- 172.21.15.38:0/1786463224 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7feab0060f20 0x7feab00633e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7feaac004630 tx=0x7feaac009b30).stop 2024-02-15T18:40:04.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.718+0000 7feac7093700 1 -- 172.21.15.38:0/1786463224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feac00f66e0 msgr2=0x7feac0107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:04.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.718+0000 7feac7093700 1 --2- 172.21.15.38:0/1786463224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feac00f66e0 0x7feac0107000 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7feab4002700 tx=0x7feab4009e00).stop 2024-02-15T18:40:04.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.718+0000 7feac7093700 1 -- 172.21.15.38:0/1786463224 shutdown_connections 2024-02-15T18:40:04.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.718+0000 7feac7093700 1 --2- 172.21.15.38:0/1786463224 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7feab0060f20 0x7feab00633e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:04.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.718+0000 7feac7093700 1 --2- 172.21.15.38:0/1786463224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feac00f66e0 0x7feac0107000 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:04.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.718+0000 7feac7093700 1 --2- 172.21.15.38:0/1786463224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feac00f7980 0x7feac01074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:04.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.718+0000 7feac7093700 1 -- 172.21.15.38:0/1786463224 >> 172.21.15.38:0/1786463224 conn(0x7feac00f1e40 msgr2=0x7feac00fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:04.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.718+0000 7feac7093700 1 -- 172.21.15.38:0/1786463224 shutdown_connections 2024-02-15T18:40:04.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.718+0000 7feac7093700 1 -- 172.21.15.38:0/1786463224 wait complete. 2024-02-15T18:40:04.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.819+0000 7ff9d759e700 1 -- 172.21.15.38:0/3233491368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d80f5440 msgr2=0x7ff9d80f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:04.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.819+0000 7ff9d759e700 1 --2- 172.21.15.38:0/3233491368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d80f5440 0x7ff9d80f5860 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7ff9c8004660 tx=0x7ff9c8009b30).stop 2024-02-15T18:40:04.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.819+0000 7ff9d759e700 1 -- 172.21.15.38:0/3233491368 shutdown_connections 2024-02-15T18:40:04.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.819+0000 7ff9d759e700 1 --2- 172.21.15.38:0/3233491368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9d80f66e0 0x7ff9d80f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:04.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.819+0000 7ff9d759e700 1 --2- 172.21.15.38:0/3233491368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d80f5440 0x7ff9d80f5860 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:04.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.819+0000 7ff9d759e700 1 -- 172.21.15.38:0/3233491368 >> 172.21.15.38:0/3233491368 conn(0x7ff9d80f0ba0 msgr2=0x7ff9d80f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:04.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.819+0000 7ff9d759e700 1 -- 172.21.15.38:0/3233491368 shutdown_connections 2024-02-15T18:40:04.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.820+0000 7ff9d759e700 1 -- 172.21.15.38:0/3233491368 wait complete. 2024-02-15T18:40:04.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.820+0000 7ff9d759e700 1 Processor -- start 2024-02-15T18:40:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.820+0000 7ff9d759e700 1 -- start start 2024-02-15T18:40:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.820+0000 7ff9d759e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9d80f5440 0x7ff9d8109af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.820+0000 7ff9d759e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d80f66e0 0x7ff9d810bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.821+0000 7ff9d759e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d80603e0 con 0x7ff9d80f66e0 2024-02-15T18:40:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.821+0000 7ff9d759e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d8060520 con 0x7ff9d80f5440 2024-02-15T18:40:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.821+0000 7ff9d5d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d80f66e0 0x7ff9d810bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.821+0000 7ff9d659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9d80f5440 0x7ff9d8109af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.821+0000 7ff9d5d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d80f66e0 0x7ff9d810bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58002/0 (socket says 172.21.15.38:58002) 2024-02-15T18:40:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.821+0000 7ff9d5d9b700 1 -- 172.21.15.38:0/3957412073 learned_addr learned my addr 172.21.15.38:0/3957412073 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:40:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.821+0000 7ff9d5d9b700 1 -- 172.21.15.38:0/3957412073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9d80f5440 msgr2=0x7ff9d8109af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.821+0000 7ff9d5d9b700 1 --2- 172.21.15.38:0/3957412073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9d80f5440 0x7ff9d8109af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.821+0000 7ff9d5d9b700 1 -- 172.21.15.38:0/3957412073 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c8005040 con 0x7ff9d80f66e0 2024-02-15T18:40:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.821+0000 7ff9d5d9b700 1 --2- 172.21.15.38:0/3957412073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d80f66e0 0x7ff9d810bff0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7ff9cc0026d0 tx=0x7ff9cc009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.821+0000 7ff9c6ffd700 1 -- 172.21.15.38:0/3957412073 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9cc0079c0 con 0x7ff9d80f66e0 2024-02-15T18:40:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.821+0000 7ff9c6ffd700 1 -- 172.21.15.38:0/3957412073 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7ff9cc00f040 con 0x7ff9d80f66e0 2024-02-15T18:40:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.821+0000 7ff9d759e700 1 -- 172.21.15.38:0/3957412073 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d810c5a0 con 0x7ff9d80f66e0 2024-02-15T18:40:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.821+0000 7ff9c6ffd700 1 -- 172.21.15.38:0/3957412073 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9cc0149f0 con 0x7ff9d80f66e0 2024-02-15T18:40:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.821+0000 7ff9d759e700 1 -- 172.21.15.38:0/3957412073 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d810cb60 con 0x7ff9d80f66e0 2024-02-15T18:40:04.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.822+0000 7ff9d759e700 1 -- 172.21.15.38:0/3957412073 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d8045ad0 con 0x7ff9d80f66e0 2024-02-15T18:40:04.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.823+0000 7ff9c6ffd700 1 -- 172.21.15.38:0/3957412073 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7ff9cc007b20 con 0x7ff9d80f66e0 2024-02-15T18:40:04.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.823+0000 7ff9c6ffd700 1 --2- 172.21.15.38:0/3957412073 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7ff9c0060f70 0x7ff9c0063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:04.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.823+0000 7ff9c6ffd700 1 -- 172.21.15.38:0/3957412073 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5812+0+0 (secure 0 0 0) 0x7ff9cc07c070 con 0x7ff9d80f66e0 2024-02-15T18:40:04.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.823+0000 7ff9d659c700 1 --2- 172.21.15.38:0/3957412073 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7ff9c0060f70 0x7ff9c0063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:04.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.824+0000 7ff9d659c700 1 --2- 172.21.15.38:0/3957412073 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7ff9c0060f70 0x7ff9c0063430 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7ff9c8005010 tx=0x7ff9c8009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:04.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:04.828+0000 7ff9c6ffd700 1 -- 172.21.15.38:0/3957412073 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff9cc04c620 con 0x7ff9d80f66e0 2024-02-15T18:40:04.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.009+0000 7ff9d759e700 1 -- 172.21.15.38:0/3957412073 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9d8057f10 con 0x7ff9c0060f70 2024-02-15T18:40:05.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.019+0000 7ff9c6ffd700 1 -- 172.21.15.38:0/3957412073 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7ff9d8057f10 con 0x7ff9c0060f70 2024-02-15T18:40:05.009 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:40:05.009 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (10m) 88s ago 12m 19.3M - 0.20.0 0881eb8f169f c2034c318e7c 2024-02-15T18:40:05.009 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (3m) 88s ago 12m 6635k - 16.2.14-521-g0e714d9a 1fbc92c859dc c2fee0824d4b 2024-02-15T18:40:05.009 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi121 smithi121 running (2m) 3s ago 11m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 05c1a5501a35 2024-02-15T18:40:05.009 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (10m) 88s ago 12m 33.4M - 6.7.4 557c83e11646 2dcccfbfbd2a 2024-02-15T18:40:05.009 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.hxoggl smithi038 running (7m) 88s ago 7m 14.8M - 16.2.4 8d91d370c2b8 be2533754b9c 2024-02-15T18:40:05.009 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.qgkzgn smithi038 running (7m) 88s ago 7m 14.4M - 16.2.4 8d91d370c2b8 dd3ffafba9bd 2024-02-15T18:40:05.009 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.mvumlh smithi121 running (7m) 3s ago 7m 116M - 16.2.4 8d91d370c2b8 88b7d876a39b 2024-02-15T18:40:05.009 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.ovuzki smithi121 running (7m) 3s ago 7m 124M - 16.2.4 8d91d370c2b8 21cacf0012f9 2024-02-15T18:40:05.009 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.khesbl smithi038 *:8443,9283 running (4m) 88s ago 14m 433M - 16.2.14-521-g0e714d9a 1fbc92c859dc e642f3c2cfc9 2024-02-15T18:40:05.009 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi121.ckiryi smithi121 *:8443,9283 running (3m) 3s ago 11m 387M - 16.2.14-521-g0e714d9a 1fbc92c859dc 82515b025594 2024-02-15T18:40:05.010 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (3m) 88s ago 14m 40.8M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc d3e8317a30ca 2024-02-15T18:40:05.010 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi121 smithi121 running (3m) 3s ago 11m 43.1M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 42e1dda1a39d 2024-02-15T18:40:05.010 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (12m) 88s ago 12m 17.1M - 0.18.1 e5a616e4b9cf 037f60e38158 2024-02-15T18:40:05.010 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi121 smithi121 *:9100 running (11m) 3s ago 11m 17.2M - 0.18.1 e5a616e4b9cf a867bff85e18 2024-02-15T18:40:05.010 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (2m) 88s ago 11m 191M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 623ac7ecbb0a 2024-02-15T18:40:05.010 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (2m) 88s ago 10m 217M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 0f9e8e1f255a 2024-02-15T18:40:05.010 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (95s) 88s ago 10m 13.4M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 37171264b5be 2024-02-15T18:40:05.010 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi121 running (64s) 3s ago 9m 255M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc d505a58b375c 2024-02-15T18:40:05.010 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi121 running (36s) 3s ago 9m 259M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 0ce6b6abb99e 2024-02-15T18:40:05.010 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi121 running (7s) 3s ago 8m 13.1M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 79bb22243769 2024-02-15T18:40:05.010 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (4m) 88s ago 11m 65.9M - 2.18.1 de242295e225 67e33f9de4e6 2024-02-15T18:40:05.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.020+0000 7ff9d759e700 1 -- 172.21.15.38:0/3957412073 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7ff9c0060f70 msgr2=0x7ff9c0063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:05.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.020+0000 7ff9d759e700 1 --2- 172.21.15.38:0/3957412073 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7ff9c0060f70 0x7ff9c0063430 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7ff9c8005010 tx=0x7ff9c8009b30).stop 2024-02-15T18:40:05.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.020+0000 7ff9d759e700 1 -- 172.21.15.38:0/3957412073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d80f66e0 msgr2=0x7ff9d810bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:05.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.020+0000 7ff9d759e700 1 --2- 172.21.15.38:0/3957412073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d80f66e0 0x7ff9d810bff0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7ff9cc0026d0 tx=0x7ff9cc009de0).stop 2024-02-15T18:40:05.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.021+0000 7ff9d759e700 1 -- 172.21.15.38:0/3957412073 shutdown_connections 2024-02-15T18:40:05.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.021+0000 7ff9d759e700 1 --2- 172.21.15.38:0/3957412073 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7ff9c0060f70 0x7ff9c0063430 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:05.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.021+0000 7ff9d759e700 1 --2- 172.21.15.38:0/3957412073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9d80f5440 0x7ff9d8109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:05.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.021+0000 7ff9d759e700 1 --2- 172.21.15.38:0/3957412073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff9d80f66e0 0x7ff9d810bff0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:05.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.021+0000 7ff9d759e700 1 -- 172.21.15.38:0/3957412073 >> 172.21.15.38:0/3957412073 conn(0x7ff9d80f0ba0 msgr2=0x7ff9d80fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:05.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.021+0000 7ff9d759e700 1 -- 172.21.15.38:0/3957412073 shutdown_connections 2024-02-15T18:40:05.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.021+0000 7ff9d759e700 1 -- 172.21.15.38:0/3957412073 wait complete. 2024-02-15T18:40:05.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.122+0000 7f4db0ca2700 1 -- 172.21.15.38:0/3484664380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dac0f4a60 msgr2=0x7f4dac0f4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.122+0000 7f4db0ca2700 1 --2- 172.21.15.38:0/3484664380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dac0f4a60 0x7f4dac0f4e80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f4d9c004660 tx=0x7f4d9c009b30).stop 2024-02-15T18:40:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.122+0000 7f4db0ca2700 1 -- 172.21.15.38:0/3484664380 shutdown_connections 2024-02-15T18:40:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.122+0000 7f4db0ca2700 1 --2- 172.21.15.38:0/3484664380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dac0f5400 0x7f4dac0fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.122+0000 7f4db0ca2700 1 --2- 172.21.15.38:0/3484664380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dac0f4a60 0x7f4dac0f4e80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.122+0000 7f4db0ca2700 1 -- 172.21.15.38:0/3484664380 >> 172.21.15.38:0/3484664380 conn(0x7f4dac0f04c0 msgr2=0x7f4dac0f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.122+0000 7f4db0ca2700 1 -- 172.21.15.38:0/3484664380 shutdown_connections 2024-02-15T18:40:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.122+0000 7f4db0ca2700 1 -- 172.21.15.38:0/3484664380 wait complete. 2024-02-15T18:40:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.122+0000 7f4db0ca2700 1 Processor -- start 2024-02-15T18:40:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.122+0000 7f4db0ca2700 1 -- start start 2024-02-15T18:40:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.123+0000 7f4db0ca2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dac0f4a60 0x7f4dac0f7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.123+0000 7f4db0ca2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dac0f5400 0x7f4dac0f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.123+0000 7f4db0ca2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dac0f6030 con 0x7f4dac0f5400 2024-02-15T18:40:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.123+0000 7f4db0ca2700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dac0f6170 con 0x7f4dac0f4a60 2024-02-15T18:40:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.123+0000 7f4da9d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dac0f5400 0x7f4dac0f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:05.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.123+0000 7f4daa59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dac0f4a60 0x7f4dac0f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:05.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.123+0000 7f4da9d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dac0f5400 0x7f4dac0f5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58022/0 (socket says 172.21.15.38:58022) 2024-02-15T18:40:05.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.123+0000 7f4da9d9b700 1 -- 172.21.15.38:0/1221992094 learned_addr learned my addr 172.21.15.38:0/1221992094 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:40:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.123+0000 7f4da9d9b700 1 -- 172.21.15.38:0/1221992094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dac0f4a60 msgr2=0x7f4dac0f7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.123+0000 7f4da9d9b700 1 --2- 172.21.15.38:0/1221992094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dac0f4a60 0x7f4dac0f7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.123+0000 7f4da9d9b700 1 -- 172.21.15.38:0/1221992094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9c005040 con 0x7f4dac0f5400 2024-02-15T18:40:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.123+0000 7f4da9d9b700 1 --2- 172.21.15.38:0/1221992094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dac0f5400 0x7f4dac0f5ad0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f4d94002700 tx=0x7f4d94009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.123+0000 7f4da2ffd700 1 -- 172.21.15.38:0/1221992094 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d940079c0 con 0x7f4dac0f5400 2024-02-15T18:40:05.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.123+0000 7f4da2ffd700 1 -- 172.21.15.38:0/1221992094 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f4d9400f040 con 0x7f4dac0f5400 2024-02-15T18:40:05.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.123+0000 7f4db0ca2700 1 -- 172.21.15.38:0/1221992094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dac0f6450 con 0x7f4dac0f5400 2024-02-15T18:40:05.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.123+0000 7f4da2ffd700 1 -- 172.21.15.38:0/1221992094 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d94014940 con 0x7f4dac0f5400 2024-02-15T18:40:05.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.124+0000 7f4db0ca2700 1 -- 172.21.15.38:0/1221992094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dac10f050 con 0x7f4dac0f5400 2024-02-15T18:40:05.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.124+0000 7f4db0ca2700 1 -- 172.21.15.38:0/1221992094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dac045ad0 con 0x7f4dac0f5400 2024-02-15T18:40:05.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.125+0000 7f4da2ffd700 1 -- 172.21.15.38:0/1221992094 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f4d9401b060 con 0x7f4dac0f5400 2024-02-15T18:40:05.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.125+0000 7f4da2ffd700 1 --2- 172.21.15.38:0/1221992094 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4d98060f20 0x7f4d980633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:05.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.125+0000 7f4da2ffd700 1 -- 172.21.15.38:0/1221992094 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f4d9407bea0 con 0x7f4dac0f5400 2024-02-15T18:40:05.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.125+0000 7f4daa59c700 1 --2- 172.21.15.38:0/1221992094 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4d98060f20 0x7f4d980633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:05.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.126+0000 7f4daa59c700 1 --2- 172.21.15.38:0/1221992094 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4d98060f20 0x7f4d980633e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f4d9c004e30 tx=0x7f4d9c009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.130+0000 7f4da2ffd700 1 -- 172.21.15.38:0/1221992094 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4d9404c3a0 con 0x7f4dac0f5400 2024-02-15T18:40:05.298 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:04.414004+0000 mgr.smithi038.khesbl (mgr.14664) 213 : audit [DBG] from='client.34264 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:40:05.298 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:04.717136+0000 mgr.smithi038.khesbl (mgr.14664) 214 : audit [DBG] from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:40:05.298 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:04.981470+0000 mgr.smithi038.khesbl (mgr.14664) 215 : cluster [DBG] pgmap v152: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 54/342 objects degraded (15.789%) 2024-02-15T18:40:05.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.374+0000 7f4db0ca2700 1 -- 172.21.15.38:0/1221992094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4dac0fefc0 con 0x7f4dac0f5400 2024-02-15T18:40:05.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.374+0000 7f4da2ffd700 1 -- 172.21.15.38:0/1221992094 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+726 (secure 0 0 0) 0x7f4d9404f9c0 con 0x7f4dac0f5400 2024-02-15T18:40:05.365 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:40:05.365 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-02-15T18:40:05.365 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:40:05.365 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:40:05.365 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-02-15T18:40:05.365 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:40:05.365 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:40:05.366 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-02-15T18:40:05.366 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 5 2024-02-15T18:40:05.366 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:40:05.366 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-02-15T18:40:05.366 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:40:05.366 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:40:05.366 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-02-15T18:40:05.366 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 9, 2024-02-15T18:40:05.366 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:40:05.366 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-02-15T18:40:05.366 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:40:05.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.376+0000 7f4db0ca2700 1 -- 172.21.15.38:0/1221992094 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4d98060f20 msgr2=0x7f4d980633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:05.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.376+0000 7f4db0ca2700 1 --2- 172.21.15.38:0/1221992094 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4d98060f20 0x7f4d980633e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f4d9c004e30 tx=0x7f4d9c009b30).stop 2024-02-15T18:40:05.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.376+0000 7f4db0ca2700 1 -- 172.21.15.38:0/1221992094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dac0f5400 msgr2=0x7f4dac0f5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:05.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.376+0000 7f4db0ca2700 1 --2- 172.21.15.38:0/1221992094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dac0f5400 0x7f4dac0f5ad0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f4d94002700 tx=0x7f4d94009d80).stop 2024-02-15T18:40:05.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.376+0000 7f4db0ca2700 1 -- 172.21.15.38:0/1221992094 shutdown_connections 2024-02-15T18:40:05.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.376+0000 7f4db0ca2700 1 --2- 172.21.15.38:0/1221992094 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4d98060f20 0x7f4d980633e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:05.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.376+0000 7f4db0ca2700 1 --2- 172.21.15.38:0/1221992094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4dac0f4a60 0x7f4dac0f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:05.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.376+0000 7f4db0ca2700 1 --2- 172.21.15.38:0/1221992094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4dac0f5400 0x7f4dac0f5ad0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:05.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.376+0000 7f4db0ca2700 1 -- 172.21.15.38:0/1221992094 >> 172.21.15.38:0/1221992094 conn(0x7f4dac0f04c0 msgr2=0x7f4dac0f8510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:05.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.376+0000 7f4db0ca2700 1 -- 172.21.15.38:0/1221992094 shutdown_connections 2024-02-15T18:40:05.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.376+0000 7f4db0ca2700 1 -- 172.21.15.38:0/1221992094 wait complete. 2024-02-15T18:40:05.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.477+0000 7f42ae496700 1 -- 172.21.15.38:0/3844845017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42a80f6860 msgr2=0x7f42a80f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:05.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.477+0000 7f42ae496700 1 --2- 172.21.15.38:0/3844845017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42a80f6860 0x7f42a80f6cc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f429c004660 tx=0x7f429c009b30).stop 2024-02-15T18:40:05.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.477+0000 7f42ae496700 1 -- 172.21.15.38:0/3844845017 shutdown_connections 2024-02-15T18:40:05.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.477+0000 7f42ae496700 1 --2- 172.21.15.38:0/3844845017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42a80f6860 0x7f42a80f6cc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:05.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.477+0000 7f42ae496700 1 --2- 172.21.15.38:0/3844845017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42a80f55c0 0x7f42a80f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:05.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.477+0000 7f42ae496700 1 -- 172.21.15.38:0/3844845017 >> 172.21.15.38:0/3844845017 conn(0x7f42a80f0d40 msgr2=0x7f42a80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:05.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.477+0000 7f42ae496700 1 -- 172.21.15.38:0/3844845017 shutdown_connections 2024-02-15T18:40:05.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.477+0000 7f42ae496700 1 -- 172.21.15.38:0/3844845017 wait complete. 2024-02-15T18:40:05.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.478+0000 7f42ae496700 1 Processor -- start 2024-02-15T18:40:05.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.478+0000 7f42ae496700 1 -- start start 2024-02-15T18:40:05.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.478+0000 7f42ae496700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42a80f55c0 0x7f42a8109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:05.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.478+0000 7f42ae496700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42a80f6860 0x7f42a810c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:05.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.478+0000 7f42ae496700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80603c0 con 0x7f42a80f55c0 2024-02-15T18:40:05.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.478+0000 7f42ae496700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a8060500 con 0x7f42a80f6860 2024-02-15T18:40:05.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.478+0000 7f42a7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42a80f55c0 0x7f42a8109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:05.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.478+0000 7f42a7fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42a80f55c0 0x7f42a8109cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58046/0 (socket says 172.21.15.38:58046) 2024-02-15T18:40:05.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.478+0000 7f42a7fff700 1 -- 172.21.15.38:0/1178097641 learned_addr learned my addr 172.21.15.38:0/1178097641 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:40:05.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.478+0000 7f42a77fe700 1 --2- 172.21.15.38:0/1178097641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42a80f6860 0x7f42a810c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:05.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.479+0000 7f42a7fff700 1 -- 172.21.15.38:0/1178097641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42a80f6860 msgr2=0x7f42a810c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:05.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.479+0000 7f42a7fff700 1 --2- 172.21.15.38:0/1178097641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42a80f6860 0x7f42a810c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:05.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.479+0000 7f42a7fff700 1 -- 172.21.15.38:0/1178097641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429c005040 con 0x7f42a80f55c0 2024-02-15T18:40:05.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.479+0000 7f42a7fff700 1 --2- 172.21.15.38:0/1178097641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42a80f55c0 0x7f42a8109cd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f4298002700 tx=0x7f4298009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:05.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.479+0000 7f42a4ff9700 1 -- 172.21.15.38:0/1178097641 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42980079c0 con 0x7f42a80f55c0 2024-02-15T18:40:05.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.479+0000 7f42ae496700 1 -- 172.21.15.38:0/1178097641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a810c780 con 0x7f42a80f55c0 2024-02-15T18:40:05.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.479+0000 7f42a4ff9700 1 -- 172.21.15.38:0/1178097641 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f429800f040 con 0x7f42a80f55c0 2024-02-15T18:40:05.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.479+0000 7f42a4ff9700 1 -- 172.21.15.38:0/1178097641 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4298014940 con 0x7f42a80f55c0 2024-02-15T18:40:05.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.479+0000 7f42ae496700 1 -- 172.21.15.38:0/1178097641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a810cd10 con 0x7f42a80f55c0 2024-02-15T18:40:05.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.480+0000 7f42ae496700 1 -- 172.21.15.38:0/1178097641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a8045ad0 con 0x7f42a80f55c0 2024-02-15T18:40:05.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.481+0000 7f42a4ff9700 1 -- 172.21.15.38:0/1178097641 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f429801b060 con 0x7f42a80f55c0 2024-02-15T18:40:05.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.481+0000 7f42a4ff9700 1 --2- 172.21.15.38:0/1178097641 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4290060f20 0x7f42900633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:05.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.481+0000 7f42a4ff9700 1 -- 172.21.15.38:0/1178097641 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f429807bf20 con 0x7f42a80f55c0 2024-02-15T18:40:05.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.481+0000 7f42a77fe700 1 --2- 172.21.15.38:0/1178097641 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4290060f20 0x7f42900633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:05.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.481+0000 7f42a77fe700 1 --2- 172.21.15.38:0/1178097641 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4290060f20 0x7f42900633e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f429c004630 tx=0x7f429c009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:05.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.485+0000 7f42a4ff9700 1 -- 172.21.15.38:0/1178097641 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f429804c3a0 con 0x7f42a80f55c0 2024-02-15T18:40:05.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:04.414004+0000 mgr.smithi038.khesbl (mgr.14664) 213 : audit [DBG] from='client.34264 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:40:05.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:04.717136+0000 mgr.smithi038.khesbl (mgr.14664) 214 : audit [DBG] from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:40:05.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:04.981470+0000 mgr.smithi038.khesbl (mgr.14664) 215 : cluster [DBG] pgmap v152: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 293 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 54/342 objects degraded (15.789%) 2024-02-15T18:40:05.605 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:40:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:40:05.308+0000 7f8a05c98380 -1 osd.5 71 log_to_monitors {default=true} 2024-02-15T18:40:05.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.705+0000 7f42ae496700 1 -- 172.21.15.38:0/1178097641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f42a80fdfa0 con 0x7f42a80f55c0 2024-02-15T18:40:05.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.706+0000 7f42a4ff9700 1 -- 172.21.15.38:0/1178097641 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1775 (secure 0 0 0) 0x7f429804f9c0 con 0x7f42a80f55c0 2024-02-15T18:40:05.696 INFO:teuthology.orchestra.run.smithi038.stdout:e16 2024-02-15T18:40:05.696 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:40:05.696 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:40:05.696 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-02-15T18:40:05.696 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:40:05.696 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:40:05.696 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-02-15T18:40:05.696 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 16 2024-02-15T18:40:05.696 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 2024-02-15T18:40:05.696 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:40:05.697 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-02-15T18:40:02.254242+0000 2024-02-15T18:40:05.697 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-02-15T18:40:05.697 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-02-15T18:40:05.697 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-02-15T18:40:05.697 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-02-15T18:40:05.697 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-02-15T18:40:05.697 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-02-15T18:40:05.697 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-02-15T18:40:05.697 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-02-15T18:40:05.697 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:40:05.697 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 1 2024-02-15T18:40:05.697 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-02-15T18:40:05.697 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=24261,1=24275} 2024-02-15T18:40:05.697 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-02-15T18:40:05.698 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-02-15T18:40:05.698 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-02-15T18:40:05.698 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-02-15T18:40:05.698 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-02-15T18:40:05.698 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-02-15T18:40:05.698 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-02-15T18:40:05.698 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-02-15T18:40:05.698 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.mvumlh{0:24261} state up:active seq 3 join_fscid=1 addr [v2:172.21.15.121:6824/1422707571,v1:172.21.15.121:6825/1422707571] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:40:05.698 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.ovuzki{1:24275} state up:stopping seq 2 join_fscid=1 addr [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:40:05.698 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:40:05.698 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:40:05.698 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-02-15T18:40:05.698 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:40:05.698 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.hxoggl{-1:14446} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.38:6826/3822880235,v1:172.21.15.38:6827/3822880235] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:40:05.699 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.qgkzgn{-1:14456} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.38:6828/703030231,v1:172.21.15.38:6829/703030231] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:40:05.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.707+0000 7f42ae496700 1 -- 172.21.15.38:0/1178097641 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4290060f20 msgr2=0x7f42900633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:05.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.707+0000 7f42ae496700 1 --2- 172.21.15.38:0/1178097641 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4290060f20 0x7f42900633e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f429c004630 tx=0x7f429c009b30).stop 2024-02-15T18:40:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.707+0000 7f42ae496700 1 -- 172.21.15.38:0/1178097641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42a80f55c0 msgr2=0x7f42a8109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.707+0000 7f42ae496700 1 --2- 172.21.15.38:0/1178097641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42a80f55c0 0x7f42a8109cd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f4298002700 tx=0x7f4298009d80).stop 2024-02-15T18:40:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.707+0000 7f42ae496700 1 -- 172.21.15.38:0/1178097641 shutdown_connections 2024-02-15T18:40:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.707+0000 7f42ae496700 1 --2- 172.21.15.38:0/1178097641 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4290060f20 0x7f42900633e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.707+0000 7f42ae496700 1 --2- 172.21.15.38:0/1178097641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42a80f55c0 0x7f42a8109cd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.707+0000 7f42ae496700 1 --2- 172.21.15.38:0/1178097641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42a80f6860 0x7f42a810c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.707+0000 7f42ae496700 1 -- 172.21.15.38:0/1178097641 >> 172.21.15.38:0/1178097641 conn(0x7f42a80f0d40 msgr2=0x7f42a80fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.708+0000 7f42ae496700 1 -- 172.21.15.38:0/1178097641 shutdown_connections 2024-02-15T18:40:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.708+0000 7f42ae496700 1 -- 172.21.15.38:0/1178097641 wait complete. 2024-02-15T18:40:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 16 2024-02-15T18:40:05.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.810+0000 7f777a4d0700 1 -- 172.21.15.38:0/2112199418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77740f66e0 msgr2=0x7f77740f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:05.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.810+0000 7f777a4d0700 1 --2- 172.21.15.38:0/2112199418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77740f66e0 0x7f77740f6b00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f7764004660 tx=0x7f7764009b30).stop 2024-02-15T18:40:05.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.810+0000 7f777a4d0700 1 -- 172.21.15.38:0/2112199418 shutdown_connections 2024-02-15T18:40:05.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.810+0000 7f777a4d0700 1 --2- 172.21.15.38:0/2112199418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77740f7980 0x7f77740f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:05.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.810+0000 7f777a4d0700 1 --2- 172.21.15.38:0/2112199418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77740f66e0 0x7f77740f6b00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:05.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.810+0000 7f777a4d0700 1 -- 172.21.15.38:0/2112199418 >> 172.21.15.38:0/2112199418 conn(0x7f77740f1e40 msgr2=0x7f77740f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:05.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.810+0000 7f777a4d0700 1 -- 172.21.15.38:0/2112199418 shutdown_connections 2024-02-15T18:40:05.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.810+0000 7f777a4d0700 1 -- 172.21.15.38:0/2112199418 wait complete. 2024-02-15T18:40:05.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.810+0000 7f777a4d0700 1 Processor -- start 2024-02-15T18:40:05.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.810+0000 7f777a4d0700 1 -- start start 2024-02-15T18:40:05.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.811+0000 7f777a4d0700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77740f66e0 0x7f7774107030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:05.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.811+0000 7f777a4d0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77740f7980 0x7f7774107520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:05.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.811+0000 7f777a4d0700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7774107ad0 con 0x7f77740f7980 2024-02-15T18:40:05.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.811+0000 7f777a4d0700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7774107c10 con 0x7f77740f66e0 2024-02-15T18:40:05.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.811+0000 7f77737fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77740f7980 0x7f7774107520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:05.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.811+0000 7f77737fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77740f7980 0x7f7774107520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58064/0 (socket says 172.21.15.38:58064) 2024-02-15T18:40:05.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.811+0000 7f77737fe700 1 -- 172.21.15.38:0/1049720957 learned_addr learned my addr 172.21.15.38:0/1049720957 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:40:05.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.811+0000 7f7773fff700 1 --2- 172.21.15.38:0/1049720957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77740f66e0 0x7f7774107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:05.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.811+0000 7f77737fe700 1 -- 172.21.15.38:0/1049720957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77740f66e0 msgr2=0x7f7774107030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:05.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.811+0000 7f77737fe700 1 --2- 172.21.15.38:0/1049720957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77740f66e0 0x7f7774107030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:05.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.811+0000 7f77737fe700 1 -- 172.21.15.38:0/1049720957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7764005040 con 0x7f77740f7980 2024-02-15T18:40:05.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.811+0000 7f77737fe700 1 --2- 172.21.15.38:0/1049720957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77740f7980 0x7f7774107520 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f775c0026d0 tx=0x7f775c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:05.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.811+0000 7f7770ff9700 1 -- 172.21.15.38:0/1049720957 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f775c0079c0 con 0x7f77740f7980 2024-02-15T18:40:05.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.811+0000 7f777a4d0700 1 -- 172.21.15.38:0/1049720957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77741085b0 con 0x7f77740f7980 2024-02-15T18:40:05.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.811+0000 7f7770ff9700 1 -- 172.21.15.38:0/1049720957 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f775c00f040 con 0x7f77740f7980 2024-02-15T18:40:05.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.811+0000 7f7770ff9700 1 -- 172.21.15.38:0/1049720957 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f775c0149f0 con 0x7f77740f7980 2024-02-15T18:40:05.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.812+0000 7f777a4d0700 1 -- 172.21.15.38:0/1049720957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7774108b70 con 0x7f77740f7980 2024-02-15T18:40:05.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.813+0000 7f777a4d0700 1 -- 172.21.15.38:0/1049720957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7774045ad0 con 0x7f77740f7980 2024-02-15T18:40:05.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.813+0000 7f7770ff9700 1 -- 172.21.15.38:0/1049720957 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f775c007b20 con 0x7f77740f7980 2024-02-15T18:40:05.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.813+0000 7f7770ff9700 1 --2- 172.21.15.38:0/1049720957 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f7760060f20 0x7f77600633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:05.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.813+0000 7f7770ff9700 1 -- 172.21.15.38:0/1049720957 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f775c07c0e0 con 0x7f77740f7980 2024-02-15T18:40:05.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.813+0000 7f7773fff700 1 --2- 172.21.15.38:0/1049720957 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f7760060f20 0x7f77600633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:05.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.814+0000 7f7773fff700 1 --2- 172.21.15.38:0/1049720957 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f7760060f20 0x7f77600633e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f7764004630 tx=0x7f7764009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:05.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:05.817+0000 7f7770ff9700 1 -- 172.21.15.38:0/1049720957 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f775c04ff90 con 0x7f77740f7980 2024-02-15T18:40:05.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:06.006+0000 7f777a4d0700 1 -- 172.21.15.38:0/1049720957 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7774108df0 con 0x7f7760060f20 2024-02-15T18:40:05.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:06.007+0000 7f7770ff9700 1 -- 172.21.15.38:0/1049720957 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7774108df0 con 0x7f7760060f20 2024-02-15T18:40:05.997 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:40:05.997 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:40:05.997 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-02-15T18:40:05.997 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:40:05.998 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-02-15T18:40:05.998 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-02-15T18:40:05.998 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-02-15T18:40:05.998 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr", 2024-02-15T18:40:05.998 INFO:teuthology.orchestra.run.smithi038.stdout: "osd" 2024-02-15T18:40:05.998 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-02-15T18:40:05.998 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "12/21 daemons upgraded", 2024-02-15T18:40:05.998 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading osd daemons", 2024-02-15T18:40:05.998 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-02-15T18:40:05.998 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:40:05.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:06.009+0000 7f777a4d0700 1 -- 172.21.15.38:0/1049720957 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f7760060f20 msgr2=0x7f77600633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:05.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:06.009+0000 7f777a4d0700 1 --2- 172.21.15.38:0/1049720957 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f7760060f20 0x7f77600633e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f7764004630 tx=0x7f7764009b30).stop 2024-02-15T18:40:05.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:06.009+0000 7f777a4d0700 1 -- 172.21.15.38:0/1049720957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77740f7980 msgr2=0x7f7774107520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:05.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:06.009+0000 7f777a4d0700 1 --2- 172.21.15.38:0/1049720957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77740f7980 0x7f7774107520 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f775c0026d0 tx=0x7f775c009de0).stop 2024-02-15T18:40:05.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:06.009+0000 7f777a4d0700 1 -- 172.21.15.38:0/1049720957 shutdown_connections 2024-02-15T18:40:05.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:06.009+0000 7f777a4d0700 1 --2- 172.21.15.38:0/1049720957 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f7760060f20 0x7f77600633e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:05.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:06.009+0000 7f777a4d0700 1 --2- 172.21.15.38:0/1049720957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77740f66e0 0x7f7774107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:06.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:06.009+0000 7f777a4d0700 1 --2- 172.21.15.38:0/1049720957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77740f7980 0x7f7774107520 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:06.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:06.009+0000 7f777a4d0700 1 -- 172.21.15.38:0/1049720957 >> 172.21.15.38:0/1049720957 conn(0x7f77740f1e40 msgr2=0x7f77740fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:06.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:06.009+0000 7f777a4d0700 1 -- 172.21.15.38:0/1049720957 shutdown_connections 2024-02-15T18:40:06.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:06.009+0000 7f777a4d0700 1 -- 172.21.15.38:0/1049720957 wait complete. 2024-02-15T18:40:06.355 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:40:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:40:06.085+0000 7f89f517b700 -1 osd.5 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-15T18:40:06.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:05.011621+0000 mgr.smithi038.khesbl (mgr.14664) 216 : audit [DBG] from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:40:06.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:05.312758+0000 mon.smithi121 (mon.1) 6 : audit [INF] from='osd.5 [v2:172.21.15.121:6816/3030235922,v1:172.21.15.121:6817/3030235922]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-15T18:40:06.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:05.313173+0000 mon.smithi038 (mon.0) 412 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-15T18:40:06.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:05.375701+0000 mon.smithi038 (mon.0) 413 : audit [DBG] from='client.? 172.21.15.38:0/1221992094' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:06.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:05.707388+0000 mon.smithi038 (mon.0) 414 : audit [DBG] from='client.? 172.21.15.38:0/1178097641' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:40:06.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:05.011621+0000 mgr.smithi038.khesbl (mgr.14664) 216 : audit [DBG] from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:40:06.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:05.312758+0000 mon.smithi121 (mon.1) 6 : audit [INF] from='osd.5 [v2:172.21.15.121:6816/3030235922,v1:172.21.15.121:6817/3030235922]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-15T18:40:06.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:05.313173+0000 mon.smithi038 (mon.0) 412 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-15T18:40:06.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:05.375701+0000 mon.smithi038 (mon.0) 413 : audit [DBG] from='client.? 172.21.15.38:0/1221992094' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:06.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:05.707388+0000 mon.smithi038 (mon.0) 414 : audit [DBG] from='client.? 172.21.15.38:0/1178097641' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:40:07.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:06.008674+0000 mgr.smithi038.khesbl (mgr.14664) 217 : audit [DBG] from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:40:07.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:06.076704+0000 mon.smithi038 (mon.0) 415 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-15T18:40:07.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:06.076752+0000 mon.smithi038 (mon.0) 416 : cluster [DBG] osdmap e74: 6 total, 5 up, 6 in 2024-02-15T18:40:07.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:06.077394+0000 mon.smithi121 (mon.1) 7 : audit [INF] from='osd.5 [v2:172.21.15.121:6816/3030235922,v1:172.21.15.121:6817/3030235922]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:40:07.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:06.077865+0000 mon.smithi038 (mon.0) 417 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:40:07.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:06.982122+0000 mgr.smithi038.khesbl (mgr.14664) 218 : cluster [DBG] pgmap v154: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 293 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 54/342 objects degraded (15.789%) 2024-02-15T18:40:07.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:06.008674+0000 mgr.smithi038.khesbl (mgr.14664) 217 : audit [DBG] from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:40:07.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:06.076704+0000 mon.smithi038 (mon.0) 415 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-15T18:40:07.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:06.076752+0000 mon.smithi038 (mon.0) 416 : cluster [DBG] osdmap e74: 6 total, 5 up, 6 in 2024-02-15T18:40:07.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:06.077394+0000 mon.smithi121 (mon.1) 7 : audit [INF] from='osd.5 [v2:172.21.15.121:6816/3030235922,v1:172.21.15.121:6817/3030235922]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:40:07.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:06.077865+0000 mon.smithi038 (mon.0) 417 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-02-15T18:40:07.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:06.982122+0000 mgr.smithi038.khesbl (mgr.14664) 218 : cluster [DBG] pgmap v154: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 293 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 54/342 objects degraded (15.789%) 2024-02-15T18:40:08.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:07.071018+0000 mon.smithi038 (mon.0) 418 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:40:08.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:07.080760+0000 mon.smithi038 (mon.0) 419 : cluster [INF] osd.5 [v2:172.21.15.121:6816/3030235922,v1:172.21.15.121:6817/3030235922] boot 2024-02-15T18:40:08.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:07.080824+0000 mon.smithi038 (mon.0) 420 : cluster [DBG] osdmap e75: 6 total, 6 up, 6 in 2024-02-15T18:40:08.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:07.083319+0000 mon.smithi038 (mon.0) 421 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:40:08.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:07.255685+0000 mon.smithi038 (mon.0) 422 : cluster [WRN] Health check update: Degraded data redundancy: 54/342 objects degraded (15.789%), 14 pgs degraded (PG_DEGRADED) 2024-02-15T18:40:08.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:08.078713+0000 mon.smithi038 (mon.0) 423 : cluster [DBG] osdmap e76: 6 total, 6 up, 6 in 2024-02-15T18:40:08.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:07.071018+0000 mon.smithi038 (mon.0) 418 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:40:08.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:07.080760+0000 mon.smithi038 (mon.0) 419 : cluster [INF] osd.5 [v2:172.21.15.121:6816/3030235922,v1:172.21.15.121:6817/3030235922] boot 2024-02-15T18:40:08.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:07.080824+0000 mon.smithi038 (mon.0) 420 : cluster [DBG] osdmap e75: 6 total, 6 up, 6 in 2024-02-15T18:40:08.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:07.083319+0000 mon.smithi038 (mon.0) 421 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:40:08.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:07.255685+0000 mon.smithi038 (mon.0) 422 : cluster [WRN] Health check update: Degraded data redundancy: 54/342 objects degraded (15.789%), 14 pgs degraded (PG_DEGRADED) 2024-02-15T18:40:08.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:08.078713+0000 mon.smithi038 (mon.0) 423 : cluster [DBG] osdmap e76: 6 total, 6 up, 6 in 2024-02-15T18:40:09.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:08.252918+0000 mon.smithi038 (mon.0) 424 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:09.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:08.982626+0000 mgr.smithi038.khesbl (mgr.14664) 219 : cluster [DBG] pgmap v157: 65 pgs: 1 activating, 2 peering, 10 active+undersized+degraded, 11 active+undersized, 41 active+clean; 293 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 44/342 objects degraded (12.865%) 2024-02-15T18:40:09.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:08.252918+0000 mon.smithi038 (mon.0) 424 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:09.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:08.982626+0000 mgr.smithi038.khesbl (mgr.14664) 219 : cluster [DBG] pgmap v157: 65 pgs: 1 activating, 2 peering, 10 active+undersized+degraded, 11 active+undersized, 41 active+clean; 293 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 44/342 objects degraded (12.865%) 2024-02-15T18:40:11.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:10.983349+0000 mgr.smithi038.khesbl (mgr.14664) 220 : cluster [DBG] pgmap v158: 65 pgs: 1 activating, 2 peering, 5 active+undersized+degraded, 8 active+undersized, 49 active+clean; 293 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 682 B/s wr, 0 op/s; 16/342 objects degraded (4.678%) 2024-02-15T18:40:11.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:10.983349+0000 mgr.smithi038.khesbl (mgr.14664) 220 : cluster [DBG] pgmap v158: 65 pgs: 1 activating, 2 peering, 5 active+undersized+degraded, 8 active+undersized, 49 active+clean; 293 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 682 B/s wr, 0 op/s; 16/342 objects degraded (4.678%) 2024-02-15T18:40:12.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:12.256470+0000 mon.smithi038 (mon.0) 425 : cluster [WRN] Health check update: Degraded data redundancy: 16/342 objects degraded (4.678%), 5 pgs degraded (PG_DEGRADED) 2024-02-15T18:40:12.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:12.301930+0000 mon.smithi038 (mon.0) 426 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:12.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:12.303833+0000 mon.smithi038 (mon.0) 427 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:40:12.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:12.304655+0000 mon.smithi038 (mon.0) 428 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:40:12.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:12.256470+0000 mon.smithi038 (mon.0) 425 : cluster [WRN] Health check update: Degraded data redundancy: 16/342 objects degraded (4.678%), 5 pgs degraded (PG_DEGRADED) 2024-02-15T18:40:12.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:12.301930+0000 mon.smithi038 (mon.0) 426 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:12.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:12.303833+0000 mon.smithi038 (mon.0) 427 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:40:12.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:12.304655+0000 mon.smithi038 (mon.0) 428 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:40:13.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:40:12.305366+0000 mgr.smithi038.khesbl (mgr.14664) 221 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:40:13.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:12.309979+0000 mon.smithi038 (mon.0) 429 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:13.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:12.317631+0000 mon.smithi038 (mon.0) 430 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:13.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:12.320716+0000 mon.smithi038 (mon.0) 431 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:13.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:12.322570+0000 mon.smithi038 (mon.0) 432 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:13.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:12.323938+0000 mon.smithi038 (mon.0) 433 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:13.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:12.325612+0000 mon.smithi038 (mon.0) 434 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:13.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:40:12.327326+0000 mgr.smithi038.khesbl (mgr.14664) 222 : cephadm [INF] Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-02-15T18:40:13.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:12.983849+0000 mgr.smithi038.khesbl (mgr.14664) 223 : cluster [DBG] pgmap v159: 65 pgs: 1 activating, 2 peering, 62 active+clean; 293 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 741 B/s wr, 0 op/s 2024-02-15T18:40:13.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:40:12.305366+0000 mgr.smithi038.khesbl (mgr.14664) 221 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:40:13.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:12.309979+0000 mon.smithi038 (mon.0) 429 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:13.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:12.317631+0000 mon.smithi038 (mon.0) 430 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:13.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:12.320716+0000 mon.smithi038 (mon.0) 431 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:13.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:12.322570+0000 mon.smithi038 (mon.0) 432 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:13.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:12.323938+0000 mon.smithi038 (mon.0) 433 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:13.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:12.325612+0000 mon.smithi038 (mon.0) 434 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:13.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:40:12.327326+0000 mgr.smithi038.khesbl (mgr.14664) 222 : cephadm [INF] Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-02-15T18:40:13.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:12.983849+0000 mgr.smithi038.khesbl (mgr.14664) 223 : cluster [DBG] pgmap v159: 65 pgs: 1 activating, 2 peering, 62 active+clean; 293 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 741 B/s wr, 0 op/s 2024-02-15T18:40:14.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:13.311647+0000 mon.smithi038 (mon.0) 435 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/342 objects degraded (4.678%), 5 pgs degraded) 2024-02-15T18:40:14.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:13.311705+0000 mon.smithi038 (mon.0) 436 : cluster [INF] Cluster is now healthy 2024-02-15T18:40:14.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:13.311647+0000 mon.smithi038 (mon.0) 435 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/342 objects degraded (4.678%), 5 pgs degraded) 2024-02-15T18:40:14.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:13.311705+0000 mon.smithi038 (mon.0) 436 : cluster [INF] Cluster is now healthy 2024-02-15T18:40:15.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:14.984434+0000 mgr.smithi038.khesbl (mgr.14664) 224 : cluster [DBG] pgmap v160: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 895 B/s wr, 3 op/s 2024-02-15T18:40:15.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:14.984434+0000 mgr.smithi038.khesbl (mgr.14664) 224 : cluster [DBG] pgmap v160: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 895 B/s wr, 3 op/s 2024-02-15T18:40:17.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:16.985130+0000 mgr.smithi038.khesbl (mgr.14664) 225 : cluster [DBG] pgmap v161: 65 pgs: 65 active+clean; 253 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1.3 KiB/s wr, 7 op/s 2024-02-15T18:40:17.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:16.985130+0000 mgr.smithi038.khesbl (mgr.14664) 225 : cluster [DBG] pgmap v161: 65 pgs: 65 active+clean; 253 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1.3 KiB/s wr, 7 op/s 2024-02-15T18:40:19.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:18.152894+0000 mon.smithi038 (mon.0) 437 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:19.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:18.265810+0000 mon.smithi038 (mon.0) 438 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:19.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:18.985555+0000 mgr.smithi038.khesbl (mgr.14664) 226 : cluster [DBG] pgmap v162: 65 pgs: 65 active+clean; 217 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1.3 KiB/s wr, 7 op/s 2024-02-15T18:40:19.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:18.152894+0000 mon.smithi038 (mon.0) 437 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:19.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:18.265810+0000 mon.smithi038 (mon.0) 438 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:19.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:18.985555+0000 mgr.smithi038.khesbl (mgr.14664) 226 : cluster [DBG] pgmap v162: 65 pgs: 65 active+clean; 217 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1.3 KiB/s wr, 7 op/s 2024-02-15T18:40:21.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:20.986068+0000 mgr.smithi038.khesbl (mgr.14664) 227 : cluster [DBG] pgmap v163: 65 pgs: 65 active+clean; 173 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1.2 KiB/s wr, 8 op/s 2024-02-15T18:40:21.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:20.986068+0000 mgr.smithi038.khesbl (mgr.14664) 227 : cluster [DBG] pgmap v163: 65 pgs: 65 active+clean; 173 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1.2 KiB/s wr, 8 op/s 2024-02-15T18:40:23.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:22.337969+0000 mon.smithi038 (mon.0) 439 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:23.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:22.339862+0000 mon.smithi038 (mon.0) 440 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:40:23.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:22.340668+0000 mon.smithi038 (mon.0) 441 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:40:23.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:40:22.341377+0000 mgr.smithi038.khesbl (mgr.14664) 228 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:40:23.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:22.347513+0000 mon.smithi038 (mon.0) 442 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:23.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:22.354624+0000 mon.smithi038 (mon.0) 443 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:23.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:22.357639+0000 mon.smithi038 (mon.0) 444 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:23.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:22.359425+0000 mon.smithi038 (mon.0) 445 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:23.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:22.361168+0000 mon.smithi038 (mon.0) 446 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:23.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:22.363538+0000 mon.smithi038 (mon.0) 447 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:23.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:40:22.365356+0000 mgr.smithi038.khesbl (mgr.14664) 229 : cephadm [INF] Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-02-15T18:40:23.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:22.986534+0000 mgr.smithi038.khesbl (mgr.14664) 230 : cluster [DBG] pgmap v164: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 938 B/s wr, 8 op/s 2024-02-15T18:40:23.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:22.337969+0000 mon.smithi038 (mon.0) 439 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:23.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:22.339862+0000 mon.smithi038 (mon.0) 440 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:40:23.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:22.340668+0000 mon.smithi038 (mon.0) 441 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:40:23.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:40:22.341377+0000 mgr.smithi038.khesbl (mgr.14664) 228 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:40:23.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:22.347513+0000 mon.smithi038 (mon.0) 442 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:23.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:22.354624+0000 mon.smithi038 (mon.0) 443 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:23.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:22.357639+0000 mon.smithi038 (mon.0) 444 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:23.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:22.359425+0000 mon.smithi038 (mon.0) 445 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:23.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:22.361168+0000 mon.smithi038 (mon.0) 446 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:23.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:22.363538+0000 mon.smithi038 (mon.0) 447 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:23.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:40:22.365356+0000 mgr.smithi038.khesbl (mgr.14664) 229 : cephadm [INF] Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-02-15T18:40:23.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:22.986534+0000 mgr.smithi038.khesbl (mgr.14664) 230 : cluster [DBG] pgmap v164: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 938 B/s wr, 8 op/s 2024-02-15T18:40:25.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:24.987064+0000 mgr.smithi038.khesbl (mgr.14664) 231 : cluster [DBG] pgmap v165: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 853 B/s wr, 8 op/s 2024-02-15T18:40:25.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:24.987064+0000 mgr.smithi038.khesbl (mgr.14664) 231 : cluster [DBG] pgmap v165: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 853 B/s wr, 8 op/s 2024-02-15T18:40:27.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:26.987638+0000 mgr.smithi038.khesbl (mgr.14664) 232 : cluster [DBG] pgmap v166: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 938 B/s wr, 6 op/s 2024-02-15T18:40:27.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:26.987638+0000 mgr.smithi038.khesbl (mgr.14664) 232 : cluster [DBG] pgmap v166: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 938 B/s wr, 6 op/s 2024-02-15T18:40:28.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:28 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:27.084246+0000 mon.smithi038 (mon.0) 448 : cluster [INF] daemon mds.cephfs.smithi121.ovuzki finished stopping rank 1 in filesystem cephfs (now has 1 ranks) 2024-02-15T18:40:28.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:28 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:27.084246+0000 mon.smithi038 (mon.0) 448 : cluster [INF] daemon mds.cephfs.smithi121.ovuzki finished stopping rank 1 in filesystem cephfs (now has 1 ranks) 2024-02-15T18:40:29.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:28.073117+0000 mon.smithi038 (mon.0) 449 : cluster [DBG] mds.1 [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845] down:stopped 2024-02-15T18:40:29.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:28.073178+0000 mon.smithi038 (mon.0) 450 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi121.mvumlh=up:active} 2 up:standby 2024-02-15T18:40:29.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:28.988327+0000 mgr.smithi038.khesbl (mgr.14664) 233 : cluster [DBG] pgmap v167: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 426 B/s wr, 2 op/s 2024-02-15T18:40:29.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:28.073117+0000 mon.smithi038 (mon.0) 449 : cluster [DBG] mds.1 [v2:172.21.15.121:6826/562926845,v1:172.21.15.121:6827/562926845] down:stopped 2024-02-15T18:40:29.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:28.073178+0000 mon.smithi038 (mon.0) 450 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi121.mvumlh=up:active} 2 up:standby 2024-02-15T18:40:29.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:28.988327+0000 mgr.smithi038.khesbl (mgr.14664) 233 : cluster [DBG] pgmap v167: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 426 B/s wr, 2 op/s 2024-02-15T18:40:30.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:30 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:29.082973+0000 mon.smithi038 (mon.0) 451 : cluster [DBG] mds.? [v2:172.21.15.121:6826/554561289,v1:172.21.15.121:6827/554561289] up:boot 2024-02-15T18:40:30.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:30 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:29.083026+0000 mon.smithi038 (mon.0) 452 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi121.mvumlh=up:active} 3 up:standby 2024-02-15T18:40:30.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:30 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:29.083263+0000 mon.smithi038 (mon.0) 453 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.ovuzki"}]: dispatch 2024-02-15T18:40:30.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:29.082973+0000 mon.smithi038 (mon.0) 451 : cluster [DBG] mds.? [v2:172.21.15.121:6826/554561289,v1:172.21.15.121:6827/554561289] up:boot 2024-02-15T18:40:30.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:29.083026+0000 mon.smithi038 (mon.0) 452 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi121.mvumlh=up:active} 3 up:standby 2024-02-15T18:40:30.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:30 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:29.083263+0000 mon.smithi038 (mon.0) 453 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.ovuzki"}]: dispatch 2024-02-15T18:40:31.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:30.988989+0000 mgr.smithi038.khesbl (mgr.14664) 234 : cluster [DBG] pgmap v168: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 341 B/s wr, 1 op/s 2024-02-15T18:40:31.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:30.988989+0000 mgr.smithi038.khesbl (mgr.14664) 234 : cluster [DBG] pgmap v168: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 341 B/s wr, 1 op/s 2024-02-15T18:40:33.479 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:32.376211+0000 mon.smithi038 (mon.0) 454 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:33.479 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:32.378177+0000 mon.smithi038 (mon.0) 455 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:40:33.479 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:32.379127+0000 mon.smithi038 (mon.0) 456 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:40:33.480 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:40:32.380143+0000 mgr.smithi038.khesbl (mgr.14664) 235 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:40:33.480 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:32.385146+0000 mon.smithi038 (mon.0) 457 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:33.480 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:32.391409+0000 mon.smithi038 (mon.0) 458 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:33.480 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:32.394265+0000 mon.smithi038 (mon.0) 459 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:33.480 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:32.395756+0000 mon.smithi038 (mon.0) 460 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:33.480 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:32.397154+0000 mon.smithi038 (mon.0) 461 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:33.480 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:32.398918+0000 mon.smithi038 (mon.0) 462 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:33.480 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:32.400830+0000 mon.smithi038 (mon.0) 463 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi038.hxoggl"]}]: dispatch 2024-02-15T18:40:33.480 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:40:32.401454+0000 mgr.smithi038.khesbl (mgr.14664) 236 : cephadm [INF] Upgrade: It appears safe to stop mds.cephfs.smithi038.hxoggl 2024-02-15T18:40:33.480 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:32.989444+0000 mgr.smithi038.khesbl (mgr.14664) 237 : cluster [DBG] pgmap v169: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-02-15T18:40:33.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:32.376211+0000 mon.smithi038 (mon.0) 454 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:33.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:32.378177+0000 mon.smithi038 (mon.0) 455 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:40:33.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:32.379127+0000 mon.smithi038 (mon.0) 456 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:40:33.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:40:32.380143+0000 mgr.smithi038.khesbl (mgr.14664) 235 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:40:33.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:32.385146+0000 mon.smithi038 (mon.0) 457 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:33.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:32.391409+0000 mon.smithi038 (mon.0) 458 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:33.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:32.394265+0000 mon.smithi038 (mon.0) 459 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:33.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:32.395756+0000 mon.smithi038 (mon.0) 460 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:33.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:32.397154+0000 mon.smithi038 (mon.0) 461 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:33.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:32.398918+0000 mon.smithi038 (mon.0) 462 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:33.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:32.400830+0000 mon.smithi038 (mon.0) 463 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi038.hxoggl"]}]: dispatch 2024-02-15T18:40:33.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:40:32.401454+0000 mgr.smithi038.khesbl (mgr.14664) 236 : cephadm [INF] Upgrade: It appears safe to stop mds.cephfs.smithi038.hxoggl 2024-02-15T18:40:33.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:32.989444+0000 mgr.smithi038.khesbl (mgr.14664) 237 : cluster [DBG] pgmap v169: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-02-15T18:40:35.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:40:34.276044+0000 mgr.smithi038.khesbl (mgr.14664) 238 : cephadm [INF] Upgrade: Updating mds.cephfs.smithi038.hxoggl 2024-02-15T18:40:35.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:34.284847+0000 mon.smithi038 (mon.0) 464 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:35.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:34.285675+0000 mon.smithi038 (mon.0) 465 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.hxoggl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:40:35.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:34.286397+0000 mon.smithi038 (mon.0) 466 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:40:35.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:40:34.287046+0000 mgr.smithi038.khesbl (mgr.14664) 239 : cephadm [INF] Deploying daemon mds.cephfs.smithi038.hxoggl on smithi038 2024-02-15T18:40:35.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:34.989854+0000 mgr.smithi038.khesbl (mgr.14664) 240 : cluster [DBG] pgmap v170: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-15T18:40:35.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:40:34.276044+0000 mgr.smithi038.khesbl (mgr.14664) 238 : cephadm [INF] Upgrade: Updating mds.cephfs.smithi038.hxoggl 2024-02-15T18:40:35.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:34.284847+0000 mon.smithi038 (mon.0) 464 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:35.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:34.285675+0000 mon.smithi038 (mon.0) 465 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.hxoggl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:40:35.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:34.286397+0000 mon.smithi038 (mon.0) 466 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:40:35.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:40:34.287046+0000 mgr.smithi038.khesbl (mgr.14664) 239 : cephadm [INF] Deploying daemon mds.cephfs.smithi038.hxoggl on smithi038 2024-02-15T18:40:35.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:34.989854+0000 mgr.smithi038.khesbl (mgr.14664) 240 : cluster [DBG] pgmap v170: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-02-15T18:40:36.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.140+0000 7f23d55be700 1 -- 172.21.15.38:0/3634654976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23d00f77d0 msgr2=0x7f23d00f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:36.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.140+0000 7f23d55be700 1 --2- 172.21.15.38:0/3634654976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23d00f77d0 0x7f23d00f7c30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f23c4004660 tx=0x7f23c4009b30).stop 2024-02-15T18:40:36.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.140+0000 7f23d55be700 1 -- 172.21.15.38:0/3634654976 shutdown_connections 2024-02-15T18:40:36.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.140+0000 7f23d55be700 1 --2- 172.21.15.38:0/3634654976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23d00f77d0 0x7f23d00f7c30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:36.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.140+0000 7f23d55be700 1 --2- 172.21.15.38:0/3634654976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23d00f6530 0x7f23d00f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:36.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.140+0000 7f23d55be700 1 -- 172.21.15.38:0/3634654976 >> 172.21.15.38:0/3634654976 conn(0x7f23d00f1c90 msgr2=0x7f23d00f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:36.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.140+0000 7f23d55be700 1 -- 172.21.15.38:0/3634654976 shutdown_connections 2024-02-15T18:40:36.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.140+0000 7f23d55be700 1 -- 172.21.15.38:0/3634654976 wait complete. 2024-02-15T18:40:36.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.140+0000 7f23d55be700 1 Processor -- start 2024-02-15T18:40:36.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.141+0000 7f23d55be700 1 -- start start 2024-02-15T18:40:36.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.141+0000 7f23d55be700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23d00f6530 0x7f23d0106e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:36.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.141+0000 7f23d55be700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23d00f77d0 0x7f23d0107310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:36.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.141+0000 7f23d55be700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d0107910 con 0x7f23d00f6530 2024-02-15T18:40:36.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.141+0000 7f23d55be700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d0108120 con 0x7f23d00f77d0 2024-02-15T18:40:36.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.141+0000 7f23cffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23d00f6530 0x7f23d0106e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:36.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.141+0000 7f23cffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23d00f6530 0x7f23d0106e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56504/0 (socket says 172.21.15.38:56504) 2024-02-15T18:40:36.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.141+0000 7f23cffff700 1 -- 172.21.15.38:0/2402122825 learned_addr learned my addr 172.21.15.38:0/2402122825 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:40:36.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.141+0000 7f23cf7fe700 1 --2- 172.21.15.38:0/2402122825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23d00f77d0 0x7f23d0107310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:36.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.142+0000 7f23cffff700 1 -- 172.21.15.38:0/2402122825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23d00f77d0 msgr2=0x7f23d0107310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:36.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.142+0000 7f23cffff700 1 --2- 172.21.15.38:0/2402122825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23d00f77d0 0x7f23d0107310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:36.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.142+0000 7f23cffff700 1 -- 172.21.15.38:0/2402122825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c4005040 con 0x7f23d00f6530 2024-02-15T18:40:36.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.142+0000 7f23cffff700 1 --2- 172.21.15.38:0/2402122825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23d00f6530 0x7f23d0106e20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f23c0002700 tx=0x7f23c0009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:36.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.142+0000 7f23ccff9700 1 -- 172.21.15.38:0/2402122825 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23c00079c0 con 0x7f23d00f6530 2024-02-15T18:40:36.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.142+0000 7f23ccff9700 1 -- 172.21.15.38:0/2402122825 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f23c000f040 con 0x7f23d00f6530 2024-02-15T18:40:36.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.142+0000 7f23d55be700 1 -- 172.21.15.38:0/2402122825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d01083a0 con 0x7f23d00f6530 2024-02-15T18:40:36.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.142+0000 7f23ccff9700 1 -- 172.21.15.38:0/2402122825 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23c0014940 con 0x7f23d00f6530 2024-02-15T18:40:36.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.142+0000 7f23d55be700 1 -- 172.21.15.38:0/2402122825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d0108960 con 0x7f23d00f6530 2024-02-15T18:40:36.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.143+0000 7f23d55be700 1 -- 172.21.15.38:0/2402122825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d0059100 con 0x7f23d00f6530 2024-02-15T18:40:36.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.144+0000 7f23ccff9700 1 -- 172.21.15.38:0/2402122825 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f23c001b060 con 0x7f23d00f6530 2024-02-15T18:40:36.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.144+0000 7f23ccff9700 1 --2- 172.21.15.38:0/2402122825 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f23b8060f70 0x7f23b8063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:36.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.144+0000 7f23ccff9700 1 -- 172.21.15.38:0/2402122825 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f23c007be80 con 0x7f23d00f6530 2024-02-15T18:40:36.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.144+0000 7f23cf7fe700 1 --2- 172.21.15.38:0/2402122825 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f23b8060f70 0x7f23b8063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:36.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.145+0000 7f23cf7fe700 1 --2- 172.21.15.38:0/2402122825 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f23b8060f70 0x7f23b8063430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f23c4004630 tx=0x7f23c4009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:36.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.149+0000 7f23ccff9700 1 -- 172.21.15.38:0/2402122825 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f23c004c380 con 0x7f23d00f6530 2024-02-15T18:40:36.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.344+0000 7f23d55be700 1 -- 172.21.15.38:0/2402122825 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f23d00feee0 con 0x7f23b8060f70 2024-02-15T18:40:36.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.345+0000 7f23ccff9700 1 -- 172.21.15.38:0/2402122825 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f23d00feee0 con 0x7f23b8060f70 2024-02-15T18:40:36.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.347+0000 7f23d55be700 1 -- 172.21.15.38:0/2402122825 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f23b8060f70 msgr2=0x7f23b8063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:36.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.347+0000 7f23d55be700 1 --2- 172.21.15.38:0/2402122825 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f23b8060f70 0x7f23b8063430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f23c4004630 tx=0x7f23c4009b30).stop 2024-02-15T18:40:36.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.347+0000 7f23d55be700 1 -- 172.21.15.38:0/2402122825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23d00f6530 msgr2=0x7f23d0106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:36.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.347+0000 7f23d55be700 1 --2- 172.21.15.38:0/2402122825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23d00f6530 0x7f23d0106e20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f23c0002700 tx=0x7f23c0009d80).stop 2024-02-15T18:40:36.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.347+0000 7f23d55be700 1 -- 172.21.15.38:0/2402122825 shutdown_connections 2024-02-15T18:40:36.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.347+0000 7f23d55be700 1 --2- 172.21.15.38:0/2402122825 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f23b8060f70 0x7f23b8063430 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:36.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.347+0000 7f23d55be700 1 --2- 172.21.15.38:0/2402122825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f23d00f6530 0x7f23d0106e20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:36.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.347+0000 7f23d55be700 1 --2- 172.21.15.38:0/2402122825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23d00f77d0 0x7f23d0107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:36.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.347+0000 7f23d55be700 1 -- 172.21.15.38:0/2402122825 >> 172.21.15.38:0/2402122825 conn(0x7f23d00f1c90 msgr2=0x7f23d00fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:36.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.347+0000 7f23d55be700 1 -- 172.21.15.38:0/2402122825 shutdown_connections 2024-02-15T18:40:36.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.347+0000 7f23d55be700 1 -- 172.21.15.38:0/2402122825 wait complete. 2024-02-15T18:40:36.347 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-02-15T18:40:36.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.453+0000 7f1f22863700 1 -- 172.21.15.38:0/1839033589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c0f55c0 msgr2=0x7f1f1c0f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:36.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.453+0000 7f1f22863700 1 --2- 172.21.15.38:0/1839033589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c0f55c0 0x7f1f1c0f59e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f1f0c004660 tx=0x7f1f0c009b30).stop 2024-02-15T18:40:36.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.453+0000 7f1f22863700 1 -- 172.21.15.38:0/1839033589 shutdown_connections 2024-02-15T18:40:36.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.453+0000 7f1f22863700 1 --2- 172.21.15.38:0/1839033589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f1c0f6860 0x7f1f1c0f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:40:36.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.453+0000 7f1f22863700 1 --2- 172.21.15.38:0/1839033589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c0f55c0 0x7f1f1c0f59e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:36.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.453+0000 7f1f22863700 1 -- 172.21.15.38:0/1839033589 >> 172.21.15.38:0/1839033589 conn(0x7f1f1c0f0d40 msgr2=0x7f1f1c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:36.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.454+0000 7f1f22863700 1 -- 172.21.15.38:0/1839033589 shutdown_connections 2024-02-15T18:40:36.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.454+0000 7f1f22863700 1 -- 172.21.15.38:0/1839033589 wait complete. 2024-02-15T18:40:36.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.454+0000 7f1f22863700 1 Processor -- start 2024-02-15T18:40:36.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.454+0000 7f1f22863700 1 -- start start 2024-02-15T18:40:36.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.455+0000 7f1f22863700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f1c0f55c0 0x7f1f1c109d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:36.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.455+0000 7f1f22863700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c0f6860 0x7f1f1c10c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:36.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.455+0000 7f1f22863700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c060470 con 0x7f1f1c0f6860 2024-02-15T18:40:36.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.455+0000 7f1f22863700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c0605b0 con 0x7f1f1c0f55c0 2024-02-15T18:40:36.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.455+0000 7f1f1b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c0f6860 0x7f1f1c10c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:36.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.455+0000 7f1f1b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c0f6860 0x7f1f1c10c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56522/0 (socket says 172.21.15.38:56522) 2024-02-15T18:40:36.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.455+0000 7f1f1b7fe700 1 -- 172.21.15.38:0/2597925821 learned_addr learned my addr 172.21.15.38:0/2597925821 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:40:36.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.455+0000 7f1f1bfff700 1 --2- 172.21.15.38:0/2597925821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f1c0f55c0 0x7f1f1c109d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:36.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.455+0000 7f1f1b7fe700 1 -- 172.21.15.38:0/2597925821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f1c0f55c0 msgr2=0x7f1f1c109d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:36.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.455+0000 7f1f1b7fe700 1 --2- 172.21.15.38:0/2597925821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f1c0f55c0 0x7f1f1c109d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:36.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.455+0000 7f1f1b7fe700 1 -- 172.21.15.38:0/2597925821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c005040 con 0x7f1f1c0f6860 2024-02-15T18:40:36.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.455+0000 7f1f1bfff700 1 --2- 172.21.15.38:0/2597925821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f1c0f55c0 0x7f1f1c109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:40:36.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.455+0000 7f1f1b7fe700 1 --2- 172.21.15.38:0/2597925821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c0f6860 0x7f1f1c10c290 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f1f1000b410 tx=0x7f1f10006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:36.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.455+0000 7f1f18ff9700 1 -- 172.21.15.38:0/2597925821 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f100125f0 con 0x7f1f1c0f6860 2024-02-15T18:40:36.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.456+0000 7f1f22863700 1 -- 172.21.15.38:0/2597925821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f1c10c8f0 con 0x7f1f1c0f6860 2024-02-15T18:40:36.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.456+0000 7f1f18ff9700 1 -- 172.21.15.38:0/2597925821 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f1f10012c40 con 0x7f1f1c0f6860 2024-02-15T18:40:36.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.456+0000 7f1f22863700 1 -- 172.21.15.38:0/2597925821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1c10ce00 con 0x7f1f1c0f6860 2024-02-15T18:40:36.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.456+0000 7f1f18ff9700 1 -- 172.21.15.38:0/2597925821 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f10002b80 con 0x7f1f1c0f6860 2024-02-15T18:40:36.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.457+0000 7f1f18ff9700 1 -- 172.21.15.38:0/2597925821 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f1f1000f030 con 0x7f1f1c0f6860 2024-02-15T18:40:36.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.457+0000 7f1f22863700 1 -- 172.21.15.38:0/2597925821 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f1c0f49b0 con 0x7f1f1c0f6860 2024-02-15T18:40:36.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.457+0000 7f1f18ff9700 1 --2- 172.21.15.38:0/2597925821 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f1f04060f20 0x7f1f040633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:36.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.457+0000 7f1f18ff9700 1 -- 172.21.15.38:0/2597925821 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f1f1007c5c0 con 0x7f1f1c0f6860 2024-02-15T18:40:36.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.457+0000 7f1f1bfff700 1 --2- 172.21.15.38:0/2597925821 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f1f04060f20 0x7f1f040633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:36.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.458+0000 7f1f1bfff700 1 --2- 172.21.15.38:0/2597925821 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f1f04060f20 0x7f1f040633e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f1f0c009d60 tx=0x7f1f0c009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:36.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.462+0000 7f1f18ff9700 1 -- 172.21.15.38:0/2597925821 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1f1004ca40 con 0x7f1f1c0f6860 2024-02-15T18:40:36.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:36 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:36.304336+0000 mon.smithi038 (mon.0) 467 : cluster [WRN] daemon mds.cephfs.smithi038.hxoggl compat changed unexpectedly 2024-02-15T18:40:36.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:36 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:36.308545+0000 mon.smithi038 (mon.0) 468 : cluster [DBG] osdmap e77: 6 total, 6 up, 6 in 2024-02-15T18:40:36.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.651+0000 7f1f22863700 1 -- 172.21.15.38:0/2597925821 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f1c0fdf80 con 0x7f1f04060f20 2024-02-15T18:40:36.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.652+0000 7f1f18ff9700 1 -- 172.21.15.38:0/2597925821 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1f1c0fdf80 con 0x7f1f04060f20 2024-02-15T18:40:36.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.653+0000 7f1f22863700 1 -- 172.21.15.38:0/2597925821 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f1f04060f20 msgr2=0x7f1f040633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:36.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.653+0000 7f1f22863700 1 --2- 172.21.15.38:0/2597925821 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f1f04060f20 0x7f1f040633e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f1f0c009d60 tx=0x7f1f0c009b30).stop 2024-02-15T18:40:36.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.653+0000 7f1f22863700 1 -- 172.21.15.38:0/2597925821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c0f6860 msgr2=0x7f1f1c10c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:36.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.653+0000 7f1f22863700 1 --2- 172.21.15.38:0/2597925821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c0f6860 0x7f1f1c10c290 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f1f1000b410 tx=0x7f1f10006ef0).stop 2024-02-15T18:40:36.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.654+0000 7f1f22863700 1 -- 172.21.15.38:0/2597925821 shutdown_connections 2024-02-15T18:40:36.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.654+0000 7f1f22863700 1 --2- 172.21.15.38:0/2597925821 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f1f04060f20 0x7f1f040633e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:36.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.654+0000 7f1f22863700 1 --2- 172.21.15.38:0/2597925821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f1c0f55c0 0x7f1f1c109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:36.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.654+0000 7f1f22863700 1 --2- 172.21.15.38:0/2597925821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f1c0f6860 0x7f1f1c10c290 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:36.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.654+0000 7f1f22863700 1 -- 172.21.15.38:0/2597925821 >> 172.21.15.38:0/2597925821 conn(0x7f1f1c0f0d40 msgr2=0x7f1f1c0fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:36.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.654+0000 7f1f22863700 1 -- 172.21.15.38:0/2597925821 shutdown_connections 2024-02-15T18:40:36.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.654+0000 7f1f22863700 1 -- 172.21.15.38:0/2597925821 wait complete. 2024-02-15T18:40:36.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.755+0000 7f88934dd700 1 -- 172.21.15.38:0/3150523826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f888c0f6860 msgr2=0x7f888c0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:36.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.755+0000 7f88934dd700 1 --2- 172.21.15.38:0/3150523826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f888c0f6860 0x7f888c0f6cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f8880004660 tx=0x7f8880009b30).stop 2024-02-15T18:40:36.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.755+0000 7f88934dd700 1 -- 172.21.15.38:0/3150523826 shutdown_connections 2024-02-15T18:40:36.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.755+0000 7f88934dd700 1 --2- 172.21.15.38:0/3150523826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f888c0f6860 0x7f888c0f6cc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:36.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.755+0000 7f88934dd700 1 --2- 172.21.15.38:0/3150523826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f888c0f55c0 0x7f888c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:40:36.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.755+0000 7f88934dd700 1 -- 172.21.15.38:0/3150523826 >> 172.21.15.38:0/3150523826 conn(0x7f888c0f0d40 msgr2=0x7f888c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:36.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.755+0000 7f88934dd700 1 -- 172.21.15.38:0/3150523826 shutdown_connections 2024-02-15T18:40:36.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.755+0000 7f88934dd700 1 -- 172.21.15.38:0/3150523826 wait complete. 2024-02-15T18:40:36.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.756+0000 7f88934dd700 1 Processor -- start 2024-02-15T18:40:36.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.756+0000 7f88934dd700 1 -- start start 2024-02-15T18:40:36.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.756+0000 7f88934dd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f888c0f55c0 0x7f888c107050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:36.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.756+0000 7f88934dd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f888c0f6860 0x7f888c107540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:36.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.756+0000 7f88934dd700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c107b80 con 0x7f888c0f6860 2024-02-15T18:40:36.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.756+0000 7f88934dd700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c108430 con 0x7f888c0f55c0 2024-02-15T18:40:36.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.756+0000 7f8890a78700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f888c0f6860 0x7f888c107540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:36.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.756+0000 7f8890a78700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f888c0f6860 0x7f888c107540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56536/0 (socket says 172.21.15.38:56536) 2024-02-15T18:40:36.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.756+0000 7f8890a78700 1 -- 172.21.15.38:0/2778369188 learned_addr learned my addr 172.21.15.38:0/2778369188 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:40:36.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.756+0000 7f8891279700 1 --2- 172.21.15.38:0/2778369188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f888c0f55c0 0x7f888c107050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:36.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.757+0000 7f8890a78700 1 -- 172.21.15.38:0/2778369188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f888c0f55c0 msgr2=0x7f888c107050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:36.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.757+0000 7f8890a78700 1 --2- 172.21.15.38:0/2778369188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f888c0f55c0 0x7f888c107050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:36.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.757+0000 7f8890a78700 1 -- 172.21.15.38:0/2778369188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8880005040 con 0x7f888c0f6860 2024-02-15T18:40:36.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.757+0000 7f8890a78700 1 --2- 172.21.15.38:0/2778369188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f888c0f6860 0x7f888c107540 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f8880009f50 tx=0x7f8880009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:36.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.757+0000 7f887dffb700 1 -- 172.21.15.38:0/2778369188 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8880018070 con 0x7f888c0f6860 2024-02-15T18:40:36.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.757+0000 7f887dffb700 1 -- 172.21.15.38:0/2778369188 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f8880007d70 con 0x7f888c0f6860 2024-02-15T18:40:36.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.757+0000 7f88934dd700 1 -- 172.21.15.38:0/2778369188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f888c1086b0 con 0x7f888c0f6860 2024-02-15T18:40:36.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.757+0000 7f887dffb700 1 -- 172.21.15.38:0/2778369188 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f888001c620 con 0x7f888c0f6860 2024-02-15T18:40:36.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.757+0000 7f88934dd700 1 -- 172.21.15.38:0/2778369188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888c108b90 con 0x7f888c0f6860 2024-02-15T18:40:36.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.758+0000 7f88934dd700 1 -- 172.21.15.38:0/2778369188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f888c0f49b0 con 0x7f888c0f6860 2024-02-15T18:40:36.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.758+0000 7f887dffb700 1 -- 172.21.15.38:0/2778369188 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f888001c8e0 con 0x7f888c0f6860 2024-02-15T18:40:36.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.759+0000 7f887dffb700 1 --2- 172.21.15.38:0/2778369188 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8878060f20 0x7f88780633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:36.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.759+0000 7f887dffb700 1 -- 172.21.15.38:0/2778369188 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f888007e070 con 0x7f888c0f6860 2024-02-15T18:40:36.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.759+0000 7f8891279700 1 --2- 172.21.15.38:0/2778369188 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8878060f20 0x7f88780633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:36.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.759+0000 7f8891279700 1 --2- 172.21.15.38:0/2778369188 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8878060f20 0x7f88780633e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f888c0f66c0 tx=0x7f8888008040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:36.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.763+0000 7f887dffb700 1 -- 172.21.15.38:0/2778369188 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8880051f20 con 0x7f888c0f6860 2024-02-15T18:40:36.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:36.304336+0000 mon.smithi038 (mon.0) 467 : cluster [WRN] daemon mds.cephfs.smithi038.hxoggl compat changed unexpectedly 2024-02-15T18:40:36.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:36 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:36.308545+0000 mon.smithi038 (mon.0) 468 : cluster [DBG] osdmap e77: 6 total, 6 up, 6 in 2024-02-15T18:40:36.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.948+0000 7f88934dd700 1 -- 172.21.15.38:0/2778369188 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f888c0fe070 con 0x7f8878060f20 2024-02-15T18:40:36.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.957+0000 7f887dffb700 1 -- 172.21.15.38:0/2778369188 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f888c0fe070 con 0x7f8878060f20 2024-02-15T18:40:36.946 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:40:36.947 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (11m) 2m ago 13m 19.3M - 0.20.0 0881eb8f169f c2034c318e7c 2024-02-15T18:40:36.947 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (3m) 2m ago 13m 6635k - 16.2.14-521-g0e714d9a 1fbc92c859dc c2fee0824d4b 2024-02-15T18:40:36.947 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi121 smithi121 running (3m) 35s ago 12m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 05c1a5501a35 2024-02-15T18:40:36.947 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (11m) 2m ago 13m 33.4M - 6.7.4 557c83e11646 2dcccfbfbd2a 2024-02-15T18:40:36.947 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.hxoggl smithi038 running (7m) 2m ago 7m 14.8M - 16.2.4 8d91d370c2b8 be2533754b9c 2024-02-15T18:40:36.947 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.qgkzgn smithi038 running (7m) 2m ago 7m 14.4M - 16.2.4 8d91d370c2b8 dd3ffafba9bd 2024-02-15T18:40:36.947 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.mvumlh smithi121 running (7m) 35s ago 7m 116M - 16.2.4 8d91d370c2b8 88b7d876a39b 2024-02-15T18:40:36.947 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.ovuzki smithi121 running (7m) 35s ago 7m 124M - 16.2.4 8d91d370c2b8 21cacf0012f9 2024-02-15T18:40:36.947 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.khesbl smithi038 *:8443,9283 running (4m) 2m ago 14m 433M - 16.2.14-521-g0e714d9a 1fbc92c859dc e642f3c2cfc9 2024-02-15T18:40:36.947 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi121.ckiryi smithi121 *:8443,9283 running (4m) 35s ago 12m 387M - 16.2.14-521-g0e714d9a 1fbc92c859dc 82515b025594 2024-02-15T18:40:36.947 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (4m) 2m ago 15m 40.8M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc d3e8317a30ca 2024-02-15T18:40:36.947 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi121 smithi121 running (3m) 35s ago 12m 43.1M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 42e1dda1a39d 2024-02-15T18:40:36.947 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (12m) 2m ago 12m 17.1M - 0.18.1 e5a616e4b9cf 037f60e38158 2024-02-15T18:40:36.947 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi121 smithi121 *:9100 running (11m) 35s ago 11m 17.2M - 0.18.1 e5a616e4b9cf a867bff85e18 2024-02-15T18:40:36.948 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (3m) 2m ago 11m 191M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 623ac7ecbb0a 2024-02-15T18:40:36.948 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (2m) 2m ago 11m 217M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 0f9e8e1f255a 2024-02-15T18:40:36.948 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (2m) 2m ago 10m 13.4M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 37171264b5be 2024-02-15T18:40:36.948 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi121 running (96s) 35s ago 10m 255M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc d505a58b375c 2024-02-15T18:40:36.948 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi121 running (68s) 35s ago 9m 259M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 0ce6b6abb99e 2024-02-15T18:40:36.948 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi121 running (39s) 35s ago 9m 13.1M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 79bb22243769 2024-02-15T18:40:36.948 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (4m) 2m ago 12m 65.9M - 2.18.1 de242295e225 67e33f9de4e6 2024-02-15T18:40:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.959+0000 7f88934dd700 1 -- 172.21.15.38:0/2778369188 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8878060f20 msgr2=0x7f88780633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.959+0000 7f88934dd700 1 --2- 172.21.15.38:0/2778369188 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8878060f20 0x7f88780633e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f888c0f66c0 tx=0x7f8888008040).stop 2024-02-15T18:40:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.959+0000 7f88934dd700 1 -- 172.21.15.38:0/2778369188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f888c0f6860 msgr2=0x7f888c107540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.959+0000 7f88934dd700 1 --2- 172.21.15.38:0/2778369188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f888c0f6860 0x7f888c107540 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f8880009f50 tx=0x7f8880009b30).stop 2024-02-15T18:40:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.959+0000 7f88934dd700 1 -- 172.21.15.38:0/2778369188 shutdown_connections 2024-02-15T18:40:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.959+0000 7f88934dd700 1 --2- 172.21.15.38:0/2778369188 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8878060f20 0x7f88780633e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.959+0000 7f88934dd700 1 --2- 172.21.15.38:0/2778369188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f888c0f55c0 0x7f888c107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.959+0000 7f88934dd700 1 --2- 172.21.15.38:0/2778369188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f888c0f6860 0x7f888c107540 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.959+0000 7f88934dd700 1 -- 172.21.15.38:0/2778369188 >> 172.21.15.38:0/2778369188 conn(0x7f888c0f0d40 msgr2=0x7f888c05fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.960+0000 7f88934dd700 1 -- 172.21.15.38:0/2778369188 shutdown_connections 2024-02-15T18:40:36.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:36.960+0000 7f88934dd700 1 -- 172.21.15.38:0/2778369188 wait complete. 2024-02-15T18:40:37.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.062+0000 7f37353ae700 1 -- 172.21.15.38:0/1439369210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37300f6870 msgr2=0x7f37300f6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:37.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.062+0000 7f37353ae700 1 --2- 172.21.15.38:0/1439369210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37300f6870 0x7f37300f6cd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f3718004660 tx=0x7f3718009b30).stop 2024-02-15T18:40:37.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.062+0000 7f37353ae700 1 -- 172.21.15.38:0/1439369210 shutdown_connections 2024-02-15T18:40:37.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.062+0000 7f37353ae700 1 --2- 172.21.15.38:0/1439369210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37300f6870 0x7f37300f6cd0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:37.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.062+0000 7f37353ae700 1 --2- 172.21.15.38:0/1439369210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37300f55d0 0x7f37300f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:40:37.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.062+0000 7f37353ae700 1 -- 172.21.15.38:0/1439369210 >> 172.21.15.38:0/1439369210 conn(0x7f37300f0d70 msgr2=0x7f37300f3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:37.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.062+0000 7f37353ae700 1 -- 172.21.15.38:0/1439369210 shutdown_connections 2024-02-15T18:40:37.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.062+0000 7f37353ae700 1 -- 172.21.15.38:0/1439369210 wait complete. 2024-02-15T18:40:37.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.063+0000 7f37353ae700 1 Processor -- start 2024-02-15T18:40:37.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.063+0000 7f37353ae700 1 -- start start 2024-02-15T18:40:37.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.063+0000 7f37353ae700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37300f55d0 0x7f3730109d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:37.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.063+0000 7f37353ae700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37300f6870 0x7f373010c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:37.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.063+0000 7f37353ae700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37300605b0 con 0x7f37300f55d0 2024-02-15T18:40:37.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.063+0000 7f37353ae700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37300606f0 con 0x7f37300f6870 2024-02-15T18:40:37.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.063+0000 7f372effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37300f55d0 0x7f3730109d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.063+0000 7f372effd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37300f55d0 0x7f3730109d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56554/0 (socket says 172.21.15.38:56554) 2024-02-15T18:40:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.063+0000 7f372effd700 1 -- 172.21.15.38:0/1929440610 learned_addr learned my addr 172.21.15.38:0/1929440610 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:40:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.063+0000 7f372e7fc700 1 --2- 172.21.15.38:0/1929440610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37300f6870 0x7f373010c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.064+0000 7f372effd700 1 -- 172.21.15.38:0/1929440610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37300f6870 msgr2=0x7f373010c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.064+0000 7f372effd700 1 --2- 172.21.15.38:0/1929440610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37300f6870 0x7f373010c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.064+0000 7f372effd700 1 -- 172.21.15.38:0/1929440610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3718005040 con 0x7f37300f55d0 2024-02-15T18:40:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.064+0000 7f372e7fc700 1 --2- 172.21.15.38:0/1929440610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37300f6870 0x7f373010c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:40:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.064+0000 7f372effd700 1 --2- 172.21.15.38:0/1929440610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37300f55d0 0x7f3730109d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f37200093f0 tx=0x7f3720006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.064+0000 7f37277fe700 1 -- 172.21.15.38:0/1929440610 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3720009ea0 con 0x7f37300f55d0 2024-02-15T18:40:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.064+0000 7f37353ae700 1 -- 172.21.15.38:0/1929440610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373010c8a0 con 0x7f37300f55d0 2024-02-15T18:40:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.064+0000 7f37277fe700 1 -- 172.21.15.38:0/1929440610 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f3720013b30 con 0x7f37300f55d0 2024-02-15T18:40:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.064+0000 7f37277fe700 1 -- 172.21.15.38:0/1929440610 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f372000dc60 con 0x7f37300f55d0 2024-02-15T18:40:37.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.064+0000 7f37353ae700 1 -- 172.21.15.38:0/1929440610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f373010cdb0 con 0x7f37300f55d0 2024-02-15T18:40:37.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.065+0000 7f37353ae700 1 -- 172.21.15.38:0/1929440610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37300f49c0 con 0x7f37300f55d0 2024-02-15T18:40:37.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.065+0000 7f37277fe700 1 -- 172.21.15.38:0/1929440610 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f372001a060 con 0x7f37300f55d0 2024-02-15T18:40:37.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.066+0000 7f37277fe700 1 --2- 172.21.15.38:0/1929440610 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f371c060f20 0x7f371c0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:37.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.066+0000 7f37277fe700 1 -- 172.21.15.38:0/1929440610 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f372007c130 con 0x7f37300f55d0 2024-02-15T18:40:37.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.066+0000 7f372e7fc700 1 --2- 172.21.15.38:0/1929440610 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f371c060f20 0x7f371c0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:37.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.066+0000 7f372e7fc700 1 --2- 172.21.15.38:0/1929440610 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f371c060f20 0x7f371c0633e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f3718004630 tx=0x7f3718009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:37.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.070+0000 7f37277fe700 1 -- 172.21.15.38:0/1929440610 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f372004c550 con 0x7f37300f55d0 2024-02-15T18:40:37.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.314+0000 7f37353ae700 1 -- 172.21.15.38:0/1929440610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3730045ad0 con 0x7f37300f55d0 2024-02-15T18:40:37.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.314+0000 7f37277fe700 1 -- 172.21.15.38:0/1929440610 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+727 (secure 0 0 0) 0x7f372004fb70 con 0x7f37300f55d0 2024-02-15T18:40:37.303 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:40:37.303 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-02-15T18:40:37.303 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:40:37.303 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:40:37.303 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-02-15T18:40:37.303 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:40:37.303 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:40:37.304 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-02-15T18:40:37.304 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 6 2024-02-15T18:40:37.304 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:40:37.304 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-02-15T18:40:37.304 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 3 2024-02-15T18:40:37.304 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:40:37.304 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-02-15T18:40:37.304 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 10, 2024-02-15T18:40:37.304 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 3 2024-02-15T18:40:37.304 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-02-15T18:40:37.304 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:40:37.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.316+0000 7f37353ae700 1 -- 172.21.15.38:0/1929440610 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f371c060f20 msgr2=0x7f371c0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:37.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.316+0000 7f37353ae700 1 --2- 172.21.15.38:0/1929440610 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f371c060f20 0x7f371c0633e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f3718004630 tx=0x7f3718009b30).stop 2024-02-15T18:40:37.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.316+0000 7f37353ae700 1 -- 172.21.15.38:0/1929440610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37300f55d0 msgr2=0x7f3730109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:37.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.316+0000 7f37353ae700 1 --2- 172.21.15.38:0/1929440610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37300f55d0 0x7f3730109d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f37200093f0 tx=0x7f3720006ec0).stop 2024-02-15T18:40:37.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.316+0000 7f37353ae700 1 -- 172.21.15.38:0/1929440610 shutdown_connections 2024-02-15T18:40:37.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.316+0000 7f37353ae700 1 --2- 172.21.15.38:0/1929440610 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f371c060f20 0x7f371c0633e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:37.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.316+0000 7f37353ae700 1 --2- 172.21.15.38:0/1929440610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37300f55d0 0x7f3730109d40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:37.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.316+0000 7f37353ae700 1 --2- 172.21.15.38:0/1929440610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37300f6870 0x7f373010c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:37.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.316+0000 7f37353ae700 1 -- 172.21.15.38:0/1929440610 >> 172.21.15.38:0/1929440610 conn(0x7f37300f0d70 msgr2=0x7f37300fa2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:37.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.316+0000 7f37353ae700 1 -- 172.21.15.38:0/1929440610 shutdown_connections 2024-02-15T18:40:37.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.316+0000 7f37353ae700 1 -- 172.21.15.38:0/1929440610 wait complete. 2024-02-15T18:40:37.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.419+0000 7f94ac43b700 1 -- 172.21.15.38:0/2274287120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94a40f6860 msgr2=0x7f94a40f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:37.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.419+0000 7f94ac43b700 1 --2- 172.21.15.38:0/2274287120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94a40f6860 0x7f94a40f6cc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f9498004660 tx=0x7f9498009b30).stop 2024-02-15T18:40:37.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.419+0000 7f94ac43b700 1 -- 172.21.15.38:0/2274287120 shutdown_connections 2024-02-15T18:40:37.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.419+0000 7f94ac43b700 1 --2- 172.21.15.38:0/2274287120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94a40f6860 0x7f94a40f6cc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:37.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.419+0000 7f94ac43b700 1 --2- 172.21.15.38:0/2274287120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a40f55c0 0x7f94a40f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:37.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.419+0000 7f94ac43b700 1 -- 172.21.15.38:0/2274287120 >> 172.21.15.38:0/2274287120 conn(0x7f94a40f0d40 msgr2=0x7f94a40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:37.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.419+0000 7f94ac43b700 1 -- 172.21.15.38:0/2274287120 shutdown_connections 2024-02-15T18:40:37.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.419+0000 7f94ac43b700 1 -- 172.21.15.38:0/2274287120 wait complete. 2024-02-15T18:40:37.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.420+0000 7f94ac43b700 1 Processor -- start 2024-02-15T18:40:37.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.420+0000 7f94ac43b700 1 -- start start 2024-02-15T18:40:37.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.420+0000 7f94ac43b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94a40f55c0 0x7f94a4109210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:37.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.420+0000 7f94ac43b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a40f6860 0x7f94a4109700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:37.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.420+0000 7f94ac43b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a4109cb0 con 0x7f94a40f55c0 2024-02-15T18:40:37.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.420+0000 7f94ac43b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a4109df0 con 0x7f94a40f6860 2024-02-15T18:40:37.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.420+0000 7f94aa1d7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94a40f55c0 0x7f94a4109210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:37.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.420+0000 7f94aa1d7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94a40f55c0 0x7f94a4109210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56566/0 (socket says 172.21.15.38:56566) 2024-02-15T18:40:37.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.420+0000 7f94aa1d7700 1 -- 172.21.15.38:0/3350941114 learned_addr learned my addr 172.21.15.38:0/3350941114 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:40:37.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.420+0000 7f94a99d6700 1 --2- 172.21.15.38:0/3350941114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a40f6860 0x7f94a4109700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:37.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.420+0000 7f94aa1d7700 1 -- 172.21.15.38:0/3350941114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a40f6860 msgr2=0x7f94a4109700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:37.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.420+0000 7f94aa1d7700 1 --2- 172.21.15.38:0/3350941114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a40f6860 0x7f94a4109700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:37.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.420+0000 7f94aa1d7700 1 -- 172.21.15.38:0/3350941114 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9498005040 con 0x7f94a40f55c0 2024-02-15T18:40:37.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.421+0000 7f94aa1d7700 1 --2- 172.21.15.38:0/3350941114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94a40f55c0 0x7f94a4109210 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f94a00026d0 tx=0x7f94a0009dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:37.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.421+0000 7f9496ffd700 1 -- 172.21.15.38:0/3350941114 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94a00079c0 con 0x7f94a40f55c0 2024-02-15T18:40:37.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.421+0000 7f9496ffd700 1 -- 172.21.15.38:0/3350941114 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f94a000f040 con 0x7f94a40f55c0 2024-02-15T18:40:37.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.421+0000 7f94ac43b700 1 -- 172.21.15.38:0/3350941114 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a410a790 con 0x7f94a40f55c0 2024-02-15T18:40:37.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.421+0000 7f9496ffd700 1 -- 172.21.15.38:0/3350941114 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94a0014950 con 0x7f94a40f55c0 2024-02-15T18:40:37.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.421+0000 7f94ac43b700 1 -- 172.21.15.38:0/3350941114 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a410ad50 con 0x7f94a40f55c0 2024-02-15T18:40:37.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.422+0000 7f94ac43b700 1 -- 172.21.15.38:0/3350941114 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a4045ad0 con 0x7f94a40f55c0 2024-02-15T18:40:37.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.422+0000 7f9496ffd700 1 -- 172.21.15.38:0/3350941114 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f94a001b060 con 0x7f94a40f55c0 2024-02-15T18:40:37.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.422+0000 7f9496ffd700 1 --2- 172.21.15.38:0/3350941114 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9490060f20 0x7f94900633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:37.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.423+0000 7f9496ffd700 1 -- 172.21.15.38:0/3350941114 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f94a007bfb0 con 0x7f94a40f55c0 2024-02-15T18:40:37.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.423+0000 7f94a99d6700 1 --2- 172.21.15.38:0/3350941114 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9490060f20 0x7f94900633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:37.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.423+0000 7f94a99d6700 1 --2- 172.21.15.38:0/3350941114 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9490060f20 0x7f94900633e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f9498004630 tx=0x7f9498009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:37.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.427+0000 7f9496ffd700 1 -- 172.21.15.38:0/3350941114 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f94a004fe00 con 0x7f94a40f55c0 2024-02-15T18:40:37.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:36.345657+0000 mgr.smithi038.khesbl (mgr.14664) 241 : audit [DBG] from='client.34290 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:40:37.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:36.358011+0000 mon.smithi038 (mon.0) 469 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi121.mvumlh=up:active} 2 up:standby 2024-02-15T18:40:37.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:36.652275+0000 mgr.smithi038.khesbl (mgr.14664) 242 : audit [DBG] from='client.34294 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:40:37.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:36.949658+0000 mgr.smithi038.khesbl (mgr.14664) 243 : audit [DBG] from='client.34298 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:40:37.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:36.990165+0000 mgr.smithi038.khesbl (mgr.14664) 244 : cluster [DBG] pgmap v172: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:37.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:37.314866+0000 mon.smithi038 (mon.0) 470 : audit [DBG] from='client.? 172.21.15.38:0/1929440610' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:37.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.643+0000 7f94ac43b700 1 -- 172.21.15.38:0/3350941114 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f94a410f830 con 0x7f94a40f55c0 2024-02-15T18:40:37.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.644+0000 7f9496ffd700 1 -- 172.21.15.38:0/3350941114 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1590 (secure 0 0 0) 0x7f94a004f9f0 con 0x7f94a40f55c0 2024-02-15T18:40:37.633 INFO:teuthology.orchestra.run.smithi038.stdout:e19 2024-02-15T18:40:37.633 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:40:37.633 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:40:37.633 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-02-15T18:40:37.633 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:40:37.633 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:40:37.633 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-02-15T18:40:37.633 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 17 2024-02-15T18:40:37.633 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 2024-02-15T18:40:37.633 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:40:37.633 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-02-15T18:40:28.060531+0000 2024-02-15T18:40:37.633 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-02-15T18:40:37.633 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-02-15T18:40:37.634 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-02-15T18:40:37.634 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-02-15T18:40:37.634 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-02-15T18:40:37.634 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-02-15T18:40:37.634 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-02-15T18:40:37.634 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-02-15T18:40:37.634 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:40:37.634 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 1 2024-02-15T18:40:37.634 INFO:teuthology.orchestra.run.smithi038.stdout:in 0 2024-02-15T18:40:37.634 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=24261} 2024-02-15T18:40:37.634 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-02-15T18:40:37.634 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-02-15T18:40:37.635 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 1 2024-02-15T18:40:37.635 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-02-15T18:40:37.635 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-02-15T18:40:37.635 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-02-15T18:40:37.635 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-02-15T18:40:37.635 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-02-15T18:40:37.635 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.mvumlh{0:24261} state up:active seq 3 join_fscid=1 addr [v2:172.21.15.121:6824/1422707571,v1:172.21.15.121:6825/1422707571] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:40:37.635 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:40:37.635 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:40:37.635 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-02-15T18:40:37.635 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:40:37.635 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.qgkzgn{-1:14456} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.38:6828/703030231,v1:172.21.15.38:6829/703030231] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:40:37.635 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.ovuzki{-1:44223} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.121:6826/554561289,v1:172.21.15.121:6827/554561289] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:40:37.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.646+0000 7f94ac43b700 1 -- 172.21.15.38:0/3350941114 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9490060f20 msgr2=0x7f94900633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:37.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.646+0000 7f94ac43b700 1 --2- 172.21.15.38:0/3350941114 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9490060f20 0x7f94900633e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f9498004630 tx=0x7f9498009b30).stop 2024-02-15T18:40:37.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.646+0000 7f94ac43b700 1 -- 172.21.15.38:0/3350941114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94a40f55c0 msgr2=0x7f94a4109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:37.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.646+0000 7f94ac43b700 1 --2- 172.21.15.38:0/3350941114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94a40f55c0 0x7f94a4109210 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f94a00026d0 tx=0x7f94a0009dc0).stop 2024-02-15T18:40:37.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.646+0000 7f94ac43b700 1 -- 172.21.15.38:0/3350941114 shutdown_connections 2024-02-15T18:40:37.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.646+0000 7f94ac43b700 1 --2- 172.21.15.38:0/3350941114 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9490060f20 0x7f94900633e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:37.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.646+0000 7f94ac43b700 1 --2- 172.21.15.38:0/3350941114 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94a40f55c0 0x7f94a4109210 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:37.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.646+0000 7f94ac43b700 1 --2- 172.21.15.38:0/3350941114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a40f6860 0x7f94a4109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:37.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.646+0000 7f94ac43b700 1 -- 172.21.15.38:0/3350941114 >> 172.21.15.38:0/3350941114 conn(0x7f94a40f0d40 msgr2=0x7f94a405fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:37.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.646+0000 7f94ac43b700 1 -- 172.21.15.38:0/3350941114 shutdown_connections 2024-02-15T18:40:37.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.646+0000 7f94ac43b700 1 -- 172.21.15.38:0/3350941114 wait complete. 2024-02-15T18:40:37.637 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 19 2024-02-15T18:40:37.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.748+0000 7f46d8811700 1 -- 172.21.15.38:0/1996275679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46d40f6860 msgr2=0x7f46d40f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:37.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.748+0000 7f46d8811700 1 --2- 172.21.15.38:0/1996275679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46d40f6860 0x7f46d40f6cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f46c4004660 tx=0x7f46c4009b30).stop 2024-02-15T18:40:37.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.748+0000 7f46d8811700 1 -- 172.21.15.38:0/1996275679 shutdown_connections 2024-02-15T18:40:37.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.748+0000 7f46d8811700 1 --2- 172.21.15.38:0/1996275679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46d40f6860 0x7f46d40f6cc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:37.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.748+0000 7f46d8811700 1 --2- 172.21.15.38:0/1996275679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46d40f55c0 0x7f46d40f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:37.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.748+0000 7f46d8811700 1 -- 172.21.15.38:0/1996275679 >> 172.21.15.38:0/1996275679 conn(0x7f46d40f0d40 msgr2=0x7f46d40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:37.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.748+0000 7f46d8811700 1 -- 172.21.15.38:0/1996275679 shutdown_connections 2024-02-15T18:40:37.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.748+0000 7f46d8811700 1 -- 172.21.15.38:0/1996275679 wait complete. 2024-02-15T18:40:37.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.749+0000 7f46d8811700 1 Processor -- start 2024-02-15T18:40:37.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.749+0000 7f46d8811700 1 -- start start 2024-02-15T18:40:37.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.749+0000 7f46d8811700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46d40f55c0 0x7f46d4105eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:37.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.749+0000 7f46d8811700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46d40f6860 0x7f46d41063a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:37.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.749+0000 7f46d8811700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d4106950 con 0x7f46d40f6860 2024-02-15T18:40:37.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.749+0000 7f46d8811700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d4106a90 con 0x7f46d40f55c0 2024-02-15T18:40:37.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.749+0000 7f46d1d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46d40f6860 0x7f46d41063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:37.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.749+0000 7f46d1d9b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46d40f6860 0x7f46d41063a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56590/0 (socket says 172.21.15.38:56590) 2024-02-15T18:40:37.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.749+0000 7f46d1d9b700 1 -- 172.21.15.38:0/187414758 learned_addr learned my addr 172.21.15.38:0/187414758 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:40:37.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.749+0000 7f46d259c700 1 --2- 172.21.15.38:0/187414758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46d40f55c0 0x7f46d4105eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:37.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.749+0000 7f46d1d9b700 1 -- 172.21.15.38:0/187414758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46d40f55c0 msgr2=0x7f46d4105eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:37.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.749+0000 7f46d1d9b700 1 --2- 172.21.15.38:0/187414758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46d40f55c0 0x7f46d4105eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:37.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.749+0000 7f46d1d9b700 1 -- 172.21.15.38:0/187414758 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c4005040 con 0x7f46d40f6860 2024-02-15T18:40:37.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.749+0000 7f46d259c700 1 --2- 172.21.15.38:0/187414758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46d40f55c0 0x7f46d4105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:40:37.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.750+0000 7f46d1d9b700 1 --2- 172.21.15.38:0/187414758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46d40f6860 0x7f46d41063a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f46c4009f50 tx=0x7f46c4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:37.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.750+0000 7f46c2ffd700 1 -- 172.21.15.38:0/187414758 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46c4018070 con 0x7f46d40f6860 2024-02-15T18:40:37.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.750+0000 7f46d8811700 1 -- 172.21.15.38:0/187414758 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d41073d0 con 0x7f46d40f6860 2024-02-15T18:40:37.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.750+0000 7f46c2ffd700 1 -- 172.21.15.38:0/187414758 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f46c4007ba0 con 0x7f46d40f6860 2024-02-15T18:40:37.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.750+0000 7f46c2ffd700 1 -- 172.21.15.38:0/187414758 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46c401c620 con 0x7f46d40f6860 2024-02-15T18:40:37.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.750+0000 7f46d8811700 1 -- 172.21.15.38:0/187414758 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d4107930 con 0x7f46d40f6860 2024-02-15T18:40:37.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.751+0000 7f46d8811700 1 -- 172.21.15.38:0/187414758 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d4045ad0 con 0x7f46d40f6860 2024-02-15T18:40:37.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.751+0000 7f46c2ffd700 1 -- 172.21.15.38:0/187414758 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f46c401c890 con 0x7f46d40f6860 2024-02-15T18:40:37.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.752+0000 7f46c2ffd700 1 --2- 172.21.15.38:0/187414758 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f46bc060f20 0x7f46bc0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:40:37.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.752+0000 7f46c2ffd700 1 -- 172.21.15.38:0/187414758 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f46c407df50 con 0x7f46d40f6860 2024-02-15T18:40:37.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.752+0000 7f46d259c700 1 --2- 172.21.15.38:0/187414758 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f46bc060f20 0x7f46bc0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:40:37.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.752+0000 7f46d259c700 1 --2- 172.21.15.38:0/187414758 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f46bc060f20 0x7f46bc0633e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f46d40f66c0 tx=0x7f46c8009c00).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:40:37.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.756+0000 7f46c2ffd700 1 -- 172.21.15.38:0/187414758 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f46c40acf60 con 0x7f46d40f6860 2024-02-15T18:40:37.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:36.345657+0000 mgr.smithi038.khesbl (mgr.14664) 241 : audit [DBG] from='client.34290 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:40:37.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:36.358011+0000 mon.smithi038 (mon.0) 469 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi121.mvumlh=up:active} 2 up:standby 2024-02-15T18:40:37.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:36.652275+0000 mgr.smithi038.khesbl (mgr.14664) 242 : audit [DBG] from='client.34294 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:40:37.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:36.949658+0000 mgr.smithi038.khesbl (mgr.14664) 243 : audit [DBG] from='client.34298 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:40:37.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:36.990165+0000 mgr.smithi038.khesbl (mgr.14664) 244 : cluster [DBG] pgmap v172: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:37.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:37.314866+0000 mon.smithi038 (mon.0) 470 : audit [DBG] from='client.? 172.21.15.38:0/1929440610' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:37.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.945+0000 7f46d8811700 1 -- 172.21.15.38:0/187414758 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f46d405f620 con 0x7f46bc060f20 2024-02-15T18:40:37.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.946+0000 7f46c2ffd700 1 -- 172.21.15.38:0/187414758 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f46d405f620 con 0x7f46bc060f20 2024-02-15T18:40:37.935 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:40:37.935 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:40:37.935 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-02-15T18:40:37.935 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:40:37.935 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-02-15T18:40:37.935 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-02-15T18:40:37.935 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-02-15T18:40:37.935 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr", 2024-02-15T18:40:37.935 INFO:teuthology.orchestra.run.smithi038.stdout: "osd" 2024-02-15T18:40:37.935 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-02-15T18:40:37.935 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "12/21 daemons upgraded", 2024-02-15T18:40:37.935 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading mds daemons", 2024-02-15T18:40:37.936 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-02-15T18:40:37.936 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:40:37.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.948+0000 7f46d8811700 1 -- 172.21.15.38:0/187414758 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f46bc060f20 msgr2=0x7f46bc0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:37.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.948+0000 7f46d8811700 1 --2- 172.21.15.38:0/187414758 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f46bc060f20 0x7f46bc0633e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f46d40f66c0 tx=0x7f46c8009c00).stop 2024-02-15T18:40:37.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.948+0000 7f46d8811700 1 -- 172.21.15.38:0/187414758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46d40f6860 msgr2=0x7f46d41063a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:40:37.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.948+0000 7f46d8811700 1 --2- 172.21.15.38:0/187414758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46d40f6860 0x7f46d41063a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f46c4009f50 tx=0x7f46c4009b30).stop 2024-02-15T18:40:37.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.948+0000 7f46d8811700 1 -- 172.21.15.38:0/187414758 shutdown_connections 2024-02-15T18:40:37.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.948+0000 7f46d8811700 1 --2- 172.21.15.38:0/187414758 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f46bc060f20 0x7f46bc0633e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:37.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.948+0000 7f46d8811700 1 --2- 172.21.15.38:0/187414758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f46d40f55c0 0x7f46d4105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:37.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.948+0000 7f46d8811700 1 --2- 172.21.15.38:0/187414758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46d40f6860 0x7f46d41063a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:40:37.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.948+0000 7f46d8811700 1 -- 172.21.15.38:0/187414758 >> 172.21.15.38:0/187414758 conn(0x7f46d40f0d40 msgr2=0x7f46d40fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:40:37.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.948+0000 7f46d8811700 1 -- 172.21.15.38:0/187414758 shutdown_connections 2024-02-15T18:40:37.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:40:37.948+0000 7f46d8811700 1 -- 172.21.15.38:0/187414758 wait complete. 2024-02-15T18:40:38.647 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:38 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:37.644562+0000 mon.smithi038 (mon.0) 471 : audit [DBG] from='client.? 172.21.15.38:0/3350941114' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:40:38.647 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:38 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:37.946330+0000 mgr.smithi038.khesbl (mgr.14664) 245 : audit [DBG] from='client.34310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:40:38.855 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:38 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:37.644562+0000 mon.smithi038 (mon.0) 471 : audit [DBG] from='client.? 172.21.15.38:0/3350941114' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:40:38.855 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:38 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:37.946330+0000 mgr.smithi038.khesbl (mgr.14664) 245 : audit [DBG] from='client.34310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:40:39.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:38.990667+0000 mgr.smithi038.khesbl (mgr.14664) 246 : cluster [DBG] pgmap v173: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:39.855 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:38.990667+0000 mgr.smithi038.khesbl (mgr.14664) 246 : cluster [DBG] pgmap v173: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:41.231 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:40.240384+0000 mon.smithi038 (mon.0) 472 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:41.232 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:40.241103+0000 mon.smithi038 (mon.0) 473 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:41.232 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:40.376986+0000 mon.smithi038 (mon.0) 474 : cluster [DBG] mds.? [v2:172.21.15.38:6826/4077527450,v1:172.21.15.38:6827/4077527450] up:boot 2024-02-15T18:40:41.232 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:40.377049+0000 mon.smithi038 (mon.0) 475 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi121.mvumlh=up:active} 3 up:standby 2024-02-15T18:40:41.232 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:40.377175+0000 mon.smithi038 (mon.0) 476 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.hxoggl"}]: dispatch 2024-02-15T18:40:41.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:40.991190+0000 mgr.smithi038.khesbl (mgr.14664) 247 : cluster [DBG] pgmap v174: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:41.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:40.240384+0000 mon.smithi038 (mon.0) 472 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:41.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:40.241103+0000 mon.smithi038 (mon.0) 473 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:41.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:40.376986+0000 mon.smithi038 (mon.0) 474 : cluster [DBG] mds.? [v2:172.21.15.38:6826/4077527450,v1:172.21.15.38:6827/4077527450] up:boot 2024-02-15T18:40:41.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:40.377049+0000 mon.smithi038 (mon.0) 475 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi121.mvumlh=up:active} 3 up:standby 2024-02-15T18:40:41.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:40.377175+0000 mon.smithi038 (mon.0) 476 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.hxoggl"}]: dispatch 2024-02-15T18:40:41.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:40.991190+0000 mgr.smithi038.khesbl (mgr.14664) 247 : cluster [DBG] pgmap v174: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:43.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:42.991552+0000 mgr.smithi038.khesbl (mgr.14664) 248 : cluster [DBG] pgmap v175: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:43.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:42.991552+0000 mgr.smithi038.khesbl (mgr.14664) 248 : cluster [DBG] pgmap v175: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:45.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:44.991935+0000 mgr.smithi038.khesbl (mgr.14664) 249 : cluster [DBG] pgmap v176: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:45.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:44.991935+0000 mgr.smithi038.khesbl (mgr.14664) 249 : cluster [DBG] pgmap v176: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:48.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:46.983479+0000 mon.smithi038 (mon.0) 477 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:48.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:46.984278+0000 mon.smithi038 (mon.0) 478 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:40:48.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:46.984916+0000 mon.smithi038 (mon.0) 479 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:40:48.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:40:46.985434+0000 mgr.smithi038.khesbl (mgr.14664) 250 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:40:48.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:46.988890+0000 mon.smithi038 (mon.0) 480 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:48.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:46.992350+0000 mgr.smithi038.khesbl (mgr.14664) 251 : cluster [DBG] pgmap v177: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:48.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:46.994410+0000 mon.smithi038 (mon.0) 481 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:48.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:46.996444+0000 mon.smithi038 (mon.0) 482 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:48.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:46.997567+0000 mon.smithi038 (mon.0) 483 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:48.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:46.998691+0000 mon.smithi038 (mon.0) 484 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:48.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:47.000170+0000 mon.smithi038 (mon.0) 485 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:48.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:47.001860+0000 mon.smithi038 (mon.0) 486 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi038.qgkzgn"]}]: dispatch 2024-02-15T18:40:48.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:40:47.002462+0000 mgr.smithi038.khesbl (mgr.14664) 252 : cephadm [INF] Upgrade: It appears safe to stop mds.cephfs.smithi038.qgkzgn 2024-02-15T18:40:48.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:46.983479+0000 mon.smithi038 (mon.0) 477 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:48.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:46.984278+0000 mon.smithi038 (mon.0) 478 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:40:48.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:46.984916+0000 mon.smithi038 (mon.0) 479 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:40:48.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:40:46.985434+0000 mgr.smithi038.khesbl (mgr.14664) 250 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:40:48.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:46.988890+0000 mon.smithi038 (mon.0) 480 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:48.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:46.992350+0000 mgr.smithi038.khesbl (mgr.14664) 251 : cluster [DBG] pgmap v177: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:48.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:46.994410+0000 mon.smithi038 (mon.0) 481 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:48.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:46.996444+0000 mon.smithi038 (mon.0) 482 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:48.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:46.997567+0000 mon.smithi038 (mon.0) 483 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:48.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:46.998691+0000 mon.smithi038 (mon.0) 484 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:48.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:47.000170+0000 mon.smithi038 (mon.0) 485 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:40:48.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:47.001860+0000 mon.smithi038 (mon.0) 486 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi038.qgkzgn"]}]: dispatch 2024-02-15T18:40:48.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:40:47.002462+0000 mgr.smithi038.khesbl (mgr.14664) 252 : cephadm [INF] Upgrade: It appears safe to stop mds.cephfs.smithi038.qgkzgn 2024-02-15T18:40:50.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:40:48.883594+0000 mgr.smithi038.khesbl (mgr.14664) 253 : cephadm [INF] Upgrade: Updating mds.cephfs.smithi038.qgkzgn 2024-02-15T18:40:50.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:48.889412+0000 mon.smithi038 (mon.0) 487 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:50.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:48.890236+0000 mon.smithi038 (mon.0) 488 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.qgkzgn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:40:50.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:48.890907+0000 mon.smithi038 (mon.0) 489 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:40:50.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:40:48.891498+0000 mgr.smithi038.khesbl (mgr.14664) 254 : cephadm [INF] Deploying daemon mds.cephfs.smithi038.qgkzgn on smithi038 2024-02-15T18:40:50.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:48.992728+0000 mgr.smithi038.khesbl (mgr.14664) 255 : cluster [DBG] pgmap v178: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:50.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:40:48.883594+0000 mgr.smithi038.khesbl (mgr.14664) 253 : cephadm [INF] Upgrade: Updating mds.cephfs.smithi038.qgkzgn 2024-02-15T18:40:50.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:48.889412+0000 mon.smithi038 (mon.0) 487 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:50.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:48.890236+0000 mon.smithi038 (mon.0) 488 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi038.qgkzgn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:40:50.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:48.890907+0000 mon.smithi038 (mon.0) 489 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:40:50.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:40:48.891498+0000 mgr.smithi038.khesbl (mgr.14664) 254 : cephadm [INF] Deploying daemon mds.cephfs.smithi038.qgkzgn on smithi038 2024-02-15T18:40:50.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:48.992728+0000 mgr.smithi038.khesbl (mgr.14664) 255 : cluster [DBG] pgmap v178: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:51.260 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:50.892752+0000 mon.smithi038 (mon.0) 490 : cluster [WRN] daemon mds.cephfs.smithi038.qgkzgn compat changed unexpectedly 2024-02-15T18:40:51.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:50.892752+0000 mon.smithi038 (mon.0) 490 : cluster [WRN] daemon mds.cephfs.smithi038.qgkzgn compat changed unexpectedly 2024-02-15T18:40:52.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:50.901101+0000 mon.smithi038 (mon.0) 491 : cluster [DBG] osdmap e78: 6 total, 6 up, 6 in 2024-02-15T18:40:52.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:50.917942+0000 mon.smithi038 (mon.0) 492 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi121.mvumlh=up:active} 2 up:standby 2024-02-15T18:40:52.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:50.993205+0000 mgr.smithi038.khesbl (mgr.14664) 256 : cluster [DBG] pgmap v180: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:52.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:50.901101+0000 mon.smithi038 (mon.0) 491 : cluster [DBG] osdmap e78: 6 total, 6 up, 6 in 2024-02-15T18:40:52.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:50.917942+0000 mon.smithi038 (mon.0) 492 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi121.mvumlh=up:active} 2 up:standby 2024-02-15T18:40:52.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:50.993205+0000 mgr.smithi038.khesbl (mgr.14664) 256 : cluster [DBG] pgmap v180: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:53.503 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:52.993639+0000 mgr.smithi038.khesbl (mgr.14664) 257 : cluster [DBG] pgmap v181: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:53.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:52.993639+0000 mgr.smithi038.khesbl (mgr.14664) 257 : cluster [DBG] pgmap v181: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:55.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:54.606791+0000 mon.smithi038 (mon.0) 493 : cluster [DBG] mds.? [v2:172.21.15.38:6828/1616551408,v1:172.21.15.38:6829/1616551408] up:boot 2024-02-15T18:40:55.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:54.606832+0000 mon.smithi038 (mon.0) 494 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi121.mvumlh=up:active} 3 up:standby 2024-02-15T18:40:55.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:54.607502+0000 mon.smithi038 (mon.0) 495 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.qgkzgn"}]: dispatch 2024-02-15T18:40:55.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:54.634243+0000 mon.smithi038 (mon.0) 496 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:55.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:54.635031+0000 mon.smithi038 (mon.0) 497 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:55.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:54.994030+0000 mgr.smithi038.khesbl (mgr.14664) 258 : cluster [DBG] pgmap v182: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:55.855 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:54.606791+0000 mon.smithi038 (mon.0) 493 : cluster [DBG] mds.? [v2:172.21.15.38:6828/1616551408,v1:172.21.15.38:6829/1616551408] up:boot 2024-02-15T18:40:55.855 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:54.606832+0000 mon.smithi038 (mon.0) 494 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi121.mvumlh=up:active} 3 up:standby 2024-02-15T18:40:55.855 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:54.607502+0000 mon.smithi038 (mon.0) 495 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi038.qgkzgn"}]: dispatch 2024-02-15T18:40:55.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:54.634243+0000 mon.smithi038 (mon.0) 496 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:40:55.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:54.635031+0000 mon.smithi038 (mon.0) 497 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:40:55.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:54.994030+0000 mgr.smithi038.khesbl (mgr.14664) 258 : cluster [DBG] pgmap v182: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:57.331 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:56.994384+0000 mgr.smithi038.khesbl (mgr.14664) 259 : cluster [DBG] pgmap v183: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:57.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:56.994384+0000 mgr.smithi038.khesbl (mgr.14664) 259 : cluster [DBG] pgmap v183: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:59.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:40:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:40:58.994930+0000 mgr.smithi038.khesbl (mgr.14664) 260 : cluster [DBG] pgmap v184: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:40:59.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:40:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:40:58.994930+0000 mgr.smithi038.khesbl (mgr.14664) 260 : cluster [DBG] pgmap v184: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:41:01.260 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:40:59.935372+0000 mon.smithi038 (mon.0) 498 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:01.260 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:00.227156+0000 mon.smithi038 (mon.0) 499 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:41:01.260 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:00.227819+0000 mon.smithi038 (mon.0) 500 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:41:01.260 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:00.236383+0000 mon.smithi038 (mon.0) 501 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:01.260 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:00.241167+0000 mon.smithi038 (mon.0) 502 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:41:01.260 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:00.243156+0000 mon.smithi038 (mon.0) 503 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:01.260 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:00.244235+0000 mon.smithi038 (mon.0) 504 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:01.260 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:00.245293+0000 mon.smithi038 (mon.0) 505 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:01.261 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:00.246656+0000 mon.smithi038 (mon.0) 506 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:01.261 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:00.248188+0000 mon.smithi038 (mon.0) 507 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi121.mvumlh"]}]: dispatch 2024-02-15T18:41:01.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:40:59.935372+0000 mon.smithi038 (mon.0) 498 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:01.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:00.227156+0000 mon.smithi038 (mon.0) 499 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:41:01.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:00.227819+0000 mon.smithi038 (mon.0) 500 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:41:01.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:00.236383+0000 mon.smithi038 (mon.0) 501 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:01.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:00.241167+0000 mon.smithi038 (mon.0) 502 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:41:01.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:00.243156+0000 mon.smithi038 (mon.0) 503 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:01.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:00.244235+0000 mon.smithi038 (mon.0) 504 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:01.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:00.245293+0000 mon.smithi038 (mon.0) 505 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:01.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:00.246656+0000 mon.smithi038 (mon.0) 506 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:01.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:00.248188+0000 mon.smithi038 (mon.0) 507 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi121.mvumlh"]}]: dispatch 2024-02-15T18:41:02.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:00.228334+0000 mgr.smithi038.khesbl (mgr.14664) 261 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:41:02.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:00.248653+0000 mgr.smithi038.khesbl (mgr.14664) 262 : cephadm [INF] Upgrade: It appears safe to stop mds.cephfs.smithi121.mvumlh 2024-02-15T18:41:02.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:00.995539+0000 mgr.smithi038.khesbl (mgr.14664) 263 : cluster [DBG] pgmap v185: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:41:02.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:01.749372+0000 mon.smithi038 (mon.0) 508 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:02.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:01.750326+0000 mon.smithi038 (mon.0) 509 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi121.mvumlh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:41:02.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:01.751289+0000 mon.smithi038 (mon.0) 510 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:41:02.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:00.228334+0000 mgr.smithi038.khesbl (mgr.14664) 261 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:41:02.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:00.248653+0000 mgr.smithi038.khesbl (mgr.14664) 262 : cephadm [INF] Upgrade: It appears safe to stop mds.cephfs.smithi121.mvumlh 2024-02-15T18:41:02.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:00.995539+0000 mgr.smithi038.khesbl (mgr.14664) 263 : cluster [DBG] pgmap v185: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:41:02.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:01.749372+0000 mon.smithi038 (mon.0) 508 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:02.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:01.750326+0000 mon.smithi038 (mon.0) 509 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi121.mvumlh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:41:02.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:01.751289+0000 mon.smithi038 (mon.0) 510 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:41:03.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:01.740059+0000 mgr.smithi038.khesbl (mgr.14664) 264 : cephadm [INF] Upgrade: Updating mds.cephfs.smithi121.mvumlh 2024-02-15T18:41:03.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:01.752019+0000 mgr.smithi038.khesbl (mgr.14664) 265 : cephadm [INF] Deploying daemon mds.cephfs.smithi121.mvumlh on smithi121 2024-02-15T18:41:03.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:01.740059+0000 mgr.smithi038.khesbl (mgr.14664) 264 : cephadm [INF] Upgrade: Updating mds.cephfs.smithi121.mvumlh 2024-02-15T18:41:03.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:01.752019+0000 mgr.smithi038.khesbl (mgr.14664) 265 : cephadm [INF] Deploying daemon mds.cephfs.smithi121.mvumlh on smithi121 2024-02-15T18:41:04.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:41:03.945+0000 7f3d723c6700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-15T18:41:04.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:02.995929+0000 mgr.smithi038.khesbl (mgr.14664) 266 : cluster [DBG] pgmap v186: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:41:04.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:03.443840+0000 mon.smithi038 (mon.0) 511 : cluster [DBG] osdmap e79: 6 total, 6 up, 6 in 2024-02-15T18:41:04.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:02.995929+0000 mgr.smithi038.khesbl (mgr.14664) 266 : cluster [DBG] pgmap v186: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:41:04.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:03.443840+0000 mon.smithi038 (mon.0) 511 : cluster [DBG] osdmap e79: 6 total, 6 up, 6 in 2024-02-15T18:41:05.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:04 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:03.946699+0000 mon.smithi038 (mon.0) 512 : cluster [WRN] Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-15T18:41:05.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:04 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:03.946733+0000 mon.smithi038 (mon.0) 513 : cluster [ERR] Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-15T18:41:05.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:04 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:03.954845+0000 mon.smithi038 (mon.0) 514 : cluster [INF] Standby daemon mds.cephfs.smithi038.hxoggl assigned to filesystem cephfs as rank 0 2024-02-15T18:41:05.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:04 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:03.954954+0000 mon.smithi038 (mon.0) 515 : cluster [INF] Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-15T18:41:05.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:04 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:03.955252+0000 mon.smithi038 (mon.0) 516 : cluster [DBG] fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-15T18:41:05.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:04 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:03.959737+0000 mon.smithi038 (mon.0) 517 : cluster [DBG] fsmap cephfs:1/1 {0=cephfs.smithi038.hxoggl=up:replay} 2 up:standby 2024-02-15T18:41:05.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:04 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:03.978572+0000 mon.smithi038 (mon.0) 518 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.mvumlh"}]: dispatch 2024-02-15T18:41:05.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:04 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:03.946699+0000 mon.smithi038 (mon.0) 512 : cluster [WRN] Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-15T18:41:05.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:04 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:03.946733+0000 mon.smithi038 (mon.0) 513 : cluster [ERR] Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-15T18:41:05.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:04 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:03.954845+0000 mon.smithi038 (mon.0) 514 : cluster [INF] Standby daemon mds.cephfs.smithi038.hxoggl assigned to filesystem cephfs as rank 0 2024-02-15T18:41:05.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:04 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:03.954954+0000 mon.smithi038 (mon.0) 515 : cluster [INF] Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-15T18:41:05.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:04 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:03.955252+0000 mon.smithi038 (mon.0) 516 : cluster [DBG] fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-15T18:41:05.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:04 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:03.959737+0000 mon.smithi038 (mon.0) 517 : cluster [DBG] fsmap cephfs:1/1 {0=cephfs.smithi038.hxoggl=up:replay} 2 up:standby 2024-02-15T18:41:05.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:04 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:03.978572+0000 mon.smithi038 (mon.0) 518 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.mvumlh"}]: dispatch 2024-02-15T18:41:06.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:04.996271+0000 mgr.smithi038.khesbl (mgr.14664) 267 : cluster [DBG] pgmap v188: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 0 op/s 2024-02-15T18:41:06.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:04.996271+0000 mgr.smithi038.khesbl (mgr.14664) 267 : cluster [DBG] pgmap v188: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 0 op/s 2024-02-15T18:41:07.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:06.996691+0000 mgr.smithi038.khesbl (mgr.14664) 268 : cluster [DBG] pgmap v189: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 6.0 MiB/s rd, 2 op/s 2024-02-15T18:41:07.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:06.996691+0000 mgr.smithi038.khesbl (mgr.14664) 268 : cluster [DBG] pgmap v189: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 6.0 MiB/s rd, 2 op/s 2024-02-15T18:41:08.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.052+0000 7faeff064700 1 -- 172.21.15.38:0/4019459940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faef80f9690 msgr2=0x7faef80fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:08.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.052+0000 7faeff064700 1 --2- 172.21.15.38:0/4019459940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faef80f9690 0x7faef80fbae0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7faee8004660 tx=0x7faee8009b30).stop 2024-02-15T18:41:08.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.052+0000 7faeff064700 1 -- 172.21.15.38:0/4019459940 shutdown_connections 2024-02-15T18:41:08.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.052+0000 7faeff064700 1 --2- 172.21.15.38:0/4019459940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faef80f9690 0x7faef80fbae0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.052+0000 7faeff064700 1 --2- 172.21.15.38:0/4019459940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faef8060310 0x7faef80f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.052+0000 7faeff064700 1 -- 172.21.15.38:0/4019459940 >> 172.21.15.38:0/4019459940 conn(0x7faef80f0b90 msgr2=0x7faef80f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:08.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.052+0000 7faeff064700 1 -- 172.21.15.38:0/4019459940 shutdown_connections 2024-02-15T18:41:08.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.052+0000 7faeff064700 1 -- 172.21.15.38:0/4019459940 wait complete. 2024-02-15T18:41:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.053+0000 7faeff064700 1 Processor -- start 2024-02-15T18:41:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.053+0000 7faeff064700 1 -- start start 2024-02-15T18:41:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.053+0000 7faeff064700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faef8060310 0x7faef8106e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.053+0000 7faeff064700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faef80f9690 0x7faef8107300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.053+0000 7faeff064700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef8107900 con 0x7faef80f9690 2024-02-15T18:41:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.053+0000 7faeff064700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef8108110 con 0x7faef8060310 2024-02-15T18:41:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.053+0000 7faefd861700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faef80f9690 0x7faef8107300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.053+0000 7faefd861700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faef80f9690 0x7faef8107300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39534/0 (socket says 172.21.15.38:39534) 2024-02-15T18:41:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.053+0000 7faefd861700 1 -- 172.21.15.38:0/2992116433 learned_addr learned my addr 172.21.15.38:0/2992116433 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:41:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.053+0000 7faefe062700 1 --2- 172.21.15.38:0/2992116433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faef8060310 0x7faef8106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:08.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.053+0000 7faefd861700 1 -- 172.21.15.38:0/2992116433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faef8060310 msgr2=0x7faef8106e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:08.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.053+0000 7faefd861700 1 --2- 172.21.15.38:0/2992116433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faef8060310 0x7faef8106e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.053+0000 7faefd861700 1 -- 172.21.15.38:0/2992116433 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee8005040 con 0x7faef80f9690 2024-02-15T18:41:08.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.054+0000 7faefd861700 1 --2- 172.21.15.38:0/2992116433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faef80f9690 0x7faef8107300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7faee8013fd0 tx=0x7faee8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:08.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.054+0000 7faeeeffd700 1 -- 172.21.15.38:0/2992116433 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faee8018070 con 0x7faef80f9690 2024-02-15T18:41:08.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.054+0000 7faeeeffd700 1 -- 172.21.15.38:0/2992116433 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7faee8007c70 con 0x7faef80f9690 2024-02-15T18:41:08.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.054+0000 7faeff064700 1 -- 172.21.15.38:0/2992116433 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef8108330 con 0x7faef80f9690 2024-02-15T18:41:08.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.054+0000 7faeeeffd700 1 -- 172.21.15.38:0/2992116433 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faee801c620 con 0x7faef80f9690 2024-02-15T18:41:08.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.054+0000 7faeff064700 1 -- 172.21.15.38:0/2992116433 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef8108890 con 0x7faef80f9690 2024-02-15T18:41:08.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.055+0000 7faeff064700 1 -- 172.21.15.38:0/2992116433 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef8059100 con 0x7faef80f9690 2024-02-15T18:41:08.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.056+0000 7faeeeffd700 1 -- 172.21.15.38:0/2992116433 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7faee801d420 con 0x7faef80f9690 2024-02-15T18:41:08.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.057+0000 7faeeeffd700 1 --2- 172.21.15.38:0/2992116433 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7faee4060f70 0x7faee4063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:08.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.057+0000 7faeeeffd700 1 -- 172.21.15.38:0/2992116433 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5898+0+0 (secure 0 0 0) 0x7faee807e5b0 con 0x7faef80f9690 2024-02-15T18:41:08.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.057+0000 7faefe062700 1 --2- 172.21.15.38:0/2992116433 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7faee4060f70 0x7faee4063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:08.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.057+0000 7faefe062700 1 --2- 172.21.15.38:0/2992116433 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7faee4060f70 0x7faee4063430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7faef40041d0 tx=0x7faef4009040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:08.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.060+0000 7faeeeffd700 1 -- 172.21.15.38:0/2992116433 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7faee804ea60 con 0x7faef80f9690 2024-02-15T18:41:08.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.252+0000 7faeff064700 1 -- 172.21.15.38:0/2992116433 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faef80fef40 con 0x7faee4060f70 2024-02-15T18:41:08.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.254+0000 7faeeeffd700 1 -- 172.21.15.38:0/2992116433 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7faef80fef40 con 0x7faee4060f70 2024-02-15T18:41:08.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.257+0000 7faeff064700 1 -- 172.21.15.38:0/2992116433 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7faee4060f70 msgr2=0x7faee4063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:08.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.257+0000 7faeff064700 1 --2- 172.21.15.38:0/2992116433 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7faee4060f70 0x7faee4063430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7faef40041d0 tx=0x7faef4009040).stop 2024-02-15T18:41:08.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.257+0000 7faeff064700 1 -- 172.21.15.38:0/2992116433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faef80f9690 msgr2=0x7faef8107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:08.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.257+0000 7faeff064700 1 --2- 172.21.15.38:0/2992116433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faef80f9690 0x7faef8107300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7faee8013fd0 tx=0x7faee8009b30).stop 2024-02-15T18:41:08.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.257+0000 7faeff064700 1 -- 172.21.15.38:0/2992116433 shutdown_connections 2024-02-15T18:41:08.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.257+0000 7faeff064700 1 --2- 172.21.15.38:0/2992116433 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7faee4060f70 0x7faee4063430 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.257+0000 7faeff064700 1 --2- 172.21.15.38:0/2992116433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faef8060310 0x7faef8106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.257+0000 7faeff064700 1 --2- 172.21.15.38:0/2992116433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faef80f9690 0x7faef8107300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.257+0000 7faeff064700 1 -- 172.21.15.38:0/2992116433 >> 172.21.15.38:0/2992116433 conn(0x7faef80f0b90 msgr2=0x7faef80f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:08.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.257+0000 7faeff064700 1 -- 172.21.15.38:0/2992116433 shutdown_connections 2024-02-15T18:41:08.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.257+0000 7faeff064700 1 -- 172.21.15.38:0/2992116433 wait complete. 2024-02-15T18:41:08.261 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-02-15T18:41:08.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.369+0000 7f126f6dd700 1 -- 172.21.15.38:0/2582153529 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12680f6860 msgr2=0x7f12680f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:08.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.369+0000 7f126f6dd700 1 --2- 172.21.15.38:0/2582153529 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12680f6860 0x7f12680f6cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f1258004660 tx=0x7f1258009b30).stop 2024-02-15T18:41:08.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.369+0000 7f126f6dd700 1 -- 172.21.15.38:0/2582153529 shutdown_connections 2024-02-15T18:41:08.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.369+0000 7f126f6dd700 1 --2- 172.21.15.38:0/2582153529 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12680f6860 0x7f12680f6cc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.369+0000 7f126f6dd700 1 --2- 172.21.15.38:0/2582153529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12680f55c0 0x7f12680f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.369+0000 7f126f6dd700 1 -- 172.21.15.38:0/2582153529 >> 172.21.15.38:0/2582153529 conn(0x7f12680f0d40 msgr2=0x7f12680f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:08.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.369+0000 7f126f6dd700 1 -- 172.21.15.38:0/2582153529 shutdown_connections 2024-02-15T18:41:08.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.369+0000 7f126f6dd700 1 -- 172.21.15.38:0/2582153529 wait complete. 2024-02-15T18:41:08.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.370+0000 7f126f6dd700 1 Processor -- start 2024-02-15T18:41:08.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.370+0000 7f126f6dd700 1 -- start start 2024-02-15T18:41:08.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.370+0000 7f126f6dd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12680f55c0 0x7f1268106fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:08.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.370+0000 7f126f6dd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12680f6860 0x7f12681074d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:08.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.370+0000 7f126f6dd700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1268107a80 con 0x7f12680f6860 2024-02-15T18:41:08.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.370+0000 7f126f6dd700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1268107bc0 con 0x7f12680f55c0 2024-02-15T18:41:08.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.370+0000 7f126cc78700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12680f6860 0x7f12681074d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:08.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.370+0000 7f126cc78700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12680f6860 0x7f12681074d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39554/0 (socket says 172.21.15.38:39554) 2024-02-15T18:41:08.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.370+0000 7f126cc78700 1 -- 172.21.15.38:0/2083641343 learned_addr learned my addr 172.21.15.38:0/2083641343 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:41:08.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.370+0000 7f126d479700 1 --2- 172.21.15.38:0/2083641343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12680f55c0 0x7f1268106fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:08.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.370+0000 7f126cc78700 1 -- 172.21.15.38:0/2083641343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12680f55c0 msgr2=0x7f1268106fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:08.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.370+0000 7f126cc78700 1 --2- 172.21.15.38:0/2083641343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12680f55c0 0x7f1268106fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.370+0000 7f126cc78700 1 -- 172.21.15.38:0/2083641343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1258005040 con 0x7f12680f6860 2024-02-15T18:41:08.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.371+0000 7f126cc78700 1 --2- 172.21.15.38:0/2083641343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12680f6860 0x7f12681074d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f12580043a0 tx=0x7f125800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:08.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.371+0000 7f125dffb700 1 -- 172.21.15.38:0/2083641343 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1258019070 con 0x7f12680f6860 2024-02-15T18:41:08.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.371+0000 7f125dffb700 1 -- 172.21.15.38:0/2083641343 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f1258014420 con 0x7f12680f6860 2024-02-15T18:41:08.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.371+0000 7f126f6dd700 1 -- 172.21.15.38:0/2083641343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1268108500 con 0x7f12680f6860 2024-02-15T18:41:08.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.371+0000 7f125dffb700 1 -- 172.21.15.38:0/2083641343 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f125801d600 con 0x7f12680f6860 2024-02-15T18:41:08.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.371+0000 7f126f6dd700 1 -- 172.21.15.38:0/2083641343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1268108a60 con 0x7f12680f6860 2024-02-15T18:41:08.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.372+0000 7f125dffb700 1 -- 172.21.15.38:0/2083641343 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f1258025070 con 0x7f12680f6860 2024-02-15T18:41:08.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.373+0000 7f125dffb700 1 --2- 172.21.15.38:0/2083641343 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f1254060f20 0x7f12540633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:08.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.373+0000 7f125dffb700 1 -- 172.21.15.38:0/2083641343 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f125807d160 con 0x7f12680f6860 2024-02-15T18:41:08.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.373+0000 7f126d479700 1 --2- 172.21.15.38:0/2083641343 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f1254060f20 0x7f12540633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:08.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.373+0000 7f126f6dd700 1 -- 172.21.15.38:0/2083641343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1268045ad0 con 0x7f12680f6860 2024-02-15T18:41:08.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.373+0000 7f126d479700 1 --2- 172.21.15.38:0/2083641343 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f1254060f20 0x7f12540633e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f1264009fd0 tx=0x7f126400b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:08.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.382+0000 7f125dffb700 1 -- 172.21.15.38:0/2083641343 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1258023080 con 0x7f12680f6860 2024-02-15T18:41:08.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.574+0000 7f126f6dd700 1 -- 172.21.15.38:0/2083641343 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f12680fe000 con 0x7f1254060f20 2024-02-15T18:41:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.576+0000 7f125dffb700 1 -- 172.21.15.38:0/2083641343 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f12680fe000 con 0x7f1254060f20 2024-02-15T18:41:08.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.578+0000 7f126f6dd700 1 -- 172.21.15.38:0/2083641343 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f1254060f20 msgr2=0x7f12540633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:08.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.578+0000 7f126f6dd700 1 --2- 172.21.15.38:0/2083641343 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f1254060f20 0x7f12540633e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f1264009fd0 tx=0x7f126400b040).stop 2024-02-15T18:41:08.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.578+0000 7f126f6dd700 1 -- 172.21.15.38:0/2083641343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12680f6860 msgr2=0x7f12681074d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:08.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.578+0000 7f126f6dd700 1 --2- 172.21.15.38:0/2083641343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12680f6860 0x7f12681074d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f12580043a0 tx=0x7f125800b040).stop 2024-02-15T18:41:08.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.578+0000 7f126f6dd700 1 -- 172.21.15.38:0/2083641343 shutdown_connections 2024-02-15T18:41:08.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.578+0000 7f126f6dd700 1 --2- 172.21.15.38:0/2083641343 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f1254060f20 0x7f12540633e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.578+0000 7f126f6dd700 1 --2- 172.21.15.38:0/2083641343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12680f55c0 0x7f1268106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.578+0000 7f126f6dd700 1 --2- 172.21.15.38:0/2083641343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12680f6860 0x7f12681074d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.578+0000 7f126f6dd700 1 -- 172.21.15.38:0/2083641343 >> 172.21.15.38:0/2083641343 conn(0x7f12680f0d40 msgr2=0x7f126805fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:08.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.578+0000 7f126f6dd700 1 -- 172.21.15.38:0/2083641343 shutdown_connections 2024-02-15T18:41:08.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.578+0000 7f126f6dd700 1 -- 172.21.15.38:0/2083641343 wait complete. 2024-02-15T18:41:08.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.682+0000 7fafc61b2700 1 -- 172.21.15.38:0/2213095679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafc00f9690 msgr2=0x7fafc00fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:08.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.682+0000 7fafc61b2700 1 --2- 172.21.15.38:0/2213095679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafc00f9690 0x7fafc00fbae0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fafb0004660 tx=0x7fafb0009b30).stop 2024-02-15T18:41:08.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.682+0000 7fafc61b2700 1 -- 172.21.15.38:0/2213095679 shutdown_connections 2024-02-15T18:41:08.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.682+0000 7fafc61b2700 1 --2- 172.21.15.38:0/2213095679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafc00f9690 0x7fafc00fbae0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.682+0000 7fafc61b2700 1 --2- 172.21.15.38:0/2213095679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafc0060310 0x7fafc00f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.682+0000 7fafc61b2700 1 -- 172.21.15.38:0/2213095679 >> 172.21.15.38:0/2213095679 conn(0x7fafc00f0b90 msgr2=0x7fafc00f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:08.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.682+0000 7fafc61b2700 1 -- 172.21.15.38:0/2213095679 shutdown_connections 2024-02-15T18:41:08.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.682+0000 7fafc61b2700 1 -- 172.21.15.38:0/2213095679 wait complete. 2024-02-15T18:41:08.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.683+0000 7fafc61b2700 1 Processor -- start 2024-02-15T18:41:08.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.683+0000 7fafc61b2700 1 -- start start 2024-02-15T18:41:08.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.683+0000 7fafc61b2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafc0060310 0x7fafc0106e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:08.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.683+0000 7fafc61b2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafc00f9690 0x7fafc0107300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:08.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.683+0000 7fafc61b2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc0107900 con 0x7fafc0060310 2024-02-15T18:41:08.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.683+0000 7fafc61b2700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc0108110 con 0x7fafc00f9690 2024-02-15T18:41:08.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.683+0000 7fafc51b0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafc0060310 0x7fafc0106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:08.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.683+0000 7fafc51b0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafc0060310 0x7fafc0106e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39580/0 (socket says 172.21.15.38:39580) 2024-02-15T18:41:08.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.683+0000 7fafc51b0700 1 -- 172.21.15.38:0/2245406544 learned_addr learned my addr 172.21.15.38:0/2245406544 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:41:08.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.683+0000 7fafc49af700 1 --2- 172.21.15.38:0/2245406544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafc00f9690 0x7fafc0107300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:08.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.684+0000 7fafc51b0700 1 -- 172.21.15.38:0/2245406544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafc00f9690 msgr2=0x7fafc0107300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:08.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.684+0000 7fafc51b0700 1 --2- 172.21.15.38:0/2245406544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafc00f9690 0x7fafc0107300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.684+0000 7fafc51b0700 1 -- 172.21.15.38:0/2245406544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb0005040 con 0x7fafc0060310 2024-02-15T18:41:08.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.684+0000 7fafc51b0700 1 --2- 172.21.15.38:0/2245406544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafc0060310 0x7fafc0106e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fafbc00c790 tx=0x7fafbc007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:08.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.684+0000 7fafb5ffb700 1 -- 172.21.15.38:0/2245406544 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafbc002590 con 0x7fafc0060310 2024-02-15T18:41:08.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.684+0000 7fafb5ffb700 1 -- 172.21.15.38:0/2245406544 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fafbc002be0 con 0x7fafc0060310 2024-02-15T18:41:08.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.684+0000 7fafc61b2700 1 -- 172.21.15.38:0/2245406544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc0108390 con 0x7fafc0060310 2024-02-15T18:41:08.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.684+0000 7fafb5ffb700 1 -- 172.21.15.38:0/2245406544 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafbc0123b0 con 0x7fafc0060310 2024-02-15T18:41:08.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.684+0000 7fafc61b2700 1 -- 172.21.15.38:0/2245406544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc0108950 con 0x7fafc0060310 2024-02-15T18:41:08.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.685+0000 7fafc61b2700 1 -- 172.21.15.38:0/2245406544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc0059100 con 0x7fafc0060310 2024-02-15T18:41:08.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.685+0000 7fafb5ffb700 1 -- 172.21.15.38:0/2245406544 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fafbc01a060 con 0x7fafc0060310 2024-02-15T18:41:08.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.686+0000 7fafb5ffb700 1 --2- 172.21.15.38:0/2245406544 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fafac060f70 0x7fafac063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:08.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.686+0000 7fafb5ffb700 1 -- 172.21.15.38:0/2245406544 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fafbc07c3d0 con 0x7fafc0060310 2024-02-15T18:41:08.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.686+0000 7fafc49af700 1 --2- 172.21.15.38:0/2245406544 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fafac060f70 0x7fafac063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:08.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.686+0000 7fafc49af700 1 --2- 172.21.15.38:0/2245406544 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fafac060f70 0x7fafac063430 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fafb0009f50 tx=0x7fafb0009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:08.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.691+0000 7fafb5ffb700 1 -- 172.21.15.38:0/2245406544 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fafbc04c7a0 con 0x7fafc0060310 2024-02-15T18:41:08.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.874+0000 7fafc61b2700 1 -- 172.21.15.38:0/2245406544 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fafc00fee70 con 0x7fafac060f70 2024-02-15T18:41:08.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.885+0000 7fafb5ffb700 1 -- 172.21.15.38:0/2245406544 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fafc00fee70 con 0x7fafac060f70 2024-02-15T18:41:08.874 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:41:08.874 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (12m) 8s ago 13m 20.9M - 0.20.0 0881eb8f169f c2034c318e7c 2024-02-15T18:41:08.874 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (4m) 8s ago 13m 6635k - 16.2.14-521-g0e714d9a 1fbc92c859dc c2fee0824d4b 2024-02-15T18:41:08.875 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi121 smithi121 running (4m) 67s ago 12m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 05c1a5501a35 2024-02-15T18:41:08.875 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (11m) 8s ago 13m 35.6M - 6.7.4 557c83e11646 2dcccfbfbd2a 2024-02-15T18:41:08.875 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.hxoggl smithi038 running (29s) 8s ago 8m 12.3M - 16.2.14-521-g0e714d9a 1fbc92c859dc 42b83e98219f 2024-02-15T18:41:08.875 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.qgkzgn smithi038 running (14s) 8s ago 8m 11.4M - 16.2.14-521-g0e714d9a 1fbc92c859dc ecbb70ac6d7d 2024-02-15T18:41:08.875 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.mvumlh smithi121 running (8m) 67s ago 8m 116M - 16.2.4 8d91d370c2b8 88b7d876a39b 2024-02-15T18:41:08.875 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.ovuzki smithi121 running (8m) 67s ago 8m 124M - 16.2.4 8d91d370c2b8 21cacf0012f9 2024-02-15T18:41:08.875 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.khesbl smithi038 *:8443,9283 running (5m) 8s ago 15m 438M - 16.2.14-521-g0e714d9a 1fbc92c859dc e642f3c2cfc9 2024-02-15T18:41:08.875 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi121.ckiryi smithi121 *:8443,9283 running (4m) 67s ago 12m 387M - 16.2.14-521-g0e714d9a 1fbc92c859dc 82515b025594 2024-02-15T18:41:08.875 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (4m) 8s ago 15m 53.0M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc d3e8317a30ca 2024-02-15T18:41:08.875 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi121 smithi121 running (4m) 67s ago 12m 43.1M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 42e1dda1a39d 2024-02-15T18:41:08.875 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (13m) 8s ago 13m 17.1M - 0.18.1 e5a616e4b9cf 037f60e38158 2024-02-15T18:41:08.875 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi121 smithi121 *:9100 running (12m) 67s ago 12m 17.2M - 0.18.1 e5a616e4b9cf a867bff85e18 2024-02-15T18:41:08.875 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (3m) 8s ago 12m 192M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 623ac7ecbb0a 2024-02-15T18:41:08.875 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (3m) 8s ago 11m 215M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 0f9e8e1f255a 2024-02-15T18:41:08.875 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (2m) 8s ago 11m 207M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 37171264b5be 2024-02-15T18:41:08.876 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi121 running (2m) 67s ago 10m 255M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc d505a58b375c 2024-02-15T18:41:08.876 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi121 running (99s) 67s ago 10m 259M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 0ce6b6abb99e 2024-02-15T18:41:08.876 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi121 running (71s) 67s ago 10m 13.1M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 79bb22243769 2024-02-15T18:41:08.876 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (5m) 8s ago 13m 66.2M - 2.18.1 de242295e225 67e33f9de4e6 2024-02-15T18:41:08.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.886+0000 7fafc61b2700 1 -- 172.21.15.38:0/2245406544 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fafac060f70 msgr2=0x7fafac063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:08.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.886+0000 7fafc61b2700 1 --2- 172.21.15.38:0/2245406544 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fafac060f70 0x7fafac063430 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fafb0009f50 tx=0x7fafb0009b30).stop 2024-02-15T18:41:08.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.886+0000 7fafc61b2700 1 -- 172.21.15.38:0/2245406544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafc0060310 msgr2=0x7fafc0106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:08.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.886+0000 7fafc61b2700 1 --2- 172.21.15.38:0/2245406544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafc0060310 0x7fafc0106e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fafbc00c790 tx=0x7fafbc007d70).stop 2024-02-15T18:41:08.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.887+0000 7fafc61b2700 1 -- 172.21.15.38:0/2245406544 shutdown_connections 2024-02-15T18:41:08.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.887+0000 7fafc61b2700 1 --2- 172.21.15.38:0/2245406544 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fafac060f70 0x7fafac063430 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.887+0000 7fafc61b2700 1 --2- 172.21.15.38:0/2245406544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafc0060310 0x7fafc0106e10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.887+0000 7fafc61b2700 1 --2- 172.21.15.38:0/2245406544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafc00f9690 0x7fafc0107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.887+0000 7fafc61b2700 1 -- 172.21.15.38:0/2245406544 >> 172.21.15.38:0/2245406544 conn(0x7fafc00f0b90 msgr2=0x7fafc00f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:08.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.887+0000 7fafc61b2700 1 -- 172.21.15.38:0/2245406544 shutdown_connections 2024-02-15T18:41:08.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.887+0000 7fafc61b2700 1 -- 172.21.15.38:0/2245406544 wait complete. 2024-02-15T18:41:08.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.989+0000 7fcfbaf15700 1 -- 172.21.15.38:0/1747874505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfb40602d0 msgr2=0x7fcfb40fb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:08.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.989+0000 7fcfbaf15700 1 --2- 172.21.15.38:0/1747874505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfb40602d0 0x7fcfb40fb6e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fcfa8004660 tx=0x7fcfa8009b30).stop 2024-02-15T18:41:08.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.989+0000 7fcfbaf15700 1 -- 172.21.15.38:0/1747874505 shutdown_connections 2024-02-15T18:41:08.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.989+0000 7fcfbaf15700 1 --2- 172.21.15.38:0/1747874505 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfb40fbbd0 0x7fcfb40fdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.989+0000 7fcfbaf15700 1 --2- 172.21.15.38:0/1747874505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfb40602d0 0x7fcfb40fb6e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.989+0000 7fcfbaf15700 1 -- 172.21.15.38:0/1747874505 >> 172.21.15.38:0/1747874505 conn(0x7fcfb40f0d40 msgr2=0x7fcfb40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:08.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.989+0000 7fcfbaf15700 1 -- 172.21.15.38:0/1747874505 shutdown_connections 2024-02-15T18:41:08.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.989+0000 7fcfbaf15700 1 -- 172.21.15.38:0/1747874505 wait complete. 2024-02-15T18:41:08.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.990+0000 7fcfbaf15700 1 Processor -- start 2024-02-15T18:41:08.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.990+0000 7fcfbaf15700 1 -- start start 2024-02-15T18:41:08.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.990+0000 7fcfbaf15700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfb40602d0 0x7fcfb41091d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:08.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.990+0000 7fcfbaf15700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfb40fbbd0 0x7fcfb41096c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:08.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.990+0000 7fcfbaf15700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb4109c70 con 0x7fcfb40fbbd0 2024-02-15T18:41:08.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.990+0000 7fcfbaf15700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb4109db0 con 0x7fcfb40602d0 2024-02-15T18:41:08.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.990+0000 7fcfb3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfb40fbbd0 0x7fcfb41096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:08.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.990+0000 7fcfb3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfb40fbbd0 0x7fcfb41096c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39594/0 (socket says 172.21.15.38:39594) 2024-02-15T18:41:08.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.990+0000 7fcfb3fff700 1 -- 172.21.15.38:0/2615145380 learned_addr learned my addr 172.21.15.38:0/2615145380 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:41:08.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.991+0000 7fcfb8cb1700 1 --2- 172.21.15.38:0/2615145380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfb40602d0 0x7fcfb41091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:08.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.991+0000 7fcfb3fff700 1 -- 172.21.15.38:0/2615145380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfb40602d0 msgr2=0x7fcfb41091d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:08.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.991+0000 7fcfb3fff700 1 --2- 172.21.15.38:0/2615145380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfb40602d0 0x7fcfb41091d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:08.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.991+0000 7fcfb3fff700 1 -- 172.21.15.38:0/2615145380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa8005040 con 0x7fcfb40fbbd0 2024-02-15T18:41:08.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.991+0000 7fcfb8cb1700 1 --2- 172.21.15.38:0/2615145380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfb40602d0 0x7fcfb41091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:41:08.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.991+0000 7fcfb3fff700 1 --2- 172.21.15.38:0/2615145380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfb40fbbd0 0x7fcfb41096c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fcfa000c790 tx=0x7fcfa0007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:08.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.991+0000 7fcfb17fa700 1 -- 172.21.15.38:0/2615145380 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfa000e680 con 0x7fcfb40fbbd0 2024-02-15T18:41:08.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.991+0000 7fcfb17fa700 1 -- 172.21.15.38:0/2615145380 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fcfa000ecd0 con 0x7fcfb40fbbd0 2024-02-15T18:41:08.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.991+0000 7fcfbaf15700 1 -- 172.21.15.38:0/2615145380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb410a750 con 0x7fcfb40fbbd0 2024-02-15T18:41:08.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.991+0000 7fcfb17fa700 1 -- 172.21.15.38:0/2615145380 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfa0013e10 con 0x7fcfb40fbbd0 2024-02-15T18:41:08.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.991+0000 7fcfbaf15700 1 -- 172.21.15.38:0/2615145380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb410ad10 con 0x7fcfb40fbbd0 2024-02-15T18:41:08.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.992+0000 7fcfbaf15700 1 -- 172.21.15.38:0/2615145380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb4045ad0 con 0x7fcfb40fbbd0 2024-02-15T18:41:08.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.993+0000 7fcfb17fa700 1 -- 172.21.15.38:0/2615145380 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fcfa001a060 con 0x7fcfb40fbbd0 2024-02-15T18:41:08.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.993+0000 7fcfb17fa700 1 --2- 172.21.15.38:0/2615145380 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcfa4060f20 0x7fcfa40633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:08.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.993+0000 7fcfb17fa700 1 -- 172.21.15.38:0/2615145380 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fcfa007c3c0 con 0x7fcfb40fbbd0 2024-02-15T18:41:08.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.993+0000 7fcfb8cb1700 1 --2- 172.21.15.38:0/2615145380 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcfa4060f20 0x7fcfa40633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:08.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:08.993+0000 7fcfb8cb1700 1 --2- 172.21.15.38:0/2615145380 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcfa4060f20 0x7fcfa40633e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fcfa8004630 tx=0x7fcfa8009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:08.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.002+0000 7fcfb17fa700 1 -- 172.21.15.38:0/2615145380 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fcfa0050220 con 0x7fcfb40fbbd0 2024-02-15T18:41:09.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.253+0000 7fcfbaf15700 1 -- 172.21.15.38:0/2615145380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcfb4101250 con 0x7fcfb40fbbd0 2024-02-15T18:41:09.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.254+0000 7fcfb17fa700 1 -- 172.21.15.38:0/2615145380 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7fcfa004fe10 con 0x7fcfb40fbbd0 2024-02-15T18:41:09.242 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:41:09.242 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-02-15T18:41:09.243 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:41:09.243 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:41:09.243 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-02-15T18:41:09.243 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:41:09.243 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:41:09.243 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-02-15T18:41:09.243 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 6 2024-02-15T18:41:09.243 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:41:09.243 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-02-15T18:41:09.243 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2, 2024-02-15T18:41:09.243 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 1 2024-02-15T18:41:09.243 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:41:09.243 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-02-15T18:41:09.243 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 12, 2024-02-15T18:41:09.244 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 1 2024-02-15T18:41:09.244 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-02-15T18:41:09.244 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:41:09.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.256+0000 7fcfbaf15700 1 -- 172.21.15.38:0/2615145380 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcfa4060f20 msgr2=0x7fcfa40633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:09.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.256+0000 7fcfbaf15700 1 --2- 172.21.15.38:0/2615145380 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcfa4060f20 0x7fcfa40633e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fcfa8004630 tx=0x7fcfa8009b30).stop 2024-02-15T18:41:09.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.256+0000 7fcfbaf15700 1 -- 172.21.15.38:0/2615145380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfb40fbbd0 msgr2=0x7fcfb41096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:09.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.256+0000 7fcfbaf15700 1 --2- 172.21.15.38:0/2615145380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfb40fbbd0 0x7fcfb41096c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fcfa000c790 tx=0x7fcfa0007d40).stop 2024-02-15T18:41:09.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.256+0000 7fcfbaf15700 1 -- 172.21.15.38:0/2615145380 shutdown_connections 2024-02-15T18:41:09.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.256+0000 7fcfbaf15700 1 --2- 172.21.15.38:0/2615145380 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcfa4060f20 0x7fcfa40633e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:09.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.256+0000 7fcfbaf15700 1 --2- 172.21.15.38:0/2615145380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfb40602d0 0x7fcfb41091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:09.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.256+0000 7fcfbaf15700 1 --2- 172.21.15.38:0/2615145380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfb40fbbd0 0x7fcfb41096c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:09.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.256+0000 7fcfbaf15700 1 -- 172.21.15.38:0/2615145380 >> 172.21.15.38:0/2615145380 conn(0x7fcfb40f0d40 msgr2=0x7fcfb40f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:09.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.257+0000 7fcfbaf15700 1 -- 172.21.15.38:0/2615145380 shutdown_connections 2024-02-15T18:41:09.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.257+0000 7fcfbaf15700 1 -- 172.21.15.38:0/2615145380 wait complete. 2024-02-15T18:41:09.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.364+0000 7f85c51b8700 1 -- 172.21.15.38:0/3526883988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85c00f7d40 msgr2=0x7f85c00f8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.364+0000 7f85c51b8700 1 --2- 172.21.15.38:0/3526883988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85c00f7d40 0x7f85c00f8160 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f85b0004660 tx=0x7f85b0009b30).stop 2024-02-15T18:41:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.364+0000 7f85c51b8700 1 -- 172.21.15.38:0/3526883988 shutdown_connections 2024-02-15T18:41:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.364+0000 7f85c51b8700 1 --2- 172.21.15.38:0/3526883988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85c00f86e0 0x7f85c00f4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.364+0000 7f85c51b8700 1 --2- 172.21.15.38:0/3526883988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85c00f7d40 0x7f85c00f8160 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.364+0000 7f85c51b8700 1 -- 172.21.15.38:0/3526883988 >> 172.21.15.38:0/3526883988 conn(0x7f85c00f04c0 msgr2=0x7f85c00f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.364+0000 7f85c51b8700 1 -- 172.21.15.38:0/3526883988 shutdown_connections 2024-02-15T18:41:09.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.364+0000 7f85c51b8700 1 -- 172.21.15.38:0/3526883988 wait complete. 2024-02-15T18:41:09.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.365+0000 7f85c51b8700 1 Processor -- start 2024-02-15T18:41:09.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.365+0000 7f85c51b8700 1 -- start start 2024-02-15T18:41:09.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.365+0000 7f85c51b8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85c00f7d40 0x7f85c0107030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:09.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.365+0000 7f85c51b8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85c00f86e0 0x7f85c0107520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:09.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.365+0000 7f85c51b8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c0107ad0 con 0x7f85c00f7d40 2024-02-15T18:41:09.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.365+0000 7f85c51b8700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c0107c10 con 0x7f85c00f86e0 2024-02-15T18:41:09.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.365+0000 7f85bed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85c00f7d40 0x7f85c0107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:09.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.365+0000 7f85bed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85c00f7d40 0x7f85c0107030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39602/0 (socket says 172.21.15.38:39602) 2024-02-15T18:41:09.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.365+0000 7f85bed9d700 1 -- 172.21.15.38:0/3882664285 learned_addr learned my addr 172.21.15.38:0/3882664285 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:41:09.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.365+0000 7f85be59c700 1 --2- 172.21.15.38:0/3882664285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85c00f86e0 0x7f85c0107520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:09.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.366+0000 7f85bed9d700 1 -- 172.21.15.38:0/3882664285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85c00f86e0 msgr2=0x7f85c0107520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:09.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.366+0000 7f85bed9d700 1 --2- 172.21.15.38:0/3882664285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85c00f86e0 0x7f85c0107520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:09.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.366+0000 7f85bed9d700 1 -- 172.21.15.38:0/3882664285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b0005040 con 0x7f85c00f7d40 2024-02-15T18:41:09.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.366+0000 7f85bed9d700 1 --2- 172.21.15.38:0/3882664285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85c00f7d40 0x7f85c0107030 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f85b0002040 tx=0x7f85b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:09.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.366+0000 7f85a77fe700 1 -- 172.21.15.38:0/3882664285 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85b0019070 con 0x7f85c00f7d40 2024-02-15T18:41:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.366+0000 7f85a77fe700 1 -- 172.21.15.38:0/3882664285 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f85b0007e80 con 0x7f85c00f7d40 2024-02-15T18:41:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.366+0000 7f85a77fe700 1 -- 172.21.15.38:0/3882664285 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85b0014db0 con 0x7f85c00f7d40 2024-02-15T18:41:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.366+0000 7f85c51b8700 1 -- 172.21.15.38:0/3882664285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c0108550 con 0x7f85c00f7d40 2024-02-15T18:41:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.366+0000 7f85c51b8700 1 -- 172.21.15.38:0/3882664285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c0108ab0 con 0x7f85c00f7d40 2024-02-15T18:41:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.367+0000 7f85a77fe700 1 -- 172.21.15.38:0/3882664285 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f85b0025070 con 0x7f85c00f7d40 2024-02-15T18:41:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.367+0000 7f85c51b8700 1 -- 172.21.15.38:0/3882664285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c0045ad0 con 0x7f85c00f7d40 2024-02-15T18:41:09.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:08.255256+0000 mgr.smithi038.khesbl (mgr.14664) 269 : audit [DBG] from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:41:09.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:08.577510+0000 mgr.smithi038.khesbl (mgr.14664) 270 : audit [DBG] from='client.34326 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:41:09.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:08.877596+0000 mgr.smithi038.khesbl (mgr.14664) 271 : audit [DBG] from='client.34330 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:41:09.359 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:08.997018+0000 mgr.smithi038.khesbl (mgr.14664) 272 : cluster [DBG] pgmap v190: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 9.6 MiB/s rd, 3 op/s 2024-02-15T18:41:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.368+0000 7f85a77fe700 1 --2- 172.21.15.38:0/3882664285 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f85a8060f20 0x7f85a80633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.368+0000 7f85a77fe700 1 -- 172.21.15.38:0/3882664285 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f85b007cef0 con 0x7f85c00f7d40 2024-02-15T18:41:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.368+0000 7f85be59c700 1 --2- 172.21.15.38:0/3882664285 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f85a8060f20 0x7f85a80633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.369+0000 7f85be59c700 1 --2- 172.21.15.38:0/3882664285 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f85a8060f20 0x7f85a80633e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f85b4009fd0 tx=0x7f85b400b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:09.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.372+0000 7f85a77fe700 1 -- 172.21.15.38:0/3882664285 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f85b0050dd0 con 0x7f85c00f7d40 2024-02-15T18:41:09.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:08.255256+0000 mgr.smithi038.khesbl (mgr.14664) 269 : audit [DBG] from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:41:09.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:08.577510+0000 mgr.smithi038.khesbl (mgr.14664) 270 : audit [DBG] from='client.34326 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:41:09.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:08.877596+0000 mgr.smithi038.khesbl (mgr.14664) 271 : audit [DBG] from='client.34330 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:41:09.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:08.997018+0000 mgr.smithi038.khesbl (mgr.14664) 272 : cluster [DBG] pgmap v190: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 9.6 MiB/s rd, 3 op/s 2024-02-15T18:41:09.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.612+0000 7f85c51b8700 1 -- 172.21.15.38:0/3882664285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f85c00ff140 con 0x7f85c00f7d40 2024-02-15T18:41:09.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.613+0000 7f85a77fe700 1 -- 172.21.15.38:0/3882664285 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1591 (secure 0 0 0) 0x7f85b0023070 con 0x7f85c00f7d40 2024-02-15T18:41:09.602 INFO:teuthology.orchestra.run.smithi038.stdout:e24 2024-02-15T18:41:09.602 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:41:09.602 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:41:09.602 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-02-15T18:41:09.602 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:41:09.602 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:41:09.602 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-02-15T18:41:09.602 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 24 2024-02-15T18:41:09.602 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 2024-02-15T18:41:09.603 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:41:09.603 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-02-15T18:41:03.954888+0000 2024-02-15T18:41:09.603 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-02-15T18:41:09.603 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-02-15T18:41:09.603 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-02-15T18:41:09.603 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-02-15T18:41:09.603 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-02-15T18:41:09.603 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-02-15T18:41:09.603 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-02-15T18:41:09.603 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 79 2024-02-15T18:41:09.603 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:41:09.604 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 1 2024-02-15T18:41:09.604 INFO:teuthology.orchestra.run.smithi038.stdout:in 0 2024-02-15T18:41:09.604 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=34314} 2024-02-15T18:41:09.604 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-02-15T18:41:09.604 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-02-15T18:41:09.604 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 1 2024-02-15T18:41:09.604 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-02-15T18:41:09.604 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-02-15T18:41:09.604 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-02-15T18:41:09.604 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-02-15T18:41:09.604 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-02-15T18:41:09.604 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.hxoggl{0:34314} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.38:6826/4077527450,v1:172.21.15.38:6827/4077527450] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:41:09.605 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:41:09.605 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:41:09.605 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-02-15T18:41:09.605 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:41:09.605 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.qgkzgn{-1:34318} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.38:6828/1616551408,v1:172.21.15.38:6829/1616551408] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:41:09.605 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.ovuzki{-1:44223} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.121:6826/554561289,v1:172.21.15.121:6827/554561289] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:41:09.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.614+0000 7f85c51b8700 1 -- 172.21.15.38:0/3882664285 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f85a8060f20 msgr2=0x7f85a80633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:09.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.614+0000 7f85c51b8700 1 --2- 172.21.15.38:0/3882664285 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f85a8060f20 0x7f85a80633e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f85b4009fd0 tx=0x7f85b400b040).stop 2024-02-15T18:41:09.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.614+0000 7f85c51b8700 1 -- 172.21.15.38:0/3882664285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85c00f7d40 msgr2=0x7f85c0107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:09.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.614+0000 7f85c51b8700 1 --2- 172.21.15.38:0/3882664285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85c00f7d40 0x7f85c0107030 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f85b0002040 tx=0x7f85b000b040).stop 2024-02-15T18:41:09.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.615+0000 7f85c51b8700 1 -- 172.21.15.38:0/3882664285 shutdown_connections 2024-02-15T18:41:09.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.615+0000 7f85c51b8700 1 --2- 172.21.15.38:0/3882664285 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f85a8060f20 0x7f85a80633e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:09.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.615+0000 7f85c51b8700 1 --2- 172.21.15.38:0/3882664285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85c00f7d40 0x7f85c0107030 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:09.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.615+0000 7f85c51b8700 1 --2- 172.21.15.38:0/3882664285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85c00f86e0 0x7f85c0107520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:09.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.615+0000 7f85c51b8700 1 -- 172.21.15.38:0/3882664285 >> 172.21.15.38:0/3882664285 conn(0x7f85c00f04c0 msgr2=0x7f85c00fb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:09.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.615+0000 7f85c51b8700 1 -- 172.21.15.38:0/3882664285 shutdown_connections 2024-02-15T18:41:09.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.615+0000 7f85c51b8700 1 -- 172.21.15.38:0/3882664285 wait complete. 2024-02-15T18:41:09.607 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 24 2024-02-15T18:41:09.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.718+0000 7fb77d017700 1 -- 172.21.15.38:0/1872828516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7780f77d0 msgr2=0x7fb7780f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:09.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.718+0000 7fb77d017700 1 --2- 172.21.15.38:0/1872828516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7780f77d0 0x7fb7780f7c30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fb76c004660 tx=0x7fb76c009b30).stop 2024-02-15T18:41:09.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.718+0000 7fb77d017700 1 -- 172.21.15.38:0/1872828516 shutdown_connections 2024-02-15T18:41:09.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.718+0000 7fb77d017700 1 --2- 172.21.15.38:0/1872828516 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7780f77d0 0x7fb7780f7c30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:09.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.718+0000 7fb77d017700 1 --2- 172.21.15.38:0/1872828516 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7780f6530 0x7fb7780f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:09.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.719+0000 7fb77d017700 1 -- 172.21.15.38:0/1872828516 >> 172.21.15.38:0/1872828516 conn(0x7fb7780f1c90 msgr2=0x7fb7780f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:09.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.719+0000 7fb77d017700 1 -- 172.21.15.38:0/1872828516 shutdown_connections 2024-02-15T18:41:09.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.719+0000 7fb77d017700 1 -- 172.21.15.38:0/1872828516 wait complete. 2024-02-15T18:41:09.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.719+0000 7fb77d017700 1 Processor -- start 2024-02-15T18:41:09.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.720+0000 7fb77d017700 1 -- start start 2024-02-15T18:41:09.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.720+0000 7fb77d017700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7780f6530 0x7fb77806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:09.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.720+0000 7fb77d017700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7780f77d0 0x7fb778069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:09.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.720+0000 7fb77d017700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77806a380 con 0x7fb7780f6530 2024-02-15T18:41:09.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.720+0000 7fb77d017700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77806a4c0 con 0x7fb7780f77d0 2024-02-15T18:41:09.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.720+0000 7fb777fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7780f6530 0x7fb77806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:09.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.720+0000 7fb777fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7780f6530 0x7fb77806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39616/0 (socket says 172.21.15.38:39616) 2024-02-15T18:41:09.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.720+0000 7fb777fff700 1 -- 172.21.15.38:0/4255045522 learned_addr learned my addr 172.21.15.38:0/4255045522 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:41:09.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.720+0000 7fb7777fe700 1 --2- 172.21.15.38:0/4255045522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7780f77d0 0x7fb778069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:09.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.720+0000 7fb777fff700 1 -- 172.21.15.38:0/4255045522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7780f77d0 msgr2=0x7fb778069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:09.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.720+0000 7fb777fff700 1 --2- 172.21.15.38:0/4255045522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7780f77d0 0x7fb778069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:09.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.720+0000 7fb777fff700 1 -- 172.21.15.38:0/4255045522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76c005040 con 0x7fb7780f6530 2024-02-15T18:41:09.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.720+0000 7fb777fff700 1 --2- 172.21.15.38:0/4255045522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7780f6530 0x7fb77806b9a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fb768002700 tx=0x7fb768009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:09.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.721+0000 7fb774ff9700 1 -- 172.21.15.38:0/4255045522 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb768009320 con 0x7fb7780f6530 2024-02-15T18:41:09.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.721+0000 7fb774ff9700 1 -- 172.21.15.38:0/4255045522 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fb76800f040 con 0x7fb7780f6530 2024-02-15T18:41:09.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.721+0000 7fb774ff9700 1 -- 172.21.15.38:0/4255045522 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7680149b0 con 0x7fb7780f6530 2024-02-15T18:41:09.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.721+0000 7fb77d017700 1 -- 172.21.15.38:0/4255045522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb77806a7a0 con 0x7fb7780f6530 2024-02-15T18:41:09.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.721+0000 7fb77d017700 1 -- 172.21.15.38:0/4255045522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb77810ede0 con 0x7fb7780f6530 2024-02-15T18:41:09.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.722+0000 7fb77d017700 1 -- 172.21.15.38:0/4255045522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb778045ad0 con 0x7fb7780f6530 2024-02-15T18:41:09.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.723+0000 7fb774ff9700 1 -- 172.21.15.38:0/4255045522 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fb768009480 con 0x7fb7780f6530 2024-02-15T18:41:09.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.723+0000 7fb774ff9700 1 --2- 172.21.15.38:0/4255045522 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fb760060f70 0x7fb760063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:09.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.723+0000 7fb774ff9700 1 -- 172.21.15.38:0/4255045522 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fb76807be90 con 0x7fb7780f6530 2024-02-15T18:41:09.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.723+0000 7fb7777fe700 1 --2- 172.21.15.38:0/4255045522 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fb760060f70 0x7fb760063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:09.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.724+0000 7fb7777fe700 1 --2- 172.21.15.38:0/4255045522 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fb760060f70 0x7fb760063430 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fb76c004e30 tx=0x7fb76c009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:09.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.727+0000 7fb774ff9700 1 -- 172.21.15.38:0/4255045522 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb768081060 con 0x7fb7780f6530 2024-02-15T18:41:09.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.919+0000 7fb77d017700 1 -- 172.21.15.38:0/4255045522 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb778057f10 con 0x7fb760060f70 2024-02-15T18:41:09.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.921+0000 7fb774ff9700 1 -- 172.21.15.38:0/4255045522 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb778057f10 con 0x7fb760060f70 2024-02-15T18:41:09.910 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:41:09.910 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:41:09.910 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-02-15T18:41:09.910 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:41:09.910 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-02-15T18:41:09.911 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-02-15T18:41:09.911 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-02-15T18:41:09.911 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr", 2024-02-15T18:41:09.911 INFO:teuthology.orchestra.run.smithi038.stdout: "osd" 2024-02-15T18:41:09.911 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-02-15T18:41:09.911 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "14/21 daemons upgraded", 2024-02-15T18:41:09.911 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading mds daemons", 2024-02-15T18:41:09.911 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-02-15T18:41:09.911 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:41:09.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.923+0000 7fb77d017700 1 -- 172.21.15.38:0/4255045522 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fb760060f70 msgr2=0x7fb760063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:09.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.923+0000 7fb77d017700 1 --2- 172.21.15.38:0/4255045522 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fb760060f70 0x7fb760063430 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fb76c004e30 tx=0x7fb76c009b30).stop 2024-02-15T18:41:09.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.923+0000 7fb77d017700 1 -- 172.21.15.38:0/4255045522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7780f6530 msgr2=0x7fb77806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:09.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.923+0000 7fb77d017700 1 --2- 172.21.15.38:0/4255045522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7780f6530 0x7fb77806b9a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fb768002700 tx=0x7fb768009e00).stop 2024-02-15T18:41:09.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.923+0000 7fb77d017700 1 -- 172.21.15.38:0/4255045522 shutdown_connections 2024-02-15T18:41:09.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.923+0000 7fb77d017700 1 --2- 172.21.15.38:0/4255045522 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fb760060f70 0x7fb760063430 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:09.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.923+0000 7fb77d017700 1 --2- 172.21.15.38:0/4255045522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7780f6530 0x7fb77806b9a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:09.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.923+0000 7fb77d017700 1 --2- 172.21.15.38:0/4255045522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7780f77d0 0x7fb778069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:09.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.923+0000 7fb77d017700 1 -- 172.21.15.38:0/4255045522 >> 172.21.15.38:0/4255045522 conn(0x7fb7780f1c90 msgr2=0x7fb7780fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:09.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.923+0000 7fb77d017700 1 -- 172.21.15.38:0/4255045522 shutdown_connections 2024-02-15T18:41:09.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:09.923+0000 7fb77d017700 1 -- 172.21.15.38:0/4255045522 wait complete. 2024-02-15T18:41:10.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:09.254245+0000 mon.smithi038 (mon.0) 519 : audit [DBG] from='client.? 172.21.15.38:0/2615145380' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:10.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:09.613417+0000 mon.smithi038 (mon.0) 520 : audit [DBG] from='client.? 172.21.15.38:0/3882664285' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:41:10.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:09.921616+0000 mgr.smithi038.khesbl (mgr.14664) 273 : audit [DBG] from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:41:10.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:09.254245+0000 mon.smithi038 (mon.0) 519 : audit [DBG] from='client.? 172.21.15.38:0/2615145380' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:10.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:09.613417+0000 mon.smithi038 (mon.0) 520 : audit [DBG] from='client.? 172.21.15.38:0/3882664285' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:41:10.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:09.921616+0000 mgr.smithi038.khesbl (mgr.14664) 273 : audit [DBG] from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:41:11.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:10.997398+0000 mgr.smithi038.khesbl (mgr.14664) 274 : cluster [DBG] pgmap v191: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-02-15T18:41:11.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:10.997398+0000 mgr.smithi038.khesbl (mgr.14664) 274 : cluster [DBG] pgmap v191: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-02-15T18:41:13.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:11.939055+0000 mon.smithi038 (mon.0) 521 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:13.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:11.939980+0000 mon.smithi038 (mon.0) 522 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:41:13.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:12.079408+0000 mon.smithi038 (mon.0) 523 : cluster [DBG] mds.? [v2:172.21.15.38:6826/4077527450,v1:172.21.15.38:6827/4077527450] up:reconnect 2024-02-15T18:41:13.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:12.079467+0000 mon.smithi038 (mon.0) 524 : cluster [DBG] mds.? [v2:172.21.15.121:6824/180558011,v1:172.21.15.121:6825/180558011] up:boot 2024-02-15T18:41:13.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:12.079543+0000 mon.smithi038 (mon.0) 525 : cluster [DBG] fsmap cephfs:1/1 {0=cephfs.smithi038.hxoggl=up:reconnect} 3 up:standby 2024-02-15T18:41:13.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:12.079674+0000 mon.smithi038 (mon.0) 526 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.mvumlh"}]: dispatch 2024-02-15T18:41:13.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:12.080863+0000 mds.cephfs.smithi038.hxoggl (mds.0) 1 : cluster [DBG] reconnect by client.14498 v1:192.168.0.1:0/2650192726 after 0.00499991 2024-02-15T18:41:13.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:12.080976+0000 mds.cephfs.smithi038.hxoggl (mds.0) 2 : cluster [DBG] reconnect by client.14504 v1:172.21.15.121:0/3468878019 after 0.00499991 2024-02-15T18:41:13.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:11.939055+0000 mon.smithi038 (mon.0) 521 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:13.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:11.939980+0000 mon.smithi038 (mon.0) 522 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:41:13.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:12.079408+0000 mon.smithi038 (mon.0) 523 : cluster [DBG] mds.? [v2:172.21.15.38:6826/4077527450,v1:172.21.15.38:6827/4077527450] up:reconnect 2024-02-15T18:41:13.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:12.079467+0000 mon.smithi038 (mon.0) 524 : cluster [DBG] mds.? [v2:172.21.15.121:6824/180558011,v1:172.21.15.121:6825/180558011] up:boot 2024-02-15T18:41:13.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:12.079543+0000 mon.smithi038 (mon.0) 525 : cluster [DBG] fsmap cephfs:1/1 {0=cephfs.smithi038.hxoggl=up:reconnect} 3 up:standby 2024-02-15T18:41:13.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:12.079674+0000 mon.smithi038 (mon.0) 526 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.mvumlh"}]: dispatch 2024-02-15T18:41:13.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:12.080863+0000 mds.cephfs.smithi038.hxoggl (mds.0) 1 : cluster [DBG] reconnect by client.14498 v1:192.168.0.1:0/2650192726 after 0.00499991 2024-02-15T18:41:13.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:12.080976+0000 mds.cephfs.smithi038.hxoggl (mds.0) 2 : cluster [DBG] reconnect by client.14504 v1:172.21.15.121:0/3468878019 after 0.00499991 2024-02-15T18:41:14.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:12.997978+0000 mgr.smithi038.khesbl (mgr.14664) 275 : cluster [DBG] pgmap v192: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-02-15T18:41:14.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:13.083918+0000 mon.smithi038 (mon.0) 527 : cluster [DBG] mds.? [v2:172.21.15.38:6826/4077527450,v1:172.21.15.38:6827/4077527450] up:rejoin 2024-02-15T18:41:14.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:13.083982+0000 mon.smithi038 (mon.0) 528 : cluster [DBG] fsmap cephfs:1/1 {0=cephfs.smithi038.hxoggl=up:rejoin} 3 up:standby 2024-02-15T18:41:14.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:13.135410+0000 mon.smithi038 (mon.0) 529 : cluster [INF] daemon mds.cephfs.smithi038.hxoggl is now active in filesystem cephfs as rank 0 2024-02-15T18:41:14.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:12.997978+0000 mgr.smithi038.khesbl (mgr.14664) 275 : cluster [DBG] pgmap v192: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-02-15T18:41:14.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:13.083918+0000 mon.smithi038 (mon.0) 527 : cluster [DBG] mds.? [v2:172.21.15.38:6826/4077527450,v1:172.21.15.38:6827/4077527450] up:rejoin 2024-02-15T18:41:14.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:13.083982+0000 mon.smithi038 (mon.0) 528 : cluster [DBG] fsmap cephfs:1/1 {0=cephfs.smithi038.hxoggl=up:rejoin} 3 up:standby 2024-02-15T18:41:14.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:13.135410+0000 mon.smithi038 (mon.0) 529 : cluster [INF] daemon mds.cephfs.smithi038.hxoggl is now active in filesystem cephfs as rank 0 2024-02-15T18:41:15.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:14.081635+0000 mon.smithi038 (mon.0) 530 : cluster [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-15T18:41:15.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:14.081692+0000 mon.smithi038 (mon.0) 531 : cluster [INF] Cluster is now healthy 2024-02-15T18:41:15.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:14.089863+0000 mon.smithi038 (mon.0) 532 : cluster [DBG] mds.? [v2:172.21.15.38:6826/4077527450,v1:172.21.15.38:6827/4077527450] up:active 2024-02-15T18:41:15.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:14.089938+0000 mon.smithi038 (mon.0) 533 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi038.hxoggl=up:active} 3 up:standby 2024-02-15T18:41:15.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:14.081635+0000 mon.smithi038 (mon.0) 530 : cluster [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-15T18:41:15.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:14.081692+0000 mon.smithi038 (mon.0) 531 : cluster [INF] Cluster is now healthy 2024-02-15T18:41:15.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:14.089863+0000 mon.smithi038 (mon.0) 532 : cluster [DBG] mds.? [v2:172.21.15.38:6826/4077527450,v1:172.21.15.38:6827/4077527450] up:active 2024-02-15T18:41:15.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:14.089938+0000 mon.smithi038 (mon.0) 533 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi038.hxoggl=up:active} 3 up:standby 2024-02-15T18:41:16.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:14.998535+0000 mgr.smithi038.khesbl (mgr.14664) 276 : cluster [DBG] pgmap v193: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-02-15T18:41:16.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:15.154125+0000 mon.smithi038 (mon.0) 534 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:16.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:15.154994+0000 mon.smithi038 (mon.0) 535 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:41:16.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:15.155908+0000 mon.smithi038 (mon.0) 536 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:41:16.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:15.160330+0000 mon.smithi038 (mon.0) 537 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:16.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:15.167264+0000 mon.smithi038 (mon.0) 538 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:41:16.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:15.170130+0000 mon.smithi038 (mon.0) 539 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:16.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:15.171707+0000 mon.smithi038 (mon.0) 540 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:16.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:15.173243+0000 mon.smithi038 (mon.0) 541 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:16.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:15.175398+0000 mon.smithi038 (mon.0) 542 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:16.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:15.177362+0000 mon.smithi038 (mon.0) 543 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi121.ovuzki"]}]: dispatch 2024-02-15T18:41:16.298 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:14.998535+0000 mgr.smithi038.khesbl (mgr.14664) 276 : cluster [DBG] pgmap v193: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-02-15T18:41:16.298 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:15.154125+0000 mon.smithi038 (mon.0) 534 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:16.298 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:15.154994+0000 mon.smithi038 (mon.0) 535 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:41:16.298 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:15.155908+0000 mon.smithi038 (mon.0) 536 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:41:16.298 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:15.160330+0000 mon.smithi038 (mon.0) 537 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:16.298 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:15.167264+0000 mon.smithi038 (mon.0) 538 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:41:16.298 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:15.170130+0000 mon.smithi038 (mon.0) 539 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:16.298 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:15.171707+0000 mon.smithi038 (mon.0) 540 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:16.298 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:15.173243+0000 mon.smithi038 (mon.0) 541 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:16.299 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:15.175398+0000 mon.smithi038 (mon.0) 542 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:16.299 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:15.177362+0000 mon.smithi038 (mon.0) 543 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi121.ovuzki"]}]: dispatch 2024-02-15T18:41:17.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:15.156698+0000 mgr.smithi038.khesbl (mgr.14664) 277 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:41:17.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:15.177950+0000 mgr.smithi038.khesbl (mgr.14664) 278 : cephadm [INF] Upgrade: It appears safe to stop mds.cephfs.smithi121.ovuzki 2024-02-15T18:41:17.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:16.854622+0000 mon.smithi038 (mon.0) 544 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:17.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:16.855601+0000 mon.smithi038 (mon.0) 545 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi121.ovuzki", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:41:17.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:16.856467+0000 mon.smithi038 (mon.0) 546 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:41:17.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:15.156698+0000 mgr.smithi038.khesbl (mgr.14664) 277 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:41:17.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:15.177950+0000 mgr.smithi038.khesbl (mgr.14664) 278 : cephadm [INF] Upgrade: It appears safe to stop mds.cephfs.smithi121.ovuzki 2024-02-15T18:41:17.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:16.854622+0000 mon.smithi038 (mon.0) 544 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:17.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:16.855601+0000 mon.smithi038 (mon.0) 545 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi121.ovuzki", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:41:17.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:16.856467+0000 mon.smithi038 (mon.0) 546 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:41:18.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:16.845030+0000 mgr.smithi038.khesbl (mgr.14664) 279 : cephadm [INF] Upgrade: Updating mds.cephfs.smithi121.ovuzki 2024-02-15T18:41:18.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:16.857170+0000 mgr.smithi038.khesbl (mgr.14664) 280 : cephadm [INF] Deploying daemon mds.cephfs.smithi121.ovuzki on smithi121 2024-02-15T18:41:18.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:16.999208+0000 mgr.smithi038.khesbl (mgr.14664) 281 : cluster [DBG] pgmap v194: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 5 op/s 2024-02-15T18:41:18.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:16.845030+0000 mgr.smithi038.khesbl (mgr.14664) 279 : cephadm [INF] Upgrade: Updating mds.cephfs.smithi121.ovuzki 2024-02-15T18:41:18.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:16.857170+0000 mgr.smithi038.khesbl (mgr.14664) 280 : cephadm [INF] Deploying daemon mds.cephfs.smithi121.ovuzki on smithi121 2024-02-15T18:41:18.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:16.999208+0000 mgr.smithi038.khesbl (mgr.14664) 281 : cluster [DBG] pgmap v194: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 5 op/s 2024-02-15T18:41:19.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:18.691628+0000 mon.smithi038 (mon.0) 547 : cluster [WRN] daemon mds.cephfs.smithi121.ovuzki compat changed unexpectedly 2024-02-15T18:41:19.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:18.700203+0000 mon.smithi038 (mon.0) 548 : cluster [DBG] osdmap e80: 6 total, 6 up, 6 in 2024-02-15T18:41:19.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:18.691628+0000 mon.smithi038 (mon.0) 547 : cluster [WRN] daemon mds.cephfs.smithi121.ovuzki compat changed unexpectedly 2024-02-15T18:41:19.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:18.700203+0000 mon.smithi038 (mon.0) 548 : cluster [DBG] osdmap e80: 6 total, 6 up, 6 in 2024-02-15T18:41:20.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:18.986954+0000 mon.smithi038 (mon.0) 549 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi038.hxoggl=up:active} 2 up:standby 2024-02-15T18:41:20.272 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:18.999638+0000 mgr.smithi038.khesbl (mgr.14664) 282 : cluster [DBG] pgmap v196: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 5.2 MiB/s rd, 204 B/s wr, 5 op/s 2024-02-15T18:41:20.275 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:19.089474+0000 mon.smithi038 (mon.0) 550 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.ovuzki"}]: dispatch 2024-02-15T18:41:20.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:18.986954+0000 mon.smithi038 (mon.0) 549 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi038.hxoggl=up:active} 2 up:standby 2024-02-15T18:41:20.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:18.999638+0000 mgr.smithi038.khesbl (mgr.14664) 282 : cluster [DBG] pgmap v196: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 5.2 MiB/s rd, 204 B/s wr, 5 op/s 2024-02-15T18:41:20.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:19.089474+0000 mon.smithi038 (mon.0) 550 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.ovuzki"}]: dispatch 2024-02-15T18:41:21.225 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:21.000305+0000 mgr.smithi038.khesbl (mgr.14664) 283 : cluster [DBG] pgmap v197: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 5.1 KiB/s wr, 6 op/s 2024-02-15T18:41:21.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:21.000305+0000 mgr.smithi038.khesbl (mgr.14664) 283 : cluster [DBG] pgmap v197: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 5.1 KiB/s wr, 6 op/s 2024-02-15T18:41:23.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:22.059105+0000 mon.smithi038 (mon.0) 551 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:23.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:22.059998+0000 mon.smithi038 (mon.0) 552 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:41:23.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:22.077918+0000 mon.smithi038 (mon.0) 553 : cluster [DBG] mds.? [v2:172.21.15.121:6826/1075229339,v1:172.21.15.121:6827/1075229339] up:boot 2024-02-15T18:41:23.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:22.077967+0000 mon.smithi038 (mon.0) 554 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi038.hxoggl=up:active} 3 up:standby 2024-02-15T18:41:23.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:22.078111+0000 mon.smithi038 (mon.0) 555 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.ovuzki"}]: dispatch 2024-02-15T18:41:23.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:23.000990+0000 mgr.smithi038.khesbl (mgr.14664) 284 : cluster [DBG] pgmap v198: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 823 KiB/s rd, 5.1 KiB/s wr, 6 op/s 2024-02-15T18:41:23.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:22.059105+0000 mon.smithi038 (mon.0) 551 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:23.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:22.059998+0000 mon.smithi038 (mon.0) 552 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:41:23.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:22.077918+0000 mon.smithi038 (mon.0) 553 : cluster [DBG] mds.? [v2:172.21.15.121:6826/1075229339,v1:172.21.15.121:6827/1075229339] up:boot 2024-02-15T18:41:23.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:22.077967+0000 mon.smithi038 (mon.0) 554 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi038.hxoggl=up:active} 3 up:standby 2024-02-15T18:41:23.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:22.078111+0000 mon.smithi038 (mon.0) 555 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi121.ovuzki"}]: dispatch 2024-02-15T18:41:23.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:23.000990+0000 mgr.smithi038.khesbl (mgr.14664) 284 : cluster [DBG] pgmap v198: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 823 KiB/s rd, 5.1 KiB/s wr, 6 op/s 2024-02-15T18:41:25.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:24.341151+0000 mon.smithi038 (mon.0) 556 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:25.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:24.342144+0000 mon.smithi038 (mon.0) 557 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:41:25.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:24.343109+0000 mon.smithi038 (mon.0) 558 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:41:25.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:24.343929+0000 mgr.smithi038.khesbl (mgr.14664) 285 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:41:25.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:24.347916+0000 mon.smithi038 (mon.0) 559 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:25.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:24.354318+0000 mon.smithi038 (mon.0) 560 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:41:25.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:24.357061+0000 mon.smithi038 (mon.0) 561 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:25.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:24.358508+0000 mon.smithi038 (mon.0) 562 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:25.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:24.359976+0000 mon.smithi038 (mon.0) 563 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:25.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:24.362079+0000 mon.smithi038 (mon.0) 564 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:25.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:24.364039+0000 mon.smithi038 (mon.0) 565 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:25.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:24.364673+0000 mgr.smithi038.khesbl (mgr.14664) 286 : cephadm [INF] Upgrade: Setting container_image for all mds 2024-02-15T18:41:25.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:24.368788+0000 mon.smithi038 (mon.0) 566 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:25.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:24.369627+0000 mon.smithi038 (mon.0) 567 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi038.hxoggl"}]: dispatch 2024-02-15T18:41:25.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:24.372925+0000 mon.smithi038 (mon.0) 568 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi038.hxoggl"}]': finished 2024-02-15T18:41:25.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:24.373671+0000 mon.smithi038 (mon.0) 569 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi038.qgkzgn"}]: dispatch 2024-02-15T18:41:25.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:24.377333+0000 mon.smithi038 (mon.0) 570 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi038.qgkzgn"}]': finished 2024-02-15T18:41:25.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:24.378028+0000 mon.smithi038 (mon.0) 571 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi121.mvumlh"}]: dispatch 2024-02-15T18:41:25.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:24.381172+0000 mon.smithi038 (mon.0) 572 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi121.mvumlh"}]': finished 2024-02-15T18:41:25.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:24.381889+0000 mon.smithi038 (mon.0) 573 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi121.ovuzki"}]: dispatch 2024-02-15T18:41:25.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:24.385510+0000 mon.smithi038 (mon.0) 574 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi121.ovuzki"}]': finished 2024-02-15T18:41:25.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:24.386236+0000 mgr.smithi038.khesbl (mgr.14664) 287 : cephadm [INF] Upgrade: Scaling up filesystem cephfs max_mds to 2 2024-02-15T18:41:25.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:24.386443+0000 mon.smithi038 (mon.0) 575 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-02-15T18:41:25.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:25.001540+0000 mgr.smithi038.khesbl (mgr.14664) 288 : cluster [DBG] pgmap v199: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.0 KiB/s rd, 5.1 KiB/s wr, 5 op/s 2024-02-15T18:41:25.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:24.341151+0000 mon.smithi038 (mon.0) 556 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:25.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:24.342144+0000 mon.smithi038 (mon.0) 557 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:41:25.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:24.343109+0000 mon.smithi038 (mon.0) 558 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:41:25.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:24.343929+0000 mgr.smithi038.khesbl (mgr.14664) 285 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:41:25.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:24.347916+0000 mon.smithi038 (mon.0) 559 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:25.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:24.354318+0000 mon.smithi038 (mon.0) 560 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:41:25.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:24.357061+0000 mon.smithi038 (mon.0) 561 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:25.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:24.358508+0000 mon.smithi038 (mon.0) 562 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:25.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:24.359976+0000 mon.smithi038 (mon.0) 563 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:25.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:24.362079+0000 mon.smithi038 (mon.0) 564 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:25.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:24.364039+0000 mon.smithi038 (mon.0) 565 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:25.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:24.364673+0000 mgr.smithi038.khesbl (mgr.14664) 286 : cephadm [INF] Upgrade: Setting container_image for all mds 2024-02-15T18:41:25.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:24.368788+0000 mon.smithi038 (mon.0) 566 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:25.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:24.369627+0000 mon.smithi038 (mon.0) 567 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi038.hxoggl"}]: dispatch 2024-02-15T18:41:25.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:24.372925+0000 mon.smithi038 (mon.0) 568 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi038.hxoggl"}]': finished 2024-02-15T18:41:25.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:24.373671+0000 mon.smithi038 (mon.0) 569 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi038.qgkzgn"}]: dispatch 2024-02-15T18:41:25.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:24.377333+0000 mon.smithi038 (mon.0) 570 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi038.qgkzgn"}]': finished 2024-02-15T18:41:25.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:24.378028+0000 mon.smithi038 (mon.0) 571 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi121.mvumlh"}]: dispatch 2024-02-15T18:41:25.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:24.381172+0000 mon.smithi038 (mon.0) 572 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi121.mvumlh"}]': finished 2024-02-15T18:41:25.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:24.381889+0000 mon.smithi038 (mon.0) 573 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi121.ovuzki"}]: dispatch 2024-02-15T18:41:25.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:24.385510+0000 mon.smithi038 (mon.0) 574 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi121.ovuzki"}]': finished 2024-02-15T18:41:25.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:24.386236+0000 mgr.smithi038.khesbl (mgr.14664) 287 : cephadm [INF] Upgrade: Scaling up filesystem cephfs max_mds to 2 2024-02-15T18:41:25.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:24.386443+0000 mon.smithi038 (mon.0) 575 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-02-15T18:41:25.773 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:25.001540+0000 mgr.smithi038.khesbl (mgr.14664) 288 : cluster [DBG] pgmap v199: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.0 KiB/s rd, 5.1 KiB/s wr, 5 op/s 2024-02-15T18:41:26.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:25.383374+0000 mon.smithi038 (mon.0) 576 : cluster [WRN] Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-15T18:41:26.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:25.388902+0000 mon.smithi038 (mon.0) 577 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-02-15T18:41:26.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:25.388995+0000 mon.smithi038 (mon.0) 578 : cluster [INF] daemon mds.cephfs.smithi038.qgkzgn assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-15T18:41:26.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:25.389078+0000 mon.smithi038 (mon.0) 579 : cluster [INF] Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-15T18:41:26.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:25.389099+0000 mon.smithi038 (mon.0) 580 : cluster [INF] Cluster is now healthy 2024-02-15T18:41:26.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:25.389389+0000 mon.smithi038 (mon.0) 581 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi038.hxoggl=up:active} 3 up:standby 2024-02-15T18:41:26.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:25.395796+0000 mon.smithi038 (mon.0) 582 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi038.hxoggl=up:active,1=cephfs.smithi038.qgkzgn=up:starting} 2 up:standby 2024-02-15T18:41:26.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:25.400345+0000 mon.smithi038 (mon.0) 583 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:26.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:25.401353+0000 mon.smithi038 (mon.0) 584 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:26.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:25.401651+0000 mon.smithi038 (mon.0) 585 : cluster [INF] daemon mds.cephfs.smithi038.qgkzgn is now active in filesystem cephfs as rank 1 2024-02-15T18:41:26.522 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:25.402190+0000 mgr.smithi038.khesbl (mgr.14664) 289 : cephadm [INF] Upgrade: Setting container_image for all rgw 2024-02-15T18:41:26.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:25.406320+0000 mon.smithi038 (mon.0) 586 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:26.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:25.407515+0000 mon.smithi038 (mon.0) 587 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:26.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:25.408131+0000 mgr.smithi038.khesbl (mgr.14664) 290 : cephadm [INF] Upgrade: Setting container_image for all rbd-mirror 2024-02-15T18:41:26.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:25.411917+0000 mon.smithi038 (mon.0) 588 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:26.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:25.413001+0000 mon.smithi038 (mon.0) 589 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:26.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:25.414136+0000 mon.smithi038 (mon.0) 590 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:26.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:25.414853+0000 mgr.smithi038.khesbl (mgr.14664) 291 : cephadm [INF] Upgrade: Setting container_image for all iscsi 2024-02-15T18:41:26.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:25.418635+0000 mon.smithi038 (mon.0) 591 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:26.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:25.419708+0000 mon.smithi038 (mon.0) 592 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:26.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:25.420332+0000 mgr.smithi038.khesbl (mgr.14664) 292 : cephadm [INF] Upgrade: Setting container_image for all nfs 2024-02-15T18:41:26.523 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:26 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:25.423929+0000 mon.smithi038 (mon.0) 593 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:26.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:25.383374+0000 mon.smithi038 (mon.0) 576 : cluster [WRN] Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-15T18:41:26.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:25.388902+0000 mon.smithi038 (mon.0) 577 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-02-15T18:41:26.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:25.388995+0000 mon.smithi038 (mon.0) 578 : cluster [INF] daemon mds.cephfs.smithi038.qgkzgn assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-15T18:41:26.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:25.389078+0000 mon.smithi038 (mon.0) 579 : cluster [INF] Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-15T18:41:26.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:25.389099+0000 mon.smithi038 (mon.0) 580 : cluster [INF] Cluster is now healthy 2024-02-15T18:41:26.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:25.389389+0000 mon.smithi038 (mon.0) 581 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi038.hxoggl=up:active} 3 up:standby 2024-02-15T18:41:26.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:25.395796+0000 mon.smithi038 (mon.0) 582 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi038.hxoggl=up:active,1=cephfs.smithi038.qgkzgn=up:starting} 2 up:standby 2024-02-15T18:41:26.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:25.400345+0000 mon.smithi038 (mon.0) 583 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:26.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:25.401353+0000 mon.smithi038 (mon.0) 584 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:26.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:25.401651+0000 mon.smithi038 (mon.0) 585 : cluster [INF] daemon mds.cephfs.smithi038.qgkzgn is now active in filesystem cephfs as rank 1 2024-02-15T18:41:26.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:25.402190+0000 mgr.smithi038.khesbl (mgr.14664) 289 : cephadm [INF] Upgrade: Setting container_image for all rgw 2024-02-15T18:41:26.608 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:25.406320+0000 mon.smithi038 (mon.0) 586 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:26.609 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:25.407515+0000 mon.smithi038 (mon.0) 587 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:26.609 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:25.408131+0000 mgr.smithi038.khesbl (mgr.14664) 290 : cephadm [INF] Upgrade: Setting container_image for all rbd-mirror 2024-02-15T18:41:26.609 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:25.411917+0000 mon.smithi038 (mon.0) 588 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:26.609 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:25.413001+0000 mon.smithi038 (mon.0) 589 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:26.609 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:25.414136+0000 mon.smithi038 (mon.0) 590 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:26.609 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:25.414853+0000 mgr.smithi038.khesbl (mgr.14664) 291 : cephadm [INF] Upgrade: Setting container_image for all iscsi 2024-02-15T18:41:26.609 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:25.418635+0000 mon.smithi038 (mon.0) 591 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:26.609 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:25.419708+0000 mon.smithi038 (mon.0) 592 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:26.609 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:25.420332+0000 mgr.smithi038.khesbl (mgr.14664) 292 : cephadm [INF] Upgrade: Setting container_image for all nfs 2024-02-15T18:41:26.609 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:26 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:25.423929+0000 mon.smithi038 (mon.0) 593 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:27.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:26.402561+0000 mon.smithi038 (mon.0) 594 : cluster [DBG] mds.? [v2:172.21.15.38:6828/1616551408,v1:172.21.15.38:6829/1616551408] up:active 2024-02-15T18:41:27.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:26.402612+0000 mon.smithi038 (mon.0) 595 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi038.hxoggl=up:active,1=cephfs.smithi038.qgkzgn=up:active} 2 up:standby 2024-02-15T18:41:27.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:26.997134+0000 mgr.smithi038.khesbl (mgr.14664) 293 : cephadm [INF] Upgrade: Updating node-exporter.smithi038 (1/2) 2024-02-15T18:41:27.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:26.997318+0000 mgr.smithi038.khesbl (mgr.14664) 294 : cephadm [INF] Deploying daemon node-exporter.smithi038 on smithi038 2024-02-15T18:41:27.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:27.001977+0000 mgr.smithi038.khesbl (mgr.14664) 295 : cluster [DBG] pgmap v200: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.1 KiB/s rd, 4.9 KiB/s wr, 5 op/s 2024-02-15T18:41:27.855 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:26.402561+0000 mon.smithi038 (mon.0) 594 : cluster [DBG] mds.? [v2:172.21.15.38:6828/1616551408,v1:172.21.15.38:6829/1616551408] up:active 2024-02-15T18:41:27.855 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:26.402612+0000 mon.smithi038 (mon.0) 595 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi038.hxoggl=up:active,1=cephfs.smithi038.qgkzgn=up:active} 2 up:standby 2024-02-15T18:41:27.855 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:26.997134+0000 mgr.smithi038.khesbl (mgr.14664) 293 : cephadm [INF] Upgrade: Updating node-exporter.smithi038 (1/2) 2024-02-15T18:41:27.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:26.997318+0000 mgr.smithi038.khesbl (mgr.14664) 294 : cephadm [INF] Deploying daemon node-exporter.smithi038 on smithi038 2024-02-15T18:41:27.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:27.001977+0000 mgr.smithi038.khesbl (mgr.14664) 295 : cluster [DBG] pgmap v200: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.1 KiB/s rd, 4.9 KiB/s wr, 5 op/s 2024-02-15T18:41:29.134 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:29.002418+0000 mgr.smithi038.khesbl (mgr.14664) 296 : cluster [DBG] pgmap v201: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 4.8 KiB/s wr, 3 op/s 2024-02-15T18:41:29.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:29.002418+0000 mgr.smithi038.khesbl (mgr.14664) 296 : cluster [DBG] pgmap v201: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 4.8 KiB/s wr, 3 op/s 2024-02-15T18:41:31.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:31.003008+0000 mgr.smithi038.khesbl (mgr.14664) 297 : cluster [DBG] pgmap v202: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.2 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-02-15T18:41:31.467 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:31.003008+0000 mgr.smithi038.khesbl (mgr.14664) 297 : cluster [DBG] pgmap v202: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.2 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-02-15T18:41:33.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:32.425950+0000 mon.smithi038 (mon.0) 596 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:33.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:33.003677+0000 mgr.smithi038.khesbl (mgr.14664) 298 : cluster [DBG] pgmap v203: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.7 KiB/s rd, 170 B/s wr, 3 op/s 2024-02-15T18:41:33.855 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:32.425950+0000 mon.smithi038 (mon.0) 596 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:33.855 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:33.003677+0000 mgr.smithi038.khesbl (mgr.14664) 298 : cluster [DBG] pgmap v203: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.7 KiB/s rd, 170 B/s wr, 3 op/s 2024-02-15T18:41:34.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:33.939794+0000 mgr.smithi038.khesbl (mgr.14664) 299 : cephadm [INF] Upgrade: Updating node-exporter.smithi121 (2/2) 2024-02-15T18:41:34.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:34 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:33.939993+0000 mgr.smithi038.khesbl (mgr.14664) 300 : cephadm [INF] Deploying daemon node-exporter.smithi121 on smithi121 2024-02-15T18:41:34.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:33.939794+0000 mgr.smithi038.khesbl (mgr.14664) 299 : cephadm [INF] Upgrade: Updating node-exporter.smithi121 (2/2) 2024-02-15T18:41:34.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:34 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:33.939993+0000 mgr.smithi038.khesbl (mgr.14664) 300 : cephadm [INF] Deploying daemon node-exporter.smithi121 on smithi121 2024-02-15T18:41:35.597 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:35.004257+0000 mgr.smithi038.khesbl (mgr.14664) 301 : cluster [DBG] pgmap v204: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.7 KiB/s rd, 170 B/s wr, 3 op/s 2024-02-15T18:41:35.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:35.004257+0000 mgr.smithi038.khesbl (mgr.14664) 301 : cluster [DBG] pgmap v204: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.7 KiB/s rd, 170 B/s wr, 3 op/s 2024-02-15T18:41:37.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:37.004882+0000 mgr.smithi038.khesbl (mgr.14664) 302 : cluster [DBG] pgmap v205: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.7 KiB/s rd, 170 B/s wr, 3 op/s 2024-02-15T18:41:37.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:37.004882+0000 mgr.smithi038.khesbl (mgr.14664) 302 : cluster [DBG] pgmap v205: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.7 KiB/s rd, 170 B/s wr, 3 op/s 2024-02-15T18:41:40.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.034+0000 7f0eb0584700 1 -- 172.21.15.38:0/183608211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ea80f55c0 msgr2=0x7f0ea80f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:40.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.034+0000 7f0eb0584700 1 --2- 172.21.15.38:0/183608211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ea80f55c0 0x7f0ea80f59e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f0ea4004660 tx=0x7f0ea4009b30).stop 2024-02-15T18:41:40.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.034+0000 7f0eb0584700 1 -- 172.21.15.38:0/183608211 shutdown_connections 2024-02-15T18:41:40.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.034+0000 7f0eb0584700 1 --2- 172.21.15.38:0/183608211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ea80f6860 0x7f0ea80f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.034+0000 7f0eb0584700 1 --2- 172.21.15.38:0/183608211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ea80f55c0 0x7f0ea80f59e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.034+0000 7f0eb0584700 1 -- 172.21.15.38:0/183608211 >> 172.21.15.38:0/183608211 conn(0x7f0ea80f0d40 msgr2=0x7f0ea80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:40.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.034+0000 7f0eb0584700 1 -- 172.21.15.38:0/183608211 shutdown_connections 2024-02-15T18:41:40.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.034+0000 7f0eb0584700 1 -- 172.21.15.38:0/183608211 wait complete. 2024-02-15T18:41:40.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.035+0000 7f0eb0584700 1 Processor -- start 2024-02-15T18:41:40.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.035+0000 7f0eb0584700 1 -- start start 2024-02-15T18:41:40.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.035+0000 7f0eb0584700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ea80f55c0 0x7f0ea80f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:40.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.035+0000 7f0eb0584700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ea80f6860 0x7f0ea80f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:40.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.035+0000 7f0eb0584700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea80f8100 con 0x7f0ea80f6860 2024-02-15T18:41:40.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.035+0000 7f0eb0584700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea80f8240 con 0x7f0ea80f55c0 2024-02-15T18:41:40.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.036+0000 7f0eadb1f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ea80f6860 0x7f0ea80f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:40.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.036+0000 7f0eadb1f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ea80f6860 0x7f0ea80f7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45006/0 (socket says 172.21.15.38:45006) 2024-02-15T18:41:40.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.036+0000 7f0eadb1f700 1 -- 172.21.15.38:0/3600953137 learned_addr learned my addr 172.21.15.38:0/3600953137 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:41:40.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.036+0000 7f0eae320700 1 --2- 172.21.15.38:0/3600953137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ea80f55c0 0x7f0ea80f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:40.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.036+0000 7f0eadb1f700 1 -- 172.21.15.38:0/3600953137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ea80f55c0 msgr2=0x7f0ea80f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:40.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.036+0000 7f0eadb1f700 1 --2- 172.21.15.38:0/3600953137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ea80f55c0 0x7f0ea80f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.036+0000 7f0eadb1f700 1 -- 172.21.15.38:0/3600953137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea4005040 con 0x7f0ea80f6860 2024-02-15T18:41:40.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.036+0000 7f0eadb1f700 1 --2- 172.21.15.38:0/3600953137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ea80f6860 0x7f0ea80f7ba0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f0e9c00c530 tx=0x7f0e9c007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:40.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.036+0000 7f0e9affd700 1 -- 172.21.15.38:0/3600953137 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e9c00f650 con 0x7f0ea80f6860 2024-02-15T18:41:40.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.036+0000 7f0e9affd700 1 -- 172.21.15.38:0/3600953137 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f0e9c00fca0 con 0x7f0ea80f6860 2024-02-15T18:41:40.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.036+0000 7f0eb0584700 1 -- 172.21.15.38:0/3600953137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea80f8520 con 0x7f0ea80f6860 2024-02-15T18:41:40.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.036+0000 7f0e9affd700 1 -- 172.21.15.38:0/3600953137 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e9c012e10 con 0x7f0ea80f6860 2024-02-15T18:41:40.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.036+0000 7f0eb0584700 1 -- 172.21.15.38:0/3600953137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea810f020 con 0x7f0ea80f6860 2024-02-15T18:41:40.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.037+0000 7f0eb0584700 1 -- 172.21.15.38:0/3600953137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea8045ad0 con 0x7f0ea80f6860 2024-02-15T18:41:40.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.038+0000 7f0e9affd700 1 -- 172.21.15.38:0/3600953137 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f0e9c010070 con 0x7f0ea80f6860 2024-02-15T18:41:40.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.038+0000 7f0e9affd700 1 --2- 172.21.15.38:0/3600953137 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f0e94060f20 0x7f0e940633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:40.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.038+0000 7f0e9affd700 1 -- 172.21.15.38:0/3600953137 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f0e9c07bdf0 con 0x7f0ea80f6860 2024-02-15T18:41:40.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.038+0000 7f0eae320700 1 --2- 172.21.15.38:0/3600953137 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f0e94060f20 0x7f0e940633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:40.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.038+0000 7f0eae320700 1 --2- 172.21.15.38:0/3600953137 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f0e94060f20 0x7f0e940633e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f0ea80f94b0 tx=0x7f0ea4009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:40.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.042+0000 7f0e9affd700 1 -- 172.21.15.38:0/3600953137 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0e9c047f60 con 0x7f0ea80f6860 2024-02-15T18:41:40.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:38.802471+0000 mon.smithi038 (mon.0) 597 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:40.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:38.803383+0000 mon.smithi038 (mon.0) 598 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:41:40.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:39.005260+0000 mgr.smithi038.khesbl (mgr.14664) 303 : cluster [DBG] pgmap v206: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.8 KiB/s rd, 170 B/s wr, 3 op/s 2024-02-15T18:41:40.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.239+0000 7f0eb0584700 1 -- 172.21.15.38:0/3600953137 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ea80faca0 con 0x7f0e94060f20 2024-02-15T18:41:40.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.240+0000 7f0e9affd700 1 -- 172.21.15.38:0/3600953137 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f0ea80faca0 con 0x7f0e94060f20 2024-02-15T18:41:40.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.242+0000 7f0eb0584700 1 -- 172.21.15.38:0/3600953137 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f0e94060f20 msgr2=0x7f0e940633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:40.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.242+0000 7f0eb0584700 1 --2- 172.21.15.38:0/3600953137 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f0e94060f20 0x7f0e940633e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f0ea80f94b0 tx=0x7f0ea4009b30).stop 2024-02-15T18:41:40.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.242+0000 7f0eb0584700 1 -- 172.21.15.38:0/3600953137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ea80f6860 msgr2=0x7f0ea80f7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:40.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.242+0000 7f0eb0584700 1 --2- 172.21.15.38:0/3600953137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ea80f6860 0x7f0ea80f7ba0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f0e9c00c530 tx=0x7f0e9c007e20).stop 2024-02-15T18:41:40.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.242+0000 7f0eb0584700 1 -- 172.21.15.38:0/3600953137 shutdown_connections 2024-02-15T18:41:40.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.242+0000 7f0eb0584700 1 --2- 172.21.15.38:0/3600953137 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f0e94060f20 0x7f0e940633e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.242+0000 7f0eb0584700 1 --2- 172.21.15.38:0/3600953137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ea80f55c0 0x7f0ea80f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.242+0000 7f0eb0584700 1 --2- 172.21.15.38:0/3600953137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ea80f6860 0x7f0ea80f7ba0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.242+0000 7f0eb0584700 1 -- 172.21.15.38:0/3600953137 >> 172.21.15.38:0/3600953137 conn(0x7f0ea80f0d40 msgr2=0x7f0ea805fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:40.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.242+0000 7f0eb0584700 1 -- 172.21.15.38:0/3600953137 shutdown_connections 2024-02-15T18:41:40.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.242+0000 7f0eb0584700 1 -- 172.21.15.38:0/3600953137 wait complete. 2024-02-15T18:41:40.241 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-02-15T18:41:40.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:38.802471+0000 mon.smithi038 (mon.0) 597 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:40.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:38.803383+0000 mon.smithi038 (mon.0) 598 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:41:40.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:39.005260+0000 mgr.smithi038.khesbl (mgr.14664) 303 : cluster [DBG] pgmap v206: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.8 KiB/s rd, 170 B/s wr, 3 op/s 2024-02-15T18:41:40.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.347+0000 7fcc2db56700 1 -- 172.21.15.38:0/151908729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc280f6530 msgr2=0x7fcc280f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:40.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.347+0000 7fcc2db56700 1 --2- 172.21.15.38:0/151908729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc280f6530 0x7fcc280f6950 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fcc1c004660 tx=0x7fcc1c009b30).stop 2024-02-15T18:41:40.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.348+0000 7fcc2db56700 1 -- 172.21.15.38:0/151908729 shutdown_connections 2024-02-15T18:41:40.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.348+0000 7fcc2db56700 1 --2- 172.21.15.38:0/151908729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc280f77d0 0x7fcc280f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.348+0000 7fcc2db56700 1 --2- 172.21.15.38:0/151908729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc280f6530 0x7fcc280f6950 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.348+0000 7fcc2db56700 1 -- 172.21.15.38:0/151908729 >> 172.21.15.38:0/151908729 conn(0x7fcc280f1cd0 msgr2=0x7fcc280f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:40.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.348+0000 7fcc2db56700 1 -- 172.21.15.38:0/151908729 shutdown_connections 2024-02-15T18:41:40.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.348+0000 7fcc2db56700 1 -- 172.21.15.38:0/151908729 wait complete. 2024-02-15T18:41:40.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.348+0000 7fcc2db56700 1 Processor -- start 2024-02-15T18:41:40.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.349+0000 7fcc2db56700 1 -- start start 2024-02-15T18:41:40.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.349+0000 7fcc2db56700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc280f6530 0x7fcc28106e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:40.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.349+0000 7fcc2db56700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc280f77d0 0x7fcc28107340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:40.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.349+0000 7fcc2db56700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc28107940 con 0x7fcc280f6530 2024-02-15T18:41:40.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.349+0000 7fcc2db56700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc28108150 con 0x7fcc280f77d0 2024-02-15T18:41:40.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.349+0000 7fcc2cb54700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc280f6530 0x7fcc28106e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:40.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.349+0000 7fcc2cb54700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc280f6530 0x7fcc28106e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45012/0 (socket says 172.21.15.38:45012) 2024-02-15T18:41:40.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.349+0000 7fcc2cb54700 1 -- 172.21.15.38:0/2882473622 learned_addr learned my addr 172.21.15.38:0/2882473622 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:41:40.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.349+0000 7fcc27fff700 1 --2- 172.21.15.38:0/2882473622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc280f77d0 0x7fcc28107340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:40.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.350+0000 7fcc27fff700 1 -- 172.21.15.38:0/2882473622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc280f6530 msgr2=0x7fcc28106e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:40.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.350+0000 7fcc27fff700 1 --2- 172.21.15.38:0/2882473622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc280f6530 0x7fcc28106e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.350+0000 7fcc27fff700 1 -- 172.21.15.38:0/2882473622 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c005040 con 0x7fcc280f77d0 2024-02-15T18:41:40.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.350+0000 7fcc27fff700 1 --2- 172.21.15.38:0/2882473622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc280f77d0 0x7fcc28107340 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fcc14002700 tx=0x7fcc14009cc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:40.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.350+0000 7fcc257fa700 1 -- 172.21.15.38:0/2882473622 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc140079c0 con 0x7fcc280f77d0 2024-02-15T18:41:40.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.350+0000 7fcc257fa700 1 -- 172.21.15.38:0/2882473622 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fcc1400f040 con 0x7fcc280f77d0 2024-02-15T18:41:40.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.350+0000 7fcc2db56700 1 -- 172.21.15.38:0/2882473622 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc281083d0 con 0x7fcc280f77d0 2024-02-15T18:41:40.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.350+0000 7fcc257fa700 1 -- 172.21.15.38:0/2882473622 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc14014800 con 0x7fcc280f77d0 2024-02-15T18:41:40.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.350+0000 7fcc2db56700 1 -- 172.21.15.38:0/2882473622 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc28108990 con 0x7fcc280f77d0 2024-02-15T18:41:40.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.352+0000 7fcc257fa700 1 -- 172.21.15.38:0/2882473622 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fcc1401b060 con 0x7fcc280f77d0 2024-02-15T18:41:40.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.352+0000 7fcc2db56700 1 -- 172.21.15.38:0/2882473622 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc28059100 con 0x7fcc280f77d0 2024-02-15T18:41:40.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.352+0000 7fcc257fa700 1 --2- 172.21.15.38:0/2882473622 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcc18060f70 0x7fcc18063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:40.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.352+0000 7fcc257fa700 1 -- 172.21.15.38:0/2882473622 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fcc1407bed0 con 0x7fcc280f77d0 2024-02-15T18:41:40.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.352+0000 7fcc2cb54700 1 --2- 172.21.15.38:0/2882473622 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcc18060f70 0x7fcc18063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:40.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.353+0000 7fcc2cb54700 1 --2- 172.21.15.38:0/2882473622 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcc18060f70 0x7fcc18063430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fcc1c004630 tx=0x7fcc1c009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:40.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.357+0000 7fcc257fa700 1 -- 172.21.15.38:0/2882473622 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fcc1404c2c0 con 0x7fcc280f77d0 2024-02-15T18:41:40.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.549+0000 7fcc2db56700 1 -- 172.21.15.38:0/2882473622 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc280fef90 con 0x7fcc18060f70 2024-02-15T18:41:40.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.550+0000 7fcc257fa700 1 -- 172.21.15.38:0/2882473622 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fcc280fef90 con 0x7fcc18060f70 2024-02-15T18:41:40.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.551+0000 7fcc2db56700 1 -- 172.21.15.38:0/2882473622 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcc18060f70 msgr2=0x7fcc18063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:40.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.551+0000 7fcc2db56700 1 --2- 172.21.15.38:0/2882473622 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcc18060f70 0x7fcc18063430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fcc1c004630 tx=0x7fcc1c009b30).stop 2024-02-15T18:41:40.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.551+0000 7fcc2db56700 1 -- 172.21.15.38:0/2882473622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc280f77d0 msgr2=0x7fcc28107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:40.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.551+0000 7fcc2db56700 1 --2- 172.21.15.38:0/2882473622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc280f77d0 0x7fcc28107340 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fcc14002700 tx=0x7fcc14009cc0).stop 2024-02-15T18:41:40.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.552+0000 7fcc2db56700 1 -- 172.21.15.38:0/2882473622 shutdown_connections 2024-02-15T18:41:40.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.552+0000 7fcc2db56700 1 --2- 172.21.15.38:0/2882473622 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fcc18060f70 0x7fcc18063430 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.552+0000 7fcc2db56700 1 --2- 172.21.15.38:0/2882473622 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc280f6530 0x7fcc28106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.552+0000 7fcc2db56700 1 --2- 172.21.15.38:0/2882473622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc280f77d0 0x7fcc28107340 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.552+0000 7fcc2db56700 1 -- 172.21.15.38:0/2882473622 >> 172.21.15.38:0/2882473622 conn(0x7fcc280f1cd0 msgr2=0x7fcc280fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:40.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.552+0000 7fcc2db56700 1 -- 172.21.15.38:0/2882473622 shutdown_connections 2024-02-15T18:41:40.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.552+0000 7fcc2db56700 1 -- 172.21.15.38:0/2882473622 wait complete. 2024-02-15T18:41:40.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.654+0000 7fee2ffff700 1 -- 172.21.15.38:0/3170038811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee300f77d0 msgr2=0x7fee300f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:40.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.654+0000 7fee2ffff700 1 --2- 172.21.15.38:0/3170038811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee300f77d0 0x7fee300f7c30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fee20004660 tx=0x7fee20009b30).stop 2024-02-15T18:41:40.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.654+0000 7fee2ffff700 1 -- 172.21.15.38:0/3170038811 shutdown_connections 2024-02-15T18:41:40.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.654+0000 7fee2ffff700 1 --2- 172.21.15.38:0/3170038811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee300f77d0 0x7fee300f7c30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.654+0000 7fee2ffff700 1 --2- 172.21.15.38:0/3170038811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee300f6530 0x7fee300f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.654+0000 7fee2ffff700 1 -- 172.21.15.38:0/3170038811 >> 172.21.15.38:0/3170038811 conn(0x7fee300f1c90 msgr2=0x7fee300f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:40.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.654+0000 7fee2ffff700 1 -- 172.21.15.38:0/3170038811 shutdown_connections 2024-02-15T18:41:40.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.654+0000 7fee2ffff700 1 -- 172.21.15.38:0/3170038811 wait complete. 2024-02-15T18:41:40.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.654+0000 7fee2ffff700 1 Processor -- start 2024-02-15T18:41:40.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.655+0000 7fee2ffff700 1 -- start start 2024-02-15T18:41:40.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.655+0000 7fee2ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee300f6530 0x7fee3006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:40.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.655+0000 7fee2ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee300f77d0 0x7fee30069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:40.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.655+0000 7fee2ffff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3006a380 con 0x7fee300f77d0 2024-02-15T18:41:40.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.655+0000 7fee2ffff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3006a4c0 con 0x7fee300f6530 2024-02-15T18:41:40.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.655+0000 7fee2e7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee300f77d0 0x7fee30069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:40.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.655+0000 7fee2e7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee300f77d0 0x7fee30069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45032/0 (socket says 172.21.15.38:45032) 2024-02-15T18:41:40.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.655+0000 7fee2e7fc700 1 -- 172.21.15.38:0/3961584419 learned_addr learned my addr 172.21.15.38:0/3961584419 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:41:40.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.655+0000 7fee2effd700 1 --2- 172.21.15.38:0/3961584419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee300f6530 0x7fee3006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:40.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.655+0000 7fee2e7fc700 1 -- 172.21.15.38:0/3961584419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee300f6530 msgr2=0x7fee3006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:40.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.655+0000 7fee2e7fc700 1 --2- 172.21.15.38:0/3961584419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee300f6530 0x7fee3006b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.655+0000 7fee2e7fc700 1 -- 172.21.15.38:0/3961584419 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee20005040 con 0x7fee300f77d0 2024-02-15T18:41:40.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.655+0000 7fee2e7fc700 1 --2- 172.21.15.38:0/3961584419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee300f77d0 0x7fee30069e20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fee20000c00 tx=0x7fee2000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:40.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.656+0000 7fee177fe700 1 -- 172.21.15.38:0/3961584419 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee20019070 con 0x7fee300f77d0 2024-02-15T18:41:40.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.656+0000 7fee2ffff700 1 -- 172.21.15.38:0/3961584419 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3006a740 con 0x7fee300f77d0 2024-02-15T18:41:40.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.656+0000 7fee177fe700 1 -- 172.21.15.38:0/3961584419 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fee20007e80 con 0x7fee300f77d0 2024-02-15T18:41:40.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.656+0000 7fee177fe700 1 -- 172.21.15.38:0/3961584419 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee20014db0 con 0x7fee300f77d0 2024-02-15T18:41:40.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.656+0000 7fee2ffff700 1 -- 172.21.15.38:0/3961584419 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3006aca0 con 0x7fee300f77d0 2024-02-15T18:41:40.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.657+0000 7fee2ffff700 1 -- 172.21.15.38:0/3961584419 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee30045ad0 con 0x7fee300f77d0 2024-02-15T18:41:40.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.657+0000 7fee177fe700 1 -- 172.21.15.38:0/3961584419 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fee20025070 con 0x7fee300f77d0 2024-02-15T18:41:40.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.658+0000 7fee177fe700 1 --2- 172.21.15.38:0/3961584419 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fee18060f10 0x7fee180633d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:40.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.658+0000 7fee177fe700 1 -- 172.21.15.38:0/3961584419 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fee2007d430 con 0x7fee300f77d0 2024-02-15T18:41:40.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.658+0000 7fee2effd700 1 --2- 172.21.15.38:0/3961584419 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fee18060f10 0x7fee180633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:40.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.658+0000 7fee2effd700 1 --2- 172.21.15.38:0/3961584419 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fee18060f10 0x7fee180633d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fee3006b730 tx=0x7fee2400b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:40.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.662+0000 7fee177fe700 1 -- 172.21.15.38:0/3961584419 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fee2004d8e0 con 0x7fee300f77d0 2024-02-15T18:41:40.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.853+0000 7fee2ffff700 1 -- 172.21.15.38:0/3961584419 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fee30057f10 con 0x7fee18060f10 2024-02-15T18:41:40.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.864+0000 7fee177fe700 1 -- 172.21.15.38:0/3961584419 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7fee30057f10 con 0x7fee18060f10 2024-02-15T18:41:40.854 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:41:40.854 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (12m) 40s ago 14m 20.9M - 0.20.0 0881eb8f169f c2034c318e7c 2024-02-15T18:41:40.854 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (4m) 40s ago 14m 6635k - 16.2.14-521-g0e714d9a 1fbc92c859dc c2fee0824d4b 2024-02-15T18:41:40.854 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi121 smithi121 running (4m) 16s ago 13m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 05c1a5501a35 2024-02-15T18:41:40.854 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (12m) 40s ago 14m 35.6M - 6.7.4 557c83e11646 2dcccfbfbd2a 2024-02-15T18:41:40.855 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.hxoggl smithi038 running (61s) 40s ago 8m 12.3M - 16.2.14-521-g0e714d9a 1fbc92c859dc 42b83e98219f 2024-02-15T18:41:40.855 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.qgkzgn smithi038 running (46s) 40s ago 8m 11.4M - 16.2.14-521-g0e714d9a 1fbc92c859dc ecbb70ac6d7d 2024-02-15T18:41:40.855 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.mvumlh smithi121 running (29s) 16s ago 8m 12.3M - 16.2.14-521-g0e714d9a 1fbc92c859dc 6b9a272c73f9 2024-02-15T18:41:40.855 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.ovuzki smithi121 running (19s) 16s ago 8m 12.2M - 16.2.14-521-g0e714d9a 1fbc92c859dc 845253fa7f6c 2024-02-15T18:41:40.855 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.khesbl smithi038 *:8443,9283 running (5m) 40s ago 15m 438M - 16.2.14-521-g0e714d9a 1fbc92c859dc e642f3c2cfc9 2024-02-15T18:41:40.855 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi121.ckiryi smithi121 *:8443,9283 running (5m) 16s ago 13m 387M - 16.2.14-521-g0e714d9a 1fbc92c859dc 82515b025594 2024-02-15T18:41:40.855 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (5m) 40s ago 16m 53.0M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc d3e8317a30ca 2024-02-15T18:41:40.855 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi121 smithi121 running (4m) 16s ago 13m 52.1M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 42e1dda1a39d 2024-02-15T18:41:40.855 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 starting - - - - 2024-02-15T18:41:40.855 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi121 smithi121 *:9100 starting - - - - 2024-02-15T18:41:40.855 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (4m) 40s ago 12m 192M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 623ac7ecbb0a 2024-02-15T18:41:40.855 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (3m) 40s ago 12m 215M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 0f9e8e1f255a 2024-02-15T18:41:40.855 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (3m) 40s ago 11m 207M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 37171264b5be 2024-02-15T18:41:40.855 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi121 running (2m) 16s ago 11m 274M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc d505a58b375c 2024-02-15T18:41:40.856 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi121 running (2m) 16s ago 10m 272M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 0ce6b6abb99e 2024-02-15T18:41:40.856 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi121 running (103s) 16s ago 10m 194M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 79bb22243769 2024-02-15T18:41:40.856 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (5m) 40s ago 13m 66.2M - 2.18.1 de242295e225 67e33f9de4e6 2024-02-15T18:41:40.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.866+0000 7fee2ffff700 1 -- 172.21.15.38:0/3961584419 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fee18060f10 msgr2=0x7fee180633d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:40.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.866+0000 7fee2ffff700 1 --2- 172.21.15.38:0/3961584419 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fee18060f10 0x7fee180633d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fee3006b730 tx=0x7fee2400b040).stop 2024-02-15T18:41:40.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.866+0000 7fee2ffff700 1 -- 172.21.15.38:0/3961584419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee300f77d0 msgr2=0x7fee30069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:40.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.866+0000 7fee2ffff700 1 --2- 172.21.15.38:0/3961584419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee300f77d0 0x7fee30069e20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fee20000c00 tx=0x7fee2000b040).stop 2024-02-15T18:41:40.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.866+0000 7fee2ffff700 1 -- 172.21.15.38:0/3961584419 shutdown_connections 2024-02-15T18:41:40.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.866+0000 7fee2ffff700 1 --2- 172.21.15.38:0/3961584419 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fee18060f10 0x7fee180633d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.866+0000 7fee2ffff700 1 --2- 172.21.15.38:0/3961584419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee300f6530 0x7fee3006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.866+0000 7fee2ffff700 1 --2- 172.21.15.38:0/3961584419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee300f77d0 0x7fee30069e20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.866+0000 7fee2ffff700 1 -- 172.21.15.38:0/3961584419 >> 172.21.15.38:0/3961584419 conn(0x7fee300f1c90 msgr2=0x7fee300fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:40.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.867+0000 7fee2ffff700 1 -- 172.21.15.38:0/3961584419 shutdown_connections 2024-02-15T18:41:40.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.867+0000 7fee2ffff700 1 -- 172.21.15.38:0/3961584419 wait complete. 2024-02-15T18:41:40.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.981+0000 7fe5a754e700 1 -- 172.21.15.38:0/3300761930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5a00f66d0 msgr2=0x7fe5a00f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.981+0000 7fe5a754e700 1 --2- 172.21.15.38:0/3300761930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5a00f66d0 0x7fe5a00f6b30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fe590004660 tx=0x7fe590009b30).stop 2024-02-15T18:41:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.981+0000 7fe5a754e700 1 -- 172.21.15.38:0/3300761930 shutdown_connections 2024-02-15T18:41:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.981+0000 7fe5a754e700 1 --2- 172.21.15.38:0/3300761930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5a00f66d0 0x7fe5a00f6b30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.981+0000 7fe5a754e700 1 --2- 172.21.15.38:0/3300761930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a00f5430 0x7fe5a00f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.981+0000 7fe5a754e700 1 -- 172.21.15.38:0/3300761930 >> 172.21.15.38:0/3300761930 conn(0x7fe5a00f0b90 msgr2=0x7fe5a00f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.981+0000 7fe5a754e700 1 -- 172.21.15.38:0/3300761930 shutdown_connections 2024-02-15T18:41:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.981+0000 7fe5a754e700 1 -- 172.21.15.38:0/3300761930 wait complete. 2024-02-15T18:41:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.982+0000 7fe5a754e700 1 Processor -- start 2024-02-15T18:41:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.982+0000 7fe5a754e700 1 -- start start 2024-02-15T18:41:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.982+0000 7fe5a754e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5a00f5430 0x7fe5a00f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.982+0000 7fe5a754e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a00f66d0 0x7fe5a00f8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.982+0000 7fe5a754e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00f8790 con 0x7fe5a00f5430 2024-02-15T18:41:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.982+0000 7fe5a754e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00f88d0 con 0x7fe5a00f66d0 2024-02-15T18:41:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.982+0000 7fe5a654c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5a00f5430 0x7fe5a00f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.982+0000 7fe5a654c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5a00f5430 0x7fe5a00f9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45042/0 (socket says 172.21.15.38:45042) 2024-02-15T18:41:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.982+0000 7fe5a654c700 1 -- 172.21.15.38:0/154595687 learned_addr learned my addr 172.21.15.38:0/154595687 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:41:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.982+0000 7fe5a5d4b700 1 --2- 172.21.15.38:0/154595687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a00f66d0 0x7fe5a00f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.982+0000 7fe5a654c700 1 -- 172.21.15.38:0/154595687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a00f66d0 msgr2=0x7fe5a00f8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:40.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.982+0000 7fe5a654c700 1 --2- 172.21.15.38:0/154595687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a00f66d0 0x7fe5a00f8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:40.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.982+0000 7fe5a654c700 1 -- 172.21.15.38:0/154595687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe590005040 con 0x7fe5a00f5430 2024-02-15T18:41:40.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.982+0000 7fe5a5d4b700 1 --2- 172.21.15.38:0/154595687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a00f66d0 0x7fe5a00f8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:41:40.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.983+0000 7fe5a654c700 1 --2- 172.21.15.38:0/154595687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5a00f5430 0x7fe5a00f9db0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fe59c00e7c0 tx=0x7fe59c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:40.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.983+0000 7fe596ffd700 1 -- 172.21.15.38:0/154595687 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe59c0092e0 con 0x7fe5a00f5430 2024-02-15T18:41:40.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.983+0000 7fe596ffd700 1 -- 172.21.15.38:0/154595687 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fe59c00f040 con 0x7fe5a00f5430 2024-02-15T18:41:40.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.983+0000 7fe596ffd700 1 -- 172.21.15.38:0/154595687 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe59c014920 con 0x7fe5a00f5430 2024-02-15T18:41:40.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.983+0000 7fe5a754e700 1 -- 172.21.15.38:0/154595687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a00f8bb0 con 0x7fe5a00f5430 2024-02-15T18:41:40.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.983+0000 7fe5a754e700 1 -- 172.21.15.38:0/154595687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a010ee10 con 0x7fe5a00f5430 2024-02-15T18:41:40.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.984+0000 7fe5a754e700 1 -- 172.21.15.38:0/154595687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5a0045ad0 con 0x7fe5a00f5430 2024-02-15T18:41:40.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.984+0000 7fe596ffd700 1 -- 172.21.15.38:0/154595687 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fe59c01b030 con 0x7fe5a00f5430 2024-02-15T18:41:40.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.985+0000 7fe596ffd700 1 --2- 172.21.15.38:0/154595687 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe58c060f70 0x7fe58c063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:40.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.985+0000 7fe596ffd700 1 -- 172.21.15.38:0/154595687 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fe59c07be30 con 0x7fe5a00f5430 2024-02-15T18:41:40.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.985+0000 7fe5a5d4b700 1 --2- 172.21.15.38:0/154595687 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe58c060f70 0x7fe58c063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:40.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.985+0000 7fe5a5d4b700 1 --2- 172.21.15.38:0/154595687 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe58c060f70 0x7fe58c063430 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fe590005010 tx=0x7fe590016040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:40.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:40.989+0000 7fe596ffd700 1 -- 172.21.15.38:0/154595687 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe59c04c2b0 con 0x7fe5a00f5430 2024-02-15T18:41:41.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.245+0000 7fe5a754e700 1 -- 172.21.15.38:0/154595687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe5a010f5e0 con 0x7fe5a00f5430 2024-02-15T18:41:41.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.245+0000 7fe596ffd700 1 -- 172.21.15.38:0/154595687 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+648 (secure 0 0 0) 0x7fe59c0192a0 con 0x7fe5a00f5430 2024-02-15T18:41:41.235 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:41:41.235 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-02-15T18:41:41.235 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:41:41.235 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:41:41.235 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-02-15T18:41:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:41:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:41:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-02-15T18:41:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 6 2024-02-15T18:41:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:41:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-02-15T18:41:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 4 2024-02-15T18:41:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:41:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-02-15T18:41:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 14 2024-02-15T18:41:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-02-15T18:41:41.236 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:41:41.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.247+0000 7fe5a754e700 1 -- 172.21.15.38:0/154595687 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe58c060f70 msgr2=0x7fe58c063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:41.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.247+0000 7fe5a754e700 1 --2- 172.21.15.38:0/154595687 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe58c060f70 0x7fe58c063430 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fe590005010 tx=0x7fe590016040).stop 2024-02-15T18:41:41.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.247+0000 7fe5a754e700 1 -- 172.21.15.38:0/154595687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5a00f5430 msgr2=0x7fe5a00f9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:41.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.247+0000 7fe5a754e700 1 --2- 172.21.15.38:0/154595687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5a00f5430 0x7fe5a00f9db0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fe59c00e7c0 tx=0x7fe59c00b040).stop 2024-02-15T18:41:41.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.247+0000 7fe5a754e700 1 -- 172.21.15.38:0/154595687 shutdown_connections 2024-02-15T18:41:41.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.248+0000 7fe5a754e700 1 --2- 172.21.15.38:0/154595687 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe58c060f70 0x7fe58c063430 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:41.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.248+0000 7fe5a754e700 1 --2- 172.21.15.38:0/154595687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5a00f5430 0x7fe5a00f9db0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:41.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.248+0000 7fe5a754e700 1 --2- 172.21.15.38:0/154595687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a00f66d0 0x7fe5a00f8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:41.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.248+0000 7fe5a754e700 1 -- 172.21.15.38:0/154595687 >> 172.21.15.38:0/154595687 conn(0x7fe5a00f0b90 msgr2=0x7fe5a005f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:41.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.248+0000 7fe5a754e700 1 -- 172.21.15.38:0/154595687 shutdown_connections 2024-02-15T18:41:41.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.248+0000 7fe5a754e700 1 -- 172.21.15.38:0/154595687 wait complete. 2024-02-15T18:41:41.341 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:40.241389+0000 mgr.smithi038.khesbl (mgr.14664) 304 : audit [DBG] from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:41:41.342 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:40.551246+0000 mgr.smithi038.khesbl (mgr.14664) 305 : audit [DBG] from='client.44271 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:41:41.342 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:40.856067+0000 mgr.smithi038.khesbl (mgr.14664) 306 : audit [DBG] from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:41:41.342 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:41.005641+0000 mgr.smithi038.khesbl (mgr.14664) 307 : cluster [DBG] pgmap v207: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 170 B/s wr, 2 op/s 2024-02-15T18:41:41.342 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:41.021477+0000 mon.smithi038 (mon.0) 599 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:41.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.355+0000 7fa0cf526700 1 -- 172.21.15.38:0/1366035770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0c80f5e60 msgr2=0x7fa0c80f62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:41.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.355+0000 7fa0cf526700 1 --2- 172.21.15.38:0/1366035770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0c80f5e60 0x7fa0c80f62c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fa0b8004770 tx=0x7fa0b800ac40).stop 2024-02-15T18:41:41.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.355+0000 7fa0cf526700 1 -- 172.21.15.38:0/1366035770 shutdown_connections 2024-02-15T18:41:41.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.355+0000 7fa0cf526700 1 --2- 172.21.15.38:0/1366035770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0c80f5e60 0x7fa0c80f62c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:41.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.355+0000 7fa0cf526700 1 --2- 172.21.15.38:0/1366035770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0c80f4bc0 0x7fa0c80f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:41.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.355+0000 7fa0cf526700 1 -- 172.21.15.38:0/1366035770 >> 172.21.15.38:0/1366035770 conn(0x7fa0c80f0320 msgr2=0x7fa0c80f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:41.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.355+0000 7fa0cf526700 1 -- 172.21.15.38:0/1366035770 shutdown_connections 2024-02-15T18:41:41.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.355+0000 7fa0cf526700 1 -- 172.21.15.38:0/1366035770 wait complete. 2024-02-15T18:41:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.356+0000 7fa0cf526700 1 Processor -- start 2024-02-15T18:41:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.356+0000 7fa0cf526700 1 -- start start 2024-02-15T18:41:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.356+0000 7fa0cf526700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0c80f4bc0 0x7fa0c8106e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.356+0000 7fa0cf526700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0c80f5e60 0x7fa0c8107300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.356+0000 7fa0cf526700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c8107900 con 0x7fa0c80f4bc0 2024-02-15T18:41:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.356+0000 7fa0cf526700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c8108110 con 0x7fa0c80f5e60 2024-02-15T18:41:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.356+0000 7fa0ce524700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0c80f4bc0 0x7fa0c8106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.356+0000 7fa0ce524700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0c80f4bc0 0x7fa0c8106e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45066/0 (socket says 172.21.15.38:45066) 2024-02-15T18:41:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.356+0000 7fa0ce524700 1 -- 172.21.15.38:0/2835413733 learned_addr learned my addr 172.21.15.38:0/2835413733 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:41:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.357+0000 7fa0cdd23700 1 --2- 172.21.15.38:0/2835413733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0c80f5e60 0x7fa0c8107300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.357+0000 7fa0ce524700 1 -- 172.21.15.38:0/2835413733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0c80f5e60 msgr2=0x7fa0c8107300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.357+0000 7fa0ce524700 1 --2- 172.21.15.38:0/2835413733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0c80f5e60 0x7fa0c8107300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.357+0000 7fa0ce524700 1 -- 172.21.15.38:0/2835413733 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b8005040 con 0x7fa0c80f4bc0 2024-02-15T18:41:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.357+0000 7fa0ce524700 1 --2- 172.21.15.38:0/2835413733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0c80f4bc0 0x7fa0c8106e10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fa0c4002700 tx=0x7fa0c4009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.357+0000 7fa0beffd700 1 -- 172.21.15.38:0/2835413733 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0c40079c0 con 0x7fa0c80f4bc0 2024-02-15T18:41:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.357+0000 7fa0cf526700 1 -- 172.21.15.38:0/2835413733 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c8108390 con 0x7fa0c80f4bc0 2024-02-15T18:41:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.357+0000 7fa0beffd700 1 -- 172.21.15.38:0/2835413733 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fa0c400f040 con 0x7fa0c80f4bc0 2024-02-15T18:41:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.357+0000 7fa0beffd700 1 -- 172.21.15.38:0/2835413733 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0c4014990 con 0x7fa0c80f4bc0 2024-02-15T18:41:41.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.357+0000 7fa0cf526700 1 -- 172.21.15.38:0/2835413733 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c8108950 con 0x7fa0c80f4bc0 2024-02-15T18:41:41.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.358+0000 7fa0cf526700 1 -- 172.21.15.38:0/2835413733 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c8059100 con 0x7fa0c80f4bc0 2024-02-15T18:41:41.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.359+0000 7fa0beffd700 1 -- 172.21.15.38:0/2835413733 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fa0c401b060 con 0x7fa0c80f4bc0 2024-02-15T18:41:41.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.359+0000 7fa0beffd700 1 --2- 172.21.15.38:0/2835413733 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fa0b4060f70 0x7fa0b4063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:41.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.359+0000 7fa0beffd700 1 -- 172.21.15.38:0/2835413733 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fa0c407c020 con 0x7fa0c80f4bc0 2024-02-15T18:41:41.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.359+0000 7fa0cdd23700 1 --2- 172.21.15.38:0/2835413733 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fa0b4060f70 0x7fa0b4063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:41.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.360+0000 7fa0cdd23700 1 --2- 172.21.15.38:0/2835413733 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fa0b4060f70 0x7fa0b4063430 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fa0b8004f40 tx=0x7fa0b8016040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.364+0000 7fa0beffd700 1 -- 172.21.15.38:0/2835413733 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa0c404c450 con 0x7fa0c80f4bc0 2024-02-15T18:41:41.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:40.241389+0000 mgr.smithi038.khesbl (mgr.14664) 304 : audit [DBG] from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:41:41.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:40.551246+0000 mgr.smithi038.khesbl (mgr.14664) 305 : audit [DBG] from='client.44271 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:41:41.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:40.856067+0000 mgr.smithi038.khesbl (mgr.14664) 306 : audit [DBG] from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:41:41.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:41.005641+0000 mgr.smithi038.khesbl (mgr.14664) 307 : cluster [DBG] pgmap v207: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 170 B/s wr, 2 op/s 2024-02-15T18:41:41.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:41.021477+0000 mon.smithi038 (mon.0) 599 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:41.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.588+0000 7fa0cf526700 1 -- 172.21.15.38:0/2835413733 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa0c80fcd50 con 0x7fa0c80f4bc0 2024-02-15T18:41:41.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.588+0000 7fa0beffd700 1 -- 172.21.15.38:0/2835413733 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1777 (secure 0 0 0) 0x7fa0c400e1a0 con 0x7fa0c80f4bc0 2024-02-15T18:41:41.578 INFO:teuthology.orchestra.run.smithi038.stdout:e32 2024-02-15T18:41:41.578 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:41:41.578 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:41:41.578 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-02-15T18:41:41.578 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:41:41.578 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:41:41.578 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-02-15T18:41:41.578 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 32 2024-02-15T18:41:41.579 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 2024-02-15T18:41:41.579 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:41:41.579 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-02-15T18:41:26.398387+0000 2024-02-15T18:41:41.579 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-02-15T18:41:41.579 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-02-15T18:41:41.579 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-02-15T18:41:41.579 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-02-15T18:41:41.579 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-02-15T18:41:41.579 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-02-15T18:41:41.579 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-02-15T18:41:41.579 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 79 2024-02-15T18:41:41.579 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:41:41.579 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-02-15T18:41:41.580 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-02-15T18:41:41.580 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=34314,1=34318} 2024-02-15T18:41:41.580 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-02-15T18:41:41.580 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-02-15T18:41:41.580 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-02-15T18:41:41.580 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-02-15T18:41:41.580 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-02-15T18:41:41.580 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-02-15T18:41:41.580 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-02-15T18:41:41.580 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-02-15T18:41:41.580 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.hxoggl{0:34314} state up:active seq 11 join_fscid=1 addr [v2:172.21.15.38:6826/4077527450,v1:172.21.15.38:6827/4077527450] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:41:41.580 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.qgkzgn{1:34318} state up:active seq 9 join_fscid=1 addr [v2:172.21.15.38:6828/1616551408,v1:172.21.15.38:6829/1616551408] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:41:41.580 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:41:41.580 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:41:41.581 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-02-15T18:41:41.581 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:41:41.581 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.mvumlh{-1:44261} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.121:6824/180558011,v1:172.21.15.121:6825/180558011] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:41:41.581 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.ovuzki{-1:44265} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.121:6826/1075229339,v1:172.21.15.121:6827/1075229339] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:41:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.590+0000 7fa0cf526700 1 -- 172.21.15.38:0/2835413733 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fa0b4060f70 msgr2=0x7fa0b4063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.590+0000 7fa0cf526700 1 --2- 172.21.15.38:0/2835413733 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fa0b4060f70 0x7fa0b4063430 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fa0b8004f40 tx=0x7fa0b8016040).stop 2024-02-15T18:41:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.590+0000 7fa0cf526700 1 -- 172.21.15.38:0/2835413733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0c80f4bc0 msgr2=0x7fa0c8106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.590+0000 7fa0cf526700 1 --2- 172.21.15.38:0/2835413733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0c80f4bc0 0x7fa0c8106e10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fa0c4002700 tx=0x7fa0c4009e00).stop 2024-02-15T18:41:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.590+0000 7fa0cf526700 1 -- 172.21.15.38:0/2835413733 shutdown_connections 2024-02-15T18:41:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.590+0000 7fa0cf526700 1 --2- 172.21.15.38:0/2835413733 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fa0b4060f70 0x7fa0b4063430 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.590+0000 7fa0cf526700 1 --2- 172.21.15.38:0/2835413733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0c80f4bc0 0x7fa0c8106e10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.590+0000 7fa0cf526700 1 --2- 172.21.15.38:0/2835413733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0c80f5e60 0x7fa0c8107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.590+0000 7fa0cf526700 1 -- 172.21.15.38:0/2835413733 >> 172.21.15.38:0/2835413733 conn(0x7fa0c80f0320 msgr2=0x7fa0c80f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.590+0000 7fa0cf526700 1 -- 172.21.15.38:0/2835413733 shutdown_connections 2024-02-15T18:41:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.590+0000 7fa0cf526700 1 -- 172.21.15.38:0/2835413733 wait complete. 2024-02-15T18:41:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 32 2024-02-15T18:41:41.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.696+0000 7f483d91e700 1 -- 172.21.15.38:0/2724839664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48380f66e0 msgr2=0x7f48380f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:41.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.696+0000 7f483d91e700 1 --2- 172.21.15.38:0/2724839664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48380f66e0 0x7f48380f6b00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f4828004660 tx=0x7f4828009b30).stop 2024-02-15T18:41:41.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.696+0000 7f483d91e700 1 -- 172.21.15.38:0/2724839664 shutdown_connections 2024-02-15T18:41:41.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.696+0000 7f483d91e700 1 --2- 172.21.15.38:0/2724839664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48380f7980 0x7f48380f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:41.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.696+0000 7f483d91e700 1 --2- 172.21.15.38:0/2724839664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48380f66e0 0x7f48380f6b00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:41.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.696+0000 7f483d91e700 1 -- 172.21.15.38:0/2724839664 >> 172.21.15.38:0/2724839664 conn(0x7f48380f1e40 msgr2=0x7f48380f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:41.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.697+0000 7f483d91e700 1 -- 172.21.15.38:0/2724839664 shutdown_connections 2024-02-15T18:41:41.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.697+0000 7f483d91e700 1 -- 172.21.15.38:0/2724839664 wait complete. 2024-02-15T18:41:41.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.697+0000 7f483d91e700 1 Processor -- start 2024-02-15T18:41:41.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.697+0000 7f483d91e700 1 -- start start 2024-02-15T18:41:41.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.697+0000 7f483d91e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48380f66e0 0x7f483806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:41.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.697+0000 7f483d91e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48380f7980 0x7f4838069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:41.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.697+0000 7f483d91e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483806a380 con 0x7f48380f66e0 2024-02-15T18:41:41.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.697+0000 7f483d91e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483806a4c0 con 0x7f48380f7980 2024-02-15T18:41:41.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.698+0000 7f4836ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48380f66e0 0x7f483806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:41.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.698+0000 7f4836ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48380f66e0 0x7f483806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45084/0 (socket says 172.21.15.38:45084) 2024-02-15T18:41:41.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.698+0000 7f4836ffd700 1 -- 172.21.15.38:0/4155096406 learned_addr learned my addr 172.21.15.38:0/4155096406 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:41:41.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.698+0000 7f48367fc700 1 --2- 172.21.15.38:0/4155096406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48380f7980 0x7f4838069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:41.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.698+0000 7f4836ffd700 1 -- 172.21.15.38:0/4155096406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48380f7980 msgr2=0x7f4838069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:41.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.698+0000 7f4836ffd700 1 --2- 172.21.15.38:0/4155096406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48380f7980 0x7f4838069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:41.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.698+0000 7f4836ffd700 1 -- 172.21.15.38:0/4155096406 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4828005040 con 0x7f48380f66e0 2024-02-15T18:41:41.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.698+0000 7f4836ffd700 1 --2- 172.21.15.38:0/4155096406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48380f66e0 0x7f483806b9a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f4828004e30 tx=0x7f4828009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:41.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.698+0000 7f481f7fe700 1 -- 172.21.15.38:0/4155096406 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4828018070 con 0x7f48380f66e0 2024-02-15T18:41:41.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.698+0000 7f481f7fe700 1 -- 172.21.15.38:0/4155096406 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f4828007ba0 con 0x7f48380f66e0 2024-02-15T18:41:41.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.698+0000 7f483d91e700 1 -- 172.21.15.38:0/4155096406 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f483806a740 con 0x7f48380f66e0 2024-02-15T18:41:41.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.698+0000 7f481f7fe700 1 -- 172.21.15.38:0/4155096406 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f482801c3c0 con 0x7f48380f66e0 2024-02-15T18:41:41.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.698+0000 7f483d91e700 1 -- 172.21.15.38:0/4155096406 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f483806aca0 con 0x7f48380f66e0 2024-02-15T18:41:41.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.699+0000 7f483d91e700 1 -- 172.21.15.38:0/4155096406 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4838045ad0 con 0x7f48380f66e0 2024-02-15T18:41:41.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.699+0000 7f481f7fe700 1 -- 172.21.15.38:0/4155096406 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f482801d420 con 0x7f48380f66e0 2024-02-15T18:41:41.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.700+0000 7f481f7fe700 1 --2- 172.21.15.38:0/4155096406 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4820060f20 0x7f48200633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:41:41.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.700+0000 7f481f7fe700 1 -- 172.21.15.38:0/4155096406 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f482807dd60 con 0x7f48380f66e0 2024-02-15T18:41:41.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.700+0000 7f48367fc700 1 --2- 172.21.15.38:0/4155096406 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4820060f20 0x7f48200633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:41:41.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.700+0000 7f48367fc700 1 --2- 172.21.15.38:0/4155096406 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4820060f20 0x7f48200633e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f482c004e20 tx=0x7f482c00b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:41:41.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.704+0000 7f481f7fe700 1 -- 172.21.15.38:0/4155096406 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f482804e200 con 0x7f48380f66e0 2024-02-15T18:41:41.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.903+0000 7f483d91e700 1 -- 172.21.15.38:0/4155096406 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4838057f10 con 0x7f4820060f20 2024-02-15T18:41:41.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.904+0000 7f481f7fe700 1 -- 172.21.15.38:0/4155096406 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f4838057f10 con 0x7f4820060f20 2024-02-15T18:41:41.893 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:41:41.893 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:41:41.893 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-02-15T18:41:41.893 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:41:41.894 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-02-15T18:41:41.894 INFO:teuthology.orchestra.run.smithi038.stdout: "osd", 2024-02-15T18:41:41.894 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-02-15T18:41:41.894 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-02-15T18:41:41.894 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr", 2024-02-15T18:41:41.894 INFO:teuthology.orchestra.run.smithi038.stdout: "mds" 2024-02-15T18:41:41.894 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-02-15T18:41:41.894 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "16/21 daemons upgraded", 2024-02-15T18:41:41.894 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading node-exporter daemons", 2024-02-15T18:41:41.894 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-02-15T18:41:41.894 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:41:41.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.906+0000 7f483d91e700 1 -- 172.21.15.38:0/4155096406 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4820060f20 msgr2=0x7f48200633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:41.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.906+0000 7f483d91e700 1 --2- 172.21.15.38:0/4155096406 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4820060f20 0x7f48200633e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f482c004e20 tx=0x7f482c00b040).stop 2024-02-15T18:41:41.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.906+0000 7f483d91e700 1 -- 172.21.15.38:0/4155096406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48380f66e0 msgr2=0x7f483806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:41:41.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.906+0000 7f483d91e700 1 --2- 172.21.15.38:0/4155096406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48380f66e0 0x7f483806b9a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f4828004e30 tx=0x7f4828009b30).stop 2024-02-15T18:41:41.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.906+0000 7f483d91e700 1 -- 172.21.15.38:0/4155096406 shutdown_connections 2024-02-15T18:41:41.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.906+0000 7f483d91e700 1 --2- 172.21.15.38:0/4155096406 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4820060f20 0x7f48200633e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:41.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.906+0000 7f483d91e700 1 --2- 172.21.15.38:0/4155096406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48380f66e0 0x7f483806b9a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:41.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.906+0000 7f483d91e700 1 --2- 172.21.15.38:0/4155096406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48380f7980 0x7f4838069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:41:41.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.906+0000 7f483d91e700 1 -- 172.21.15.38:0/4155096406 >> 172.21.15.38:0/4155096406 conn(0x7f48380f1e40 msgr2=0x7f48380fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:41:41.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.906+0000 7f483d91e700 1 -- 172.21.15.38:0/4155096406 shutdown_connections 2024-02-15T18:41:41.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:41:41.906+0000 7f483d91e700 1 -- 172.21.15.38:0/4155096406 wait complete. 2024-02-15T18:41:42.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:41.247181+0000 mon.smithi038 (mon.0) 600 : audit [DBG] from='client.? 172.21.15.38:0/154595687' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:42.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:41.590068+0000 mon.smithi038 (mon.0) 601 : audit [DBG] from='client.? 172.21.15.38:0/2835413733' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:41:42.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:41.905231+0000 mgr.smithi038.khesbl (mgr.14664) 308 : audit [DBG] from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:41:42.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:41.247181+0000 mon.smithi038 (mon.0) 600 : audit [DBG] from='client.? 172.21.15.38:0/154595687' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:42.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:41.590068+0000 mon.smithi038 (mon.0) 601 : audit [DBG] from='client.? 172.21.15.38:0/2835413733' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:41:42.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:41.905231+0000 mgr.smithi038.khesbl (mgr.14664) 308 : audit [DBG] from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:41:43.504 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:43.006092+0000 mgr.smithi038.khesbl (mgr.14664) 309 : cluster [DBG] pgmap v208: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 511 B/s rd, 85 B/s wr, 0 op/s 2024-02-15T18:41:43.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:43.006092+0000 mgr.smithi038.khesbl (mgr.14664) 309 : cluster [DBG] pgmap v208: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 511 B/s rd, 85 B/s wr, 0 op/s 2024-02-15T18:41:45.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:43.878463+0000 mon.smithi038 (mon.0) 602 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:45.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:43.879206+0000 mon.smithi038 (mon.0) 603 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:41:45.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:43.879790+0000 mon.smithi038 (mon.0) 604 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:41:45.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:43.880289+0000 mgr.smithi038.khesbl (mgr.14664) 310 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:41:45.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:43.887728+0000 mon.smithi038 (mon.0) 605 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:45.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:43.892299+0000 mon.smithi038 (mon.0) 606 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:41:45.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:43.894322+0000 mon.smithi038 (mon.0) 607 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:43.895367+0000 mon.smithi038 (mon.0) 608 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:43.896383+0000 mon.smithi038 (mon.0) 609 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:43.897707+0000 mon.smithi038 (mon.0) 610 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:43.899250+0000 mon.smithi038 (mon.0) 611 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:43.900136+0000 mon.smithi038 (mon.0) 612 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:43.901023+0000 mon.smithi038 (mon.0) 613 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:43.901937+0000 mon.smithi038 (mon.0) 614 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:43.902781+0000 mon.smithi038 (mon.0) 615 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:43.903620+0000 mon.smithi038 (mon.0) 616 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:44 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:43.904790+0000 mon.smithi038 (mon.0) 617 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:43.878463+0000 mon.smithi038 (mon.0) 602 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:45.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:43.879206+0000 mon.smithi038 (mon.0) 603 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:41:45.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:43.879790+0000 mon.smithi038 (mon.0) 604 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:41:45.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:43.880289+0000 mgr.smithi038.khesbl (mgr.14664) 310 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:41:45.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:43.887728+0000 mon.smithi038 (mon.0) 605 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:45.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:43.892299+0000 mon.smithi038 (mon.0) 606 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:41:45.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:43.894322+0000 mon.smithi038 (mon.0) 607 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.356 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:43.895367+0000 mon.smithi038 (mon.0) 608 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:43.896383+0000 mon.smithi038 (mon.0) 609 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:43.897707+0000 mon.smithi038 (mon.0) 610 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:43.899250+0000 mon.smithi038 (mon.0) 611 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:43.900136+0000 mon.smithi038 (mon.0) 612 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:43.901023+0000 mon.smithi038 (mon.0) 613 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:43.901937+0000 mon.smithi038 (mon.0) 614 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:43.902781+0000 mon.smithi038 (mon.0) 615 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:43.903620+0000 mon.smithi038 (mon.0) 616 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:45.357 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:44 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:43.904790+0000 mon.smithi038 (mon.0) 617 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:46.260 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:45.006513+0000 mgr.smithi038.khesbl (mgr.14664) 311 : cluster [DBG] pgmap v209: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-02-15T18:41:46.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:45.006513+0000 mgr.smithi038.khesbl (mgr.14664) 311 : cluster [DBG] pgmap v209: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-02-15T18:41:47.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:45.491966+0000 mgr.smithi038.khesbl (mgr.14664) 312 : cephadm [INF] Upgrade: Updating prometheus.smithi038 2024-02-15T18:41:47.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:46 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:45.502990+0000 mgr.smithi038.khesbl (mgr.14664) 313 : cephadm [INF] Deploying daemon prometheus.smithi038 on smithi038 2024-02-15T18:41:47.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:46 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:45.491966+0000 mgr.smithi038.khesbl (mgr.14664) 312 : cephadm [INF] Upgrade: Updating prometheus.smithi038 2024-02-15T18:41:47.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:46 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:45.502990+0000 mgr.smithi038.khesbl (mgr.14664) 313 : cephadm [INF] Deploying daemon prometheus.smithi038 on smithi038 2024-02-15T18:41:48.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:47.007078+0000 mgr.smithi038.khesbl (mgr.14664) 314 : cluster [DBG] pgmap v210: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:41:48.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:47.007078+0000 mgr.smithi038.khesbl (mgr.14664) 314 : cluster [DBG] pgmap v210: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:41:49.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:49.007605+0000 mgr.smithi038.khesbl (mgr.14664) 315 : cluster [DBG] pgmap v211: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:41:49.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:49.007605+0000 mgr.smithi038.khesbl (mgr.14664) 315 : cluster [DBG] pgmap v211: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:41:51.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:51.008020+0000 mgr.smithi038.khesbl (mgr.14664) 316 : cluster [DBG] pgmap v212: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:41:51.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:51.008020+0000 mgr.smithi038.khesbl (mgr.14664) 316 : cluster [DBG] pgmap v212: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:41:53.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:52 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:51.718802+0000 mon.smithi038 (mon.0) 618 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:53.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:52 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:51.719560+0000 mon.smithi038 (mon.0) 619 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:41:53.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:51.718802+0000 mon.smithi038 (mon.0) 618 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:53.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:52 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:51.719560+0000 mon.smithi038 (mon.0) 619 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:41:54.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:53.008425+0000 mgr.smithi038.khesbl (mgr.14664) 317 : cluster [DBG] pgmap v213: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:41:54.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:53.008425+0000 mgr.smithi038.khesbl (mgr.14664) 317 : cluster [DBG] pgmap v213: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:41:55.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:55.008896+0000 mgr.smithi038.khesbl (mgr.14664) 318 : cluster [DBG] pgmap v214: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:41:55.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:55.008896+0000 mgr.smithi038.khesbl (mgr.14664) 318 : cluster [DBG] pgmap v214: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:41:58.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:56.861836+0000 mon.smithi038 (mon.0) 620 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:58.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:56.862592+0000 mon.smithi038 (mon.0) 621 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:41:58.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:56.863218+0000 mon.smithi038 (mon.0) 622 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:41:58.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:56.863734+0000 mgr.smithi038.khesbl (mgr.14664) 319 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:41:58.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:56.871120+0000 mon.smithi038 (mon.0) 623 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:58.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:56.878354+0000 mon.smithi038 (mon.0) 624 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:41:58.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:56.878721+0000 mgr.smithi038.khesbl (mgr.14664) 320 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:41:58.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:56.880090+0000 mon.smithi038 (mon.0) 625 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:41:58.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:56.882236+0000 mon.smithi038 (mon.0) 626 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:56.883347+0000 mon.smithi038 (mon.0) 627 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:56.884409+0000 mon.smithi038 (mon.0) 628 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:56.885747+0000 mon.smithi038 (mon.0) 629 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:56.887305+0000 mon.smithi038 (mon.0) 630 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:56.888178+0000 mon.smithi038 (mon.0) 631 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:56.889065+0000 mon.smithi038 (mon.0) 632 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:56.889951+0000 mon.smithi038 (mon.0) 633 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:56.890847+0000 mon.smithi038 (mon.0) 634 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:56.891764+0000 mon.smithi038 (mon.0) 635 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:56.892997+0000 mon.smithi038 (mon.0) 636 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:41:56.894083+0000 mon.smithi038 (mon.0) 637 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:57.009314+0000 mgr.smithi038.khesbl (mgr.14664) 321 : cluster [DBG] pgmap v215: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:41:58.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:56.861836+0000 mon.smithi038 (mon.0) 620 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:58.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:56.862592+0000 mon.smithi038 (mon.0) 621 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:41:58.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:56.863218+0000 mon.smithi038 (mon.0) 622 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:41:58.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:56.863734+0000 mgr.smithi038.khesbl (mgr.14664) 319 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:41:58.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:56.871120+0000 mon.smithi038 (mon.0) 623 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:41:58.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:56.878354+0000 mon.smithi038 (mon.0) 624 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:41:58.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:56.878721+0000 mgr.smithi038.khesbl (mgr.14664) 320 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:41:58.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:56.880090+0000 mon.smithi038 (mon.0) 625 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:41:58.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:56.882236+0000 mon.smithi038 (mon.0) 626 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:56.883347+0000 mon.smithi038 (mon.0) 627 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:56.884409+0000 mon.smithi038 (mon.0) 628 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:56.885747+0000 mon.smithi038 (mon.0) 629 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:56.887305+0000 mon.smithi038 (mon.0) 630 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.106 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:56.888178+0000 mon.smithi038 (mon.0) 631 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:56.889065+0000 mon.smithi038 (mon.0) 632 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:56.889951+0000 mon.smithi038 (mon.0) 633 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:56.890847+0000 mon.smithi038 (mon.0) 634 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:56.891764+0000 mon.smithi038 (mon.0) 635 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:56.892997+0000 mon.smithi038 (mon.0) 636 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:41:56.894083+0000 mon.smithi038 (mon.0) 637 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:41:58.107 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:57.009314+0000 mgr.smithi038.khesbl (mgr.14664) 321 : cluster [DBG] pgmap v215: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:41:59.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:58.653203+0000 mgr.smithi038.khesbl (mgr.14664) 322 : cephadm [INF] Upgrade: Updating alertmanager.smithi038 2024-02-15T18:41:59.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:41:58.660393+0000 mgr.smithi038.khesbl (mgr.14664) 323 : cephadm [INF] Deploying daemon alertmanager.smithi038 on smithi038 2024-02-15T18:41:59.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:41:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:41:59.009707+0000 mgr.smithi038.khesbl (mgr.14664) 324 : cluster [DBG] pgmap v216: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:41:59.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:58.653203+0000 mgr.smithi038.khesbl (mgr.14664) 322 : cephadm [INF] Upgrade: Updating alertmanager.smithi038 2024-02-15T18:41:59.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:41:58.660393+0000 mgr.smithi038.khesbl (mgr.14664) 323 : cephadm [INF] Deploying daemon alertmanager.smithi038 on smithi038 2024-02-15T18:41:59.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:41:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:41:59.009707+0000 mgr.smithi038.khesbl (mgr.14664) 324 : cluster [DBG] pgmap v216: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:01.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:01.010170+0000 mgr.smithi038.khesbl (mgr.14664) 325 : cluster [DBG] pgmap v217: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:01.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:01.010170+0000 mgr.smithi038.khesbl (mgr.14664) 325 : cluster [DBG] pgmap v217: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:03.396 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:03.010698+0000 mgr.smithi038.khesbl (mgr.14664) 326 : cluster [DBG] pgmap v218: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:03.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:03.010698+0000 mgr.smithi038.khesbl (mgr.14664) 326 : cluster [DBG] pgmap v218: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:05.331 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:05.011175+0000 mgr.smithi038.khesbl (mgr.14664) 327 : cluster [DBG] pgmap v219: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:05.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:05.011175+0000 mgr.smithi038.khesbl (mgr.14664) 327 : cluster [DBG] pgmap v219: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:07.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:07.011682+0000 mgr.smithi038.khesbl (mgr.14664) 328 : cluster [DBG] pgmap v220: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:07.451 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:07.011682+0000 mgr.smithi038.khesbl (mgr.14664) 328 : cluster [DBG] pgmap v220: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:10.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:08.842376+0000 mon.smithi038 (mon.0) 638 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:10.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:08.843136+0000 mon.smithi038 (mon.0) 639 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:42:10.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:09.012037+0000 mgr.smithi038.khesbl (mgr.14664) 329 : cluster [DBG] pgmap v221: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:10.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:08.842376+0000 mon.smithi038 (mon.0) 638 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:10.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:08.843136+0000 mon.smithi038 (mon.0) 639 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:42:10.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:09.012037+0000 mgr.smithi038.khesbl (mgr.14664) 329 : cluster [DBG] pgmap v221: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:11.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:11.012430+0000 mgr.smithi038.khesbl (mgr.14664) 330 : cluster [DBG] pgmap v222: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:11.425 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:11.012430+0000 mgr.smithi038.khesbl (mgr.14664) 330 : cluster [DBG] pgmap v222: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:12.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.027+0000 7f344286b700 1 -- 172.21.15.38:0/3993051063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f343c060310 msgr2=0x7f343c0f92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.027+0000 7f344286b700 1 --2- 172.21.15.38:0/3993051063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f343c060310 0x7f343c0f92a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f342c004660 tx=0x7f342c009b30).stop 2024-02-15T18:42:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.027+0000 7f344286b700 1 -- 172.21.15.38:0/3993051063 shutdown_connections 2024-02-15T18:42:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.027+0000 7f344286b700 1 --2- 172.21.15.38:0/3993051063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f343c0f9820 0x7f343c0fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.027+0000 7f344286b700 1 --2- 172.21.15.38:0/3993051063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f343c060310 0x7f343c0f92a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.027+0000 7f344286b700 1 -- 172.21.15.38:0/3993051063 >> 172.21.15.38:0/3993051063 conn(0x7f343c0f0d40 msgr2=0x7f343c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:42:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.027+0000 7f344286b700 1 -- 172.21.15.38:0/3993051063 shutdown_connections 2024-02-15T18:42:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.027+0000 7f344286b700 1 -- 172.21.15.38:0/3993051063 wait complete. 2024-02-15T18:42:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.028+0000 7f344286b700 1 Processor -- start 2024-02-15T18:42:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.028+0000 7f344286b700 1 -- start start 2024-02-15T18:42:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.028+0000 7f344286b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f343c060310 0x7f343c0f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.028+0000 7f344286b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f343c0f9820 0x7f343c0f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:12.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.028+0000 7f344286b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0f5700 con 0x7f343c060310 2024-02-15T18:42:12.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.028+0000 7f344286b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0f5840 con 0x7f343c0f9820 2024-02-15T18:42:12.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.028+0000 7f343bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f343c060310 0x7f343c0f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:12.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.028+0000 7f343bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f343c060310 0x7f343c0f6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34462/0 (socket says 172.21.15.38:34462) 2024-02-15T18:42:12.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.028+0000 7f343bfff700 1 -- 172.21.15.38:0/2213984792 learned_addr learned my addr 172.21.15.38:0/2213984792 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:42:12.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.028+0000 7f343b7fe700 1 --2- 172.21.15.38:0/2213984792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f343c0f9820 0x7f343c0f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:12.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.029+0000 7f343bfff700 1 -- 172.21.15.38:0/2213984792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f343c0f9820 msgr2=0x7f343c0f51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:12.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.029+0000 7f343bfff700 1 --2- 172.21.15.38:0/2213984792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f343c0f9820 0x7f343c0f51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:12.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.029+0000 7f343bfff700 1 -- 172.21.15.38:0/2213984792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c005040 con 0x7f343c060310 2024-02-15T18:42:12.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.029+0000 7f343bfff700 1 --2- 172.21.15.38:0/2213984792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f343c060310 0x7f343c0f6d20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f342c009f50 tx=0x7f342c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:42:12.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.029+0000 7f3438ff9700 1 -- 172.21.15.38:0/2213984792 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f342c018070 con 0x7f343c060310 2024-02-15T18:42:12.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.029+0000 7f344286b700 1 -- 172.21.15.38:0/2213984792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343c0f5ac0 con 0x7f343c060310 2024-02-15T18:42:12.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.029+0000 7f3438ff9700 1 -- 172.21.15.38:0/2213984792 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f342c007d70 con 0x7f343c060310 2024-02-15T18:42:12.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.029+0000 7f3438ff9700 1 -- 172.21.15.38:0/2213984792 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f342c01c620 con 0x7f343c060310 2024-02-15T18:42:12.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.029+0000 7f344286b700 1 -- 172.21.15.38:0/2213984792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343c0f6020 con 0x7f343c060310 2024-02-15T18:42:12.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.030+0000 7f344286b700 1 -- 172.21.15.38:0/2213984792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343c045ad0 con 0x7f343c060310 2024-02-15T18:42:12.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.030+0000 7f3438ff9700 1 -- 172.21.15.38:0/2213984792 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f342c01c8e0 con 0x7f343c060310 2024-02-15T18:42:12.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.031+0000 7f3438ff9700 1 --2- 172.21.15.38:0/2213984792 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3424060f20 0x7f34240633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:12.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.031+0000 7f3438ff9700 1 -- 172.21.15.38:0/2213984792 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f342c07e0c0 con 0x7f343c060310 2024-02-15T18:42:12.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.031+0000 7f343b7fe700 1 --2- 172.21.15.38:0/2213984792 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3424060f20 0x7f34240633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:12.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.031+0000 7f343b7fe700 1 --2- 172.21.15.38:0/2213984792 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3424060f20 0x7f34240633e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f343c0600e0 tx=0x7f3430009040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:42:12.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.035+0000 7f3438ff9700 1 -- 172.21.15.38:0/2213984792 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f342c04e4f0 con 0x7f343c060310 2024-02-15T18:42:12.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.224+0000 7f344286b700 1 -- 172.21.15.38:0/2213984792 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f343c0ff0c0 con 0x7f3424060f20 2024-02-15T18:42:12.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.225+0000 7f3438ff9700 1 -- 172.21.15.38:0/2213984792 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+440 (secure 0 0 0) 0x7f343c0ff0c0 con 0x7f3424060f20 2024-02-15T18:42:12.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.226+0000 7f344286b700 1 -- 172.21.15.38:0/2213984792 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3424060f20 msgr2=0x7f34240633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:12.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.227+0000 7f344286b700 1 --2- 172.21.15.38:0/2213984792 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3424060f20 0x7f34240633e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f343c0600e0 tx=0x7f3430009040).stop 2024-02-15T18:42:12.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.227+0000 7f344286b700 1 -- 172.21.15.38:0/2213984792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f343c060310 msgr2=0x7f343c0f6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:12.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.227+0000 7f344286b700 1 --2- 172.21.15.38:0/2213984792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f343c060310 0x7f343c0f6d20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f342c009f50 tx=0x7f342c009b30).stop 2024-02-15T18:42:12.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.227+0000 7f344286b700 1 -- 172.21.15.38:0/2213984792 shutdown_connections 2024-02-15T18:42:12.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.227+0000 7f344286b700 1 --2- 172.21.15.38:0/2213984792 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3424060f20 0x7f34240633e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:12.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.227+0000 7f344286b700 1 --2- 172.21.15.38:0/2213984792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f343c060310 0x7f343c0f6d20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:12.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.227+0000 7f344286b700 1 --2- 172.21.15.38:0/2213984792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f343c0f9820 0x7f343c0f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:12.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.227+0000 7f344286b700 1 -- 172.21.15.38:0/2213984792 >> 172.21.15.38:0/2213984792 conn(0x7f343c0f0d40 msgr2=0x7f343c0f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:42:12.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.227+0000 7f344286b700 1 -- 172.21.15.38:0/2213984792 shutdown_connections 2024-02-15T18:42:12.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.227+0000 7f344286b700 1 -- 172.21.15.38:0/2213984792 wait complete. 2024-02-15T18:42:12.225 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-02-15T18:42:12.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.333+0000 7fb8726d2700 1 -- 172.21.15.38:0/2340967350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb86c0f9830 msgr2=0x7fb86c0fbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:12.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.333+0000 7fb8726d2700 1 --2- 172.21.15.38:0/2340967350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb86c0f9830 0x7fb86c0fbc80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fb860004660 tx=0x7fb860009b30).stop 2024-02-15T18:42:12.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.333+0000 7fb8726d2700 1 -- 172.21.15.38:0/2340967350 shutdown_connections 2024-02-15T18:42:12.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.333+0000 7fb8726d2700 1 --2- 172.21.15.38:0/2340967350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb86c0f9830 0x7fb86c0fbc80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:12.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.333+0000 7fb8726d2700 1 --2- 172.21.15.38:0/2340967350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb86c060270 0x7fb86c0f9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:12.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.333+0000 7fb8726d2700 1 -- 172.21.15.38:0/2340967350 >> 172.21.15.38:0/2340967350 conn(0x7fb86c0f0d40 msgr2=0x7fb86c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:42:12.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.333+0000 7fb8726d2700 1 -- 172.21.15.38:0/2340967350 shutdown_connections 2024-02-15T18:42:12.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.333+0000 7fb8726d2700 1 -- 172.21.15.38:0/2340967350 wait complete. 2024-02-15T18:42:12.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.334+0000 7fb8726d2700 1 Processor -- start 2024-02-15T18:42:12.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.334+0000 7fb8726d2700 1 -- start start 2024-02-15T18:42:12.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.334+0000 7fb8726d2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb86c060270 0x7fb86c106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:12.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.334+0000 7fb8726d2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb86c0f9830 0x7fb86c1074a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:12.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.334+0000 7fb8726d2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c107a50 con 0x7fb86c060270 2024-02-15T18:42:12.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.334+0000 7fb8726d2700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c107b90 con 0x7fb86c0f9830 2024-02-15T18:42:12.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.334+0000 7fb86bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb86c060270 0x7fb86c106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:12.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.335+0000 7fb86bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb86c060270 0x7fb86c106fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34476/0 (socket says 172.21.15.38:34476) 2024-02-15T18:42:12.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.335+0000 7fb86bfff700 1 -- 172.21.15.38:0/3444379656 learned_addr learned my addr 172.21.15.38:0/3444379656 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:42:12.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.335+0000 7fb86b7fe700 1 --2- 172.21.15.38:0/3444379656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb86c0f9830 0x7fb86c1074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:12.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.335+0000 7fb86bfff700 1 -- 172.21.15.38:0/3444379656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb86c0f9830 msgr2=0x7fb86c1074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:12.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.335+0000 7fb86bfff700 1 --2- 172.21.15.38:0/3444379656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb86c0f9830 0x7fb86c1074a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:12.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.335+0000 7fb86bfff700 1 -- 172.21.15.38:0/3444379656 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb860005040 con 0x7fb86c060270 2024-02-15T18:42:12.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.335+0000 7fb86bfff700 1 --2- 172.21.15.38:0/3444379656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb86c060270 0x7fb86c106fb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fb85c00e7f0 tx=0x7fb85c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:42:12.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.335+0000 7fb868ff9700 1 -- 172.21.15.38:0/3444379656 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb85c0092e0 con 0x7fb86c060270 2024-02-15T18:42:12.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.335+0000 7fb868ff9700 1 -- 172.21.15.38:0/3444379656 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fb85c00f040 con 0x7fb86c060270 2024-02-15T18:42:12.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.335+0000 7fb8726d2700 1 -- 172.21.15.38:0/3444379656 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86c108530 con 0x7fb86c060270 2024-02-15T18:42:12.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.335+0000 7fb868ff9700 1 -- 172.21.15.38:0/3444379656 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb85c014850 con 0x7fb86c060270 2024-02-15T18:42:12.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.335+0000 7fb8726d2700 1 -- 172.21.15.38:0/3444379656 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86c108af0 con 0x7fb86c060270 2024-02-15T18:42:12.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.337+0000 7fb8726d2700 1 -- 172.21.15.38:0/3444379656 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86c045ad0 con 0x7fb86c060270 2024-02-15T18:42:12.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.337+0000 7fb868ff9700 1 -- 172.21.15.38:0/3444379656 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fb85c01b030 con 0x7fb86c060270 2024-02-15T18:42:12.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.338+0000 7fb868ff9700 1 --2- 172.21.15.38:0/3444379656 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fb854060f20 0x7fb8540633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:12.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.338+0000 7fb868ff9700 1 -- 172.21.15.38:0/3444379656 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fb85c07bfa0 con 0x7fb86c060270 2024-02-15T18:42:12.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.338+0000 7fb86b7fe700 1 --2- 172.21.15.38:0/3444379656 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fb854060f20 0x7fb8540633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:12.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.338+0000 7fb86b7fe700 1 --2- 172.21.15.38:0/3444379656 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fb854060f20 0x7fb8540633e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fb860009720 tx=0x7fb860016040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:42:12.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.343+0000 7fb868ff9700 1 -- 172.21.15.38:0/3444379656 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb85c050000 con 0x7fb86c060270 2024-02-15T18:42:12.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.535+0000 7fb8726d2700 1 -- 172.21.15.38:0/3444379656 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb86c0ff150 con 0x7fb854060f20 2024-02-15T18:42:12.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.536+0000 7fb868ff9700 1 -- 172.21.15.38:0/3444379656 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+440 (secure 0 0 0) 0x7fb86000f070 con 0x7fb854060f20 2024-02-15T18:42:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.538+0000 7fb8726d2700 1 -- 172.21.15.38:0/3444379656 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fb854060f20 msgr2=0x7fb8540633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.538+0000 7fb8726d2700 1 --2- 172.21.15.38:0/3444379656 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fb854060f20 0x7fb8540633e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fb860009720 tx=0x7fb860016040).stop 2024-02-15T18:42:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.538+0000 7fb8726d2700 1 -- 172.21.15.38:0/3444379656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb86c060270 msgr2=0x7fb86c106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.538+0000 7fb8726d2700 1 --2- 172.21.15.38:0/3444379656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb86c060270 0x7fb86c106fb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fb85c00e7f0 tx=0x7fb85c00b040).stop 2024-02-15T18:42:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.538+0000 7fb8726d2700 1 -- 172.21.15.38:0/3444379656 shutdown_connections 2024-02-15T18:42:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.538+0000 7fb8726d2700 1 --2- 172.21.15.38:0/3444379656 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fb854060f20 0x7fb8540633e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:12.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.538+0000 7fb8726d2700 1 --2- 172.21.15.38:0/3444379656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb86c060270 0x7fb86c106fb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:12.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.538+0000 7fb8726d2700 1 --2- 172.21.15.38:0/3444379656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb86c0f9830 0x7fb86c1074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:12.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.538+0000 7fb8726d2700 1 -- 172.21.15.38:0/3444379656 >> 172.21.15.38:0/3444379656 conn(0x7fb86c0f0d40 msgr2=0x7fb86c0f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:42:12.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.538+0000 7fb8726d2700 1 -- 172.21.15.38:0/3444379656 shutdown_connections 2024-02-15T18:42:12.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.538+0000 7fb8726d2700 1 -- 172.21.15.38:0/3444379656 wait complete. 2024-02-15T18:42:12.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.639+0000 7f1eab59e700 1 -- 172.21.15.38:0/599020620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1eac0f9690 msgr2=0x7f1eac0fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:12.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.639+0000 7f1eab59e700 1 --2- 172.21.15.38:0/599020620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1eac0f9690 0x7f1eac0fbae0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f1ea0004660 tx=0x7f1ea0009b30).stop 2024-02-15T18:42:12.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.639+0000 7f1eab59e700 1 -- 172.21.15.38:0/599020620 shutdown_connections 2024-02-15T18:42:12.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.639+0000 7f1eab59e700 1 --2- 172.21.15.38:0/599020620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1eac0f9690 0x7f1eac0fbae0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:12.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.639+0000 7f1eab59e700 1 --2- 172.21.15.38:0/599020620 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1eac060310 0x7f1eac0f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:12.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.639+0000 7f1eab59e700 1 -- 172.21.15.38:0/599020620 >> 172.21.15.38:0/599020620 conn(0x7f1eac0f0b90 msgr2=0x7f1eac0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:42:12.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.640+0000 7f1eab59e700 1 -- 172.21.15.38:0/599020620 shutdown_connections 2024-02-15T18:42:12.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.640+0000 7f1eab59e700 1 -- 172.21.15.38:0/599020620 wait complete. 2024-02-15T18:42:12.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.640+0000 7f1eab59e700 1 Processor -- start 2024-02-15T18:42:12.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.640+0000 7f1eab59e700 1 -- start start 2024-02-15T18:42:12.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.640+0000 7f1eab59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1eac060310 0x7f1eac108fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:12.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.640+0000 7f1eab59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1eac0f9690 0x7f1eac109490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:12.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.640+0000 7f1eab59e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac109a40 con 0x7f1eac060310 2024-02-15T18:42:12.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.640+0000 7f1eab59e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac109b80 con 0x7f1eac0f9690 2024-02-15T18:42:12.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.640+0000 7f1eaa59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1eac060310 0x7f1eac108fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:12.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.640+0000 7f1eaa59c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1eac060310 0x7f1eac108fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34482/0 (socket says 172.21.15.38:34482) 2024-02-15T18:42:12.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.640+0000 7f1eaa59c700 1 -- 172.21.15.38:0/2191540025 learned_addr learned my addr 172.21.15.38:0/2191540025 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:42:12.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.641+0000 7f1ea9d9b700 1 --2- 172.21.15.38:0/2191540025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1eac0f9690 0x7f1eac109490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:12.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.641+0000 7f1eaa59c700 1 -- 172.21.15.38:0/2191540025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1eac0f9690 msgr2=0x7f1eac109490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:12.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.641+0000 7f1eaa59c700 1 --2- 172.21.15.38:0/2191540025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1eac0f9690 0x7f1eac109490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:12.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.641+0000 7f1eaa59c700 1 -- 172.21.15.38:0/2191540025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea0005040 con 0x7f1eac060310 2024-02-15T18:42:12.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.641+0000 7f1eaa59c700 1 --2- 172.21.15.38:0/2191540025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1eac060310 0x7f1eac108fa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f1e9c00e7f0 tx=0x7f1e9c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:42:12.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.641+0000 7f1e9affd700 1 -- 172.21.15.38:0/2191540025 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e9c0092e0 con 0x7f1eac060310 2024-02-15T18:42:12.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.641+0000 7f1e9affd700 1 -- 172.21.15.38:0/2191540025 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f1e9c00f040 con 0x7f1eac060310 2024-02-15T18:42:12.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.641+0000 7f1eab59e700 1 -- 172.21.15.38:0/2191540025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eac10a520 con 0x7f1eac060310 2024-02-15T18:42:12.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.641+0000 7f1e9affd700 1 -- 172.21.15.38:0/2191540025 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e9c014850 con 0x7f1eac060310 2024-02-15T18:42:12.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.641+0000 7f1eab59e700 1 -- 172.21.15.38:0/2191540025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eac10aae0 con 0x7f1eac060310 2024-02-15T18:42:12.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.642+0000 7f1eab59e700 1 -- 172.21.15.38:0/2191540025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eac045ad0 con 0x7f1eac060310 2024-02-15T18:42:12.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.642+0000 7f1e9affd700 1 -- 172.21.15.38:0/2191540025 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f1e9c01b030 con 0x7f1eac060310 2024-02-15T18:42:12.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.643+0000 7f1e9affd700 1 --2- 172.21.15.38:0/2191540025 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f1e94060f70 0x7f1e94063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:12.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.643+0000 7f1e9affd700 1 -- 172.21.15.38:0/2191540025 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f1e9c07bee0 con 0x7f1eac060310 2024-02-15T18:42:12.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.643+0000 7f1ea9d9b700 1 --2- 172.21.15.38:0/2191540025 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f1e94060f70 0x7f1e94063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:12.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.643+0000 7f1ea9d9b700 1 --2- 172.21.15.38:0/2191540025 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f1e94060f70 0x7f1e94063430 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f1ea0009720 tx=0x7f1ea0016040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:42:12.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.647+0000 7f1e9affd700 1 -- 172.21.15.38:0/2191540025 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1e9c04fdf0 con 0x7f1eac060310 2024-02-15T18:42:12.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.830+0000 7f1eab59e700 1 -- 172.21.15.38:0/2191540025 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1eac0f5610 con 0x7f1e94060f70 2024-02-15T18:42:12.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.841+0000 7f1e9affd700 1 -- 172.21.15.38:0/2191540025 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f1eac0f5610 con 0x7f1e94060f70 2024-02-15T18:42:12.829 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:42:12.829 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 starting - - - - 2024-02-15T18:42:12.829 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (5m) 15s ago 15m 6635k - 16.2.14-521-g0e714d9a 1fbc92c859dc c2fee0824d4b 2024-02-15T18:42:12.829 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi121 smithi121 running (5m) 31s ago 13m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 05c1a5501a35 2024-02-15T18:42:12.829 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (12m) 15s ago 14m 35.6M - 6.7.4 557c83e11646 2dcccfbfbd2a 2024-02-15T18:42:12.830 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.hxoggl smithi038 running (93s) 15s ago 9m 64.5M - 16.2.14-521-g0e714d9a 1fbc92c859dc 42b83e98219f 2024-02-15T18:42:12.830 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.qgkzgn smithi038 running (78s) 15s ago 9m 13.9M - 16.2.14-521-g0e714d9a 1fbc92c859dc ecbb70ac6d7d 2024-02-15T18:42:12.830 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.mvumlh smithi121 running (61s) 31s ago 9m 12.4M - 16.2.14-521-g0e714d9a 1fbc92c859dc 6b9a272c73f9 2024-02-15T18:42:12.830 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.ovuzki smithi121 running (51s) 31s ago 9m 12.3M - 16.2.14-521-g0e714d9a 1fbc92c859dc 845253fa7f6c 2024-02-15T18:42:12.830 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.khesbl smithi038 *:8443,9283 running (6m) 15s ago 16m 440M - 16.2.14-521-g0e714d9a 1fbc92c859dc e642f3c2cfc9 2024-02-15T18:42:12.830 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi121.ckiryi smithi121 *:8443,9283 running (5m) 31s ago 13m 387M - 16.2.14-521-g0e714d9a 1fbc92c859dc 82515b025594 2024-02-15T18:42:12.830 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (5m) 15s ago 16m 57.4M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc d3e8317a30ca 2024-02-15T18:42:12.830 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi121 smithi121 running (5m) 31s ago 13m 53.7M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 42e1dda1a39d 2024-02-15T18:42:12.830 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (41s) 15s ago 14m 11.7M - 1.3.1 1dbe0e931976 e1b1d762ce83 2024-02-15T18:42:12.830 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi121 smithi121 *:9100 running (34s) 31s ago 13m 7134k - 1.3.1 1dbe0e931976 d93a49dc70f8 2024-02-15T18:42:12.830 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (4m) 15s ago 13m 196M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 623ac7ecbb0a 2024-02-15T18:42:12.830 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (4m) 15s ago 12m 221M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 0f9e8e1f255a 2024-02-15T18:42:12.830 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (3m) 15s ago 12m 212M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 37171264b5be 2024-02-15T18:42:12.830 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi121 running (3m) 31s ago 11m 275M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc d505a58b375c 2024-02-15T18:42:12.831 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi121 running (2m) 31s ago 11m 272M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 0ce6b6abb99e 2024-02-15T18:42:12.831 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi121 running (2m) 31s ago 11m 194M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 79bb22243769 2024-02-15T18:42:12.831 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (21s) 15s ago 14m 34.2M - 2.33.4 514e6a882f6e 919f6aa6ab04 2024-02-15T18:42:12.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.843+0000 7f1eab59e700 1 -- 172.21.15.38:0/2191540025 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f1e94060f70 msgr2=0x7f1e94063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:12.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.843+0000 7f1eab59e700 1 --2- 172.21.15.38:0/2191540025 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f1e94060f70 0x7f1e94063430 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f1ea0009720 tx=0x7f1ea0016040).stop 2024-02-15T18:42:12.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.843+0000 7f1eab59e700 1 -- 172.21.15.38:0/2191540025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1eac060310 msgr2=0x7f1eac108fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:12.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.843+0000 7f1eab59e700 1 --2- 172.21.15.38:0/2191540025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1eac060310 0x7f1eac108fa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f1e9c00e7f0 tx=0x7f1e9c00b040).stop 2024-02-15T18:42:12.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.843+0000 7f1eab59e700 1 -- 172.21.15.38:0/2191540025 shutdown_connections 2024-02-15T18:42:12.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.843+0000 7f1eab59e700 1 --2- 172.21.15.38:0/2191540025 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f1e94060f70 0x7f1e94063430 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:12.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.844+0000 7f1eab59e700 1 --2- 172.21.15.38:0/2191540025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1eac060310 0x7f1eac108fa0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:12.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.844+0000 7f1eab59e700 1 --2- 172.21.15.38:0/2191540025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1eac0f9690 0x7f1eac109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:12.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.844+0000 7f1eab59e700 1 -- 172.21.15.38:0/2191540025 >> 172.21.15.38:0/2191540025 conn(0x7f1eac0f0b90 msgr2=0x7f1eac0f4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:42:12.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.844+0000 7f1eab59e700 1 -- 172.21.15.38:0/2191540025 shutdown_connections 2024-02-15T18:42:12.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.844+0000 7f1eab59e700 1 -- 172.21.15.38:0/2191540025 wait complete. 2024-02-15T18:42:12.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.960+0000 7f15bb5c8700 1 -- 172.21.15.38:0/3274548081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b4060310 msgr2=0x7f15b40f92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:12.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.960+0000 7f15bb5c8700 1 --2- 172.21.15.38:0/3274548081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b4060310 0x7f15b40f92a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f15ac0099e0 tx=0x7f15ac004d10).stop 2024-02-15T18:42:12.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.960+0000 7f15bb5c8700 1 -- 172.21.15.38:0/3274548081 shutdown_connections 2024-02-15T18:42:12.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.960+0000 7f15bb5c8700 1 --2- 172.21.15.38:0/3274548081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15b40f9820 0x7f15b40fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:12.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.960+0000 7f15bb5c8700 1 --2- 172.21.15.38:0/3274548081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b4060310 0x7f15b40f92a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:12.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.960+0000 7f15bb5c8700 1 -- 172.21.15.38:0/3274548081 >> 172.21.15.38:0/3274548081 conn(0x7f15b40f0d40 msgr2=0x7f15b40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:42:12.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.960+0000 7f15bb5c8700 1 -- 172.21.15.38:0/3274548081 shutdown_connections 2024-02-15T18:42:12.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.960+0000 7f15bb5c8700 1 -- 172.21.15.38:0/3274548081 wait complete. 2024-02-15T18:42:12.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.960+0000 7f15bb5c8700 1 Processor -- start 2024-02-15T18:42:12.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.960+0000 7f15bb5c8700 1 -- start start 2024-02-15T18:42:12.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.961+0000 7f15bb5c8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b4060310 0x7f15b4107120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:12.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.961+0000 7f15bb5c8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15b40f9820 0x7f15b4107610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:12.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.961+0000 7f15bb5c8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b4107bc0 con 0x7f15b4060310 2024-02-15T18:42:12.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.961+0000 7f15bb5c8700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b4107d00 con 0x7f15b40f9820 2024-02-15T18:42:12.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.961+0000 7f15b9364700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b4060310 0x7f15b4107120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:12.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.961+0000 7f15b9364700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b4060310 0x7f15b4107120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34502/0 (socket says 172.21.15.38:34502) 2024-02-15T18:42:12.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.961+0000 7f15b9364700 1 -- 172.21.15.38:0/496239547 learned_addr learned my addr 172.21.15.38:0/496239547 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:42:12.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.961+0000 7f15b9364700 1 -- 172.21.15.38:0/496239547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15b40f9820 msgr2=0x7f15b4107610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:12.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.961+0000 7f15b9364700 1 --2- 172.21.15.38:0/496239547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15b40f9820 0x7f15b4107610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:42:12.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.961+0000 7f15b9364700 1 -- 172.21.15.38:0/496239547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac009640 con 0x7f15b4060310 2024-02-15T18:42:12.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.961+0000 7f15b9364700 1 --2- 172.21.15.38:0/496239547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b4060310 0x7f15b4107120 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f15ac004b70 tx=0x7f15ac004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:42:12.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.961+0000 7f15b1ffb700 1 -- 172.21.15.38:0/496239547 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15ac00e070 con 0x7f15b4060310 2024-02-15T18:42:12.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.961+0000 7f15b1ffb700 1 -- 172.21.15.38:0/496239547 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f15ac00bea0 con 0x7f15b4060310 2024-02-15T18:42:12.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.961+0000 7f15bb5c8700 1 -- 172.21.15.38:0/496239547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b4108640 con 0x7f15b4060310 2024-02-15T18:42:12.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.961+0000 7f15b1ffb700 1 -- 172.21.15.38:0/496239547 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15ac018070 con 0x7f15b4060310 2024-02-15T18:42:12.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.961+0000 7f15bb5c8700 1 -- 172.21.15.38:0/496239547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b4108b20 con 0x7f15b4060310 2024-02-15T18:42:12.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.962+0000 7f15bb5c8700 1 -- 172.21.15.38:0/496239547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b4045ad0 con 0x7f15b4060310 2024-02-15T18:42:12.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.963+0000 7f15b1ffb700 1 -- 172.21.15.38:0/496239547 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f15ac024060 con 0x7f15b4060310 2024-02-15T18:42:12.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.963+0000 7f15b1ffb700 1 --2- 172.21.15.38:0/496239547 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f159c060f20 0x7f159c0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:12.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.963+0000 7f15b1ffb700 1 -- 172.21.15.38:0/496239547 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f15ac07d880 con 0x7f15b4060310 2024-02-15T18:42:12.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.963+0000 7f15b8b63700 1 --2- 172.21.15.38:0/496239547 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f159c060f20 0x7f159c0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:12.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.964+0000 7f15b8b63700 1 --2- 172.21.15.38:0/496239547 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f159c060f20 0x7f159c0633e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f15b40600e0 tx=0x7f15a80099e0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:42:12.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:12.968+0000 7f15b1ffb700 1 -- 172.21.15.38:0/496239547 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f15ac051760 con 0x7f15b4060310 2024-02-15T18:42:13.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.214+0000 7f15bb5c8700 1 -- 172.21.15.38:0/496239547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f15b4027a00 con 0x7f15b4060310 2024-02-15T18:42:13.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.215+0000 7f15b1ffb700 1 -- 172.21.15.38:0/496239547 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+648 (secure 0 0 0) 0x7f15ac051350 con 0x7f15b4060310 2024-02-15T18:42:13.203 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:42:13.203 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-02-15T18:42:13.203 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:42:13.204 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:42:13.204 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-02-15T18:42:13.204 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:42:13.204 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:42:13.204 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-02-15T18:42:13.204 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 6 2024-02-15T18:42:13.204 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:42:13.204 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-02-15T18:42:13.204 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 4 2024-02-15T18:42:13.204 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:42:13.204 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-02-15T18:42:13.204 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 14 2024-02-15T18:42:13.204 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-02-15T18:42:13.205 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:42:13.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.217+0000 7f15bb5c8700 1 -- 172.21.15.38:0/496239547 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f159c060f20 msgr2=0x7f159c0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:13.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.217+0000 7f15bb5c8700 1 --2- 172.21.15.38:0/496239547 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f159c060f20 0x7f159c0633e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f15b40600e0 tx=0x7f15a80099e0).stop 2024-02-15T18:42:13.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.217+0000 7f15bb5c8700 1 -- 172.21.15.38:0/496239547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b4060310 msgr2=0x7f15b4107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:13.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.217+0000 7f15bb5c8700 1 --2- 172.21.15.38:0/496239547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b4060310 0x7f15b4107120 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f15ac004b70 tx=0x7f15ac004d10).stop 2024-02-15T18:42:13.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.217+0000 7f15bb5c8700 1 -- 172.21.15.38:0/496239547 shutdown_connections 2024-02-15T18:42:13.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.217+0000 7f15bb5c8700 1 --2- 172.21.15.38:0/496239547 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f159c060f20 0x7f159c0633e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:13.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.217+0000 7f15bb5c8700 1 --2- 172.21.15.38:0/496239547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15b4060310 0x7f15b4107120 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:13.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.217+0000 7f15bb5c8700 1 --2- 172.21.15.38:0/496239547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15b40f9820 0x7f15b4107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:42:13.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.217+0000 7f15bb5c8700 1 -- 172.21.15.38:0/496239547 >> 172.21.15.38:0/496239547 conn(0x7f15b40f0d40 msgr2=0x7f15b40f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:42:13.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.217+0000 7f15bb5c8700 1 -- 172.21.15.38:0/496239547 shutdown_connections 2024-02-15T18:42:13.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.217+0000 7f15bb5c8700 1 -- 172.21.15.38:0/496239547 wait complete. 2024-02-15T18:42:13.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.328+0000 7f08f5a2d700 1 -- 172.21.15.38:0/111033697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f00f55c0 msgr2=0x7f08f00f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:13.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.328+0000 7f08f5a2d700 1 --2- 172.21.15.38:0/111033697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f00f55c0 0x7f08f00f59e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f08e4004660 tx=0x7f08e4009b30).stop 2024-02-15T18:42:13.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.328+0000 7f08f5a2d700 1 -- 172.21.15.38:0/111033697 shutdown_connections 2024-02-15T18:42:13.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.328+0000 7f08f5a2d700 1 --2- 172.21.15.38:0/111033697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08f00f6860 0x7f08f00f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:13.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.328+0000 7f08f5a2d700 1 --2- 172.21.15.38:0/111033697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f00f55c0 0x7f08f00f59e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:13.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.328+0000 7f08f5a2d700 1 -- 172.21.15.38:0/111033697 >> 172.21.15.38:0/111033697 conn(0x7f08f00f0d40 msgr2=0x7f08f00f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:42:13.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.328+0000 7f08f5a2d700 1 -- 172.21.15.38:0/111033697 shutdown_connections 2024-02-15T18:42:13.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.328+0000 7f08f5a2d700 1 -- 172.21.15.38:0/111033697 wait complete. 2024-02-15T18:42:13.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.329+0000 7f08f5a2d700 1 Processor -- start 2024-02-15T18:42:13.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.329+0000 7f08f5a2d700 1 -- start start 2024-02-15T18:42:13.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.329+0000 7f08f5a2d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f00f55c0 0x7f08f0109d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:13.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.329+0000 7f08eeffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f00f55c0 0x7f08f0109d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:13.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.329+0000 7f08eeffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f00f55c0 0x7f08f0109d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34532/0 (socket says 172.21.15.38:34532) 2024-02-15T18:42:13.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.329+0000 7f08f5a2d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08f00f6860 0x7f08f010c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:13.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.329+0000 7f08eeffd700 1 -- 172.21.15.38:0/1231198572 learned_addr learned my addr 172.21.15.38:0/1231198572 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:42:13.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.329+0000 7f08f5a2d700 1 -- 172.21.15.38:0/1231198572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f00603e0 con 0x7f08f00f55c0 2024-02-15T18:42:13.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.329+0000 7f08f5a2d700 1 -- 172.21.15.38:0/1231198572 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f0060520 con 0x7f08f00f6860 2024-02-15T18:42:13.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.329+0000 7f08eeffd700 1 -- 172.21.15.38:0/1231198572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08f00f6860 msgr2=0x7f08f010c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:13.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.329+0000 7f08ee7fc700 1 --2- 172.21.15.38:0/1231198572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08f00f6860 0x7f08f010c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:13.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.329+0000 7f08eeffd700 1 --2- 172.21.15.38:0/1231198572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08f00f6860 0x7f08f010c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:13.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.329+0000 7f08eeffd700 1 -- 172.21.15.38:0/1231198572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e4005040 con 0x7f08f00f55c0 2024-02-15T18:42:13.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.329+0000 7f08ee7fc700 1 --2- 172.21.15.38:0/1231198572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08f00f6860 0x7f08f010c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:42:13.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.330+0000 7f08eeffd700 1 --2- 172.21.15.38:0/1231198572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f00f55c0 0x7f08f0109d00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f08e4004e30 tx=0x7f08e4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:42:13.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.330+0000 7f08d77fe700 1 -- 172.21.15.38:0/1231198572 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08e4018070 con 0x7f08f00f55c0 2024-02-15T18:42:13.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.330+0000 7f08f5a2d700 1 -- 172.21.15.38:0/1231198572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f010c750 con 0x7f08f00f55c0 2024-02-15T18:42:13.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.330+0000 7f08d77fe700 1 -- 172.21.15.38:0/1231198572 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f08e4007ba0 con 0x7f08f00f55c0 2024-02-15T18:42:13.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.330+0000 7f08d77fe700 1 -- 172.21.15.38:0/1231198572 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08e401c620 con 0x7f08f00f55c0 2024-02-15T18:42:13.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.330+0000 7f08f5a2d700 1 -- 172.21.15.38:0/1231198572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f010ccb0 con 0x7f08f00f55c0 2024-02-15T18:42:13.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.331+0000 7f08f5a2d700 1 -- 172.21.15.38:0/1231198572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f0045ad0 con 0x7f08f00f55c0 2024-02-15T18:42:13.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.331+0000 7f08d77fe700 1 -- 172.21.15.38:0/1231198572 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f08e401d9a0 con 0x7f08f00f55c0 2024-02-15T18:42:13.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.332+0000 7f08d77fe700 1 --2- 172.21.15.38:0/1231198572 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f08d8060f20 0x7f08d80633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:13.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.332+0000 7f08d77fe700 1 -- 172.21.15.38:0/1231198572 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f08e407def0 con 0x7f08f00f55c0 2024-02-15T18:42:13.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.332+0000 7f08ee7fc700 1 --2- 172.21.15.38:0/1231198572 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f08d8060f20 0x7f08d80633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:13.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.332+0000 7f08ee7fc700 1 --2- 172.21.15.38:0/1231198572 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f08d8060f20 0x7f08d80633e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f08e0009fd0 tx=0x7f08e000b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:42:13.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.336+0000 7f08d77fe700 1 -- 172.21.15.38:0/1231198572 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f08e404e320 con 0x7f08f00f55c0 2024-02-15T18:42:13.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.558+0000 7f08f5a2d700 1 -- 172.21.15.38:0/1231198572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f08f00fdfa0 con 0x7f08f00f55c0 2024-02-15T18:42:13.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.559+0000 7f08d77fe700 1 -- 172.21.15.38:0/1231198572 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1777 (secure 0 0 0) 0x7f08e4051940 con 0x7f08f00f55c0 2024-02-15T18:42:13.548 INFO:teuthology.orchestra.run.smithi038.stdout:e32 2024-02-15T18:42:13.548 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:42:13.548 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:42:13.548 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-02-15T18:42:13.548 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:42:13.548 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:42:13.548 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-02-15T18:42:13.548 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 32 2024-02-15T18:42:13.548 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 2024-02-15T18:42:13.548 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:42:13.548 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-02-15T18:41:26.398387+0000 2024-02-15T18:42:13.548 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-02-15T18:42:13.549 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-02-15T18:42:13.549 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-02-15T18:42:13.549 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-02-15T18:42:13.549 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-02-15T18:42:13.549 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-02-15T18:42:13.549 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-02-15T18:42:13.549 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 79 2024-02-15T18:42:13.549 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:42:13.549 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-02-15T18:42:13.549 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-02-15T18:42:13.549 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=34314,1=34318} 2024-02-15T18:42:13.549 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-02-15T18:42:13.549 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-02-15T18:42:13.549 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-02-15T18:42:13.550 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-02-15T18:42:13.550 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-02-15T18:42:13.550 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-02-15T18:42:13.550 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-02-15T18:42:13.550 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-02-15T18:42:13.550 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.hxoggl{0:34314} state up:active seq 11 join_fscid=1 addr [v2:172.21.15.38:6826/4077527450,v1:172.21.15.38:6827/4077527450] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:42:13.550 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.qgkzgn{1:34318} state up:active seq 9 join_fscid=1 addr [v2:172.21.15.38:6828/1616551408,v1:172.21.15.38:6829/1616551408] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:42:13.550 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:42:13.550 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:42:13.550 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-02-15T18:42:13.550 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:42:13.550 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.mvumlh{-1:44261} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.121:6824/180558011,v1:172.21.15.121:6825/180558011] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:42:13.550 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.ovuzki{-1:44265} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.121:6826/1075229339,v1:172.21.15.121:6827/1075229339] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:42:13.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.561+0000 7f08f5a2d700 1 -- 172.21.15.38:0/1231198572 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f08d8060f20 msgr2=0x7f08d80633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:13.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.561+0000 7f08f5a2d700 1 --2- 172.21.15.38:0/1231198572 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f08d8060f20 0x7f08d80633e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f08e0009fd0 tx=0x7f08e000b040).stop 2024-02-15T18:42:13.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.561+0000 7f08f5a2d700 1 -- 172.21.15.38:0/1231198572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f00f55c0 msgr2=0x7f08f0109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:13.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.561+0000 7f08f5a2d700 1 --2- 172.21.15.38:0/1231198572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f00f55c0 0x7f08f0109d00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f08e4004e30 tx=0x7f08e4009b30).stop 2024-02-15T18:42:13.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.561+0000 7f08f5a2d700 1 -- 172.21.15.38:0/1231198572 shutdown_connections 2024-02-15T18:42:13.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.561+0000 7f08f5a2d700 1 --2- 172.21.15.38:0/1231198572 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f08d8060f20 0x7f08d80633e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:13.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.561+0000 7f08f5a2d700 1 --2- 172.21.15.38:0/1231198572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08f00f55c0 0x7f08f0109d00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:13.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.561+0000 7f08f5a2d700 1 --2- 172.21.15.38:0/1231198572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08f00f6860 0x7f08f010c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:13.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.561+0000 7f08f5a2d700 1 -- 172.21.15.38:0/1231198572 >> 172.21.15.38:0/1231198572 conn(0x7f08f00f0d40 msgr2=0x7f08f00fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:42:13.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.561+0000 7f08f5a2d700 1 -- 172.21.15.38:0/1231198572 shutdown_connections 2024-02-15T18:42:13.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.561+0000 7f08f5a2d700 1 -- 172.21.15.38:0/1231198572 wait complete. 2024-02-15T18:42:13.552 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 32 2024-02-15T18:42:13.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:12.225323+0000 mgr.smithi038.khesbl (mgr.14664) 331 : audit [DBG] from='client.34370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:42:13.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:12.536472+0000 mgr.smithi038.khesbl (mgr.14664) 332 : audit [DBG] from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:42:13.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:12.831580+0000 mgr.smithi038.khesbl (mgr.14664) 333 : audit [DBG] from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:42:13.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:13.012793+0000 mgr.smithi038.khesbl (mgr.14664) 334 : cluster [DBG] pgmap v223: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:13.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:13.215478+0000 mon.smithi038 (mon.0) 640 : audit [DBG] from='client.? 172.21.15.38:0/496239547' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:13.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.664+0000 7fd36ed7c700 1 -- 172.21.15.38:0/328586220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3680f66d0 msgr2=0x7fd3680f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:13.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.664+0000 7fd36ed7c700 1 --2- 172.21.15.38:0/328586220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3680f66d0 0x7fd3680f6b30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fd358004660 tx=0x7fd358009b30).stop 2024-02-15T18:42:13.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.664+0000 7fd36ed7c700 1 -- 172.21.15.38:0/328586220 shutdown_connections 2024-02-15T18:42:13.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.664+0000 7fd36ed7c700 1 --2- 172.21.15.38:0/328586220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3680f66d0 0x7fd3680f6b30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:13.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.664+0000 7fd36ed7c700 1 --2- 172.21.15.38:0/328586220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3680f5430 0x7fd3680f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:13.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.664+0000 7fd36ed7c700 1 -- 172.21.15.38:0/328586220 >> 172.21.15.38:0/328586220 conn(0x7fd3680f0b90 msgr2=0x7fd3680f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:42:13.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.664+0000 7fd36ed7c700 1 -- 172.21.15.38:0/328586220 shutdown_connections 2024-02-15T18:42:13.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.664+0000 7fd36ed7c700 1 -- 172.21.15.38:0/328586220 wait complete. 2024-02-15T18:42:13.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.665+0000 7fd36ed7c700 1 Processor -- start 2024-02-15T18:42:13.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.665+0000 7fd36ed7c700 1 -- start start 2024-02-15T18:42:13.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.665+0000 7fd36ed7c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3680f5430 0x7fd3680f9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:13.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.665+0000 7fd36ed7c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3680f66d0 0x7fd3680f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:13.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.665+0000 7fd36ed7c700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3680f7f70 con 0x7fd3680f5430 2024-02-15T18:42:13.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.665+0000 7fd36ed7c700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3680f80b0 con 0x7fd3680f66d0 2024-02-15T18:42:13.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.665+0000 7fd36dd7a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3680f5430 0x7fd3680f9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:13.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.665+0000 7fd36dd7a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3680f5430 0x7fd3680f9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35824/0 (socket says 172.21.15.38:35824) 2024-02-15T18:42:13.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.665+0000 7fd36dd7a700 1 -- 172.21.15.38:0/2358615947 learned_addr learned my addr 172.21.15.38:0/2358615947 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:42:13.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.665+0000 7fd36d579700 1 --2- 172.21.15.38:0/2358615947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3680f66d0 0x7fd3680f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:13.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.665+0000 7fd36dd7a700 1 -- 172.21.15.38:0/2358615947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3680f66d0 msgr2=0x7fd3680f7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:13.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.665+0000 7fd36dd7a700 1 --2- 172.21.15.38:0/2358615947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3680f66d0 0x7fd3680f7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:13.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.665+0000 7fd36dd7a700 1 -- 172.21.15.38:0/2358615947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd358005040 con 0x7fd3680f5430 2024-02-15T18:42:13.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.665+0000 7fd36d579700 1 --2- 172.21.15.38:0/2358615947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3680f66d0 0x7fd3680f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:42:13.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.665+0000 7fd36dd7a700 1 --2- 172.21.15.38:0/2358615947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3680f5430 0x7fd3680f9590 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fd364002700 tx=0x7fd364009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:42:13.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.666+0000 7fd35e7fc700 1 -- 172.21.15.38:0/2358615947 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd364009320 con 0x7fd3680f5430 2024-02-15T18:42:13.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.666+0000 7fd35e7fc700 1 -- 172.21.15.38:0/2358615947 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fd36400f040 con 0x7fd3680f5430 2024-02-15T18:42:13.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.666+0000 7fd36ed7c700 1 -- 172.21.15.38:0/2358615947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3680f8390 con 0x7fd3680f5430 2024-02-15T18:42:13.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.666+0000 7fd35e7fc700 1 -- 172.21.15.38:0/2358615947 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3640149d0 con 0x7fd3680f5430 2024-02-15T18:42:13.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.666+0000 7fd36ed7c700 1 -- 172.21.15.38:0/2358615947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd36810edf0 con 0x7fd3680f5430 2024-02-15T18:42:13.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.666+0000 7fd36ed7c700 1 -- 172.21.15.38:0/2358615947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd368045ad0 con 0x7fd3680f5430 2024-02-15T18:42:13.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.667+0000 7fd35e7fc700 1 -- 172.21.15.38:0/2358615947 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fd3640077b0 con 0x7fd3680f5430 2024-02-15T18:42:13.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.667+0000 7fd35e7fc700 1 --2- 172.21.15.38:0/2358615947 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd354060f70 0x7fd354063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:13.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.667+0000 7fd35e7fc700 1 -- 172.21.15.38:0/2358615947 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fd36407c3c0 con 0x7fd3680f5430 2024-02-15T18:42:13.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.667+0000 7fd36d579700 1 --2- 172.21.15.38:0/2358615947 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd354060f70 0x7fd354063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:13.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.668+0000 7fd36d579700 1 --2- 172.21.15.38:0/2358615947 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd354060f70 0x7fd354063430 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fd358005010 tx=0x7fd358009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:42:13.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.672+0000 7fd35e7fc700 1 -- 172.21.15.38:0/2358615947 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd36404c7f0 con 0x7fd3680f5430 2024-02-15T18:42:13.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:12.225323+0000 mgr.smithi038.khesbl (mgr.14664) 331 : audit [DBG] from='client.34370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:42:13.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:12.536472+0000 mgr.smithi038.khesbl (mgr.14664) 332 : audit [DBG] from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:42:13.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:12.831580+0000 mgr.smithi038.khesbl (mgr.14664) 333 : audit [DBG] from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:42:13.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:13.012793+0000 mgr.smithi038.khesbl (mgr.14664) 334 : cluster [DBG] pgmap v223: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:13.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:13.215478+0000 mon.smithi038 (mon.0) 640 : audit [DBG] from='client.? 172.21.15.38:0/496239547' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:13.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.881+0000 7fd36ed7c700 1 -- 172.21.15.38:0/2358615947 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3680fab10 con 0x7fd354060f70 2024-02-15T18:42:13.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.882+0000 7fd35e7fc700 1 -- 172.21.15.38:0/2358615947 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+440 (secure 0 0 0) 0x7fd3680fab10 con 0x7fd354060f70 2024-02-15T18:42:13.871 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:42:13.871 INFO:teuthology.orchestra.run.smithi038.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:42:13.871 INFO:teuthology.orchestra.run.smithi038.stdout: "in_progress": true, 2024-02-15T18:42:13.871 INFO:teuthology.orchestra.run.smithi038.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:42:13.871 INFO:teuthology.orchestra.run.smithi038.stdout: "services_complete": [ 2024-02-15T18:42:13.871 INFO:teuthology.orchestra.run.smithi038.stdout: "osd", 2024-02-15T18:42:13.871 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-02-15T18:42:13.871 INFO:teuthology.orchestra.run.smithi038.stdout: "mon", 2024-02-15T18:42:13.871 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr", 2024-02-15T18:42:13.871 INFO:teuthology.orchestra.run.smithi038.stdout: "mds" 2024-02-15T18:42:13.872 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-02-15T18:42:13.872 INFO:teuthology.orchestra.run.smithi038.stdout: "progress": "16/21 daemons upgraded", 2024-02-15T18:42:13.872 INFO:teuthology.orchestra.run.smithi038.stdout: "message": "Currently upgrading alertmanager daemons", 2024-02-15T18:42:13.872 INFO:teuthology.orchestra.run.smithi038.stdout: "is_paused": false 2024-02-15T18:42:13.872 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:42:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.884+0000 7fd36ed7c700 1 -- 172.21.15.38:0/2358615947 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd354060f70 msgr2=0x7fd354063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.884+0000 7fd36ed7c700 1 --2- 172.21.15.38:0/2358615947 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd354060f70 0x7fd354063430 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fd358005010 tx=0x7fd358009b30).stop 2024-02-15T18:42:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.884+0000 7fd36ed7c700 1 -- 172.21.15.38:0/2358615947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3680f5430 msgr2=0x7fd3680f9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.884+0000 7fd36ed7c700 1 --2- 172.21.15.38:0/2358615947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3680f5430 0x7fd3680f9590 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fd364002700 tx=0x7fd364009d80).stop 2024-02-15T18:42:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.885+0000 7fd36ed7c700 1 -- 172.21.15.38:0/2358615947 shutdown_connections 2024-02-15T18:42:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.885+0000 7fd36ed7c700 1 --2- 172.21.15.38:0/2358615947 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd354060f70 0x7fd354063430 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:13.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.885+0000 7fd36ed7c700 1 --2- 172.21.15.38:0/2358615947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3680f5430 0x7fd3680f9590 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:13.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.885+0000 7fd36ed7c700 1 --2- 172.21.15.38:0/2358615947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3680f66d0 0x7fd3680f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:13.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.885+0000 7fd36ed7c700 1 -- 172.21.15.38:0/2358615947 >> 172.21.15.38:0/2358615947 conn(0x7fd3680f0b90 msgr2=0x7fd36805fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:42:13.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.885+0000 7fd36ed7c700 1 -- 172.21.15.38:0/2358615947 shutdown_connections 2024-02-15T18:42:13.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:13.885+0000 7fd36ed7c700 1 -- 172.21.15.38:0/2358615947 wait complete. 2024-02-15T18:42:14.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:13.559699+0000 mon.smithi038 (mon.0) 641 : audit [DBG] from='client.? 172.21.15.38:0/1231198572' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:42:14.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:13.882921+0000 mgr.smithi038.khesbl (mgr.14664) 335 : audit [DBG] from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:42:14.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:13.993574+0000 mon.smithi038 (mon.0) 642 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:14.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.283998+0000 mon.smithi038 (mon.0) 643 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:42:14.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.284684+0000 mon.smithi038 (mon.0) 644 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:42:14.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.290304+0000 mon.smithi038 (mon.0) 645 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:14.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:13.559699+0000 mon.smithi038 (mon.0) 641 : audit [DBG] from='client.? 172.21.15.38:0/1231198572' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:42:14.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:13.882921+0000 mgr.smithi038.khesbl (mgr.14664) 335 : audit [DBG] from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:42:14.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:13.993574+0000 mon.smithi038 (mon.0) 642 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:14.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.283998+0000 mon.smithi038 (mon.0) 643 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:42:14.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.284684+0000 mon.smithi038 (mon.0) 644 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:42:14.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.290304+0000 mon.smithi038 (mon.0) 645 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:15.426 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:42:14.285200+0000 mgr.smithi038.khesbl (mgr.14664) 336 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:42:15.426 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.296323+0000 mon.smithi038 (mon.0) 646 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-15T18:42:15.426 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.296701+0000 mgr.smithi038.khesbl (mgr.14664) 337 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-15T18:42:15.427 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.299507+0000 mon.smithi038 (mon.0) 647 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi038.front.sepia.ceph.com:9093"}]: dispatch 2024-02-15T18:42:15.427 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.299778+0000 mgr.smithi038.khesbl (mgr.14664) 338 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi038.front.sepia.ceph.com:9093"}]: dispatch 2024-02-15T18:42:15.427 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.303203+0000 mon.smithi038 (mon.0) 648 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:15.427 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.306160+0000 mon.smithi038 (mon.0) 649 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:42:15.427 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.309602+0000 mon.smithi038 (mon.0) 650 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.427 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.311347+0000 mon.smithi038 (mon.0) 651 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.427 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.312631+0000 mon.smithi038 (mon.0) 652 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.427 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.314177+0000 mon.smithi038 (mon.0) 653 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.427 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.315861+0000 mon.smithi038 (mon.0) 654 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.427 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.317110+0000 mon.smithi038 (mon.0) 655 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.427 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.318198+0000 mon.smithi038 (mon.0) 656 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.427 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.319579+0000 mon.smithi038 (mon.0) 657 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.427 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.320531+0000 mon.smithi038 (mon.0) 658 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.428 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.321521+0000 mon.smithi038 (mon.0) 659 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.428 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.322781+0000 mon.smithi038 (mon.0) 660 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.428 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.323921+0000 mon.smithi038 (mon.0) 661 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.428 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:14.325053+0000 mon.smithi038 (mon.0) 662 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.428 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:15.013196+0000 mgr.smithi038.khesbl (mgr.14664) 339 : cluster [DBG] pgmap v224: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:15.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:42:14.285200+0000 mgr.smithi038.khesbl (mgr.14664) 336 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:42:15.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.296323+0000 mon.smithi038 (mon.0) 646 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-15T18:42:15.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.296701+0000 mgr.smithi038.khesbl (mgr.14664) 337 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-15T18:42:15.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.299507+0000 mon.smithi038 (mon.0) 647 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi038.front.sepia.ceph.com:9093"}]: dispatch 2024-02-15T18:42:15.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.299778+0000 mgr.smithi038.khesbl (mgr.14664) 338 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi038.front.sepia.ceph.com:9093"}]: dispatch 2024-02-15T18:42:15.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.303203+0000 mon.smithi038 (mon.0) 648 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:15.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.306160+0000 mon.smithi038 (mon.0) 649 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:42:15.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.309602+0000 mon.smithi038 (mon.0) 650 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.311347+0000 mon.smithi038 (mon.0) 651 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.312631+0000 mon.smithi038 (mon.0) 652 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.314177+0000 mon.smithi038 (mon.0) 653 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.315861+0000 mon.smithi038 (mon.0) 654 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.317110+0000 mon.smithi038 (mon.0) 655 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.318198+0000 mon.smithi038 (mon.0) 656 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.319579+0000 mon.smithi038 (mon.0) 657 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.320531+0000 mon.smithi038 (mon.0) 658 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.606 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.321521+0000 mon.smithi038 (mon.0) 659 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.322781+0000 mon.smithi038 (mon.0) 660 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.323921+0000 mon.smithi038 (mon.0) 661 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:14.325053+0000 mon.smithi038 (mon.0) 662 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:15.607 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:15.013196+0000 mgr.smithi038.khesbl (mgr.14664) 339 : cluster [DBG] pgmap v224: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:17.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:42:15.784022+0000 mgr.smithi038.khesbl (mgr.14664) 340 : cephadm [INF] Upgrade: Updating grafana.smithi038 2024-02-15T18:42:17.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:15.955082+0000 mon.smithi038 (mon.0) 663 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:17.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:15.960010+0000 mon.smithi038 (mon.0) 664 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:17.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:15.962433+0000 mon.smithi038 (mon.0) 665 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-15T18:42:17.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:15.962789+0000 mgr.smithi038.khesbl (mgr.14664) 341 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-15T18:42:17.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:42:15.967633+0000 mgr.smithi038.khesbl (mgr.14664) 342 : cephadm [INF] Deploying daemon grafana.smithi038 on smithi038 2024-02-15T18:42:17.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:42:15.784022+0000 mgr.smithi038.khesbl (mgr.14664) 340 : cephadm [INF] Upgrade: Updating grafana.smithi038 2024-02-15T18:42:17.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:15.955082+0000 mon.smithi038 (mon.0) 663 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:17.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:15.960010+0000 mon.smithi038 (mon.0) 664 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:17.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:15.962433+0000 mon.smithi038 (mon.0) 665 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-15T18:42:17.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:15.962789+0000 mgr.smithi038.khesbl (mgr.14664) 341 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-15T18:42:17.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:42:15.967633+0000 mgr.smithi038.khesbl (mgr.14664) 342 : cephadm [INF] Deploying daemon grafana.smithi038 on smithi038 2024-02-15T18:42:18.270 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:17.013596+0000 mgr.smithi038.khesbl (mgr.14664) 343 : cluster [DBG] pgmap v225: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:18.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:17.013596+0000 mgr.smithi038.khesbl (mgr.14664) 343 : cluster [DBG] pgmap v225: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:19.270 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:19.013939+0000 mgr.smithi038.khesbl (mgr.14664) 344 : cluster [DBG] pgmap v226: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:19.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:19.013939+0000 mgr.smithi038.khesbl (mgr.14664) 344 : cluster [DBG] pgmap v226: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:21.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:21.014315+0000 mgr.smithi038.khesbl (mgr.14664) 345 : cluster [DBG] pgmap v227: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:21.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:21.014315+0000 mgr.smithi038.khesbl (mgr.14664) 345 : cluster [DBG] pgmap v227: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:23.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:23.014663+0000 mgr.smithi038.khesbl (mgr.14664) 346 : cluster [DBG] pgmap v228: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:23.770 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:23.014663+0000 mgr.smithi038.khesbl (mgr.14664) 346 : cluster [DBG] pgmap v228: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:25.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:25.015340+0000 mgr.smithi038.khesbl (mgr.14664) 347 : cluster [DBG] pgmap v229: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:25.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:25.015340+0000 mgr.smithi038.khesbl (mgr.14664) 347 : cluster [DBG] pgmap v229: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:27.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:27.015997+0000 mgr.smithi038.khesbl (mgr.14664) 348 : cluster [DBG] pgmap v230: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:27.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:27.015997+0000 mgr.smithi038.khesbl (mgr.14664) 348 : cluster [DBG] pgmap v230: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:29.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:29.016462+0000 mgr.smithi038.khesbl (mgr.14664) 349 : cluster [DBG] pgmap v231: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:29.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:29.016462+0000 mgr.smithi038.khesbl (mgr.14664) 349 : cluster [DBG] pgmap v231: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:31.855 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:30.584039+0000 mon.smithi038 (mon.0) 666 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:31.855 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:30.584796+0000 mon.smithi038 (mon.0) 667 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:42:31.855 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:31.016846+0000 mgr.smithi038.khesbl (mgr.14664) 350 : cluster [DBG] pgmap v232: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:32.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:30.584039+0000 mon.smithi038 (mon.0) 666 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:32.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:30.584796+0000 mon.smithi038 (mon.0) 667 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:42:32.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:31.016846+0000 mgr.smithi038.khesbl (mgr.14664) 350 : cluster [DBG] pgmap v232: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:33.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:33.017558+0000 mgr.smithi038.khesbl (mgr.14664) 351 : cluster [DBG] pgmap v233: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:33.770 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:33.017558+0000 mgr.smithi038.khesbl (mgr.14664) 351 : cluster [DBG] pgmap v233: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:35.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:35.018009+0000 mgr.smithi038.khesbl (mgr.14664) 352 : cluster [DBG] pgmap v234: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:35.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:35.018009+0000 mgr.smithi038.khesbl (mgr.14664) 352 : cluster [DBG] pgmap v234: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:37.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:37.018564+0000 mgr.smithi038.khesbl (mgr.14664) 353 : cluster [DBG] pgmap v235: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:37.407 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:37.018564+0000 mgr.smithi038.khesbl (mgr.14664) 353 : cluster [DBG] pgmap v235: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:39.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:39.018994+0000 mgr.smithi038.khesbl (mgr.14664) 354 : cluster [DBG] pgmap v236: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:39.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:39.018994+0000 mgr.smithi038.khesbl (mgr.14664) 354 : cluster [DBG] pgmap v236: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:41.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:41.019374+0000 mgr.smithi038.khesbl (mgr.14664) 355 : cluster [DBG] pgmap v237: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:41.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:41.019374+0000 mgr.smithi038.khesbl (mgr.14664) 355 : cluster [DBG] pgmap v237: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:42.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.562398+0000 mon.smithi038 (mon.0) 668 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:42.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.563195+0000 mon.smithi038 (mon.0) 669 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:42:42.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.563843+0000 mon.smithi038 (mon.0) 670 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:42:42.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:42:41.564381+0000 mgr.smithi038.khesbl (mgr.14664) 356 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:42:42.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.571701+0000 mon.smithi038 (mon.0) 671 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:42.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.577642+0000 mon.smithi038 (mon.0) 672 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-15T18:42:42.856 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.577970+0000 mgr.smithi038.khesbl (mgr.14664) 357 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-15T18:42:42.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.578766+0000 mon.smithi038 (mon.0) 673 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi038.front.sepia.ceph.com:3000"}]: dispatch 2024-02-15T18:42:42.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.579070+0000 mgr.smithi038.khesbl (mgr.14664) 358 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi038.front.sepia.ceph.com:3000"}]: dispatch 2024-02-15T18:42:42.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.586998+0000 mon.smithi038 (mon.0) 674 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:42.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.589886+0000 mon.smithi038 (mon.0) 675 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:42:42.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.594478+0000 mon.smithi038 (mon.0) 676 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:42.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.595878+0000 mon.smithi038 (mon.0) 677 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:42.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.597474+0000 mon.smithi038 (mon.0) 678 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:42.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.599380+0000 mon.smithi038 (mon.0) 679 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:42.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.601238+0000 mon.smithi038 (mon.0) 680 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:42.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.602248+0000 mon.smithi038 (mon.0) 681 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:42.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.603414+0000 mon.smithi038 (mon.0) 682 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:42.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.604610+0000 mon.smithi038 (mon.0) 683 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:42.857 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.605606+0000 mon.smithi038 (mon.0) 684 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:42.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.606615+0000 mon.smithi038 (mon.0) 685 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:42.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.607867+0000 mon.smithi038 (mon.0) 686 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:42.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.608982+0000 mon.smithi038 (mon.0) 687 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:42.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.610077+0000 mon.smithi038 (mon.0) 688 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:42.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.611191+0000 mon.smithi038 (mon.0) 689 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:42.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:42:41.611782+0000 mgr.smithi038.khesbl (mgr.14664) 359 : cephadm [INF] Upgrade: Finalizing container_image settings 2024-02-15T18:42:42.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.615641+0000 mon.smithi038 (mon.0) 690 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:42.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.616292+0000 mon.smithi038 (mon.0) 691 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-15T18:42:42.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.619058+0000 mon.smithi038 (mon.0) 692 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-15T18:42:42.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.619712+0000 mon.smithi038 (mon.0) 693 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:42:42.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.622656+0000 mon.smithi038 (mon.0) 694 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-15T18:42:42.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.623229+0000 mon.smithi038 (mon.0) 695 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-15T18:42:42.858 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.626091+0000 mon.smithi038 (mon.0) 696 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-02-15T18:42:42.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.626706+0000 mon.smithi038 (mon.0) 697 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-15T18:42:42.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.629447+0000 mon.smithi038 (mon.0) 698 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-02-15T18:42:42.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.629969+0000 mon.smithi038 (mon.0) 699 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-15T18:42:42.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.632854+0000 mon.smithi038 (mon.0) 700 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-02-15T18:42:42.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.633389+0000 mon.smithi038 (mon.0) 701 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-15T18:42:42.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.636113+0000 mon.smithi038 (mon.0) 702 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-15T18:42:42.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.636705+0000 mon.smithi038 (mon.0) 703 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-15T18:42:42.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.639364+0000 mon.smithi038 (mon.0) 704 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-15T18:42:42.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.639939+0000 mon.smithi038 (mon.0) 705 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:42:42.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.640468+0000 mon.smithi038 (mon.0) 706 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-15T18:42:42.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.644789+0000 mon.smithi038 (mon.0) 707 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-15T18:42:42.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.645312+0000 mon.smithi038 (mon.0) 708 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-15T18:42:42.859 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.650395+0000 mon.smithi038 (mon.0) 709 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-15T18:42:42.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.650978+0000 mon.smithi038 (mon.0) 710 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:42:42.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.651510+0000 mon.smithi038 (mon.0) 711 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:42:42.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.652016+0000 mon.smithi038 (mon.0) 712 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:42:42.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.652542+0000 mon.smithi038 (mon.0) 713 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:42:42.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.653046+0000 mon.smithi038 (mon.0) 714 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-02-15T18:42:42.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.655785+0000 mon.smithi038 (mon.0) 715 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-02-15T18:42:42.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:42:41.656206+0000 mgr.smithi038.khesbl (mgr.14664) 360 : cephadm [INF] Upgrade: Complete! 2024-02-15T18:42:42.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.656542+0000 mon.smithi038 (mon.0) 716 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-15T18:42:42.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.661404+0000 mon.smithi038 (mon.0) 717 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-15T18:42:42.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.661760+0000 mon.smithi038 (mon.0) 718 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:42:42.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.956340+0000 mon.smithi038 (mon.0) 719 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:42:42.860 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.957055+0000 mon.smithi038 (mon.0) 720 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:42:42.861 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:42:41.957664+0000 mgr.smithi038.khesbl (mgr.14664) 361 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:42:42.861 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.965552+0000 mon.smithi038 (mon.0) 721 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:42.861 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.970549+0000 mon.smithi038 (mon.0) 722 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:42:42.861 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.971872+0000 mon.smithi038 (mon.0) 723 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:42:42.861 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.972492+0000 mon.smithi038 (mon.0) 724 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:42:42.861 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:42:41.973028+0000 mgr.smithi038.khesbl (mgr.14664) 362 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:42:42.861 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:41.976394+0000 mon.smithi038 (mon.0) 725 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:43.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.562398+0000 mon.smithi038 (mon.0) 668 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:43.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.563195+0000 mon.smithi038 (mon.0) 669 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:42:43.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.563843+0000 mon.smithi038 (mon.0) 670 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:42:43.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:42:41.564381+0000 mgr.smithi038.khesbl (mgr.14664) 356 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:42:43.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.571701+0000 mon.smithi038 (mon.0) 671 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:43.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.577642+0000 mon.smithi038 (mon.0) 672 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-15T18:42:43.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.577970+0000 mgr.smithi038.khesbl (mgr.14664) 357 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-15T18:42:43.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.578766+0000 mon.smithi038 (mon.0) 673 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi038.front.sepia.ceph.com:3000"}]: dispatch 2024-02-15T18:42:43.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.579070+0000 mgr.smithi038.khesbl (mgr.14664) 358 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi038.front.sepia.ceph.com:3000"}]: dispatch 2024-02-15T18:42:43.022 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.586998+0000 mon.smithi038 (mon.0) 674 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:43.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.589886+0000 mon.smithi038 (mon.0) 675 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:42:43.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.594478+0000 mon.smithi038 (mon.0) 676 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:43.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.595878+0000 mon.smithi038 (mon.0) 677 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:43.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.597474+0000 mon.smithi038 (mon.0) 678 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:43.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.599380+0000 mon.smithi038 (mon.0) 679 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:43.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.601238+0000 mon.smithi038 (mon.0) 680 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:43.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.602248+0000 mon.smithi038 (mon.0) 681 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:43.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.603414+0000 mon.smithi038 (mon.0) 682 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:43.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.604610+0000 mon.smithi038 (mon.0) 683 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:43.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.605606+0000 mon.smithi038 (mon.0) 684 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:43.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.606615+0000 mon.smithi038 (mon.0) 685 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:43.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.607867+0000 mon.smithi038 (mon.0) 686 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:43.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.608982+0000 mon.smithi038 (mon.0) 687 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:43.023 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.610077+0000 mon.smithi038 (mon.0) 688 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:43.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.611191+0000 mon.smithi038 (mon.0) 689 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:43.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:42:41.611782+0000 mgr.smithi038.khesbl (mgr.14664) 359 : cephadm [INF] Upgrade: Finalizing container_image settings 2024-02-15T18:42:43.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.615641+0000 mon.smithi038 (mon.0) 690 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:43.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.616292+0000 mon.smithi038 (mon.0) 691 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-15T18:42:43.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.619058+0000 mon.smithi038 (mon.0) 692 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-15T18:42:43.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.619712+0000 mon.smithi038 (mon.0) 693 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:42:43.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.622656+0000 mon.smithi038 (mon.0) 694 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-15T18:42:43.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.623229+0000 mon.smithi038 (mon.0) 695 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-15T18:42:43.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.626091+0000 mon.smithi038 (mon.0) 696 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-02-15T18:42:43.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.626706+0000 mon.smithi038 (mon.0) 697 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-15T18:42:43.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.629447+0000 mon.smithi038 (mon.0) 698 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-02-15T18:42:43.024 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.629969+0000 mon.smithi038 (mon.0) 699 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-15T18:42:43.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.632854+0000 mon.smithi038 (mon.0) 700 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-02-15T18:42:43.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.633389+0000 mon.smithi038 (mon.0) 701 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-15T18:42:43.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.636113+0000 mon.smithi038 (mon.0) 702 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-15T18:42:43.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.636705+0000 mon.smithi038 (mon.0) 703 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-15T18:42:43.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.639364+0000 mon.smithi038 (mon.0) 704 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-15T18:42:43.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.639939+0000 mon.smithi038 (mon.0) 705 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:42:43.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.640468+0000 mon.smithi038 (mon.0) 706 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-15T18:42:43.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.644789+0000 mon.smithi038 (mon.0) 707 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-15T18:42:43.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.645312+0000 mon.smithi038 (mon.0) 708 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-15T18:42:43.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.650395+0000 mon.smithi038 (mon.0) 709 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-15T18:42:43.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.650978+0000 mon.smithi038 (mon.0) 710 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:42:43.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.651510+0000 mon.smithi038 (mon.0) 711 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:42:43.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.652016+0000 mon.smithi038 (mon.0) 712 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:42:43.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.652542+0000 mon.smithi038 (mon.0) 713 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:42:43.025 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.653046+0000 mon.smithi038 (mon.0) 714 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-02-15T18:42:43.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.655785+0000 mon.smithi038 (mon.0) 715 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-02-15T18:42:43.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:42:41.656206+0000 mgr.smithi038.khesbl (mgr.14664) 360 : cephadm [INF] Upgrade: Complete! 2024-02-15T18:42:43.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.656542+0000 mon.smithi038 (mon.0) 716 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-15T18:42:43.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.661404+0000 mon.smithi038 (mon.0) 717 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-15T18:42:43.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.661760+0000 mon.smithi038 (mon.0) 718 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:42:43.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.956340+0000 mon.smithi038 (mon.0) 719 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:42:43.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.957055+0000 mon.smithi038 (mon.0) 720 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:42:43.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:42:41.957664+0000 mgr.smithi038.khesbl (mgr.14664) 361 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:42:43.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.965552+0000 mon.smithi038 (mon.0) 721 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:43.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.970549+0000 mon.smithi038 (mon.0) 722 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:42:43.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.971872+0000 mon.smithi038 (mon.0) 723 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:42:43.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.972492+0000 mon.smithi038 (mon.0) 724 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:42:43.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:42:41.973028+0000 mgr.smithi038.khesbl (mgr.14664) 362 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:42:43.026 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:41.976394+0000 mon.smithi038 (mon.0) 725 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:43.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:43.019892+0000 mgr.smithi038.khesbl (mgr.14664) 363 : cluster [DBG] pgmap v238: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:43.855 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:43.397962+0000 mon.smithi038 (mon.0) 726 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:43.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.998+0000 7f8c22772700 1 -- 172.21.15.38:0/820696150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1c0f51a0 msgr2=0x7f8c1c0f7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:43.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.998+0000 7f8c22772700 1 --2- 172.21.15.38:0/820696150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1c0f51a0 0x7f8c1c0f7920 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f8c0c004660 tx=0x7f8c0c009b30).stop 2024-02-15T18:42:43.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.998+0000 7f8c22772700 1 -- 172.21.15.38:0/820696150 shutdown_connections 2024-02-15T18:42:43.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.998+0000 7f8c22772700 1 --2- 172.21.15.38:0/820696150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1c0f51a0 0x7f8c1c0f7920 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:43.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.998+0000 7f8c22772700 1 --2- 172.21.15.38:0/820696150 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c1c0f4800 0x7f8c1c0f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:43.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.998+0000 7f8c22772700 1 -- 172.21.15.38:0/820696150 >> 172.21.15.38:0/820696150 conn(0x7f8c1c0f0320 msgr2=0x7f8c1c0f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:42:43.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.998+0000 7f8c22772700 1 -- 172.21.15.38:0/820696150 shutdown_connections 2024-02-15T18:42:43.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.998+0000 7f8c22772700 1 -- 172.21.15.38:0/820696150 wait complete. 2024-02-15T18:42:43.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.998+0000 7f8c22772700 1 Processor -- start 2024-02-15T18:42:43.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.999+0000 7f8c22772700 1 -- start start 2024-02-15T18:42:43.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.999+0000 7f8c22772700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1c0f4800 0x7f8c1c106eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:43.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.999+0000 7f8c22772700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c1c0f51a0 0x7f8c1c1073a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:43.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.999+0000 7f8c22772700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c107950 con 0x7f8c1c0f4800 2024-02-15T18:42:43.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.999+0000 7f8c22772700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c108150 con 0x7f8c1c0f51a0 2024-02-15T18:42:43.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.999+0000 7f8c21770700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1c0f4800 0x7f8c1c106eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:43.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.999+0000 7f8c21770700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1c0f4800 0x7f8c1c106eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33856/0 (socket says 172.21.15.38:33856) 2024-02-15T18:42:43.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.999+0000 7f8c21770700 1 -- 172.21.15.38:0/2243101258 learned_addr learned my addr 172.21.15.38:0/2243101258 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:42:43.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.999+0000 7f8c20f6f700 1 --2- 172.21.15.38:0/2243101258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c1c0f51a0 0x7f8c1c1073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:43.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.999+0000 7f8c21770700 1 -- 172.21.15.38:0/2243101258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c1c0f51a0 msgr2=0x7f8c1c1073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:43.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.999+0000 7f8c21770700 1 --2- 172.21.15.38:0/2243101258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c1c0f51a0 0x7f8c1c1073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:43.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.999+0000 7f8c21770700 1 -- 172.21.15.38:0/2243101258 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0c005040 con 0x7f8c1c0f4800 2024-02-15T18:42:43.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:43.999+0000 7f8c21770700 1 --2- 172.21.15.38:0/2243101258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1c0f4800 0x7f8c1c106eb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f8c1800e7c0 tx=0x7f8c1800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:42:43.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.000+0000 7f8c11ffb700 1 -- 172.21.15.38:0/2243101258 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c180092e0 con 0x7f8c1c0f4800 2024-02-15T18:42:43.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.000+0000 7f8c11ffb700 1 -- 172.21.15.38:0/2243101258 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f8c1800f040 con 0x7f8c1c0f4800 2024-02-15T18:42:43.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.000+0000 7f8c11ffb700 1 -- 172.21.15.38:0/2243101258 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c18014920 con 0x7f8c1c0f4800 2024-02-15T18:42:43.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.000+0000 7f8c22772700 1 -- 172.21.15.38:0/2243101258 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1c108430 con 0x7f8c1c0f4800 2024-02-15T18:42:43.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.000+0000 7f8c22772700 1 -- 172.21.15.38:0/2243101258 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1c1089f0 con 0x7f8c1c0f4800 2024-02-15T18:42:43.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.001+0000 7f8c22772700 1 -- 172.21.15.38:0/2243101258 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1c059100 con 0x7f8c1c0f4800 2024-02-15T18:42:43.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.001+0000 7f8c11ffb700 1 -- 172.21.15.38:0/2243101258 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f8c1801b030 con 0x7f8c1c0f4800 2024-02-15T18:42:43.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.001+0000 7f8c11ffb700 1 --2- 172.21.15.38:0/2243101258 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8c08060f70 0x7f8c08063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:43.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.002+0000 7f8c11ffb700 1 -- 172.21.15.38:0/2243101258 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f8c1807bfe0 con 0x7f8c1c0f4800 2024-02-15T18:42:43.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.002+0000 7f8c20f6f700 1 --2- 172.21.15.38:0/2243101258 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8c08060f70 0x7f8c08063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:43.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.002+0000 7f8c20f6f700 1 --2- 172.21.15.38:0/2243101258 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8c08060f70 0x7f8c08063430 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f8c0c009720 tx=0x7f8c0c016040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:42:43.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.006+0000 7f8c11ffb700 1 -- 172.21.15.38:0/2243101258 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8c1804fe40 con 0x7f8c1c0f4800 2024-02-15T18:42:44.020 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:43.019892+0000 mgr.smithi038.khesbl (mgr.14664) 363 : cluster [DBG] pgmap v238: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:44.020 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:43.397962+0000 mon.smithi038 (mon.0) 726 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:42:44.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.198+0000 7f8c22772700 1 -- 172.21.15.38:0/2243101258 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c1c0008d0 con 0x7f8c08060f70 2024-02-15T18:42:44.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.199+0000 7f8c11ffb700 1 -- 172.21.15.38:0/2243101258 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f8c1c0008d0 con 0x7f8c08060f70 2024-02-15T18:42:44.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.201+0000 7f8c22772700 1 -- 172.21.15.38:0/2243101258 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8c08060f70 msgr2=0x7f8c08063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:44.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.201+0000 7f8c22772700 1 --2- 172.21.15.38:0/2243101258 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8c08060f70 0x7f8c08063430 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f8c0c009720 tx=0x7f8c0c016040).stop 2024-02-15T18:42:44.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.201+0000 7f8c22772700 1 -- 172.21.15.38:0/2243101258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1c0f4800 msgr2=0x7f8c1c106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:44.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.201+0000 7f8c22772700 1 --2- 172.21.15.38:0/2243101258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1c0f4800 0x7f8c1c106eb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f8c1800e7c0 tx=0x7f8c1800b040).stop 2024-02-15T18:42:44.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.201+0000 7f8c22772700 1 -- 172.21.15.38:0/2243101258 shutdown_connections 2024-02-15T18:42:44.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.201+0000 7f8c22772700 1 --2- 172.21.15.38:0/2243101258 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8c08060f70 0x7f8c08063430 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:44.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.201+0000 7f8c22772700 1 --2- 172.21.15.38:0/2243101258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1c0f4800 0x7f8c1c106eb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:44.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.201+0000 7f8c22772700 1 --2- 172.21.15.38:0/2243101258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c1c0f51a0 0x7f8c1c1073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:44.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.201+0000 7f8c22772700 1 -- 172.21.15.38:0/2243101258 >> 172.21.15.38:0/2243101258 conn(0x7f8c1c0f0320 msgr2=0x7f8c1c0fb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:42:44.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.201+0000 7f8c22772700 1 -- 172.21.15.38:0/2243101258 shutdown_connections 2024-02-15T18:42:44.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:44.201+0000 7f8c22772700 1 -- 172.21.15.38:0/2243101258 wait complete. 2024-02-15T18:42:44.942 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'ceph orch ps' 2024-02-15T18:42:45.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:44.200272+0000 mgr.smithi038.khesbl (mgr.14664) 364 : audit [DBG] from='client.34394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:42:45.271 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:45.020340+0000 mgr.smithi038.khesbl (mgr.14664) 365 : cluster [DBG] pgmap v239: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:45.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:44.200272+0000 mgr.smithi038.khesbl (mgr.14664) 364 : audit [DBG] from='client.34394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:42:45.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:45.020340+0000 mgr.smithi038.khesbl (mgr.14664) 365 : cluster [DBG] pgmap v239: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:47.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.164+0000 7f38972a6700 1 -- 172.21.15.38:0/2965677025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38900f7980 msgr2=0x7f38900f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:47.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.164+0000 7f38972a6700 1 --2- 172.21.15.38:0/2965677025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38900f7980 0x7f38900f7de0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f3880004660 tx=0x7f3880009b30).stop 2024-02-15T18:42:47.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.164+0000 7f38972a6700 1 -- 172.21.15.38:0/2965677025 shutdown_connections 2024-02-15T18:42:47.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.164+0000 7f38972a6700 1 --2- 172.21.15.38:0/2965677025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38900f7980 0x7f38900f7de0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:47.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.164+0000 7f38972a6700 1 --2- 172.21.15.38:0/2965677025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38900f66e0 0x7f38900f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:47.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.164+0000 7f38972a6700 1 -- 172.21.15.38:0/2965677025 >> 172.21.15.38:0/2965677025 conn(0x7f38900f1e40 msgr2=0x7f38900f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:42:47.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.164+0000 7f38972a6700 1 -- 172.21.15.38:0/2965677025 shutdown_connections 2024-02-15T18:42:47.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.164+0000 7f38972a6700 1 -- 172.21.15.38:0/2965677025 wait complete. 2024-02-15T18:42:47.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.164+0000 7f38972a6700 1 Processor -- start 2024-02-15T18:42:47.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.164+0000 7f38972a6700 1 -- start start 2024-02-15T18:42:47.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.165+0000 7f38972a6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38900f66e0 0x7f389006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:47.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.165+0000 7f38972a6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38900f7980 0x7f3890069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:47.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.165+0000 7f38972a6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389006a380 con 0x7f38900f66e0 2024-02-15T18:42:47.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.165+0000 7f38972a6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389006a4c0 con 0x7f38900f7980 2024-02-15T18:42:47.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.165+0000 7f3895042700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38900f66e0 0x7f389006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:47.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.165+0000 7f3895042700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38900f66e0 0x7f389006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33880/0 (socket says 172.21.15.38:33880) 2024-02-15T18:42:47.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.165+0000 7f3895042700 1 -- 172.21.15.38:0/1577791982 learned_addr learned my addr 172.21.15.38:0/1577791982 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:42:47.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.165+0000 7f3894841700 1 --2- 172.21.15.38:0/1577791982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38900f7980 0x7f3890069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:47.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.165+0000 7f3895042700 1 -- 172.21.15.38:0/1577791982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38900f7980 msgr2=0x7f3890069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:47.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.165+0000 7f3895042700 1 --2- 172.21.15.38:0/1577791982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38900f7980 0x7f3890069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:47.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.165+0000 7f3895042700 1 -- 172.21.15.38:0/1577791982 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3880005040 con 0x7f38900f66e0 2024-02-15T18:42:47.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.165+0000 7f3895042700 1 --2- 172.21.15.38:0/1577791982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38900f66e0 0x7f389006b9a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f388c00c530 tx=0x7f388c007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:42:47.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.165+0000 7f3885ffb700 1 -- 172.21.15.38:0/1577791982 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f388c00f650 con 0x7f38900f66e0 2024-02-15T18:42:47.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.165+0000 7f3885ffb700 1 -- 172.21.15.38:0/1577791982 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f388c00fca0 con 0x7f38900f66e0 2024-02-15T18:42:47.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.165+0000 7f3885ffb700 1 -- 172.21.15.38:0/1577791982 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f388c012e60 con 0x7f38900f66e0 2024-02-15T18:42:47.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.165+0000 7f38972a6700 1 -- 172.21.15.38:0/1577791982 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f389006a7a0 con 0x7f38900f66e0 2024-02-15T18:42:47.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.166+0000 7f38972a6700 1 -- 172.21.15.38:0/1577791982 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f389010f030 con 0x7f38900f66e0 2024-02-15T18:42:47.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.167+0000 7f38972a6700 1 -- 172.21.15.38:0/1577791982 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3890045ad0 con 0x7f38900f66e0 2024-02-15T18:42:47.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.167+0000 7f3885ffb700 1 -- 172.21.15.38:0/1577791982 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f388c019030 con 0x7f38900f66e0 2024-02-15T18:42:47.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.167+0000 7f3885ffb700 1 --2- 172.21.15.38:0/1577791982 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f387c060f20 0x7f387c0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:47.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.167+0000 7f3885ffb700 1 -- 172.21.15.38:0/1577791982 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f388c07bd60 con 0x7f38900f66e0 2024-02-15T18:42:47.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.168+0000 7f3894841700 1 --2- 172.21.15.38:0/1577791982 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f387c060f20 0x7f387c0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:47.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.168+0000 7f3894841700 1 --2- 172.21.15.38:0/1577791982 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f387c060f20 0x7f387c0633e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f3880009d60 tx=0x7f3880009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:42:47.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.172+0000 7f3885ffb700 1 -- 172.21.15.38:0/1577791982 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f388c048f60 con 0x7f38900f66e0 2024-02-15T18:42:47.270 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:47.020919+0000 mgr.smithi038.khesbl (mgr.14664) 366 : cluster [DBG] pgmap v240: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:47.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.356+0000 7f38972a6700 1 -- 172.21.15.38:0/1577791982 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f38900ff130 con 0x7f387c060f20 2024-02-15T18:42:47.354 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:47.020919+0000 mgr.smithi038.khesbl (mgr.14664) 366 : cluster [DBG] pgmap v240: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:47.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.366+0000 7f3885ffb700 1 -- 172.21.15.38:0/1577791982 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f38900ff130 con 0x7f387c060f20 2024-02-15T18:42:47.355 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:42:47.355 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (39s) 5s ago 15m 13.2M - 0.23.0 ba2b418f427c 874f0582274f 2024-02-15T18:42:47.355 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (5m) 5s ago 15m 6635k - 16.2.14-521-g0e714d9a 1fbc92c859dc c2fee0824d4b 2024-02-15T18:42:47.355 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi121 smithi121 running (5m) 66s ago 14m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 05c1a5501a35 2024-02-15T18:42:47.355 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (17s) 5s ago 15m 44.0M - 8.3.5 dad864ee21e9 86b1e2a0208e 2024-02-15T18:42:47.356 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.hxoggl smithi038 running (2m) 5s ago 9m 64.7M - 16.2.14-521-g0e714d9a 1fbc92c859dc 42b83e98219f 2024-02-15T18:42:47.356 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.qgkzgn smithi038 running (113s) 5s ago 9m 14.8M - 16.2.14-521-g0e714d9a 1fbc92c859dc ecbb70ac6d7d 2024-02-15T18:42:47.356 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.mvumlh smithi121 running (95s) 66s ago 9m 12.4M - 16.2.14-521-g0e714d9a 1fbc92c859dc 6b9a272c73f9 2024-02-15T18:42:47.356 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.ovuzki smithi121 running (85s) 66s ago 9m 12.3M - 16.2.14-521-g0e714d9a 1fbc92c859dc 845253fa7f6c 2024-02-15T18:42:47.356 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.khesbl smithi038 *:8443,9283 running (6m) 5s ago 17m 442M - 16.2.14-521-g0e714d9a 1fbc92c859dc e642f3c2cfc9 2024-02-15T18:42:47.356 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi121.ckiryi smithi121 *:8443,9283 running (6m) 66s ago 14m 387M - 16.2.14-521-g0e714d9a 1fbc92c859dc 82515b025594 2024-02-15T18:42:47.356 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (6m) 5s ago 17m 52.5M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc d3e8317a30ca 2024-02-15T18:42:47.356 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi121 smithi121 running (5m) 66s ago 14m 53.7M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 42e1dda1a39d 2024-02-15T18:42:47.356 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (75s) 5s ago 15m 14.0M - 1.3.1 1dbe0e931976 e1b1d762ce83 2024-02-15T18:42:47.356 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi121 smithi121 *:9100 running (69s) 66s ago 14m 7134k - 1.3.1 1dbe0e931976 d93a49dc70f8 2024-02-15T18:42:47.356 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (5m) 5s ago 13m 196M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 623ac7ecbb0a 2024-02-15T18:42:47.356 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (4m) 5s ago 13m 221M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 0f9e8e1f255a 2024-02-15T18:42:47.356 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (4m) 5s ago 12m 212M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 37171264b5be 2024-02-15T18:42:47.357 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi121 running (3m) 66s ago 12m 275M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc d505a58b375c 2024-02-15T18:42:47.357 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi121 running (3m) 66s ago 12m 272M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 0ce6b6abb99e 2024-02-15T18:42:47.357 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi121 running (2m) 66s ago 11m 194M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 79bb22243769 2024-02-15T18:42:47.357 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (56s) 5s ago 14m 57.5M - 2.33.4 514e6a882f6e 919f6aa6ab04 2024-02-15T18:42:47.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.368+0000 7f38972a6700 1 -- 172.21.15.38:0/1577791982 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f387c060f20 msgr2=0x7f387c0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:47.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.368+0000 7f38972a6700 1 --2- 172.21.15.38:0/1577791982 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f387c060f20 0x7f387c0633e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f3880009d60 tx=0x7f3880009b30).stop 2024-02-15T18:42:47.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.368+0000 7f38972a6700 1 -- 172.21.15.38:0/1577791982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38900f66e0 msgr2=0x7f389006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:47.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.368+0000 7f38972a6700 1 --2- 172.21.15.38:0/1577791982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38900f66e0 0x7f389006b9a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f388c00c530 tx=0x7f388c007e10).stop 2024-02-15T18:42:47.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.368+0000 7f38972a6700 1 -- 172.21.15.38:0/1577791982 shutdown_connections 2024-02-15T18:42:47.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.368+0000 7f38972a6700 1 --2- 172.21.15.38:0/1577791982 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f387c060f20 0x7f387c0633e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:47.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.368+0000 7f38972a6700 1 --2- 172.21.15.38:0/1577791982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38900f66e0 0x7f389006b9a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:47.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.368+0000 7f38972a6700 1 --2- 172.21.15.38:0/1577791982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38900f7980 0x7f3890069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:47.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.368+0000 7f38972a6700 1 -- 172.21.15.38:0/1577791982 >> 172.21.15.38:0/1577791982 conn(0x7f38900f1e40 msgr2=0x7f38900fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:42:47.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.368+0000 7f38972a6700 1 -- 172.21.15.38:0/1577791982 shutdown_connections 2024-02-15T18:42:47.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:47.368+0000 7f38972a6700 1 -- 172.21.15.38:0/1577791982 wait complete. 2024-02-15T18:42:48.169 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'ceph versions' 2024-02-15T18:42:48.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:48 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:47.358482+0000 mgr.smithi038.khesbl (mgr.14664) 367 : audit [DBG] from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:42:48.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:48 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:47.358482+0000 mgr.smithi038.khesbl (mgr.14664) 367 : audit [DBG] from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:42:49.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:49.021512+0000 mgr.smithi038.khesbl (mgr.14664) 368 : cluster [DBG] pgmap v241: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:49.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:49.021512+0000 mgr.smithi038.khesbl (mgr.14664) 368 : cluster [DBG] pgmap v241: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:50.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.528+0000 7fd266895700 1 -- 172.21.15.38:0/1782879832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2600602d0 msgr2=0x7fd2600f92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:50.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.528+0000 7fd266895700 1 --2- 172.21.15.38:0/1782879832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2600602d0 0x7fd2600f92e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fd250004660 tx=0x7fd250009b30).stop 2024-02-15T18:42:50.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.528+0000 7fd266895700 1 -- 172.21.15.38:0/1782879832 shutdown_connections 2024-02-15T18:42:50.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.528+0000 7fd266895700 1 --2- 172.21.15.38:0/1782879832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2600f9860 0x7fd2600fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:50.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.528+0000 7fd266895700 1 --2- 172.21.15.38:0/1782879832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2600602d0 0x7fd2600f92e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:50.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.528+0000 7fd266895700 1 -- 172.21.15.38:0/1782879832 >> 172.21.15.38:0/1782879832 conn(0x7fd2600f0d40 msgr2=0x7fd2600f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:42:50.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.528+0000 7fd266895700 1 -- 172.21.15.38:0/1782879832 shutdown_connections 2024-02-15T18:42:50.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.528+0000 7fd266895700 1 -- 172.21.15.38:0/1782879832 wait complete. 2024-02-15T18:42:50.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.529+0000 7fd266895700 1 Processor -- start 2024-02-15T18:42:50.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.529+0000 7fd266895700 1 -- start start 2024-02-15T18:42:50.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.529+0000 7fd266895700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2600602d0 0x7fd260106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:50.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.529+0000 7fd266895700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2600f9860 0x7fd2601074a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:50.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.529+0000 7fd266895700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd260107a50 con 0x7fd2600602d0 2024-02-15T18:42:50.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.529+0000 7fd266895700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd260107b90 con 0x7fd2600f9860 2024-02-15T18:42:50.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.529+0000 7fd25ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2600602d0 0x7fd260106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:50.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.529+0000 7fd25ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2600602d0 0x7fd260106fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33900/0 (socket says 172.21.15.38:33900) 2024-02-15T18:42:50.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.529+0000 7fd25ffff700 1 -- 172.21.15.38:0/2823752460 learned_addr learned my addr 172.21.15.38:0/2823752460 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:42:50.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.529+0000 7fd25f7fe700 1 --2- 172.21.15.38:0/2823752460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2600f9860 0x7fd2601074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:50.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.530+0000 7fd25ffff700 1 -- 172.21.15.38:0/2823752460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2600f9860 msgr2=0x7fd2601074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:50.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.530+0000 7fd25ffff700 1 --2- 172.21.15.38:0/2823752460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2600f9860 0x7fd2601074a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:50.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.530+0000 7fd25ffff700 1 -- 172.21.15.38:0/2823752460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd250005040 con 0x7fd2600602d0 2024-02-15T18:42:50.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.530+0000 7fd25ffff700 1 --2- 172.21.15.38:0/2823752460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2600602d0 0x7fd260106fb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fd250009f50 tx=0x7fd250009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:42:50.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.530+0000 7fd25cff9700 1 -- 172.21.15.38:0/2823752460 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd250018070 con 0x7fd2600602d0 2024-02-15T18:42:50.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.530+0000 7fd25cff9700 1 -- 172.21.15.38:0/2823752460 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fd250007ba0 con 0x7fd2600602d0 2024-02-15T18:42:50.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.530+0000 7fd266895700 1 -- 172.21.15.38:0/2823752460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2601084d0 con 0x7fd2600602d0 2024-02-15T18:42:50.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.530+0000 7fd25cff9700 1 -- 172.21.15.38:0/2823752460 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd25001c420 con 0x7fd2600602d0 2024-02-15T18:42:50.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.530+0000 7fd266895700 1 -- 172.21.15.38:0/2823752460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd260108a30 con 0x7fd2600602d0 2024-02-15T18:42:50.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.531+0000 7fd266895700 1 -- 172.21.15.38:0/2823752460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd260045ad0 con 0x7fd2600602d0 2024-02-15T18:42:50.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.531+0000 7fd25cff9700 1 -- 172.21.15.38:0/2823752460 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fd25001d420 con 0x7fd2600602d0 2024-02-15T18:42:50.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.532+0000 7fd25cff9700 1 --2- 172.21.15.38:0/2823752460 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd248060f20 0x7fd2480633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:42:50.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.532+0000 7fd25cff9700 1 -- 172.21.15.38:0/2823752460 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fd25007de10 con 0x7fd2600602d0 2024-02-15T18:42:50.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.532+0000 7fd25f7fe700 1 --2- 172.21.15.38:0/2823752460 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd248060f20 0x7fd2480633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:42:50.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.532+0000 7fd25f7fe700 1 --2- 172.21.15.38:0/2823752460 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd248060f20 0x7fd2480633e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fd2600600a0 tx=0x7fd25400b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:42:50.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.536+0000 7fd25cff9700 1 -- 172.21.15.38:0/2823752460 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd250051c10 con 0x7fd2600602d0 2024-02-15T18:42:50.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.781+0000 7fd266895700 1 -- 172.21.15.38:0/2823752460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd2600ff0c0 con 0x7fd2600602d0 2024-02-15T18:42:50.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.782+0000 7fd25cff9700 1 -- 172.21.15.38:0/2823752460 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+648 (secure 0 0 0) 0x7fd250051800 con 0x7fd2600602d0 2024-02-15T18:42:50.771 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:42:50.771 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-02-15T18:42:50.771 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:42:50.772 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:42:50.772 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-02-15T18:42:50.772 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:42:50.772 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:42:50.772 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-02-15T18:42:50.772 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 6 2024-02-15T18:42:50.772 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:42:50.772 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-02-15T18:42:50.772 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 4 2024-02-15T18:42:50.772 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:42:50.772 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-02-15T18:42:50.772 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 14 2024-02-15T18:42:50.773 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-02-15T18:42:50.773 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:42:50.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.784+0000 7fd266895700 1 -- 172.21.15.38:0/2823752460 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd248060f20 msgr2=0x7fd2480633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:50.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.784+0000 7fd266895700 1 --2- 172.21.15.38:0/2823752460 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd248060f20 0x7fd2480633e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fd2600600a0 tx=0x7fd25400b040).stop 2024-02-15T18:42:50.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.784+0000 7fd266895700 1 -- 172.21.15.38:0/2823752460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2600602d0 msgr2=0x7fd260106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:42:50.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.784+0000 7fd266895700 1 --2- 172.21.15.38:0/2823752460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2600602d0 0x7fd260106fb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fd250009f50 tx=0x7fd250009b30).stop 2024-02-15T18:42:50.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.784+0000 7fd266895700 1 -- 172.21.15.38:0/2823752460 shutdown_connections 2024-02-15T18:42:50.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.784+0000 7fd266895700 1 --2- 172.21.15.38:0/2823752460 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd248060f20 0x7fd2480633e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:50.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.784+0000 7fd266895700 1 --2- 172.21.15.38:0/2823752460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2600602d0 0x7fd260106fb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:50.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.784+0000 7fd266895700 1 --2- 172.21.15.38:0/2823752460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd2600f9860 0x7fd2601074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:42:50.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.784+0000 7fd266895700 1 -- 172.21.15.38:0/2823752460 >> 172.21.15.38:0/2823752460 conn(0x7fd2600f0d40 msgr2=0x7fd2600f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:42:50.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.784+0000 7fd266895700 1 -- 172.21.15.38:0/2823752460 shutdown_connections 2024-02-15T18:42:50.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:42:50.784+0000 7fd266895700 1 -- 172.21.15.38:0/2823752460 wait complete. 2024-02-15T18:42:51.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:50 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:42:50.783558+0000 mon.smithi038 (mon.0) 727 : audit [DBG] from='client.? 172.21.15.38:0/2823752460' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:51.220 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:50 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:42:50.783558+0000 mon.smithi038 (mon.0) 727 : audit [DBG] from='client.? 172.21.15.38:0/2823752460' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:42:51.643 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-02-15T18:42:52.021 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:51.022148+0000 mgr.smithi038.khesbl (mgr.14664) 369 : cluster [DBG] pgmap v242: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:52.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:51.022148+0000 mgr.smithi038.khesbl (mgr.14664) 369 : cluster [DBG] pgmap v242: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:53.558 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:53.022591+0000 mgr.smithi038.khesbl (mgr.14664) 370 : cluster [DBG] pgmap v243: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:53.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:53.022591+0000 mgr.smithi038.khesbl (mgr.14664) 370 : cluster [DBG] pgmap v243: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:53.815 INFO:teuthology.orchestra.run.smithi038.stdout:wait for servicemap items w/ changing names to refresh 2024-02-15T18:42:54.633 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'sleep 60' 2024-02-15T18:42:55.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:55.023054+0000 mgr.smithi038.khesbl (mgr.14664) 371 : cluster [DBG] pgmap v244: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:55.418 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:55.023054+0000 mgr.smithi038.khesbl (mgr.14664) 371 : cluster [DBG] pgmap v244: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:57.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:57.023592+0000 mgr.smithi038.khesbl (mgr.14664) 372 : cluster [DBG] pgmap v245: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:57.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:57 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:57.023592+0000 mgr.smithi038.khesbl (mgr.14664) 372 : cluster [DBG] pgmap v245: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:59.270 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:42:59 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:42:59.024320+0000 mgr.smithi038.khesbl (mgr.14664) 373 : cluster [DBG] pgmap v246: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:42:59.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:42:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:42:59.024320+0000 mgr.smithi038.khesbl (mgr.14664) 373 : cluster [DBG] pgmap v246: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:01.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:01.024920+0000 mgr.smithi038.khesbl (mgr.14664) 374 : cluster [DBG] pgmap v247: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:01.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:01.024920+0000 mgr.smithi038.khesbl (mgr.14664) 374 : cluster [DBG] pgmap v247: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:03.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:03.025451+0000 mgr.smithi038.khesbl (mgr.14664) 375 : cluster [DBG] pgmap v248: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:03.770 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:03.025451+0000 mgr.smithi038.khesbl (mgr.14664) 375 : cluster [DBG] pgmap v248: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:05.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:05.025976+0000 mgr.smithi038.khesbl (mgr.14664) 376 : cluster [DBG] pgmap v249: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:05.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:05 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:05.025976+0000 mgr.smithi038.khesbl (mgr.14664) 376 : cluster [DBG] pgmap v249: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:07.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:07.026578+0000 mgr.smithi038.khesbl (mgr.14664) 377 : cluster [DBG] pgmap v250: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:07.524 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:07.026578+0000 mgr.smithi038.khesbl (mgr.14664) 377 : cluster [DBG] pgmap v250: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:09.270 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:09 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:09.027019+0000 mgr.smithi038.khesbl (mgr.14664) 378 : cluster [DBG] pgmap v251: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:09.354 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:09.027019+0000 mgr.smithi038.khesbl (mgr.14664) 378 : cluster [DBG] pgmap v251: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:11.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:11.027543+0000 mgr.smithi038.khesbl (mgr.14664) 379 : cluster [DBG] pgmap v252: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:11.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:11 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:11.027543+0000 mgr.smithi038.khesbl (mgr.14664) 379 : cluster [DBG] pgmap v252: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:13.605 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:13.027911+0000 mgr.smithi038.khesbl (mgr.14664) 380 : cluster [DBG] pgmap v253: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:13.770 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:13.027911+0000 mgr.smithi038.khesbl (mgr.14664) 380 : cluster [DBG] pgmap v253: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:15.354 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:15.028458+0000 mgr.smithi038.khesbl (mgr.14664) 381 : cluster [DBG] pgmap v254: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:15.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:15.028458+0000 mgr.smithi038.khesbl (mgr.14664) 381 : cluster [DBG] pgmap v254: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:17.354 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:17.028913+0000 mgr.smithi038.khesbl (mgr.14664) 382 : cluster [DBG] pgmap v255: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:17.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:17.028913+0000 mgr.smithi038.khesbl (mgr.14664) 382 : cluster [DBG] pgmap v255: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:19.270 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:19.029420+0000 mgr.smithi038.khesbl (mgr.14664) 383 : cluster [DBG] pgmap v256: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:19.354 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:19.029420+0000 mgr.smithi038.khesbl (mgr.14664) 383 : cluster [DBG] pgmap v256: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:21.354 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:21.029988+0000 mgr.smithi038.khesbl (mgr.14664) 384 : cluster [DBG] pgmap v257: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:21.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:21.029988+0000 mgr.smithi038.khesbl (mgr.14664) 384 : cluster [DBG] pgmap v257: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:23.604 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:23.030571+0000 mgr.smithi038.khesbl (mgr.14664) 385 : cluster [DBG] pgmap v258: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:23.770 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:23.030571+0000 mgr.smithi038.khesbl (mgr.14664) 385 : cluster [DBG] pgmap v258: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:25.354 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:25.031181+0000 mgr.smithi038.khesbl (mgr.14664) 386 : cluster [DBG] pgmap v259: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:25.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:25 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:25.031181+0000 mgr.smithi038.khesbl (mgr.14664) 386 : cluster [DBG] pgmap v259: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:27.354 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:27 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:27.031644+0000 mgr.smithi038.khesbl (mgr.14664) 387 : cluster [DBG] pgmap v260: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:27.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:27.031644+0000 mgr.smithi038.khesbl (mgr.14664) 387 : cluster [DBG] pgmap v260: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:29.270 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:29 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:29.032249+0000 mgr.smithi038.khesbl (mgr.14664) 388 : cluster [DBG] pgmap v261: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:29.354 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:29 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:29.032249+0000 mgr.smithi038.khesbl (mgr.14664) 388 : cluster [DBG] pgmap v261: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:31.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:31 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:31.032854+0000 mgr.smithi038.khesbl (mgr.14664) 389 : cluster [DBG] pgmap v262: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:31.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:31 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:31.032854+0000 mgr.smithi038.khesbl (mgr.14664) 389 : cluster [DBG] pgmap v262: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:33.604 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:33 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:33.033273+0000 mgr.smithi038.khesbl (mgr.14664) 390 : cluster [DBG] pgmap v263: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:33.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:33 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:33.033273+0000 mgr.smithi038.khesbl (mgr.14664) 390 : cluster [DBG] pgmap v263: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:35.354 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:35 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:35.033792+0000 mgr.smithi038.khesbl (mgr.14664) 391 : cluster [DBG] pgmap v264: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:35.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:35.033792+0000 mgr.smithi038.khesbl (mgr.14664) 391 : cluster [DBG] pgmap v264: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:37.354 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:37 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:37.034226+0000 mgr.smithi038.khesbl (mgr.14664) 392 : cluster [DBG] pgmap v265: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:37.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:37 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:37.034226+0000 mgr.smithi038.khesbl (mgr.14664) 392 : cluster [DBG] pgmap v265: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:39.270 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:39 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:39.034720+0000 mgr.smithi038.khesbl (mgr.14664) 393 : cluster [DBG] pgmap v266: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:39.354 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:39 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:39.034720+0000 mgr.smithi038.khesbl (mgr.14664) 393 : cluster [DBG] pgmap v266: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:41.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:41 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:41.035203+0000 mgr.smithi038.khesbl (mgr.14664) 394 : cluster [DBG] pgmap v267: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:41.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:41 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:41.035203+0000 mgr.smithi038.khesbl (mgr.14664) 394 : cluster [DBG] pgmap v267: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:42.354 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:42 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:43:41.981428+0000 mon.smithi038 (mon.0) 728 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:43:42.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:42 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:43:41.981428+0000 mon.smithi038 (mon.0) 728 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:43:43.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:43:42.299799+0000 mon.smithi038 (mon.0) 729 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:43:43.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:43:42.300458+0000 mon.smithi038 (mon.0) 730 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:43:43.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cephadm 2024-02-15T18:43:42.301005+0000 mgr.smithi038.khesbl (mgr.14664) 395 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:43:43.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:43:42.304799+0000 mon.smithi038 (mon.0) 731 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:43:43.355 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:43 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:43.035671+0000 mgr.smithi038.khesbl (mgr.14664) 396 : cluster [DBG] pgmap v268: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:43.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:43:42.299799+0000 mon.smithi038 (mon.0) 729 : audit [DBG] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:43:43.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:43:42.300458+0000 mon.smithi038 (mon.0) 730 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:43:43.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cephadm 2024-02-15T18:43:42.301005+0000 mgr.smithi038.khesbl (mgr.14664) 395 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:43:43.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:43:42.304799+0000 mon.smithi038 (mon.0) 731 : audit [INF] from='mgr.14664 172.21.15.38:0/2021914404' entity='mgr.smithi038.khesbl' 2024-02-15T18:43:43.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:43.035671+0000 mgr.smithi038.khesbl (mgr.14664) 396 : cluster [DBG] pgmap v268: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:45.354 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:45 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:45.036218+0000 mgr.smithi038.khesbl (mgr.14664) 397 : cluster [DBG] pgmap v269: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:45.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:45 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:45.036218+0000 mgr.smithi038.khesbl (mgr.14664) 397 : cluster [DBG] pgmap v269: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:47.354 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:47 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:47.036919+0000 mgr.smithi038.khesbl (mgr.14664) 398 : cluster [DBG] pgmap v270: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:47.521 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:47 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:47.036919+0000 mgr.smithi038.khesbl (mgr.14664) 398 : cluster [DBG] pgmap v270: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:49.270 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:49 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:49.037500+0000 mgr.smithi038.khesbl (mgr.14664) 399 : cluster [DBG] pgmap v271: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:49.354 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:49 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:49.037500+0000 mgr.smithi038.khesbl (mgr.14664) 399 : cluster [DBG] pgmap v271: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:51.354 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:51.037924+0000 mgr.smithi038.khesbl (mgr.14664) 400 : cluster [DBG] pgmap v272: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:51.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:51 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:51.037924+0000 mgr.smithi038.khesbl (mgr.14664) 400 : cluster [DBG] pgmap v272: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:53.604 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:53.038471+0000 mgr.smithi038.khesbl (mgr.14664) 401 : cluster [DBG] pgmap v273: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:53.770 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:53 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:53.038471+0000 mgr.smithi038.khesbl (mgr.14664) 401 : cluster [DBG] pgmap v273: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:55.354 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:55.039041+0000 mgr.smithi038.khesbl (mgr.14664) 402 : cluster [DBG] pgmap v274: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:55.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:55 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:55.039041+0000 mgr.smithi038.khesbl (mgr.14664) 402 : cluster [DBG] pgmap v274: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:57.756 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'ceph orch ps' 2024-02-15T18:43:58.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:43:58 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:57.039720+0000 mgr.smithi038.khesbl (mgr.14664) 403 : cluster [DBG] pgmap v275: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:43:58.604 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:43:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:57.039720+0000 mgr.smithi038.khesbl (mgr.14664) 403 : cluster [DBG] pgmap v275: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:00.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.046+0000 7fdd7dc91700 1 -- 172.21.15.38:0/646134163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd780f5430 msgr2=0x7fdd780f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:00.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.046+0000 7fdd7dc91700 1 --2- 172.21.15.38:0/646134163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd780f5430 0x7fdd780f5850 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fdd6c004660 tx=0x7fdd6c009b30).stop 2024-02-15T18:44:00.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.046+0000 7fdd7dc91700 1 -- 172.21.15.38:0/646134163 shutdown_connections 2024-02-15T18:44:00.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.046+0000 7fdd7dc91700 1 --2- 172.21.15.38:0/646134163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd780f66d0 0x7fdd780f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:00.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.046+0000 7fdd7dc91700 1 --2- 172.21.15.38:0/646134163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd780f5430 0x7fdd780f5850 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:00.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.046+0000 7fdd7dc91700 1 -- 172.21.15.38:0/646134163 >> 172.21.15.38:0/646134163 conn(0x7fdd780f0b90 msgr2=0x7fdd780f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:00.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.046+0000 7fdd7dc91700 1 -- 172.21.15.38:0/646134163 shutdown_connections 2024-02-15T18:44:00.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.046+0000 7fdd7dc91700 1 -- 172.21.15.38:0/646134163 wait complete. 2024-02-15T18:44:00.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.047+0000 7fdd7dc91700 1 Processor -- start 2024-02-15T18:44:00.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.047+0000 7fdd7dc91700 1 -- start start 2024-02-15T18:44:00.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.047+0000 7fdd7dc91700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd780f5430 0x7fdd7810bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:00.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.047+0000 7fdd7dc91700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd780f66d0 0x7fdd7810e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:00.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.047+0000 7fdd7dc91700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780602b0 con 0x7fdd780f5430 2024-02-15T18:44:00.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.047+0000 7fdd7dc91700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780603f0 con 0x7fdd780f66d0 2024-02-15T18:44:00.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.048+0000 7fdd7cc8f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd780f5430 0x7fdd7810bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:00.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.048+0000 7fdd7cc8f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd780f5430 0x7fdd7810bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41582/0 (socket says 172.21.15.38:41582) 2024-02-15T18:44:00.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.048+0000 7fdd7cc8f700 1 -- 172.21.15.38:0/1892365139 learned_addr learned my addr 172.21.15.38:0/1892365139 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:00.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.048+0000 7fdd7cc8f700 1 -- 172.21.15.38:0/1892365139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd780f66d0 msgr2=0x7fdd7810e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:00.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.048+0000 7fdd77fff700 1 --2- 172.21.15.38:0/1892365139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd780f66d0 0x7fdd7810e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:00.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.048+0000 7fdd7cc8f700 1 --2- 172.21.15.38:0/1892365139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd780f66d0 0x7fdd7810e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:00.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.048+0000 7fdd7cc8f700 1 -- 172.21.15.38:0/1892365139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c005040 con 0x7fdd780f5430 2024-02-15T18:44:00.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.048+0000 7fdd77fff700 1 --2- 172.21.15.38:0/1892365139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd780f66d0 0x7fdd7810e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:44:00.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.048+0000 7fdd7cc8f700 1 --2- 172.21.15.38:0/1892365139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd780f5430 0x7fdd7810bf40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fdd6c013fd0 tx=0x7fdd6c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:00.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.048+0000 7fdd757fa700 1 -- 172.21.15.38:0/1892365139 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd6c018070 con 0x7fdd780f5430 2024-02-15T18:44:00.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.048+0000 7fdd757fa700 1 -- 172.21.15.38:0/1892365139 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fdd6c007d70 con 0x7fdd780f5430 2024-02-15T18:44:00.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.048+0000 7fdd757fa700 1 -- 172.21.15.38:0/1892365139 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd6c01c620 con 0x7fdd780f5430 2024-02-15T18:44:00.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.048+0000 7fdd7dc91700 1 -- 172.21.15.38:0/1892365139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7810e930 con 0x7fdd780f5430 2024-02-15T18:44:00.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.048+0000 7fdd7dc91700 1 -- 172.21.15.38:0/1892365139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7810ed00 con 0x7fdd780f5430 2024-02-15T18:44:00.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.049+0000 7fdd7dc91700 1 -- 172.21.15.38:0/1892365139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd78059100 con 0x7fdd780f5430 2024-02-15T18:44:00.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.050+0000 7fdd757fa700 1 -- 172.21.15.38:0/1892365139 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fdd6c01d420 con 0x7fdd780f5430 2024-02-15T18:44:00.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.050+0000 7fdd757fa700 1 --2- 172.21.15.38:0/1892365139 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fdd68060f70 0x7fdd68063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:00.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.050+0000 7fdd757fa700 1 -- 172.21.15.38:0/1892365139 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fdd6c07df90 con 0x7fdd780f5430 2024-02-15T18:44:00.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.050+0000 7fdd77fff700 1 --2- 172.21.15.38:0/1892365139 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fdd68060f70 0x7fdd68063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:00.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.050+0000 7fdd77fff700 1 --2- 172.21.15.38:0/1892365139 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fdd68060f70 0x7fdd68063430 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fdd64004a70 tx=0x7fdd64009040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:00.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.054+0000 7fdd757fa700 1 -- 172.21.15.38:0/1892365139 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fdd6c04e520 con 0x7fdd780f5430 2024-02-15T18:44:00.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.240+0000 7fdd7dc91700 1 -- 172.21.15.38:0/1892365139 --> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd780fdd70 con 0x7fdd68060f70 2024-02-15T18:44:00.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.249+0000 7fdd757fa700 1 -- 172.21.15.38:0/1892365139 <== mgr.14664 v2:172.21.15.38:6800/3254855982 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7fdd780fdd70 con 0x7fdd68060f70 2024-02-15T18:44:00.237 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:44:00.237 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.smithi038 smithi038 *:9093,9094 running (112s) 78s ago 16m 13.2M - 0.23.0 ba2b418f427c 874f0582274f 2024-02-15T18:44:00.238 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi038 smithi038 running (6m) 78s ago 16m 6635k - 16.2.14-521-g0e714d9a 1fbc92c859dc c2fee0824d4b 2024-02-15T18:44:00.238 INFO:teuthology.orchestra.run.smithi038.stdout:crash.smithi121 smithi121 running (6m) 2m ago 15m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 05c1a5501a35 2024-02-15T18:44:00.238 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.smithi038 smithi038 *:3000 running (90s) 78s ago 16m 44.0M - 8.3.5 dad864ee21e9 86b1e2a0208e 2024-02-15T18:44:00.238 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.hxoggl smithi038 running (3m) 78s ago 11m 64.7M - 16.2.14-521-g0e714d9a 1fbc92c859dc 42b83e98219f 2024-02-15T18:44:00.238 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi038.qgkzgn smithi038 running (3m) 78s ago 10m 14.8M - 16.2.14-521-g0e714d9a 1fbc92c859dc ecbb70ac6d7d 2024-02-15T18:44:00.238 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.mvumlh smithi121 running (2m) 2m ago 11m 12.4M - 16.2.14-521-g0e714d9a 1fbc92c859dc 6b9a272c73f9 2024-02-15T18:44:00.238 INFO:teuthology.orchestra.run.smithi038.stdout:mds.cephfs.smithi121.ovuzki smithi121 running (2m) 2m ago 11m 12.3M - 16.2.14-521-g0e714d9a 1fbc92c859dc 845253fa7f6c 2024-02-15T18:44:00.238 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi038.khesbl smithi038 *:8443,9283 running (8m) 78s ago 18m 442M - 16.2.14-521-g0e714d9a 1fbc92c859dc e642f3c2cfc9 2024-02-15T18:44:00.238 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.smithi121.ckiryi smithi121 *:8443,9283 running (7m) 2m ago 15m 387M - 16.2.14-521-g0e714d9a 1fbc92c859dc 82515b025594 2024-02-15T18:44:00.238 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi038 smithi038 running (7m) 78s ago 18m 52.5M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc d3e8317a30ca 2024-02-15T18:44:00.238 INFO:teuthology.orchestra.run.smithi038.stdout:mon.smithi121 smithi121 running (7m) 2m ago 15m 53.7M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 42e1dda1a39d 2024-02-15T18:44:00.238 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi038 smithi038 *:9100 running (2m) 78s ago 16m 14.0M - 1.3.1 1dbe0e931976 e1b1d762ce83 2024-02-15T18:44:00.238 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter.smithi121 smithi121 *:9100 running (2m) 2m ago 15m 7134k - 1.3.1 1dbe0e931976 d93a49dc70f8 2024-02-15T18:44:00.238 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (6m) 78s ago 14m 196M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 623ac7ecbb0a 2024-02-15T18:44:00.238 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (6m) 78s ago 14m 221M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 0f9e8e1f255a 2024-02-15T18:44:00.239 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (5m) 78s ago 13m 212M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 37171264b5be 2024-02-15T18:44:00.239 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi121 running (4m) 2m ago 13m 275M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc d505a58b375c 2024-02-15T18:44:00.239 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi121 running (4m) 2m ago 13m 272M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 0ce6b6abb99e 2024-02-15T18:44:00.239 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi121 running (4m) 2m ago 12m 194M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 79bb22243769 2024-02-15T18:44:00.239 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.smithi038 smithi038 *:9095 running (2m) 78s ago 15m 57.5M - 2.33.4 514e6a882f6e 919f6aa6ab04 2024-02-15T18:44:00.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.251+0000 7fdd7dc91700 1 -- 172.21.15.38:0/1892365139 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fdd68060f70 msgr2=0x7fdd68063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:00.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.251+0000 7fdd7dc91700 1 --2- 172.21.15.38:0/1892365139 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fdd68060f70 0x7fdd68063430 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fdd64004a70 tx=0x7fdd64009040).stop 2024-02-15T18:44:00.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.251+0000 7fdd7dc91700 1 -- 172.21.15.38:0/1892365139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd780f5430 msgr2=0x7fdd7810bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:00.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.251+0000 7fdd7dc91700 1 --2- 172.21.15.38:0/1892365139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd780f5430 0x7fdd7810bf40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fdd6c013fd0 tx=0x7fdd6c009b30).stop 2024-02-15T18:44:00.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.251+0000 7fdd7dc91700 1 -- 172.21.15.38:0/1892365139 shutdown_connections 2024-02-15T18:44:00.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.251+0000 7fdd7dc91700 1 --2- 172.21.15.38:0/1892365139 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fdd68060f70 0x7fdd68063430 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:00.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.251+0000 7fdd7dc91700 1 --2- 172.21.15.38:0/1892365139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd780f5430 0x7fdd7810bf40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:00.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.251+0000 7fdd7dc91700 1 --2- 172.21.15.38:0/1892365139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd780f66d0 0x7fdd7810e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:00.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.251+0000 7fdd7dc91700 1 -- 172.21.15.38:0/1892365139 >> 172.21.15.38:0/1892365139 conn(0x7fdd780f0b90 msgr2=0x7fdd780fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:00.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.251+0000 7fdd7dc91700 1 -- 172.21.15.38:0/1892365139 shutdown_connections 2024-02-15T18:44:00.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:00.251+0000 7fdd7dc91700 1 -- 172.21.15.38:0/1892365139 wait complete. 2024-02-15T18:44:00.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:00 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:43:59.040282+0000 mgr.smithi038.khesbl (mgr.14664) 404 : cluster [DBG] pgmap v276: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:00.604 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:43:59.040282+0000 mgr.smithi038.khesbl (mgr.14664) 404 : cluster [DBG] pgmap v276: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:01.025 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'ceph versions' 2024-02-15T18:44:01.604 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:00.241191+0000 mgr.smithi038.khesbl (mgr.14664) 405 : audit [DBG] from='client.34406 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:44:01.746 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:01 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:00.241191+0000 mgr.smithi038.khesbl (mgr.14664) 405 : audit [DBG] from='client.34406 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:44:02.451 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:02 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:44:01.040800+0000 mgr.smithi038.khesbl (mgr.14664) 406 : cluster [DBG] pgmap v277: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:02.604 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:44:01.040800+0000 mgr.smithi038.khesbl (mgr.14664) 406 : cluster [DBG] pgmap v277: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:03.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.073+0000 7f863f43b700 1 -- 172.21.15.38:0/251544211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8638060250 msgr2=0x7f86380f9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:03.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.073+0000 7f863f43b700 1 --2- 172.21.15.38:0/251544211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8638060250 0x7f86380f9360 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f8628004660 tx=0x7f8628009b30).stop 2024-02-15T18:44:03.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.073+0000 7f863f43b700 1 -- 172.21.15.38:0/251544211 shutdown_connections 2024-02-15T18:44:03.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.073+0000 7f863f43b700 1 --2- 172.21.15.38:0/251544211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86380f9850 0x7f86380fbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:03.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.073+0000 7f863f43b700 1 --2- 172.21.15.38:0/251544211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8638060250 0x7f86380f9360 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:03.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.073+0000 7f863f43b700 1 -- 172.21.15.38:0/251544211 >> 172.21.15.38:0/251544211 conn(0x7f86380f0d40 msgr2=0x7f86380f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:03.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.073+0000 7f863f43b700 1 -- 172.21.15.38:0/251544211 shutdown_connections 2024-02-15T18:44:03.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.073+0000 7f863f43b700 1 -- 172.21.15.38:0/251544211 wait complete. 2024-02-15T18:44:03.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.074+0000 7f863f43b700 1 Processor -- start 2024-02-15T18:44:03.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.074+0000 7f863f43b700 1 -- start start 2024-02-15T18:44:03.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.074+0000 7f863f43b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8638060250 0x7f86381091d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:03.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.074+0000 7f863f43b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86380f9850 0x7f86381096c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:03.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.074+0000 7f863f43b700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8638109c70 con 0x7f8638060250 2024-02-15T18:44:03.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.074+0000 7f863f43b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8638109db0 con 0x7f86380f9850 2024-02-15T18:44:03.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.074+0000 7f863d1d7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8638060250 0x7f86381091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:03.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.074+0000 7f863d1d7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8638060250 0x7f86381091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41592/0 (socket says 172.21.15.38:41592) 2024-02-15T18:44:03.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.074+0000 7f863d1d7700 1 -- 172.21.15.38:0/3814105419 learned_addr learned my addr 172.21.15.38:0/3814105419 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:03.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.074+0000 7f863c9d6700 1 --2- 172.21.15.38:0/3814105419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86380f9850 0x7f86381096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:03.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.075+0000 7f863d1d7700 1 -- 172.21.15.38:0/3814105419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86380f9850 msgr2=0x7f86381096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:03.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.075+0000 7f863d1d7700 1 --2- 172.21.15.38:0/3814105419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86380f9850 0x7f86381096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:03.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.075+0000 7f863d1d7700 1 -- 172.21.15.38:0/3814105419 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8628005040 con 0x7f8638060250 2024-02-15T18:44:03.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.075+0000 7f863d1d7700 1 --2- 172.21.15.38:0/3814105419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8638060250 0x7f86381091d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f8628013fd0 tx=0x7f8628009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:03.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.075+0000 7f862dffb700 1 -- 172.21.15.38:0/3814105419 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8628018070 con 0x7f8638060250 2024-02-15T18:44:03.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.075+0000 7f862dffb700 1 -- 172.21.15.38:0/3814105419 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f8628007ba0 con 0x7f8638060250 2024-02-15T18:44:03.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.075+0000 7f862dffb700 1 -- 172.21.15.38:0/3814105419 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f862801c620 con 0x7f8638060250 2024-02-15T18:44:03.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.075+0000 7f863f43b700 1 -- 172.21.15.38:0/3814105419 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f863810a6f0 con 0x7f8638060250 2024-02-15T18:44:03.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.075+0000 7f863f43b700 1 -- 172.21.15.38:0/3814105419 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f863810ac50 con 0x7f8638060250 2024-02-15T18:44:03.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.076+0000 7f863f43b700 1 -- 172.21.15.38:0/3814105419 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8638045ad0 con 0x7f8638060250 2024-02-15T18:44:03.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.077+0000 7f862dffb700 1 -- 172.21.15.38:0/3814105419 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f862801c8e0 con 0x7f8638060250 2024-02-15T18:44:03.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.077+0000 7f862dffb700 1 --2- 172.21.15.38:0/3814105419 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8624060f20 0x7f86240633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:03.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.077+0000 7f862dffb700 1 -- 172.21.15.38:0/3814105419 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f862807df90 con 0x7f8638060250 2024-02-15T18:44:03.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.077+0000 7f863c9d6700 1 --2- 172.21.15.38:0/3814105419 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8624060f20 0x7f86240633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:03.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.077+0000 7f863c9d6700 1 --2- 172.21.15.38:0/3814105419 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8624060f20 0x7f86240633e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f8638060020 tx=0x7f8634009be0).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:03.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.081+0000 7f862dffb700 1 -- 172.21.15.38:0/3814105419 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8628051df0 con 0x7f8638060250 2024-02-15T18:44:03.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.326+0000 7f863f43b700 1 -- 172.21.15.38:0/3814105419 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f863810b410 con 0x7f8638060250 2024-02-15T18:44:03.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.327+0000 7f862dffb700 1 -- 172.21.15.38:0/3814105419 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+648 (secure 0 0 0) 0x7f86280519e0 con 0x7f8638060250 2024-02-15T18:44:03.315 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-02-15T18:44:03.315 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": { 2024-02-15T18:44:03.315 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:44:03.315 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:44:03.315 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": { 2024-02-15T18:44:03.315 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:44:03.315 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:44:03.315 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": { 2024-02-15T18:44:03.315 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 6 2024-02-15T18:44:03.315 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:44:03.316 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-02-15T18:44:03.316 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 4 2024-02-15T18:44:03.316 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-02-15T18:44:03.316 INFO:teuthology.orchestra.run.smithi038.stdout: "overall": { 2024-02-15T18:44:03.316 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 14 2024-02-15T18:44:03.316 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-02-15T18:44:03.316 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-02-15T18:44:03.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.328+0000 7f863f43b700 1 -- 172.21.15.38:0/3814105419 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8624060f20 msgr2=0x7f86240633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:03.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.328+0000 7f863f43b700 1 --2- 172.21.15.38:0/3814105419 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8624060f20 0x7f86240633e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f8638060020 tx=0x7f8634009be0).stop 2024-02-15T18:44:03.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.328+0000 7f863f43b700 1 -- 172.21.15.38:0/3814105419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8638060250 msgr2=0x7f86381091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:03.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.328+0000 7f863f43b700 1 --2- 172.21.15.38:0/3814105419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8638060250 0x7f86381091d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f8628013fd0 tx=0x7f8628009b30).stop 2024-02-15T18:44:03.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.328+0000 7f863f43b700 1 -- 172.21.15.38:0/3814105419 shutdown_connections 2024-02-15T18:44:03.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.328+0000 7f863f43b700 1 --2- 172.21.15.38:0/3814105419 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8624060f20 0x7f86240633e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:03.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.328+0000 7f863f43b700 1 --2- 172.21.15.38:0/3814105419 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8638060250 0x7f86381091d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:03.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.328+0000 7f863f43b700 1 --2- 172.21.15.38:0/3814105419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86380f9850 0x7f86381096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:03.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.328+0000 7f863f43b700 1 -- 172.21.15.38:0/3814105419 >> 172.21.15.38:0/3814105419 conn(0x7f86380f0d40 msgr2=0x7f86380f9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:03.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.329+0000 7f863f43b700 1 -- 172.21.15.38:0/3814105419 shutdown_connections 2024-02-15T18:44:03.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:03.329+0000 7f863f43b700 1 -- 172.21.15.38:0/3814105419 wait complete. 2024-02-15T18:44:03.604 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:03.327231+0000 mon.smithi038 (mon.0) 732 : audit [DBG] from='client.? 172.21.15.38:0/3814105419' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:44:03.770 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:03 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:03.327231+0000 mon.smithi038 (mon.0) 732 : audit [DBG] from='client.? 172.21.15.38:0/3814105419' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:44:04.187 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-02-15T18:44:04.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:04 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:44:03.041147+0000 mgr.smithi038.khesbl (mgr.14664) 407 : cluster [DBG] pgmap v278: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:04.604 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:04 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:44:03.041147+0000 mgr.smithi038.khesbl (mgr.14664) 407 : cluster [DBG] pgmap v278: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:06.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.424+0000 7fd5c98aa700 1 -- 172.21.15.38:0/1801072033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c40f5ef0 msgr2=0x7fd5c40f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:06.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.424+0000 7fd5c98aa700 1 --2- 172.21.15.38:0/1801072033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c40f5ef0 0x7fd5c40f6920 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fd5b8004660 tx=0x7fd5b8009b30).stop 2024-02-15T18:44:06.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.424+0000 7fd5c98aa700 1 -- 172.21.15.38:0/1801072033 shutdown_connections 2024-02-15T18:44:06.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.424+0000 7fd5c98aa700 1 --2- 172.21.15.38:0/1801072033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c40f5ef0 0x7fd5c40f6920 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:06.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.424+0000 7fd5c98aa700 1 --2- 172.21.15.38:0/1801072033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5c40f55e0 0x7fd5c40f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:44:06.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.424+0000 7fd5c98aa700 1 -- 172.21.15.38:0/1801072033 >> 172.21.15.38:0/1801072033 conn(0x7fd5c40f0de0 msgr2=0x7fd5c40f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:06.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.424+0000 7fd5c98aa700 1 -- 172.21.15.38:0/1801072033 shutdown_connections 2024-02-15T18:44:06.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.424+0000 7fd5c98aa700 1 -- 172.21.15.38:0/1801072033 wait complete. 2024-02-15T18:44:06.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.425+0000 7fd5c98aa700 1 Processor -- start 2024-02-15T18:44:06.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.425+0000 7fd5c98aa700 1 -- start start 2024-02-15T18:44:06.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.425+0000 7fd5c98aa700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5c40f55e0 0x7fd5c4105f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:06.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.425+0000 7fd5c98aa700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c40f5ef0 0x7fd5c4106460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:06.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.425+0000 7fd5c98aa700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c4106aa0 con 0x7fd5c40f5ef0 2024-02-15T18:44:06.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.425+0000 7fd5c98aa700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c4107350 con 0x7fd5c40f55e0 2024-02-15T18:44:06.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.426+0000 7fd5c27fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c40f5ef0 0x7fd5c4106460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:06.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.426+0000 7fd5c27fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c40f5ef0 0x7fd5c4106460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42990/0 (socket says 172.21.15.38:42990) 2024-02-15T18:44:06.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.426+0000 7fd5c27fc700 1 -- 172.21.15.38:0/3309894363 learned_addr learned my addr 172.21.15.38:0/3309894363 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:06.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.426+0000 7fd5c2ffd700 1 --2- 172.21.15.38:0/3309894363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5c40f55e0 0x7fd5c4105f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:06.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.426+0000 7fd5c27fc700 1 -- 172.21.15.38:0/3309894363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5c40f55e0 msgr2=0x7fd5c4105f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:06.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.426+0000 7fd5c27fc700 1 --2- 172.21.15.38:0/3309894363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5c40f55e0 0x7fd5c4105f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:06.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.426+0000 7fd5c27fc700 1 -- 172.21.15.38:0/3309894363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b8005040 con 0x7fd5c40f5ef0 2024-02-15T18:44:06.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.426+0000 7fd5c27fc700 1 --2- 172.21.15.38:0/3309894363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c40f5ef0 0x7fd5c4106460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fd5b8009830 tx=0x7fd5b8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:06.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.426+0000 7fd5ab7fe700 1 -- 172.21.15.38:0/3309894363 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5b801a070 con 0x7fd5c40f5ef0 2024-02-15T18:44:06.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.426+0000 7fd5ab7fe700 1 -- 172.21.15.38:0/3309894363 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fd5b8012ac0 con 0x7fd5c40f5ef0 2024-02-15T18:44:06.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.426+0000 7fd5c98aa700 1 -- 172.21.15.38:0/3309894363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c41075d0 con 0x7fd5c40f5ef0 2024-02-15T18:44:06.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.426+0000 7fd5ab7fe700 1 -- 172.21.15.38:0/3309894363 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5b801fac0 con 0x7fd5c40f5ef0 2024-02-15T18:44:06.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.426+0000 7fd5c98aa700 1 -- 172.21.15.38:0/3309894363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c4107ab0 con 0x7fd5c40f5ef0 2024-02-15T18:44:06.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.427+0000 7fd5c98aa700 1 -- 172.21.15.38:0/3309894363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c40f4a70 con 0x7fd5c40f5ef0 2024-02-15T18:44:06.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.428+0000 7fd5ab7fe700 1 -- 172.21.15.38:0/3309894363 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fd5b8012c30 con 0x7fd5c40f5ef0 2024-02-15T18:44:06.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.428+0000 7fd5ab7fe700 1 --2- 172.21.15.38:0/3309894363 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd5ac060f20 0x7fd5ac0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:06.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.428+0000 7fd5ab7fe700 1 -- 172.21.15.38:0/3309894363 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fd5b8080440 con 0x7fd5c40f5ef0 2024-02-15T18:44:06.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.428+0000 7fd5c2ffd700 1 --2- 172.21.15.38:0/3309894363 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd5ac060f20 0x7fd5ac0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:06.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.429+0000 7fd5c2ffd700 1 --2- 172.21.15.38:0/3309894363 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd5ac060f20 0x7fd5ac0633e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fd5b4006fd0 tx=0x7fd5b4008040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:06.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.432+0000 7fd5ab7fe700 1 -- 172.21.15.38:0/3309894363 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd5b8054320 con 0x7fd5c40f5ef0 2024-02-15T18:44:06.604 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:44:05.041560+0000 mgr.smithi038.khesbl (mgr.14664) 408 : cluster [DBG] pgmap v279: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:06.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.681+0000 7fd5c98aa700 1 -- 172.21.15.38:0/3309894363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd5c40fe0b0 con 0x7fd5c40f5ef0 2024-02-15T18:44:06.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.681+0000 7fd5ab7fe700 1 -- 172.21.15.38:0/3309894363 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+648 (secure 0 0 0) 0x7fd5b8053f10 con 0x7fd5c40f5ef0 2024-02-15T18:44:06.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.683+0000 7fd5c98aa700 1 -- 172.21.15.38:0/3309894363 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd5ac060f20 msgr2=0x7fd5ac0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:06.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.683+0000 7fd5c98aa700 1 --2- 172.21.15.38:0/3309894363 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd5ac060f20 0x7fd5ac0633e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fd5b4006fd0 tx=0x7fd5b4008040).stop 2024-02-15T18:44:06.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.683+0000 7fd5c98aa700 1 -- 172.21.15.38:0/3309894363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c40f5ef0 msgr2=0x7fd5c4106460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:06.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.683+0000 7fd5c98aa700 1 --2- 172.21.15.38:0/3309894363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c40f5ef0 0x7fd5c4106460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fd5b8009830 tx=0x7fd5b8009b30).stop 2024-02-15T18:44:06.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.683+0000 7fd5c98aa700 1 -- 172.21.15.38:0/3309894363 shutdown_connections 2024-02-15T18:44:06.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.683+0000 7fd5c98aa700 1 --2- 172.21.15.38:0/3309894363 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd5ac060f20 0x7fd5ac0633e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:06.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.683+0000 7fd5c98aa700 1 --2- 172.21.15.38:0/3309894363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5c40f55e0 0x7fd5c4105f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:06.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.683+0000 7fd5c98aa700 1 --2- 172.21.15.38:0/3309894363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5c40f5ef0 0x7fd5c4106460 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:06.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.683+0000 7fd5c98aa700 1 -- 172.21.15.38:0/3309894363 >> 172.21.15.38:0/3309894363 conn(0x7fd5c40f0de0 msgr2=0x7fd5c40fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:06.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.683+0000 7fd5c98aa700 1 -- 172.21.15.38:0/3309894363 shutdown_connections 2024-02-15T18:44:06.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:06.683+0000 7fd5c98aa700 1 -- 172.21.15.38:0/3309894363 wait complete. 2024-02-15T18:44:06.681 INFO:teuthology.orchestra.run.smithi038.stdout:true 2024-02-15T18:44:06.699 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:06 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:44:05.041560+0000 mgr.smithi038.khesbl (mgr.14664) 408 : cluster [DBG] pgmap v279: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:07.478 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-02-15T18:44:07.770 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:07 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:06.682165+0000 mon.smithi038 (mon.0) 733 : audit [DBG] from='client.? 172.21.15.38:0/3309894363' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:44:07.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:06.682165+0000 mon.smithi038 (mon.0) 733 : audit [DBG] from='client.? 172.21.15.38:0/3309894363' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:44:08.648 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:08 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:44:07.041941+0000 mgr.smithi038.khesbl (mgr.14664) 409 : cluster [DBG] pgmap v280: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:08.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:44:07.041941+0000 mgr.smithi038.khesbl (mgr.14664) 409 : cluster [DBG] pgmap v280: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:09.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.767+0000 7efed0f55700 1 -- 172.21.15.38:0/2425587814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efecc0f77d0 msgr2=0x7efecc0f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:09.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.767+0000 7efed0f55700 1 --2- 172.21.15.38:0/2425587814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efecc0f77d0 0x7efecc0f7c30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7efeb4004660 tx=0x7efeb4009b30).stop 2024-02-15T18:44:09.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.767+0000 7efed0f55700 1 -- 172.21.15.38:0/2425587814 shutdown_connections 2024-02-15T18:44:09.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.767+0000 7efed0f55700 1 --2- 172.21.15.38:0/2425587814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efecc0f77d0 0x7efecc0f7c30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:09.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.767+0000 7efed0f55700 1 --2- 172.21.15.38:0/2425587814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efecc0f6530 0x7efecc0f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:09.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.767+0000 7efed0f55700 1 -- 172.21.15.38:0/2425587814 >> 172.21.15.38:0/2425587814 conn(0x7efecc0f1cd0 msgr2=0x7efecc0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:09.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.767+0000 7efed0f55700 1 -- 172.21.15.38:0/2425587814 shutdown_connections 2024-02-15T18:44:09.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.767+0000 7efed0f55700 1 -- 172.21.15.38:0/2425587814 wait complete. 2024-02-15T18:44:09.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.768+0000 7efed0f55700 1 Processor -- start 2024-02-15T18:44:09.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.768+0000 7efed0f55700 1 -- start start 2024-02-15T18:44:09.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.768+0000 7efed0f55700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efecc0f6530 0x7efecc106db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:09.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.768+0000 7efed0f55700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efecc0f77d0 0x7efecc1072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:09.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.768+0000 7efed0f55700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc1078a0 con 0x7efecc0f6530 2024-02-15T18:44:09.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.768+0000 7efed0f55700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc1080e0 con 0x7efecc0f77d0 2024-02-15T18:44:09.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.768+0000 7efecb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efecc0f6530 0x7efecc106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:09.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.768+0000 7efecb7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efecc0f6530 0x7efecc106db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43018/0 (socket says 172.21.15.38:43018) 2024-02-15T18:44:09.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.768+0000 7efecb7fe700 1 -- 172.21.15.38:0/855037338 learned_addr learned my addr 172.21.15.38:0/855037338 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:09.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.769+0000 7efecaffd700 1 --2- 172.21.15.38:0/855037338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efecc0f77d0 0x7efecc1072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:09.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.769+0000 7efecb7fe700 1 -- 172.21.15.38:0/855037338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efecc0f77d0 msgr2=0x7efecc1072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:09.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.769+0000 7efecb7fe700 1 --2- 172.21.15.38:0/855037338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efecc0f77d0 0x7efecc1072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:09.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.769+0000 7efecb7fe700 1 -- 172.21.15.38:0/855037338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb4005040 con 0x7efecc0f6530 2024-02-15T18:44:09.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.769+0000 7efecb7fe700 1 --2- 172.21.15.38:0/855037338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efecc0f6530 0x7efecc106db0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7efebc002700 tx=0x7efebc009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:09.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.769+0000 7efec3fff700 1 -- 172.21.15.38:0/855037338 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efebc0079c0 con 0x7efecc0f6530 2024-02-15T18:44:09.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.769+0000 7efec3fff700 1 -- 172.21.15.38:0/855037338 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7efebc00f040 con 0x7efecc0f6530 2024-02-15T18:44:09.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.769+0000 7efed0f55700 1 -- 172.21.15.38:0/855037338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efecc1083c0 con 0x7efecc0f6530 2024-02-15T18:44:09.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.769+0000 7efec3fff700 1 -- 172.21.15.38:0/855037338 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efebc014940 con 0x7efecc0f6530 2024-02-15T18:44:09.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.769+0000 7efed0f55700 1 -- 172.21.15.38:0/855037338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efecc108900 con 0x7efecc0f6530 2024-02-15T18:44:09.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.770+0000 7efed0f55700 1 -- 172.21.15.38:0/855037338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efecc059100 con 0x7efecc0f6530 2024-02-15T18:44:09.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.771+0000 7efec3fff700 1 -- 172.21.15.38:0/855037338 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7efebc01b060 con 0x7efecc0f6530 2024-02-15T18:44:09.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.771+0000 7efec3fff700 1 --2- 172.21.15.38:0/855037338 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7efeb8060f70 0x7efeb8063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:09.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.771+0000 7efec3fff700 1 -- 172.21.15.38:0/855037338 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7efebc07bfb0 con 0x7efecc0f6530 2024-02-15T18:44:09.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.771+0000 7efecaffd700 1 --2- 172.21.15.38:0/855037338 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7efeb8060f70 0x7efeb8063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:09.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.771+0000 7efecaffd700 1 --2- 172.21.15.38:0/855037338 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7efeb8060f70 0x7efeb8063430 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7efeb4004630 tx=0x7efeb4009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:09.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:09.775+0000 7efec3fff700 1 -- 172.21.15.38:0/855037338 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7efebc04c3e0 con 0x7efecc0f6530 2024-02-15T18:44:10.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:10.017+0000 7efed0f55700 1 -- 172.21.15.38:0/855037338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efecc0fef00 con 0x7efecc0f6530 2024-02-15T18:44:10.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:10.018+0000 7efec3fff700 1 -- 172.21.15.38:0/855037338 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+648 (secure 0 0 0) 0x7efebc04fa00 con 0x7efecc0f6530 2024-02-15T18:44:10.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:10.019+0000 7efed0f55700 1 -- 172.21.15.38:0/855037338 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7efeb8060f70 msgr2=0x7efeb8063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:10.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:10.019+0000 7efed0f55700 1 --2- 172.21.15.38:0/855037338 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7efeb8060f70 0x7efeb8063430 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7efeb4004630 tx=0x7efeb4009b30).stop 2024-02-15T18:44:10.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:10.019+0000 7efed0f55700 1 -- 172.21.15.38:0/855037338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efecc0f6530 msgr2=0x7efecc106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:10.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:10.019+0000 7efed0f55700 1 --2- 172.21.15.38:0/855037338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efecc0f6530 0x7efecc106db0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7efebc002700 tx=0x7efebc009d80).stop 2024-02-15T18:44:10.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:10.019+0000 7efed0f55700 1 -- 172.21.15.38:0/855037338 shutdown_connections 2024-02-15T18:44:10.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:10.019+0000 7efed0f55700 1 --2- 172.21.15.38:0/855037338 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7efeb8060f70 0x7efeb8063430 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:10.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:10.019+0000 7efed0f55700 1 --2- 172.21.15.38:0/855037338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efecc0f6530 0x7efecc106db0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:10.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:10.020+0000 7efed0f55700 1 --2- 172.21.15.38:0/855037338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efecc0f77d0 0x7efecc1072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:10.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:10.020+0000 7efed0f55700 1 -- 172.21.15.38:0/855037338 >> 172.21.15.38:0/855037338 conn(0x7efecc0f1cd0 msgr2=0x7efecc0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:10.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:10.020+0000 7efed0f55700 1 -- 172.21.15.38:0/855037338 shutdown_connections 2024-02-15T18:44:10.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:10.020+0000 7efed0f55700 1 -- 172.21.15.38:0/855037338 wait complete. 2024-02-15T18:44:10.018 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)" 2024-02-15T18:44:10.697 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:44:09.042631+0000 mgr.smithi038.khesbl (mgr.14664) 410 : cluster [DBG] pgmap v281: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:10.698 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:10 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:10.018470+0000 mon.smithi038 (mon.0) 734 : audit [DBG] from='client.? 172.21.15.38:0/855037338' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:44:10.722 DEBUG:teuthology.parallel:result is None 2024-02-15T18:44:10.722 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-15T18:44:10.734 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi038.front.sepia.ceph.com 2024-02-15T18:44:10.734 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-15T18:44:10.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:44:09.042631+0000 mgr.smithi038.khesbl (mgr.14664) 410 : cluster [DBG] pgmap v281: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:10.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:10.018470+0000 mon.smithi038 (mon.0) 734 : audit [DBG] from='client.? 172.21.15.38:0/855037338' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:44:12.729 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:12 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:44:11.043027+0000 mgr.smithi038.khesbl (mgr.14664) 411 : cluster [DBG] pgmap v282: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:12.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:12 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:44:11.043027+0000 mgr.smithi038.khesbl (mgr.14664) 411 : cluster [DBG] pgmap v282: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:13.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.063+0000 7f832aac6700 1 -- 172.21.15.38:0/779249152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240f6860 msgr2=0x7f83240f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:13.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.063+0000 7f832aac6700 1 --2- 172.21.15.38:0/779249152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240f6860 0x7f83240f6cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f8310004660 tx=0x7f8310009b30).stop 2024-02-15T18:44:13.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.063+0000 7f832aac6700 1 -- 172.21.15.38:0/779249152 shutdown_connections 2024-02-15T18:44:13.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.063+0000 7f832aac6700 1 --2- 172.21.15.38:0/779249152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240f6860 0x7f83240f6cc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:13.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.063+0000 7f832aac6700 1 --2- 172.21.15.38:0/779249152 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83240f55c0 0x7f83240f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:13.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.063+0000 7f832aac6700 1 -- 172.21.15.38:0/779249152 >> 172.21.15.38:0/779249152 conn(0x7f83240f0d40 msgr2=0x7f83240f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:13.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.063+0000 7f832aac6700 1 -- 172.21.15.38:0/779249152 shutdown_connections 2024-02-15T18:44:13.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.063+0000 7f832aac6700 1 -- 172.21.15.38:0/779249152 wait complete. 2024-02-15T18:44:13.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.064+0000 7f832aac6700 1 Processor -- start 2024-02-15T18:44:13.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.064+0000 7f832aac6700 1 -- start start 2024-02-15T18:44:13.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.064+0000 7f832aac6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83240f55c0 0x7f8324109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:13.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.064+0000 7f832aac6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240f6860 0x7f832410c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:13.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.064+0000 7f832aac6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83240603c0 con 0x7f83240f6860 2024-02-15T18:44:13.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.064+0000 7f832aac6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8324060500 con 0x7f83240f55c0 2024-02-15T18:44:13.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.064+0000 7f8323fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240f6860 0x7f832410c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:13.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.064+0000 7f8323fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240f6860 0x7f832410c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43050/0 (socket says 172.21.15.38:43050) 2024-02-15T18:44:13.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.064+0000 7f8323fff700 1 -- 172.21.15.38:0/4270683367 learned_addr learned my addr 172.21.15.38:0/4270683367 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:13.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.064+0000 7f8328862700 1 --2- 172.21.15.38:0/4270683367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83240f55c0 0x7f8324109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:13.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.065+0000 7f8323fff700 1 -- 172.21.15.38:0/4270683367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83240f55c0 msgr2=0x7f8324109cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:13.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.065+0000 7f8323fff700 1 --2- 172.21.15.38:0/4270683367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83240f55c0 0x7f8324109cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:13.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.065+0000 7f8323fff700 1 -- 172.21.15.38:0/4270683367 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8310005040 con 0x7f83240f6860 2024-02-15T18:44:13.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.065+0000 7f8323fff700 1 --2- 172.21.15.38:0/4270683367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240f6860 0x7f832410c1d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f8310009f50 tx=0x7f8310009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:13.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.065+0000 7f83217fa700 1 -- 172.21.15.38:0/4270683367 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8310018070 con 0x7f83240f6860 2024-02-15T18:44:13.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.065+0000 7f83217fa700 1 -- 172.21.15.38:0/4270683367 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f8310007ba0 con 0x7f83240f6860 2024-02-15T18:44:13.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.065+0000 7f832aac6700 1 -- 172.21.15.38:0/4270683367 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832410c720 con 0x7f83240f6860 2024-02-15T18:44:13.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.065+0000 7f83217fa700 1 -- 172.21.15.38:0/4270683367 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f831001c420 con 0x7f83240f6860 2024-02-15T18:44:13.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.065+0000 7f832aac6700 1 -- 172.21.15.38:0/4270683367 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832410cc50 con 0x7f83240f6860 2024-02-15T18:44:13.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.066+0000 7f832aac6700 1 -- 172.21.15.38:0/4270683367 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83240f49b0 con 0x7f83240f6860 2024-02-15T18:44:13.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.066+0000 7f83217fa700 1 -- 172.21.15.38:0/4270683367 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f831001d420 con 0x7f83240f6860 2024-02-15T18:44:13.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.067+0000 7f83217fa700 1 --2- 172.21.15.38:0/4270683367 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8314060f20 0x7f83140633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:13.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.067+0000 7f83217fa700 1 -- 172.21.15.38:0/4270683367 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f831007de10 con 0x7f83240f6860 2024-02-15T18:44:13.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.067+0000 7f8328862700 1 --2- 172.21.15.38:0/4270683367 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8314060f20 0x7f83140633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:13.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.067+0000 7f8328862700 1 --2- 172.21.15.38:0/4270683367 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8314060f20 0x7f83140633e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f8324060040 tx=0x7f831800b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:13.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.071+0000 7f83217fa700 1 -- 172.21.15.38:0/4270683367 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f831004e1e0 con 0x7f83240f6860 2024-02-15T18:44:13.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.291+0000 7f832aac6700 1 -- 172.21.15.38:0/4270683367 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8324045ad0 con 0x7f83240f6860 2024-02-15T18:44:13.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.291+0000 7f83217fa700 1 -- 172.21.15.38:0/4270683367 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1777 (secure 0 0 0) 0x7f8310051800 con 0x7f83240f6860 2024-02-15T18:44:13.279 INFO:teuthology.orchestra.run.smithi038.stdout:e32 2024-02-15T18:44:13.280 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:44:13.280 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:44:13.280 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-02-15T18:44:13.280 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:13.280 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:44:13.280 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-02-15T18:44:13.280 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 32 2024-02-15T18:44:13.280 INFO:teuthology.orchestra.run.smithi038.stdout:flags 12 2024-02-15T18:44:13.280 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-02-15T18:32:46.871444+0000 2024-02-15T18:44:13.280 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-02-15T18:41:26.398387+0000 2024-02-15T18:44:13.280 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-02-15T18:44:13.280 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-02-15T18:44:13.280 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 60 2024-02-15T18:44:13.281 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-02-15T18:44:13.281 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-02-15T18:44:13.281 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-02-15T18:44:13.281 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-02-15T18:44:13.281 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 79 2024-02-15T18:44:13.281 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:44:13.281 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 2 2024-02-15T18:44:13.281 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1 2024-02-15T18:44:13.281 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=34314,1=34318} 2024-02-15T18:44:13.281 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-02-15T18:44:13.281 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-02-15T18:44:13.281 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-02-15T18:44:13.281 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3] 2024-02-15T18:44:13.281 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-02-15T18:44:13.282 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-02-15T18:44:13.282 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-02-15T18:44:13.282 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 1 2024-02-15T18:44:13.282 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.hxoggl{0:34314} state up:active seq 11 join_fscid=1 addr [v2:172.21.15.38:6826/4077527450,v1:172.21.15.38:6827/4077527450] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:44:13.282 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi038.qgkzgn{1:34318} state up:active seq 9 join_fscid=1 addr [v2:172.21.15.38:6828/1616551408,v1:172.21.15.38:6829/1616551408] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:44:13.282 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:13.282 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:13.282 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-02-15T18:44:13.282 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:13.282 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.mvumlh{-1:44261} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.121:6824/180558011,v1:172.21.15.121:6825/180558011] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:44:13.282 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.cephfs.smithi121.ovuzki{-1:44265} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.121:6826/1075229339,v1:172.21.15.121:6827/1075229339] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:44:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.293+0000 7f832aac6700 1 -- 172.21.15.38:0/4270683367 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8314060f20 msgr2=0x7f83140633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.293+0000 7f832aac6700 1 --2- 172.21.15.38:0/4270683367 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8314060f20 0x7f83140633e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f8324060040 tx=0x7f831800b040).stop 2024-02-15T18:44:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.293+0000 7f832aac6700 1 -- 172.21.15.38:0/4270683367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240f6860 msgr2=0x7f832410c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.293+0000 7f832aac6700 1 --2- 172.21.15.38:0/4270683367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240f6860 0x7f832410c1d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f8310009f50 tx=0x7f8310009b30).stop 2024-02-15T18:44:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.294+0000 7f832aac6700 1 -- 172.21.15.38:0/4270683367 shutdown_connections 2024-02-15T18:44:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.294+0000 7f832aac6700 1 --2- 172.21.15.38:0/4270683367 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8314060f20 0x7f83140633e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.294+0000 7f832aac6700 1 --2- 172.21.15.38:0/4270683367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83240f55c0 0x7f8324109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.294+0000 7f832aac6700 1 --2- 172.21.15.38:0/4270683367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240f6860 0x7f832410c1d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:13.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.294+0000 7f832aac6700 1 -- 172.21.15.38:0/4270683367 >> 172.21.15.38:0/4270683367 conn(0x7f83240f0d40 msgr2=0x7f83240fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:13.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.294+0000 7f832aac6700 1 -- 172.21.15.38:0/4270683367 shutdown_connections 2024-02-15T18:44:13.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:13.294+0000 7f832aac6700 1 -- 172.21.15.38:0/4270683367 wait complete. 2024-02-15T18:44:13.284 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 32 2024-02-15T18:44:13.680 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:13 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:13.292223+0000 mon.smithi038 (mon.0) 735 : audit [DBG] from='client.? 172.21.15.38:0/4270683367' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:44:13.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:13 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:13.292223+0000 mon.smithi038 (mon.0) 735 : audit [DBG] from='client.? 172.21.15.38:0/4270683367' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:44:14.074 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-02-15T18:44:14.087 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-15T18:44:14.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.223+0000 7f0d680fc700 1 -- 172.21.15.38:0/1743836407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d600f53b0 msgr2=0x7f0d600f57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:14.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.223+0000 7f0d680fc700 1 --2- 172.21.15.38:0/1743836407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d600f53b0 0x7f0d600f57d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f0d5c004660 tx=0x7f0d5c009b30).stop 2024-02-15T18:44:14.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.223+0000 7f0d680fc700 1 -- 172.21.15.38:0/1743836407 shutdown_connections 2024-02-15T18:44:14.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.223+0000 7f0d680fc700 1 --2- 172.21.15.38:0/1743836407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d600f5cc0 0x7f0d600f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:14.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.223+0000 7f0d680fc700 1 --2- 172.21.15.38:0/1743836407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d600f53b0 0x7f0d600f57d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:14.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.223+0000 7f0d680fc700 1 -- 172.21.15.38:0/1743836407 >> 172.21.15.38:0/1743836407 conn(0x7f0d600f0bd0 msgr2=0x7f0d600f2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:14.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.223+0000 7f0d680fc700 1 -- 172.21.15.38:0/1743836407 shutdown_connections 2024-02-15T18:44:14.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.223+0000 7f0d680fc700 1 -- 172.21.15.38:0/1743836407 wait complete. 2024-02-15T18:44:14.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.224+0000 7f0d680fc700 1 Processor -- start 2024-02-15T18:44:14.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.224+0000 7f0d680fc700 1 -- start start 2024-02-15T18:44:14.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.224+0000 7f0d680fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d600f53b0 0x7f0d6005fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:14.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.224+0000 7f0d680fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d600f5cc0 0x7f0d6010ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:14.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.224+0000 7f0d680fc700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6010bef0 con 0x7f0d600f5cc0 2024-02-15T18:44:14.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.224+0000 7f0d680fc700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6010c030 con 0x7f0d600f53b0 2024-02-15T18:44:14.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.224+0000 7f0d65697700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d600f5cc0 0x7f0d6010ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:14.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.224+0000 7f0d65697700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d600f5cc0 0x7f0d6010ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59170/0 (socket says 172.21.15.38:59170) 2024-02-15T18:44:14.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.224+0000 7f0d65697700 1 -- 172.21.15.38:0/3458760530 learned_addr learned my addr 172.21.15.38:0/3458760530 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:14.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.224+0000 7f0d65697700 1 -- 172.21.15.38:0/3458760530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d600f53b0 msgr2=0x7f0d6005fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:14.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.224+0000 7f0d65e98700 1 --2- 172.21.15.38:0/3458760530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d600f53b0 0x7f0d6005fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:14.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.224+0000 7f0d65697700 1 --2- 172.21.15.38:0/3458760530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d600f53b0 0x7f0d6005fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:14.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.224+0000 7f0d65697700 1 -- 172.21.15.38:0/3458760530 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d5c005040 con 0x7f0d600f5cc0 2024-02-15T18:44:14.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.225+0000 7f0d65697700 1 --2- 172.21.15.38:0/3458760530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d600f5cc0 0x7f0d6010ba00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f0d5400c530 tx=0x7f0d54007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:14.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.225+0000 7f0d527fc700 1 -- 172.21.15.38:0/3458760530 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d5400f650 con 0x7f0d600f5cc0 2024-02-15T18:44:14.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.225+0000 7f0d527fc700 1 -- 172.21.15.38:0/3458760530 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f0d5400fca0 con 0x7f0d600f5cc0 2024-02-15T18:44:14.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.225+0000 7f0d680fc700 1 -- 172.21.15.38:0/3458760530 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d6010c2c0 con 0x7f0d600f5cc0 2024-02-15T18:44:14.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.225+0000 7f0d527fc700 1 -- 172.21.15.38:0/3458760530 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d54012e10 con 0x7f0d600f5cc0 2024-02-15T18:44:14.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.225+0000 7f0d680fc700 1 -- 172.21.15.38:0/3458760530 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d6010c880 con 0x7f0d600f5cc0 2024-02-15T18:44:14.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.226+0000 7f0d680fc700 1 -- 172.21.15.38:0/3458760530 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d60027a00 con 0x7f0d600f5cc0 2024-02-15T18:44:14.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.226+0000 7f0d527fc700 1 -- 172.21.15.38:0/3458760530 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f0d54010070 con 0x7f0d600f5cc0 2024-02-15T18:44:14.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.226+0000 7f0d527fc700 1 --2- 172.21.15.38:0/3458760530 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f0d4c060fe0 0x7f0d4c0634a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:14.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.227+0000 7f0d527fc700 1 -- 172.21.15.38:0/3458760530 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f0d5407be90 con 0x7f0d600f5cc0 2024-02-15T18:44:14.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.227+0000 7f0d65e98700 1 --2- 172.21.15.38:0/3458760530 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f0d4c060fe0 0x7f0d4c0634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:14.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.227+0000 7f0d65e98700 1 --2- 172.21.15.38:0/3458760530 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f0d4c060fe0 0x7f0d4c0634a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f0d5c009d60 tx=0x7f0d5c009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:14.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.231+0000 7f0d527fc700 1 -- 172.21.15.38:0/3458760530 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0d54047f60 con 0x7f0d600f5cc0 2024-02-15T18:44:14.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.401+0000 7f0d680fc700 1 -- 172.21.15.38:0/3458760530 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d600586e0 con 0x7f0d600f5cc0 2024-02-15T18:44:14.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.402+0000 7f0d527fc700 1 -- 172.21.15.38:0/3458760530 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+14899 (secure 0 0 0) 0x7f0d5404fcf0 con 0x7f0d600f5cc0 2024-02-15T18:44:14.391 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:14.391 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":80,"fsid":"7e05753e-cc2f-11ee-95ba-87774f69a715","created":"2024-02-15T18:25:33.849561+0000","modified":"2024-02-15T18:41:18.691680+0000","last_up_change":"2024-02-15T18:40:07.070626+0000","last_in_change":"2024-02-15T18:30:56.975140+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-15T18:30:09.268924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-15T18:32:45.509930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-15T18:32:46.291705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"eee9713b-ff79-4169-8c04-b1def80d2045","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":46,"up_from":50,"up_thru":75,"down_at":47,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4082243527},{"type":"v1","addr":"172.21.15.38:6803","nonce":4082243527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4082243527},{"type":"v1","addr":"172.21.15.38:6805","nonce":4082243527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4082243527},{"type":"v1","addr":"172.21.15.38:6809","nonce":4082243527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4082243527},{"type":"v1","addr":"172.21.15.38:6807","nonce":4082243527}]},"public_addr":"172.21.15.38:6803/4082243527","cluster_addr":"172.21.15.38:6805/4082243527","heartbeat_back_addr":"172.21.15.38:6809/4082243527","heartbeat_front_addr":"172.21.15.38:6807/4082243527","state":["exists","up"]},{"osd":1,"uuid":"740eed48-d34e-4e71-a859-035f01b1fdd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":51,"up_from":55,"up_thru":75,"down_at":52,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":341024688},{"type":"v1","addr":"172.21.15.38:6811","nonce":341024688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":341024688},{"type":"v1","addr":"172.21.15.38:6813","nonce":341024688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":341024688},{"type":"v1","addr":"172.21.15.38:6817","nonce":341024688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":341024688},{"type":"v1","addr":"172.21.15.38:6815","nonce":341024688}]},"public_addr":"172.21.15.38:6811/341024688","cluster_addr":"172.21.15.38:6813/341024688","heartbeat_back_addr":"172.21.15.38:6817/341024688","heartbeat_front_addr":"172.21.15.38:6815/341024688","state":["exists","up"]},{"osd":2,"uuid":"8a397970-64c0-47d7-94c8-77fcf9fe579d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":56,"up_from":60,"up_thru":75,"down_at":57,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":209130240},{"type":"v1","addr":"172.21.15.38:6819","nonce":209130240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":209130240},{"type":"v1","addr":"172.21.15.38:6821","nonce":209130240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":209130240},{"type":"v1","addr":"172.21.15.38:6825","nonce":209130240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":209130240},{"type":"v1","addr":"172.21.15.38:6823","nonce":209130240}]},"public_addr":"172.21.15.38:6819/209130240","cluster_addr":"172.21.15.38:6821/209130240","heartbeat_back_addr":"172.21.15.38:6825/209130240","heartbeat_front_addr":"172.21.15.38:6823/209130240","state":["exists","up"]},{"osd":3,"uuid":"0a590f02-56f3-43e7-894f-741f95d2facd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":61,"up_from":65,"up_thru":75,"down_at":62,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1455526151},{"type":"v1","addr":"172.21.15.121:6801","nonce":1455526151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1455526151},{"type":"v1","addr":"172.21.15.121:6803","nonce":1455526151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1455526151},{"type":"v1","addr":"172.21.15.121:6807","nonce":1455526151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1455526151},{"type":"v1","addr":"172.21.15.121:6805","nonce":1455526151}]},"public_addr":"172.21.15.121:6801/1455526151","cluster_addr":"172.21.15.121:6803/1455526151","heartbeat_back_addr":"172.21.15.121:6807/1455526151","heartbeat_front_addr":"172.21.15.121:6805/1455526151","state":["exists","up"]},{"osd":4,"uuid":"5e13d4f8-2233-4674-9b52-a3fb9dde34ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":32,"last_clean_end":66,"up_from":70,"up_thru":75,"down_at":67,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1905463927},{"type":"v1","addr":"172.21.15.121:6809","nonce":1905463927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1905463927},{"type":"v1","addr":"172.21.15.121:6811","nonce":1905463927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1905463927},{"type":"v1","addr":"172.21.15.121:6815","nonce":1905463927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1905463927},{"type":"v1","addr":"172.21.15.121:6813","nonce":1905463927}]},"public_addr":"172.21.15.121:6809/1905463927","cluster_addr":"172.21.15.121:6811/1905463927","heartbeat_back_addr":"172.21.15.121:6815/1905463927","heartbeat_front_addr":"172.21.15.121:6813/1905463927","state":["exists","up"]},{"osd":5,"uuid":"6cd02c52-b9f4-494e-b67e-7019e4c224af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":37,"last_clean_end":71,"up_from":75,"up_thru":75,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3030235922},{"type":"v1","addr":"172.21.15.121:6817","nonce":3030235922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3030235922},{"type":"v1","addr":"172.21.15.121:6819","nonce":3030235922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3030235922},{"type":"v1","addr":"172.21.15.121:6823","nonce":3030235922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3030235922},{"type":"v1","addr":"172.21.15.121:6821","nonce":3030235922}]},"public_addr":"172.21.15.121:6817/3030235922","cluster_addr":"172.21.15.121:6819/3030235922","heartbeat_back_addr":"172.21.15.121:6823/3030235922","heartbeat_front_addr":"172.21.15.121:6821/3030235922","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:29:06.518109+0000","dead_epoch":47},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:29:40.582228+0000","dead_epoch":52},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:30:07.799238+0000","dead_epoch":57},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:30:29.057360+0000","dead_epoch":62},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:30:51.020867+0000","dead_epoch":67},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:31:13.005231+0000","dead_epoch":72}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6825/1422707571":"2024-02-16T18:41:03.437341+0000","172.21.15.121:6824/1422707571":"2024-02-16T18:41:03.437341+0000","172.21.15.38:6801/4189944318":"2024-02-16T18:26:02.772407+0000","172.21.15.38:6800/4189944318":"2024-02-16T18:26:02.772407+0000","172.21.15.38:0/2413778685":"2024-02-16T18:26:23.102293+0000","172.21.15.38:0/1003542778":"2024-02-16T18:35:17.589056+0000","172.21.15.38:0/2424130559":"2024-02-16T18:35:17.589056+0000","172.21.15.38:0/4035950047":"2024-02-16T18:26:02.772407+0000","172.21.15.38:0/788545352":"2024-02-16T18:26:23.102293+0000","172.21.15.121:6828/330429733":"2024-02-16T18:36:01.922927+0000","172.21.15.38:6800/2095530547":"2024-02-16T18:26:40.033295+0000","172.21.15.121:0/3752917867":"2024-02-16T18:36:01.922927+0000","172.21.15.38:0/1774333839":"2024-02-16T18:26:40.033295+0000","172.21.15.38:0/3518675152":"2024-02-16T18:26:40.033295+0000","172.21.15.121:0/1457708352":"2024-02-16T18:36:01.922927+0000","172.21.15.38:6801/2095530547":"2024-02-16T18:26:40.033295+0000","172.21.15.38:6800/3009942445":"2024-02-16T18:26:23.102293+0000","172.21.15.38:0/3436541874":"2024-02-16T18:26:02.772407+0000","172.21.15.121:0/1860609213":"2024-02-16T18:36:01.922927+0000","172.21.15.121:0/3029797785":"2024-02-16T18:36:01.922927+0000","172.21.15.38:6801/3009942445":"2024-02-16T18:26:23.102293+0000","172.21.15.38:6800/2900497610":"2024-02-16T18:35:17.589056+0000","172.21.15.121:6829/330429733":"2024-02-16T18:36:01.922927+0000","172.21.15.38:6801/2900497610":"2024-02-16T18:35:17.589056+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-15T18:44:14.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.404+0000 7f0d680fc700 1 -- 172.21.15.38:0/3458760530 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f0d4c060fe0 msgr2=0x7f0d4c0634a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:14.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.404+0000 7f0d680fc700 1 --2- 172.21.15.38:0/3458760530 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f0d4c060fe0 0x7f0d4c0634a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f0d5c009d60 tx=0x7f0d5c009b30).stop 2024-02-15T18:44:14.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.404+0000 7f0d680fc700 1 -- 172.21.15.38:0/3458760530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d600f5cc0 msgr2=0x7f0d6010ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:14.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.404+0000 7f0d680fc700 1 --2- 172.21.15.38:0/3458760530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d600f5cc0 0x7f0d6010ba00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f0d5400c530 tx=0x7f0d54007e20).stop 2024-02-15T18:44:14.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.404+0000 7f0d680fc700 1 -- 172.21.15.38:0/3458760530 shutdown_connections 2024-02-15T18:44:14.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.404+0000 7f0d680fc700 1 --2- 172.21.15.38:0/3458760530 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f0d4c060fe0 0x7f0d4c0634a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:14.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.404+0000 7f0d680fc700 1 --2- 172.21.15.38:0/3458760530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d600f53b0 0x7f0d6005fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:14.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.404+0000 7f0d680fc700 1 --2- 172.21.15.38:0/3458760530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d600f5cc0 0x7f0d6010ba00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:14.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.404+0000 7f0d680fc700 1 -- 172.21.15.38:0/3458760530 >> 172.21.15.38:0/3458760530 conn(0x7f0d600f0bd0 msgr2=0x7f0d600fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:14.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.404+0000 7f0d680fc700 1 -- 172.21.15.38:0/3458760530 shutdown_connections 2024-02-15T18:44:14.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.404+0000 7f0d680fc700 1 -- 172.21.15.38:0/3458760530 wait complete. 2024-02-15T18:44:14.405 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-15T18:30:09.268924+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-15T18:32:45.509930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-15T18:32:46.291705+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-15T18:44:14.405 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-02-15T18:44:14.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.540+0000 7f517d358700 1 -- 172.21.15.38:0/2131097500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51780f6490 msgr2=0x7f51780f68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:14.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.540+0000 7f517d358700 1 --2- 172.21.15.38:0/2131097500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51780f6490 0x7f51780f68b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f5168004660 tx=0x7f5168009b30).stop 2024-02-15T18:44:14.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.540+0000 7f517d358700 1 -- 172.21.15.38:0/2131097500 shutdown_connections 2024-02-15T18:44:14.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.540+0000 7f517d358700 1 --2- 172.21.15.38:0/2131097500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51780f7730 0x7f51780f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:14.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.540+0000 7f517d358700 1 --2- 172.21.15.38:0/2131097500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51780f6490 0x7f51780f68b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:14.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.540+0000 7f517d358700 1 -- 172.21.15.38:0/2131097500 >> 172.21.15.38:0/2131097500 conn(0x7f51780f1bd0 msgr2=0x7f51780f4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:14.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.540+0000 7f517d358700 1 -- 172.21.15.38:0/2131097500 shutdown_connections 2024-02-15T18:44:14.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.540+0000 7f517d358700 1 -- 172.21.15.38:0/2131097500 wait complete. 2024-02-15T18:44:14.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.541+0000 7f517d358700 1 Processor -- start 2024-02-15T18:44:14.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.541+0000 7f517d358700 1 -- start start 2024-02-15T18:44:14.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.541+0000 7f517d358700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51780f6490 0x7f5178106d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:14.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.541+0000 7f517d358700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51780f7730 0x7f51781071f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:14.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.541+0000 7f517d358700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51781077f0 con 0x7f51780f6490 2024-02-15T18:44:14.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.541+0000 7f517d358700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5178107930 con 0x7f51780f7730 2024-02-15T18:44:14.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.541+0000 7f5176ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51780f6490 0x7f5178106d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:14.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.541+0000 7f5176ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51780f6490 0x7f5178106d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59196/0 (socket says 172.21.15.38:59196) 2024-02-15T18:44:14.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.541+0000 7f5176ffd700 1 -- 172.21.15.38:0/284925094 learned_addr learned my addr 172.21.15.38:0/284925094 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:14.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.541+0000 7f5176ffd700 1 -- 172.21.15.38:0/284925094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51780f7730 msgr2=0x7f51781071f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:14.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.541+0000 7f5176ffd700 1 --2- 172.21.15.38:0/284925094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51780f7730 0x7f51781071f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:44:14.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.541+0000 7f5176ffd700 1 -- 172.21.15.38:0/284925094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5168005040 con 0x7f51780f6490 2024-02-15T18:44:14.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.541+0000 7f5176ffd700 1 --2- 172.21.15.38:0/284925094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51780f6490 0x7f5178106d00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f5168013fd0 tx=0x7f5168009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:14.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.541+0000 7f516f7fe700 1 -- 172.21.15.38:0/284925094 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5168018070 con 0x7f51780f6490 2024-02-15T18:44:14.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.541+0000 7f516f7fe700 1 -- 172.21.15.38:0/284925094 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f5168007d70 con 0x7f51780f6490 2024-02-15T18:44:14.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.541+0000 7f517d358700 1 -- 172.21.15.38:0/284925094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517810c2e0 con 0x7f51780f6490 2024-02-15T18:44:14.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.541+0000 7f516f7fe700 1 -- 172.21.15.38:0/284925094 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f516801c620 con 0x7f51780f6490 2024-02-15T18:44:14.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.542+0000 7f517d358700 1 -- 172.21.15.38:0/284925094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517810c840 con 0x7f51780f6490 2024-02-15T18:44:14.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.542+0000 7f517d358700 1 -- 172.21.15.38:0/284925094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5178027a00 con 0x7f51780f6490 2024-02-15T18:44:14.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.543+0000 7f516f7fe700 1 -- 172.21.15.38:0/284925094 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f516801d420 con 0x7f51780f6490 2024-02-15T18:44:14.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.543+0000 7f516f7fe700 1 --2- 172.21.15.38:0/284925094 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5164060f20 0x7f51640633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:14.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.543+0000 7f516f7fe700 1 -- 172.21.15.38:0/284925094 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f516807e090 con 0x7f51780f6490 2024-02-15T18:44:14.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.543+0000 7f51767fc700 1 --2- 172.21.15.38:0/284925094 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5164060f20 0x7f51640633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:14.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.544+0000 7f51767fc700 1 --2- 172.21.15.38:0/284925094 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5164060f20 0x7f51640633e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f51600047d0 tx=0x7f5160007a40).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:14.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.547+0000 7f516f7fe700 1 -- 172.21.15.38:0/284925094 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f516804e470 con 0x7f51780f6490 2024-02-15T18:44:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.719+0000 7f517d358700 1 -- 172.21.15.38:0/284925094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f51780fed60 con 0x7f51780f6490 2024-02-15T18:44:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.720+0000 7f516f7fe700 1 -- 172.21.15.38:0/284925094 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v80) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5168051a90 con 0x7f51780f6490 2024-02-15T18:44:14.709 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-02-15T18:44:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.722+0000 7f517d358700 1 -- 172.21.15.38:0/284925094 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5164060f20 msgr2=0x7f51640633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.722+0000 7f517d358700 1 --2- 172.21.15.38:0/284925094 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5164060f20 0x7f51640633e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f51600047d0 tx=0x7f5160007a40).stop 2024-02-15T18:44:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.722+0000 7f517d358700 1 -- 172.21.15.38:0/284925094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51780f6490 msgr2=0x7f5178106d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.722+0000 7f517d358700 1 --2- 172.21.15.38:0/284925094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51780f6490 0x7f5178106d00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f5168013fd0 tx=0x7f5168009b30).stop 2024-02-15T18:44:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.722+0000 7f517d358700 1 -- 172.21.15.38:0/284925094 shutdown_connections 2024-02-15T18:44:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.722+0000 7f517d358700 1 --2- 172.21.15.38:0/284925094 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5164060f20 0x7f51640633e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.722+0000 7f517d358700 1 --2- 172.21.15.38:0/284925094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51780f6490 0x7f5178106d00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.722+0000 7f517d358700 1 --2- 172.21.15.38:0/284925094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51780f7730 0x7f51781071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:44:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.722+0000 7f517d358700 1 -- 172.21.15.38:0/284925094 >> 172.21.15.38:0/284925094 conn(0x7f51780f1bd0 msgr2=0x7f51780fb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.722+0000 7f517d358700 1 -- 172.21.15.38:0/284925094 shutdown_connections 2024-02-15T18:44:14.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.722+0000 7f517d358700 1 -- 172.21.15.38:0/284925094 wait complete. 2024-02-15T18:44:14.721 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:14 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:44:13.043374+0000 mgr.smithi038.khesbl (mgr.14664) 412 : cluster [DBG] pgmap v283: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:14.722 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-15T18:44:14.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.856+0000 7fe098e29700 1 -- 172.21.15.38:0/3980172504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0940f5390 msgr2=0x7fe0940f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:14.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.856+0000 7fe098e29700 1 --2- 172.21.15.38:0/3980172504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0940f5390 0x7fe0940f57b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fe084004660 tx=0x7fe084009b30).stop 2024-02-15T18:44:14.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.856+0000 7fe098e29700 1 -- 172.21.15.38:0/3980172504 shutdown_connections 2024-02-15T18:44:14.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.856+0000 7fe098e29700 1 --2- 172.21.15.38:0/3980172504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0940f5ca0 0x7fe0940f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:14.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.856+0000 7fe098e29700 1 --2- 172.21.15.38:0/3980172504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0940f5390 0x7fe0940f57b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:14.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.856+0000 7fe098e29700 1 -- 172.21.15.38:0/3980172504 >> 172.21.15.38:0/3980172504 conn(0x7fe0940f0b70 msgr2=0x7fe0940f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.856+0000 7fe098e29700 1 -- 172.21.15.38:0/3980172504 shutdown_connections 2024-02-15T18:44:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.856+0000 7fe098e29700 1 -- 172.21.15.38:0/3980172504 wait complete. 2024-02-15T18:44:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.857+0000 7fe098e29700 1 Processor -- start 2024-02-15T18:44:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.857+0000 7fe098e29700 1 -- start start 2024-02-15T18:44:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.857+0000 7fe098e29700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0940f5390 0x7fe094108eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.857+0000 7fe098e29700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0940f5ca0 0x7fe0941093a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.857+0000 7fe098e29700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe094109950 con 0x7fe0940f5390 2024-02-15T18:44:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.857+0000 7fe098e29700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe094109a90 con 0x7fe0940f5ca0 2024-02-15T18:44:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.857+0000 7fe09259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0940f5390 0x7fe094108eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.857+0000 7fe09259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0940f5390 0x7fe094108eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59214/0 (socket says 172.21.15.38:59214) 2024-02-15T18:44:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.857+0000 7fe09259c700 1 -- 172.21.15.38:0/2549164072 learned_addr learned my addr 172.21.15.38:0/2549164072 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.857+0000 7fe091d9b700 1 --2- 172.21.15.38:0/2549164072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0940f5ca0 0x7fe0941093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.857+0000 7fe09259c700 1 -- 172.21.15.38:0/2549164072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0940f5ca0 msgr2=0x7fe0941093a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.857+0000 7fe09259c700 1 --2- 172.21.15.38:0/2549164072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0940f5ca0 0x7fe0941093a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.857+0000 7fe09259c700 1 -- 172.21.15.38:0/2549164072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe084005040 con 0x7fe0940f5390 2024-02-15T18:44:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.857+0000 7fe091d9b700 1 --2- 172.21.15.38:0/2549164072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0940f5ca0 0x7fe0941093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:44:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.857+0000 7fe09259c700 1 --2- 172.21.15.38:0/2549164072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0940f5390 0x7fe094108eb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fe084009f50 tx=0x7fe084009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.858+0000 7fe08affd700 1 -- 172.21.15.38:0/2549164072 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe084018070 con 0x7fe0940f5390 2024-02-15T18:44:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.858+0000 7fe08affd700 1 -- 172.21.15.38:0/2549164072 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fe084007d70 con 0x7fe0940f5390 2024-02-15T18:44:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.858+0000 7fe098e29700 1 -- 172.21.15.38:0/2549164072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe09410e4d0 con 0x7fe0940f5390 2024-02-15T18:44:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.858+0000 7fe08affd700 1 -- 172.21.15.38:0/2549164072 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe08401c620 con 0x7fe0940f5390 2024-02-15T18:44:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.858+0000 7fe098e29700 1 -- 172.21.15.38:0/2549164072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09410ea90 con 0x7fe0940f5390 2024-02-15T18:44:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.858+0000 7fe098e29700 1 -- 172.21.15.38:0/2549164072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0940fdc90 con 0x7fe0940f5390 2024-02-15T18:44:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.859+0000 7fe08affd700 1 -- 172.21.15.38:0/2549164072 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fe08401c890 con 0x7fe0940f5390 2024-02-15T18:44:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.859+0000 7fe08affd700 1 --2- 172.21.15.38:0/2549164072 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe080060f20 0x7fe0800633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.859+0000 7fe08affd700 1 -- 172.21.15.38:0/2549164072 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fe08407dfc0 con 0x7fe0940f5390 2024-02-15T18:44:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.860+0000 7fe091d9b700 1 --2- 172.21.15.38:0/2549164072 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe080060f20 0x7fe0800633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.860+0000 7fe091d9b700 1 --2- 172.21.15.38:0/2549164072 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe080060f20 0x7fe0800633e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fe07c0041d0 tx=0x7fe07c009040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:14.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:14.864+0000 7fe08affd700 1 -- 172.21.15.38:0/2549164072 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe084051ea0 con 0x7fe0940f5390 2024-02-15T18:44:14.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:14 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:44:13.043374+0000 mgr.smithi038.khesbl (mgr.14664) 412 : cluster [DBG] pgmap v283: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:15.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.033+0000 7fe098e29700 1 -- 172.21.15.38:0/2549164072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fe09410f120 con 0x7fe0940f5390 2024-02-15T18:44:15.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.033+0000 7fe08affd700 1 -- 172.21.15.38:0/2549164072 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v80) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe084051a90 con 0x7fe0940f5390 2024-02-15T18:44:15.022 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-02-15T18:44:15.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.035+0000 7fe098e29700 1 -- 172.21.15.38:0/2549164072 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe080060f20 msgr2=0x7fe0800633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:15.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.035+0000 7fe098e29700 1 --2- 172.21.15.38:0/2549164072 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe080060f20 0x7fe0800633e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fe07c0041d0 tx=0x7fe07c009040).stop 2024-02-15T18:44:15.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.035+0000 7fe098e29700 1 -- 172.21.15.38:0/2549164072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0940f5390 msgr2=0x7fe094108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:15.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.035+0000 7fe098e29700 1 --2- 172.21.15.38:0/2549164072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0940f5390 0x7fe094108eb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fe084009f50 tx=0x7fe084009b30).stop 2024-02-15T18:44:15.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.035+0000 7fe098e29700 1 -- 172.21.15.38:0/2549164072 shutdown_connections 2024-02-15T18:44:15.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.035+0000 7fe098e29700 1 --2- 172.21.15.38:0/2549164072 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fe080060f20 0x7fe0800633e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:15.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.035+0000 7fe098e29700 1 --2- 172.21.15.38:0/2549164072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0940f5390 0x7fe094108eb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:15.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.035+0000 7fe098e29700 1 --2- 172.21.15.38:0/2549164072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0940f5ca0 0x7fe0941093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:15.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.035+0000 7fe098e29700 1 -- 172.21.15.38:0/2549164072 >> 172.21.15.38:0/2549164072 conn(0x7fe0940f0b70 msgr2=0x7fe09405fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:15.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.035+0000 7fe098e29700 1 -- 172.21.15.38:0/2549164072 shutdown_connections 2024-02-15T18:44:15.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.035+0000 7fe098e29700 1 -- 172.21.15.38:0/2549164072 wait complete. 2024-02-15T18:44:15.036 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-15T18:44:15.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.170+0000 7fdcd30c6700 1 -- 172.21.15.38:0/78781608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdccc0f96a0 msgr2=0x7fdccc0fba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:15.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.170+0000 7fdcd30c6700 1 --2- 172.21.15.38:0/78781608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdccc0f96a0 0x7fdccc0fba90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fdcb8004660 tx=0x7fdcb8009b30).stop 2024-02-15T18:44:15.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.170+0000 7fdcd30c6700 1 -- 172.21.15.38:0/78781608 shutdown_connections 2024-02-15T18:44:15.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.170+0000 7fdcd30c6700 1 --2- 172.21.15.38:0/78781608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdccc0f96a0 0x7fdccc0fba90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:15.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.170+0000 7fdcd30c6700 1 --2- 172.21.15.38:0/78781608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdccc05fd90 0x7fdccc0f91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:15.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.170+0000 7fdcd30c6700 1 -- 172.21.15.38:0/78781608 >> 172.21.15.38:0/78781608 conn(0x7fdccc0f0b70 msgr2=0x7fdccc0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:15.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.170+0000 7fdcd30c6700 1 -- 172.21.15.38:0/78781608 shutdown_connections 2024-02-15T18:44:15.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.170+0000 7fdcd30c6700 1 -- 172.21.15.38:0/78781608 wait complete. 2024-02-15T18:44:15.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.171+0000 7fdcd30c6700 1 Processor -- start 2024-02-15T18:44:15.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.171+0000 7fdcd30c6700 1 -- start start 2024-02-15T18:44:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.171+0000 7fdcd30c6700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdccc05fd90 0x7fdccc106c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.171+0000 7fdcd30c6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdccc0f96a0 0x7fdccc107170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.171+0000 7fdcd30c6700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc107770 con 0x7fdccc05fd90 2024-02-15T18:44:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.171+0000 7fdcd30c6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc1078e0 con 0x7fdccc0f96a0 2024-02-15T18:44:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.171+0000 7fdcd0e62700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdccc05fd90 0x7fdccc106c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.171+0000 7fdcd0e62700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdccc05fd90 0x7fdccc106c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59226/0 (socket says 172.21.15.38:59226) 2024-02-15T18:44:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.171+0000 7fdcd0e62700 1 -- 172.21.15.38:0/2405785595 learned_addr learned my addr 172.21.15.38:0/2405785595 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.171+0000 7fdccbfff700 1 --2- 172.21.15.38:0/2405785595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdccc0f96a0 0x7fdccc107170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.172+0000 7fdcd0e62700 1 -- 172.21.15.38:0/2405785595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdccc0f96a0 msgr2=0x7fdccc107170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.172+0000 7fdcd0e62700 1 --2- 172.21.15.38:0/2405785595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdccc0f96a0 0x7fdccc107170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.172+0000 7fdcd0e62700 1 -- 172.21.15.38:0/2405785595 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb8005040 con 0x7fdccc05fd90 2024-02-15T18:44:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.172+0000 7fdcd0e62700 1 --2- 172.21.15.38:0/2405785595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdccc05fd90 0x7fdccc106c80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fdcc0002700 tx=0x7fdcc0009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.172+0000 7fdcc97fa700 1 -- 172.21.15.38:0/2405785595 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcc00079c0 con 0x7fdccc05fd90 2024-02-15T18:44:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.172+0000 7fdcc97fa700 1 -- 172.21.15.38:0/2405785595 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fdcc000f040 con 0x7fdccc05fd90 2024-02-15T18:44:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.172+0000 7fdcd30c6700 1 -- 172.21.15.38:0/2405785595 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdccc10c350 con 0x7fdccc05fd90 2024-02-15T18:44:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.172+0000 7fdcc97fa700 1 -- 172.21.15.38:0/2405785595 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcc0014940 con 0x7fdccc05fd90 2024-02-15T18:44:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.172+0000 7fdcd30c6700 1 -- 172.21.15.38:0/2405785595 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdccc10c8e0 con 0x7fdccc05fd90 2024-02-15T18:44:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.173+0000 7fdcd30c6700 1 -- 172.21.15.38:0/2405785595 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdccc046330 con 0x7fdccc05fd90 2024-02-15T18:44:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.173+0000 7fdcc97fa700 1 -- 172.21.15.38:0/2405785595 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fdcc001b060 con 0x7fdccc05fd90 2024-02-15T18:44:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.174+0000 7fdcc97fa700 1 --2- 172.21.15.38:0/2405785595 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fdcbc060f20 0x7fdcbc0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.174+0000 7fdcc97fa700 1 -- 172.21.15.38:0/2405785595 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fdcc007bf70 con 0x7fdccc05fd90 2024-02-15T18:44:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.174+0000 7fdccbfff700 1 --2- 172.21.15.38:0/2405785595 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fdcbc060f20 0x7fdcbc0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.174+0000 7fdccbfff700 1 --2- 172.21.15.38:0/2405785595 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fdcbc060f20 0x7fdcbc0633e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fdcb8004630 tx=0x7fdcb8009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:15.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.178+0000 7fdcc97fa700 1 -- 172.21.15.38:0/2405785595 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fdcc004c3a0 con 0x7fdccc05fd90 2024-02-15T18:44:15.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.349+0000 7fdcd30c6700 1 -- 172.21.15.38:0/2405785595 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fdccc058780 con 0x7fdccc05fd90 2024-02-15T18:44:15.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.349+0000 7fdcc97fa700 1 -- 172.21.15.38:0/2405785595 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v80) v1 ==== 107+0+11 (secure 0 0 0) 0x7fdcc004f9c0 con 0x7fdccc05fd90 2024-02-15T18:44:15.338 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-02-15T18:44:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.351+0000 7fdcd30c6700 1 -- 172.21.15.38:0/2405785595 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fdcbc060f20 msgr2=0x7fdcbc0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.351+0000 7fdcd30c6700 1 --2- 172.21.15.38:0/2405785595 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fdcbc060f20 0x7fdcbc0633e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fdcb8004630 tx=0x7fdcb8009b30).stop 2024-02-15T18:44:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.351+0000 7fdcd30c6700 1 -- 172.21.15.38:0/2405785595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdccc05fd90 msgr2=0x7fdccc106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.351+0000 7fdcd30c6700 1 --2- 172.21.15.38:0/2405785595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdccc05fd90 0x7fdccc106c80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fdcc0002700 tx=0x7fdcc0009d80).stop 2024-02-15T18:44:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.351+0000 7fdcd30c6700 1 -- 172.21.15.38:0/2405785595 shutdown_connections 2024-02-15T18:44:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.351+0000 7fdcd30c6700 1 --2- 172.21.15.38:0/2405785595 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fdcbc060f20 0x7fdcbc0633e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.351+0000 7fdcd30c6700 1 --2- 172.21.15.38:0/2405785595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdccc05fd90 0x7fdccc106c80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.351+0000 7fdcd30c6700 1 --2- 172.21.15.38:0/2405785595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdccc0f96a0 0x7fdccc107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.351+0000 7fdcd30c6700 1 -- 172.21.15.38:0/2405785595 >> 172.21.15.38:0/2405785595 conn(0x7fdccc0f0b70 msgr2=0x7fdccc0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.351+0000 7fdcd30c6700 1 -- 172.21.15.38:0/2405785595 shutdown_connections 2024-02-15T18:44:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.351+0000 7fdcd30c6700 1 -- 172.21.15.38:0/2405785595 wait complete. 2024-02-15T18:44:15.352 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-15T18:44:15.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.484+0000 7f8820446700 1 -- 172.21.15.38:0/3420503804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88180f5cc0 msgr2=0x7f88180f66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:15.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.484+0000 7f8820446700 1 --2- 172.21.15.38:0/3420503804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88180f5cc0 0x7f88180f66f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f8814004660 tx=0x7f8814009b30).stop 2024-02-15T18:44:15.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.484+0000 7f8820446700 1 -- 172.21.15.38:0/3420503804 shutdown_connections 2024-02-15T18:44:15.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.484+0000 7f8820446700 1 --2- 172.21.15.38:0/3420503804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88180f5cc0 0x7f88180f66f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:15.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.484+0000 7f8820446700 1 --2- 172.21.15.38:0/3420503804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88180f53b0 0x7f88180f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:15.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.484+0000 7f8820446700 1 -- 172.21.15.38:0/3420503804 >> 172.21.15.38:0/3420503804 conn(0x7f88180f0bd0 msgr2=0x7f88180f2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:15.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.484+0000 7f8820446700 1 -- 172.21.15.38:0/3420503804 shutdown_connections 2024-02-15T18:44:15.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.484+0000 7f8820446700 1 -- 172.21.15.38:0/3420503804 wait complete. 2024-02-15T18:44:15.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.485+0000 7f8820446700 1 Processor -- start 2024-02-15T18:44:15.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.485+0000 7f8820446700 1 -- start start 2024-02-15T18:44:15.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.485+0000 7f8820446700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88180f53b0 0x7f8818106c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:15.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.485+0000 7f8820446700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88180f5cc0 0x7f8818107130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:15.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.485+0000 7f8820446700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8818107730 con 0x7f88180f5cc0 2024-02-15T18:44:15.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.485+0000 7f8820446700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8818107870 con 0x7f88180f53b0 2024-02-15T18:44:15.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.485+0000 7f881d9e1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88180f5cc0 0x7f8818107130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:15.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.485+0000 7f881d9e1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88180f5cc0 0x7f8818107130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59234/0 (socket says 172.21.15.38:59234) 2024-02-15T18:44:15.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.485+0000 7f881d9e1700 1 -- 172.21.15.38:0/761189161 learned_addr learned my addr 172.21.15.38:0/761189161 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:15.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.485+0000 7f881e1e2700 1 --2- 172.21.15.38:0/761189161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88180f53b0 0x7f8818106c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:15.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.485+0000 7f881d9e1700 1 -- 172.21.15.38:0/761189161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88180f53b0 msgr2=0x7f8818106c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:15.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.485+0000 7f881d9e1700 1 --2- 172.21.15.38:0/761189161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88180f53b0 0x7f8818106c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:15.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.485+0000 7f881d9e1700 1 -- 172.21.15.38:0/761189161 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8814005040 con 0x7f88180f5cc0 2024-02-15T18:44:15.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.486+0000 7f881d9e1700 1 --2- 172.21.15.38:0/761189161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88180f5cc0 0x7f8818107130 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f8814000c00 tx=0x7f881400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:15.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.486+0000 7f880affd700 1 -- 172.21.15.38:0/761189161 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8814019070 con 0x7f88180f5cc0 2024-02-15T18:44:15.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.486+0000 7f880affd700 1 -- 172.21.15.38:0/761189161 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f8814007e80 con 0x7f88180f5cc0 2024-02-15T18:44:15.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.486+0000 7f880affd700 1 -- 172.21.15.38:0/761189161 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8814014db0 con 0x7f88180f5cc0 2024-02-15T18:44:15.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.486+0000 7f8820446700 1 -- 172.21.15.38:0/761189161 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881810c220 con 0x7f88180f5cc0 2024-02-15T18:44:15.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.486+0000 7f8820446700 1 -- 172.21.15.38:0/761189161 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f881810c780 con 0x7f88180f5cc0 2024-02-15T18:44:15.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.487+0000 7f8820446700 1 -- 172.21.15.38:0/761189161 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8818027a00 con 0x7f88180f5cc0 2024-02-15T18:44:15.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.487+0000 7f880affd700 1 -- 172.21.15.38:0/761189161 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f8814025070 con 0x7f88180f5cc0 2024-02-15T18:44:15.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.487+0000 7f880affd700 1 --2- 172.21.15.38:0/761189161 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8804060f80 0x7f8804063440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:15.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.487+0000 7f880affd700 1 -- 172.21.15.38:0/761189161 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f881407d090 con 0x7f88180f5cc0 2024-02-15T18:44:15.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.488+0000 7f881e1e2700 1 --2- 172.21.15.38:0/761189161 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8804060f80 0x7f8804063440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:15.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.488+0000 7f881e1e2700 1 --2- 172.21.15.38:0/761189161 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8804060f80 0x7f8804063440 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f8818108130 tx=0x7f880c00b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:15.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.492+0000 7f880affd700 1 -- 172.21.15.38:0/761189161 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f881404d4d0 con 0x7f88180f5cc0 2024-02-15T18:44:15.604 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:14.403207+0000 mon.smithi038 (mon.0) 736 : audit [DBG] from='client.? 172.21.15.38:0/3458760530' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-15T18:44:15.604 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:14.721477+0000 mon.smithi038 (mon.0) 737 : audit [DBG] from='client.? 172.21.15.38:0/284925094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-15T18:44:15.604 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:15.034783+0000 mon.smithi038 (mon.0) 738 : audit [DBG] from='client.? 172.21.15.38:0/2549164072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-15T18:44:15.604 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:15 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:15.351084+0000 mon.smithi038 (mon.0) 739 : audit [DBG] from='client.? 172.21.15.38:0/2405785595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-15T18:44:15.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.705+0000 7f8820446700 1 -- 172.21.15.38:0/761189161 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88180fedb0 con 0x7f88180f5cc0 2024-02-15T18:44:15.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.706+0000 7f880affd700 1 -- 172.21.15.38:0/761189161 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f8814023070 con 0x7f88180f5cc0 2024-02-15T18:44:15.695 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:15.695 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44261,"name":"cephfs.smithi121.mvumlh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6825/180558011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":180558011},{"type":"v1","addr":"172.21.15.121:6825","nonce":180558011}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44265,"name":"cephfs.smithi121.ovuzki","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6827/1075229339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1075229339},{"type":"v1","addr":"172.21.15.121:6827","nonce":1075229339}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:41:26.398387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34314,"mds_1":34318},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34314":{"gid":34314,"name":"cephfs.smithi038.hxoggl","rank":0,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.38:6827/4077527450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":4077527450},{"type":"v1","addr":"172.21.15.38:6827","nonce":4077527450}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34318":{"gid":34318,"name":"cephfs.smithi038.qgkzgn","rank":1,"incarnation":31,"state":"up:active","state_seq":9,"addr":"172.21.15.38:6829/1616551408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1616551408},{"type":"v1","addr":"172.21.15.38:6829","nonce":1616551408}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:15.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.708+0000 7f8820446700 1 -- 172.21.15.38:0/761189161 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8804060f80 msgr2=0x7f8804063440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:15.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.708+0000 7f8820446700 1 --2- 172.21.15.38:0/761189161 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8804060f80 0x7f8804063440 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f8818108130 tx=0x7f880c00b040).stop 2024-02-15T18:44:15.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.708+0000 7f8820446700 1 -- 172.21.15.38:0/761189161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88180f5cc0 msgr2=0x7f8818107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:15.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.708+0000 7f8820446700 1 --2- 172.21.15.38:0/761189161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88180f5cc0 0x7f8818107130 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f8814000c00 tx=0x7f881400b040).stop 2024-02-15T18:44:15.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.708+0000 7f8820446700 1 -- 172.21.15.38:0/761189161 shutdown_connections 2024-02-15T18:44:15.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.708+0000 7f8820446700 1 --2- 172.21.15.38:0/761189161 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8804060f80 0x7f8804063440 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:15.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.708+0000 7f8820446700 1 --2- 172.21.15.38:0/761189161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88180f53b0 0x7f8818106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:15.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.708+0000 7f8820446700 1 --2- 172.21.15.38:0/761189161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88180f5cc0 0x7f8818107130 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:15.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.708+0000 7f8820446700 1 -- 172.21.15.38:0/761189161 >> 172.21.15.38:0/761189161 conn(0x7f88180f0bd0 msgr2=0x7f88180fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:15.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.708+0000 7f8820446700 1 -- 172.21.15.38:0/761189161 shutdown_connections 2024-02-15T18:44:15.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.708+0000 7f8820446700 1 -- 172.21.15.38:0/761189161 wait complete. 2024-02-15T18:44:15.698 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 32 2024-02-15T18:44:15.707 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:14.403207+0000 mon.smithi038 (mon.0) 736 : audit [DBG] from='client.? 172.21.15.38:0/3458760530' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-15T18:44:15.707 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:14.721477+0000 mon.smithi038 (mon.0) 737 : audit [DBG] from='client.? 172.21.15.38:0/284925094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-15T18:44:15.707 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:15.034783+0000 mon.smithi038 (mon.0) 738 : audit [DBG] from='client.? 172.21.15.38:0/2549164072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-15T18:44:15.707 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:15 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:15.351084+0000 mon.smithi038 (mon.0) 739 : audit [DBG] from='client.? 172.21.15.38:0/2405785595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-15T18:44:15.708 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 10, 'max_mds': 2, 'flags': 18} 2024-02-15T18:44:15.709 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 11 2024-02-15T18:44:15.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.841+0000 7f3cb2931700 1 -- 172.21.15.38:0/2316832276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cac0f64e0 msgr2=0x7f3cac0f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:15.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.841+0000 7f3cb2931700 1 --2- 172.21.15.38:0/2316832276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cac0f64e0 0x7f3cac0f6900 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f3c9c004660 tx=0x7f3c9c009b30).stop 2024-02-15T18:44:15.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.841+0000 7f3cb2931700 1 -- 172.21.15.38:0/2316832276 shutdown_connections 2024-02-15T18:44:15.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.841+0000 7f3cb2931700 1 --2- 172.21.15.38:0/2316832276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cac0f7780 0x7f3cac0f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:15.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.841+0000 7f3cb2931700 1 --2- 172.21.15.38:0/2316832276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cac0f64e0 0x7f3cac0f6900 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:15.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.841+0000 7f3cb2931700 1 -- 172.21.15.38:0/2316832276 >> 172.21.15.38:0/2316832276 conn(0x7f3cac0f1c40 msgr2=0x7f3cac0f4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:15.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.841+0000 7f3cb2931700 1 -- 172.21.15.38:0/2316832276 shutdown_connections 2024-02-15T18:44:15.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.841+0000 7f3cb2931700 1 -- 172.21.15.38:0/2316832276 wait complete. 2024-02-15T18:44:15.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.842+0000 7f3cb2931700 1 Processor -- start 2024-02-15T18:44:15.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.842+0000 7f3cb2931700 1 -- start start 2024-02-15T18:44:15.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.842+0000 7f3cb2931700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cac0f64e0 0x7f3cac06b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:15.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.842+0000 7f3cb2931700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cac0f7780 0x7f3cac069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:15.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.842+0000 7f3cb2931700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac06bd70 con 0x7f3cac0f7780 2024-02-15T18:44:15.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.842+0000 7f3cb2931700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac06a310 con 0x7f3cac0f64e0 2024-02-15T18:44:15.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.842+0000 7f3cab7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cac0f7780 0x7f3cac069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:15.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.842+0000 7f3cab7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cac0f7780 0x7f3cac069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59250/0 (socket says 172.21.15.38:59250) 2024-02-15T18:44:15.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.842+0000 7f3cab7fe700 1 -- 172.21.15.38:0/1718754947 learned_addr learned my addr 172.21.15.38:0/1718754947 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:15.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.842+0000 7f3cabfff700 1 --2- 172.21.15.38:0/1718754947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cac0f64e0 0x7f3cac06b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:15.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.842+0000 7f3cab7fe700 1 -- 172.21.15.38:0/1718754947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cac0f64e0 msgr2=0x7f3cac06b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:15.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.842+0000 7f3cab7fe700 1 --2- 172.21.15.38:0/1718754947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cac0f64e0 0x7f3cac06b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:15.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.842+0000 7f3cab7fe700 1 -- 172.21.15.38:0/1718754947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c005040 con 0x7f3cac0f7780 2024-02-15T18:44:15.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.842+0000 7f3cab7fe700 1 --2- 172.21.15.38:0/1718754947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cac0f7780 0x7f3cac069e20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f3ca000c530 tx=0x7f3ca0007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:15.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.843+0000 7f3ca8ff9700 1 -- 172.21.15.38:0/1718754947 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ca000f650 con 0x7f3cac0f7780 2024-02-15T18:44:15.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.843+0000 7f3ca8ff9700 1 -- 172.21.15.38:0/1718754947 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f3ca000fca0 con 0x7f3cac0f7780 2024-02-15T18:44:15.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.843+0000 7f3cb2931700 1 -- 172.21.15.38:0/1718754947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cac06a590 con 0x7f3cac0f7780 2024-02-15T18:44:15.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.843+0000 7f3ca8ff9700 1 -- 172.21.15.38:0/1718754947 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ca0012d40 con 0x7f3cac0f7780 2024-02-15T18:44:15.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.843+0000 7f3cb2931700 1 -- 172.21.15.38:0/1718754947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cac06ab50 con 0x7f3cac0f7780 2024-02-15T18:44:15.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.844+0000 7f3cb2931700 1 -- 172.21.15.38:0/1718754947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cac046330 con 0x7f3cac0f7780 2024-02-15T18:44:15.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.844+0000 7f3ca8ff9700 1 -- 172.21.15.38:0/1718754947 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f3ca0010070 con 0x7f3cac0f7780 2024-02-15T18:44:15.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.844+0000 7f3ca8ff9700 1 --2- 172.21.15.38:0/1718754947 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3c94060f20 0x7f3c940633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:15.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.844+0000 7f3ca8ff9700 1 -- 172.21.15.38:0/1718754947 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f3ca007bde0 con 0x7f3cac0f7780 2024-02-15T18:44:15.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.844+0000 7f3cabfff700 1 --2- 172.21.15.38:0/1718754947 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3c94060f20 0x7f3c940633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:15.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.845+0000 7f3cabfff700 1 --2- 172.21.15.38:0/1718754947 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3c94060f20 0x7f3c940633e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f3c9c009f50 tx=0x7f3c9c009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:15.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:15.848+0000 7f3ca8ff9700 1 -- 172.21.15.38:0/1718754947 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3ca0047f60 con 0x7f3cac0f7780 2024-02-15T18:44:16.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.068+0000 7f3cb2931700 1 -- 172.21.15.38:0/1718754947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 11, "format": "json"} v 0) v1 -- 0x7f3cac027a00 con 0x7f3cac0f7780 2024-02-15T18:44:16.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.068+0000 7f3ca8ff9700 1 -- 172.21.15.38:0/1718754947 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 11, "format": "json"}]=0 dumped fsmap epoch 11 v32) v1 ==== 107+0+4593 (secure 0 0 0) 0x7f3ca004fc40 con 0x7f3cac0f7780 2024-02-15T18:44:16.057 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:16.057 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14446,"name":"cephfs.smithi038.hxoggl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.38:6827/3822880235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3822880235},{"type":"v1","addr":"172.21.15.38:6827","nonce":3822880235}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14456,"name":"cephfs.smithi038.qgkzgn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.38:6829/703030231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":703030231},{"type":"v1","addr":"172.21.15.38:6829","nonce":703030231}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:33:02.882026+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":24261,"mds_1":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi121.mvumlh","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6825/1422707571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1422707571},{"type":"v1","addr":"172.21.15.121:6825","nonce":1422707571}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24275":{"gid":24275,"name":"cephfs.smithi121.ovuzki","rank":1,"incarnation":7,"state":"up:active","state_seq":2,"addr":"172.21.15.121:6827/562926845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":562926845},{"type":"v1","addr":"172.21.15.121:6827","nonce":562926845}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:16.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.070+0000 7f3cb2931700 1 -- 172.21.15.38:0/1718754947 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3c94060f20 msgr2=0x7f3c940633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:16.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.070+0000 7f3cb2931700 1 --2- 172.21.15.38:0/1718754947 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3c94060f20 0x7f3c940633e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f3c9c009f50 tx=0x7f3c9c009b30).stop 2024-02-15T18:44:16.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.070+0000 7f3cb2931700 1 -- 172.21.15.38:0/1718754947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cac0f7780 msgr2=0x7f3cac069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:16.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.070+0000 7f3cb2931700 1 --2- 172.21.15.38:0/1718754947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cac0f7780 0x7f3cac069e20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f3ca000c530 tx=0x7f3ca0007e20).stop 2024-02-15T18:44:16.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.070+0000 7f3cb2931700 1 -- 172.21.15.38:0/1718754947 shutdown_connections 2024-02-15T18:44:16.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.070+0000 7f3cb2931700 1 --2- 172.21.15.38:0/1718754947 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3c94060f20 0x7f3c940633e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:16.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.070+0000 7f3cb2931700 1 --2- 172.21.15.38:0/1718754947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cac0f64e0 0x7f3cac06b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:16.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.070+0000 7f3cb2931700 1 --2- 172.21.15.38:0/1718754947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cac0f7780 0x7f3cac069e20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:16.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.070+0000 7f3cb2931700 1 -- 172.21.15.38:0/1718754947 >> 172.21.15.38:0/1718754947 conn(0x7f3cac0f1c40 msgr2=0x7f3cac0fb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:16.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.071+0000 7f3cb2931700 1 -- 172.21.15.38:0/1718754947 shutdown_connections 2024-02-15T18:44:16.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.071+0000 7f3cb2931700 1 -- 172.21.15.38:0/1718754947 wait complete. 2024-02-15T18:44:16.060 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 11 2024-02-15T18:44:16.071 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 12 2024-02-15T18:44:16.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.206+0000 7f4523334700 1 -- 172.21.15.38:0/2762740038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f451c0f5ca0 msgr2=0x7f451c0f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:16.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.206+0000 7f4523334700 1 --2- 172.21.15.38:0/2762740038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f451c0f5ca0 0x7f451c0f66d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f450c004660 tx=0x7f450c009b30).stop 2024-02-15T18:44:16.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.206+0000 7f4523334700 1 -- 172.21.15.38:0/2762740038 shutdown_connections 2024-02-15T18:44:16.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.206+0000 7f4523334700 1 --2- 172.21.15.38:0/2762740038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f451c0f5ca0 0x7f451c0f66d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:16.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.206+0000 7f4523334700 1 --2- 172.21.15.38:0/2762740038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f451c0f5390 0x7f451c0f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:16.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.206+0000 7f4523334700 1 -- 172.21.15.38:0/2762740038 >> 172.21.15.38:0/2762740038 conn(0x7f451c0f0b70 msgr2=0x7f451c0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:16.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.206+0000 7f4523334700 1 -- 172.21.15.38:0/2762740038 shutdown_connections 2024-02-15T18:44:16.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.206+0000 7f4523334700 1 -- 172.21.15.38:0/2762740038 wait complete. 2024-02-15T18:44:16.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.206+0000 7f4523334700 1 Processor -- start 2024-02-15T18:44:16.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.206+0000 7f4523334700 1 -- start start 2024-02-15T18:44:16.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.207+0000 7f4523334700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f451c0f5390 0x7f451c05fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:16.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.207+0000 7f4523334700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f451c0f5ca0 0x7f451c10bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:16.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.207+0000 7f4523334700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451c10bfc0 con 0x7f451c0f5390 2024-02-15T18:44:16.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.207+0000 7f4523334700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451c10c100 con 0x7f451c0f5ca0 2024-02-15T18:44:16.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.207+0000 7f45210d0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f451c0f5390 0x7f451c05fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:16.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.207+0000 7f45210d0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f451c0f5390 0x7f451c05fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59270/0 (socket says 172.21.15.38:59270) 2024-02-15T18:44:16.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.207+0000 7f45210d0700 1 -- 172.21.15.38:0/568806339 learned_addr learned my addr 172.21.15.38:0/568806339 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:16.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.207+0000 7f45208cf700 1 --2- 172.21.15.38:0/568806339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f451c0f5ca0 0x7f451c10bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:16.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.207+0000 7f45210d0700 1 -- 172.21.15.38:0/568806339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f451c0f5ca0 msgr2=0x7f451c10bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:16.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.207+0000 7f45210d0700 1 --2- 172.21.15.38:0/568806339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f451c0f5ca0 0x7f451c10bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:16.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.207+0000 7f45210d0700 1 -- 172.21.15.38:0/568806339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c005040 con 0x7f451c0f5390 2024-02-15T18:44:16.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.207+0000 7f45210d0700 1 --2- 172.21.15.38:0/568806339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f451c0f5390 0x7f451c05fcb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f4518002700 tx=0x7f4518009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:16.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.207+0000 7f4511ffb700 1 -- 172.21.15.38:0/568806339 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4518009320 con 0x7f451c0f5390 2024-02-15T18:44:16.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.207+0000 7f4511ffb700 1 -- 172.21.15.38:0/568806339 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f451800f040 con 0x7f451c0f5390 2024-02-15T18:44:16.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.207+0000 7f4523334700 1 -- 172.21.15.38:0/568806339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f451c10c360 con 0x7f451c0f5390 2024-02-15T18:44:16.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.207+0000 7f4511ffb700 1 -- 172.21.15.38:0/568806339 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45180149b0 con 0x7f451c0f5390 2024-02-15T18:44:16.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.208+0000 7f4523334700 1 -- 172.21.15.38:0/568806339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451c10c920 con 0x7f451c0f5390 2024-02-15T18:44:16.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.208+0000 7f4523334700 1 -- 172.21.15.38:0/568806339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f451c027a00 con 0x7f451c0f5390 2024-02-15T18:44:16.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.209+0000 7f4511ffb700 1 -- 172.21.15.38:0/568806339 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f4518009480 con 0x7f451c0f5390 2024-02-15T18:44:16.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.209+0000 7f4511ffb700 1 --2- 172.21.15.38:0/568806339 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4508060f20 0x7f45080633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:16.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.209+0000 7f4511ffb700 1 -- 172.21.15.38:0/568806339 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f451807c670 con 0x7f451c0f5390 2024-02-15T18:44:16.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.209+0000 7f45208cf700 1 --2- 172.21.15.38:0/568806339 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4508060f20 0x7f45080633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:16.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.210+0000 7f45208cf700 1 --2- 172.21.15.38:0/568806339 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4508060f20 0x7f45080633e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f450c004630 tx=0x7f450c009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:16.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.214+0000 7f4511ffb700 1 -- 172.21.15.38:0/568806339 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4518050540 con 0x7f451c0f5390 2024-02-15T18:44:16.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.429+0000 7f4523334700 1 -- 172.21.15.38:0/568806339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 12, "format": "json"} v 0) v1 -- 0x7f451c0586e0 con 0x7f451c0f5390 2024-02-15T18:44:16.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.430+0000 7f4511ffb700 1 -- 172.21.15.38:0/568806339 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 12, "format": "json"}]=0 dumped fsmap epoch 12 v32) v1 ==== 107+0+4594 (secure 0 0 0) 0x7f4518050130 con 0x7f451c0f5390 2024-02-15T18:44:16.419 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:16.419 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14446,"name":"cephfs.smithi038.hxoggl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.38:6827/3822880235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3822880235},{"type":"v1","addr":"172.21.15.38:6827","nonce":3822880235}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14456,"name":"cephfs.smithi038.qgkzgn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.38:6829/703030231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":703030231},{"type":"v1","addr":"172.21.15.38:6829","nonce":703030231}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:34:24.368128+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":24261,"mds_1":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi121.mvumlh","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6825/1422707571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1422707571},{"type":"v1","addr":"172.21.15.121:6825","nonce":1422707571}]},"join_fscid":1,"export_targets":[1],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24275":{"gid":24275,"name":"cephfs.smithi121.ovuzki","rank":1,"incarnation":7,"state":"up:active","state_seq":2,"addr":"172.21.15.121:6827/562926845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":562926845},{"type":"v1","addr":"172.21.15.121:6827","nonce":562926845}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:16.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.432+0000 7f4523334700 1 -- 172.21.15.38:0/568806339 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4508060f20 msgr2=0x7f45080633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:16.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.432+0000 7f4523334700 1 --2- 172.21.15.38:0/568806339 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4508060f20 0x7f45080633e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f450c004630 tx=0x7f450c009b30).stop 2024-02-15T18:44:16.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.432+0000 7f4523334700 1 -- 172.21.15.38:0/568806339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f451c0f5390 msgr2=0x7f451c05fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:16.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.432+0000 7f4523334700 1 --2- 172.21.15.38:0/568806339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f451c0f5390 0x7f451c05fcb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f4518002700 tx=0x7f4518009de0).stop 2024-02-15T18:44:16.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.432+0000 7f4523334700 1 -- 172.21.15.38:0/568806339 shutdown_connections 2024-02-15T18:44:16.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.432+0000 7f4523334700 1 --2- 172.21.15.38:0/568806339 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4508060f20 0x7f45080633e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:16.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.432+0000 7f4523334700 1 --2- 172.21.15.38:0/568806339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f451c0f5390 0x7f451c05fcb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:16.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.432+0000 7f4523334700 1 --2- 172.21.15.38:0/568806339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f451c0f5ca0 0x7f451c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:16.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.432+0000 7f4523334700 1 -- 172.21.15.38:0/568806339 >> 172.21.15.38:0/568806339 conn(0x7f451c0f0b70 msgr2=0x7f451c0fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:16.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.432+0000 7f4523334700 1 -- 172.21.15.38:0/568806339 shutdown_connections 2024-02-15T18:44:16.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.432+0000 7f4523334700 1 -- 172.21.15.38:0/568806339 wait complete. 2024-02-15T18:44:16.422 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 12 2024-02-15T18:44:16.433 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 13 2024-02-15T18:44:16.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.565+0000 7f5933439700 1 -- 172.21.15.38:0/3820587154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f592c0f9600 msgr2=0x7f592c0fb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:16.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.565+0000 7f5933439700 1 --2- 172.21.15.38:0/3820587154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f592c0f9600 0x7f592c0fb9f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f591c004660 tx=0x7f591c009b30).stop 2024-02-15T18:44:16.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.565+0000 7f5933439700 1 -- 172.21.15.38:0/3820587154 shutdown_connections 2024-02-15T18:44:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.565+0000 7f5933439700 1 --2- 172.21.15.38:0/3820587154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f592c0f9600 0x7f592c0fb9f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.565+0000 7f5933439700 1 --2- 172.21.15.38:0/3820587154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f592c05ff10 0x7f592c0f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.565+0000 7f5933439700 1 -- 172.21.15.38:0/3820587154 >> 172.21.15.38:0/3820587154 conn(0x7f592c0f0b50 msgr2=0x7f592c0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.566+0000 7f5933439700 1 -- 172.21.15.38:0/3820587154 shutdown_connections 2024-02-15T18:44:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.566+0000 7f5933439700 1 -- 172.21.15.38:0/3820587154 wait complete. 2024-02-15T18:44:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.566+0000 7f5933439700 1 Processor -- start 2024-02-15T18:44:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.566+0000 7f5933439700 1 -- start start 2024-02-15T18:44:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.567+0000 7f5933439700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f592c05ff10 0x7f592c106ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.567+0000 7f5933439700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f592c0f9600 0x7f592c1071d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.567+0000 7f5933439700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c1077d0 con 0x7f592c05ff10 2024-02-15T18:44:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.567+0000 7f5933439700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c107910 con 0x7f592c0f9600 2024-02-15T18:44:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.567+0000 7f59311d5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f592c05ff10 0x7f592c106ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.567+0000 7f59311d5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f592c05ff10 0x7f592c106ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59286/0 (socket says 172.21.15.38:59286) 2024-02-15T18:44:16.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.567+0000 7f59311d5700 1 -- 172.21.15.38:0/345136027 learned_addr learned my addr 172.21.15.38:0/345136027 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.567+0000 7f59309d4700 1 --2- 172.21.15.38:0/345136027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f592c0f9600 0x7f592c1071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.567+0000 7f59311d5700 1 -- 172.21.15.38:0/345136027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f592c0f9600 msgr2=0x7f592c1071d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.567+0000 7f59311d5700 1 --2- 172.21.15.38:0/345136027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f592c0f9600 0x7f592c1071d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.567+0000 7f59311d5700 1 -- 172.21.15.38:0/345136027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c005040 con 0x7f592c05ff10 2024-02-15T18:44:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.567+0000 7f59311d5700 1 --2- 172.21.15.38:0/345136027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f592c05ff10 0x7f592c106ce0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f5928002700 tx=0x7f5928009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.567+0000 7f5921ffb700 1 -- 172.21.15.38:0/345136027 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5928009320 con 0x7f592c05ff10 2024-02-15T18:44:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.567+0000 7f5921ffb700 1 -- 172.21.15.38:0/345136027 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f592800f040 con 0x7f592c05ff10 2024-02-15T18:44:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.567+0000 7f5921ffb700 1 -- 172.21.15.38:0/345136027 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59280149d0 con 0x7f592c05ff10 2024-02-15T18:44:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.567+0000 7f5933439700 1 -- 172.21.15.38:0/345136027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c10c320 con 0x7f592c05ff10 2024-02-15T18:44:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.568+0000 7f5933439700 1 -- 172.21.15.38:0/345136027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c10c8e0 con 0x7f592c05ff10 2024-02-15T18:44:16.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.568+0000 7f5933439700 1 -- 172.21.15.38:0/345136027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592c027a00 con 0x7f592c05ff10 2024-02-15T18:44:16.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.569+0000 7f5921ffb700 1 -- 172.21.15.38:0/345136027 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f59280077b0 con 0x7f592c05ff10 2024-02-15T18:44:16.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.569+0000 7f5921ffb700 1 --2- 172.21.15.38:0/345136027 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5918060f20 0x7f59180633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:16.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.569+0000 7f5921ffb700 1 -- 172.21.15.38:0/345136027 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f592807c300 con 0x7f592c05ff10 2024-02-15T18:44:16.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.569+0000 7f59309d4700 1 --2- 172.21.15.38:0/345136027 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5918060f20 0x7f59180633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:16.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.569+0000 7f59309d4700 1 --2- 172.21.15.38:0/345136027 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5918060f20 0x7f59180633e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f591c004e30 tx=0x7f591c009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:16.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.573+0000 7f5921ffb700 1 -- 172.21.15.38:0/345136027 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f592804c7b0 con 0x7f592c05ff10 2024-02-15T18:44:16.770 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:44:15.043760+0000 mgr.smithi038.khesbl (mgr.14664) 413 : cluster [DBG] pgmap v284: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:16.770 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:15.707685+0000 mon.smithi038 (mon.0) 740 : audit [DBG] from='client.? 172.21.15.38:0/761189161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-15T18:44:16.770 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:16 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:16.070025+0000 mon.smithi038 (mon.0) 741 : audit [DBG] from='client.? 172.21.15.38:0/1718754947' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 11, "format": "json"}]: dispatch 2024-02-15T18:44:16.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.786+0000 7f5933439700 1 -- 172.21.15.38:0/345136027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 13, "format": "json"} v 0) v1 -- 0x7f592c0fed70 con 0x7f592c05ff10 2024-02-15T18:44:16.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.787+0000 7f5921ffb700 1 -- 172.21.15.38:0/345136027 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 13, "format": "json"}]=0 dumped fsmap epoch 13 v32) v1 ==== 107+0+4595 (secure 0 0 0) 0x7f592804c030 con 0x7f592c05ff10 2024-02-15T18:44:16.776 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:16.776 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14446,"name":"cephfs.smithi038.hxoggl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.38:6827/3822880235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3822880235},{"type":"v1","addr":"172.21.15.38:6827","nonce":3822880235}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14456,"name":"cephfs.smithi038.qgkzgn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.38:6829/703030231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":703030231},{"type":"v1","addr":"172.21.15.38:6829","nonce":703030231}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:35:07.485909+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":24261,"mds_1":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi121.mvumlh","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6825/1422707571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1422707571},{"type":"v1","addr":"172.21.15.121:6825","nonce":1422707571}]},"join_fscid":1,"export_targets":[1],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24275":{"gid":24275,"name":"cephfs.smithi121.ovuzki","rank":1,"incarnation":7,"state":"up:active","state_seq":2,"addr":"172.21.15.121:6827/562926845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":562926845},{"type":"v1","addr":"172.21.15.121:6827","nonce":562926845}]},"join_fscid":1,"export_targets":[0],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:16.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.789+0000 7f5933439700 1 -- 172.21.15.38:0/345136027 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5918060f20 msgr2=0x7f59180633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:16.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.789+0000 7f5933439700 1 --2- 172.21.15.38:0/345136027 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5918060f20 0x7f59180633e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f591c004e30 tx=0x7f591c009b30).stop 2024-02-15T18:44:16.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.789+0000 7f5933439700 1 -- 172.21.15.38:0/345136027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f592c05ff10 msgr2=0x7f592c106ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:16.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.789+0000 7f5933439700 1 --2- 172.21.15.38:0/345136027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f592c05ff10 0x7f592c106ce0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f5928002700 tx=0x7f5928009d80).stop 2024-02-15T18:44:16.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.789+0000 7f5933439700 1 -- 172.21.15.38:0/345136027 shutdown_connections 2024-02-15T18:44:16.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.789+0000 7f5933439700 1 --2- 172.21.15.38:0/345136027 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f5918060f20 0x7f59180633e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:16.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.789+0000 7f5933439700 1 --2- 172.21.15.38:0/345136027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f592c05ff10 0x7f592c106ce0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:16.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.789+0000 7f5933439700 1 --2- 172.21.15.38:0/345136027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f592c0f9600 0x7f592c1071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:16.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.789+0000 7f5933439700 1 -- 172.21.15.38:0/345136027 >> 172.21.15.38:0/345136027 conn(0x7f592c0f0b50 msgr2=0x7f592c0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:16.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.789+0000 7f5933439700 1 -- 172.21.15.38:0/345136027 shutdown_connections 2024-02-15T18:44:16.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.789+0000 7f5933439700 1 -- 172.21.15.38:0/345136027 wait complete. 2024-02-15T18:44:16.779 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 13 2024-02-15T18:44:16.790 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 14 2024-02-15T18:44:16.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:44:15.043760+0000 mgr.smithi038.khesbl (mgr.14664) 413 : cluster [DBG] pgmap v284: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:16.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:15.707685+0000 mon.smithi038 (mon.0) 740 : audit [DBG] from='client.? 172.21.15.38:0/761189161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-15T18:44:16.855 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:16 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:16.070025+0000 mon.smithi038 (mon.0) 741 : audit [DBG] from='client.? 172.21.15.38:0/1718754947' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 11, "format": "json"}]: dispatch 2024-02-15T18:44:16.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.922+0000 7f6d76abf700 1 -- 172.21.15.38:0/324377819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d700f5ca0 msgr2=0x7f6d700f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:16.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.922+0000 7f6d76abf700 1 --2- 172.21.15.38:0/324377819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d700f5ca0 0x7f6d700f66d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f6d5c004660 tx=0x7f6d5c009b30).stop 2024-02-15T18:44:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.922+0000 7f6d76abf700 1 -- 172.21.15.38:0/324377819 shutdown_connections 2024-02-15T18:44:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.922+0000 7f6d76abf700 1 --2- 172.21.15.38:0/324377819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d700f5ca0 0x7f6d700f66d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.922+0000 7f6d76abf700 1 --2- 172.21.15.38:0/324377819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d700f5390 0x7f6d700f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.922+0000 7f6d76abf700 1 -- 172.21.15.38:0/324377819 >> 172.21.15.38:0/324377819 conn(0x7f6d700f0b70 msgr2=0x7f6d700f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.922+0000 7f6d76abf700 1 -- 172.21.15.38:0/324377819 shutdown_connections 2024-02-15T18:44:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.922+0000 7f6d76abf700 1 -- 172.21.15.38:0/324377819 wait complete. 2024-02-15T18:44:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.923+0000 7f6d76abf700 1 Processor -- start 2024-02-15T18:44:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.923+0000 7f6d76abf700 1 -- start start 2024-02-15T18:44:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.923+0000 7f6d76abf700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d700f5390 0x7f6d7005fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.923+0000 7f6d76abf700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d700f5ca0 0x7f6d7010bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.923+0000 7f6d76abf700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7010bfc0 con 0x7f6d700f5390 2024-02-15T18:44:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.923+0000 7f6d76abf700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7010c100 con 0x7f6d700f5ca0 2024-02-15T18:44:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.923+0000 7f6d7485b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d700f5390 0x7f6d7005fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.923+0000 7f6d7485b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d700f5390 0x7f6d7005fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59316/0 (socket says 172.21.15.38:59316) 2024-02-15T18:44:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.923+0000 7f6d7485b700 1 -- 172.21.15.38:0/4042345809 learned_addr learned my addr 172.21.15.38:0/4042345809 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.923+0000 7f6d6ffff700 1 --2- 172.21.15.38:0/4042345809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d700f5ca0 0x7f6d7010bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.923+0000 7f6d7485b700 1 -- 172.21.15.38:0/4042345809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d700f5ca0 msgr2=0x7f6d7010bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.923+0000 7f6d7485b700 1 --2- 172.21.15.38:0/4042345809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d700f5ca0 0x7f6d7010bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.923+0000 7f6d7485b700 1 -- 172.21.15.38:0/4042345809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c005040 con 0x7f6d700f5390 2024-02-15T18:44:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.924+0000 7f6d7485b700 1 --2- 172.21.15.38:0/4042345809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d700f5390 0x7f6d7005fcb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f6d6400c7c0 tx=0x7f6d64007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.924+0000 7f6d6d7fa700 1 -- 172.21.15.38:0/4042345809 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d6400e680 con 0x7f6d700f5390 2024-02-15T18:44:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.924+0000 7f6d6d7fa700 1 -- 172.21.15.38:0/4042345809 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f6d6400ecd0 con 0x7f6d700f5390 2024-02-15T18:44:16.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.924+0000 7f6d76abf700 1 -- 172.21.15.38:0/4042345809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d7010c360 con 0x7f6d700f5390 2024-02-15T18:44:16.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.924+0000 7f6d6d7fa700 1 -- 172.21.15.38:0/4042345809 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d64013e90 con 0x7f6d700f5390 2024-02-15T18:44:16.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.924+0000 7f6d76abf700 1 -- 172.21.15.38:0/4042345809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7010c920 con 0x7f6d700f5390 2024-02-15T18:44:16.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.925+0000 7f6d76abf700 1 -- 172.21.15.38:0/4042345809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d70027a00 con 0x7f6d700f5390 2024-02-15T18:44:16.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.925+0000 7f6d6d7fa700 1 -- 172.21.15.38:0/4042345809 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f6d6401a060 con 0x7f6d700f5390 2024-02-15T18:44:16.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.926+0000 7f6d6d7fa700 1 --2- 172.21.15.38:0/4042345809 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f6d60060f20 0x7f6d600633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:16.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.926+0000 7f6d6d7fa700 1 -- 172.21.15.38:0/4042345809 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f6d6407c440 con 0x7f6d700f5390 2024-02-15T18:44:16.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.926+0000 7f6d6ffff700 1 --2- 172.21.15.38:0/4042345809 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f6d60060f20 0x7f6d600633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:16.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.926+0000 7f6d6ffff700 1 --2- 172.21.15.38:0/4042345809 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f6d60060f20 0x7f6d600633e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f6d5c004e30 tx=0x7f6d5c009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:16.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:16.930+0000 7f6d6d7fa700 1 -- 172.21.15.38:0/4042345809 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6d640501e0 con 0x7f6d700f5390 2024-02-15T18:44:17.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.150+0000 7f6d76abf700 1 -- 172.21.15.38:0/4042345809 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 14, "format": "json"} v 0) v1 -- 0x7f6d700586e0 con 0x7f6d700f5390 2024-02-15T18:44:17.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.150+0000 7f6d6d7fa700 1 -- 172.21.15.38:0/4042345809 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 14, "format": "json"}]=0 dumped fsmap epoch 14 v32) v1 ==== 107+0+4594 (secure 0 0 0) 0x7f6d6404c030 con 0x7f6d700f5390 2024-02-15T18:44:17.139 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:17.139 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14446,"name":"cephfs.smithi038.hxoggl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.38:6827/3822880235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3822880235},{"type":"v1","addr":"172.21.15.38:6827","nonce":3822880235}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14456,"name":"cephfs.smithi038.qgkzgn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.38:6829/703030231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":703030231},{"type":"v1","addr":"172.21.15.38:6829","nonce":703030231}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:39:05.842870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":24261,"mds_1":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi121.mvumlh","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6825/1422707571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1422707571},{"type":"v1","addr":"172.21.15.121:6825","nonce":1422707571}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24275":{"gid":24275,"name":"cephfs.smithi121.ovuzki","rank":1,"incarnation":7,"state":"up:active","state_seq":2,"addr":"172.21.15.121:6827/562926845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":562926845},{"type":"v1","addr":"172.21.15.121:6827","nonce":562926845}]},"join_fscid":1,"export_targets":[0],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:17.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.152+0000 7f6d76abf700 1 -- 172.21.15.38:0/4042345809 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f6d60060f20 msgr2=0x7f6d600633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:17.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.152+0000 7f6d76abf700 1 --2- 172.21.15.38:0/4042345809 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f6d60060f20 0x7f6d600633e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f6d5c004e30 tx=0x7f6d5c009b30).stop 2024-02-15T18:44:17.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.152+0000 7f6d76abf700 1 -- 172.21.15.38:0/4042345809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d700f5390 msgr2=0x7f6d7005fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:17.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.152+0000 7f6d76abf700 1 --2- 172.21.15.38:0/4042345809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d700f5390 0x7f6d7005fcb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f6d6400c7c0 tx=0x7f6d64007d70).stop 2024-02-15T18:44:17.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.153+0000 7f6d76abf700 1 -- 172.21.15.38:0/4042345809 shutdown_connections 2024-02-15T18:44:17.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.153+0000 7f6d76abf700 1 --2- 172.21.15.38:0/4042345809 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f6d60060f20 0x7f6d600633e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:17.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.153+0000 7f6d76abf700 1 --2- 172.21.15.38:0/4042345809 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d700f5390 0x7f6d7005fcb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:17.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.153+0000 7f6d76abf700 1 --2- 172.21.15.38:0/4042345809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d700f5ca0 0x7f6d7010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:17.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.153+0000 7f6d76abf700 1 -- 172.21.15.38:0/4042345809 >> 172.21.15.38:0/4042345809 conn(0x7f6d700f0b70 msgr2=0x7f6d700fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:17.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.153+0000 7f6d76abf700 1 -- 172.21.15.38:0/4042345809 shutdown_connections 2024-02-15T18:44:17.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.153+0000 7f6d76abf700 1 -- 172.21.15.38:0/4042345809 wait complete. 2024-02-15T18:44:17.143 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 14 2024-02-15T18:44:17.153 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 15 2024-02-15T18:44:17.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.286+0000 7f32728f7700 1 -- 172.21.15.38:0/208384396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f326c0f5ca0 msgr2=0x7f326c0f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:17.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.286+0000 7f32728f7700 1 --2- 172.21.15.38:0/208384396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f326c0f5ca0 0x7f326c0f66d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f325c004660 tx=0x7f325c009b30).stop 2024-02-15T18:44:17.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.286+0000 7f32728f7700 1 -- 172.21.15.38:0/208384396 shutdown_connections 2024-02-15T18:44:17.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.286+0000 7f32728f7700 1 --2- 172.21.15.38:0/208384396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f326c0f5ca0 0x7f326c0f66d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:17.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.286+0000 7f32728f7700 1 --2- 172.21.15.38:0/208384396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f326c0f5390 0x7f326c0f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:17.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.286+0000 7f32728f7700 1 -- 172.21.15.38:0/208384396 >> 172.21.15.38:0/208384396 conn(0x7f326c0f0b70 msgr2=0x7f326c0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:17.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.286+0000 7f32728f7700 1 -- 172.21.15.38:0/208384396 shutdown_connections 2024-02-15T18:44:17.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.286+0000 7f32728f7700 1 -- 172.21.15.38:0/208384396 wait complete. 2024-02-15T18:44:17.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.287+0000 7f32728f7700 1 Processor -- start 2024-02-15T18:44:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.287+0000 7f32728f7700 1 -- start start 2024-02-15T18:44:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.287+0000 7f32728f7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f326c0f5390 0x7f326c05fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.287+0000 7f32728f7700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f326c0f5ca0 0x7f326c10bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.287+0000 7f32728f7700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c10bfc0 con 0x7f326c0f5ca0 2024-02-15T18:44:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.287+0000 7f32728f7700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c10c100 con 0x7f326c0f5390 2024-02-15T18:44:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.287+0000 7f326b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f326c0f5ca0 0x7f326c10bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.287+0000 7f326b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f326c0f5ca0 0x7f326c10bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59330/0 (socket says 172.21.15.38:59330) 2024-02-15T18:44:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.287+0000 7f326b7fe700 1 -- 172.21.15.38:0/1675295081 learned_addr learned my addr 172.21.15.38:0/1675295081 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.287+0000 7f326bfff700 1 --2- 172.21.15.38:0/1675295081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f326c0f5390 0x7f326c05fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.287+0000 7f326b7fe700 1 -- 172.21.15.38:0/1675295081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f326c0f5390 msgr2=0x7f326c05fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.287+0000 7f326b7fe700 1 --2- 172.21.15.38:0/1675295081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f326c0f5390 0x7f326c05fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.287+0000 7f326b7fe700 1 -- 172.21.15.38:0/1675295081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325c005040 con 0x7f326c0f5ca0 2024-02-15T18:44:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.287+0000 7f326b7fe700 1 --2- 172.21.15.38:0/1675295081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f326c0f5ca0 0x7f326c10bad0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f325c0096f0 tx=0x7f325c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.288+0000 7f3268ff9700 1 -- 172.21.15.38:0/1675295081 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f325c019070 con 0x7f326c0f5ca0 2024-02-15T18:44:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.288+0000 7f32728f7700 1 -- 172.21.15.38:0/1675295081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326c10c300 con 0x7f326c0f5ca0 2024-02-15T18:44:17.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.288+0000 7f3268ff9700 1 -- 172.21.15.38:0/1675295081 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f325c014420 con 0x7f326c0f5ca0 2024-02-15T18:44:17.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.288+0000 7f3268ff9700 1 -- 172.21.15.38:0/1675295081 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f325c01d600 con 0x7f326c0f5ca0 2024-02-15T18:44:17.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.288+0000 7f32728f7700 1 -- 172.21.15.38:0/1675295081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326c10c860 con 0x7f326c0f5ca0 2024-02-15T18:44:17.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.289+0000 7f32728f7700 1 -- 172.21.15.38:0/1675295081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326c027a00 con 0x7f326c0f5ca0 2024-02-15T18:44:17.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.289+0000 7f3268ff9700 1 -- 172.21.15.38:0/1675295081 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f325c025070 con 0x7f326c0f5ca0 2024-02-15T18:44:17.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.289+0000 7f3268ff9700 1 --2- 172.21.15.38:0/1675295081 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3258060f20 0x7f32580633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:17.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.290+0000 7f3268ff9700 1 -- 172.21.15.38:0/1675295081 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f325c07d120 con 0x7f326c0f5ca0 2024-02-15T18:44:17.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.290+0000 7f326bfff700 1 --2- 172.21.15.38:0/1675295081 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3258060f20 0x7f32580633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:17.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.290+0000 7f326bfff700 1 --2- 172.21.15.38:0/1675295081 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3258060f20 0x7f32580633e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f3254007f80 tx=0x7f325400b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:17.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.294+0000 7f3268ff9700 1 -- 172.21.15.38:0/1675295081 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f325c050f20 con 0x7f326c0f5ca0 2024-02-15T18:44:17.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.509+0000 7f32728f7700 1 -- 172.21.15.38:0/1675295081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 15, "format": "json"} v 0) v1 -- 0x7f326c058710 con 0x7f326c0f5ca0 2024-02-15T18:44:17.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.510+0000 7f3268ff9700 1 -- 172.21.15.38:0/1675295081 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 15, "format": "json"}]=0 dumped fsmap epoch 15 v32) v1 ==== 107+0+4593 (secure 0 0 0) 0x7f325c023020 con 0x7f326c0f5ca0 2024-02-15T18:44:17.499 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:17.499 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14446,"name":"cephfs.smithi038.hxoggl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.38:6827/3822880235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3822880235},{"type":"v1","addr":"172.21.15.38:6827","nonce":3822880235}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14456,"name":"cephfs.smithi038.qgkzgn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.38:6829/703030231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":703030231},{"type":"v1","addr":"172.21.15.38:6829","nonce":703030231}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:39:38.050426+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":24261,"mds_1":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi121.mvumlh","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6825/1422707571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1422707571},{"type":"v1","addr":"172.21.15.121:6825","nonce":1422707571}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24275":{"gid":24275,"name":"cephfs.smithi121.ovuzki","rank":1,"incarnation":7,"state":"up:active","state_seq":2,"addr":"172.21.15.121:6827/562926845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":562926845},{"type":"v1","addr":"172.21.15.121:6827","nonce":562926845}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:17.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.512+0000 7f32728f7700 1 -- 172.21.15.38:0/1675295081 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3258060f20 msgr2=0x7f32580633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:17.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.512+0000 7f32728f7700 1 --2- 172.21.15.38:0/1675295081 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3258060f20 0x7f32580633e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f3254007f80 tx=0x7f325400b040).stop 2024-02-15T18:44:17.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.512+0000 7f32728f7700 1 -- 172.21.15.38:0/1675295081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f326c0f5ca0 msgr2=0x7f326c10bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:17.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.512+0000 7f32728f7700 1 --2- 172.21.15.38:0/1675295081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f326c0f5ca0 0x7f326c10bad0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f325c0096f0 tx=0x7f325c00b040).stop 2024-02-15T18:44:17.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.512+0000 7f32728f7700 1 -- 172.21.15.38:0/1675295081 shutdown_connections 2024-02-15T18:44:17.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.512+0000 7f32728f7700 1 --2- 172.21.15.38:0/1675295081 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3258060f20 0x7f32580633e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:17.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.512+0000 7f32728f7700 1 --2- 172.21.15.38:0/1675295081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f326c0f5390 0x7f326c05fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:17.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.512+0000 7f32728f7700 1 --2- 172.21.15.38:0/1675295081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f326c0f5ca0 0x7f326c10bad0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:17.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.512+0000 7f32728f7700 1 -- 172.21.15.38:0/1675295081 >> 172.21.15.38:0/1675295081 conn(0x7f326c0f0b70 msgr2=0x7f326c0fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:17.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.512+0000 7f32728f7700 1 -- 172.21.15.38:0/1675295081 shutdown_connections 2024-02-15T18:44:17.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.512+0000 7f32728f7700 1 -- 172.21.15.38:0/1675295081 wait complete. 2024-02-15T18:44:17.502 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 15 2024-02-15T18:44:17.511 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:16.431138+0000 mon.smithi038 (mon.0) 742 : audit [DBG] from='client.? 172.21.15.38:0/568806339' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 12, "format": "json"}]: dispatch 2024-02-15T18:44:17.511 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:16.788534+0000 mon.smithi038 (mon.0) 743 : audit [DBG] from='client.? 172.21.15.38:0/345136027' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 13, "format": "json"}]: dispatch 2024-02-15T18:44:17.512 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:17 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:17.152129+0000 mon.smithi038 (mon.0) 744 : audit [DBG] from='client.? 172.21.15.38:0/4042345809' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 14, "format": "json"}]: dispatch 2024-02-15T18:44:17.513 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 16 2024-02-15T18:44:17.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.644+0000 7f81d3bca700 1 -- 172.21.15.38:0/1753858169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81cc0f6490 msgr2=0x7f81cc0f68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:17.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.644+0000 7f81d3bca700 1 --2- 172.21.15.38:0/1753858169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81cc0f6490 0x7f81cc0f68b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f81c8004660 tx=0x7f81c8009b30).stop 2024-02-15T18:44:17.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.644+0000 7f81d3bca700 1 -- 172.21.15.38:0/1753858169 shutdown_connections 2024-02-15T18:44:17.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.644+0000 7f81d3bca700 1 --2- 172.21.15.38:0/1753858169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81cc0f7730 0x7f81cc0f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:17.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.644+0000 7f81d3bca700 1 --2- 172.21.15.38:0/1753858169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81cc0f6490 0x7f81cc0f68b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:17.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.644+0000 7f81d3bca700 1 -- 172.21.15.38:0/1753858169 >> 172.21.15.38:0/1753858169 conn(0x7f81cc0f1bd0 msgr2=0x7f81cc0f4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:17.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.644+0000 7f81d3bca700 1 -- 172.21.15.38:0/1753858169 shutdown_connections 2024-02-15T18:44:17.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.645+0000 7f81d3bca700 1 -- 172.21.15.38:0/1753858169 wait complete. 2024-02-15T18:44:17.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.645+0000 7f81d3bca700 1 Processor -- start 2024-02-15T18:44:17.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.645+0000 7f81d3bca700 1 -- start start 2024-02-15T18:44:17.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.645+0000 7f81d3bca700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81cc0f6490 0x7f81cc106d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:17.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.645+0000 7f81d3bca700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81cc0f7730 0x7f81cc107210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:17.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.645+0000 7f81d3bca700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc107810 con 0x7f81cc0f7730 2024-02-15T18:44:17.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.645+0000 7f81d3bca700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc107950 con 0x7f81cc0f6490 2024-02-15T18:44:17.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.645+0000 7f81d1165700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81cc0f7730 0x7f81cc107210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:17.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.645+0000 7f81d1165700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81cc0f7730 0x7f81cc107210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59338/0 (socket says 172.21.15.38:59338) 2024-02-15T18:44:17.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.645+0000 7f81d1165700 1 -- 172.21.15.38:0/1250691869 learned_addr learned my addr 172.21.15.38:0/1250691869 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:17.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.645+0000 7f81d1966700 1 --2- 172.21.15.38:0/1250691869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81cc0f6490 0x7f81cc106d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:17.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.646+0000 7f81d1165700 1 -- 172.21.15.38:0/1250691869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81cc0f6490 msgr2=0x7f81cc106d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:17.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.646+0000 7f81d1165700 1 --2- 172.21.15.38:0/1250691869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81cc0f6490 0x7f81cc106d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:17.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.646+0000 7f81d1165700 1 -- 172.21.15.38:0/1250691869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c8005040 con 0x7f81cc0f7730 2024-02-15T18:44:17.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.646+0000 7f81d1966700 1 --2- 172.21.15.38:0/1250691869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81cc0f6490 0x7f81cc106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:44:17.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.646+0000 7f81d1165700 1 --2- 172.21.15.38:0/1250691869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81cc0f7730 0x7f81cc107210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f81bc002700 tx=0x7f81bc009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:17.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.646+0000 7f81c27fc700 1 -- 172.21.15.38:0/1250691869 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81bc009320 con 0x7f81cc0f7730 2024-02-15T18:44:17.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.646+0000 7f81d3bca700 1 -- 172.21.15.38:0/1250691869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81cc10c360 con 0x7f81cc0f7730 2024-02-15T18:44:17.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.646+0000 7f81c27fc700 1 -- 172.21.15.38:0/1250691869 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f81bc00f040 con 0x7f81cc0f7730 2024-02-15T18:44:17.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.646+0000 7f81c27fc700 1 -- 172.21.15.38:0/1250691869 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81bc014970 con 0x7f81cc0f7730 2024-02-15T18:44:17.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.646+0000 7f81d3bca700 1 -- 172.21.15.38:0/1250691869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81cc10c8a0 con 0x7f81cc0f7730 2024-02-15T18:44:17.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.647+0000 7f81d3bca700 1 -- 172.21.15.38:0/1250691869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81cc027a00 con 0x7f81cc0f7730 2024-02-15T18:44:17.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.647+0000 7f81c27fc700 1 -- 172.21.15.38:0/1250691869 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f81bc0077b0 con 0x7f81cc0f7730 2024-02-15T18:44:17.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.648+0000 7f81c27fc700 1 --2- 172.21.15.38:0/1250691869 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f81b8060f20 0x7f81b80633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:17.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.648+0000 7f81c27fc700 1 -- 172.21.15.38:0/1250691869 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f81bc07c570 con 0x7f81cc0f7730 2024-02-15T18:44:17.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.648+0000 7f81d1966700 1 --2- 172.21.15.38:0/1250691869 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f81b8060f20 0x7f81b80633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:17.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.648+0000 7f81d1966700 1 --2- 172.21.15.38:0/1250691869 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f81b8060f20 0x7f81b80633e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f81cc108210 tx=0x7f81c8009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:17.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.652+0000 7f81c27fc700 1 -- 172.21.15.38:0/1250691869 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f81bc04c9a0 con 0x7f81cc0f7730 2024-02-15T18:44:17.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.863+0000 7f81d3bca700 1 -- 172.21.15.38:0/1250691869 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7f81cc0fed60 con 0x7f81cc0f7730 2024-02-15T18:44:17.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.864+0000 7f81c27fc700 1 -- 172.21.15.38:0/1250691869 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v32) v1 ==== 107+0+4595 (secure 0 0 0) 0x7f81bc04ffc0 con 0x7f81cc0f7730 2024-02-15T18:44:17.853 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:17.853 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14446,"name":"cephfs.smithi038.hxoggl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.38:6827/3822880235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3822880235},{"type":"v1","addr":"172.21.15.38:6827","nonce":3822880235}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14456,"name":"cephfs.smithi038.qgkzgn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.38:6829/703030231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":703030231},{"type":"v1","addr":"172.21.15.38:6829","nonce":703030231}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:40:02.254242+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0,1],"up":{"mds_0":24261,"mds_1":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi121.mvumlh","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6825/1422707571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1422707571},{"type":"v1","addr":"172.21.15.121:6825","nonce":1422707571}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24275":{"gid":24275,"name":"cephfs.smithi121.ovuzki","rank":1,"incarnation":7,"state":"up:stopping","state_seq":2,"addr":"172.21.15.121:6827/562926845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":562926845},{"type":"v1","addr":"172.21.15.121:6827","nonce":562926845}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:17.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:16.431138+0000 mon.smithi038 (mon.0) 742 : audit [DBG] from='client.? 172.21.15.38:0/568806339' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 12, "format": "json"}]: dispatch 2024-02-15T18:44:17.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:16.788534+0000 mon.smithi038 (mon.0) 743 : audit [DBG] from='client.? 172.21.15.38:0/345136027' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 13, "format": "json"}]: dispatch 2024-02-15T18:44:17.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:17 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:17.152129+0000 mon.smithi038 (mon.0) 744 : audit [DBG] from='client.? 172.21.15.38:0/4042345809' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 14, "format": "json"}]: dispatch 2024-02-15T18:44:17.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.866+0000 7f81d3bca700 1 -- 172.21.15.38:0/1250691869 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f81b8060f20 msgr2=0x7f81b80633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:17.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.866+0000 7f81d3bca700 1 --2- 172.21.15.38:0/1250691869 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f81b8060f20 0x7f81b80633e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f81cc108210 tx=0x7f81c8009b30).stop 2024-02-15T18:44:17.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.866+0000 7f81d3bca700 1 -- 172.21.15.38:0/1250691869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81cc0f7730 msgr2=0x7f81cc107210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:17.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.866+0000 7f81d3bca700 1 --2- 172.21.15.38:0/1250691869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81cc0f7730 0x7f81cc107210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f81bc002700 tx=0x7f81bc009e00).stop 2024-02-15T18:44:17.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.866+0000 7f81d3bca700 1 -- 172.21.15.38:0/1250691869 shutdown_connections 2024-02-15T18:44:17.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.866+0000 7f81d3bca700 1 --2- 172.21.15.38:0/1250691869 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f81b8060f20 0x7f81b80633e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:17.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.866+0000 7f81d3bca700 1 --2- 172.21.15.38:0/1250691869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81cc0f6490 0x7f81cc106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:17.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.866+0000 7f81d3bca700 1 --2- 172.21.15.38:0/1250691869 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81cc0f7730 0x7f81cc107210 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:17.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.866+0000 7f81d3bca700 1 -- 172.21.15.38:0/1250691869 >> 172.21.15.38:0/1250691869 conn(0x7f81cc0f1bd0 msgr2=0x7f81cc0fb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:17.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.866+0000 7f81d3bca700 1 -- 172.21.15.38:0/1250691869 shutdown_connections 2024-02-15T18:44:17.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.866+0000 7f81d3bca700 1 -- 172.21.15.38:0/1250691869 wait complete. 2024-02-15T18:44:17.856 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 16 2024-02-15T18:44:17.867 DEBUG:tasks.fs:max_mds reduced in epoch 16 2024-02-15T18:44:17.867 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-02-15T18:44:17.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.999+0000 7fd737c98700 1 -- 172.21.15.38:0/2917288658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7300f9610 msgr2=0x7fd7300fba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:17.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.999+0000 7fd737c98700 1 --2- 172.21.15.38:0/2917288658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7300f9610 0x7fd7300fba00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fd724004660 tx=0x7fd724009b30).stop 2024-02-15T18:44:17.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.999+0000 7fd737c98700 1 -- 172.21.15.38:0/2917288658 shutdown_connections 2024-02-15T18:44:17.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.999+0000 7fd737c98700 1 --2- 172.21.15.38:0/2917288658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7300f9610 0x7fd7300fba00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:17.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.999+0000 7fd737c98700 1 --2- 172.21.15.38:0/2917288658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd73005fd80 0x7fd7300f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:17.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.999+0000 7fd737c98700 1 -- 172.21.15.38:0/2917288658 >> 172.21.15.38:0/2917288658 conn(0x7fd7300f0b60 msgr2=0x7fd7300f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:17.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.999+0000 7fd737c98700 1 -- 172.21.15.38:0/2917288658 shutdown_connections 2024-02-15T18:44:17.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.999+0000 7fd737c98700 1 -- 172.21.15.38:0/2917288658 wait complete. 2024-02-15T18:44:17.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:17.999+0000 7fd737c98700 1 Processor -- start 2024-02-15T18:44:17.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.000+0000 7fd737c98700 1 -- start start 2024-02-15T18:44:17.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.000+0000 7fd737c98700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd73005fd80 0x7fd730106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:17.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.000+0000 7fd737c98700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7300f9610 0x7fd7301070a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:17.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.000+0000 7fd737c98700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd730107650 con 0x7fd7300f9610 2024-02-15T18:44:17.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.000+0000 7fd737c98700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd730107790 con 0x7fd73005fd80 2024-02-15T18:44:17.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.000+0000 7fd735233700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7300f9610 0x7fd7301070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:17.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.000+0000 7fd735233700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7300f9610 0x7fd7301070a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59362/0 (socket says 172.21.15.38:59362) 2024-02-15T18:44:17.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.000+0000 7fd735233700 1 -- 172.21.15.38:0/702536140 learned_addr learned my addr 172.21.15.38:0/702536140 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:17.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.000+0000 7fd735a34700 1 --2- 172.21.15.38:0/702536140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd73005fd80 0x7fd730106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:17.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.000+0000 7fd735233700 1 -- 172.21.15.38:0/702536140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd73005fd80 msgr2=0x7fd730106bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:17.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.000+0000 7fd735233700 1 --2- 172.21.15.38:0/702536140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd73005fd80 0x7fd730106bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:17.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.000+0000 7fd735233700 1 -- 172.21.15.38:0/702536140 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd724005040 con 0x7fd7300f9610 2024-02-15T18:44:17.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.000+0000 7fd735233700 1 --2- 172.21.15.38:0/702536140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7300f9610 0x7fd7301070a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fd724009f50 tx=0x7fd724009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:17.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.000+0000 7fd7227fc700 1 -- 172.21.15.38:0/702536140 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd724018070 con 0x7fd7300f9610 2024-02-15T18:44:17.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.000+0000 7fd737c98700 1 -- 172.21.15.38:0/702536140 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd73010c1d0 con 0x7fd7300f9610 2024-02-15T18:44:17.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.000+0000 7fd7227fc700 1 -- 172.21.15.38:0/702536140 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fd724007ba0 con 0x7fd7300f9610 2024-02-15T18:44:17.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.000+0000 7fd7227fc700 1 -- 172.21.15.38:0/702536140 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd72401c420 con 0x7fd7300f9610 2024-02-15T18:44:17.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.001+0000 7fd737c98700 1 -- 172.21.15.38:0/702536140 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73010c790 con 0x7fd7300f9610 2024-02-15T18:44:17.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.001+0000 7fd737c98700 1 -- 172.21.15.38:0/702536140 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd730027a00 con 0x7fd7300f9610 2024-02-15T18:44:17.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.002+0000 7fd7227fc700 1 -- 172.21.15.38:0/702536140 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fd72401d420 con 0x7fd7300f9610 2024-02-15T18:44:17.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.002+0000 7fd7227fc700 1 --2- 172.21.15.38:0/702536140 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd71c060fe0 0x7fd71c0634a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:17.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.002+0000 7fd7227fc700 1 -- 172.21.15.38:0/702536140 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fd72407de10 con 0x7fd7300f9610 2024-02-15T18:44:17.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.002+0000 7fd735a34700 1 --2- 172.21.15.38:0/702536140 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd71c060fe0 0x7fd71c0634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:17.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.003+0000 7fd735a34700 1 --2- 172.21.15.38:0/702536140 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd71c060fe0 0x7fd71c0634a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fd73005fb50 tx=0x7fd72c00b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:17.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.006+0000 7fd7227fc700 1 -- 172.21.15.38:0/702536140 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd724051c10 con 0x7fd7300f9610 2024-02-15T18:44:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.220+0000 7fd737c98700 1 -- 172.21.15.38:0/702536140 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7fd7300fed20 con 0x7fd7300f9610 2024-02-15T18:44:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.221+0000 7fd7227fc700 1 -- 172.21.15.38:0/702536140 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v32) v1 ==== 107+0+3800 (secure 0 0 0) 0x7fd724051800 con 0x7fd7300f9610 2024-02-15T18:44:18.210 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:18.210 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14446,"name":"cephfs.smithi038.hxoggl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.38:6827/3822880235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3822880235},{"type":"v1","addr":"172.21.15.38:6827","nonce":3822880235}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14456,"name":"cephfs.smithi038.qgkzgn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.38:6829/703030231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":703030231},{"type":"v1","addr":"172.21.15.38:6829","nonce":703030231}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:40:28.060531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi121.mvumlh","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6825/1422707571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1422707571},{"type":"v1","addr":"172.21.15.121:6825","nonce":1422707571}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:18.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.223+0000 7fd737c98700 1 -- 172.21.15.38:0/702536140 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd71c060fe0 msgr2=0x7fd71c0634a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:18.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.223+0000 7fd737c98700 1 --2- 172.21.15.38:0/702536140 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd71c060fe0 0x7fd71c0634a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fd73005fb50 tx=0x7fd72c00b040).stop 2024-02-15T18:44:18.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.223+0000 7fd737c98700 1 -- 172.21.15.38:0/702536140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7300f9610 msgr2=0x7fd7301070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:18.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.223+0000 7fd737c98700 1 --2- 172.21.15.38:0/702536140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7300f9610 0x7fd7301070a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fd724009f50 tx=0x7fd724009b30).stop 2024-02-15T18:44:18.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.223+0000 7fd737c98700 1 -- 172.21.15.38:0/702536140 shutdown_connections 2024-02-15T18:44:18.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.223+0000 7fd737c98700 1 --2- 172.21.15.38:0/702536140 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd71c060fe0 0x7fd71c0634a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:18.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.223+0000 7fd737c98700 1 --2- 172.21.15.38:0/702536140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd73005fd80 0x7fd730106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:18.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.223+0000 7fd737c98700 1 --2- 172.21.15.38:0/702536140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7300f9610 0x7fd7301070a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:18.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.223+0000 7fd737c98700 1 -- 172.21.15.38:0/702536140 >> 172.21.15.38:0/702536140 conn(0x7fd7300f0b60 msgr2=0x7fd7300f2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:18.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.223+0000 7fd737c98700 1 -- 172.21.15.38:0/702536140 shutdown_connections 2024-02-15T18:44:18.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.223+0000 7fd737c98700 1 -- 172.21.15.38:0/702536140 wait complete. 2024-02-15T18:44:18.213 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 17 2024-02-15T18:44:18.227 DEBUG:tasks.fs:max_mds reduced in epoch 17 2024-02-15T18:44:18.227 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-02-15T18:44:18.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.359+0000 7f898ee8e700 1 -- 172.21.15.38:0/927445589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89880f5390 msgr2=0x7f89880f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:18.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.359+0000 7f898ee8e700 1 --2- 172.21.15.38:0/927445589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89880f5390 0x7f89880f57b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f897c004660 tx=0x7f897c009b30).stop 2024-02-15T18:44:18.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.359+0000 7f898ee8e700 1 -- 172.21.15.38:0/927445589 shutdown_connections 2024-02-15T18:44:18.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.359+0000 7f898ee8e700 1 --2- 172.21.15.38:0/927445589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89880f5ca0 0x7f89880f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:18.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.359+0000 7f898ee8e700 1 --2- 172.21.15.38:0/927445589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89880f5390 0x7f89880f57b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:18.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.359+0000 7f898ee8e700 1 -- 172.21.15.38:0/927445589 >> 172.21.15.38:0/927445589 conn(0x7f89880f0b70 msgr2=0x7f89880f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:18.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.359+0000 7f898ee8e700 1 -- 172.21.15.38:0/927445589 shutdown_connections 2024-02-15T18:44:18.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.359+0000 7f898ee8e700 1 -- 172.21.15.38:0/927445589 wait complete. 2024-02-15T18:44:18.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.360+0000 7f898ee8e700 1 Processor -- start 2024-02-15T18:44:18.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.360+0000 7f898ee8e700 1 -- start start 2024-02-15T18:44:18.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.360+0000 7f898ee8e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89880f5390 0x7f8988106cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:18.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.360+0000 7f898ee8e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89880f5ca0 0x7f89881071c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:18.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.360+0000 7f898ee8e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89881077c0 con 0x7f89880f5390 2024-02-15T18:44:18.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.360+0000 7f898ee8e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8988107900 con 0x7f89880f5ca0 2024-02-15T18:44:18.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.360+0000 7f898cc2a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89880f5390 0x7f8988106cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:18.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.360+0000 7f898cc2a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89880f5390 0x7f8988106cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59386/0 (socket says 172.21.15.38:59386) 2024-02-15T18:44:18.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.360+0000 7f898cc2a700 1 -- 172.21.15.38:0/3256085262 learned_addr learned my addr 172.21.15.38:0/3256085262 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:18.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.360+0000 7f8987fff700 1 --2- 172.21.15.38:0/3256085262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89880f5ca0 0x7f89881071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:18.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.360+0000 7f898cc2a700 1 -- 172.21.15.38:0/3256085262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89880f5ca0 msgr2=0x7f89881071c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:18.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.360+0000 7f898cc2a700 1 --2- 172.21.15.38:0/3256085262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89880f5ca0 0x7f89881071c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:18.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.360+0000 7f898cc2a700 1 -- 172.21.15.38:0/3256085262 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c005040 con 0x7f89880f5390 2024-02-15T18:44:18.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.360+0000 7f898cc2a700 1 --2- 172.21.15.38:0/3256085262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89880f5390 0x7f8988106cd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f897c013fd0 tx=0x7f897c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:18.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.361+0000 7f89857fa700 1 -- 172.21.15.38:0/3256085262 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f897c018070 con 0x7f89880f5390 2024-02-15T18:44:18.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.361+0000 7f89857fa700 1 -- 172.21.15.38:0/3256085262 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f897c0097d0 con 0x7f89880f5390 2024-02-15T18:44:18.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.361+0000 7f898ee8e700 1 -- 172.21.15.38:0/3256085262 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898810c2b0 con 0x7f89880f5390 2024-02-15T18:44:18.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.361+0000 7f89857fa700 1 -- 172.21.15.38:0/3256085262 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f897c01c3c0 con 0x7f89880f5390 2024-02-15T18:44:18.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.361+0000 7f898ee8e700 1 -- 172.21.15.38:0/3256085262 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898810c810 con 0x7f89880f5390 2024-02-15T18:44:18.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.362+0000 7f898ee8e700 1 -- 172.21.15.38:0/3256085262 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89880fdc90 con 0x7f89880f5390 2024-02-15T18:44:18.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.362+0000 7f89857fa700 1 -- 172.21.15.38:0/3256085262 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f897c01d420 con 0x7f89880f5390 2024-02-15T18:44:18.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.362+0000 7f89857fa700 1 --2- 172.21.15.38:0/3256085262 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8978060f20 0x7f89780633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:18.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.363+0000 7f89857fa700 1 -- 172.21.15.38:0/3256085262 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f897c07dd70 con 0x7f89880f5390 2024-02-15T18:44:18.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.363+0000 7f8987fff700 1 --2- 172.21.15.38:0/3256085262 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8978060f20 0x7f89780633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:18.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.363+0000 7f8987fff700 1 --2- 172.21.15.38:0/3256085262 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8978060f20 0x7f89780633e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f8974004c40 tx=0x7f897400b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:18.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.367+0000 7f89857fa700 1 -- 172.21.15.38:0/3256085262 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f897c04e1a0 con 0x7f89880f5390 2024-02-15T18:44:18.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:44:17.044187+0000 mgr.smithi038.khesbl (mgr.14664) 414 : cluster [DBG] pgmap v285: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:18.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:17.511408+0000 mon.smithi038 (mon.0) 745 : audit [DBG] from='client.? 172.21.15.38:0/1675295081' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-02-15T18:44:18.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:17.865573+0000 mon.smithi038 (mon.0) 746 : audit [DBG] from='client.? 172.21.15.38:0/1250691869' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-02-15T18:44:18.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:18 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:18.222344+0000 mon.smithi038 (mon.0) 747 : audit [DBG] from='client.? 172.21.15.38:0/702536140' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-02-15T18:44:18.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.581+0000 7f898ee8e700 1 -- 172.21.15.38:0/3256085262 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f898810d260 con 0x7f89880f5390 2024-02-15T18:44:18.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.581+0000 7f89857fa700 1 -- 172.21.15.38:0/3256085262 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v32) v1 ==== 107+0+4579 (secure 0 0 0) 0x7f897c0517c0 con 0x7f89880f5390 2024-02-15T18:44:18.570 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:18.570 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14446,"name":"cephfs.smithi038.hxoggl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.38:6827/3822880235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3822880235},{"type":"v1","addr":"172.21.15.38:6827","nonce":3822880235}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14456,"name":"cephfs.smithi038.qgkzgn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.38:6829/703030231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":703030231},{"type":"v1","addr":"172.21.15.38:6829","nonce":703030231}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":44223,"name":"cephfs.smithi121.ovuzki","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6827/554561289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":554561289},{"type":"v1","addr":"172.21.15.121:6827","nonce":554561289}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:40:28.060531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi121.mvumlh","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6825/1422707571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1422707571},{"type":"v1","addr":"172.21.15.121:6825","nonce":1422707571}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:18.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.583+0000 7f898ee8e700 1 -- 172.21.15.38:0/3256085262 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8978060f20 msgr2=0x7f89780633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:18.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.583+0000 7f898ee8e700 1 --2- 172.21.15.38:0/3256085262 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8978060f20 0x7f89780633e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f8974004c40 tx=0x7f897400b040).stop 2024-02-15T18:44:18.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.583+0000 7f898ee8e700 1 -- 172.21.15.38:0/3256085262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89880f5390 msgr2=0x7f8988106cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:18.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.583+0000 7f898ee8e700 1 --2- 172.21.15.38:0/3256085262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89880f5390 0x7f8988106cd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f897c013fd0 tx=0x7f897c009b30).stop 2024-02-15T18:44:18.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.583+0000 7f898ee8e700 1 -- 172.21.15.38:0/3256085262 shutdown_connections 2024-02-15T18:44:18.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.583+0000 7f898ee8e700 1 --2- 172.21.15.38:0/3256085262 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f8978060f20 0x7f89780633e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:18.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.583+0000 7f898ee8e700 1 --2- 172.21.15.38:0/3256085262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89880f5390 0x7f8988106cd0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:18.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.583+0000 7f898ee8e700 1 --2- 172.21.15.38:0/3256085262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89880f5ca0 0x7f89881071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:18.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.583+0000 7f898ee8e700 1 -- 172.21.15.38:0/3256085262 >> 172.21.15.38:0/3256085262 conn(0x7f89880f0b70 msgr2=0x7f898805fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:18.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.583+0000 7f898ee8e700 1 -- 172.21.15.38:0/3256085262 shutdown_connections 2024-02-15T18:44:18.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.583+0000 7f898ee8e700 1 -- 172.21.15.38:0/3256085262 wait complete. 2024-02-15T18:44:18.573 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 18 2024-02-15T18:44:18.584 DEBUG:tasks.fs:max_mds reduced in epoch 18 2024-02-15T18:44:18.584 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-02-15T18:44:18.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.717+0000 7fd53278a700 1 -- 172.21.15.38:0/4263246530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c0f5390 msgr2=0x7fd52c0f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:18.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.717+0000 7fd53278a700 1 --2- 172.21.15.38:0/4263246530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c0f5390 0x7fd52c0f57b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fd520004660 tx=0x7fd520009b30).stop 2024-02-15T18:44:18.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.717+0000 7fd53278a700 1 -- 172.21.15.38:0/4263246530 shutdown_connections 2024-02-15T18:44:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.717+0000 7fd53278a700 1 --2- 172.21.15.38:0/4263246530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd52c0f6630 0x7fd52c0f6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.718+0000 7fd53278a700 1 --2- 172.21.15.38:0/4263246530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c0f5390 0x7fd52c0f57b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.718+0000 7fd53278a700 1 -- 172.21.15.38:0/4263246530 >> 172.21.15.38:0/4263246530 conn(0x7fd52c0f0b10 msgr2=0x7fd52c0f2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.718+0000 7fd53278a700 1 -- 172.21.15.38:0/4263246530 shutdown_connections 2024-02-15T18:44:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.718+0000 7fd53278a700 1 -- 172.21.15.38:0/4263246530 wait complete. 2024-02-15T18:44:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.718+0000 7fd53278a700 1 Processor -- start 2024-02-15T18:44:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.718+0000 7fd53278a700 1 -- start start 2024-02-15T18:44:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.718+0000 7fd53278a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd52c0f5390 0x7fd52c106ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.718+0000 7fd53278a700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c0f6630 0x7fd52c1071d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.718+0000 7fd53278a700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c1077d0 con 0x7fd52c0f6630 2024-02-15T18:44:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.718+0000 7fd53278a700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c107910 con 0x7fd52c0f5390 2024-02-15T18:44:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.719+0000 7fd52b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c0f6630 0x7fd52c1071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:18.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.719+0000 7fd52b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c0f6630 0x7fd52c1071d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59398/0 (socket says 172.21.15.38:59398) 2024-02-15T18:44:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.719+0000 7fd52b7fe700 1 -- 172.21.15.38:0/3453733822 learned_addr learned my addr 172.21.15.38:0/3453733822 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.719+0000 7fd52bfff700 1 --2- 172.21.15.38:0/3453733822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd52c0f5390 0x7fd52c106ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.719+0000 7fd52b7fe700 1 -- 172.21.15.38:0/3453733822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd52c0f5390 msgr2=0x7fd52c106ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.719+0000 7fd52b7fe700 1 --2- 172.21.15.38:0/3453733822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd52c0f5390 0x7fd52c106ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.719+0000 7fd52b7fe700 1 -- 172.21.15.38:0/3453733822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd520005040 con 0x7fd52c0f6630 2024-02-15T18:44:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.719+0000 7fd52b7fe700 1 --2- 172.21.15.38:0/3453733822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c0f6630 0x7fd52c1071d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fd51c00e7f0 tx=0x7fd51c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.719+0000 7fd528ff9700 1 -- 172.21.15.38:0/3453733822 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd51c0092e0 con 0x7fd52c0f6630 2024-02-15T18:44:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.719+0000 7fd528ff9700 1 -- 172.21.15.38:0/3453733822 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fd51c00f040 con 0x7fd52c0f6630 2024-02-15T18:44:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.719+0000 7fd53278a700 1 -- 172.21.15.38:0/3453733822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52c10c320 con 0x7fd52c0f6630 2024-02-15T18:44:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.719+0000 7fd528ff9700 1 -- 172.21.15.38:0/3453733822 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd51c014850 con 0x7fd52c0f6630 2024-02-15T18:44:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.719+0000 7fd53278a700 1 -- 172.21.15.38:0/3453733822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52c10c8e0 con 0x7fd52c0f6630 2024-02-15T18:44:18.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.720+0000 7fd53278a700 1 -- 172.21.15.38:0/3453733822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52c0fdc60 con 0x7fd52c0f6630 2024-02-15T18:44:18.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.720+0000 7fd528ff9700 1 -- 172.21.15.38:0/3453733822 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fd51c01b030 con 0x7fd52c0f6630 2024-02-15T18:44:18.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.721+0000 7fd528ff9700 1 --2- 172.21.15.38:0/3453733822 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd514060f20 0x7fd5140633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:18.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.721+0000 7fd528ff9700 1 -- 172.21.15.38:0/3453733822 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fd51c07be80 con 0x7fd52c0f6630 2024-02-15T18:44:18.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.721+0000 7fd52bfff700 1 --2- 172.21.15.38:0/3453733822 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd514060f20 0x7fd5140633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:18.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.721+0000 7fd52bfff700 1 --2- 172.21.15.38:0/3453733822 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd514060f20 0x7fd5140633e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fd520004630 tx=0x7fd520016040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:18.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.725+0000 7fd528ff9700 1 -- 172.21.15.38:0/3453733822 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd51c04c360 con 0x7fd52c0f6630 2024-02-15T18:44:18.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:44:17.044187+0000 mgr.smithi038.khesbl (mgr.14664) 414 : cluster [DBG] pgmap v285: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:18.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:17.511408+0000 mon.smithi038 (mon.0) 745 : audit [DBG] from='client.? 172.21.15.38:0/1675295081' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-02-15T18:44:18.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:17.865573+0000 mon.smithi038 (mon.0) 746 : audit [DBG] from='client.? 172.21.15.38:0/1250691869' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-02-15T18:44:18.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:18 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:18.222344+0000 mon.smithi038 (mon.0) 747 : audit [DBG] from='client.? 172.21.15.38:0/702536140' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-02-15T18:44:18.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.944+0000 7fd53278a700 1 -- 172.21.15.38:0/3453733822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7fd52c10cf20 con 0x7fd52c0f6630 2024-02-15T18:44:18.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.945+0000 7fd528ff9700 1 -- 172.21.15.38:0/3453733822 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v32) v1 ==== 107+0+3800 (secure 0 0 0) 0x7fd51c019020 con 0x7fd52c0f6630 2024-02-15T18:44:18.934 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:18.934 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14456,"name":"cephfs.smithi038.qgkzgn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.38:6829/703030231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":703030231},{"type":"v1","addr":"172.21.15.38:6829","nonce":703030231}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":44223,"name":"cephfs.smithi121.ovuzki","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6827/554561289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":554561289},{"type":"v1","addr":"172.21.15.121:6827","nonce":554561289}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:40:28.060531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi121.mvumlh","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6825/1422707571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1422707571},{"type":"v1","addr":"172.21.15.121:6825","nonce":1422707571}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:18.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.947+0000 7fd53278a700 1 -- 172.21.15.38:0/3453733822 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd514060f20 msgr2=0x7fd5140633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:18.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.947+0000 7fd53278a700 1 --2- 172.21.15.38:0/3453733822 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd514060f20 0x7fd5140633e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fd520004630 tx=0x7fd520016040).stop 2024-02-15T18:44:18.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.947+0000 7fd53278a700 1 -- 172.21.15.38:0/3453733822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c0f6630 msgr2=0x7fd52c1071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:18.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.947+0000 7fd53278a700 1 --2- 172.21.15.38:0/3453733822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c0f6630 0x7fd52c1071d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fd51c00e7f0 tx=0x7fd51c00b040).stop 2024-02-15T18:44:18.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.947+0000 7fd53278a700 1 -- 172.21.15.38:0/3453733822 shutdown_connections 2024-02-15T18:44:18.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.947+0000 7fd53278a700 1 --2- 172.21.15.38:0/3453733822 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd514060f20 0x7fd5140633e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:18.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.947+0000 7fd53278a700 1 --2- 172.21.15.38:0/3453733822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd52c0f5390 0x7fd52c106ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:18.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.947+0000 7fd53278a700 1 --2- 172.21.15.38:0/3453733822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd52c0f6630 0x7fd52c1071d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:18.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.947+0000 7fd53278a700 1 -- 172.21.15.38:0/3453733822 >> 172.21.15.38:0/3453733822 conn(0x7fd52c0f0b10 msgr2=0x7fd52c05fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:18.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.947+0000 7fd53278a700 1 -- 172.21.15.38:0/3453733822 shutdown_connections 2024-02-15T18:44:18.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:18.947+0000 7fd53278a700 1 -- 172.21.15.38:0/3453733822 wait complete. 2024-02-15T18:44:18.937 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 19 2024-02-15T18:44:18.948 DEBUG:tasks.fs:max_mds reduced in epoch 19 2024-02-15T18:44:18.948 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-02-15T18:44:19.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.081+0000 7ff40ba37700 1 -- 172.21.15.38:0/1590389967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff40405fe30 msgr2=0x7ff4040f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:19.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.081+0000 7ff40ba37700 1 --2- 172.21.15.38:0/1590389967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff40405fe30 0x7ff4040f9110 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7ff3f4004660 tx=0x7ff3f4009b30).stop 2024-02-15T18:44:19.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.081+0000 7ff40ba37700 1 -- 172.21.15.38:0/1590389967 shutdown_connections 2024-02-15T18:44:19.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.081+0000 7ff40ba37700 1 --2- 172.21.15.38:0/1590389967 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4040f9600 0x7ff4040fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:19.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.081+0000 7ff40ba37700 1 --2- 172.21.15.38:0/1590389967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff40405fe30 0x7ff4040f9110 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:19.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.081+0000 7ff40ba37700 1 -- 172.21.15.38:0/1590389967 >> 172.21.15.38:0/1590389967 conn(0x7ff4040f0b70 msgr2=0x7ff4040f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:19.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.081+0000 7ff40ba37700 1 -- 172.21.15.38:0/1590389967 shutdown_connections 2024-02-15T18:44:19.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.081+0000 7ff40ba37700 1 -- 172.21.15.38:0/1590389967 wait complete. 2024-02-15T18:44:19.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.082+0000 7ff40ba37700 1 Processor -- start 2024-02-15T18:44:19.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.082+0000 7ff40ba37700 1 -- start start 2024-02-15T18:44:19.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.082+0000 7ff40ba37700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff40405fe30 0x7ff404106c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:19.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.082+0000 7ff40ba37700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4040f9600 0x7ff404107180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:19.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.082+0000 7ff40ba37700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff404107780 con 0x7ff4040f9600 2024-02-15T18:44:19.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.082+0000 7ff40ba37700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4041078f0 con 0x7ff40405fe30 2024-02-15T18:44:19.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.082+0000 7ff408fd2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4040f9600 0x7ff404107180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:19.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.082+0000 7ff408fd2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4040f9600 0x7ff404107180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59426/0 (socket says 172.21.15.38:59426) 2024-02-15T18:44:19.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.082+0000 7ff408fd2700 1 -- 172.21.15.38:0/1974767161 learned_addr learned my addr 172.21.15.38:0/1974767161 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:19.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.082+0000 7ff4097d3700 1 --2- 172.21.15.38:0/1974767161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff40405fe30 0x7ff404106c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:19.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.082+0000 7ff408fd2700 1 -- 172.21.15.38:0/1974767161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff40405fe30 msgr2=0x7ff404106c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:19.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.082+0000 7ff408fd2700 1 --2- 172.21.15.38:0/1974767161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff40405fe30 0x7ff404106c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:19.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.082+0000 7ff408fd2700 1 -- 172.21.15.38:0/1974767161 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f4005040 con 0x7ff4040f9600 2024-02-15T18:44:19.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.082+0000 7ff4097d3700 1 --2- 172.21.15.38:0/1974767161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff40405fe30 0x7ff404106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:44:19.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.083+0000 7ff408fd2700 1 --2- 172.21.15.38:0/1974767161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4040f9600 0x7ff404107180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7ff3f80026d0 tx=0x7ff3f8009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:19.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.083+0000 7ff401ffb700 1 -- 172.21.15.38:0/1974767161 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3f8009320 con 0x7ff4040f9600 2024-02-15T18:44:19.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.083+0000 7ff40ba37700 1 -- 172.21.15.38:0/1974767161 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40410c360 con 0x7ff4040f9600 2024-02-15T18:44:19.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.083+0000 7ff401ffb700 1 -- 172.21.15.38:0/1974767161 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7ff3f800f040 con 0x7ff4040f9600 2024-02-15T18:44:19.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.083+0000 7ff401ffb700 1 -- 172.21.15.38:0/1974767161 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3f80149b0 con 0x7ff4040f9600 2024-02-15T18:44:19.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.083+0000 7ff40ba37700 1 -- 172.21.15.38:0/1974767161 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40410c8f0 con 0x7ff4040f9600 2024-02-15T18:44:19.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.084+0000 7ff40ba37700 1 -- 172.21.15.38:0/1974767161 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff404046330 con 0x7ff4040f9600 2024-02-15T18:44:19.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.084+0000 7ff401ffb700 1 -- 172.21.15.38:0/1974767161 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7ff3f8009480 con 0x7ff4040f9600 2024-02-15T18:44:19.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.085+0000 7ff401ffb700 1 --2- 172.21.15.38:0/1974767161 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7ff3ec060f20 0x7ff3ec0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:19.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.085+0000 7ff401ffb700 1 -- 172.21.15.38:0/1974767161 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7ff3f807c790 con 0x7ff4040f9600 2024-02-15T18:44:19.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.085+0000 7ff4097d3700 1 --2- 172.21.15.38:0/1974767161 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7ff3ec060f20 0x7ff3ec0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:19.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.085+0000 7ff4097d3700 1 --2- 172.21.15.38:0/1974767161 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7ff3ec060f20 0x7ff3ec0633e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7ff3f4009d60 tx=0x7ff3f4009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:19.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.089+0000 7ff401ffb700 1 -- 172.21.15.38:0/1974767161 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff3f804cb00 con 0x7ff4040f9600 2024-02-15T18:44:19.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.302+0000 7ff40ba37700 1 -- 172.21.15.38:0/1974767161 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7ff4040586e0 con 0x7ff4040f9600 2024-02-15T18:44:19.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.303+0000 7ff401ffb700 1 -- 172.21.15.38:0/1974767161 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v32) v1 ==== 107+0+4579 (secure 0 0 0) 0x7ff3f8050120 con 0x7ff4040f9600 2024-02-15T18:44:19.292 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:19.292 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14456,"name":"cephfs.smithi038.qgkzgn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.38:6829/703030231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":703030231},{"type":"v1","addr":"172.21.15.38:6829","nonce":703030231}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":34314,"name":"cephfs.smithi038.hxoggl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6827/4077527450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":4077527450},{"type":"v1","addr":"172.21.15.38:6827","nonce":4077527450}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":44223,"name":"cephfs.smithi121.ovuzki","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6827/554561289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":554561289},{"type":"v1","addr":"172.21.15.121:6827","nonce":554561289}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:40:28.060531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi121.mvumlh","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6825/1422707571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1422707571},{"type":"v1","addr":"172.21.15.121:6825","nonce":1422707571}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:19.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.305+0000 7ff40ba37700 1 -- 172.21.15.38:0/1974767161 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7ff3ec060f20 msgr2=0x7ff3ec0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:19.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.305+0000 7ff40ba37700 1 --2- 172.21.15.38:0/1974767161 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7ff3ec060f20 0x7ff3ec0633e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7ff3f4009d60 tx=0x7ff3f4009b30).stop 2024-02-15T18:44:19.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.305+0000 7ff40ba37700 1 -- 172.21.15.38:0/1974767161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4040f9600 msgr2=0x7ff404107180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:19.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.305+0000 7ff40ba37700 1 --2- 172.21.15.38:0/1974767161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4040f9600 0x7ff404107180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7ff3f80026d0 tx=0x7ff3f8009de0).stop 2024-02-15T18:44:19.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.305+0000 7ff40ba37700 1 -- 172.21.15.38:0/1974767161 shutdown_connections 2024-02-15T18:44:19.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.305+0000 7ff40ba37700 1 --2- 172.21.15.38:0/1974767161 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7ff3ec060f20 0x7ff3ec0633e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:19.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.305+0000 7ff40ba37700 1 --2- 172.21.15.38:0/1974767161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff40405fe30 0x7ff404106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:19.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.305+0000 7ff40ba37700 1 --2- 172.21.15.38:0/1974767161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4040f9600 0x7ff404107180 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:19.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.305+0000 7ff40ba37700 1 -- 172.21.15.38:0/1974767161 >> 172.21.15.38:0/1974767161 conn(0x7ff4040f0b70 msgr2=0x7ff4040f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:19.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.305+0000 7ff40ba37700 1 -- 172.21.15.38:0/1974767161 shutdown_connections 2024-02-15T18:44:19.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.305+0000 7ff40ba37700 1 -- 172.21.15.38:0/1974767161 wait complete. 2024-02-15T18:44:19.296 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 20 2024-02-15T18:44:19.306 DEBUG:tasks.fs:max_mds reduced in epoch 20 2024-02-15T18:44:19.306 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-02-15T18:44:19.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.440+0000 7f4af93b2700 1 -- 172.21.15.38:0/1831577692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af40f7730 msgr2=0x7f4af40f7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:19.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.440+0000 7f4af93b2700 1 --2- 172.21.15.38:0/1831577692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af40f7730 0x7f4af40f7b90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f4adc004660 tx=0x7f4adc009b30).stop 2024-02-15T18:44:19.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.441+0000 7f4af93b2700 1 -- 172.21.15.38:0/1831577692 shutdown_connections 2024-02-15T18:44:19.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.441+0000 7f4af93b2700 1 --2- 172.21.15.38:0/1831577692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af40f7730 0x7f4af40f7b90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:19.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.441+0000 7f4af93b2700 1 --2- 172.21.15.38:0/1831577692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4af40f6490 0x7f4af40f68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:19.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.441+0000 7f4af93b2700 1 -- 172.21.15.38:0/1831577692 >> 172.21.15.38:0/1831577692 conn(0x7f4af40f1bd0 msgr2=0x7f4af40f4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:19.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.441+0000 7f4af93b2700 1 -- 172.21.15.38:0/1831577692 shutdown_connections 2024-02-15T18:44:19.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.441+0000 7f4af93b2700 1 -- 172.21.15.38:0/1831577692 wait complete. 2024-02-15T18:44:19.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.441+0000 7f4af93b2700 1 Processor -- start 2024-02-15T18:44:19.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.441+0000 7f4af93b2700 1 -- start start 2024-02-15T18:44:19.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.441+0000 7f4af93b2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af40f6490 0x7f4af4106ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:19.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.441+0000 7f4af93b2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4af40f7730 0x7f4af41071d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:19.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.441+0000 7f4af93b2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af41077d0 con 0x7f4af40f6490 2024-02-15T18:44:19.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.441+0000 7f4af93b2700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af4107910 con 0x7f4af40f7730 2024-02-15T18:44:19.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.442+0000 7f4af2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af40f6490 0x7f4af4106ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:19.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.442+0000 7f4af2ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af40f6490 0x7f4af4106ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59448/0 (socket says 172.21.15.38:59448) 2024-02-15T18:44:19.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.442+0000 7f4af2ffd700 1 -- 172.21.15.38:0/3465728577 learned_addr learned my addr 172.21.15.38:0/3465728577 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:19.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.442+0000 7f4af27fc700 1 --2- 172.21.15.38:0/3465728577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4af40f7730 0x7f4af41071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:19.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.442+0000 7f4af2ffd700 1 -- 172.21.15.38:0/3465728577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4af40f7730 msgr2=0x7f4af41071d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:19.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.442+0000 7f4af2ffd700 1 --2- 172.21.15.38:0/3465728577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4af40f7730 0x7f4af41071d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:19.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.442+0000 7f4af2ffd700 1 -- 172.21.15.38:0/3465728577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc005040 con 0x7f4af40f6490 2024-02-15T18:44:19.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.442+0000 7f4af2ffd700 1 --2- 172.21.15.38:0/3465728577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af40f6490 0x7f4af4106ce0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f4ae4002700 tx=0x7f4ae4009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:19.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.442+0000 7f4aeb7fe700 1 -- 172.21.15.38:0/3465728577 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ae4009320 con 0x7f4af40f6490 2024-02-15T18:44:19.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.442+0000 7f4aeb7fe700 1 -- 172.21.15.38:0/3465728577 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f4ae400f040 con 0x7f4af40f6490 2024-02-15T18:44:19.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.442+0000 7f4af93b2700 1 -- 172.21.15.38:0/3465728577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af410c320 con 0x7f4af40f6490 2024-02-15T18:44:19.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.442+0000 7f4aeb7fe700 1 -- 172.21.15.38:0/3465728577 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ae40149d0 con 0x7f4af40f6490 2024-02-15T18:44:19.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.442+0000 7f4af93b2700 1 -- 172.21.15.38:0/3465728577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af410c8e0 con 0x7f4af40f6490 2024-02-15T18:44:19.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.443+0000 7f4af93b2700 1 -- 172.21.15.38:0/3465728577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af4027a00 con 0x7f4af40f6490 2024-02-15T18:44:19.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.443+0000 7f4aeb7fe700 1 -- 172.21.15.38:0/3465728577 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f4ae4014b30 con 0x7f4af40f6490 2024-02-15T18:44:19.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.444+0000 7f4aeb7fe700 1 --2- 172.21.15.38:0/3465728577 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4ae0060f20 0x7f4ae00633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:19.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.444+0000 7f4aeb7fe700 1 -- 172.21.15.38:0/3465728577 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f4ae407c3c0 con 0x7f4af40f6490 2024-02-15T18:44:19.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.444+0000 7f4af27fc700 1 --2- 172.21.15.38:0/3465728577 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4ae0060f20 0x7f4ae00633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:19.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.444+0000 7f4af27fc700 1 --2- 172.21.15.38:0/3465728577 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4ae0060f20 0x7f4ae00633e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f4adc004e30 tx=0x7f4adc009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:19.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.448+0000 7f4aeb7fe700 1 -- 172.21.15.38:0/3465728577 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4ae404c7f0 con 0x7f4af40f6490 2024-02-15T18:44:19.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.662+0000 7f4af93b2700 1 -- 172.21.15.38:0/3465728577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f4af40fed60 con 0x7f4af40f6490 2024-02-15T18:44:19.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.663+0000 7f4aeb7fe700 1 -- 172.21.15.38:0/3465728577 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v32) v1 ==== 107+0+3803 (secure 0 0 0) 0x7f4ae404fe10 con 0x7f4af40f6490 2024-02-15T18:44:19.652 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:19.652 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34314,"name":"cephfs.smithi038.hxoggl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6827/4077527450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":4077527450},{"type":"v1","addr":"172.21.15.38:6827","nonce":4077527450}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":44223,"name":"cephfs.smithi121.ovuzki","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6827/554561289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":554561289},{"type":"v1","addr":"172.21.15.121:6827","nonce":554561289}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:40:28.060531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi121.mvumlh","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6825/1422707571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1422707571},{"type":"v1","addr":"172.21.15.121:6825","nonce":1422707571}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:19.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.665+0000 7f4af93b2700 1 -- 172.21.15.38:0/3465728577 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4ae0060f20 msgr2=0x7f4ae00633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:19.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.665+0000 7f4af93b2700 1 --2- 172.21.15.38:0/3465728577 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4ae0060f20 0x7f4ae00633e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f4adc004e30 tx=0x7f4adc009b30).stop 2024-02-15T18:44:19.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.665+0000 7f4af93b2700 1 -- 172.21.15.38:0/3465728577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af40f6490 msgr2=0x7f4af4106ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:19.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.665+0000 7f4af93b2700 1 --2- 172.21.15.38:0/3465728577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af40f6490 0x7f4af4106ce0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f4ae4002700 tx=0x7f4ae4009d80).stop 2024-02-15T18:44:19.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.665+0000 7f4af93b2700 1 -- 172.21.15.38:0/3465728577 shutdown_connections 2024-02-15T18:44:19.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.665+0000 7f4af93b2700 1 --2- 172.21.15.38:0/3465728577 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f4ae0060f20 0x7f4ae00633e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:19.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.665+0000 7f4af93b2700 1 --2- 172.21.15.38:0/3465728577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af40f6490 0x7f4af4106ce0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:19.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.665+0000 7f4af93b2700 1 --2- 172.21.15.38:0/3465728577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4af40f7730 0x7f4af41071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:19.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.665+0000 7f4af93b2700 1 -- 172.21.15.38:0/3465728577 >> 172.21.15.38:0/3465728577 conn(0x7f4af40f1bd0 msgr2=0x7f4af40fb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:19.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.665+0000 7f4af93b2700 1 -- 172.21.15.38:0/3465728577 shutdown_connections 2024-02-15T18:44:19.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.665+0000 7f4af93b2700 1 -- 172.21.15.38:0/3465728577 wait complete. 2024-02-15T18:44:19.655 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 21 2024-02-15T18:44:19.665 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:18.582930+0000 mon.smithi038 (mon.0) 748 : audit [DBG] from='client.? 172.21.15.38:0/3256085262' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-02-15T18:44:19.665 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:18.946526+0000 mon.smithi038 (mon.0) 749 : audit [DBG] from='client.? 172.21.15.38:0/3453733822' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-02-15T18:44:19.665 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:19 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:19.304708+0000 mon.smithi038 (mon.0) 750 : audit [DBG] from='client.? 172.21.15.38:0/1974767161' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-02-15T18:44:19.666 DEBUG:tasks.fs:max_mds reduced in epoch 21 2024-02-15T18:44:19.666 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-02-15T18:44:19.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.798+0000 7f410f855700 1 -- 172.21.15.38:0/445827665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41080f6490 msgr2=0x7f41080f68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:19.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.798+0000 7f410f855700 1 --2- 172.21.15.38:0/445827665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41080f6490 0x7f41080f68b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f40f8004660 tx=0x7f40f8009b30).stop 2024-02-15T18:44:19.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.798+0000 7f410f855700 1 -- 172.21.15.38:0/445827665 shutdown_connections 2024-02-15T18:44:19.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.798+0000 7f410f855700 1 --2- 172.21.15.38:0/445827665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41080f7730 0x7f41080f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:19.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.798+0000 7f410f855700 1 --2- 172.21.15.38:0/445827665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41080f6490 0x7f41080f68b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:19.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.798+0000 7f410f855700 1 -- 172.21.15.38:0/445827665 >> 172.21.15.38:0/445827665 conn(0x7f41080f1bd0 msgr2=0x7f41080f4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:19.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.798+0000 7f410f855700 1 -- 172.21.15.38:0/445827665 shutdown_connections 2024-02-15T18:44:19.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.798+0000 7f410f855700 1 -- 172.21.15.38:0/445827665 wait complete. 2024-02-15T18:44:19.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.799+0000 7f410f855700 1 Processor -- start 2024-02-15T18:44:19.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.799+0000 7f410f855700 1 -- start start 2024-02-15T18:44:19.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.799+0000 7f410f855700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41080f6490 0x7f4108106d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:19.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.799+0000 7f410f855700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41080f7730 0x7f41081071f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:19.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.799+0000 7f410f855700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41081077f0 con 0x7f41080f7730 2024-02-15T18:44:19.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.799+0000 7f410f855700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4108107930 con 0x7f41080f6490 2024-02-15T18:44:19.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.799+0000 7f410cdf0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41080f7730 0x7f41081071f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:19.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.799+0000 7f410cdf0700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41080f7730 0x7f41081071f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59466/0 (socket says 172.21.15.38:59466) 2024-02-15T18:44:19.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.799+0000 7f410cdf0700 1 -- 172.21.15.38:0/3651106641 learned_addr learned my addr 172.21.15.38:0/3651106641 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:19.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.799+0000 7f410d5f1700 1 --2- 172.21.15.38:0/3651106641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41080f6490 0x7f4108106d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:19.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.800+0000 7f410cdf0700 1 -- 172.21.15.38:0/3651106641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41080f6490 msgr2=0x7f4108106d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:19.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.800+0000 7f410cdf0700 1 --2- 172.21.15.38:0/3651106641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41080f6490 0x7f4108106d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:19.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.800+0000 7f410cdf0700 1 -- 172.21.15.38:0/3651106641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f8005040 con 0x7f41080f7730 2024-02-15T18:44:19.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.800+0000 7f410cdf0700 1 --2- 172.21.15.38:0/3651106641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41080f7730 0x7f41081071f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f41040026d0 tx=0x7f4104009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:19.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.800+0000 7f40fdffb700 1 -- 172.21.15.38:0/3651106641 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41040079c0 con 0x7f41080f7730 2024-02-15T18:44:19.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.800+0000 7f410f855700 1 -- 172.21.15.38:0/3651106641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f410810c340 con 0x7f41080f7730 2024-02-15T18:44:19.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.800+0000 7f40fdffb700 1 -- 172.21.15.38:0/3651106641 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f410400f040 con 0x7f41080f7730 2024-02-15T18:44:19.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.800+0000 7f40fdffb700 1 -- 172.21.15.38:0/3651106641 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41040149f0 con 0x7f41080f7730 2024-02-15T18:44:19.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.800+0000 7f410f855700 1 -- 172.21.15.38:0/3651106641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410810c900 con 0x7f41080f7730 2024-02-15T18:44:19.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.801+0000 7f410f855700 1 -- 172.21.15.38:0/3651106641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4108027a00 con 0x7f41080f7730 2024-02-15T18:44:19.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.801+0000 7f40fdffb700 1 -- 172.21.15.38:0/3651106641 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f4104014b50 con 0x7f41080f7730 2024-02-15T18:44:19.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.802+0000 7f40fdffb700 1 --2- 172.21.15.38:0/3651106641 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f40f4060f20 0x7f40f40633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:19.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.802+0000 7f40fdffb700 1 -- 172.21.15.38:0/3651106641 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f410407c110 con 0x7f41080f7730 2024-02-15T18:44:19.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.802+0000 7f410d5f1700 1 --2- 172.21.15.38:0/3651106641 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f40f4060f20 0x7f40f40633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:19.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.802+0000 7f410d5f1700 1 --2- 172.21.15.38:0/3651106641 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f40f4060f20 0x7f40f40633e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f40f8005010 tx=0x7f40f8009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:19.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:19.806+0000 7f40fdffb700 1 -- 172.21.15.38:0/3651106641 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f410404c540 con 0x7f41080f7730 2024-02-15T18:44:19.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:18.582930+0000 mon.smithi038 (mon.0) 748 : audit [DBG] from='client.? 172.21.15.38:0/3256085262' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-02-15T18:44:19.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:18.946526+0000 mon.smithi038 (mon.0) 749 : audit [DBG] from='client.? 172.21.15.38:0/3453733822' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-02-15T18:44:19.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:19 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:19.304708+0000 mon.smithi038 (mon.0) 750 : audit [DBG] from='client.? 172.21.15.38:0/1974767161' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-02-15T18:44:20.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.019+0000 7f410f855700 1 -- 172.21.15.38:0/3651106641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f41080fed60 con 0x7f41080f7730 2024-02-15T18:44:20.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.020+0000 7f40fdffb700 1 -- 172.21.15.38:0/3651106641 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v32) v1 ==== 107+0+4582 (secure 0 0 0) 0x7f410404fb60 con 0x7f41080f7730 2024-02-15T18:44:20.009 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:20.009 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34314,"name":"cephfs.smithi038.hxoggl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6827/4077527450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":4077527450},{"type":"v1","addr":"172.21.15.38:6827","nonce":4077527450}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":34318,"name":"cephfs.smithi038.qgkzgn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6829/1616551408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1616551408},{"type":"v1","addr":"172.21.15.38:6829","nonce":1616551408}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44223,"name":"cephfs.smithi121.ovuzki","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6827/554561289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":554561289},{"type":"v1","addr":"172.21.15.121:6827","nonce":554561289}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:40:28.060531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24261},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24261":{"gid":24261,"name":"cephfs.smithi121.mvumlh","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6825/1422707571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1422707571},{"type":"v1","addr":"172.21.15.121:6825","nonce":1422707571}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:20.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.022+0000 7f410f855700 1 -- 172.21.15.38:0/3651106641 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f40f4060f20 msgr2=0x7f40f40633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:20.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.022+0000 7f410f855700 1 --2- 172.21.15.38:0/3651106641 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f40f4060f20 0x7f40f40633e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f40f8005010 tx=0x7f40f8009b30).stop 2024-02-15T18:44:20.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.022+0000 7f410f855700 1 -- 172.21.15.38:0/3651106641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41080f7730 msgr2=0x7f41081071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:20.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.022+0000 7f410f855700 1 --2- 172.21.15.38:0/3651106641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41080f7730 0x7f41081071f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f41040026d0 tx=0x7f4104009de0).stop 2024-02-15T18:44:20.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.022+0000 7f410f855700 1 -- 172.21.15.38:0/3651106641 shutdown_connections 2024-02-15T18:44:20.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.022+0000 7f410f855700 1 --2- 172.21.15.38:0/3651106641 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f40f4060f20 0x7f40f40633e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:20.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.022+0000 7f410f855700 1 --2- 172.21.15.38:0/3651106641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41080f6490 0x7f4108106d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:20.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.022+0000 7f410f855700 1 --2- 172.21.15.38:0/3651106641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41080f7730 0x7f41081071f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:20.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.022+0000 7f410f855700 1 -- 172.21.15.38:0/3651106641 >> 172.21.15.38:0/3651106641 conn(0x7f41080f1bd0 msgr2=0x7f41080fb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:20.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.022+0000 7f410f855700 1 -- 172.21.15.38:0/3651106641 shutdown_connections 2024-02-15T18:44:20.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.022+0000 7f410f855700 1 -- 172.21.15.38:0/3651106641 wait complete. 2024-02-15T18:44:20.012 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 22 2024-02-15T18:44:20.023 DEBUG:tasks.fs:max_mds reduced in epoch 22 2024-02-15T18:44:20.023 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-02-15T18:44:20.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.156+0000 7fee78d29700 1 -- 172.21.15.38:0/3748807150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee740f95f0 msgr2=0x7fee740fba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:20.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.156+0000 7fee78d29700 1 --2- 172.21.15.38:0/3748807150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee740f95f0 0x7fee740fba40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fee68004660 tx=0x7fee68009b30).stop 2024-02-15T18:44:20.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.157+0000 7fee78d29700 1 -- 172.21.15.38:0/3748807150 shutdown_connections 2024-02-15T18:44:20.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.157+0000 7fee78d29700 1 --2- 172.21.15.38:0/3748807150 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee740f95f0 0x7fee740fba40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:20.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.157+0000 7fee78d29700 1 --2- 172.21.15.38:0/3748807150 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee74060320 0x7fee740f9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:20.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.157+0000 7fee78d29700 1 -- 172.21.15.38:0/3748807150 >> 172.21.15.38:0/3748807150 conn(0x7fee740f0b10 msgr2=0x7fee740f2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:20.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.157+0000 7fee78d29700 1 -- 172.21.15.38:0/3748807150 shutdown_connections 2024-02-15T18:44:20.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.157+0000 7fee78d29700 1 -- 172.21.15.38:0/3748807150 wait complete. 2024-02-15T18:44:20.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.157+0000 7fee78d29700 1 Processor -- start 2024-02-15T18:44:20.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.157+0000 7fee78d29700 1 -- start start 2024-02-15T18:44:20.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.157+0000 7fee78d29700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee74060320 0x7fee740f6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:20.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.157+0000 7fee78d29700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee740f95f0 0x7fee740f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:20.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.157+0000 7fee78d29700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee740f6ec0 con 0x7fee74060320 2024-02-15T18:44:20.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.157+0000 7fee78d29700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee740f5460 con 0x7fee740f95f0 2024-02-15T18:44:20.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.157+0000 7fee7259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee74060320 0x7fee740f6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:20.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.157+0000 7fee7259c700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee74060320 0x7fee740f6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59492/0 (socket says 172.21.15.38:59492) 2024-02-15T18:44:20.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.157+0000 7fee7259c700 1 -- 172.21.15.38:0/715346363 learned_addr learned my addr 172.21.15.38:0/715346363 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:20.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.158+0000 7fee71d9b700 1 --2- 172.21.15.38:0/715346363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee740f95f0 0x7fee740f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:20.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.158+0000 7fee7259c700 1 -- 172.21.15.38:0/715346363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee740f95f0 msgr2=0x7fee740f4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:20.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.158+0000 7fee7259c700 1 --2- 172.21.15.38:0/715346363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee740f95f0 0x7fee740f4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:20.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.158+0000 7fee7259c700 1 -- 172.21.15.38:0/715346363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee68005040 con 0x7fee74060320 2024-02-15T18:44:20.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.158+0000 7fee7259c700 1 --2- 172.21.15.38:0/715346363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee74060320 0x7fee740f6910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fee640026d0 tx=0x7fee64009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:20.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.158+0000 7fee62ffd700 1 -- 172.21.15.38:0/715346363 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee64009320 con 0x7fee74060320 2024-02-15T18:44:20.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.158+0000 7fee62ffd700 1 -- 172.21.15.38:0/715346363 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fee6400f040 con 0x7fee74060320 2024-02-15T18:44:20.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.158+0000 7fee78d29700 1 -- 172.21.15.38:0/715346363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee740f56e0 con 0x7fee74060320 2024-02-15T18:44:20.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.158+0000 7fee62ffd700 1 -- 172.21.15.38:0/715346363 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee640149b0 con 0x7fee74060320 2024-02-15T18:44:20.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.158+0000 7fee78d29700 1 -- 172.21.15.38:0/715346363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee740f5ca0 con 0x7fee74060320 2024-02-15T18:44:20.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.159+0000 7fee78d29700 1 -- 172.21.15.38:0/715346363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee74027a00 con 0x7fee74060320 2024-02-15T18:44:20.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.159+0000 7fee62ffd700 1 -- 172.21.15.38:0/715346363 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fee64009480 con 0x7fee74060320 2024-02-15T18:44:20.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.160+0000 7fee62ffd700 1 --2- 172.21.15.38:0/715346363 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fee5c060f20 0x7fee5c0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:20.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.160+0000 7fee62ffd700 1 -- 172.21.15.38:0/715346363 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fee6407c620 con 0x7fee74060320 2024-02-15T18:44:20.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.160+0000 7fee71d9b700 1 --2- 172.21.15.38:0/715346363 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fee5c060f20 0x7fee5c0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:20.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.160+0000 7fee71d9b700 1 --2- 172.21.15.38:0/715346363 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fee5c060f20 0x7fee5c0633e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fee68004e30 tx=0x7fee68009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:20.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.164+0000 7fee62ffd700 1 -- 172.21.15.38:0/715346363 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fee64050500 con 0x7fee74060320 2024-02-15T18:44:20.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.380+0000 7fee78d29700 1 -- 172.21.15.38:0/715346363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7fee740fed60 con 0x7fee74060320 2024-02-15T18:44:20.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.380+0000 7fee62ffd700 1 -- 172.21.15.38:0/715346363 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v32) v1 ==== 107+0+3791 (secure 0 0 0) 0x7fee640500f0 con 0x7fee74060320 2024-02-15T18:44:20.369 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:20.369 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34314,"name":"cephfs.smithi038.hxoggl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6827/4077527450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":4077527450},{"type":"v1","addr":"172.21.15.38:6827","nonce":4077527450}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":34318,"name":"cephfs.smithi038.qgkzgn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6829/1616551408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1616551408},{"type":"v1","addr":"172.21.15.38:6829","nonce":1616551408}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44223,"name":"cephfs.smithi121.ovuzki","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6827/554561289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":554561289},{"type":"v1","addr":"172.21.15.121:6827","nonce":554561289}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:41:03.946565+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[1],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:20.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.382+0000 7fee78d29700 1 -- 172.21.15.38:0/715346363 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fee5c060f20 msgr2=0x7fee5c0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:20.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.382+0000 7fee78d29700 1 --2- 172.21.15.38:0/715346363 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fee5c060f20 0x7fee5c0633e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fee68004e30 tx=0x7fee68009b30).stop 2024-02-15T18:44:20.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.382+0000 7fee78d29700 1 -- 172.21.15.38:0/715346363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee74060320 msgr2=0x7fee740f6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:20.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.382+0000 7fee78d29700 1 --2- 172.21.15.38:0/715346363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee74060320 0x7fee740f6910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fee640026d0 tx=0x7fee64009de0).stop 2024-02-15T18:44:20.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.382+0000 7fee78d29700 1 -- 172.21.15.38:0/715346363 shutdown_connections 2024-02-15T18:44:20.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.382+0000 7fee78d29700 1 --2- 172.21.15.38:0/715346363 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fee5c060f20 0x7fee5c0633e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:20.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.382+0000 7fee78d29700 1 --2- 172.21.15.38:0/715346363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee74060320 0x7fee740f6910 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:20.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.382+0000 7fee78d29700 1 --2- 172.21.15.38:0/715346363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee740f95f0 0x7fee740f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:20.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.382+0000 7fee78d29700 1 -- 172.21.15.38:0/715346363 >> 172.21.15.38:0/715346363 conn(0x7fee740f0b10 msgr2=0x7fee740f2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:20.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.382+0000 7fee78d29700 1 -- 172.21.15.38:0/715346363 shutdown_connections 2024-02-15T18:44:20.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.382+0000 7fee78d29700 1 -- 172.21.15.38:0/715346363 wait complete. 2024-02-15T18:44:20.372 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 23 2024-02-15T18:44:20.384 DEBUG:tasks.fs:max_mds reduced in epoch 23 2024-02-15T18:44:20.384 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-02-15T18:44:20.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.517+0000 7eff3de54700 1 -- 172.21.15.38:0/2060799646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3805fd90 msgr2=0x7eff380f91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.517+0000 7eff3de54700 1 --2- 172.21.15.38:0/2060799646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3805fd90 0x7eff380f91b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7eff2c004660 tx=0x7eff2c009b30).stop 2024-02-15T18:44:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.517+0000 7eff3de54700 1 -- 172.21.15.38:0/2060799646 shutdown_connections 2024-02-15T18:44:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.517+0000 7eff3de54700 1 --2- 172.21.15.38:0/2060799646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff380f96a0 0x7eff380fba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.517+0000 7eff3de54700 1 --2- 172.21.15.38:0/2060799646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3805fd90 0x7eff380f91b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.517+0000 7eff3de54700 1 -- 172.21.15.38:0/2060799646 >> 172.21.15.38:0/2060799646 conn(0x7eff380f0b70 msgr2=0x7eff380f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.517+0000 7eff3de54700 1 -- 172.21.15.38:0/2060799646 shutdown_connections 2024-02-15T18:44:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.517+0000 7eff3de54700 1 -- 172.21.15.38:0/2060799646 wait complete. 2024-02-15T18:44:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.517+0000 7eff3de54700 1 Processor -- start 2024-02-15T18:44:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.518+0000 7eff3de54700 1 -- start start 2024-02-15T18:44:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.518+0000 7eff3de54700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3805fd90 0x7eff38106cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.518+0000 7eff3de54700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff380f96a0 0x7eff381071e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.518+0000 7eff3de54700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff381077e0 con 0x7eff3805fd90 2024-02-15T18:44:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.518+0000 7eff3de54700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff38107920 con 0x7eff380f96a0 2024-02-15T18:44:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.518+0000 7eff377fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3805fd90 0x7eff38106cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.518+0000 7eff377fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3805fd90 0x7eff38106cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59516/0 (socket says 172.21.15.38:59516) 2024-02-15T18:44:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.518+0000 7eff377fe700 1 -- 172.21.15.38:0/4116223456 learned_addr learned my addr 172.21.15.38:0/4116223456 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.518+0000 7eff36ffd700 1 --2- 172.21.15.38:0/4116223456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff380f96a0 0x7eff381071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.518+0000 7eff377fe700 1 -- 172.21.15.38:0/4116223456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff380f96a0 msgr2=0x7eff381071e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.518+0000 7eff377fe700 1 --2- 172.21.15.38:0/4116223456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff380f96a0 0x7eff381071e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.518+0000 7eff377fe700 1 -- 172.21.15.38:0/4116223456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c005040 con 0x7eff3805fd90 2024-02-15T18:44:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.518+0000 7eff36ffd700 1 --2- 172.21.15.38:0/4116223456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff380f96a0 0x7eff381071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:44:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.518+0000 7eff377fe700 1 --2- 172.21.15.38:0/4116223456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3805fd90 0x7eff38106cf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7eff2c004e30 tx=0x7eff2c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.519+0000 7eff1ffff700 1 -- 172.21.15.38:0/4116223456 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff2c018070 con 0x7eff3805fd90 2024-02-15T18:44:20.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.519+0000 7eff3de54700 1 -- 172.21.15.38:0/4116223456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3810c2d0 con 0x7eff3805fd90 2024-02-15T18:44:20.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.519+0000 7eff1ffff700 1 -- 172.21.15.38:0/4116223456 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7eff2c007ba0 con 0x7eff3805fd90 2024-02-15T18:44:20.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.519+0000 7eff1ffff700 1 -- 172.21.15.38:0/4116223456 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff2c01c620 con 0x7eff3805fd90 2024-02-15T18:44:20.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.519+0000 7eff3de54700 1 -- 172.21.15.38:0/4116223456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3810c830 con 0x7eff3805fd90 2024-02-15T18:44:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.520+0000 7eff3de54700 1 -- 172.21.15.38:0/4116223456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff38046330 con 0x7eff3805fd90 2024-02-15T18:44:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.520+0000 7eff1ffff700 1 -- 172.21.15.38:0/4116223456 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7eff2c01d420 con 0x7eff3805fd90 2024-02-15T18:44:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.520+0000 7eff1ffff700 1 --2- 172.21.15.38:0/4116223456 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7eff20060f20 0x7eff200633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.520+0000 7eff1ffff700 1 -- 172.21.15.38:0/4116223456 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7eff2c07df00 con 0x7eff3805fd90 2024-02-15T18:44:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.520+0000 7eff36ffd700 1 --2- 172.21.15.38:0/4116223456 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7eff20060f20 0x7eff200633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:20.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.521+0000 7eff36ffd700 1 --2- 172.21.15.38:0/4116223456 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7eff20060f20 0x7eff200633e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7eff28009fd0 tx=0x7eff2800b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:20.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.524+0000 7eff1ffff700 1 -- 172.21.15.38:0/4116223456 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7eff2c04e330 con 0x7eff3805fd90 2024-02-15T18:44:20.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.738+0000 7eff3de54700 1 -- 172.21.15.38:0/4116223456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7eff38027a00 con 0x7eff3805fd90 2024-02-15T18:44:20.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.739+0000 7eff1ffff700 1 -- 172.21.15.38:0/4116223456 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v32) v1 ==== 107+0+3802 (secure 0 0 0) 0x7eff2c051950 con 0x7eff3805fd90 2024-02-15T18:44:20.728 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:20.728 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34318,"name":"cephfs.smithi038.qgkzgn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6829/1616551408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1616551408},{"type":"v1","addr":"172.21.15.38:6829","nonce":1616551408}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44223,"name":"cephfs.smithi121.ovuzki","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6827/554561289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":554561289},{"type":"v1","addr":"172.21.15.121:6827","nonce":554561289}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:41:03.954888+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34314},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34314":{"gid":34314,"name":"cephfs.smithi038.hxoggl","rank":0,"incarnation":24,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6827/4077527450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":4077527450},{"type":"v1","addr":"172.21.15.38:6827","nonce":4077527450}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:20.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.741+0000 7eff3de54700 1 -- 172.21.15.38:0/4116223456 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7eff20060f20 msgr2=0x7eff200633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:20.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.741+0000 7eff3de54700 1 --2- 172.21.15.38:0/4116223456 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7eff20060f20 0x7eff200633e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7eff28009fd0 tx=0x7eff2800b040).stop 2024-02-15T18:44:20.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.741+0000 7eff3de54700 1 -- 172.21.15.38:0/4116223456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3805fd90 msgr2=0x7eff38106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:20.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.741+0000 7eff3de54700 1 --2- 172.21.15.38:0/4116223456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3805fd90 0x7eff38106cf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7eff2c004e30 tx=0x7eff2c009b30).stop 2024-02-15T18:44:20.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.741+0000 7eff3de54700 1 -- 172.21.15.38:0/4116223456 shutdown_connections 2024-02-15T18:44:20.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.741+0000 7eff3de54700 1 --2- 172.21.15.38:0/4116223456 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7eff20060f20 0x7eff200633e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:20.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.741+0000 7eff3de54700 1 --2- 172.21.15.38:0/4116223456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3805fd90 0x7eff38106cf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:20.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.741+0000 7eff3de54700 1 --2- 172.21.15.38:0/4116223456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff380f96a0 0x7eff381071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:20.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.741+0000 7eff3de54700 1 -- 172.21.15.38:0/4116223456 >> 172.21.15.38:0/4116223456 conn(0x7eff380f0b70 msgr2=0x7eff380f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:20.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.741+0000 7eff3de54700 1 -- 172.21.15.38:0/4116223456 shutdown_connections 2024-02-15T18:44:20.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.741+0000 7eff3de54700 1 -- 172.21.15.38:0/4116223456 wait complete. 2024-02-15T18:44:20.731 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 24 2024-02-15T18:44:20.741 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:20 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:44:19.044579+0000 mgr.smithi038.khesbl (mgr.14664) 415 : cluster [DBG] pgmap v286: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:20.741 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:20 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:19.664535+0000 mon.smithi038 (mon.0) 751 : audit [DBG] from='client.? 172.21.15.38:0/3465728577' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-02-15T18:44:20.741 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:20 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:20.021654+0000 mon.smithi038 (mon.0) 752 : audit [DBG] from='client.? 172.21.15.38:0/3651106641' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-02-15T18:44:20.741 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:20 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:20.381950+0000 mon.smithi038 (mon.0) 753 : audit [DBG] from='client.? 172.21.15.38:0/715346363' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-02-15T18:44:20.742 DEBUG:tasks.fs:max_mds reduced in epoch 24 2024-02-15T18:44:20.743 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-02-15T18:44:20.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:20 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:44:19.044579+0000 mgr.smithi038.khesbl (mgr.14664) 415 : cluster [DBG] pgmap v286: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:20.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:20 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:19.664535+0000 mon.smithi038 (mon.0) 751 : audit [DBG] from='client.? 172.21.15.38:0/3465728577' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-02-15T18:44:20.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:20 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:20.021654+0000 mon.smithi038 (mon.0) 752 : audit [DBG] from='client.? 172.21.15.38:0/3651106641' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-02-15T18:44:20.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:20 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:20.381950+0000 mon.smithi038 (mon.0) 753 : audit [DBG] from='client.? 172.21.15.38:0/715346363' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-02-15T18:44:20.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.875+0000 7f3ab9b98700 1 -- 172.21.15.38:0/3057908862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab40f5390 msgr2=0x7f3ab40f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:20.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.875+0000 7f3ab9b98700 1 --2- 172.21.15.38:0/3057908862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab40f5390 0x7f3ab40f57b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f3a9c004660 tx=0x7f3a9c009b30).stop 2024-02-15T18:44:20.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.875+0000 7f3ab9b98700 1 -- 172.21.15.38:0/3057908862 shutdown_connections 2024-02-15T18:44:20.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.875+0000 7f3ab9b98700 1 --2- 172.21.15.38:0/3057908862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ab40f5ca0 0x7f3ab40f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:20.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.875+0000 7f3ab9b98700 1 --2- 172.21.15.38:0/3057908862 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab40f5390 0x7f3ab40f57b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:20.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.875+0000 7f3ab9b98700 1 -- 172.21.15.38:0/3057908862 >> 172.21.15.38:0/3057908862 conn(0x7f3ab40f0b70 msgr2=0x7f3ab40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:20.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.875+0000 7f3ab9b98700 1 -- 172.21.15.38:0/3057908862 shutdown_connections 2024-02-15T18:44:20.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.875+0000 7f3ab9b98700 1 -- 172.21.15.38:0/3057908862 wait complete. 2024-02-15T18:44:20.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.875+0000 7f3ab9b98700 1 Processor -- start 2024-02-15T18:44:20.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.876+0000 7f3ab9b98700 1 -- start start 2024-02-15T18:44:20.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.876+0000 7f3ab9b98700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab40f5390 0x7f3ab405fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:20.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.876+0000 7f3ab9b98700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ab40f5ca0 0x7f3ab410baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:20.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.876+0000 7f3ab9b98700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab410bf90 con 0x7f3ab40f5390 2024-02-15T18:44:20.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.876+0000 7f3ab9b98700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab410c0d0 con 0x7f3ab40f5ca0 2024-02-15T18:44:20.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.876+0000 7f3ab37fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab40f5390 0x7f3ab405fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:20.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.876+0000 7f3ab37fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab40f5390 0x7f3ab405fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59530/0 (socket says 172.21.15.38:59530) 2024-02-15T18:44:20.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.876+0000 7f3ab37fe700 1 -- 172.21.15.38:0/1915847708 learned_addr learned my addr 172.21.15.38:0/1915847708 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:20.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.876+0000 7f3ab37fe700 1 -- 172.21.15.38:0/1915847708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ab40f5ca0 msgr2=0x7f3ab410baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:20.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.876+0000 7f3ab2ffd700 1 --2- 172.21.15.38:0/1915847708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ab40f5ca0 0x7f3ab410baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:20.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.876+0000 7f3ab37fe700 1 --2- 172.21.15.38:0/1915847708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ab40f5ca0 0x7f3ab410baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:20.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.876+0000 7f3ab37fe700 1 -- 172.21.15.38:0/1915847708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c005040 con 0x7f3ab40f5390 2024-02-15T18:44:20.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.876+0000 7f3ab37fe700 1 --2- 172.21.15.38:0/1915847708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab40f5390 0x7f3ab405fcb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f3a9c0096f0 tx=0x7f3a9c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:20.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.877+0000 7f3aabfff700 1 -- 172.21.15.38:0/1915847708 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a9c019070 con 0x7f3ab40f5390 2024-02-15T18:44:20.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.877+0000 7f3ab9b98700 1 -- 172.21.15.38:0/1915847708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab410c2d0 con 0x7f3ab40f5390 2024-02-15T18:44:20.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.877+0000 7f3aabfff700 1 -- 172.21.15.38:0/1915847708 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f3a9c014420 con 0x7f3ab40f5390 2024-02-15T18:44:20.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.877+0000 7f3aabfff700 1 -- 172.21.15.38:0/1915847708 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a9c01d600 con 0x7f3ab40f5390 2024-02-15T18:44:20.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.877+0000 7f3ab9b98700 1 -- 172.21.15.38:0/1915847708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab410c830 con 0x7f3ab40f5390 2024-02-15T18:44:20.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.878+0000 7f3ab9b98700 1 -- 172.21.15.38:0/1915847708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab4027a00 con 0x7f3ab40f5390 2024-02-15T18:44:20.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.878+0000 7f3aabfff700 1 -- 172.21.15.38:0/1915847708 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f3a9c025070 con 0x7f3ab40f5390 2024-02-15T18:44:20.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.878+0000 7f3aabfff700 1 --2- 172.21.15.38:0/1915847708 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3aa0060f20 0x7f3aa00633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:20.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.878+0000 7f3aabfff700 1 -- 172.21.15.38:0/1915847708 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f3a9c07d0e0 con 0x7f3ab40f5390 2024-02-15T18:44:20.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.878+0000 7f3ab2ffd700 1 --2- 172.21.15.38:0/1915847708 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3aa0060f20 0x7f3aa00633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:20.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.879+0000 7f3ab2ffd700 1 --2- 172.21.15.38:0/1915847708 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3aa0060f20 0x7f3aa00633e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f3aa4009da0 tx=0x7f3aa400b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:20.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:20.883+0000 7f3aabfff700 1 -- 172.21.15.38:0/1915847708 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3a9c050ee0 con 0x7f3ab40f5390 2024-02-15T18:44:21.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.099+0000 7f3ab9b98700 1 -- 172.21.15.38:0/1915847708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f3ab40586e0 con 0x7f3ab40f5390 2024-02-15T18:44:21.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.100+0000 7f3aabfff700 1 -- 172.21.15.38:0/1915847708 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v32) v1 ==== 107+0+4584 (secure 0 0 0) 0x7f3a9c023020 con 0x7f3ab40f5390 2024-02-15T18:44:21.088 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:21.089 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34318,"name":"cephfs.smithi038.qgkzgn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6829/1616551408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1616551408},{"type":"v1","addr":"172.21.15.38:6829","nonce":1616551408}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44223,"name":"cephfs.smithi121.ovuzki","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6827/554561289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":554561289},{"type":"v1","addr":"172.21.15.121:6827","nonce":554561289}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":44261,"name":"cephfs.smithi121.mvumlh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6825/180558011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":180558011},{"type":"v1","addr":"172.21.15.121:6825","nonce":180558011}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:41:12.074295+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34314},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34314":{"gid":34314,"name":"cephfs.smithi038.hxoggl","rank":0,"incarnation":24,"state":"up:reconnect","state_seq":9,"addr":"172.21.15.38:6827/4077527450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":4077527450},{"type":"v1","addr":"172.21.15.38:6827","nonce":4077527450}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:21.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.101+0000 7f3ab9b98700 1 -- 172.21.15.38:0/1915847708 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3aa0060f20 msgr2=0x7f3aa00633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:21.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.101+0000 7f3ab9b98700 1 --2- 172.21.15.38:0/1915847708 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3aa0060f20 0x7f3aa00633e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f3aa4009da0 tx=0x7f3aa400b040).stop 2024-02-15T18:44:21.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.101+0000 7f3ab9b98700 1 -- 172.21.15.38:0/1915847708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab40f5390 msgr2=0x7f3ab405fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:21.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.101+0000 7f3ab9b98700 1 --2- 172.21.15.38:0/1915847708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab40f5390 0x7f3ab405fcb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f3a9c0096f0 tx=0x7f3a9c00b040).stop 2024-02-15T18:44:21.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.101+0000 7f3ab9b98700 1 -- 172.21.15.38:0/1915847708 shutdown_connections 2024-02-15T18:44:21.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.101+0000 7f3ab9b98700 1 --2- 172.21.15.38:0/1915847708 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3aa0060f20 0x7f3aa00633e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:21.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.101+0000 7f3ab9b98700 1 --2- 172.21.15.38:0/1915847708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ab40f5390 0x7f3ab405fcb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:21.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.101+0000 7f3ab9b98700 1 --2- 172.21.15.38:0/1915847708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ab40f5ca0 0x7f3ab410baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:21.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.101+0000 7f3ab9b98700 1 -- 172.21.15.38:0/1915847708 >> 172.21.15.38:0/1915847708 conn(0x7f3ab40f0b70 msgr2=0x7f3ab40fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:21.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.101+0000 7f3ab9b98700 1 -- 172.21.15.38:0/1915847708 shutdown_connections 2024-02-15T18:44:21.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.101+0000 7f3ab9b98700 1 -- 172.21.15.38:0/1915847708 wait complete. 2024-02-15T18:44:21.091 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 25 2024-02-15T18:44:21.102 DEBUG:tasks.fs:max_mds reduced in epoch 25 2024-02-15T18:44:21.102 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-02-15T18:44:21.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.235+0000 7f3855626700 1 -- 172.21.15.38:0/468224072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38500f5390 msgr2=0x7f38500f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.235+0000 7f3855626700 1 --2- 172.21.15.38:0/468224072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38500f5390 0x7f38500f57b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f3840004660 tx=0x7f3840009b30).stop 2024-02-15T18:44:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.235+0000 7f3855626700 1 -- 172.21.15.38:0/468224072 shutdown_connections 2024-02-15T18:44:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.235+0000 7f3855626700 1 --2- 172.21.15.38:0/468224072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38500f6630 0x7f38500f6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.235+0000 7f3855626700 1 --2- 172.21.15.38:0/468224072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38500f5390 0x7f38500f57b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.235+0000 7f3855626700 1 -- 172.21.15.38:0/468224072 >> 172.21.15.38:0/468224072 conn(0x7f38500f0b10 msgr2=0x7f38500f2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.235+0000 7f3855626700 1 -- 172.21.15.38:0/468224072 shutdown_connections 2024-02-15T18:44:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.235+0000 7f3855626700 1 -- 172.21.15.38:0/468224072 wait complete. 2024-02-15T18:44:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.236+0000 7f3855626700 1 Processor -- start 2024-02-15T18:44:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.236+0000 7f3855626700 1 -- start start 2024-02-15T18:44:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.236+0000 7f3855626700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38500f5390 0x7f3850108f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:21.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.236+0000 7f3855626700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38500f6630 0x7f3850109400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:21.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.236+0000 7f3855626700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38501099b0 con 0x7f38500f6630 2024-02-15T18:44:21.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.236+0000 7f3855626700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3850109af0 con 0x7f38500f5390 2024-02-15T18:44:21.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.236+0000 7f384e7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38500f6630 0x7f3850109400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:21.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.236+0000 7f384e7fc700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38500f6630 0x7f3850109400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59550/0 (socket says 172.21.15.38:59550) 2024-02-15T18:44:21.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.236+0000 7f384e7fc700 1 -- 172.21.15.38:0/2731212215 learned_addr learned my addr 172.21.15.38:0/2731212215 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:21.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.236+0000 7f384effd700 1 --2- 172.21.15.38:0/2731212215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38500f5390 0x7f3850108f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:21.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.236+0000 7f384e7fc700 1 -- 172.21.15.38:0/2731212215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38500f5390 msgr2=0x7f3850108f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:21.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.236+0000 7f384e7fc700 1 --2- 172.21.15.38:0/2731212215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38500f5390 0x7f3850108f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:21.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.236+0000 7f384e7fc700 1 -- 172.21.15.38:0/2731212215 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3840005040 con 0x7f38500f6630 2024-02-15T18:44:21.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.237+0000 7f384e7fc700 1 --2- 172.21.15.38:0/2731212215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38500f6630 0x7f3850109400 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f38440026d0 tx=0x7f3844009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:21.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.237+0000 7f38377fe700 1 -- 172.21.15.38:0/2731212215 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38440079c0 con 0x7f38500f6630 2024-02-15T18:44:21.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.237+0000 7f3855626700 1 -- 172.21.15.38:0/2731212215 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385010e540 con 0x7f38500f6630 2024-02-15T18:44:21.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.237+0000 7f38377fe700 1 -- 172.21.15.38:0/2731212215 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f384400f040 con 0x7f38500f6630 2024-02-15T18:44:21.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.237+0000 7f38377fe700 1 -- 172.21.15.38:0/2731212215 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38440149f0 con 0x7f38500f6630 2024-02-15T18:44:21.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.237+0000 7f3855626700 1 -- 172.21.15.38:0/2731212215 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385010eb00 con 0x7f38500f6630 2024-02-15T18:44:21.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.238+0000 7f3855626700 1 -- 172.21.15.38:0/2731212215 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38500fdc60 con 0x7f38500f6630 2024-02-15T18:44:21.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.238+0000 7f38377fe700 1 -- 172.21.15.38:0/2731212215 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f3844007b20 con 0x7f38500f6630 2024-02-15T18:44:21.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.238+0000 7f38377fe700 1 --2- 172.21.15.38:0/2731212215 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3838060f20 0x7f38380633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:21.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.239+0000 7f38377fe700 1 -- 172.21.15.38:0/2731212215 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f384407c140 con 0x7f38500f6630 2024-02-15T18:44:21.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.239+0000 7f384effd700 1 --2- 172.21.15.38:0/2731212215 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3838060f20 0x7f38380633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:21.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.239+0000 7f384effd700 1 --2- 172.21.15.38:0/2731212215 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3838060f20 0x7f38380633e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f3840004630 tx=0x7f3840009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:21.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.243+0000 7f38377fe700 1 -- 172.21.15.38:0/2731212215 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f384404ffa0 con 0x7f38500f6630 2024-02-15T18:44:21.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.460+0000 7f3855626700 1 -- 172.21.15.38:0/2731212215 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f385010f490 con 0x7f38500f6630 2024-02-15T18:44:21.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.460+0000 7f38377fe700 1 -- 172.21.15.38:0/2731212215 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v32) v1 ==== 107+0+4582 (secure 0 0 0) 0x7f384404fb90 con 0x7f38500f6630 2024-02-15T18:44:21.449 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:21.449 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34318,"name":"cephfs.smithi038.qgkzgn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6829/1616551408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1616551408},{"type":"v1","addr":"172.21.15.38:6829","nonce":1616551408}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44223,"name":"cephfs.smithi121.ovuzki","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6827/554561289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":554561289},{"type":"v1","addr":"172.21.15.121:6827","nonce":554561289}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":44261,"name":"cephfs.smithi121.mvumlh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6825/180558011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":180558011},{"type":"v1","addr":"172.21.15.121:6825","nonce":180558011}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:41:13.076011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34314},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34314":{"gid":34314,"name":"cephfs.smithi038.hxoggl","rank":0,"incarnation":24,"state":"up:rejoin","state_seq":10,"addr":"172.21.15.38:6827/4077527450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":4077527450},{"type":"v1","addr":"172.21.15.38:6827","nonce":4077527450}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.462+0000 7f3855626700 1 -- 172.21.15.38:0/2731212215 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3838060f20 msgr2=0x7f38380633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.462+0000 7f3855626700 1 --2- 172.21.15.38:0/2731212215 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3838060f20 0x7f38380633e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f3840004630 tx=0x7f3840009b30).stop 2024-02-15T18:44:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.462+0000 7f3855626700 1 -- 172.21.15.38:0/2731212215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38500f6630 msgr2=0x7f3850109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.462+0000 7f3855626700 1 --2- 172.21.15.38:0/2731212215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38500f6630 0x7f3850109400 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f38440026d0 tx=0x7f3844009de0).stop 2024-02-15T18:44:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.462+0000 7f3855626700 1 -- 172.21.15.38:0/2731212215 shutdown_connections 2024-02-15T18:44:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.462+0000 7f3855626700 1 --2- 172.21.15.38:0/2731212215 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f3838060f20 0x7f38380633e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.462+0000 7f3855626700 1 --2- 172.21.15.38:0/2731212215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38500f5390 0x7f3850108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.462+0000 7f3855626700 1 --2- 172.21.15.38:0/2731212215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38500f6630 0x7f3850109400 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.462+0000 7f3855626700 1 -- 172.21.15.38:0/2731212215 >> 172.21.15.38:0/2731212215 conn(0x7f38500f0b10 msgr2=0x7f385005fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.462+0000 7f3855626700 1 -- 172.21.15.38:0/2731212215 shutdown_connections 2024-02-15T18:44:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.462+0000 7f3855626700 1 -- 172.21.15.38:0/2731212215 wait complete. 2024-02-15T18:44:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-02-15T18:44:21.463 DEBUG:tasks.fs:max_mds reduced in epoch 26 2024-02-15T18:44:21.463 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-02-15T18:44:21.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.595+0000 7f9f32894700 1 -- 172.21.15.38:0/2159413532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f2c0f64b0 msgr2=0x7f9f2c0f68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:21.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.595+0000 7f9f32894700 1 --2- 172.21.15.38:0/2159413532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f2c0f64b0 0x7f9f2c0f68d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f9f1c004660 tx=0x7f9f1c009b30).stop 2024-02-15T18:44:21.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.595+0000 7f9f32894700 1 -- 172.21.15.38:0/2159413532 shutdown_connections 2024-02-15T18:44:21.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.595+0000 7f9f32894700 1 --2- 172.21.15.38:0/2159413532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f2c0f7750 0x7f9f2c0f7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:21.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.595+0000 7f9f32894700 1 --2- 172.21.15.38:0/2159413532 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f2c0f64b0 0x7f9f2c0f68d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:21.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.595+0000 7f9f32894700 1 -- 172.21.15.38:0/2159413532 >> 172.21.15.38:0/2159413532 conn(0x7f9f2c0f1bd0 msgr2=0x7f9f2c0f4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:21.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.595+0000 7f9f32894700 1 -- 172.21.15.38:0/2159413532 shutdown_connections 2024-02-15T18:44:21.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.595+0000 7f9f32894700 1 -- 172.21.15.38:0/2159413532 wait complete. 2024-02-15T18:44:21.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.596+0000 7f9f32894700 1 Processor -- start 2024-02-15T18:44:21.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.596+0000 7f9f32894700 1 -- start start 2024-02-15T18:44:21.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.596+0000 7f9f32894700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f2c0f64b0 0x7f9f2c106d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:21.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.596+0000 7f9f32894700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f2c0f7750 0x7f9f2c107200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:21.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.596+0000 7f9f32894700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c107800 con 0x7f9f2c0f64b0 2024-02-15T18:44:21.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.596+0000 7f9f32894700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c107940 con 0x7f9f2c0f7750 2024-02-15T18:44:21.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.596+0000 7f9f2bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f2c0f64b0 0x7f9f2c106d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:21.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.596+0000 7f9f2bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f2c0f64b0 0x7f9f2c106d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59566/0 (socket says 172.21.15.38:59566) 2024-02-15T18:44:21.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.596+0000 7f9f2bfff700 1 -- 172.21.15.38:0/891964973 learned_addr learned my addr 172.21.15.38:0/891964973 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:21.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.597+0000 7f9f2b7fe700 1 --2- 172.21.15.38:0/891964973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f2c0f7750 0x7f9f2c107200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:21.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.597+0000 7f9f2bfff700 1 -- 172.21.15.38:0/891964973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f2c0f7750 msgr2=0x7f9f2c107200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:21.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.597+0000 7f9f2bfff700 1 --2- 172.21.15.38:0/891964973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f2c0f7750 0x7f9f2c107200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:21.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.597+0000 7f9f2bfff700 1 -- 172.21.15.38:0/891964973 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1c005040 con 0x7f9f2c0f64b0 2024-02-15T18:44:21.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.597+0000 7f9f2bfff700 1 --2- 172.21.15.38:0/891964973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f2c0f64b0 0x7f9f2c106d10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f9f1c013fd0 tx=0x7f9f1c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:21.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.597+0000 7f9f28ff9700 1 -- 172.21.15.38:0/891964973 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f1c018070 con 0x7f9f2c0f64b0 2024-02-15T18:44:21.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.597+0000 7f9f28ff9700 1 -- 172.21.15.38:0/891964973 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f9f1c007d70 con 0x7f9f2c0f64b0 2024-02-15T18:44:21.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.597+0000 7f9f32894700 1 -- 172.21.15.38:0/891964973 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2c10c2f0 con 0x7f9f2c0f64b0 2024-02-15T18:44:21.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.597+0000 7f9f28ff9700 1 -- 172.21.15.38:0/891964973 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f1c01c620 con 0x7f9f2c0f64b0 2024-02-15T18:44:21.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.597+0000 7f9f32894700 1 -- 172.21.15.38:0/891964973 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2c10c850 con 0x7f9f2c0f64b0 2024-02-15T18:44:21.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.598+0000 7f9f32894700 1 -- 172.21.15.38:0/891964973 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2c046330 con 0x7f9f2c0f64b0 2024-02-15T18:44:21.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.598+0000 7f9f28ff9700 1 -- 172.21.15.38:0/891964973 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f9f1c01d420 con 0x7f9f2c0f64b0 2024-02-15T18:44:21.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.599+0000 7f9f28ff9700 1 --2- 172.21.15.38:0/891964973 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9f14060f20 0x7f9f140633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:21.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.599+0000 7f9f28ff9700 1 -- 172.21.15.38:0/891964973 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f9f1c07df90 con 0x7f9f2c0f64b0 2024-02-15T18:44:21.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.599+0000 7f9f2b7fe700 1 --2- 172.21.15.38:0/891964973 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9f14060f20 0x7f9f140633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:21.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.599+0000 7f9f2b7fe700 1 --2- 172.21.15.38:0/891964973 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9f14060f20 0x7f9f140633e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f9f200041d0 tx=0x7f9f20009040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:21.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.603+0000 7f9f28ff9700 1 -- 172.21.15.38:0/891964973 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9f1c04e520 con 0x7f9f2c0f64b0 2024-02-15T18:44:21.770 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:20.740629+0000 mon.smithi038 (mon.0) 754 : audit [DBG] from='client.? 172.21.15.38:0/4116223456' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-02-15T18:44:21.770 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:21 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:21.101141+0000 mon.smithi038 (mon.0) 755 : audit [DBG] from='client.? 172.21.15.38:0/1915847708' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-02-15T18:44:21.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.820+0000 7f9f32894700 1 -- 172.21.15.38:0/891964973 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f9f2c0586e0 con 0x7f9f2c0f64b0 2024-02-15T18:44:21.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.821+0000 7f9f28ff9700 1 -- 172.21.15.38:0/891964973 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v32) v1 ==== 107+0+4582 (secure 0 0 0) 0x7f9f1c051b40 con 0x7f9f2c0f64b0 2024-02-15T18:44:21.810 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:21.810 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34318,"name":"cephfs.smithi038.qgkzgn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6829/1616551408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1616551408},{"type":"v1","addr":"172.21.15.38:6829","nonce":1616551408}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44223,"name":"cephfs.smithi121.ovuzki","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6827/554561289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":554561289},{"type":"v1","addr":"172.21.15.121:6827","nonce":554561289}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":44261,"name":"cephfs.smithi121.mvumlh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6825/180558011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":180558011},{"type":"v1","addr":"172.21.15.121:6825","nonce":180558011}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:41:14.081538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34314},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34314":{"gid":34314,"name":"cephfs.smithi038.hxoggl","rank":0,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.38:6827/4077527450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":4077527450},{"type":"v1","addr":"172.21.15.38:6827","nonce":4077527450}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:21.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.823+0000 7f9f32894700 1 -- 172.21.15.38:0/891964973 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9f14060f20 msgr2=0x7f9f140633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:21.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.823+0000 7f9f32894700 1 --2- 172.21.15.38:0/891964973 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9f14060f20 0x7f9f140633e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f9f200041d0 tx=0x7f9f20009040).stop 2024-02-15T18:44:21.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.823+0000 7f9f32894700 1 -- 172.21.15.38:0/891964973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f2c0f64b0 msgr2=0x7f9f2c106d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:21.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.823+0000 7f9f32894700 1 --2- 172.21.15.38:0/891964973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f2c0f64b0 0x7f9f2c106d10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f9f1c013fd0 tx=0x7f9f1c009b30).stop 2024-02-15T18:44:21.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.823+0000 7f9f32894700 1 -- 172.21.15.38:0/891964973 shutdown_connections 2024-02-15T18:44:21.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.823+0000 7f9f32894700 1 --2- 172.21.15.38:0/891964973 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f9f14060f20 0x7f9f140633e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:21.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.823+0000 7f9f32894700 1 --2- 172.21.15.38:0/891964973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f2c0f64b0 0x7f9f2c106d10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:21.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.823+0000 7f9f32894700 1 --2- 172.21.15.38:0/891964973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f2c0f7750 0x7f9f2c107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:21.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.823+0000 7f9f32894700 1 -- 172.21.15.38:0/891964973 >> 172.21.15.38:0/891964973 conn(0x7f9f2c0f1bd0 msgr2=0x7f9f2c0fb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:21.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.823+0000 7f9f32894700 1 -- 172.21.15.38:0/891964973 shutdown_connections 2024-02-15T18:44:21.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.823+0000 7f9f32894700 1 -- 172.21.15.38:0/891964973 wait complete. 2024-02-15T18:44:21.813 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 27 2024-02-15T18:44:21.824 DEBUG:tasks.fs:max_mds reduced in epoch 27 2024-02-15T18:44:21.824 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-02-15T18:44:21.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:20.740629+0000 mon.smithi038 (mon.0) 754 : audit [DBG] from='client.? 172.21.15.38:0/4116223456' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-02-15T18:44:21.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:21 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:21.101141+0000 mon.smithi038 (mon.0) 755 : audit [DBG] from='client.? 172.21.15.38:0/1915847708' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-02-15T18:44:21.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.956+0000 7f10f3831700 1 -- 172.21.15.38:0/576080955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10ec0f5390 msgr2=0x7f10ec0f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:21.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.956+0000 7f10f3831700 1 --2- 172.21.15.38:0/576080955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10ec0f5390 0x7f10ec0f57b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f10e0004660 tx=0x7f10e0009b30).stop 2024-02-15T18:44:21.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.956+0000 7f10f3831700 1 -- 172.21.15.38:0/576080955 shutdown_connections 2024-02-15T18:44:21.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.956+0000 7f10f3831700 1 --2- 172.21.15.38:0/576080955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10ec0f5ca0 0x7f10ec0f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:21.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.956+0000 7f10f3831700 1 --2- 172.21.15.38:0/576080955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10ec0f5390 0x7f10ec0f57b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:21.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.956+0000 7f10f3831700 1 -- 172.21.15.38:0/576080955 >> 172.21.15.38:0/576080955 conn(0x7f10ec0f0b70 msgr2=0x7f10ec0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:21.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.956+0000 7f10f3831700 1 -- 172.21.15.38:0/576080955 shutdown_connections 2024-02-15T18:44:21.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.956+0000 7f10f3831700 1 -- 172.21.15.38:0/576080955 wait complete. 2024-02-15T18:44:21.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.957+0000 7f10f3831700 1 Processor -- start 2024-02-15T18:44:21.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.957+0000 7f10f3831700 1 -- start start 2024-02-15T18:44:21.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.957+0000 7f10f3831700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10ec0f5390 0x7f10ec05fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:21.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.957+0000 7f10f3831700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10ec0f5ca0 0x7f10ec10bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:21.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.957+0000 7f10f3831700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec10bfb0 con 0x7f10ec0f5390 2024-02-15T18:44:21.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.957+0000 7f10f3831700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec10c0f0 con 0x7f10ec0f5ca0 2024-02-15T18:44:21.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.957+0000 7f10f15cd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10ec0f5390 0x7f10ec05fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:21.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.957+0000 7f10f15cd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10ec0f5390 0x7f10ec05fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59586/0 (socket says 172.21.15.38:59586) 2024-02-15T18:44:21.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.957+0000 7f10f15cd700 1 -- 172.21.15.38:0/170865625 learned_addr learned my addr 172.21.15.38:0/170865625 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:21.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.957+0000 7f10f0dcc700 1 --2- 172.21.15.38:0/170865625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10ec0f5ca0 0x7f10ec10bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:21.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.958+0000 7f10f15cd700 1 -- 172.21.15.38:0/170865625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10ec0f5ca0 msgr2=0x7f10ec10bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:21.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.958+0000 7f10f15cd700 1 --2- 172.21.15.38:0/170865625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10ec0f5ca0 0x7f10ec10bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:21.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.958+0000 7f10f15cd700 1 -- 172.21.15.38:0/170865625 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e0005040 con 0x7f10ec0f5390 2024-02-15T18:44:21.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.958+0000 7f10f15cd700 1 --2- 172.21.15.38:0/170865625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10ec0f5390 0x7f10ec05fc90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f10e0009f50 tx=0x7f10e0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:21.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.958+0000 7f10ddffb700 1 -- 172.21.15.38:0/170865625 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10e0018070 con 0x7f10ec0f5390 2024-02-15T18:44:21.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.958+0000 7f10f3831700 1 -- 172.21.15.38:0/170865625 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10ec10c2f0 con 0x7f10ec0f5390 2024-02-15T18:44:21.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.958+0000 7f10ddffb700 1 -- 172.21.15.38:0/170865625 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f10e0007ba0 con 0x7f10ec0f5390 2024-02-15T18:44:21.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.958+0000 7f10ddffb700 1 -- 172.21.15.38:0/170865625 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10e001c420 con 0x7f10ec0f5390 2024-02-15T18:44:21.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.958+0000 7f10f3831700 1 -- 172.21.15.38:0/170865625 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ec10c7a0 con 0x7f10ec0f5390 2024-02-15T18:44:21.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.959+0000 7f10f3831700 1 -- 172.21.15.38:0/170865625 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10ec046330 con 0x7f10ec0f5390 2024-02-15T18:44:21.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.960+0000 7f10ddffb700 1 -- 172.21.15.38:0/170865625 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f10e001d420 con 0x7f10ec0f5390 2024-02-15T18:44:21.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.960+0000 7f10ddffb700 1 --2- 172.21.15.38:0/170865625 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f10d8060f20 0x7f10d80633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:21.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.960+0000 7f10ddffb700 1 -- 172.21.15.38:0/170865625 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f10e007ddf0 con 0x7f10ec0f5390 2024-02-15T18:44:21.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.960+0000 7f10f0dcc700 1 --2- 172.21.15.38:0/170865625 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f10d8060f20 0x7f10d80633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:21.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.960+0000 7f10f0dcc700 1 --2- 172.21.15.38:0/170865625 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f10d8060f20 0x7f10d80633e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f10e8004ce0 tx=0x7f10e800b040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:21.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:21.964+0000 7f10ddffb700 1 -- 172.21.15.38:0/170865625 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f10e0051bf0 con 0x7f10ec0f5390 2024-02-15T18:44:22.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.179+0000 7f10f3831700 1 -- 172.21.15.38:0/170865625 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f10ec0586e0 con 0x7f10ec0f5390 2024-02-15T18:44:22.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.179+0000 7f10ddffb700 1 -- 172.21.15.38:0/170865625 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v32) v1 ==== 107+0+3803 (secure 0 0 0) 0x7f10e00517e0 con 0x7f10ec0f5390 2024-02-15T18:44:22.168 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:22.169 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34318,"name":"cephfs.smithi038.qgkzgn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6829/1616551408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1616551408},{"type":"v1","addr":"172.21.15.38:6829","nonce":1616551408}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44261,"name":"cephfs.smithi121.mvumlh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6825/180558011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":180558011},{"type":"v1","addr":"172.21.15.121:6825","nonce":180558011}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:41:14.081538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34314},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34314":{"gid":34314,"name":"cephfs.smithi038.hxoggl","rank":0,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.38:6827/4077527450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":4077527450},{"type":"v1","addr":"172.21.15.38:6827","nonce":4077527450}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:22.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.181+0000 7f10f3831700 1 -- 172.21.15.38:0/170865625 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f10d8060f20 msgr2=0x7f10d80633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:22.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.181+0000 7f10f3831700 1 --2- 172.21.15.38:0/170865625 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f10d8060f20 0x7f10d80633e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f10e8004ce0 tx=0x7f10e800b040).stop 2024-02-15T18:44:22.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.181+0000 7f10f3831700 1 -- 172.21.15.38:0/170865625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10ec0f5390 msgr2=0x7f10ec05fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:22.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.181+0000 7f10f3831700 1 --2- 172.21.15.38:0/170865625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10ec0f5390 0x7f10ec05fc90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f10e0009f50 tx=0x7f10e0009b30).stop 2024-02-15T18:44:22.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.181+0000 7f10f3831700 1 -- 172.21.15.38:0/170865625 shutdown_connections 2024-02-15T18:44:22.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.181+0000 7f10f3831700 1 --2- 172.21.15.38:0/170865625 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7f10d8060f20 0x7f10d80633e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:22.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.181+0000 7f10f3831700 1 --2- 172.21.15.38:0/170865625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10ec0f5390 0x7f10ec05fc90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:22.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.181+0000 7f10f3831700 1 --2- 172.21.15.38:0/170865625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10ec0f5ca0 0x7f10ec10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:22.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.181+0000 7f10f3831700 1 -- 172.21.15.38:0/170865625 >> 172.21.15.38:0/170865625 conn(0x7f10ec0f0b70 msgr2=0x7f10ec0fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:22.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.181+0000 7f10f3831700 1 -- 172.21.15.38:0/170865625 shutdown_connections 2024-02-15T18:44:22.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.181+0000 7f10f3831700 1 -- 172.21.15.38:0/170865625 wait complete. 2024-02-15T18:44:22.172 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-02-15T18:44:22.182 DEBUG:tasks.fs:max_mds reduced in epoch 28 2024-02-15T18:44:22.182 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-02-15T18:44:22.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.314+0000 7fc7b3b74700 1 -- 172.21.15.38:0/2193009295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7ac0f5390 msgr2=0x7fc7ac0f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:22.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.314+0000 7fc7b3b74700 1 --2- 172.21.15.38:0/2193009295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7ac0f5390 0x7fc7ac0f57b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fc7a8004660 tx=0x7fc7a8009b30).stop 2024-02-15T18:44:22.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.314+0000 7fc7b3b74700 1 -- 172.21.15.38:0/2193009295 shutdown_connections 2024-02-15T18:44:22.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.314+0000 7fc7b3b74700 1 --2- 172.21.15.38:0/2193009295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7ac0f5ca0 0x7fc7ac0f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:22.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.314+0000 7fc7b3b74700 1 --2- 172.21.15.38:0/2193009295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7ac0f5390 0x7fc7ac0f57b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:22.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.314+0000 7fc7b3b74700 1 -- 172.21.15.38:0/2193009295 >> 172.21.15.38:0/2193009295 conn(0x7fc7ac0f0b70 msgr2=0x7fc7ac0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:22.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.314+0000 7fc7b3b74700 1 -- 172.21.15.38:0/2193009295 shutdown_connections 2024-02-15T18:44:22.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.314+0000 7fc7b3b74700 1 -- 172.21.15.38:0/2193009295 wait complete. 2024-02-15T18:44:22.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.315+0000 7fc7b3b74700 1 Processor -- start 2024-02-15T18:44:22.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.315+0000 7fc7b3b74700 1 -- start start 2024-02-15T18:44:22.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.315+0000 7fc7b3b74700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7ac0f5390 0x7fc7ac105bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:22.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.315+0000 7fc7b3b74700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7ac0f5ca0 0x7fc7ac1060a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:22.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.315+0000 7fc7b3b74700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac106650 con 0x7fc7ac0f5ca0 2024-02-15T18:44:22.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.315+0000 7fc7b3b74700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac106790 con 0x7fc7ac0f5390 2024-02-15T18:44:22.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.315+0000 7fc7b110f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7ac0f5ca0 0x7fc7ac1060a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:22.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.315+0000 7fc7b110f700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7ac0f5ca0 0x7fc7ac1060a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59600/0 (socket says 172.21.15.38:59600) 2024-02-15T18:44:22.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.315+0000 7fc7b110f700 1 -- 172.21.15.38:0/3036471069 learned_addr learned my addr 172.21.15.38:0/3036471069 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:22.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.315+0000 7fc7b1910700 1 --2- 172.21.15.38:0/3036471069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7ac0f5390 0x7fc7ac105bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:22.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.316+0000 7fc7b110f700 1 -- 172.21.15.38:0/3036471069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7ac0f5390 msgr2=0x7fc7ac105bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:22.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.316+0000 7fc7b110f700 1 --2- 172.21.15.38:0/3036471069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7ac0f5390 0x7fc7ac105bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:22.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.316+0000 7fc7b110f700 1 -- 172.21.15.38:0/3036471069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a8005040 con 0x7fc7ac0f5ca0 2024-02-15T18:44:22.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.316+0000 7fc7b110f700 1 --2- 172.21.15.38:0/3036471069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7ac0f5ca0 0x7fc7ac1060a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fc79c0026d0 tx=0x7fc79c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:22.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.316+0000 7fc7a27fc700 1 -- 172.21.15.38:0/3036471069 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc79c0079c0 con 0x7fc7ac0f5ca0 2024-02-15T18:44:22.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.316+0000 7fc7a27fc700 1 -- 172.21.15.38:0/3036471069 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fc79c00f040 con 0x7fc7ac0f5ca0 2024-02-15T18:44:22.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.316+0000 7fc7b3b74700 1 -- 172.21.15.38:0/3036471069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7ac10b230 con 0x7fc7ac0f5ca0 2024-02-15T18:44:22.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.316+0000 7fc7a27fc700 1 -- 172.21.15.38:0/3036471069 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc79c0149f0 con 0x7fc7ac0f5ca0 2024-02-15T18:44:22.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.316+0000 7fc7b3b74700 1 -- 172.21.15.38:0/3036471069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7ac10b7f0 con 0x7fc7ac0f5ca0 2024-02-15T18:44:22.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.317+0000 7fc7b3b74700 1 -- 172.21.15.38:0/3036471069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7ac027a00 con 0x7fc7ac0f5ca0 2024-02-15T18:44:22.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.317+0000 7fc7a27fc700 1 -- 172.21.15.38:0/3036471069 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fc79c007b20 con 0x7fc7ac0f5ca0 2024-02-15T18:44:22.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.318+0000 7fc7a27fc700 1 --2- 172.21.15.38:0/3036471069 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc798060f20 0x7fc7980633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:22.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.318+0000 7fc7a27fc700 1 -- 172.21.15.38:0/3036471069 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fc79c07c120 con 0x7fc7ac0f5ca0 2024-02-15T18:44:22.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.318+0000 7fc7b1910700 1 --2- 172.21.15.38:0/3036471069 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc798060f20 0x7fc7980633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:22.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.321+0000 7fc7b1910700 1 --2- 172.21.15.38:0/3036471069 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc798060f20 0x7fc7980633e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fc7a8004630 tx=0x7fc7a8009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:22.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.323+0000 7fc7a27fc700 1 -- 172.21.15.38:0/3036471069 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc79c04ff80 con 0x7fc7ac0f5ca0 2024-02-15T18:44:22.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: cluster 2024-02-15T18:44:21.045001+0000 mgr.smithi038.khesbl (mgr.14664) 416 : cluster [DBG] pgmap v287: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:22.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:21.461728+0000 mon.smithi038 (mon.0) 756 : audit [DBG] from='client.? 172.21.15.38:0/2731212215' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-02-15T18:44:22.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:21.822466+0000 mon.smithi038 (mon.0) 757 : audit [DBG] from='client.? 172.21.15.38:0/891964973' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-02-15T18:44:22.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:22 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:22.181123+0000 mon.smithi038 (mon.0) 758 : audit [DBG] from='client.? 172.21.15.38:0/170865625' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-02-15T18:44:22.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.538+0000 7fc7b3b74700 1 -- 172.21.15.38:0/3036471069 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7fc7ac0fdc90 con 0x7fc7ac0f5ca0 2024-02-15T18:44:22.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.539+0000 7fc7a27fc700 1 -- 172.21.15.38:0/3036471069 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v32) v1 ==== 107+0+4585 (secure 0 0 0) 0x7fc79c04fb70 con 0x7fc7ac0f5ca0 2024-02-15T18:44:22.528 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:22.528 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34318,"name":"cephfs.smithi038.qgkzgn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6829/1616551408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1616551408},{"type":"v1","addr":"172.21.15.38:6829","nonce":1616551408}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44261,"name":"cephfs.smithi121.mvumlh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6825/180558011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":180558011},{"type":"v1","addr":"172.21.15.121:6825","nonce":180558011}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44265,"name":"cephfs.smithi121.ovuzki","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6827/1075229339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1075229339},{"type":"v1","addr":"172.21.15.121:6827","nonce":1075229339}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:41:14.081538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34314},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34314":{"gid":34314,"name":"cephfs.smithi038.hxoggl","rank":0,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.38:6827/4077527450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":4077527450},{"type":"v1","addr":"172.21.15.38:6827","nonce":4077527450}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:22.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.541+0000 7fc7b3b74700 1 -- 172.21.15.38:0/3036471069 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc798060f20 msgr2=0x7fc7980633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:22.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.541+0000 7fc7b3b74700 1 --2- 172.21.15.38:0/3036471069 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc798060f20 0x7fc7980633e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fc7a8004630 tx=0x7fc7a8009b30).stop 2024-02-15T18:44:22.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.541+0000 7fc7b3b74700 1 -- 172.21.15.38:0/3036471069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7ac0f5ca0 msgr2=0x7fc7ac1060a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:22.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.541+0000 7fc7b3b74700 1 --2- 172.21.15.38:0/3036471069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7ac0f5ca0 0x7fc7ac1060a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fc79c0026d0 tx=0x7fc79c009de0).stop 2024-02-15T18:44:22.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.541+0000 7fc7b3b74700 1 -- 172.21.15.38:0/3036471069 shutdown_connections 2024-02-15T18:44:22.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.541+0000 7fc7b3b74700 1 --2- 172.21.15.38:0/3036471069 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc798060f20 0x7fc7980633e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:22.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.541+0000 7fc7b3b74700 1 --2- 172.21.15.38:0/3036471069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7ac0f5390 0x7fc7ac105bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:22.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.541+0000 7fc7b3b74700 1 --2- 172.21.15.38:0/3036471069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7ac0f5ca0 0x7fc7ac1060a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:22.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.541+0000 7fc7b3b74700 1 -- 172.21.15.38:0/3036471069 >> 172.21.15.38:0/3036471069 conn(0x7fc7ac0f0b70 msgr2=0x7fc7ac0fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:22.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.541+0000 7fc7b3b74700 1 -- 172.21.15.38:0/3036471069 shutdown_connections 2024-02-15T18:44:22.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.541+0000 7fc7b3b74700 1 -- 172.21.15.38:0/3036471069 wait complete. 2024-02-15T18:44:22.531 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 29 2024-02-15T18:44:22.542 DEBUG:tasks.fs:max_mds reduced in epoch 29 2024-02-15T18:44:22.542 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-02-15T18:44:22.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.675+0000 7fd0fd891700 1 -- 172.21.15.38:0/613130210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0f80f96b0 msgr2=0x7fd0f80fbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:22.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.675+0000 7fd0fd891700 1 --2- 172.21.15.38:0/613130210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0f80f96b0 0x7fd0f80fbaa0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fd0e0004660 tx=0x7fd0e0009b30).stop 2024-02-15T18:44:22.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.675+0000 7fd0fd891700 1 -- 172.21.15.38:0/613130210 shutdown_connections 2024-02-15T18:44:22.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.675+0000 7fd0fd891700 1 --2- 172.21.15.38:0/613130210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0f80f96b0 0x7fd0f80fbaa0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:22.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.675+0000 7fd0fd891700 1 --2- 172.21.15.38:0/613130210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0f80602a0 0x7fd0f80f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:22.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.675+0000 7fd0fd891700 1 -- 172.21.15.38:0/613130210 >> 172.21.15.38:0/613130210 conn(0x7fd0f80f0b30 msgr2=0x7fd0f80f2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:22.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.675+0000 7fd0fd891700 1 -- 172.21.15.38:0/613130210 shutdown_connections 2024-02-15T18:44:22.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.675+0000 7fd0fd891700 1 -- 172.21.15.38:0/613130210 wait complete. 2024-02-15T18:44:22.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.676+0000 7fd0fd891700 1 Processor -- start 2024-02-15T18:44:22.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.676+0000 7fd0fd891700 1 -- start start 2024-02-15T18:44:22.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.676+0000 7fd0fd891700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0f80602a0 0x7fd0f8106d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:22.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.676+0000 7fd0fd891700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0f80f96b0 0x7fd0f8107200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:22.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.676+0000 7fd0fd891700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f8107800 con 0x7fd0f80602a0 2024-02-15T18:44:22.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.676+0000 7fd0fd891700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f8107940 con 0x7fd0f80f96b0 2024-02-15T18:44:22.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.676+0000 7fd0f6ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0f80602a0 0x7fd0f8106d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:22.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.676+0000 7fd0f6ffd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0f80602a0 0x7fd0f8106d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59624/0 (socket says 172.21.15.38:59624) 2024-02-15T18:44:22.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.676+0000 7fd0f6ffd700 1 -- 172.21.15.38:0/2258485358 learned_addr learned my addr 172.21.15.38:0/2258485358 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:22.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.676+0000 7fd0f67fc700 1 --2- 172.21.15.38:0/2258485358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0f80f96b0 0x7fd0f8107200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:22.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.676+0000 7fd0f6ffd700 1 -- 172.21.15.38:0/2258485358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0f80f96b0 msgr2=0x7fd0f8107200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:22.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.676+0000 7fd0f6ffd700 1 --2- 172.21.15.38:0/2258485358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0f80f96b0 0x7fd0f8107200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:22.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.676+0000 7fd0f6ffd700 1 -- 172.21.15.38:0/2258485358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e0005040 con 0x7fd0f80602a0 2024-02-15T18:44:22.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.676+0000 7fd0f6ffd700 1 --2- 172.21.15.38:0/2258485358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0f80602a0 0x7fd0f8106d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fd0e800e7f0 tx=0x7fd0e800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:22.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.676+0000 7fd0ef7fe700 1 -- 172.21.15.38:0/2258485358 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0e80092e0 con 0x7fd0f80602a0 2024-02-15T18:44:22.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.677+0000 7fd0ef7fe700 1 -- 172.21.15.38:0/2258485358 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fd0e800f040 con 0x7fd0f80602a0 2024-02-15T18:44:22.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.677+0000 7fd0fd891700 1 -- 172.21.15.38:0/2258485358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f810c350 con 0x7fd0f80602a0 2024-02-15T18:44:22.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.677+0000 7fd0ef7fe700 1 -- 172.21.15.38:0/2258485358 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0e8014850 con 0x7fd0f80602a0 2024-02-15T18:44:22.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.677+0000 7fd0fd891700 1 -- 172.21.15.38:0/2258485358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f810c910 con 0x7fd0f80602a0 2024-02-15T18:44:22.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.678+0000 7fd0fd891700 1 -- 172.21.15.38:0/2258485358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f8027a00 con 0x7fd0f80602a0 2024-02-15T18:44:22.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.678+0000 7fd0ef7fe700 1 -- 172.21.15.38:0/2258485358 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fd0e801b030 con 0x7fd0f80602a0 2024-02-15T18:44:22.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.678+0000 7fd0ef7fe700 1 --2- 172.21.15.38:0/2258485358 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd0e4060f20 0x7fd0e40633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:22.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.679+0000 7fd0ef7fe700 1 -- 172.21.15.38:0/2258485358 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fd0e807be20 con 0x7fd0f80602a0 2024-02-15T18:44:22.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.679+0000 7fd0f67fc700 1 --2- 172.21.15.38:0/2258485358 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd0e4060f20 0x7fd0e40633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:22.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.679+0000 7fd0f67fc700 1 --2- 172.21.15.38:0/2258485358 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd0e4060f20 0x7fd0e40633e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fd0e0009720 tx=0x7fd0e0016040).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:22.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.683+0000 7fd0ef7fe700 1 -- 172.21.15.38:0/2258485358 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd0e804c300 con 0x7fd0f80602a0 2024-02-15T18:44:22.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: cluster 2024-02-15T18:44:21.045001+0000 mgr.smithi038.khesbl (mgr.14664) 416 : cluster [DBG] pgmap v287: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:44:22.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:21.461728+0000 mon.smithi038 (mon.0) 756 : audit [DBG] from='client.? 172.21.15.38:0/2731212215' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-02-15T18:44:22.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:21.822466+0000 mon.smithi038 (mon.0) 757 : audit [DBG] from='client.? 172.21.15.38:0/891964973' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-02-15T18:44:22.855 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:22 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:22.181123+0000 mon.smithi038 (mon.0) 758 : audit [DBG] from='client.? 172.21.15.38:0/170865625' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-02-15T18:44:22.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.897+0000 7fd0fd891700 1 -- 172.21.15.38:0/2258485358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fd0f80feda0 con 0x7fd0f80602a0 2024-02-15T18:44:22.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.898+0000 7fd0ef7fe700 1 -- 172.21.15.38:0/2258485358 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v32) v1 ==== 107+0+4585 (secure 0 0 0) 0x7fd0e8019020 con 0x7fd0f80602a0 2024-02-15T18:44:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:22.886 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34318,"name":"cephfs.smithi038.qgkzgn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6829/1616551408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1616551408},{"type":"v1","addr":"172.21.15.38:6829","nonce":1616551408}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44261,"name":"cephfs.smithi121.mvumlh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6825/180558011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":180558011},{"type":"v1","addr":"172.21.15.121:6825","nonce":180558011}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44265,"name":"cephfs.smithi121.ovuzki","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6827/1075229339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1075229339},{"type":"v1","addr":"172.21.15.121:6827","nonce":1075229339}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:41:25.383284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0],"up":{"mds_0":34314},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34314":{"gid":34314,"name":"cephfs.smithi038.hxoggl","rank":0,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.38:6827/4077527450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":4077527450},{"type":"v1","addr":"172.21.15.38:6827","nonce":4077527450}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:22.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.899+0000 7fd0fd891700 1 -- 172.21.15.38:0/2258485358 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd0e4060f20 msgr2=0x7fd0e40633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:22.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.899+0000 7fd0fd891700 1 --2- 172.21.15.38:0/2258485358 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd0e4060f20 0x7fd0e40633e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fd0e0009720 tx=0x7fd0e0016040).stop 2024-02-15T18:44:22.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.899+0000 7fd0fd891700 1 -- 172.21.15.38:0/2258485358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0f80602a0 msgr2=0x7fd0f8106d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:22.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.899+0000 7fd0fd891700 1 --2- 172.21.15.38:0/2258485358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0f80602a0 0x7fd0f8106d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fd0e800e7f0 tx=0x7fd0e800b040).stop 2024-02-15T18:44:22.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.899+0000 7fd0fd891700 1 -- 172.21.15.38:0/2258485358 shutdown_connections 2024-02-15T18:44:22.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.899+0000 7fd0fd891700 1 --2- 172.21.15.38:0/2258485358 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fd0e4060f20 0x7fd0e40633e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:22.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.899+0000 7fd0fd891700 1 --2- 172.21.15.38:0/2258485358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0f80602a0 0x7fd0f8106d10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:22.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.899+0000 7fd0fd891700 1 --2- 172.21.15.38:0/2258485358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0f80f96b0 0x7fd0f8107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:22.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.899+0000 7fd0fd891700 1 -- 172.21.15.38:0/2258485358 >> 172.21.15.38:0/2258485358 conn(0x7fd0f80f0b30 msgr2=0x7fd0f80f2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:22.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.899+0000 7fd0fd891700 1 -- 172.21.15.38:0/2258485358 shutdown_connections 2024-02-15T18:44:22.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:22.900+0000 7fd0fd891700 1 -- 172.21.15.38:0/2258485358 wait complete. 2024-02-15T18:44:22.890 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 30 2024-02-15T18:44:22.901 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-02-15T18:44:23.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.033+0000 7fc1b5051700 1 -- 172.21.15.38:0/609802487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1b00f5ca0 msgr2=0x7fc1b00f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:23.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.033+0000 7fc1b5051700 1 --2- 172.21.15.38:0/609802487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1b00f5ca0 0x7fc1b00f66d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fc1a0004660 tx=0x7fc1a0009b30).stop 2024-02-15T18:44:23.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.033+0000 7fc1b5051700 1 -- 172.21.15.38:0/609802487 shutdown_connections 2024-02-15T18:44:23.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.033+0000 7fc1b5051700 1 --2- 172.21.15.38:0/609802487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1b00f5ca0 0x7fc1b00f66d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:23.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.033+0000 7fc1b5051700 1 --2- 172.21.15.38:0/609802487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1b00f5390 0x7fc1b00f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:23.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.033+0000 7fc1b5051700 1 -- 172.21.15.38:0/609802487 >> 172.21.15.38:0/609802487 conn(0x7fc1b00f0b70 msgr2=0x7fc1b00f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:23.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.033+0000 7fc1b5051700 1 -- 172.21.15.38:0/609802487 shutdown_connections 2024-02-15T18:44:23.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.033+0000 7fc1b5051700 1 -- 172.21.15.38:0/609802487 wait complete. 2024-02-15T18:44:23.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.033+0000 7fc1b5051700 1 Processor -- start 2024-02-15T18:44:23.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.033+0000 7fc1b5051700 1 -- start start 2024-02-15T18:44:23.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.033+0000 7fc1b5051700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1b00f5390 0x7fc1b0105c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:23.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.033+0000 7fc1b5051700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1b00f5ca0 0x7fc1b01060f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:23.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.033+0000 7fc1b5051700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b01066f0 con 0x7fc1b00f5390 2024-02-15T18:44:23.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.034+0000 7fc1b5051700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b0106830 con 0x7fc1b00f5ca0 2024-02-15T18:44:23.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.034+0000 7fc1aed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1b00f5390 0x7fc1b0105c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:23.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.034+0000 7fc1aed9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1b00f5390 0x7fc1b0105c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59636/0 (socket says 172.21.15.38:59636) 2024-02-15T18:44:23.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.034+0000 7fc1aed9d700 1 -- 172.21.15.38:0/4100324192 learned_addr learned my addr 172.21.15.38:0/4100324192 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-02-15T18:44:23.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.034+0000 7fc1ae59c700 1 --2- 172.21.15.38:0/4100324192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1b00f5ca0 0x7fc1b01060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:23.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.034+0000 7fc1aed9d700 1 -- 172.21.15.38:0/4100324192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1b00f5ca0 msgr2=0x7fc1b01060f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:23.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.034+0000 7fc1aed9d700 1 --2- 172.21.15.38:0/4100324192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1b00f5ca0 0x7fc1b01060f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:23.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.034+0000 7fc1aed9d700 1 -- 172.21.15.38:0/4100324192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a0005040 con 0x7fc1b00f5390 2024-02-15T18:44:23.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.034+0000 7fc1aed9d700 1 --2- 172.21.15.38:0/4100324192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1b00f5390 0x7fc1b0105c00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fc1a40026d0 tx=0x7fc1a4009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:23.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.034+0000 7fc1977fe700 1 -- 172.21.15.38:0/4100324192 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1a40079c0 con 0x7fc1b00f5390 2024-02-15T18:44:23.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.034+0000 7fc1977fe700 1 -- 172.21.15.38:0/4100324192 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fc1a400f040 con 0x7fc1b00f5390 2024-02-15T18:44:23.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.034+0000 7fc1b5051700 1 -- 172.21.15.38:0/4100324192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b010b240 con 0x7fc1b00f5390 2024-02-15T18:44:23.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.034+0000 7fc1977fe700 1 -- 172.21.15.38:0/4100324192 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1a40149f0 con 0x7fc1b00f5390 2024-02-15T18:44:23.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.034+0000 7fc1b5051700 1 -- 172.21.15.38:0/4100324192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b010b800 con 0x7fc1b00f5390 2024-02-15T18:44:23.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.035+0000 7fc1b5051700 1 -- 172.21.15.38:0/4100324192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b0027a00 con 0x7fc1b00f5390 2024-02-15T18:44:23.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.036+0000 7fc1977fe700 1 -- 172.21.15.38:0/4100324192 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fc1a4014b50 con 0x7fc1b00f5390 2024-02-15T18:44:23.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.036+0000 7fc1977fe700 1 --2- 172.21.15.38:0/4100324192 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc198060f20 0x7fc1980633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:44:23.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.036+0000 7fc1977fe700 1 -- 172.21.15.38:0/4100324192 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fc1a407c110 con 0x7fc1b00f5390 2024-02-15T18:44:23.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.036+0000 7fc1ae59c700 1 --2- 172.21.15.38:0/4100324192 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc198060f20 0x7fc1980633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:44:23.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.036+0000 7fc1ae59c700 1 --2- 172.21.15.38:0/4100324192 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc198060f20 0x7fc1980633e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fc1a0004630 tx=0x7fc1a0009b30).ready entity=mgr.14664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:44:23.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.040+0000 7fc1977fe700 1 -- 172.21.15.38:0/4100324192 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc1a404c540 con 0x7fc1b00f5390 2024-02-15T18:44:23.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.254+0000 7fc1b5051700 1 -- 172.21.15.38:0/4100324192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7fc1b00fdc90 con 0x7fc1b00f5390 2024-02-15T18:44:23.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.255+0000 7fc1977fe700 1 -- 172.21.15.38:0/4100324192 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v32) v1 ==== 107+0+4602 (secure 0 0 0) 0x7fc1a404fb60 con 0x7fc1b00f5390 2024-02-15T18:44:23.244 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-02-15T18:44:23.244 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44261,"name":"cephfs.smithi121.mvumlh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6825/180558011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":180558011},{"type":"v1","addr":"172.21.15.121:6825","nonce":180558011}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44265,"name":"cephfs.smithi121.ovuzki","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6827/1075229339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1075229339},{"type":"v1","addr":"172.21.15.121:6827","nonce":1075229339}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-15T18:32:46.871444+0000","modified":"2024-02-15T18:41:25.389045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34314,"mds_1":34318},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34314":{"gid":34314,"name":"cephfs.smithi038.hxoggl","rank":0,"incarnation":24,"state":"up:active","state_seq":11,"addr":"172.21.15.38:6827/4077527450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":4077527450},{"type":"v1","addr":"172.21.15.38:6827","nonce":4077527450}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34318":{"gid":34318,"name":"cephfs.smithi038.qgkzgn","rank":1,"incarnation":31,"state":"up:starting","state_seq":1,"addr":"172.21.15.38:6829/1616551408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":1616551408},{"type":"v1","addr":"172.21.15.38:6829","nonce":1616551408}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:44:23.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.257+0000 7fc1b5051700 1 -- 172.21.15.38:0/4100324192 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc198060f20 msgr2=0x7fc1980633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:23.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.257+0000 7fc1b5051700 1 --2- 172.21.15.38:0/4100324192 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc198060f20 0x7fc1980633e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fc1a0004630 tx=0x7fc1a0009b30).stop 2024-02-15T18:44:23.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.257+0000 7fc1b5051700 1 -- 172.21.15.38:0/4100324192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1b00f5390 msgr2=0x7fc1b0105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:44:23.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.257+0000 7fc1b5051700 1 --2- 172.21.15.38:0/4100324192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1b00f5390 0x7fc1b0105c00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fc1a40026d0 tx=0x7fc1a4009de0).stop 2024-02-15T18:44:23.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.257+0000 7fc1b5051700 1 -- 172.21.15.38:0/4100324192 shutdown_connections 2024-02-15T18:44:23.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.257+0000 7fc1b5051700 1 --2- 172.21.15.38:0/4100324192 >> [v2:172.21.15.38:6800/3254855982,v1:172.21.15.38:6801/3254855982] conn(0x7fc198060f20 0x7fc1980633e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:23.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.257+0000 7fc1b5051700 1 --2- 172.21.15.38:0/4100324192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1b00f5390 0x7fc1b0105c00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:23.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.257+0000 7fc1b5051700 1 --2- 172.21.15.38:0/4100324192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1b00f5ca0 0x7fc1b01060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:44:23.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.257+0000 7fc1b5051700 1 -- 172.21.15.38:0/4100324192 >> 172.21.15.38:0/4100324192 conn(0x7fc1b00f0b70 msgr2=0x7fc1b00fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:44:23.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.257+0000 7fc1b5051700 1 -- 172.21.15.38:0/4100324192 shutdown_connections 2024-02-15T18:44:23.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-02-15T18:44:23.257+0000 7fc1b5051700 1 -- 172.21.15.38:0/4100324192 wait complete. 2024-02-15T18:44:23.247 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 31 2024-02-15T18:44:23.258 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-02-15T18:44:23.270 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-15T18:44:23.270 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-15T18:44:23.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-15T18:44:23.271 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-15T18:44:23.317 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-02-15T18:44:23.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:44:23.317 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-15T18:44:23.359 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-15T18:44:23.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-15T18:44:23.359 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-02-15T18:44:23.411 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2024-02-15T18:44:23.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:44:23.411 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-15T18:44:23.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:44:23.454 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-02-15T18:44:23.473 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-15T18:44:23.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:44:23.474 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-15T18:44:23.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:22.540467+0000 mon.smithi038 (mon.0) 759 : audit [DBG] from='client.? 172.21.15.38:0/3036471069' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-02-15T18:44:23.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:22.899214+0000 mon.smithi038 (mon.0) 760 : audit [DBG] from='client.? 172.21.15.38:0/2258485358' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-02-15T18:44:23.520 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:23 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: audit 2024-02-15T18:44:23.256527+0000 mon.smithi038 (mon.0) 761 : audit [DBG] from='client.? 172.21.15.38:0/4100324192' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-02-15T18:44:23.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:44:23.569 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-02-15T18:44:23.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:44:23.671 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-02-15T18:44:23.692 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-15T18:44:23.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:44:23.693 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-15T18:44:23.761 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:22.540467+0000 mon.smithi038 (mon.0) 759 : audit [DBG] from='client.? 172.21.15.38:0/3036471069' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-02-15T18:44:23.762 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:22.899214+0000 mon.smithi038 (mon.0) 760 : audit [DBG] from='client.? 172.21.15.38:0/2258485358' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-02-15T18:44:23.762 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:23 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: audit 2024-02-15T18:44:23.256527+0000 mon.smithi038 (mon.0) 761 : audit [DBG] from='client.? 172.21.15.38:0/4100324192' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-02-15T18:44:23.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:44:23.790 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link delete ceph-brx 2024-02-15T18:44:23.888 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-15T18:44:23.899 INFO:tasks.cephadm:Teardown begin 2024-02-15T18:44:23.899 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-15T18:44:23.929 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-15T18:44:23.960 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-15T18:44:23.961 DEBUG:teuthology.orchestra.run.smithi038:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-15T18:44:23.988 DEBUG:teuthology.orchestra.run.smithi121:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-15T18:44:24.018 INFO:tasks.cephadm:Stopping all daemons... 2024-02-15T18:44:24.019 INFO:tasks.cephadm.mon.smithi038:Stopping mon.smithi038... 2024-02-15T18:44:24.019 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mon.smithi038 2024-02-15T18:44:24.270 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:24 smithi038 systemd[1]: Stopping Ceph mon.smithi038 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:44:24.270 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:24 smithi038 bash[99542]: Error: no container with name or ID "ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi038" found: no such container 2024-02-15T18:44:24.771 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:44:24.342+0000 7f3d763ce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi038 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-15T18:44:24.772 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:24 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038[77926]: debug 2024-02-15T18:44:24.342+0000 7f3d763ce700 -1 mon.smithi038@0(leader) e2 *** Got Signal Terminated *** 2024-02-15T18:44:25.270 INFO:journalctl@ceph.mon.smithi038.smithi038.stdout:Feb 15 18:44:24 smithi038 bash[99583]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi038 2024-02-15T18:44:25.479 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mon.smithi038.service' 2024-02-15T18:44:25.528 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-15T18:44:25.528 INFO:tasks.cephadm.mon.smithi038:Stopped mon.smithi038 2024-02-15T18:44:25.528 INFO:tasks.cephadm.mon.smithi121:Stopping mon.smithi121... 2024-02-15T18:44:25.528 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mon.smithi121 2024-02-15T18:44:25.819 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:25 smithi121 systemd[1]: Stopping Ceph mon.smithi121 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:44:25.819 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:25 smithi121 bash[70942]: Error: no container with name or ID "ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon.smithi121" found: no such container 2024-02-15T18:44:26.104 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:44:25.830+0000 7fca22f0b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi121 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-15T18:44:26.105 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:25 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121[59733]: debug 2024-02-15T18:44:25.830+0000 7fca22f0b700 -1 mon.smithi121@1(peon) e2 *** Got Signal Terminated *** 2024-02-15T18:44:26.854 INFO:journalctl@ceph.mon.smithi121.smithi121.stdout:Feb 15 18:44:26 smithi121 bash[70982]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-mon-smithi121 2024-02-15T18:44:27.031 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@mon.smithi121.service' 2024-02-15T18:44:27.109 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-15T18:44:27.109 INFO:tasks.cephadm.mon.smithi121:Stopped mon.smithi121 2024-02-15T18:44:27.109 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-15T18:44:27.109 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.0 2024-02-15T18:44:27.415 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:44:27 smithi038 systemd[1]: Stopping Ceph osd.0 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:44:27.415 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:44:27 smithi038 bash[99715]: Error: no container with name or ID "ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-15T18:44:27.770 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:44:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-0[81296]: debug 2024-02-15T18:44:27.426+0000 7f777f26b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:44:27.770 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:44:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-0[81296]: debug 2024-02-15T18:44:27.426+0000 7f777f26b700 -1 osd.0 80 *** Got signal Terminated *** 2024-02-15T18:44:27.770 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:44:27 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-0[81296]: debug 2024-02-15T18:44:27.426+0000 7f777f26b700 -1 osd.0 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:44:33.142 INFO:journalctl@ceph.osd.0.smithi038.stdout:Feb 15 18:44:32 smithi038 bash[99755]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-0 2024-02-15T18:44:34.839 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.0.service' 2024-02-15T18:44:34.885 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-15T18:44:34.886 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-15T18:44:34.886 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-15T18:44:34.886 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.1 2024-02-15T18:44:35.203 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:44:34 smithi038 systemd[1]: Stopping Ceph osd.1 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:44:35.203 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:44:34 smithi038 bash[99992]: Error: no container with name or ID "ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-15T18:44:35.520 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:44:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-1[83993]: debug 2024-02-15T18:44:35.213+0000 7fa0b65c8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:44:35.520 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:44:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-1[83993]: debug 2024-02-15T18:44:35.213+0000 7fa0b65c8700 -1 osd.1 80 *** Got signal Terminated *** 2024-02-15T18:44:35.520 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:44:35 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-1[83993]: debug 2024-02-15T18:44:35.213+0000 7fa0b65c8700 -1 osd.1 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:44:41.020 INFO:journalctl@ceph.osd.1.smithi038.stdout:Feb 15 18:44:40 smithi038 bash[100030]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-1 2024-02-15T18:44:42.757 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.1.service' 2024-02-15T18:44:42.803 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-15T18:44:42.803 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-15T18:44:42.803 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-15T18:44:42.803 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.2 2024-02-15T18:44:43.020 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:44:42 smithi038 systemd[1]: Stopping Ceph osd.2 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:44:43.021 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:44:42 smithi038 bash[100271]: Error: no container with name or ID "ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-15T18:44:43.520 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:44:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-2[86649]: debug 2024-02-15T18:44:43.158+0000 7f26449ae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:44:43.520 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:44:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-2[86649]: debug 2024-02-15T18:44:43.158+0000 7f26449ae700 -1 osd.2 80 *** Got signal Terminated *** 2024-02-15T18:44:43.520 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:44:43 smithi038 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-2[86649]: debug 2024-02-15T18:44:43.158+0000 7f26449ae700 -1 osd.2 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:44:48.932 INFO:journalctl@ceph.osd.2.smithi038.stdout:Feb 15 18:44:48 smithi038 bash[100312]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-2 2024-02-15T18:44:50.754 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.2.service' 2024-02-15T18:44:50.803 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-15T18:44:50.803 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-15T18:44:50.804 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-15T18:44:50.804 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.3 2024-02-15T18:44:51.104 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:44:50 smithi121 systemd[1]: Stopping Ceph osd.3 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:44:51.104 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:44:50 smithi121 bash[71110]: Error: no container with name or ID "ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-15T18:44:51.604 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:44:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-3[62228]: debug 2024-02-15T18:44:51.129+0000 7f5332b9f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:44:51.604 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:44:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-3[62228]: debug 2024-02-15T18:44:51.129+0000 7f5332b9f700 -1 osd.3 80 *** Got signal Terminated *** 2024-02-15T18:44:51.604 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:44:51 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-3[62228]: debug 2024-02-15T18:44:51.129+0000 7f5332b9f700 -1 osd.3 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:44:53.604 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:44:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-4[64474]: debug 2024-02-15T18:44:53.267+0000 7f9cc9bca700 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:31.120282+0000 front 2024-02-15T18:44:31.120433+0000 (oldest deadline 2024-02-15T18:44:52.820132+0000) 2024-02-15T18:44:53.605 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:44:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-3[62228]: debug 2024-02-15T18:44:53.349+0000 7f532fb1c700 -1 osd.3 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:26.672586+0000 front 2024-02-15T18:44:26.672577+0000 (oldest deadline 2024-02-15T18:44:52.572432+0000) 2024-02-15T18:44:53.605 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:44:53 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:44:53.365+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:29.743576+0000 front 2024-02-15T18:44:29.743513+0000 (oldest deadline 2024-02-15T18:44:52.643285+0000) 2024-02-15T18:44:54.604 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:44:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-4[64474]: debug 2024-02-15T18:44:54.218+0000 7f9cc9bca700 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:31.120282+0000 front 2024-02-15T18:44:31.120433+0000 (oldest deadline 2024-02-15T18:44:52.820132+0000) 2024-02-15T18:44:54.605 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:44:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-3[62228]: debug 2024-02-15T18:44:54.322+0000 7f532fb1c700 -1 osd.3 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:26.672586+0000 front 2024-02-15T18:44:26.672577+0000 (oldest deadline 2024-02-15T18:44:52.572432+0000) 2024-02-15T18:44:54.605 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:44:54 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:44:54.401+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:29.743576+0000 front 2024-02-15T18:44:29.743513+0000 (oldest deadline 2024-02-15T18:44:52.643285+0000) 2024-02-15T18:44:55.604 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:44:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-4[64474]: debug 2024-02-15T18:44:55.265+0000 7f9cc9bca700 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:31.120282+0000 front 2024-02-15T18:44:31.120433+0000 (oldest deadline 2024-02-15T18:44:52.820132+0000) 2024-02-15T18:44:55.604 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:44:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-3[62228]: debug 2024-02-15T18:44:55.287+0000 7f532fb1c700 -1 osd.3 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:26.672586+0000 front 2024-02-15T18:44:26.672577+0000 (oldest deadline 2024-02-15T18:44:52.572432+0000) 2024-02-15T18:44:55.605 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:44:55 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:44:55.388+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:29.743576+0000 front 2024-02-15T18:44:29.743513+0000 (oldest deadline 2024-02-15T18:44:52.643285+0000) 2024-02-15T18:44:56.604 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:44:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-4[64474]: debug 2024-02-15T18:44:56.261+0000 7f9cc9bca700 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:31.120282+0000 front 2024-02-15T18:44:31.120433+0000 (oldest deadline 2024-02-15T18:44:52.820132+0000) 2024-02-15T18:44:56.605 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:44:56 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:44:56.382+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:29.743576+0000 front 2024-02-15T18:44:29.743513+0000 (oldest deadline 2024-02-15T18:44:52.643285+0000) 2024-02-15T18:44:57.104 INFO:journalctl@ceph.osd.3.smithi121.stdout:Feb 15 18:44:56 smithi121 bash[71151]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-3 2024-02-15T18:44:57.501 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:44:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-4[64474]: debug 2024-02-15T18:44:57.250+0000 7f9cc9bca700 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:31.120282+0000 front 2024-02-15T18:44:31.120433+0000 (oldest deadline 2024-02-15T18:44:52.820132+0000) 2024-02-15T18:44:57.502 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:44:57 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:44:57.384+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:29.743576+0000 front 2024-02-15T18:44:29.743513+0000 (oldest deadline 2024-02-15T18:44:52.643285+0000) 2024-02-15T18:44:58.604 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:44:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-4[64474]: debug 2024-02-15T18:44:58.238+0000 7f9cc9bca700 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:31.120282+0000 front 2024-02-15T18:44:31.120433+0000 (oldest deadline 2024-02-15T18:44:52.820132+0000) 2024-02-15T18:44:58.605 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:44:58 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:44:58.351+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:29.743576+0000 front 2024-02-15T18:44:29.743513+0000 (oldest deadline 2024-02-15T18:44:52.643285+0000) 2024-02-15T18:44:58.732 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.3.service' 2024-02-15T18:44:58.776 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-15T18:44:58.776 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-15T18:44:58.777 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-15T18:44:58.777 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.4 2024-02-15T18:44:59.104 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:44:58 smithi121 systemd[1]: Stopping Ceph osd.4 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:44:59.104 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:44:58 smithi121 bash[71391]: Error: no container with name or ID "ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-15T18:44:59.604 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:44:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:44:59.303+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:29.743576+0000 front 2024-02-15T18:44:29.743513+0000 (oldest deadline 2024-02-15T18:44:52.643285+0000) 2024-02-15T18:44:59.604 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:44:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-4[64474]: debug 2024-02-15T18:44:59.126+0000 7f9cccc4d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:44:59.604 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:44:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-4[64474]: debug 2024-02-15T18:44:59.126+0000 7f9cccc4d700 -1 osd.4 80 *** Got signal Terminated *** 2024-02-15T18:44:59.605 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:44:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-4[64474]: debug 2024-02-15T18:44:59.126+0000 7f9cccc4d700 -1 osd.4 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:44:59.605 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:44:59 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-4[64474]: debug 2024-02-15T18:44:59.252+0000 7f9cc9bca700 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:31.120282+0000 front 2024-02-15T18:44:31.120433+0000 (oldest deadline 2024-02-15T18:44:52.820132+0000) 2024-02-15T18:45:00.604 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:45:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-4[64474]: debug 2024-02-15T18:45:00.212+0000 7f9cc9bca700 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:31.120282+0000 front 2024-02-15T18:44:31.120433+0000 (oldest deadline 2024-02-15T18:44:52.820132+0000) 2024-02-15T18:45:00.604 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:00 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:00.349+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:29.743576+0000 front 2024-02-15T18:44:29.743513+0000 (oldest deadline 2024-02-15T18:44:52.643285+0000) 2024-02-15T18:45:01.604 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:45:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-4[64474]: debug 2024-02-15T18:45:01.202+0000 7f9cc9bca700 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:31.120282+0000 front 2024-02-15T18:44:31.120433+0000 (oldest deadline 2024-02-15T18:44:52.820132+0000) 2024-02-15T18:45:01.604 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:01 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:01.343+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:29.743576+0000 front 2024-02-15T18:44:29.743513+0000 (oldest deadline 2024-02-15T18:44:52.643285+0000) 2024-02-15T18:45:02.604 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:45:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-4[64474]: debug 2024-02-15T18:45:02.199+0000 7f9cc9bca700 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:31.120282+0000 front 2024-02-15T18:44:31.120433+0000 (oldest deadline 2024-02-15T18:44:52.820132+0000) 2024-02-15T18:45:02.604 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:02.309+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:29.743576+0000 front 2024-02-15T18:44:29.743513+0000 (oldest deadline 2024-02-15T18:44:52.643285+0000) 2024-02-15T18:45:02.605 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:02 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:02.309+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-02-15T18:44:38.444503+0000 front 2024-02-15T18:44:38.444434+0000 (oldest deadline 2024-02-15T18:45:01.944309+0000) 2024-02-15T18:45:03.604 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:45:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-4[64474]: debug 2024-02-15T18:45:03.153+0000 7f9cc9bca700 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:31.120282+0000 front 2024-02-15T18:44:31.120433+0000 (oldest deadline 2024-02-15T18:44:52.820132+0000) 2024-02-15T18:45:03.604 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:45:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-4[64474]: debug 2024-02-15T18:45:03.153+0000 7f9cc9bca700 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-02-15T18:44:39.622179+0000 front 2024-02-15T18:44:39.622242+0000 (oldest deadline 2024-02-15T18:45:02.522043+0000) 2024-02-15T18:45:03.605 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:03.264+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:29.743576+0000 front 2024-02-15T18:44:29.743513+0000 (oldest deadline 2024-02-15T18:44:52.643285+0000) 2024-02-15T18:45:03.605 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:03 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:03.264+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-02-15T18:44:38.444503+0000 front 2024-02-15T18:44:38.444434+0000 (oldest deadline 2024-02-15T18:45:01.944309+0000) 2024-02-15T18:45:04.599 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:04 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:04.292+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:29.743576+0000 front 2024-02-15T18:44:29.743513+0000 (oldest deadline 2024-02-15T18:44:52.643285+0000) 2024-02-15T18:45:04.600 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:04 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:04.292+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-02-15T18:44:38.444503+0000 front 2024-02-15T18:44:38.444434+0000 (oldest deadline 2024-02-15T18:45:01.944309+0000) 2024-02-15T18:45:04.854 INFO:journalctl@ceph.osd.4.smithi121.stdout:Feb 15 18:45:04 smithi121 bash[71431]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-4 2024-02-15T18:45:05.604 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:05.339+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:29.743576+0000 front 2024-02-15T18:44:29.743513+0000 (oldest deadline 2024-02-15T18:44:52.643285+0000) 2024-02-15T18:45:05.604 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:05 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:05.339+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-02-15T18:44:38.444503+0000 front 2024-02-15T18:44:38.444434+0000 (oldest deadline 2024-02-15T18:45:01.944309+0000) 2024-02-15T18:45:06.333 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.4.service' 2024-02-15T18:45:06.375 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-15T18:45:06.375 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-15T18:45:06.375 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-15T18:45:06.376 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.5 2024-02-15T18:45:06.604 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:06.325+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:29.743576+0000 front 2024-02-15T18:44:29.743513+0000 (oldest deadline 2024-02-15T18:44:52.643285+0000) 2024-02-15T18:45:06.605 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:06.325+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-02-15T18:44:38.444503+0000 front 2024-02-15T18:44:38.444434+0000 (oldest deadline 2024-02-15T18:45:01.944309+0000) 2024-02-15T18:45:06.605 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:06 smithi121 systemd[1]: Stopping Ceph osd.5 for 7e05753e-cc2f-11ee-95ba-87774f69a715... 2024-02-15T18:45:06.605 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:06 smithi121 bash[71668]: Error: no container with name or ID "ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-15T18:45:07.104 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:06.745+0000 7f89fd208700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:45:07.104 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:06.745+0000 7f89fd208700 -1 osd.5 80 *** Got signal Terminated *** 2024-02-15T18:45:07.104 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:06 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:06.745+0000 7f89fd208700 -1 osd.5 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:45:07.604 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:07.334+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:29.743576+0000 front 2024-02-15T18:44:29.743513+0000 (oldest deadline 2024-02-15T18:44:52.643285+0000) 2024-02-15T18:45:07.604 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:07 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:07.334+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-02-15T18:44:38.444503+0000 front 2024-02-15T18:44:38.444434+0000 (oldest deadline 2024-02-15T18:45:01.944309+0000) 2024-02-15T18:45:08.604 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:08.336+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:29.743576+0000 front 2024-02-15T18:44:29.743513+0000 (oldest deadline 2024-02-15T18:44:52.643285+0000) 2024-02-15T18:45:08.604 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:08 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:08.336+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-02-15T18:44:38.444503+0000 front 2024-02-15T18:44:38.444434+0000 (oldest deadline 2024-02-15T18:45:01.944309+0000) 2024-02-15T18:45:09.604 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:09.293+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:29.743576+0000 front 2024-02-15T18:44:29.743513+0000 (oldest deadline 2024-02-15T18:44:52.643285+0000) 2024-02-15T18:45:09.604 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:09 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:09.293+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-02-15T18:44:38.444503+0000 front 2024-02-15T18:44:38.444434+0000 (oldest deadline 2024-02-15T18:45:01.944309+0000) 2024-02-15T18:45:10.604 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:10.284+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:29.743576+0000 front 2024-02-15T18:44:29.743513+0000 (oldest deadline 2024-02-15T18:44:52.643285+0000) 2024-02-15T18:45:10.604 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:10 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:10.284+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-02-15T18:44:38.444503+0000 front 2024-02-15T18:44:38.444434+0000 (oldest deadline 2024-02-15T18:45:01.944309+0000) 2024-02-15T18:45:11.604 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:11.290+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-02-15T18:44:29.743576+0000 front 2024-02-15T18:44:29.743513+0000 (oldest deadline 2024-02-15T18:44:52.643285+0000) 2024-02-15T18:45:11.604 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:11.290+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-02-15T18:44:38.444503+0000 front 2024-02-15T18:44:38.444434+0000 (oldest deadline 2024-02-15T18:45:01.944309+0000) 2024-02-15T18:45:11.604 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:11 smithi121 ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5[66750]: debug 2024-02-15T18:45:11.290+0000 7f89fa986700 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-02-15T18:44:47.745706+0000 front 2024-02-15T18:44:47.745713+0000 (oldest deadline 2024-02-15T18:45:11.245563+0000) 2024-02-15T18:45:12.354 INFO:journalctl@ceph.osd.5.smithi121.stdout:Feb 15 18:45:12 smithi121 bash[71706]: ceph-7e05753e-cc2f-11ee-95ba-87774f69a715-osd-5 2024-02-15T18:45:13.919 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7e05753e-cc2f-11ee-95ba-87774f69a715@osd.5.service' 2024-02-15T18:45:13.962 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-15T18:45:13.962 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-15T18:45:13.962 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-15T18:45:35.890 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-15T18:45:46.903 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-15T18:45:46.942 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-15T18:45:46.973 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-15T18:45:46.974 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561675/remote/ubuntu@smithi038.front.sepia.ceph.com/crash 2024-02-15T18:45:46.975 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/crash -- . 2024-02-15T18:45:47.013 INFO:teuthology.orchestra.run.smithi038.stderr:tar: /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-15T18:45:47.013 INFO:teuthology.orchestra.run.smithi038.stderr:tar: Error is not recoverable: exiting now 2024-02-15T18:45:47.015 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561675/remote/ubuntu@smithi121.front.sepia.ceph.com/crash 2024-02-15T18:45:47.016 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/crash -- . 2024-02-15T18:45:47.042 INFO:teuthology.orchestra.run.smithi121.stderr:tar: /var/lib/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-15T18:45:47.042 INFO:teuthology.orchestra.run.smithi121.stderr:tar: Error is not recoverable: exiting now 2024-02-15T18:45:47.043 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-15T18:45:47.043 DEBUG:teuthology.orchestra.run.smithi038:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v 'Degraded data redundancy' | egrep -v '\(PG_' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-02-15T18:45:47.092 INFO:teuthology.orchestra.run.smithi038.stdout:2024-02-15T18:32:57.924130+0000 mon.smithi038 (mon.0) 506 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-02-15T18:45:47.092 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-15T18:45:47.092 DEBUG:teuthology.orchestra.run.smithi038:> sudo egrep '\[SEC\]' /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v 'Degraded data redundancy' | egrep -v '\(PG_' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-02-15T18:45:47.167 DEBUG:teuthology.orchestra.run.smithi038:> sudo egrep '\[ERR\]' /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v 'Degraded data redundancy' | egrep -v '\(PG_' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-02-15T18:45:47.243 DEBUG:teuthology.orchestra.run.smithi038:> sudo egrep '\[WRN\]' /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v 'Degraded data redundancy' | egrep -v '\(PG_' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-02-15T18:45:47.317 INFO:teuthology.orchestra.run.smithi038.stdout:2024-02-15T18:32:57.924130+0000 mon.smithi038 (mon.0) 506 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-02-15T18:45:47.318 INFO:tasks.cephadm:Compressing logs... 2024-02-15T18:45:47.318 DEBUG:teuthology.orchestra.run.smithi038:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-15T18:45:47.360 DEBUG:teuthology.orchestra.run.smithi121:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-15T18:45:47.385 INFO:teuthology.orchestra.run.smithi038.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-15T18:45:47.386 INFO:teuthology.orchestra.run.smithi038.stderr:: No such file or directory 2024-02-15T18:45:47.386 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-15T18:45:47.386 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose /var/log/ceph/cephadm.log: -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-osd.2.log 2024-02-15T18:45:47.387 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mon.smithi038.log 2024-02-15T18:45:47.387 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-15T18:45:47.387 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mon.smithi038.log: gzip -5 --verbose -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-15T18:45:47.388 INFO:teuthology.orchestra.run.smithi121.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-15T18:45:47.388 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-15T18:45:47.388 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-15T18:45:47.389 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-15T18:45:47.389 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-15T18:45:47.389 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-osd.3.log 2024-02-15T18:45:47.389 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.cephadm.log: -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi121.mvumlh.log 2024-02-15T18:45:47.390 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-osd.3.log: 86.7% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-15T18:45:47.390 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi121.mvumlh.log: gzip -5 --verbose -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mgr.smithi121.ckiryi.log 2024-02-15T18:45:47.390 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi121.ovuzki.log 2024-02-15T18:45:47.391 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mgr.smithi121.ckiryi.log: /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi121.ovuzki.log: 91.3% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-15T18:45:47.392 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.log 2024-02-15T18:45:47.392 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-osd.0.log 2024-02-15T18:45:47.392 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.cephadm.log: /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-osd.0.log: 86.5% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-15T18:45:47.392 INFO:teuthology.orchestra.run.smithi038.stderr: 91.2% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-15T18:45:47.392 INFO:teuthology.orchestra.run.smithi038.stderr: 90.7% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-15T18:45:47.392 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mgr.smithi038.khesbl.log 2024-02-15T18:45:47.393 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi038.qgkzgn.log 2024-02-15T18:45:47.393 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mgr.smithi038.khesbl.log: gzip -5 --verbose -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi038.hxoggl.log 2024-02-15T18:45:47.393 INFO:teuthology.orchestra.run.smithi038.stderr: 91.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-15T18:45:47.393 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi038.qgkzgn.log: gzip -5 --verbose -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.log 2024-02-15T18:45:47.393 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.log: 89.5% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-15T18:45:47.394 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-15T18:45:47.394 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-osd.4.log 2024-02-15T18:45:47.394 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-osd.5.log 2024-02-15T18:45:47.401 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi038.hxoggl.log: /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.log: 87.9% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-15T18:45:47.401 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-osd.1.log 2024-02-15T18:45:47.404 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-osd.4.log: /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-osd.5.log: 90.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-15T18:45:47.404 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mon.smithi121.log 2024-02-15T18:45:47.404 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-osd.1.log: 93.7% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi038.qgkzgn.log.gz 2024-02-15T18:45:47.425 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mon.smithi121.log: 89.4% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mgr.smithi121.ckiryi.log.gz 2024-02-15T18:45:47.710 INFO:teuthology.orchestra.run.smithi121.stderr: 92.6% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mon.smithi121.log.gz 2024-02-15T18:45:47.727 INFO:teuthology.orchestra.run.smithi038.stderr: 90.7% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mgr.smithi038.khesbl.log.gz 2024-02-15T18:45:47.956 INFO:teuthology.orchestra.run.smithi038.stderr: 90.6% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mon.smithi038.log.gz 2024-02-15T18:45:50.850 INFO:teuthology.orchestra.run.smithi038.stderr: 93.4% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-osd.2.log.gz 2024-02-15T18:45:51.715 INFO:teuthology.orchestra.run.smithi038.stderr: 93.4% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-osd.0.log.gz 2024-02-15T18:45:52.017 INFO:teuthology.orchestra.run.smithi121.stderr: 93.6% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-osd.5.log.gz 2024-02-15T18:45:52.018 INFO:teuthology.orchestra.run.smithi038.stderr: 95.1% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi038.hxoggl.log.gz 2024-02-15T18:45:52.048 INFO:teuthology.orchestra.run.smithi038.stderr: 93.5% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-osd.1.log.gz 2024-02-15T18:45:52.050 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-02-15T18:45:52.050 INFO:teuthology.orchestra.run.smithi038.stderr:real 0m4.675s 2024-02-15T18:45:52.050 INFO:teuthology.orchestra.run.smithi038.stderr:user 0m17.397s 2024-02-15T18:45:52.051 INFO:teuthology.orchestra.run.smithi038.stderr:sys 0m0.610s 2024-02-15T18:45:52.479 INFO:teuthology.orchestra.run.smithi121.stderr: 93.2% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-osd.4.log.gz 2024-02-15T18:45:52.544 INFO:teuthology.orchestra.run.smithi121.stderr: 93.5% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-osd.3.log.gz 2024-02-15T18:46:29.927 INFO:teuthology.orchestra.run.smithi121.stderr: 93.6% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi121.ovuzki.log.gz 2024-02-15T18:46:30.427 INFO:teuthology.orchestra.run.smithi121.stderr: 93.6% -- replaced with /var/log/ceph/7e05753e-cc2f-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi121.mvumlh.log.gz 2024-02-15T18:46:30.429 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-02-15T18:46:30.429 INFO:teuthology.orchestra.run.smithi121.stderr:real 0m43.051s 2024-02-15T18:46:30.429 INFO:teuthology.orchestra.run.smithi121.stderr:user 1m37.284s 2024-02-15T18:46:30.429 INFO:teuthology.orchestra.run.smithi121.stderr:sys 0m3.308s 2024-02-15T18:46:30.430 INFO:tasks.cephadm:Archiving logs... 2024-02-15T18:46:30.430 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/log/ceph to /home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561675/remote/ubuntu@smithi038.front.sepia.ceph.com/log 2024-02-15T18:46:30.431 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-15T18:46:33.089 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/log/ceph to /home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561675/remote/ubuntu@smithi121.front.sepia.ceph.com/log 2024-02-15T18:46:33.090 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-15T18:46:48.489 INFO:tasks.cephadm:Removing cluster... 2024-02-15T18:46:48.489 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 --force 2024-02-15T18:46:48.832 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7e05753e-cc2f-11ee-95ba-87774f69a715 --force 2024-02-15T18:46:49.268 INFO:tasks.cephadm:Removing cephadm ... 2024-02-15T18:46:49.268 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-15T18:46:49.288 DEBUG:teuthology.orchestra.run.smithi121:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-15T18:46:49.306 INFO:tasks.cephadm:Teardown complete 2024-02-15T18:46:49.306 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-15T18:46:49.318 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-15T18:46:49.319 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-15T18:46:49.330 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-15T18:46:49.424 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-15T18:46:49.435 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-15T18:46:49.435 DEBUG:teuthology.orchestra.run.smithi038:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-15T18:46:49.438 DEBUG:teuthology.orchestra.run.smithi121:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-15T18:46:49.453 INFO:teuthology.orchestra.run.smithi038.stderr:bash: ntpq: command not found 2024-02-15T18:46:49.454 INFO:teuthology.orchestra.run.smithi121.stderr:bash: ntpq: command not found 2024-02-15T18:46:49.457 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-15T18:46:49.458 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-02-15T18:46:49.458 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 58 +1266us[+1266us] +/- 116ms 2024-02-15T18:46:49.458 INFO:teuthology.orchestra.run.smithi038.stdout:^* hv02.front.sepia.ceph.com 3 6 377 58 +176us[ +198us] +/- 67ms 2024-02-15T18:46:49.458 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 -988us[ -967us] +/- 74ms 2024-02-15T18:46:49.458 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-15T18:46:49.458 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-15T18:46:49.458 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2024-02-15T18:46:49.458 INFO:teuthology.orchestra.run.smithi121.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 62 +1214us[+1219us] +/- 116ms 2024-02-15T18:46:49.459 INFO:teuthology.orchestra.run.smithi121.stdout:^* hv02.front.sepia.ceph.com 3 7 377 59 +159us[ +164us] +/- 67ms 2024-02-15T18:46:49.459 INFO:teuthology.orchestra.run.smithi121.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 -1020us[-1015us] +/- 74ms 2024-02-15T18:46:49.459 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-15T18:46:49.460 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-15T18:46:49.472 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-15T18:46:49.473 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-15T18:46:49.484 DEBUG:teuthology.orchestra.run.smithi038:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-15T18:46:49.567 DEBUG:teuthology.orchestra.run.smithi121:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-15T18:46:49.628 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-15T18:46:49.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-15T18:46:49.659 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-15T18:46:49.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-15T18:46:49.689 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-15T18:46:49.701 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-15T18:46:49.744 INFO:teuthology.task.internal:Duration was 1913.938757 seconds 2024-02-15T18:46:49.744 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-15T18:46:49.755 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-15T18:46:49.755 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-15T18:46:49.758 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-15T18:46:49.802 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-15T18:46:49.803 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-15T18:46:50.032 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-15T18:46:50.033 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi038.front.sepia.ceph.com 2024-02-15T18:46:50.033 DEBUG:teuthology.orchestra.run.smithi038:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-15T18:46:50.056 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi121.front.sepia.ceph.com 2024-02-15T18:46:50.056 DEBUG:teuthology.orchestra.run.smithi121:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-15T18:46:50.079 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-15T18:46:50.079 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-15T18:46:50.098 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-15T18:46:50.174 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-15T18:46:50.174 DEBUG:teuthology.orchestra.run.smithi038:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-15T18:46:50.176 DEBUG:teuthology.orchestra.run.smithi121:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-15T18:46:50.304 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-15T18:46:50.317 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-15T18:46:50.317 DEBUG:teuthology.orchestra.run.smithi038:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-15T18:46:50.345 DEBUG:teuthology.orchestra.run.smithi121:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-15T18:46:50.372 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-15T18:46:50.385 DEBUG:teuthology.orchestra.run.smithi038:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-15T18:46:50.387 DEBUG:teuthology.orchestra.run.smithi121:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-15T18:46:50.412 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = core 2024-02-15T18:46:50.439 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = core 2024-02-15T18:46:50.452 DEBUG:teuthology.orchestra.run.smithi038:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-15T18:46:50.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-15T18:46:50.483 DEBUG:teuthology.orchestra.run.smithi121:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-15T18:46:50.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-15T18:46:50.509 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-15T18:46:50.521 INFO:teuthology.task.internal:Transferring archived files... 2024-02-15T18:46:50.522 DEBUG:teuthology.misc:Transferring archived files from smithi038:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561675/remote/smithi038 2024-02-15T18:46:50.523 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-15T18:46:50.569 DEBUG:teuthology.misc:Transferring archived files from smithi121:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561675/remote/smithi121 2024-02-15T18:46:50.570 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-15T18:46:50.610 INFO:teuthology.task.internal:Removing archive directory... 2024-02-15T18:46:50.611 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-15T18:46:50.613 DEBUG:teuthology.orchestra.run.smithi121:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-15T18:46:50.661 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-15T18:46:50.676 INFO:teuthology.task.internal:Not uploading archives. 2024-02-15T18:46:50.676 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-15T18:46:50.696 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-15T18:46:50.696 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-15T18:46:50.699 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-15T18:46:50.714 INFO:teuthology.orchestra.run.smithi038.stdout: 393224 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 15 18:46 /home/ubuntu/cephtest 2024-02-15T18:46:50.718 INFO:teuthology.orchestra.run.smithi121.stdout: 525357 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 15 18:46 /home/ubuntu/cephtest 2024-02-15T18:46:50.720 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-15T18:46:50.733 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-15T18:46:50.759 INFO:teuthology.nuke:Checking targets against current locks 2024-02-15T18:46:50.789 DEBUG:teuthology.nuke:shortname: smithi038 2024-02-15T18:46:50.789 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-15T18:46:50.813 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561675', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-15 18:04:35.389983', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEd41pZ143Adv9uvpx+XlukqK5F+agsvwBdxonJpz8AHilhRrPX5g0VWtWM0VufLhQp8BISm/7dTbXjZcJltzNo='} 2024-02-15T18:46:50.817 DEBUG:teuthology.nuke:shortname: smithi121 2024-02-15T18:46:50.817 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-15T18:46:50.840 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561675', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-15 18:04:35.388771', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOZ9r2D1mj/bG7GzVc470FwrHqZw9kuRYD6jtyL+QgMEKVZ53Sbt7UlfCnUk/hTnZLHHznVyp1T4GMNzJFZdAXk='} 2024-02-15T18:46:50.869 INFO:teuthology.orchestra.console.smithi038:Power off 2024-02-15T18:46:50.869 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-15T18:46:50.884 INFO:teuthology.orchestra.console.smithi121:Power off 2024-02-15T18:46:50.884 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-15T18:46:50.894 DEBUG:teuthology.orchestra.console.smithi038:power off output: Chassis Power Control: Down/Off 2024-02-15T18:46:50.894 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-15T18:46:50.905 DEBUG:teuthology.orchestra.console.smithi121:power off output: Chassis Power Control: Down/Off 2024-02-15T18:46:50.906 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-15T18:46:50.918 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on 2024-02-15T18:46:50.926 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is on 2024-02-15T18:46:54.919 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-15T18:46:54.928 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-15T18:46:55.031 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on 2024-02-15T18:46:55.039 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is on 2024-02-15T18:46:59.036 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-15T18:46:59.045 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-15T18:46:59.147 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is off 2024-02-15T18:46:59.154 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is off 2024-02-15T18:46:59.248 INFO:teuthology.orchestra.console.smithi038:Power off completed 2024-02-15T18:46:59.255 INFO:teuthology.orchestra.console.smithi121:Power off completed 2024-02-15T18:46:59.359 INFO:teuthology.run:Summary data: description: rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} duration: 1913.938756942749 failure_reason: '"2024-02-15T18:32:57.924130+0000 mon.smithi038 (mon.0) 506 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)" in cluster log' flavor: default owner: scheduled_lflores@teuthology success: false 2024-02-15T18:46:59.359 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-15T18:46:59.442 INFO:teuthology.run:FAIL